Logging to experiments/gym_fwalker2d/Wo01/Mon-07-Nov-2022-10-30-38-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2531
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 67.
Path 5 | total_timesteps 81.
Path 6 | total_timesteps 99.
Path 7 | total_timesteps 112.
Path 8 | total_timesteps 135.
Path 9 | total_timesteps 148.
Path 10 | total_timesteps 177.
Path 11 | total_timesteps 190.
Path 12 | total_timesteps 204.
Path 13 | total_timesteps 252.
Path 14 | total_timesteps 264.
Path 15 | total_timesteps 291.
Path 16 | total_timesteps 307.
Path 17 | total_timesteps 324.
Path 18 | total_timesteps 357.
Path 19 | total_timesteps 377.
Path 20 | total_timesteps 398.
Path 21 | total_timesteps 410.
Path 22 | total_timesteps 435.
Path 23 | total_timesteps 462.
Path 24 | total_timesteps 486.
Path 25 | total_timesteps 497.
Path 26 | total_timesteps 509.
Path 27 | total_timesteps 544.
Path 28 | total_timesteps 576.
Path 29 | total_timesteps 605.
Path 30 | total_timesteps 626.
Path 31 | total_timesteps 655.
Path 32 | total_timesteps 672.
Path 33 | total_timesteps 693.
Path 34 | total_timesteps 705.
Path 35 | total_timesteps 720.
Path 36 | total_timesteps 743.
Path 37 | total_timesteps 759.
Path 38 | total_timesteps 772.
Path 39 | total_timesteps 792.
Path 40 | total_timesteps 809.
Path 41 | total_timesteps 824.
Path 42 | total_timesteps 846.
Path 43 | total_timesteps 868.
Path 44 | total_timesteps 889.
Path 45 | total_timesteps 900.
Path 46 | total_timesteps 914.
Path 47 | total_timesteps 929.
Path 48 | total_timesteps 941.
Path 49 | total_timesteps 952.
Path 50 | total_timesteps 974.
Path 51 | total_timesteps 989.
Path 52 | total_timesteps 1014.
Path 53 | total_timesteps 1052.
Path 54 | total_timesteps 1072.
Path 55 | total_timesteps 1094.
Path 56 | total_timesteps 1124.
Path 57 | total_timesteps 1137.
Path 58 | total_timesteps 1154.
Path 59 | total_timesteps 1187.
Path 60 | total_timesteps 1207.
Path 61 | total_timesteps 1229.
Path 62 | total_timesteps 1245.
Path 63 | total_timesteps 1261.
Path 64 | total_timesteps 1290.
Path 65 | total_timesteps 1313.
Path 66 | total_timesteps 1337.
Path 67 | total_timesteps 1357.
Path 68 | total_timesteps 1370.
Path 69 | total_timesteps 1389.
Path 70 | total_timesteps 1406.
Path 71 | total_timesteps 1420.
Path 72 | total_timesteps 1437.
Path 73 | total_timesteps 1455.
Path 74 | total_timesteps 1471.
Path 75 | total_timesteps 1491.
Path 76 | total_timesteps 1515.
Path 77 | total_timesteps 1525.
Path 78 | total_timesteps 1545.
Path 79 | total_timesteps 1572.
Path 80 | total_timesteps 1591.
Path 81 | total_timesteps 1609.
Path 82 | total_timesteps 1626.
Path 83 | total_timesteps 1651.
Path 84 | total_timesteps 1671.
Path 85 | total_timesteps 1692.
Path 86 | total_timesteps 1706.
Path 87 | total_timesteps 1730.
Path 88 | total_timesteps 1741.
Path 89 | total_timesteps 1753.
Path 90 | total_timesteps 1788.
Path 91 | total_timesteps 1801.
Path 92 | total_timesteps 1823.
Path 93 | total_timesteps 1837.
Path 94 | total_timesteps 1874.
Path 95 | total_timesteps 1894.
Path 96 | total_timesteps 1919.
Path 97 | total_timesteps 1930.
Path 98 | total_timesteps 1942.
Path 99 | total_timesteps 1965.
Path 100 | total_timesteps 1992.
Path 101 | total_timesteps 2011.
Path 102 | total_timesteps 2037.
Path 103 | total_timesteps 2054.
Path 104 | total_timesteps 2073.
Path 105 | total_timesteps 2090.
Path 106 | total_timesteps 2104.
Path 107 | total_timesteps 2130.
Path 108 | total_timesteps 2155.
Path 109 | total_timesteps 2178.
Path 110 | total_timesteps 2194.
Path 111 | total_timesteps 2207.
Path 112 | total_timesteps 2226.
Path 113 | total_timesteps 2240.
Path 114 | total_timesteps 2260.
Path 115 | total_timesteps 2275.
Path 116 | total_timesteps 2307.
Path 117 | total_timesteps 2350.
Path 118 | total_timesteps 2362.
Path 119 | total_timesteps 2377.
Path 120 | total_timesteps 2398.
Path 121 | total_timesteps 2413.
Path 122 | total_timesteps 2428.
Path 123 | total_timesteps 2449.
Path 124 | total_timesteps 2474.
Path 125 | total_timesteps 2486.
Path 126 | total_timesteps 2498.
Path 127 | total_timesteps 2544.
Path 128 | total_timesteps 2565.
Path 129 | total_timesteps 2585.
Path 130 | total_timesteps 2615.
Path 131 | total_timesteps 2636.
Path 132 | total_timesteps 2654.
Path 133 | total_timesteps 2666.
Path 134 | total_timesteps 2681.
Path 135 | total_timesteps 2693.
Path 136 | total_timesteps 2705.
Path 137 | total_timesteps 2727.
Path 138 | total_timesteps 2742.
Path 139 | total_timesteps 2756.
Path 140 | total_timesteps 2784.
Path 141 | total_timesteps 2814.
Path 142 | total_timesteps 2840.
Path 143 | total_timesteps 2873.
Path 144 | total_timesteps 2884.
Path 145 | total_timesteps 2900.
Path 146 | total_timesteps 2923.
Path 147 | total_timesteps 2942.
Path 148 | total_timesteps 2955.
Path 149 | total_timesteps 2972.
Path 150 | total_timesteps 2982.
Path 151 | total_timesteps 2998.
Path 152 | total_timesteps 3018.
Path 153 | total_timesteps 3033.
Path 154 | total_timesteps 3044.
Path 155 | total_timesteps 3072.
Path 156 | total_timesteps 3085.
Path 157 | total_timesteps 3098.
Path 158 | total_timesteps 3110.
Path 159 | total_timesteps 3140.
Path 160 | total_timesteps 3157.
Path 161 | total_timesteps 3167.
Path 162 | total_timesteps 3185.
Path 163 | total_timesteps 3226.
Path 164 | total_timesteps 3246.
Path 165 | total_timesteps 3281.
Path 166 | total_timesteps 3297.
Path 167 | total_timesteps 3338.
Path 168 | total_timesteps 3360.
Path 169 | total_timesteps 3380.
Path 170 | total_timesteps 3406.
Path 171 | total_timesteps 3435.
Path 172 | total_timesteps 3460.
Path 173 | total_timesteps 3487.
Path 174 | total_timesteps 3505.
Path 175 | total_timesteps 3524.
Path 176 | total_timesteps 3549.
Path 177 | total_timesteps 3574.
Path 178 | total_timesteps 3605.
Path 179 | total_timesteps 3663.
Path 180 | total_timesteps 3693.
Path 181 | total_timesteps 3704.
Path 182 | total_timesteps 3719.
Path 183 | total_timesteps 3747.
Path 184 | total_timesteps 3775.
Path 185 | total_timesteps 3785.
Path 186 | total_timesteps 3804.
Path 187 | total_timesteps 3820.
Path 188 | total_timesteps 3861.
Path 189 | total_timesteps 3884.
Path 190 | total_timesteps 3922.
Path 191 | total_timesteps 3938.
Path 192 | total_timesteps 3955.
Path 193 | total_timesteps 3970.
Path 194 | total_timesteps 4004.
Path 195 | total_timesteps 4022.
Path 196 | total_timesteps 4042.
Path 197 | total_timesteps 4079.
Path 198 | total_timesteps 4090.
Path 199 | total_timesteps 4112.
Path 200 | total_timesteps 4127.
Path 201 | total_timesteps 4156.
Path 202 | total_timesteps 4169.
Path 203 | total_timesteps 4206.
Path 204 | total_timesteps 4228.
Path 205 | total_timesteps 4243.
Path 206 | total_timesteps 4266.
Path 207 | total_timesteps 4310.
Path 208 | total_timesteps 4346.
Path 209 | total_timesteps 4365.
Path 210 | total_timesteps 4380.
Path 211 | total_timesteps 4429.
Path 212 | total_timesteps 4449.
Path 213 | total_timesteps 4484.
Path 214 | total_timesteps 4504.
Path 215 | total_timesteps 4528.
Path 216 | total_timesteps 4551.
Path 217 | total_timesteps 4573.
Path 218 | total_timesteps 4590.
Path 219 | total_timesteps 4618.
Path 220 | total_timesteps 4628.
Path 221 | total_timesteps 4640.
Path 222 | total_timesteps 4689.
Path 223 | total_timesteps 4701.
Path 224 | total_timesteps 4721.
Path 225 | total_timesteps 4738.
Path 226 | total_timesteps 4753.
Path 227 | total_timesteps 4770.
Path 228 | total_timesteps 4790.
Path 229 | total_timesteps 4811.
Path 230 | total_timesteps 4835.
Path 231 | total_timesteps 4851.
Path 232 | total_timesteps 4876.
Path 233 | total_timesteps 4898.
Path 234 | total_timesteps 4911.
Path 235 | total_timesteps 4928.
Path 236 | total_timesteps 4942.
Path 237 | total_timesteps 4959.
Path 238 | total_timesteps 4988.
Path 239 | total_timesteps 4997.
Path 240 | total_timesteps 5026.
Path 241 | total_timesteps 5040.
Path 242 | total_timesteps 5060.
Path 243 | total_timesteps 5083.
Path 244 | total_timesteps 5102.
Path 245 | total_timesteps 5119.
Path 246 | total_timesteps 5131.
Path 247 | total_timesteps 5168.
Path 248 | total_timesteps 5181.
Path 249 | total_timesteps 5205.
Path 250 | total_timesteps 5236.
Path 251 | total_timesteps 5258.
Path 252 | total_timesteps 5274.
Path 253 | total_timesteps 5289.
Path 254 | total_timesteps 5314.
Path 255 | total_timesteps 5365.
Path 256 | total_timesteps 5391.
Path 257 | total_timesteps 5414.
Path 258 | total_timesteps 5424.
Path 259 | total_timesteps 5458.
Path 260 | total_timesteps 5475.
Path 261 | total_timesteps 5496.
Path 262 | total_timesteps 5550.
Path 263 | total_timesteps 5574.
Path 264 | total_timesteps 5593.
Path 265 | total_timesteps 5610.
Path 266 | total_timesteps 5631.
Path 267 | total_timesteps 5644.
Path 268 | total_timesteps 5664.
Path 269 | total_timesteps 5680.
Path 270 | total_timesteps 5705.
Path 271 | total_timesteps 5717.
Path 272 | total_timesteps 5728.
Path 273 | total_timesteps 5745.
Path 274 | total_timesteps 5757.
Path 275 | total_timesteps 5766.
Path 276 | total_timesteps 5786.
Path 277 | total_timesteps 5803.
Path 278 | total_timesteps 5822.
Path 279 | total_timesteps 5834.
Path 280 | total_timesteps 5848.
Path 281 | total_timesteps 5863.
Path 282 | total_timesteps 5882.
Path 283 | total_timesteps 5895.
Path 284 | total_timesteps 5913.
Path 285 | total_timesteps 5923.
Path 286 | total_timesteps 5938.
Path 287 | total_timesteps 5955.
Path 288 | total_timesteps 5979.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.545913815498352
Validation loss = 0.40558403730392456
Validation loss = 0.3650318384170532
Validation loss = 0.34661710262298584
Validation loss = 0.3325902819633484
Validation loss = 0.32909974455833435
Validation loss = 0.3316575586795807
Validation loss = 0.33535894751548767
Validation loss = 0.3387902081012726
Validation loss = 0.3550412654876709
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 92.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 124.
Path 11 | total_timesteps 134.
Path 12 | total_timesteps 144.
Path 13 | total_timesteps 154.
Path 14 | total_timesteps 161.
Path 15 | total_timesteps 169.
Path 16 | total_timesteps 180.
Path 17 | total_timesteps 189.
Path 18 | total_timesteps 197.
Path 19 | total_timesteps 210.
Path 20 | total_timesteps 223.
Path 21 | total_timesteps 234.
Path 22 | total_timesteps 242.
Path 23 | total_timesteps 253.
Path 24 | total_timesteps 262.
Path 25 | total_timesteps 284.
Path 26 | total_timesteps 294.
Path 27 | total_timesteps 302.
Path 28 | total_timesteps 308.
Path 29 | total_timesteps 316.
Path 30 | total_timesteps 330.
Path 31 | total_timesteps 341.
Path 32 | total_timesteps 353.
Path 33 | total_timesteps 368.
Path 34 | total_timesteps 376.
Path 35 | total_timesteps 388.
Path 36 | total_timesteps 394.
Path 37 | total_timesteps 405.
Path 38 | total_timesteps 416.
Path 39 | total_timesteps 426.
Path 40 | total_timesteps 434.
Path 41 | total_timesteps 442.
Path 42 | total_timesteps 451.
Path 43 | total_timesteps 461.
Path 44 | total_timesteps 470.
Path 45 | total_timesteps 477.
Path 46 | total_timesteps 491.
Path 47 | total_timesteps 502.
Path 48 | total_timesteps 514.
Path 49 | total_timesteps 525.
Path 50 | total_timesteps 540.
Path 51 | total_timesteps 549.
Path 52 | total_timesteps 556.
Path 53 | total_timesteps 566.
Path 54 | total_timesteps 574.
Path 55 | total_timesteps 585.
Path 56 | total_timesteps 592.
Path 57 | total_timesteps 604.
Path 58 | total_timesteps 615.
Path 59 | total_timesteps 631.
Path 60 | total_timesteps 643.
Path 61 | total_timesteps 654.
Path 62 | total_timesteps 663.
Path 63 | total_timesteps 674.
Path 64 | total_timesteps 685.
Path 65 | total_timesteps 695.
Path 66 | total_timesteps 703.
Path 67 | total_timesteps 710.
Path 68 | total_timesteps 718.
Path 69 | total_timesteps 727.
Path 70 | total_timesteps 742.
Path 71 | total_timesteps 751.
Path 72 | total_timesteps 764.
Path 73 | total_timesteps 778.
Path 74 | total_timesteps 788.
Path 75 | total_timesteps 796.
Path 76 | total_timesteps 803.
Path 77 | total_timesteps 814.
Path 78 | total_timesteps 824.
Path 79 | total_timesteps 831.
Path 80 | total_timesteps 845.
Path 81 | total_timesteps 854.
Path 82 | total_timesteps 862.
Path 83 | total_timesteps 870.
Path 84 | total_timesteps 883.
Path 85 | total_timesteps 897.
Path 86 | total_timesteps 904.
Path 87 | total_timesteps 914.
Path 88 | total_timesteps 923.
Path 89 | total_timesteps 935.
Path 90 | total_timesteps 944.
Path 91 | total_timesteps 953.
Path 92 | total_timesteps 970.
Path 93 | total_timesteps 981.
Path 94 | total_timesteps 996.
Path 95 | total_timesteps 1003.
Path 96 | total_timesteps 1012.
Path 97 | total_timesteps 1020.
Path 98 | total_timesteps 1028.
Path 99 | total_timesteps 1036.
Path 100 | total_timesteps 1049.
Path 101 | total_timesteps 1062.
Path 102 | total_timesteps 1070.
Path 103 | total_timesteps 1078.
Path 104 | total_timesteps 1096.
Path 105 | total_timesteps 1107.
Path 106 | total_timesteps 1115.
Path 107 | total_timesteps 1123.
Path 108 | total_timesteps 1132.
Path 109 | total_timesteps 1142.
Path 110 | total_timesteps 1163.
Path 111 | total_timesteps 1175.
Path 112 | total_timesteps 1182.
Path 113 | total_timesteps 1191.
Path 114 | total_timesteps 1201.
Path 115 | total_timesteps 1216.
Path 116 | total_timesteps 1226.
Path 117 | total_timesteps 1240.
Path 118 | total_timesteps 1251.
Path 119 | total_timesteps 1264.
Path 120 | total_timesteps 1271.
Path 121 | total_timesteps 1284.
Path 122 | total_timesteps 1295.
Path 123 | total_timesteps 1305.
Path 124 | total_timesteps 1312.
Path 125 | total_timesteps 1324.
Path 126 | total_timesteps 1334.
Path 127 | total_timesteps 1344.
Path 128 | total_timesteps 1360.
Path 129 | total_timesteps 1372.
Path 130 | total_timesteps 1383.
Path 131 | total_timesteps 1394.
Path 132 | total_timesteps 1402.
Path 133 | total_timesteps 1411.
Path 134 | total_timesteps 1424.
Path 135 | total_timesteps 1432.
Path 136 | total_timesteps 1444.
Path 137 | total_timesteps 1452.
Path 138 | total_timesteps 1458.
Path 139 | total_timesteps 1469.
Path 140 | total_timesteps 1477.
Path 141 | total_timesteps 1491.
Path 142 | total_timesteps 1503.
Path 143 | total_timesteps 1514.
Path 144 | total_timesteps 1524.
Path 145 | total_timesteps 1537.
Path 146 | total_timesteps 1548.
Path 147 | total_timesteps 1563.
Path 148 | total_timesteps 1571.
Path 149 | total_timesteps 1585.
Path 150 | total_timesteps 1596.
Path 151 | total_timesteps 1604.
Path 152 | total_timesteps 1613.
Path 153 | total_timesteps 1624.
Path 154 | total_timesteps 1631.
Path 155 | total_timesteps 1640.
Path 156 | total_timesteps 1647.
Path 157 | total_timesteps 1656.
Path 158 | total_timesteps 1666.
Path 159 | total_timesteps 1676.
Path 160 | total_timesteps 1686.
Path 161 | total_timesteps 1698.
Path 162 | total_timesteps 1707.
Path 163 | total_timesteps 1717.
Path 164 | total_timesteps 1729.
Path 165 | total_timesteps 1739.
Path 166 | total_timesteps 1747.
Path 167 | total_timesteps 1758.
Path 168 | total_timesteps 1774.
Path 169 | total_timesteps 1787.
Path 170 | total_timesteps 1797.
Path 171 | total_timesteps 1807.
Path 172 | total_timesteps 1815.
Path 173 | total_timesteps 1826.
Path 174 | total_timesteps 1834.
Path 175 | total_timesteps 1843.
Path 176 | total_timesteps 1852.
Path 177 | total_timesteps 1862.
Path 178 | total_timesteps 1872.
Path 179 | total_timesteps 1881.
Path 180 | total_timesteps 1891.
Path 181 | total_timesteps 1903.
Path 182 | total_timesteps 1912.
Path 183 | total_timesteps 1922.
Path 184 | total_timesteps 1931.
Path 185 | total_timesteps 1942.
Path 186 | total_timesteps 1949.
Path 187 | total_timesteps 1965.
Path 188 | total_timesteps 1974.
Path 189 | total_timesteps 1983.
Path 190 | total_timesteps 1992.
Path 191 | total_timesteps 2002.
Path 192 | total_timesteps 2013.
Path 193 | total_timesteps 2020.
Path 194 | total_timesteps 2034.
Path 195 | total_timesteps 2045.
Path 196 | total_timesteps 2055.
Path 197 | total_timesteps 2066.
Path 198 | total_timesteps 2076.
Path 199 | total_timesteps 2086.
Path 200 | total_timesteps 2096.
Path 201 | total_timesteps 2109.
Path 202 | total_timesteps 2118.
Path 203 | total_timesteps 2128.
Path 204 | total_timesteps 2137.
Path 205 | total_timesteps 2148.
Path 206 | total_timesteps 2157.
Path 207 | total_timesteps 2164.
Path 208 | total_timesteps 2176.
Path 209 | total_timesteps 2183.
Path 210 | total_timesteps 2191.
Path 211 | total_timesteps 2200.
Path 212 | total_timesteps 2207.
Path 213 | total_timesteps 2217.
Path 214 | total_timesteps 2227.
Path 215 | total_timesteps 2235.
Path 216 | total_timesteps 2242.
Path 217 | total_timesteps 2251.
Path 218 | total_timesteps 2261.
Path 219 | total_timesteps 2272.
Path 220 | total_timesteps 2295.
Path 221 | total_timesteps 2305.
Path 222 | total_timesteps 2315.
Path 223 | total_timesteps 2324.
Path 224 | total_timesteps 2334.
Path 225 | total_timesteps 2342.
Path 226 | total_timesteps 2360.
Path 227 | total_timesteps 2372.
Path 228 | total_timesteps 2382.
Path 229 | total_timesteps 2393.
Path 230 | total_timesteps 2401.
Path 231 | total_timesteps 2410.
Path 232 | total_timesteps 2418.
Path 233 | total_timesteps 2426.
Path 234 | total_timesteps 2436.
Path 235 | total_timesteps 2444.
Path 236 | total_timesteps 2454.
Path 237 | total_timesteps 2466.
Path 238 | total_timesteps 2476.
Path 239 | total_timesteps 2483.
Path 240 | total_timesteps 2491.
Path 241 | total_timesteps 2501.
Path 242 | total_timesteps 2512.
Path 243 | total_timesteps 2523.
Path 244 | total_timesteps 2532.
Path 245 | total_timesteps 2543.
Path 246 | total_timesteps 2558.
Path 247 | total_timesteps 2569.
Path 248 | total_timesteps 2578.
Path 249 | total_timesteps 2586.
Path 250 | total_timesteps 2593.
Path 251 | total_timesteps 2600.
Path 252 | total_timesteps 2610.
Path 253 | total_timesteps 2617.
Path 254 | total_timesteps 2627.
Path 255 | total_timesteps 2636.
Path 256 | total_timesteps 2649.
Path 257 | total_timesteps 2660.
Path 258 | total_timesteps 2667.
Path 259 | total_timesteps 2677.
Path 260 | total_timesteps 2688.
Path 261 | total_timesteps 2698.
Path 262 | total_timesteps 2707.
Path 263 | total_timesteps 2720.
Path 264 | total_timesteps 2743.
Path 265 | total_timesteps 2752.
Path 266 | total_timesteps 2763.
Path 267 | total_timesteps 2769.
Path 268 | total_timesteps 2780.
Path 269 | total_timesteps 2789.
Path 270 | total_timesteps 2799.
Path 271 | total_timesteps 2812.
Path 272 | total_timesteps 2823.
Path 273 | total_timesteps 2832.
Path 274 | total_timesteps 2841.
Path 275 | total_timesteps 2849.
Path 276 | total_timesteps 2859.
Path 277 | total_timesteps 2865.
Path 278 | total_timesteps 2873.
Path 279 | total_timesteps 2880.
Path 280 | total_timesteps 2890.
Path 281 | total_timesteps 2896.
Path 282 | total_timesteps 2906.
Path 283 | total_timesteps 2913.
Path 284 | total_timesteps 2924.
Path 285 | total_timesteps 2934.
Path 286 | total_timesteps 2943.
Path 287 | total_timesteps 2950.
Path 288 | total_timesteps 2960.
Path 289 | total_timesteps 2969.
Path 290 | total_timesteps 2976.
Path 291 | total_timesteps 2984.
Path 292 | total_timesteps 2995.
Path 293 | total_timesteps 3005.
Path 294 | total_timesteps 3015.
Path 295 | total_timesteps 3021.
Path 296 | total_timesteps 3034.
Path 297 | total_timesteps 3045.
Path 298 | total_timesteps 3055.
Path 299 | total_timesteps 3064.
Path 300 | total_timesteps 3079.
Path 301 | total_timesteps 3093.
Path 302 | total_timesteps 3104.
Path 303 | total_timesteps 3111.
Path 304 | total_timesteps 3123.
Path 305 | total_timesteps 3132.
Path 306 | total_timesteps 3140.
Path 307 | total_timesteps 3148.
Path 308 | total_timesteps 3156.
Path 309 | total_timesteps 3164.
Path 310 | total_timesteps 3173.
Path 311 | total_timesteps 3183.
Path 312 | total_timesteps 3193.
Path 313 | total_timesteps 3202.
Path 314 | total_timesteps 3212.
Path 315 | total_timesteps 3219.
Path 316 | total_timesteps 3226.
Path 317 | total_timesteps 3234.
Path 318 | total_timesteps 3247.
Path 319 | total_timesteps 3259.
Path 320 | total_timesteps 3270.
Path 321 | total_timesteps 3279.
Path 322 | total_timesteps 3289.
Path 323 | total_timesteps 3299.
Path 324 | total_timesteps 3311.
Path 325 | total_timesteps 3320.
Path 326 | total_timesteps 3329.
Path 327 | total_timesteps 3338.
Path 328 | total_timesteps 3347.
Path 329 | total_timesteps 3357.
Path 330 | total_timesteps 3364.
Path 331 | total_timesteps 3373.
Path 332 | total_timesteps 3381.
Path 333 | total_timesteps 3392.
Path 334 | total_timesteps 3401.
Path 335 | total_timesteps 3412.
Path 336 | total_timesteps 3424.
Path 337 | total_timesteps 3431.
Path 338 | total_timesteps 3439.
Path 339 | total_timesteps 3448.
Path 340 | total_timesteps 3457.
Path 341 | total_timesteps 3467.
Path 342 | total_timesteps 3477.
Path 343 | total_timesteps 3486.
Path 344 | total_timesteps 3497.
Path 345 | total_timesteps 3512.
Path 346 | total_timesteps 3521.
Path 347 | total_timesteps 3534.
Path 348 | total_timesteps 3551.
Path 349 | total_timesteps 3562.
Path 350 | total_timesteps 3573.
Path 351 | total_timesteps 3580.
Path 352 | total_timesteps 3589.
Path 353 | total_timesteps 3599.
Path 354 | total_timesteps 3608.
Path 355 | total_timesteps 3617.
Path 356 | total_timesteps 3627.
Path 357 | total_timesteps 3637.
Path 358 | total_timesteps 3646.
Path 359 | total_timesteps 3654.
Path 360 | total_timesteps 3665.
Path 361 | total_timesteps 3677.
Path 362 | total_timesteps 3685.
Path 363 | total_timesteps 3693.
Path 364 | total_timesteps 3711.
Path 365 | total_timesteps 3722.
Path 366 | total_timesteps 3733.
Path 367 | total_timesteps 3744.
Path 368 | total_timesteps 3755.
Path 369 | total_timesteps 3768.
Path 370 | total_timesteps 3780.
Path 371 | total_timesteps 3794.
Path 372 | total_timesteps 3807.
Path 373 | total_timesteps 3816.
Path 374 | total_timesteps 3830.
Path 375 | total_timesteps 3838.
Path 376 | total_timesteps 3848.
Path 377 | total_timesteps 3861.
Path 378 | total_timesteps 3873.
Path 379 | total_timesteps 3882.
Path 380 | total_timesteps 3891.
Path 381 | total_timesteps 3902.
Path 382 | total_timesteps 3913.
Path 383 | total_timesteps 3921.
Path 384 | total_timesteps 3928.
Path 385 | total_timesteps 3939.
Path 386 | total_timesteps 3950.
Path 387 | total_timesteps 3961.
Path 388 | total_timesteps 3970.
Path 389 | total_timesteps 3977.
Path 390 | total_timesteps 3983.
Path 391 | total_timesteps 3992.
Path 392 | total_timesteps 4003.
Path 393 | total_timesteps 4011.
Path 394 | total_timesteps 4021.
Path 395 | total_timesteps 4029.
Path 396 | total_timesteps 4043.
Path 397 | total_timesteps 4050.
Path 398 | total_timesteps 4062.
Path 399 | total_timesteps 4075.
Path 400 | total_timesteps 4085.
Path 401 | total_timesteps 4094.
Path 402 | total_timesteps 4106.
Path 403 | total_timesteps 4113.
Path 404 | total_timesteps 4123.
Path 405 | total_timesteps 4138.
Path 406 | total_timesteps 4146.
Path 407 | total_timesteps 4156.
Path 408 | total_timesteps 4164.
Path 409 | total_timesteps 4172.
Path 410 | total_timesteps 4181.
Path 411 | total_timesteps 4196.
Path 412 | total_timesteps 4206.
Path 413 | total_timesteps 4212.
Path 414 | total_timesteps 4219.
Path 415 | total_timesteps 4231.
Path 416 | total_timesteps 4239.
Path 417 | total_timesteps 4248.
Path 418 | total_timesteps 4254.
Path 419 | total_timesteps 4264.
Path 420 | total_timesteps 4274.
Path 421 | total_timesteps 4286.
Path 422 | total_timesteps 4295.
Path 423 | total_timesteps 4303.
Path 424 | total_timesteps 4310.
Path 425 | total_timesteps 4320.
Path 426 | total_timesteps 4331.
Path 427 | total_timesteps 4339.
Path 428 | total_timesteps 4347.
Path 429 | total_timesteps 4357.
Path 430 | total_timesteps 4369.
Path 431 | total_timesteps 4380.
Path 432 | total_timesteps 4392.
Path 433 | total_timesteps 4400.
Path 434 | total_timesteps 4410.
Path 435 | total_timesteps 4420.
Path 436 | total_timesteps 4434.
Path 437 | total_timesteps 4442.
Path 438 | total_timesteps 4452.
Path 439 | total_timesteps 4461.
Path 440 | total_timesteps 4469.
Path 441 | total_timesteps 4476.
Path 442 | total_timesteps 4491.
Path 443 | total_timesteps 4502.
Path 444 | total_timesteps 4511.
Path 445 | total_timesteps 4521.
Path 446 | total_timesteps 4530.
Path 447 | total_timesteps 4539.
Path 448 | total_timesteps 4548.
Path 449 | total_timesteps 4557.
Path 450 | total_timesteps 4573.
Path 451 | total_timesteps 4583.
Path 452 | total_timesteps 4590.
Path 453 | total_timesteps 4599.
Path 454 | total_timesteps 4614.
Path 455 | total_timesteps 4625.
Path 456 | total_timesteps 4633.
Path 457 | total_timesteps 4641.
Path 458 | total_timesteps 4650.
Path 459 | total_timesteps 4659.
Path 460 | total_timesteps 4668.
Path 461 | total_timesteps 4675.
Path 462 | total_timesteps 4683.
Path 463 | total_timesteps 4696.
Path 464 | total_timesteps 4707.
Path 465 | total_timesteps 4718.
Path 466 | total_timesteps 4732.
Path 467 | total_timesteps 4743.
Path 468 | total_timesteps 4755.
Path 469 | total_timesteps 4763.
Path 470 | total_timesteps 4772.
Path 471 | total_timesteps 4781.
Path 472 | total_timesteps 4788.
Path 473 | total_timesteps 4797.
Path 474 | total_timesteps 4812.
Path 475 | total_timesteps 4833.
Path 476 | total_timesteps 4842.
Path 477 | total_timesteps 4850.
Path 478 | total_timesteps 4858.
Path 479 | total_timesteps 4867.
Path 480 | total_timesteps 4875.
Path 481 | total_timesteps 4886.
Path 482 | total_timesteps 4896.
Path 483 | total_timesteps 4905.
Path 484 | total_timesteps 4914.
Path 485 | total_timesteps 4920.
Path 486 | total_timesteps 4926.
Path 487 | total_timesteps 4935.
Path 488 | total_timesteps 4945.
Path 489 | total_timesteps 4955.
Path 490 | total_timesteps 4964.
Path 491 | total_timesteps 4972.
Path 492 | total_timesteps 4983.
Path 493 | total_timesteps 4991.
Path 494 | total_timesteps 4999.
Path 495 | total_timesteps 5010.
Path 496 | total_timesteps 5019.
Path 497 | total_timesteps 5030.
Path 498 | total_timesteps 5038.
Path 499 | total_timesteps 5045.
Path 500 | total_timesteps 5060.
Path 501 | total_timesteps 5067.
Path 502 | total_timesteps 5074.
Path 503 | total_timesteps 5088.
Path 504 | total_timesteps 5095.
Path 505 | total_timesteps 5105.
Path 506 | total_timesteps 5116.
Path 507 | total_timesteps 5129.
Path 508 | total_timesteps 5138.
Path 509 | total_timesteps 5145.
Path 510 | total_timesteps 5154.
Path 511 | total_timesteps 5163.
Path 512 | total_timesteps 5170.
Path 513 | total_timesteps 5178.
Path 514 | total_timesteps 5188.
Path 515 | total_timesteps 5194.
Path 516 | total_timesteps 5212.
Path 517 | total_timesteps 5223.
Path 518 | total_timesteps 5231.
Path 519 | total_timesteps 5238.
Path 520 | total_timesteps 5252.
Path 521 | total_timesteps 5263.
Path 522 | total_timesteps 5271.
Path 523 | total_timesteps 5283.
Path 524 | total_timesteps 5292.
Path 525 | total_timesteps 5302.
Path 526 | total_timesteps 5310.
Path 527 | total_timesteps 5322.
Path 528 | total_timesteps 5329.
Path 529 | total_timesteps 5341.
Path 530 | total_timesteps 5348.
Path 531 | total_timesteps 5355.
Path 532 | total_timesteps 5367.
Path 533 | total_timesteps 5379.
Path 534 | total_timesteps 5390.
Path 535 | total_timesteps 5402.
Path 536 | total_timesteps 5414.
Path 537 | total_timesteps 5423.
Path 538 | total_timesteps 5437.
Path 539 | total_timesteps 5447.
Path 540 | total_timesteps 5455.
Path 541 | total_timesteps 5463.
Path 542 | total_timesteps 5473.
Path 543 | total_timesteps 5483.
Path 544 | total_timesteps 5493.
Path 545 | total_timesteps 5504.
Path 546 | total_timesteps 5514.
Path 547 | total_timesteps 5528.
Path 548 | total_timesteps 5542.
Path 549 | total_timesteps 5554.
Path 550 | total_timesteps 5565.
Path 551 | total_timesteps 5576.
Path 552 | total_timesteps 5586.
Path 553 | total_timesteps 5595.
Path 554 | total_timesteps 5602.
Path 555 | total_timesteps 5610.
Path 556 | total_timesteps 5623.
Path 557 | total_timesteps 5634.
Path 558 | total_timesteps 5646.
Path 559 | total_timesteps 5656.
Path 560 | total_timesteps 5664.
Path 561 | total_timesteps 5675.
Path 562 | total_timesteps 5687.
Path 563 | total_timesteps 5697.
Path 564 | total_timesteps 5706.
Path 565 | total_timesteps 5714.
Path 566 | total_timesteps 5727.
Path 567 | total_timesteps 5743.
Path 568 | total_timesteps 5752.
Path 569 | total_timesteps 5761.
Path 570 | total_timesteps 5769.
Path 571 | total_timesteps 5782.
Path 572 | total_timesteps 5796.
Path 573 | total_timesteps 5804.
Path 574 | total_timesteps 5813.
Path 575 | total_timesteps 5824.
Path 576 | total_timesteps 5835.
Path 577 | total_timesteps 5844.
Path 578 | total_timesteps 5854.
Path 579 | total_timesteps 5865.
Path 580 | total_timesteps 5874.
Path 581 | total_timesteps 5882.
Path 582 | total_timesteps 5890.
Path 583 | total_timesteps 5902.
Path 584 | total_timesteps 5926.
Path 585 | total_timesteps 5938.
Path 586 | total_timesteps 5947.
Path 587 | total_timesteps 5956.
Path 588 | total_timesteps 5963.
Path 589 | total_timesteps 5972.
Path 590 | total_timesteps 5981.
Path 591 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.55    |
| Iteration     | 0        |
| MaximumReturn | 0.572    |
| MinimumReturn | -22      |
| TotalSamples  | 8007     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.3478845953941345
Validation loss = 0.2940956950187683
Validation loss = 0.29507315158843994
Validation loss = 0.28825843334198
Validation loss = 0.2962987422943115
Validation loss = 0.3250651955604553
Validation loss = 0.3434027135372162
Validation loss = 0.30068352818489075
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 14.
Path 3 | total_timesteps 22.
Path 4 | total_timesteps 33.
Path 5 | total_timesteps 42.
Path 6 | total_timesteps 53.
Path 7 | total_timesteps 65.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 93.
Path 11 | total_timesteps 102.
Path 12 | total_timesteps 112.
Path 13 | total_timesteps 122.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 146.
Path 16 | total_timesteps 153.
Path 17 | total_timesteps 166.
Path 18 | total_timesteps 174.
Path 19 | total_timesteps 189.
Path 20 | total_timesteps 198.
Path 21 | total_timesteps 213.
Path 22 | total_timesteps 223.
Path 23 | total_timesteps 234.
Path 24 | total_timesteps 247.
Path 25 | total_timesteps 256.
Path 26 | total_timesteps 265.
Path 27 | total_timesteps 272.
Path 28 | total_timesteps 281.
Path 29 | total_timesteps 289.
Path 30 | total_timesteps 299.
Path 31 | total_timesteps 310.
Path 32 | total_timesteps 318.
Path 33 | total_timesteps 325.
Path 34 | total_timesteps 333.
Path 35 | total_timesteps 345.
Path 36 | total_timesteps 354.
Path 37 | total_timesteps 366.
Path 38 | total_timesteps 376.
Path 39 | total_timesteps 383.
Path 40 | total_timesteps 401.
Path 41 | total_timesteps 409.
Path 42 | total_timesteps 418.
Path 43 | total_timesteps 425.
Path 44 | total_timesteps 442.
Path 45 | total_timesteps 453.
Path 46 | total_timesteps 462.
Path 47 | total_timesteps 473.
Path 48 | total_timesteps 479.
Path 49 | total_timesteps 488.
Path 50 | total_timesteps 499.
Path 51 | total_timesteps 508.
Path 52 | total_timesteps 517.
Path 53 | total_timesteps 530.
Path 54 | total_timesteps 542.
Path 55 | total_timesteps 552.
Path 56 | total_timesteps 560.
Path 57 | total_timesteps 575.
Path 58 | total_timesteps 585.
Path 59 | total_timesteps 601.
Path 60 | total_timesteps 614.
Path 61 | total_timesteps 622.
Path 62 | total_timesteps 631.
Path 63 | total_timesteps 638.
Path 64 | total_timesteps 647.
Path 65 | total_timesteps 654.
Path 66 | total_timesteps 665.
Path 67 | total_timesteps 676.
Path 68 | total_timesteps 685.
Path 69 | total_timesteps 696.
Path 70 | total_timesteps 704.
Path 71 | total_timesteps 718.
Path 72 | total_timesteps 731.
Path 73 | total_timesteps 740.
Path 74 | total_timesteps 748.
Path 75 | total_timesteps 755.
Path 76 | total_timesteps 763.
Path 77 | total_timesteps 773.
Path 78 | total_timesteps 782.
Path 79 | total_timesteps 796.
Path 80 | total_timesteps 807.
Path 81 | total_timesteps 816.
Path 82 | total_timesteps 827.
Path 83 | total_timesteps 839.
Path 84 | total_timesteps 847.
Path 85 | total_timesteps 855.
Path 86 | total_timesteps 867.
Path 87 | total_timesteps 874.
Path 88 | total_timesteps 882.
Path 89 | total_timesteps 893.
Path 90 | total_timesteps 909.
Path 91 | total_timesteps 918.
Path 92 | total_timesteps 926.
Path 93 | total_timesteps 936.
Path 94 | total_timesteps 945.
Path 95 | total_timesteps 955.
Path 96 | total_timesteps 964.
Path 97 | total_timesteps 971.
Path 98 | total_timesteps 982.
Path 99 | total_timesteps 995.
Path 100 | total_timesteps 1005.
Path 101 | total_timesteps 1013.
Path 102 | total_timesteps 1024.
Path 103 | total_timesteps 1035.
Path 104 | total_timesteps 1042.
Path 105 | total_timesteps 1049.
Path 106 | total_timesteps 1058.
Path 107 | total_timesteps 1067.
Path 108 | total_timesteps 1080.
Path 109 | total_timesteps 1090.
Path 110 | total_timesteps 1102.
Path 111 | total_timesteps 1110.
Path 112 | total_timesteps 1124.
Path 113 | total_timesteps 1134.
Path 114 | total_timesteps 1149.
Path 115 | total_timesteps 1158.
Path 116 | total_timesteps 1171.
Path 117 | total_timesteps 1179.
Path 118 | total_timesteps 1188.
Path 119 | total_timesteps 1195.
Path 120 | total_timesteps 1206.
Path 121 | total_timesteps 1217.
Path 122 | total_timesteps 1228.
Path 123 | total_timesteps 1236.
Path 124 | total_timesteps 1245.
Path 125 | total_timesteps 1255.
Path 126 | total_timesteps 1267.
Path 127 | total_timesteps 1281.
Path 128 | total_timesteps 1296.
Path 129 | total_timesteps 1305.
Path 130 | total_timesteps 1313.
Path 131 | total_timesteps 1322.
Path 132 | total_timesteps 1333.
Path 133 | total_timesteps 1345.
Path 134 | total_timesteps 1354.
Path 135 | total_timesteps 1364.
Path 136 | total_timesteps 1373.
Path 137 | total_timesteps 1380.
Path 138 | total_timesteps 1389.
Path 139 | total_timesteps 1401.
Path 140 | total_timesteps 1415.
Path 141 | total_timesteps 1424.
Path 142 | total_timesteps 1434.
Path 143 | total_timesteps 1441.
Path 144 | total_timesteps 1451.
Path 145 | total_timesteps 1461.
Path 146 | total_timesteps 1470.
Path 147 | total_timesteps 1483.
Path 148 | total_timesteps 1493.
Path 149 | total_timesteps 1502.
Path 150 | total_timesteps 1510.
Path 151 | total_timesteps 1521.
Path 152 | total_timesteps 1532.
Path 153 | total_timesteps 1541.
Path 154 | total_timesteps 1553.
Path 155 | total_timesteps 1565.
Path 156 | total_timesteps 1576.
Path 157 | total_timesteps 1595.
Path 158 | total_timesteps 1601.
Path 159 | total_timesteps 1611.
Path 160 | total_timesteps 1624.
Path 161 | total_timesteps 1648.
Path 162 | total_timesteps 1657.
Path 163 | total_timesteps 1663.
Path 164 | total_timesteps 1671.
Path 165 | total_timesteps 1682.
Path 166 | total_timesteps 1693.
Path 167 | total_timesteps 1702.
Path 168 | total_timesteps 1709.
Path 169 | total_timesteps 1717.
Path 170 | total_timesteps 1725.
Path 171 | total_timesteps 1734.
Path 172 | total_timesteps 1749.
Path 173 | total_timesteps 1759.
Path 174 | total_timesteps 1771.
Path 175 | total_timesteps 1779.
Path 176 | total_timesteps 1787.
Path 177 | total_timesteps 1795.
Path 178 | total_timesteps 1806.
Path 179 | total_timesteps 1820.
Path 180 | total_timesteps 1832.
Path 181 | total_timesteps 1850.
Path 182 | total_timesteps 1860.
Path 183 | total_timesteps 1867.
Path 184 | total_timesteps 1877.
Path 185 | total_timesteps 1888.
Path 186 | total_timesteps 1898.
Path 187 | total_timesteps 1906.
Path 188 | total_timesteps 1917.
Path 189 | total_timesteps 1930.
Path 190 | total_timesteps 1940.
Path 191 | total_timesteps 1950.
Path 192 | total_timesteps 1962.
Path 193 | total_timesteps 1969.
Path 194 | total_timesteps 1977.
Path 195 | total_timesteps 1985.
Path 196 | total_timesteps 1996.
Path 197 | total_timesteps 2009.
Path 198 | total_timesteps 2017.
Path 199 | total_timesteps 2028.
Path 200 | total_timesteps 2039.
Path 201 | total_timesteps 2049.
Path 202 | total_timesteps 2063.
Path 203 | total_timesteps 2071.
Path 204 | total_timesteps 2082.
Path 205 | total_timesteps 2099.
Path 206 | total_timesteps 2110.
Path 207 | total_timesteps 2119.
Path 208 | total_timesteps 2126.
Path 209 | total_timesteps 2139.
Path 210 | total_timesteps 2157.
Path 211 | total_timesteps 2166.
Path 212 | total_timesteps 2176.
Path 213 | total_timesteps 2188.
Path 214 | total_timesteps 2199.
Path 215 | total_timesteps 2212.
Path 216 | total_timesteps 2219.
Path 217 | total_timesteps 2227.
Path 218 | total_timesteps 2237.
Path 219 | total_timesteps 2244.
Path 220 | total_timesteps 2257.
Path 221 | total_timesteps 2270.
Path 222 | total_timesteps 2278.
Path 223 | total_timesteps 2291.
Path 224 | total_timesteps 2302.
Path 225 | total_timesteps 2317.
Path 226 | total_timesteps 2327.
Path 227 | total_timesteps 2336.
Path 228 | total_timesteps 2346.
Path 229 | total_timesteps 2364.
Path 230 | total_timesteps 2371.
Path 231 | total_timesteps 2381.
Path 232 | total_timesteps 2389.
Path 233 | total_timesteps 2400.
Path 234 | total_timesteps 2407.
Path 235 | total_timesteps 2415.
Path 236 | total_timesteps 2426.
Path 237 | total_timesteps 2436.
Path 238 | total_timesteps 2447.
Path 239 | total_timesteps 2458.
Path 240 | total_timesteps 2466.
Path 241 | total_timesteps 2482.
Path 242 | total_timesteps 2493.
Path 243 | total_timesteps 2502.
Path 244 | total_timesteps 2511.
Path 245 | total_timesteps 2521.
Path 246 | total_timesteps 2528.
Path 247 | total_timesteps 2534.
Path 248 | total_timesteps 2546.
Path 249 | total_timesteps 2555.
Path 250 | total_timesteps 2563.
Path 251 | total_timesteps 2578.
Path 252 | total_timesteps 2586.
Path 253 | total_timesteps 2593.
Path 254 | total_timesteps 2601.
Path 255 | total_timesteps 2608.
Path 256 | total_timesteps 2620.
Path 257 | total_timesteps 2627.
Path 258 | total_timesteps 2636.
Path 259 | total_timesteps 2648.
Path 260 | total_timesteps 2663.
Path 261 | total_timesteps 2671.
Path 262 | total_timesteps 2681.
Path 263 | total_timesteps 2687.
Path 264 | total_timesteps 2695.
Path 265 | total_timesteps 2702.
Path 266 | total_timesteps 2714.
Path 267 | total_timesteps 2727.
Path 268 | total_timesteps 2740.
Path 269 | total_timesteps 2753.
Path 270 | total_timesteps 2763.
Path 271 | total_timesteps 2775.
Path 272 | total_timesteps 2785.
Path 273 | total_timesteps 2794.
Path 274 | total_timesteps 2801.
Path 275 | total_timesteps 2810.
Path 276 | total_timesteps 2823.
Path 277 | total_timesteps 2832.
Path 278 | total_timesteps 2843.
Path 279 | total_timesteps 2851.
Path 280 | total_timesteps 2860.
Path 281 | total_timesteps 2868.
Path 282 | total_timesteps 2878.
Path 283 | total_timesteps 2886.
Path 284 | total_timesteps 2900.
Path 285 | total_timesteps 2909.
Path 286 | total_timesteps 2916.
Path 287 | total_timesteps 2930.
Path 288 | total_timesteps 2942.
Path 289 | total_timesteps 2952.
Path 290 | total_timesteps 2960.
Path 291 | total_timesteps 2973.
Path 292 | total_timesteps 2986.
Path 293 | total_timesteps 2997.
Path 294 | total_timesteps 3016.
Path 295 | total_timesteps 3033.
Path 296 | total_timesteps 3040.
Path 297 | total_timesteps 3052.
Path 298 | total_timesteps 3060.
Path 299 | total_timesteps 3069.
Path 300 | total_timesteps 3080.
Path 301 | total_timesteps 3090.
Path 302 | total_timesteps 3100.
Path 303 | total_timesteps 3114.
Path 304 | total_timesteps 3122.
Path 305 | total_timesteps 3132.
Path 306 | total_timesteps 3144.
Path 307 | total_timesteps 3153.
Path 308 | total_timesteps 3164.
Path 309 | total_timesteps 3172.
Path 310 | total_timesteps 3180.
Path 311 | total_timesteps 3189.
Path 312 | total_timesteps 3198.
Path 313 | total_timesteps 3208.
Path 314 | total_timesteps 3215.
Path 315 | total_timesteps 3224.
Path 316 | total_timesteps 3231.
Path 317 | total_timesteps 3247.
Path 318 | total_timesteps 3254.
Path 319 | total_timesteps 3263.
Path 320 | total_timesteps 3274.
Path 321 | total_timesteps 3285.
Path 322 | total_timesteps 3297.
Path 323 | total_timesteps 3305.
Path 324 | total_timesteps 3313.
Path 325 | total_timesteps 3323.
Path 326 | total_timesteps 3332.
Path 327 | total_timesteps 3339.
Path 328 | total_timesteps 3347.
Path 329 | total_timesteps 3359.
Path 330 | total_timesteps 3368.
Path 331 | total_timesteps 3376.
Path 332 | total_timesteps 3387.
Path 333 | total_timesteps 3395.
Path 334 | total_timesteps 3403.
Path 335 | total_timesteps 3414.
Path 336 | total_timesteps 3420.
Path 337 | total_timesteps 3428.
Path 338 | total_timesteps 3444.
Path 339 | total_timesteps 3452.
Path 340 | total_timesteps 3459.
Path 341 | total_timesteps 3466.
Path 342 | total_timesteps 3485.
Path 343 | total_timesteps 3496.
Path 344 | total_timesteps 3511.
Path 345 | total_timesteps 3520.
Path 346 | total_timesteps 3530.
Path 347 | total_timesteps 3540.
Path 348 | total_timesteps 3547.
Path 349 | total_timesteps 3557.
Path 350 | total_timesteps 3564.
Path 351 | total_timesteps 3575.
Path 352 | total_timesteps 3582.
Path 353 | total_timesteps 3597.
Path 354 | total_timesteps 3609.
Path 355 | total_timesteps 3618.
Path 356 | total_timesteps 3628.
Path 357 | total_timesteps 3637.
Path 358 | total_timesteps 3646.
Path 359 | total_timesteps 3653.
Path 360 | total_timesteps 3663.
Path 361 | total_timesteps 3672.
Path 362 | total_timesteps 3684.
Path 363 | total_timesteps 3695.
Path 364 | total_timesteps 3704.
Path 365 | total_timesteps 3711.
Path 366 | total_timesteps 3725.
Path 367 | total_timesteps 3736.
Path 368 | total_timesteps 3746.
Path 369 | total_timesteps 3757.
Path 370 | total_timesteps 3765.
Path 371 | total_timesteps 3773.
Path 372 | total_timesteps 3785.
Path 373 | total_timesteps 3795.
Path 374 | total_timesteps 3808.
Path 375 | total_timesteps 3820.
Path 376 | total_timesteps 3840.
Path 377 | total_timesteps 3855.
Path 378 | total_timesteps 3863.
Path 379 | total_timesteps 3879.
Path 380 | total_timesteps 3891.
Path 381 | total_timesteps 3903.
Path 382 | total_timesteps 3910.
Path 383 | total_timesteps 3918.
Path 384 | total_timesteps 3933.
Path 385 | total_timesteps 3944.
Path 386 | total_timesteps 3953.
Path 387 | total_timesteps 3963.
Path 388 | total_timesteps 3970.
Path 389 | total_timesteps 3979.
Path 390 | total_timesteps 3999.
Path 391 | total_timesteps 4013.
Path 392 | total_timesteps 4021.
Path 393 | total_timesteps 4031.
Path 394 | total_timesteps 4041.
Path 395 | total_timesteps 4055.
Path 396 | total_timesteps 4062.
Path 397 | total_timesteps 4072.
Path 398 | total_timesteps 4079.
Path 399 | total_timesteps 4086.
Path 400 | total_timesteps 4100.
Path 401 | total_timesteps 4113.
Path 402 | total_timesteps 4124.
Path 403 | total_timesteps 4132.
Path 404 | total_timesteps 4144.
Path 405 | total_timesteps 4160.
Path 406 | total_timesteps 4171.
Path 407 | total_timesteps 4179.
Path 408 | total_timesteps 4187.
Path 409 | total_timesteps 4197.
Path 410 | total_timesteps 4208.
Path 411 | total_timesteps 4219.
Path 412 | total_timesteps 4226.
Path 413 | total_timesteps 4236.
Path 414 | total_timesteps 4245.
Path 415 | total_timesteps 4254.
Path 416 | total_timesteps 4264.
Path 417 | total_timesteps 4273.
Path 418 | total_timesteps 4281.
Path 419 | total_timesteps 4298.
Path 420 | total_timesteps 4304.
Path 421 | total_timesteps 4311.
Path 422 | total_timesteps 4322.
Path 423 | total_timesteps 4346.
Path 424 | total_timesteps 4356.
Path 425 | total_timesteps 4377.
Path 426 | total_timesteps 4386.
Path 427 | total_timesteps 4393.
Path 428 | total_timesteps 4401.
Path 429 | total_timesteps 4412.
Path 430 | total_timesteps 4419.
Path 431 | total_timesteps 4428.
Path 432 | total_timesteps 4438.
Path 433 | total_timesteps 4448.
Path 434 | total_timesteps 4455.
Path 435 | total_timesteps 4462.
Path 436 | total_timesteps 4471.
Path 437 | total_timesteps 4485.
Path 438 | total_timesteps 4497.
Path 439 | total_timesteps 4511.
Path 440 | total_timesteps 4521.
Path 441 | total_timesteps 4529.
Path 442 | total_timesteps 4538.
Path 443 | total_timesteps 4551.
Path 444 | total_timesteps 4567.
Path 445 | total_timesteps 4576.
Path 446 | total_timesteps 4588.
Path 447 | total_timesteps 4596.
Path 448 | total_timesteps 4607.
Path 449 | total_timesteps 4615.
Path 450 | total_timesteps 4624.
Path 451 | total_timesteps 4637.
Path 452 | total_timesteps 4647.
Path 453 | total_timesteps 4659.
Path 454 | total_timesteps 4667.
Path 455 | total_timesteps 4685.
Path 456 | total_timesteps 4692.
Path 457 | total_timesteps 4704.
Path 458 | total_timesteps 4713.
Path 459 | total_timesteps 4722.
Path 460 | total_timesteps 4730.
Path 461 | total_timesteps 4739.
Path 462 | total_timesteps 4748.
Path 463 | total_timesteps 4757.
Path 464 | total_timesteps 4766.
Path 465 | total_timesteps 4773.
Path 466 | total_timesteps 4785.
Path 467 | total_timesteps 4794.
Path 468 | total_timesteps 4803.
Path 469 | total_timesteps 4812.
Path 470 | total_timesteps 4819.
Path 471 | total_timesteps 4830.
Path 472 | total_timesteps 4838.
Path 473 | total_timesteps 4848.
Path 474 | total_timesteps 4860.
Path 475 | total_timesteps 4880.
Path 476 | total_timesteps 4898.
Path 477 | total_timesteps 4906.
Path 478 | total_timesteps 4917.
Path 479 | total_timesteps 4930.
Path 480 | total_timesteps 4939.
Path 481 | total_timesteps 4949.
Path 482 | total_timesteps 4960.
Path 483 | total_timesteps 4969.
Path 484 | total_timesteps 4979.
Path 485 | total_timesteps 4987.
Path 486 | total_timesteps 4997.
Path 487 | total_timesteps 5007.
Path 488 | total_timesteps 5018.
Path 489 | total_timesteps 5030.
Path 490 | total_timesteps 5041.
Path 491 | total_timesteps 5055.
Path 492 | total_timesteps 5063.
Path 493 | total_timesteps 5077.
Path 494 | total_timesteps 5084.
Path 495 | total_timesteps 5095.
Path 496 | total_timesteps 5107.
Path 497 | total_timesteps 5118.
Path 498 | total_timesteps 5127.
Path 499 | total_timesteps 5137.
Path 500 | total_timesteps 5147.
Path 501 | total_timesteps 5154.
Path 502 | total_timesteps 5168.
Path 503 | total_timesteps 5175.
Path 504 | total_timesteps 5183.
Path 505 | total_timesteps 5192.
Path 506 | total_timesteps 5209.
Path 507 | total_timesteps 5220.
Path 508 | total_timesteps 5228.
Path 509 | total_timesteps 5237.
Path 510 | total_timesteps 5251.
Path 511 | total_timesteps 5260.
Path 512 | total_timesteps 5270.
Path 513 | total_timesteps 5277.
Path 514 | total_timesteps 5286.
Path 515 | total_timesteps 5294.
Path 516 | total_timesteps 5301.
Path 517 | total_timesteps 5307.
Path 518 | total_timesteps 5317.
Path 519 | total_timesteps 5331.
Path 520 | total_timesteps 5341.
Path 521 | total_timesteps 5348.
Path 522 | total_timesteps 5356.
Path 523 | total_timesteps 5365.
Path 524 | total_timesteps 5374.
Path 525 | total_timesteps 5387.
Path 526 | total_timesteps 5395.
Path 527 | total_timesteps 5403.
Path 528 | total_timesteps 5413.
Path 529 | total_timesteps 5423.
Path 530 | total_timesteps 5432.
Path 531 | total_timesteps 5443.
Path 532 | total_timesteps 5455.
Path 533 | total_timesteps 5464.
Path 534 | total_timesteps 5473.
Path 535 | total_timesteps 5482.
Path 536 | total_timesteps 5493.
Path 537 | total_timesteps 5504.
Path 538 | total_timesteps 5512.
Path 539 | total_timesteps 5520.
Path 540 | total_timesteps 5530.
Path 541 | total_timesteps 5543.
Path 542 | total_timesteps 5552.
Path 543 | total_timesteps 5564.
Path 544 | total_timesteps 5572.
Path 545 | total_timesteps 5582.
Path 546 | total_timesteps 5592.
Path 547 | total_timesteps 5602.
Path 548 | total_timesteps 5616.
Path 549 | total_timesteps 5623.
Path 550 | total_timesteps 5633.
Path 551 | total_timesteps 5647.
Path 552 | total_timesteps 5659.
Path 553 | total_timesteps 5674.
Path 554 | total_timesteps 5683.
Path 555 | total_timesteps 5692.
Path 556 | total_timesteps 5703.
Path 557 | total_timesteps 5711.
Path 558 | total_timesteps 5722.
Path 559 | total_timesteps 5731.
Path 560 | total_timesteps 5738.
Path 561 | total_timesteps 5749.
Path 562 | total_timesteps 5758.
Path 563 | total_timesteps 5769.
Path 564 | total_timesteps 5776.
Path 565 | total_timesteps 5789.
Path 566 | total_timesteps 5802.
Path 567 | total_timesteps 5810.
Path 568 | total_timesteps 5821.
Path 569 | total_timesteps 5829.
Path 570 | total_timesteps 5850.
Path 571 | total_timesteps 5859.
Path 572 | total_timesteps 5871.
Path 573 | total_timesteps 5879.
Path 574 | total_timesteps 5895.
Path 575 | total_timesteps 5907.
Path 576 | total_timesteps 5917.
Path 577 | total_timesteps 5929.
Path 578 | total_timesteps 5941.
Path 579 | total_timesteps 5957.
Path 580 | total_timesteps 5966.
Path 581 | total_timesteps 5978.
Path 582 | total_timesteps 5985.
Path 583 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.67    |
| Iteration     | 1        |
| MaximumReturn | 1.79     |
| MinimumReturn | -17.4    |
| TotalSamples  | 12008    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.28583621978759766
Validation loss = 0.2860032618045807
Validation loss = 0.2958466708660126
Validation loss = 0.29010188579559326
Validation loss = 0.3124888837337494
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 75.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 91.
Path 11 | total_timesteps 105.
Path 12 | total_timesteps 114.
Path 13 | total_timesteps 123.
Path 14 | total_timesteps 130.
Path 15 | total_timesteps 139.
Path 16 | total_timesteps 148.
Path 17 | total_timesteps 157.
Path 18 | total_timesteps 166.
Path 19 | total_timesteps 176.
Path 20 | total_timesteps 187.
Path 21 | total_timesteps 194.
Path 22 | total_timesteps 209.
Path 23 | total_timesteps 220.
Path 24 | total_timesteps 227.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 246.
Path 27 | total_timesteps 254.
Path 28 | total_timesteps 263.
Path 29 | total_timesteps 272.
Path 30 | total_timesteps 284.
Path 31 | total_timesteps 292.
Path 32 | total_timesteps 302.
Path 33 | total_timesteps 312.
Path 34 | total_timesteps 322.
Path 35 | total_timesteps 331.
Path 36 | total_timesteps 339.
Path 37 | total_timesteps 350.
Path 38 | total_timesteps 359.
Path 39 | total_timesteps 370.
Path 40 | total_timesteps 379.
Path 41 | total_timesteps 389.
Path 42 | total_timesteps 400.
Path 43 | total_timesteps 412.
Path 44 | total_timesteps 421.
Path 45 | total_timesteps 430.
Path 46 | total_timesteps 440.
Path 47 | total_timesteps 449.
Path 48 | total_timesteps 462.
Path 49 | total_timesteps 474.
Path 50 | total_timesteps 484.
Path 51 | total_timesteps 504.
Path 52 | total_timesteps 517.
Path 53 | total_timesteps 526.
Path 54 | total_timesteps 535.
Path 55 | total_timesteps 548.
Path 56 | total_timesteps 557.
Path 57 | total_timesteps 567.
Path 58 | total_timesteps 578.
Path 59 | total_timesteps 586.
Path 60 | total_timesteps 592.
Path 61 | total_timesteps 601.
Path 62 | total_timesteps 610.
Path 63 | total_timesteps 623.
Path 64 | total_timesteps 632.
Path 65 | total_timesteps 640.
Path 66 | total_timesteps 648.
Path 67 | total_timesteps 660.
Path 68 | total_timesteps 670.
Path 69 | total_timesteps 678.
Path 70 | total_timesteps 686.
Path 71 | total_timesteps 700.
Path 72 | total_timesteps 708.
Path 73 | total_timesteps 716.
Path 74 | total_timesteps 725.
Path 75 | total_timesteps 736.
Path 76 | total_timesteps 751.
Path 77 | total_timesteps 761.
Path 78 | total_timesteps 774.
Path 79 | total_timesteps 784.
Path 80 | total_timesteps 793.
Path 81 | total_timesteps 805.
Path 82 | total_timesteps 812.
Path 83 | total_timesteps 829.
Path 84 | total_timesteps 839.
Path 85 | total_timesteps 849.
Path 86 | total_timesteps 857.
Path 87 | total_timesteps 865.
Path 88 | total_timesteps 873.
Path 89 | total_timesteps 883.
Path 90 | total_timesteps 889.
Path 91 | total_timesteps 898.
Path 92 | total_timesteps 904.
Path 93 | total_timesteps 913.
Path 94 | total_timesteps 922.
Path 95 | total_timesteps 931.
Path 96 | total_timesteps 939.
Path 97 | total_timesteps 946.
Path 98 | total_timesteps 954.
Path 99 | total_timesteps 962.
Path 100 | total_timesteps 973.
Path 101 | total_timesteps 983.
Path 102 | total_timesteps 1002.
Path 103 | total_timesteps 1012.
Path 104 | total_timesteps 1019.
Path 105 | total_timesteps 1028.
Path 106 | total_timesteps 1036.
Path 107 | total_timesteps 1054.
Path 108 | total_timesteps 1064.
Path 109 | total_timesteps 1074.
Path 110 | total_timesteps 1081.
Path 111 | total_timesteps 1090.
Path 112 | total_timesteps 1101.
Path 113 | total_timesteps 1112.
Path 114 | total_timesteps 1125.
Path 115 | total_timesteps 1139.
Path 116 | total_timesteps 1150.
Path 117 | total_timesteps 1162.
Path 118 | total_timesteps 1171.
Path 119 | total_timesteps 1180.
Path 120 | total_timesteps 1189.
Path 121 | total_timesteps 1199.
Path 122 | total_timesteps 1209.
Path 123 | total_timesteps 1220.
Path 124 | total_timesteps 1238.
Path 125 | total_timesteps 1246.
Path 126 | total_timesteps 1254.
Path 127 | total_timesteps 1263.
Path 128 | total_timesteps 1270.
Path 129 | total_timesteps 1279.
Path 130 | total_timesteps 1289.
Path 131 | total_timesteps 1299.
Path 132 | total_timesteps 1306.
Path 133 | total_timesteps 1314.
Path 134 | total_timesteps 1326.
Path 135 | total_timesteps 1333.
Path 136 | total_timesteps 1341.
Path 137 | total_timesteps 1349.
Path 138 | total_timesteps 1358.
Path 139 | total_timesteps 1371.
Path 140 | total_timesteps 1381.
Path 141 | total_timesteps 1390.
Path 142 | total_timesteps 1404.
Path 143 | total_timesteps 1414.
Path 144 | total_timesteps 1430.
Path 145 | total_timesteps 1440.
Path 146 | total_timesteps 1451.
Path 147 | total_timesteps 1462.
Path 148 | total_timesteps 1472.
Path 149 | total_timesteps 1480.
Path 150 | total_timesteps 1488.
Path 151 | total_timesteps 1506.
Path 152 | total_timesteps 1516.
Path 153 | total_timesteps 1525.
Path 154 | total_timesteps 1534.
Path 155 | total_timesteps 1547.
Path 156 | total_timesteps 1555.
Path 157 | total_timesteps 1565.
Path 158 | total_timesteps 1576.
Path 159 | total_timesteps 1583.
Path 160 | total_timesteps 1593.
Path 161 | total_timesteps 1602.
Path 162 | total_timesteps 1614.
Path 163 | total_timesteps 1625.
Path 164 | total_timesteps 1638.
Path 165 | total_timesteps 1645.
Path 166 | total_timesteps 1654.
Path 167 | total_timesteps 1665.
Path 168 | total_timesteps 1678.
Path 169 | total_timesteps 1687.
Path 170 | total_timesteps 1696.
Path 171 | total_timesteps 1703.
Path 172 | total_timesteps 1711.
Path 173 | total_timesteps 1719.
Path 174 | total_timesteps 1732.
Path 175 | total_timesteps 1742.
Path 176 | total_timesteps 1752.
Path 177 | total_timesteps 1761.
Path 178 | total_timesteps 1771.
Path 179 | total_timesteps 1779.
Path 180 | total_timesteps 1790.
Path 181 | total_timesteps 1807.
Path 182 | total_timesteps 1818.
Path 183 | total_timesteps 1824.
Path 184 | total_timesteps 1837.
Path 185 | total_timesteps 1844.
Path 186 | total_timesteps 1852.
Path 187 | total_timesteps 1868.
Path 188 | total_timesteps 1876.
Path 189 | total_timesteps 1885.
Path 190 | total_timesteps 1895.
Path 191 | total_timesteps 1907.
Path 192 | total_timesteps 1917.
Path 193 | total_timesteps 1929.
Path 194 | total_timesteps 1942.
Path 195 | total_timesteps 1949.
Path 196 | total_timesteps 1959.
Path 197 | total_timesteps 1968.
Path 198 | total_timesteps 1976.
Path 199 | total_timesteps 1985.
Path 200 | total_timesteps 1994.
Path 201 | total_timesteps 2004.
Path 202 | total_timesteps 2011.
Path 203 | total_timesteps 2018.
Path 204 | total_timesteps 2028.
Path 205 | total_timesteps 2036.
Path 206 | total_timesteps 2048.
Path 207 | total_timesteps 2054.
Path 208 | total_timesteps 2064.
Path 209 | total_timesteps 2074.
Path 210 | total_timesteps 2088.
Path 211 | total_timesteps 2101.
Path 212 | total_timesteps 2107.
Path 213 | total_timesteps 2120.
Path 214 | total_timesteps 2127.
Path 215 | total_timesteps 2140.
Path 216 | total_timesteps 2152.
Path 217 | total_timesteps 2166.
Path 218 | total_timesteps 2177.
Path 219 | total_timesteps 2189.
Path 220 | total_timesteps 2200.
Path 221 | total_timesteps 2209.
Path 222 | total_timesteps 2225.
Path 223 | total_timesteps 2235.
Path 224 | total_timesteps 2243.
Path 225 | total_timesteps 2256.
Path 226 | total_timesteps 2267.
Path 227 | total_timesteps 2278.
Path 228 | total_timesteps 2289.
Path 229 | total_timesteps 2298.
Path 230 | total_timesteps 2306.
Path 231 | total_timesteps 2316.
Path 232 | total_timesteps 2324.
Path 233 | total_timesteps 2331.
Path 234 | total_timesteps 2339.
Path 235 | total_timesteps 2354.
Path 236 | total_timesteps 2360.
Path 237 | total_timesteps 2369.
Path 238 | total_timesteps 2377.
Path 239 | total_timesteps 2390.
Path 240 | total_timesteps 2399.
Path 241 | total_timesteps 2407.
Path 242 | total_timesteps 2419.
Path 243 | total_timesteps 2428.
Path 244 | total_timesteps 2440.
Path 245 | total_timesteps 2449.
Path 246 | total_timesteps 2459.
Path 247 | total_timesteps 2467.
Path 248 | total_timesteps 2475.
Path 249 | total_timesteps 2483.
Path 250 | total_timesteps 2492.
Path 251 | total_timesteps 2502.
Path 252 | total_timesteps 2511.
Path 253 | total_timesteps 2518.
Path 254 | total_timesteps 2533.
Path 255 | total_timesteps 2541.
Path 256 | total_timesteps 2553.
Path 257 | total_timesteps 2563.
Path 258 | total_timesteps 2574.
Path 259 | total_timesteps 2588.
Path 260 | total_timesteps 2595.
Path 261 | total_timesteps 2609.
Path 262 | total_timesteps 2620.
Path 263 | total_timesteps 2626.
Path 264 | total_timesteps 2632.
Path 265 | total_timesteps 2647.
Path 266 | total_timesteps 2664.
Path 267 | total_timesteps 2671.
Path 268 | total_timesteps 2680.
Path 269 | total_timesteps 2689.
Path 270 | total_timesteps 2697.
Path 271 | total_timesteps 2705.
Path 272 | total_timesteps 2718.
Path 273 | total_timesteps 2726.
Path 274 | total_timesteps 2733.
Path 275 | total_timesteps 2740.
Path 276 | total_timesteps 2748.
Path 277 | total_timesteps 2757.
Path 278 | total_timesteps 2769.
Path 279 | total_timesteps 2777.
Path 280 | total_timesteps 2787.
Path 281 | total_timesteps 2793.
Path 282 | total_timesteps 2803.
Path 283 | total_timesteps 2813.
Path 284 | total_timesteps 2823.
Path 285 | total_timesteps 2834.
Path 286 | total_timesteps 2843.
Path 287 | total_timesteps 2854.
Path 288 | total_timesteps 2861.
Path 289 | total_timesteps 2872.
Path 290 | total_timesteps 2887.
Path 291 | total_timesteps 2896.
Path 292 | total_timesteps 2905.
Path 293 | total_timesteps 2914.
Path 294 | total_timesteps 2923.
Path 295 | total_timesteps 2931.
Path 296 | total_timesteps 2940.
Path 297 | total_timesteps 2949.
Path 298 | total_timesteps 2958.
Path 299 | total_timesteps 2970.
Path 300 | total_timesteps 2978.
Path 301 | total_timesteps 2988.
Path 302 | total_timesteps 2997.
Path 303 | total_timesteps 3005.
Path 304 | total_timesteps 3014.
Path 305 | total_timesteps 3023.
Path 306 | total_timesteps 3039.
Path 307 | total_timesteps 3048.
Path 308 | total_timesteps 3058.
Path 309 | total_timesteps 3071.
Path 310 | total_timesteps 3080.
Path 311 | total_timesteps 3091.
Path 312 | total_timesteps 3104.
Path 313 | total_timesteps 3111.
Path 314 | total_timesteps 3124.
Path 315 | total_timesteps 3132.
Path 316 | total_timesteps 3142.
Path 317 | total_timesteps 3153.
Path 318 | total_timesteps 3160.
Path 319 | total_timesteps 3172.
Path 320 | total_timesteps 3185.
Path 321 | total_timesteps 3197.
Path 322 | total_timesteps 3208.
Path 323 | total_timesteps 3218.
Path 324 | total_timesteps 3225.
Path 325 | total_timesteps 3235.
Path 326 | total_timesteps 3248.
Path 327 | total_timesteps 3259.
Path 328 | total_timesteps 3270.
Path 329 | total_timesteps 3280.
Path 330 | total_timesteps 3292.
Path 331 | total_timesteps 3303.
Path 332 | total_timesteps 3311.
Path 333 | total_timesteps 3325.
Path 334 | total_timesteps 3342.
Path 335 | total_timesteps 3351.
Path 336 | total_timesteps 3358.
Path 337 | total_timesteps 3370.
Path 338 | total_timesteps 3379.
Path 339 | total_timesteps 3389.
Path 340 | total_timesteps 3401.
Path 341 | total_timesteps 3411.
Path 342 | total_timesteps 3425.
Path 343 | total_timesteps 3434.
Path 344 | total_timesteps 3450.
Path 345 | total_timesteps 3462.
Path 346 | total_timesteps 3471.
Path 347 | total_timesteps 3481.
Path 348 | total_timesteps 3497.
Path 349 | total_timesteps 3504.
Path 350 | total_timesteps 3514.
Path 351 | total_timesteps 3525.
Path 352 | total_timesteps 3536.
Path 353 | total_timesteps 3545.
Path 354 | total_timesteps 3555.
Path 355 | total_timesteps 3564.
Path 356 | total_timesteps 3573.
Path 357 | total_timesteps 3583.
Path 358 | total_timesteps 3595.
Path 359 | total_timesteps 3603.
Path 360 | total_timesteps 3621.
Path 361 | total_timesteps 3633.
Path 362 | total_timesteps 3643.
Path 363 | total_timesteps 3652.
Path 364 | total_timesteps 3661.
Path 365 | total_timesteps 3676.
Path 366 | total_timesteps 3686.
Path 367 | total_timesteps 3697.
Path 368 | total_timesteps 3706.
Path 369 | total_timesteps 3718.
Path 370 | total_timesteps 3729.
Path 371 | total_timesteps 3737.
Path 372 | total_timesteps 3745.
Path 373 | total_timesteps 3753.
Path 374 | total_timesteps 3763.
Path 375 | total_timesteps 3775.
Path 376 | total_timesteps 3786.
Path 377 | total_timesteps 3796.
Path 378 | total_timesteps 3805.
Path 379 | total_timesteps 3812.
Path 380 | total_timesteps 3820.
Path 381 | total_timesteps 3827.
Path 382 | total_timesteps 3838.
Path 383 | total_timesteps 3849.
Path 384 | total_timesteps 3857.
Path 385 | total_timesteps 3865.
Path 386 | total_timesteps 3875.
Path 387 | total_timesteps 3884.
Path 388 | total_timesteps 3894.
Path 389 | total_timesteps 3902.
Path 390 | total_timesteps 3911.
Path 391 | total_timesteps 3919.
Path 392 | total_timesteps 3934.
Path 393 | total_timesteps 3942.
Path 394 | total_timesteps 3949.
Path 395 | total_timesteps 3964.
Path 396 | total_timesteps 3972.
Path 397 | total_timesteps 3982.
Path 398 | total_timesteps 3991.
Path 399 | total_timesteps 4004.
Path 400 | total_timesteps 4017.
Path 401 | total_timesteps 4026.
Path 402 | total_timesteps 4038.
Path 403 | total_timesteps 4059.
Path 404 | total_timesteps 4068.
Path 405 | total_timesteps 4076.
Path 406 | total_timesteps 4085.
Path 407 | total_timesteps 4093.
Path 408 | total_timesteps 4101.
Path 409 | total_timesteps 4110.
Path 410 | total_timesteps 4120.
Path 411 | total_timesteps 4128.
Path 412 | total_timesteps 4138.
Path 413 | total_timesteps 4147.
Path 414 | total_timesteps 4155.
Path 415 | total_timesteps 4167.
Path 416 | total_timesteps 4176.
Path 417 | total_timesteps 4186.
Path 418 | total_timesteps 4197.
Path 419 | total_timesteps 4206.
Path 420 | total_timesteps 4216.
Path 421 | total_timesteps 4224.
Path 422 | total_timesteps 4234.
Path 423 | total_timesteps 4246.
Path 424 | total_timesteps 4255.
Path 425 | total_timesteps 4264.
Path 426 | total_timesteps 4284.
Path 427 | total_timesteps 4293.
Path 428 | total_timesteps 4300.
Path 429 | total_timesteps 4311.
Path 430 | total_timesteps 4321.
Path 431 | total_timesteps 4328.
Path 432 | total_timesteps 4338.
Path 433 | total_timesteps 4347.
Path 434 | total_timesteps 4357.
Path 435 | total_timesteps 4368.
Path 436 | total_timesteps 4387.
Path 437 | total_timesteps 4399.
Path 438 | total_timesteps 4407.
Path 439 | total_timesteps 4417.
Path 440 | total_timesteps 4426.
Path 441 | total_timesteps 4435.
Path 442 | total_timesteps 4443.
Path 443 | total_timesteps 4457.
Path 444 | total_timesteps 4465.
Path 445 | total_timesteps 4474.
Path 446 | total_timesteps 4484.
Path 447 | total_timesteps 4492.
Path 448 | total_timesteps 4500.
Path 449 | total_timesteps 4512.
Path 450 | total_timesteps 4521.
Path 451 | total_timesteps 4529.
Path 452 | total_timesteps 4537.
Path 453 | total_timesteps 4546.
Path 454 | total_timesteps 4559.
Path 455 | total_timesteps 4568.
Path 456 | total_timesteps 4576.
Path 457 | total_timesteps 4585.
Path 458 | total_timesteps 4598.
Path 459 | total_timesteps 4609.
Path 460 | total_timesteps 4618.
Path 461 | total_timesteps 4626.
Path 462 | total_timesteps 4633.
Path 463 | total_timesteps 4644.
Path 464 | total_timesteps 4653.
Path 465 | total_timesteps 4675.
Path 466 | total_timesteps 4685.
Path 467 | total_timesteps 4694.
Path 468 | total_timesteps 4704.
Path 469 | total_timesteps 4716.
Path 470 | total_timesteps 4728.
Path 471 | total_timesteps 4735.
Path 472 | total_timesteps 4744.
Path 473 | total_timesteps 4754.
Path 474 | total_timesteps 4766.
Path 475 | total_timesteps 4776.
Path 476 | total_timesteps 4786.
Path 477 | total_timesteps 4794.
Path 478 | total_timesteps 4803.
Path 479 | total_timesteps 4813.
Path 480 | total_timesteps 4821.
Path 481 | total_timesteps 4833.
Path 482 | total_timesteps 4840.
Path 483 | total_timesteps 4850.
Path 484 | total_timesteps 4861.
Path 485 | total_timesteps 4876.
Path 486 | total_timesteps 4886.
Path 487 | total_timesteps 4894.
Path 488 | total_timesteps 4906.
Path 489 | total_timesteps 4916.
Path 490 | total_timesteps 4926.
Path 491 | total_timesteps 4935.
Path 492 | total_timesteps 4950.
Path 493 | total_timesteps 4959.
Path 494 | total_timesteps 4967.
Path 495 | total_timesteps 4981.
Path 496 | total_timesteps 4989.
Path 497 | total_timesteps 5000.
Path 498 | total_timesteps 5009.
Path 499 | total_timesteps 5024.
Path 500 | total_timesteps 5031.
Path 501 | total_timesteps 5046.
Path 502 | total_timesteps 5058.
Path 503 | total_timesteps 5069.
Path 504 | total_timesteps 5078.
Path 505 | total_timesteps 5086.
Path 506 | total_timesteps 5097.
Path 507 | total_timesteps 5105.
Path 508 | total_timesteps 5114.
Path 509 | total_timesteps 5126.
Path 510 | total_timesteps 5134.
Path 511 | total_timesteps 5144.
Path 512 | total_timesteps 5158.
Path 513 | total_timesteps 5166.
Path 514 | total_timesteps 5173.
Path 515 | total_timesteps 5180.
Path 516 | total_timesteps 5193.
Path 517 | total_timesteps 5204.
Path 518 | total_timesteps 5213.
Path 519 | total_timesteps 5222.
Path 520 | total_timesteps 5232.
Path 521 | total_timesteps 5243.
Path 522 | total_timesteps 5253.
Path 523 | total_timesteps 5263.
Path 524 | total_timesteps 5272.
Path 525 | total_timesteps 5281.
Path 526 | total_timesteps 5293.
Path 527 | total_timesteps 5305.
Path 528 | total_timesteps 5314.
Path 529 | total_timesteps 5325.
Path 530 | total_timesteps 5333.
Path 531 | total_timesteps 5345.
Path 532 | total_timesteps 5357.
Path 533 | total_timesteps 5364.
Path 534 | total_timesteps 5372.
Path 535 | total_timesteps 5381.
Path 536 | total_timesteps 5390.
Path 537 | total_timesteps 5400.
Path 538 | total_timesteps 5410.
Path 539 | total_timesteps 5416.
Path 540 | total_timesteps 5424.
Path 541 | total_timesteps 5434.
Path 542 | total_timesteps 5446.
Path 543 | total_timesteps 5455.
Path 544 | total_timesteps 5465.
Path 545 | total_timesteps 5472.
Path 546 | total_timesteps 5480.
Path 547 | total_timesteps 5488.
Path 548 | total_timesteps 5497.
Path 549 | total_timesteps 5508.
Path 550 | total_timesteps 5515.
Path 551 | total_timesteps 5524.
Path 552 | total_timesteps 5530.
Path 553 | total_timesteps 5538.
Path 554 | total_timesteps 5549.
Path 555 | total_timesteps 5559.
Path 556 | total_timesteps 5572.
Path 557 | total_timesteps 5581.
Path 558 | total_timesteps 5591.
Path 559 | total_timesteps 5600.
Path 560 | total_timesteps 5615.
Path 561 | total_timesteps 5622.
Path 562 | total_timesteps 5633.
Path 563 | total_timesteps 5639.
Path 564 | total_timesteps 5648.
Path 565 | total_timesteps 5657.
Path 566 | total_timesteps 5668.
Path 567 | total_timesteps 5681.
Path 568 | total_timesteps 5692.
Path 569 | total_timesteps 5699.
Path 570 | total_timesteps 5713.
Path 571 | total_timesteps 5720.
Path 572 | total_timesteps 5726.
Path 573 | total_timesteps 5734.
Path 574 | total_timesteps 5744.
Path 575 | total_timesteps 5760.
Path 576 | total_timesteps 5770.
Path 577 | total_timesteps 5780.
Path 578 | total_timesteps 5789.
Path 579 | total_timesteps 5796.
Path 580 | total_timesteps 5809.
Path 581 | total_timesteps 5818.
Path 582 | total_timesteps 5827.
Path 583 | total_timesteps 5835.
Path 584 | total_timesteps 5843.
Path 585 | total_timesteps 5853.
Path 586 | total_timesteps 5863.
Path 587 | total_timesteps 5872.
Path 588 | total_timesteps 5881.
Path 589 | total_timesteps 5888.
Path 590 | total_timesteps 5900.
Path 591 | total_timesteps 5910.
Path 592 | total_timesteps 5918.
Path 593 | total_timesteps 5925.
Path 594 | total_timesteps 5933.
Path 595 | total_timesteps 5946.
Path 596 | total_timesteps 5954.
Path 597 | total_timesteps 5966.
Path 598 | total_timesteps 5974.
Path 599 | total_timesteps 5982.
Path 600 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.71    |
| Iteration     | 2        |
| MaximumReturn | 0.146    |
| MinimumReturn | -19.6    |
| TotalSamples  | 16010    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.2809910476207733
Validation loss = 0.27878329157829285
Validation loss = 0.28671562671661377
Validation loss = 0.29192033410072327
Validation loss = 0.29204273223876953
Validation loss = 0.2911067605018616
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 80.
Path 9 | total_timesteps 90.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 119.
Path 13 | total_timesteps 127.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 146.
Path 16 | total_timesteps 154.
Path 17 | total_timesteps 163.
Path 18 | total_timesteps 171.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 187.
Path 21 | total_timesteps 195.
Path 22 | total_timesteps 208.
Path 23 | total_timesteps 216.
Path 24 | total_timesteps 226.
Path 25 | total_timesteps 238.
Path 26 | total_timesteps 251.
Path 27 | total_timesteps 261.
Path 28 | total_timesteps 277.
Path 29 | total_timesteps 293.
Path 30 | total_timesteps 308.
Path 31 | total_timesteps 318.
Path 32 | total_timesteps 328.
Path 33 | total_timesteps 340.
Path 34 | total_timesteps 347.
Path 35 | total_timesteps 354.
Path 36 | total_timesteps 364.
Path 37 | total_timesteps 372.
Path 38 | total_timesteps 383.
Path 39 | total_timesteps 391.
Path 40 | total_timesteps 404.
Path 41 | total_timesteps 411.
Path 42 | total_timesteps 420.
Path 43 | total_timesteps 428.
Path 44 | total_timesteps 440.
Path 45 | total_timesteps 452.
Path 46 | total_timesteps 462.
Path 47 | total_timesteps 469.
Path 48 | total_timesteps 479.
Path 49 | total_timesteps 487.
Path 50 | total_timesteps 502.
Path 51 | total_timesteps 512.
Path 52 | total_timesteps 521.
Path 53 | total_timesteps 532.
Path 54 | total_timesteps 544.
Path 55 | total_timesteps 551.
Path 56 | total_timesteps 558.
Path 57 | total_timesteps 569.
Path 58 | total_timesteps 585.
Path 59 | total_timesteps 593.
Path 60 | total_timesteps 601.
Path 61 | total_timesteps 611.
Path 62 | total_timesteps 618.
Path 63 | total_timesteps 628.
Path 64 | total_timesteps 637.
Path 65 | total_timesteps 650.
Path 66 | total_timesteps 665.
Path 67 | total_timesteps 672.
Path 68 | total_timesteps 683.
Path 69 | total_timesteps 695.
Path 70 | total_timesteps 707.
Path 71 | total_timesteps 717.
Path 72 | total_timesteps 728.
Path 73 | total_timesteps 742.
Path 74 | total_timesteps 753.
Path 75 | total_timesteps 775.
Path 76 | total_timesteps 783.
Path 77 | total_timesteps 790.
Path 78 | total_timesteps 801.
Path 79 | total_timesteps 813.
Path 80 | total_timesteps 819.
Path 81 | total_timesteps 841.
Path 82 | total_timesteps 848.
Path 83 | total_timesteps 858.
Path 84 | total_timesteps 865.
Path 85 | total_timesteps 873.
Path 86 | total_timesteps 883.
Path 87 | total_timesteps 897.
Path 88 | total_timesteps 906.
Path 89 | total_timesteps 917.
Path 90 | total_timesteps 937.
Path 91 | total_timesteps 946.
Path 92 | total_timesteps 956.
Path 93 | total_timesteps 964.
Path 94 | total_timesteps 974.
Path 95 | total_timesteps 985.
Path 96 | total_timesteps 993.
Path 97 | total_timesteps 1004.
Path 98 | total_timesteps 1013.
Path 99 | total_timesteps 1022.
Path 100 | total_timesteps 1030.
Path 101 | total_timesteps 1039.
Path 102 | total_timesteps 1049.
Path 103 | total_timesteps 1056.
Path 104 | total_timesteps 1065.
Path 105 | total_timesteps 1076.
Path 106 | total_timesteps 1086.
Path 107 | total_timesteps 1095.
Path 108 | total_timesteps 1103.
Path 109 | total_timesteps 1112.
Path 110 | total_timesteps 1121.
Path 111 | total_timesteps 1130.
Path 112 | total_timesteps 1139.
Path 113 | total_timesteps 1149.
Path 114 | total_timesteps 1159.
Path 115 | total_timesteps 1170.
Path 116 | total_timesteps 1178.
Path 117 | total_timesteps 1187.
Path 118 | total_timesteps 1197.
Path 119 | total_timesteps 1203.
Path 120 | total_timesteps 1212.
Path 121 | total_timesteps 1221.
Path 122 | total_timesteps 1228.
Path 123 | total_timesteps 1238.
Path 124 | total_timesteps 1257.
Path 125 | total_timesteps 1266.
Path 126 | total_timesteps 1276.
Path 127 | total_timesteps 1283.
Path 128 | total_timesteps 1290.
Path 129 | total_timesteps 1299.
Path 130 | total_timesteps 1308.
Path 131 | total_timesteps 1317.
Path 132 | total_timesteps 1326.
Path 133 | total_timesteps 1337.
Path 134 | total_timesteps 1346.
Path 135 | total_timesteps 1360.
Path 136 | total_timesteps 1378.
Path 137 | total_timesteps 1386.
Path 138 | total_timesteps 1396.
Path 139 | total_timesteps 1406.
Path 140 | total_timesteps 1416.
Path 141 | total_timesteps 1424.
Path 142 | total_timesteps 1432.
Path 143 | total_timesteps 1441.
Path 144 | total_timesteps 1450.
Path 145 | total_timesteps 1460.
Path 146 | total_timesteps 1467.
Path 147 | total_timesteps 1478.
Path 148 | total_timesteps 1486.
Path 149 | total_timesteps 1495.
Path 150 | total_timesteps 1512.
Path 151 | total_timesteps 1524.
Path 152 | total_timesteps 1532.
Path 153 | total_timesteps 1541.
Path 154 | total_timesteps 1557.
Path 155 | total_timesteps 1571.
Path 156 | total_timesteps 1581.
Path 157 | total_timesteps 1588.
Path 158 | total_timesteps 1597.
Path 159 | total_timesteps 1611.
Path 160 | total_timesteps 1623.
Path 161 | total_timesteps 1631.
Path 162 | total_timesteps 1640.
Path 163 | total_timesteps 1648.
Path 164 | total_timesteps 1655.
Path 165 | total_timesteps 1665.
Path 166 | total_timesteps 1673.
Path 167 | total_timesteps 1684.
Path 168 | total_timesteps 1696.
Path 169 | total_timesteps 1703.
Path 170 | total_timesteps 1717.
Path 171 | total_timesteps 1724.
Path 172 | total_timesteps 1733.
Path 173 | total_timesteps 1748.
Path 174 | total_timesteps 1756.
Path 175 | total_timesteps 1764.
Path 176 | total_timesteps 1773.
Path 177 | total_timesteps 1780.
Path 178 | total_timesteps 1787.
Path 179 | total_timesteps 1797.
Path 180 | total_timesteps 1806.
Path 181 | total_timesteps 1817.
Path 182 | total_timesteps 1826.
Path 183 | total_timesteps 1841.
Path 184 | total_timesteps 1849.
Path 185 | total_timesteps 1857.
Path 186 | total_timesteps 1864.
Path 187 | total_timesteps 1874.
Path 188 | total_timesteps 1890.
Path 189 | total_timesteps 1903.
Path 190 | total_timesteps 1914.
Path 191 | total_timesteps 1924.
Path 192 | total_timesteps 1931.
Path 193 | total_timesteps 1943.
Path 194 | total_timesteps 1951.
Path 195 | total_timesteps 1960.
Path 196 | total_timesteps 1968.
Path 197 | total_timesteps 1978.
Path 198 | total_timesteps 1990.
Path 199 | total_timesteps 1999.
Path 200 | total_timesteps 2007.
Path 201 | total_timesteps 2018.
Path 202 | total_timesteps 2028.
Path 203 | total_timesteps 2036.
Path 204 | total_timesteps 2043.
Path 205 | total_timesteps 2055.
Path 206 | total_timesteps 2066.
Path 207 | total_timesteps 2075.
Path 208 | total_timesteps 2086.
Path 209 | total_timesteps 2095.
Path 210 | total_timesteps 2107.
Path 211 | total_timesteps 2114.
Path 212 | total_timesteps 2123.
Path 213 | total_timesteps 2135.
Path 214 | total_timesteps 2145.
Path 215 | total_timesteps 2153.
Path 216 | total_timesteps 2162.
Path 217 | total_timesteps 2169.
Path 218 | total_timesteps 2181.
Path 219 | total_timesteps 2192.
Path 220 | total_timesteps 2203.
Path 221 | total_timesteps 2213.
Path 222 | total_timesteps 2225.
Path 223 | total_timesteps 2240.
Path 224 | total_timesteps 2247.
Path 225 | total_timesteps 2253.
Path 226 | total_timesteps 2259.
Path 227 | total_timesteps 2267.
Path 228 | total_timesteps 2276.
Path 229 | total_timesteps 2284.
Path 230 | total_timesteps 2301.
Path 231 | total_timesteps 2309.
Path 232 | total_timesteps 2319.
Path 233 | total_timesteps 2332.
Path 234 | total_timesteps 2341.
Path 235 | total_timesteps 2351.
Path 236 | total_timesteps 2358.
Path 237 | total_timesteps 2369.
Path 238 | total_timesteps 2375.
Path 239 | total_timesteps 2385.
Path 240 | total_timesteps 2393.
Path 241 | total_timesteps 2404.
Path 242 | total_timesteps 2414.
Path 243 | total_timesteps 2422.
Path 244 | total_timesteps 2429.
Path 245 | total_timesteps 2439.
Path 246 | total_timesteps 2447.
Path 247 | total_timesteps 2456.
Path 248 | total_timesteps 2465.
Path 249 | total_timesteps 2476.
Path 250 | total_timesteps 2489.
Path 251 | total_timesteps 2499.
Path 252 | total_timesteps 2509.
Path 253 | total_timesteps 2517.
Path 254 | total_timesteps 2525.
Path 255 | total_timesteps 2536.
Path 256 | total_timesteps 2543.
Path 257 | total_timesteps 2551.
Path 258 | total_timesteps 2560.
Path 259 | total_timesteps 2568.
Path 260 | total_timesteps 2578.
Path 261 | total_timesteps 2588.
Path 262 | total_timesteps 2597.
Path 263 | total_timesteps 2610.
Path 264 | total_timesteps 2623.
Path 265 | total_timesteps 2632.
Path 266 | total_timesteps 2643.
Path 267 | total_timesteps 2656.
Path 268 | total_timesteps 2669.
Path 269 | total_timesteps 2682.
Path 270 | total_timesteps 2693.
Path 271 | total_timesteps 2705.
Path 272 | total_timesteps 2717.
Path 273 | total_timesteps 2724.
Path 274 | total_timesteps 2733.
Path 275 | total_timesteps 2742.
Path 276 | total_timesteps 2752.
Path 277 | total_timesteps 2760.
Path 278 | total_timesteps 2769.
Path 279 | total_timesteps 2779.
Path 280 | total_timesteps 2786.
Path 281 | total_timesteps 2795.
Path 282 | total_timesteps 2805.
Path 283 | total_timesteps 2813.
Path 284 | total_timesteps 2822.
Path 285 | total_timesteps 2831.
Path 286 | total_timesteps 2838.
Path 287 | total_timesteps 2852.
Path 288 | total_timesteps 2864.
Path 289 | total_timesteps 2872.
Path 290 | total_timesteps 2883.
Path 291 | total_timesteps 2895.
Path 292 | total_timesteps 2904.
Path 293 | total_timesteps 2913.
Path 294 | total_timesteps 2921.
Path 295 | total_timesteps 2931.
Path 296 | total_timesteps 2941.
Path 297 | total_timesteps 2950.
Path 298 | total_timesteps 2959.
Path 299 | total_timesteps 2967.
Path 300 | total_timesteps 2976.
Path 301 | total_timesteps 2984.
Path 302 | total_timesteps 2994.
Path 303 | total_timesteps 3001.
Path 304 | total_timesteps 3009.
Path 305 | total_timesteps 3022.
Path 306 | total_timesteps 3028.
Path 307 | total_timesteps 3039.
Path 308 | total_timesteps 3048.
Path 309 | total_timesteps 3061.
Path 310 | total_timesteps 3073.
Path 311 | total_timesteps 3086.
Path 312 | total_timesteps 3093.
Path 313 | total_timesteps 3102.
Path 314 | total_timesteps 3110.
Path 315 | total_timesteps 3118.
Path 316 | total_timesteps 3126.
Path 317 | total_timesteps 3137.
Path 318 | total_timesteps 3145.
Path 319 | total_timesteps 3155.
Path 320 | total_timesteps 3168.
Path 321 | total_timesteps 3177.
Path 322 | total_timesteps 3187.
Path 323 | total_timesteps 3195.
Path 324 | total_timesteps 3206.
Path 325 | total_timesteps 3214.
Path 326 | total_timesteps 3224.
Path 327 | total_timesteps 3236.
Path 328 | total_timesteps 3246.
Path 329 | total_timesteps 3258.
Path 330 | total_timesteps 3279.
Path 331 | total_timesteps 3286.
Path 332 | total_timesteps 3293.
Path 333 | total_timesteps 3302.
Path 334 | total_timesteps 3310.
Path 335 | total_timesteps 3317.
Path 336 | total_timesteps 3325.
Path 337 | total_timesteps 3335.
Path 338 | total_timesteps 3343.
Path 339 | total_timesteps 3351.
Path 340 | total_timesteps 3360.
Path 341 | total_timesteps 3369.
Path 342 | total_timesteps 3377.
Path 343 | total_timesteps 3388.
Path 344 | total_timesteps 3399.
Path 345 | total_timesteps 3409.
Path 346 | total_timesteps 3422.
Path 347 | total_timesteps 3432.
Path 348 | total_timesteps 3439.
Path 349 | total_timesteps 3448.
Path 350 | total_timesteps 3457.
Path 351 | total_timesteps 3466.
Path 352 | total_timesteps 3479.
Path 353 | total_timesteps 3487.
Path 354 | total_timesteps 3495.
Path 355 | total_timesteps 3506.
Path 356 | total_timesteps 3514.
Path 357 | total_timesteps 3523.
Path 358 | total_timesteps 3533.
Path 359 | total_timesteps 3540.
Path 360 | total_timesteps 3552.
Path 361 | total_timesteps 3563.
Path 362 | total_timesteps 3577.
Path 363 | total_timesteps 3592.
Path 364 | total_timesteps 3602.
Path 365 | total_timesteps 3612.
Path 366 | total_timesteps 3619.
Path 367 | total_timesteps 3630.
Path 368 | total_timesteps 3640.
Path 369 | total_timesteps 3648.
Path 370 | total_timesteps 3656.
Path 371 | total_timesteps 3665.
Path 372 | total_timesteps 3674.
Path 373 | total_timesteps 3684.
Path 374 | total_timesteps 3693.
Path 375 | total_timesteps 3704.
Path 376 | total_timesteps 3711.
Path 377 | total_timesteps 3722.
Path 378 | total_timesteps 3733.
Path 379 | total_timesteps 3742.
Path 380 | total_timesteps 3749.
Path 381 | total_timesteps 3756.
Path 382 | total_timesteps 3768.
Path 383 | total_timesteps 3776.
Path 384 | total_timesteps 3786.
Path 385 | total_timesteps 3795.
Path 386 | total_timesteps 3805.
Path 387 | total_timesteps 3813.
Path 388 | total_timesteps 3823.
Path 389 | total_timesteps 3837.
Path 390 | total_timesteps 3848.
Path 391 | total_timesteps 3860.
Path 392 | total_timesteps 3867.
Path 393 | total_timesteps 3876.
Path 394 | total_timesteps 3884.
Path 395 | total_timesteps 3895.
Path 396 | total_timesteps 3906.
Path 397 | total_timesteps 3915.
Path 398 | total_timesteps 3930.
Path 399 | total_timesteps 3939.
Path 400 | total_timesteps 3950.
Path 401 | total_timesteps 3956.
Path 402 | total_timesteps 3965.
Path 403 | total_timesteps 3973.
Path 404 | total_timesteps 3985.
Path 405 | total_timesteps 3999.
Path 406 | total_timesteps 4008.
Path 407 | total_timesteps 4016.
Path 408 | total_timesteps 4025.
Path 409 | total_timesteps 4036.
Path 410 | total_timesteps 4045.
Path 411 | total_timesteps 4055.
Path 412 | total_timesteps 4063.
Path 413 | total_timesteps 4074.
Path 414 | total_timesteps 4083.
Path 415 | total_timesteps 4092.
Path 416 | total_timesteps 4101.
Path 417 | total_timesteps 4109.
Path 418 | total_timesteps 4118.
Path 419 | total_timesteps 4127.
Path 420 | total_timesteps 4138.
Path 421 | total_timesteps 4150.
Path 422 | total_timesteps 4156.
Path 423 | total_timesteps 4170.
Path 424 | total_timesteps 4183.
Path 425 | total_timesteps 4191.
Path 426 | total_timesteps 4202.
Path 427 | total_timesteps 4213.
Path 428 | total_timesteps 4221.
Path 429 | total_timesteps 4229.
Path 430 | total_timesteps 4244.
Path 431 | total_timesteps 4255.
Path 432 | total_timesteps 4263.
Path 433 | total_timesteps 4272.
Path 434 | total_timesteps 4281.
Path 435 | total_timesteps 4289.
Path 436 | total_timesteps 4298.
Path 437 | total_timesteps 4307.
Path 438 | total_timesteps 4318.
Path 439 | total_timesteps 4327.
Path 440 | total_timesteps 4336.
Path 441 | total_timesteps 4344.
Path 442 | total_timesteps 4355.
Path 443 | total_timesteps 4364.
Path 444 | total_timesteps 4377.
Path 445 | total_timesteps 4384.
Path 446 | total_timesteps 4394.
Path 447 | total_timesteps 4406.
Path 448 | total_timesteps 4414.
Path 449 | total_timesteps 4424.
Path 450 | total_timesteps 4432.
Path 451 | total_timesteps 4445.
Path 452 | total_timesteps 4452.
Path 453 | total_timesteps 4460.
Path 454 | total_timesteps 4468.
Path 455 | total_timesteps 4476.
Path 456 | total_timesteps 4487.
Path 457 | total_timesteps 4496.
Path 458 | total_timesteps 4504.
Path 459 | total_timesteps 4516.
Path 460 | total_timesteps 4524.
Path 461 | total_timesteps 4533.
Path 462 | total_timesteps 4550.
Path 463 | total_timesteps 4561.
Path 464 | total_timesteps 4579.
Path 465 | total_timesteps 4591.
Path 466 | total_timesteps 4599.
Path 467 | total_timesteps 4608.
Path 468 | total_timesteps 4618.
Path 469 | total_timesteps 4632.
Path 470 | total_timesteps 4644.
Path 471 | total_timesteps 4653.
Path 472 | total_timesteps 4661.
Path 473 | total_timesteps 4669.
Path 474 | total_timesteps 4682.
Path 475 | total_timesteps 4691.
Path 476 | total_timesteps 4700.
Path 477 | total_timesteps 4712.
Path 478 | total_timesteps 4721.
Path 479 | total_timesteps 4737.
Path 480 | total_timesteps 4747.
Path 481 | total_timesteps 4762.
Path 482 | total_timesteps 4772.
Path 483 | total_timesteps 4782.
Path 484 | total_timesteps 4790.
Path 485 | total_timesteps 4800.
Path 486 | total_timesteps 4807.
Path 487 | total_timesteps 4815.
Path 488 | total_timesteps 4826.
Path 489 | total_timesteps 4840.
Path 490 | total_timesteps 4850.
Path 491 | total_timesteps 4861.
Path 492 | total_timesteps 4872.
Path 493 | total_timesteps 4880.
Path 494 | total_timesteps 4887.
Path 495 | total_timesteps 4899.
Path 496 | total_timesteps 4908.
Path 497 | total_timesteps 4917.
Path 498 | total_timesteps 4930.
Path 499 | total_timesteps 4939.
Path 500 | total_timesteps 4948.
Path 501 | total_timesteps 4956.
Path 502 | total_timesteps 4964.
Path 503 | total_timesteps 4973.
Path 504 | total_timesteps 4982.
Path 505 | total_timesteps 4990.
Path 506 | total_timesteps 4999.
Path 507 | total_timesteps 5010.
Path 508 | total_timesteps 5019.
Path 509 | total_timesteps 5027.
Path 510 | total_timesteps 5035.
Path 511 | total_timesteps 5045.
Path 512 | total_timesteps 5052.
Path 513 | total_timesteps 5062.
Path 514 | total_timesteps 5072.
Path 515 | total_timesteps 5081.
Path 516 | total_timesteps 5092.
Path 517 | total_timesteps 5103.
Path 518 | total_timesteps 5115.
Path 519 | total_timesteps 5124.
Path 520 | total_timesteps 5135.
Path 521 | total_timesteps 5147.
Path 522 | total_timesteps 5155.
Path 523 | total_timesteps 5164.
Path 524 | total_timesteps 5172.
Path 525 | total_timesteps 5180.
Path 526 | total_timesteps 5188.
Path 527 | total_timesteps 5199.
Path 528 | total_timesteps 5207.
Path 529 | total_timesteps 5216.
Path 530 | total_timesteps 5225.
Path 531 | total_timesteps 5242.
Path 532 | total_timesteps 5252.
Path 533 | total_timesteps 5269.
Path 534 | total_timesteps 5279.
Path 535 | total_timesteps 5292.
Path 536 | total_timesteps 5299.
Path 537 | total_timesteps 5311.
Path 538 | total_timesteps 5328.
Path 539 | total_timesteps 5336.
Path 540 | total_timesteps 5359.
Path 541 | total_timesteps 5367.
Path 542 | total_timesteps 5379.
Path 543 | total_timesteps 5393.
Path 544 | total_timesteps 5406.
Path 545 | total_timesteps 5419.
Path 546 | total_timesteps 5427.
Path 547 | total_timesteps 5447.
Path 548 | total_timesteps 5456.
Path 549 | total_timesteps 5472.
Path 550 | total_timesteps 5483.
Path 551 | total_timesteps 5495.
Path 552 | total_timesteps 5503.
Path 553 | total_timesteps 5511.
Path 554 | total_timesteps 5519.
Path 555 | total_timesteps 5530.
Path 556 | total_timesteps 5538.
Path 557 | total_timesteps 5547.
Path 558 | total_timesteps 5557.
Path 559 | total_timesteps 5568.
Path 560 | total_timesteps 5579.
Path 561 | total_timesteps 5596.
Path 562 | total_timesteps 5607.
Path 563 | total_timesteps 5622.
Path 564 | total_timesteps 5632.
Path 565 | total_timesteps 5641.
Path 566 | total_timesteps 5653.
Path 567 | total_timesteps 5664.
Path 568 | total_timesteps 5673.
Path 569 | total_timesteps 5684.
Path 570 | total_timesteps 5691.
Path 571 | total_timesteps 5705.
Path 572 | total_timesteps 5713.
Path 573 | total_timesteps 5733.
Path 574 | total_timesteps 5742.
Path 575 | total_timesteps 5752.
Path 576 | total_timesteps 5765.
Path 577 | total_timesteps 5777.
Path 578 | total_timesteps 5790.
Path 579 | total_timesteps 5796.
Path 580 | total_timesteps 5807.
Path 581 | total_timesteps 5815.
Path 582 | total_timesteps 5827.
Path 583 | total_timesteps 5837.
Path 584 | total_timesteps 5846.
Path 585 | total_timesteps 5853.
Path 586 | total_timesteps 5865.
Path 587 | total_timesteps 5873.
Path 588 | total_timesteps 5883.
Path 589 | total_timesteps 5895.
Path 590 | total_timesteps 5901.
Path 591 | total_timesteps 5909.
Path 592 | total_timesteps 5917.
Path 593 | total_timesteps 5928.
Path 594 | total_timesteps 5937.
Path 595 | total_timesteps 5949.
Path 596 | total_timesteps 5957.
Path 597 | total_timesteps 5969.
Path 598 | total_timesteps 5981.
Path 599 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.76    |
| Iteration     | 3        |
| MaximumReturn | 1.77     |
| MinimumReturn | -15.1    |
| TotalSamples  | 20010    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.2865408957004547
Validation loss = 0.28913867473602295
Validation loss = 0.2927190363407135
Validation loss = 0.300601065158844
Validation loss = 0.3010239899158478
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 33.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 54.
Path 7 | total_timesteps 60.
Path 8 | total_timesteps 68.
Path 9 | total_timesteps 78.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 96.
Path 12 | total_timesteps 105.
Path 13 | total_timesteps 123.
Path 14 | total_timesteps 137.
Path 15 | total_timesteps 147.
Path 16 | total_timesteps 155.
Path 17 | total_timesteps 172.
Path 18 | total_timesteps 179.
Path 19 | total_timesteps 188.
Path 20 | total_timesteps 195.
Path 21 | total_timesteps 207.
Path 22 | total_timesteps 217.
Path 23 | total_timesteps 227.
Path 24 | total_timesteps 238.
Path 25 | total_timesteps 251.
Path 26 | total_timesteps 258.
Path 27 | total_timesteps 269.
Path 28 | total_timesteps 278.
Path 29 | total_timesteps 287.
Path 30 | total_timesteps 296.
Path 31 | total_timesteps 308.
Path 32 | total_timesteps 319.
Path 33 | total_timesteps 330.
Path 34 | total_timesteps 342.
Path 35 | total_timesteps 350.
Path 36 | total_timesteps 358.
Path 37 | total_timesteps 367.
Path 38 | total_timesteps 376.
Path 39 | total_timesteps 386.
Path 40 | total_timesteps 395.
Path 41 | total_timesteps 404.
Path 42 | total_timesteps 411.
Path 43 | total_timesteps 420.
Path 44 | total_timesteps 432.
Path 45 | total_timesteps 443.
Path 46 | total_timesteps 453.
Path 47 | total_timesteps 466.
Path 48 | total_timesteps 483.
Path 49 | total_timesteps 493.
Path 50 | total_timesteps 502.
Path 51 | total_timesteps 509.
Path 52 | total_timesteps 525.
Path 53 | total_timesteps 533.
Path 54 | total_timesteps 542.
Path 55 | total_timesteps 550.
Path 56 | total_timesteps 562.
Path 57 | total_timesteps 573.
Path 58 | total_timesteps 584.
Path 59 | total_timesteps 592.
Path 60 | total_timesteps 601.
Path 61 | total_timesteps 618.
Path 62 | total_timesteps 630.
Path 63 | total_timesteps 641.
Path 64 | total_timesteps 652.
Path 65 | total_timesteps 666.
Path 66 | total_timesteps 675.
Path 67 | total_timesteps 684.
Path 68 | total_timesteps 701.
Path 69 | total_timesteps 708.
Path 70 | total_timesteps 715.
Path 71 | total_timesteps 722.
Path 72 | total_timesteps 732.
Path 73 | total_timesteps 741.
Path 74 | total_timesteps 751.
Path 75 | total_timesteps 762.
Path 76 | total_timesteps 771.
Path 77 | total_timesteps 779.
Path 78 | total_timesteps 791.
Path 79 | total_timesteps 800.
Path 80 | total_timesteps 807.
Path 81 | total_timesteps 820.
Path 82 | total_timesteps 830.
Path 83 | total_timesteps 839.
Path 84 | total_timesteps 854.
Path 85 | total_timesteps 866.
Path 86 | total_timesteps 875.
Path 87 | total_timesteps 882.
Path 88 | total_timesteps 889.
Path 89 | total_timesteps 897.
Path 90 | total_timesteps 906.
Path 91 | total_timesteps 916.
Path 92 | total_timesteps 927.
Path 93 | total_timesteps 937.
Path 94 | total_timesteps 945.
Path 95 | total_timesteps 951.
Path 96 | total_timesteps 962.
Path 97 | total_timesteps 972.
Path 98 | total_timesteps 981.
Path 99 | total_timesteps 990.
Path 100 | total_timesteps 1000.
Path 101 | total_timesteps 1011.
Path 102 | total_timesteps 1018.
Path 103 | total_timesteps 1027.
Path 104 | total_timesteps 1039.
Path 105 | total_timesteps 1050.
Path 106 | total_timesteps 1067.
Path 107 | total_timesteps 1078.
Path 108 | total_timesteps 1088.
Path 109 | total_timesteps 1095.
Path 110 | total_timesteps 1104.
Path 111 | total_timesteps 1112.
Path 112 | total_timesteps 1123.
Path 113 | total_timesteps 1132.
Path 114 | total_timesteps 1143.
Path 115 | total_timesteps 1155.
Path 116 | total_timesteps 1171.
Path 117 | total_timesteps 1181.
Path 118 | total_timesteps 1189.
Path 119 | total_timesteps 1200.
Path 120 | total_timesteps 1209.
Path 121 | total_timesteps 1218.
Path 122 | total_timesteps 1227.
Path 123 | total_timesteps 1238.
Path 124 | total_timesteps 1250.
Path 125 | total_timesteps 1258.
Path 126 | total_timesteps 1268.
Path 127 | total_timesteps 1280.
Path 128 | total_timesteps 1288.
Path 129 | total_timesteps 1296.
Path 130 | total_timesteps 1304.
Path 131 | total_timesteps 1317.
Path 132 | total_timesteps 1328.
Path 133 | total_timesteps 1335.
Path 134 | total_timesteps 1342.
Path 135 | total_timesteps 1353.
Path 136 | total_timesteps 1365.
Path 137 | total_timesteps 1374.
Path 138 | total_timesteps 1387.
Path 139 | total_timesteps 1399.
Path 140 | total_timesteps 1410.
Path 141 | total_timesteps 1419.
Path 142 | total_timesteps 1431.
Path 143 | total_timesteps 1443.
Path 144 | total_timesteps 1450.
Path 145 | total_timesteps 1457.
Path 146 | total_timesteps 1463.
Path 147 | total_timesteps 1472.
Path 148 | total_timesteps 1485.
Path 149 | total_timesteps 1494.
Path 150 | total_timesteps 1505.
Path 151 | total_timesteps 1513.
Path 152 | total_timesteps 1523.
Path 153 | total_timesteps 1540.
Path 154 | total_timesteps 1551.
Path 155 | total_timesteps 1565.
Path 156 | total_timesteps 1574.
Path 157 | total_timesteps 1586.
Path 158 | total_timesteps 1594.
Path 159 | total_timesteps 1605.
Path 160 | total_timesteps 1614.
Path 161 | total_timesteps 1623.
Path 162 | total_timesteps 1632.
Path 163 | total_timesteps 1644.
Path 164 | total_timesteps 1657.
Path 165 | total_timesteps 1665.
Path 166 | total_timesteps 1674.
Path 167 | total_timesteps 1682.
Path 168 | total_timesteps 1693.
Path 169 | total_timesteps 1707.
Path 170 | total_timesteps 1719.
Path 171 | total_timesteps 1729.
Path 172 | total_timesteps 1744.
Path 173 | total_timesteps 1751.
Path 174 | total_timesteps 1758.
Path 175 | total_timesteps 1771.
Path 176 | total_timesteps 1780.
Path 177 | total_timesteps 1808.
Path 178 | total_timesteps 1817.
Path 179 | total_timesteps 1826.
Path 180 | total_timesteps 1834.
Path 181 | total_timesteps 1841.
Path 182 | total_timesteps 1850.
Path 183 | total_timesteps 1861.
Path 184 | total_timesteps 1869.
Path 185 | total_timesteps 1879.
Path 186 | total_timesteps 1889.
Path 187 | total_timesteps 1898.
Path 188 | total_timesteps 1912.
Path 189 | total_timesteps 1928.
Path 190 | total_timesteps 1937.
Path 191 | total_timesteps 1951.
Path 192 | total_timesteps 1962.
Path 193 | total_timesteps 1971.
Path 194 | total_timesteps 1979.
Path 195 | total_timesteps 1987.
Path 196 | total_timesteps 2001.
Path 197 | total_timesteps 2010.
Path 198 | total_timesteps 2019.
Path 199 | total_timesteps 2029.
Path 200 | total_timesteps 2037.
Path 201 | total_timesteps 2045.
Path 202 | total_timesteps 2054.
Path 203 | total_timesteps 2061.
Path 204 | total_timesteps 2069.
Path 205 | total_timesteps 2085.
Path 206 | total_timesteps 2096.
Path 207 | total_timesteps 2105.
Path 208 | total_timesteps 2112.
Path 209 | total_timesteps 2123.
Path 210 | total_timesteps 2130.
Path 211 | total_timesteps 2141.
Path 212 | total_timesteps 2153.
Path 213 | total_timesteps 2162.
Path 214 | total_timesteps 2171.
Path 215 | total_timesteps 2184.
Path 216 | total_timesteps 2191.
Path 217 | total_timesteps 2198.
Path 218 | total_timesteps 2211.
Path 219 | total_timesteps 2220.
Path 220 | total_timesteps 2227.
Path 221 | total_timesteps 2244.
Path 222 | total_timesteps 2263.
Path 223 | total_timesteps 2273.
Path 224 | total_timesteps 2283.
Path 225 | total_timesteps 2293.
Path 226 | total_timesteps 2304.
Path 227 | total_timesteps 2311.
Path 228 | total_timesteps 2322.
Path 229 | total_timesteps 2329.
Path 230 | total_timesteps 2339.
Path 231 | total_timesteps 2347.
Path 232 | total_timesteps 2355.
Path 233 | total_timesteps 2367.
Path 234 | total_timesteps 2375.
Path 235 | total_timesteps 2385.
Path 236 | total_timesteps 2393.
Path 237 | total_timesteps 2402.
Path 238 | total_timesteps 2418.
Path 239 | total_timesteps 2426.
Path 240 | total_timesteps 2432.
Path 241 | total_timesteps 2441.
Path 242 | total_timesteps 2449.
Path 243 | total_timesteps 2461.
Path 244 | total_timesteps 2468.
Path 245 | total_timesteps 2476.
Path 246 | total_timesteps 2485.
Path 247 | total_timesteps 2498.
Path 248 | total_timesteps 2507.
Path 249 | total_timesteps 2513.
Path 250 | total_timesteps 2521.
Path 251 | total_timesteps 2534.
Path 252 | total_timesteps 2545.
Path 253 | total_timesteps 2555.
Path 254 | total_timesteps 2566.
Path 255 | total_timesteps 2576.
Path 256 | total_timesteps 2588.
Path 257 | total_timesteps 2602.
Path 258 | total_timesteps 2614.
Path 259 | total_timesteps 2623.
Path 260 | total_timesteps 2631.
Path 261 | total_timesteps 2637.
Path 262 | total_timesteps 2645.
Path 263 | total_timesteps 2656.
Path 264 | total_timesteps 2665.
Path 265 | total_timesteps 2672.
Path 266 | total_timesteps 2681.
Path 267 | total_timesteps 2689.
Path 268 | total_timesteps 2699.
Path 269 | total_timesteps 2709.
Path 270 | total_timesteps 2718.
Path 271 | total_timesteps 2734.
Path 272 | total_timesteps 2742.
Path 273 | total_timesteps 2750.
Path 274 | total_timesteps 2759.
Path 275 | total_timesteps 2772.
Path 276 | total_timesteps 2780.
Path 277 | total_timesteps 2788.
Path 278 | total_timesteps 2799.
Path 279 | total_timesteps 2808.
Path 280 | total_timesteps 2819.
Path 281 | total_timesteps 2829.
Path 282 | total_timesteps 2849.
Path 283 | total_timesteps 2861.
Path 284 | total_timesteps 2878.
Path 285 | total_timesteps 2888.
Path 286 | total_timesteps 2897.
Path 287 | total_timesteps 2908.
Path 288 | total_timesteps 2915.
Path 289 | total_timesteps 2924.
Path 290 | total_timesteps 2934.
Path 291 | total_timesteps 2942.
Path 292 | total_timesteps 2950.
Path 293 | total_timesteps 2960.
Path 294 | total_timesteps 2968.
Path 295 | total_timesteps 2977.
Path 296 | total_timesteps 2990.
Path 297 | total_timesteps 3001.
Path 298 | total_timesteps 3010.
Path 299 | total_timesteps 3019.
Path 300 | total_timesteps 3031.
Path 301 | total_timesteps 3040.
Path 302 | total_timesteps 3051.
Path 303 | total_timesteps 3059.
Path 304 | total_timesteps 3069.
Path 305 | total_timesteps 3079.
Path 306 | total_timesteps 3089.
Path 307 | total_timesteps 3102.
Path 308 | total_timesteps 3110.
Path 309 | total_timesteps 3118.
Path 310 | total_timesteps 3127.
Path 311 | total_timesteps 3137.
Path 312 | total_timesteps 3145.
Path 313 | total_timesteps 3156.
Path 314 | total_timesteps 3166.
Path 315 | total_timesteps 3176.
Path 316 | total_timesteps 3184.
Path 317 | total_timesteps 3193.
Path 318 | total_timesteps 3201.
Path 319 | total_timesteps 3211.
Path 320 | total_timesteps 3221.
Path 321 | total_timesteps 3228.
Path 322 | total_timesteps 3238.
Path 323 | total_timesteps 3247.
Path 324 | total_timesteps 3255.
Path 325 | total_timesteps 3264.
Path 326 | total_timesteps 3274.
Path 327 | total_timesteps 3288.
Path 328 | total_timesteps 3295.
Path 329 | total_timesteps 3306.
Path 330 | total_timesteps 3319.
Path 331 | total_timesteps 3326.
Path 332 | total_timesteps 3336.
Path 333 | total_timesteps 3342.
Path 334 | total_timesteps 3351.
Path 335 | total_timesteps 3360.
Path 336 | total_timesteps 3369.
Path 337 | total_timesteps 3377.
Path 338 | total_timesteps 3387.
Path 339 | total_timesteps 3398.
Path 340 | total_timesteps 3405.
Path 341 | total_timesteps 3416.
Path 342 | total_timesteps 3427.
Path 343 | total_timesteps 3436.
Path 344 | total_timesteps 3453.
Path 345 | total_timesteps 3461.
Path 346 | total_timesteps 3474.
Path 347 | total_timesteps 3491.
Path 348 | total_timesteps 3497.
Path 349 | total_timesteps 3505.
Path 350 | total_timesteps 3511.
Path 351 | total_timesteps 3519.
Path 352 | total_timesteps 3527.
Path 353 | total_timesteps 3539.
Path 354 | total_timesteps 3552.
Path 355 | total_timesteps 3562.
Path 356 | total_timesteps 3569.
Path 357 | total_timesteps 3583.
Path 358 | total_timesteps 3592.
Path 359 | total_timesteps 3600.
Path 360 | total_timesteps 3612.
Path 361 | total_timesteps 3623.
Path 362 | total_timesteps 3637.
Path 363 | total_timesteps 3649.
Path 364 | total_timesteps 3660.
Path 365 | total_timesteps 3668.
Path 366 | total_timesteps 3677.
Path 367 | total_timesteps 3686.
Path 368 | total_timesteps 3698.
Path 369 | total_timesteps 3706.
Path 370 | total_timesteps 3717.
Path 371 | total_timesteps 3727.
Path 372 | total_timesteps 3734.
Path 373 | total_timesteps 3745.
Path 374 | total_timesteps 3759.
Path 375 | total_timesteps 3773.
Path 376 | total_timesteps 3786.
Path 377 | total_timesteps 3794.
Path 378 | total_timesteps 3802.
Path 379 | total_timesteps 3811.
Path 380 | total_timesteps 3821.
Path 381 | total_timesteps 3833.
Path 382 | total_timesteps 3847.
Path 383 | total_timesteps 3859.
Path 384 | total_timesteps 3869.
Path 385 | total_timesteps 3878.
Path 386 | total_timesteps 3887.
Path 387 | total_timesteps 3898.
Path 388 | total_timesteps 3912.
Path 389 | total_timesteps 3918.
Path 390 | total_timesteps 3926.
Path 391 | total_timesteps 3934.
Path 392 | total_timesteps 3945.
Path 393 | total_timesteps 3952.
Path 394 | total_timesteps 3963.
Path 395 | total_timesteps 3971.
Path 396 | total_timesteps 3984.
Path 397 | total_timesteps 3991.
Path 398 | total_timesteps 4001.
Path 399 | total_timesteps 4010.
Path 400 | total_timesteps 4019.
Path 401 | total_timesteps 4028.
Path 402 | total_timesteps 4040.
Path 403 | total_timesteps 4050.
Path 404 | total_timesteps 4061.
Path 405 | total_timesteps 4071.
Path 406 | total_timesteps 4082.
Path 407 | total_timesteps 4092.
Path 408 | total_timesteps 4100.
Path 409 | total_timesteps 4108.
Path 410 | total_timesteps 4119.
Path 411 | total_timesteps 4126.
Path 412 | total_timesteps 4135.
Path 413 | total_timesteps 4144.
Path 414 | total_timesteps 4153.
Path 415 | total_timesteps 4164.
Path 416 | total_timesteps 4172.
Path 417 | total_timesteps 4183.
Path 418 | total_timesteps 4194.
Path 419 | total_timesteps 4203.
Path 420 | total_timesteps 4211.
Path 421 | total_timesteps 4223.
Path 422 | total_timesteps 4231.
Path 423 | total_timesteps 4246.
Path 424 | total_timesteps 4259.
Path 425 | total_timesteps 4271.
Path 426 | total_timesteps 4280.
Path 427 | total_timesteps 4290.
Path 428 | total_timesteps 4297.
Path 429 | total_timesteps 4305.
Path 430 | total_timesteps 4320.
Path 431 | total_timesteps 4328.
Path 432 | total_timesteps 4336.
Path 433 | total_timesteps 4346.
Path 434 | total_timesteps 4356.
Path 435 | total_timesteps 4365.
Path 436 | total_timesteps 4375.
Path 437 | total_timesteps 4382.
Path 438 | total_timesteps 4392.
Path 439 | total_timesteps 4399.
Path 440 | total_timesteps 4413.
Path 441 | total_timesteps 4422.
Path 442 | total_timesteps 4432.
Path 443 | total_timesteps 4440.
Path 444 | total_timesteps 4448.
Path 445 | total_timesteps 4461.
Path 446 | total_timesteps 4472.
Path 447 | total_timesteps 4482.
Path 448 | total_timesteps 4491.
Path 449 | total_timesteps 4508.
Path 450 | total_timesteps 4520.
Path 451 | total_timesteps 4528.
Path 452 | total_timesteps 4537.
Path 453 | total_timesteps 4545.
Path 454 | total_timesteps 4555.
Path 455 | total_timesteps 4562.
Path 456 | total_timesteps 4571.
Path 457 | total_timesteps 4581.
Path 458 | total_timesteps 4591.
Path 459 | total_timesteps 4598.
Path 460 | total_timesteps 4607.
Path 461 | total_timesteps 4617.
Path 462 | total_timesteps 4627.
Path 463 | total_timesteps 4637.
Path 464 | total_timesteps 4646.
Path 465 | total_timesteps 4658.
Path 466 | total_timesteps 4667.
Path 467 | total_timesteps 4681.
Path 468 | total_timesteps 4689.
Path 469 | total_timesteps 4708.
Path 470 | total_timesteps 4722.
Path 471 | total_timesteps 4735.
Path 472 | total_timesteps 4745.
Path 473 | total_timesteps 4753.
Path 474 | total_timesteps 4769.
Path 475 | total_timesteps 4776.
Path 476 | total_timesteps 4787.
Path 477 | total_timesteps 4797.
Path 478 | total_timesteps 4807.
Path 479 | total_timesteps 4817.
Path 480 | total_timesteps 4825.
Path 481 | total_timesteps 4837.
Path 482 | total_timesteps 4846.
Path 483 | total_timesteps 4856.
Path 484 | total_timesteps 4862.
Path 485 | total_timesteps 4871.
Path 486 | total_timesteps 4884.
Path 487 | total_timesteps 4896.
Path 488 | total_timesteps 4903.
Path 489 | total_timesteps 4916.
Path 490 | total_timesteps 4926.
Path 491 | total_timesteps 4940.
Path 492 | total_timesteps 4953.
Path 493 | total_timesteps 4962.
Path 494 | total_timesteps 4971.
Path 495 | total_timesteps 4979.
Path 496 | total_timesteps 4993.
Path 497 | total_timesteps 5001.
Path 498 | total_timesteps 5015.
Path 499 | total_timesteps 5022.
Path 500 | total_timesteps 5035.
Path 501 | total_timesteps 5045.
Path 502 | total_timesteps 5053.
Path 503 | total_timesteps 5069.
Path 504 | total_timesteps 5078.
Path 505 | total_timesteps 5087.
Path 506 | total_timesteps 5095.
Path 507 | total_timesteps 5107.
Path 508 | total_timesteps 5116.
Path 509 | total_timesteps 5126.
Path 510 | total_timesteps 5138.
Path 511 | total_timesteps 5149.
Path 512 | total_timesteps 5157.
Path 513 | total_timesteps 5173.
Path 514 | total_timesteps 5185.
Path 515 | total_timesteps 5194.
Path 516 | total_timesteps 5207.
Path 517 | total_timesteps 5217.
Path 518 | total_timesteps 5226.
Path 519 | total_timesteps 5236.
Path 520 | total_timesteps 5244.
Path 521 | total_timesteps 5252.
Path 522 | total_timesteps 5261.
Path 523 | total_timesteps 5268.
Path 524 | total_timesteps 5281.
Path 525 | total_timesteps 5290.
Path 526 | total_timesteps 5298.
Path 527 | total_timesteps 5316.
Path 528 | total_timesteps 5325.
Path 529 | total_timesteps 5341.
Path 530 | total_timesteps 5348.
Path 531 | total_timesteps 5359.
Path 532 | total_timesteps 5370.
Path 533 | total_timesteps 5377.
Path 534 | total_timesteps 5387.
Path 535 | total_timesteps 5397.
Path 536 | total_timesteps 5405.
Path 537 | total_timesteps 5415.
Path 538 | total_timesteps 5427.
Path 539 | total_timesteps 5439.
Path 540 | total_timesteps 5448.
Path 541 | total_timesteps 5458.
Path 542 | total_timesteps 5467.
Path 543 | total_timesteps 5481.
Path 544 | total_timesteps 5493.
Path 545 | total_timesteps 5504.
Path 546 | total_timesteps 5511.
Path 547 | total_timesteps 5519.
Path 548 | total_timesteps 5533.
Path 549 | total_timesteps 5548.
Path 550 | total_timesteps 5559.
Path 551 | total_timesteps 5573.
Path 552 | total_timesteps 5586.
Path 553 | total_timesteps 5599.
Path 554 | total_timesteps 5608.
Path 555 | total_timesteps 5619.
Path 556 | total_timesteps 5628.
Path 557 | total_timesteps 5642.
Path 558 | total_timesteps 5652.
Path 559 | total_timesteps 5663.
Path 560 | total_timesteps 5676.
Path 561 | total_timesteps 5684.
Path 562 | total_timesteps 5691.
Path 563 | total_timesteps 5705.
Path 564 | total_timesteps 5719.
Path 565 | total_timesteps 5730.
Path 566 | total_timesteps 5740.
Path 567 | total_timesteps 5750.
Path 568 | total_timesteps 5758.
Path 569 | total_timesteps 5775.
Path 570 | total_timesteps 5785.
Path 571 | total_timesteps 5795.
Path 572 | total_timesteps 5804.
Path 573 | total_timesteps 5813.
Path 574 | total_timesteps 5827.
Path 575 | total_timesteps 5839.
Path 576 | total_timesteps 5850.
Path 577 | total_timesteps 5859.
Path 578 | total_timesteps 5874.
Path 579 | total_timesteps 5882.
Path 580 | total_timesteps 5892.
Path 581 | total_timesteps 5907.
Path 582 | total_timesteps 5919.
Path 583 | total_timesteps 5928.
Path 584 | total_timesteps 5936.
Path 585 | total_timesteps 5948.
Path 586 | total_timesteps 5956.
Path 587 | total_timesteps 5963.
Path 588 | total_timesteps 5971.
Path 589 | total_timesteps 5981.
Path 590 | total_timesteps 5989.
Path 591 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.66    |
| Iteration     | 4        |
| MaximumReturn | 1.26     |
| MinimumReturn | -21.3    |
| TotalSamples  | 24018    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.2846277356147766
Validation loss = 0.2886676490306854
Validation loss = 0.2903335988521576
Validation loss = 0.2954794466495514
Validation loss = 0.29834845662117004
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 47.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 77.
Path 9 | total_timesteps 90.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 121.
Path 13 | total_timesteps 129.
Path 14 | total_timesteps 139.
Path 15 | total_timesteps 155.
Path 16 | total_timesteps 168.
Path 17 | total_timesteps 176.
Path 18 | total_timesteps 187.
Path 19 | total_timesteps 196.
Path 20 | total_timesteps 208.
Path 21 | total_timesteps 219.
Path 22 | total_timesteps 235.
Path 23 | total_timesteps 247.
Path 24 | total_timesteps 258.
Path 25 | total_timesteps 267.
Path 26 | total_timesteps 282.
Path 27 | total_timesteps 295.
Path 28 | total_timesteps 306.
Path 29 | total_timesteps 313.
Path 30 | total_timesteps 321.
Path 31 | total_timesteps 333.
Path 32 | total_timesteps 339.
Path 33 | total_timesteps 346.
Path 34 | total_timesteps 356.
Path 35 | total_timesteps 364.
Path 36 | total_timesteps 372.
Path 37 | total_timesteps 380.
Path 38 | total_timesteps 392.
Path 39 | total_timesteps 403.
Path 40 | total_timesteps 412.
Path 41 | total_timesteps 421.
Path 42 | total_timesteps 429.
Path 43 | total_timesteps 438.
Path 44 | total_timesteps 445.
Path 45 | total_timesteps 455.
Path 46 | total_timesteps 465.
Path 47 | total_timesteps 476.
Path 48 | total_timesteps 487.
Path 49 | total_timesteps 502.
Path 50 | total_timesteps 510.
Path 51 | total_timesteps 517.
Path 52 | total_timesteps 526.
Path 53 | total_timesteps 538.
Path 54 | total_timesteps 549.
Path 55 | total_timesteps 558.
Path 56 | total_timesteps 566.
Path 57 | total_timesteps 578.
Path 58 | total_timesteps 590.
Path 59 | total_timesteps 604.
Path 60 | total_timesteps 613.
Path 61 | total_timesteps 626.
Path 62 | total_timesteps 634.
Path 63 | total_timesteps 648.
Path 64 | total_timesteps 655.
Path 65 | total_timesteps 664.
Path 66 | total_timesteps 672.
Path 67 | total_timesteps 682.
Path 68 | total_timesteps 692.
Path 69 | total_timesteps 700.
Path 70 | total_timesteps 716.
Path 71 | total_timesteps 730.
Path 72 | total_timesteps 740.
Path 73 | total_timesteps 751.
Path 74 | total_timesteps 760.
Path 75 | total_timesteps 772.
Path 76 | total_timesteps 779.
Path 77 | total_timesteps 790.
Path 78 | total_timesteps 801.
Path 79 | total_timesteps 810.
Path 80 | total_timesteps 821.
Path 81 | total_timesteps 834.
Path 82 | total_timesteps 842.
Path 83 | total_timesteps 855.
Path 84 | total_timesteps 865.
Path 85 | total_timesteps 875.
Path 86 | total_timesteps 885.
Path 87 | total_timesteps 894.
Path 88 | total_timesteps 905.
Path 89 | total_timesteps 913.
Path 90 | total_timesteps 920.
Path 91 | total_timesteps 928.
Path 92 | total_timesteps 937.
Path 93 | total_timesteps 944.
Path 94 | total_timesteps 957.
Path 95 | total_timesteps 965.
Path 96 | total_timesteps 977.
Path 97 | total_timesteps 988.
Path 98 | total_timesteps 997.
Path 99 | total_timesteps 1008.
Path 100 | total_timesteps 1015.
Path 101 | total_timesteps 1027.
Path 102 | total_timesteps 1037.
Path 103 | total_timesteps 1047.
Path 104 | total_timesteps 1055.
Path 105 | total_timesteps 1065.
Path 106 | total_timesteps 1075.
Path 107 | total_timesteps 1085.
Path 108 | total_timesteps 1096.
Path 109 | total_timesteps 1103.
Path 110 | total_timesteps 1115.
Path 111 | total_timesteps 1127.
Path 112 | total_timesteps 1135.
Path 113 | total_timesteps 1146.
Path 114 | total_timesteps 1161.
Path 115 | total_timesteps 1168.
Path 116 | total_timesteps 1175.
Path 117 | total_timesteps 1187.
Path 118 | total_timesteps 1198.
Path 119 | total_timesteps 1208.
Path 120 | total_timesteps 1221.
Path 121 | total_timesteps 1229.
Path 122 | total_timesteps 1239.
Path 123 | total_timesteps 1249.
Path 124 | total_timesteps 1255.
Path 125 | total_timesteps 1263.
Path 126 | total_timesteps 1272.
Path 127 | total_timesteps 1281.
Path 128 | total_timesteps 1295.
Path 129 | total_timesteps 1309.
Path 130 | total_timesteps 1320.
Path 131 | total_timesteps 1328.
Path 132 | total_timesteps 1336.
Path 133 | total_timesteps 1344.
Path 134 | total_timesteps 1355.
Path 135 | total_timesteps 1374.
Path 136 | total_timesteps 1383.
Path 137 | total_timesteps 1390.
Path 138 | total_timesteps 1404.
Path 139 | total_timesteps 1412.
Path 140 | total_timesteps 1423.
Path 141 | total_timesteps 1431.
Path 142 | total_timesteps 1441.
Path 143 | total_timesteps 1457.
Path 144 | total_timesteps 1469.
Path 145 | total_timesteps 1481.
Path 146 | total_timesteps 1488.
Path 147 | total_timesteps 1496.
Path 148 | total_timesteps 1511.
Path 149 | total_timesteps 1519.
Path 150 | total_timesteps 1528.
Path 151 | total_timesteps 1536.
Path 152 | total_timesteps 1543.
Path 153 | total_timesteps 1552.
Path 154 | total_timesteps 1560.
Path 155 | total_timesteps 1575.
Path 156 | total_timesteps 1583.
Path 157 | total_timesteps 1591.
Path 158 | total_timesteps 1603.
Path 159 | total_timesteps 1613.
Path 160 | total_timesteps 1621.
Path 161 | total_timesteps 1630.
Path 162 | total_timesteps 1646.
Path 163 | total_timesteps 1659.
Path 164 | total_timesteps 1670.
Path 165 | total_timesteps 1676.
Path 166 | total_timesteps 1691.
Path 167 | total_timesteps 1699.
Path 168 | total_timesteps 1710.
Path 169 | total_timesteps 1721.
Path 170 | total_timesteps 1731.
Path 171 | total_timesteps 1744.
Path 172 | total_timesteps 1753.
Path 173 | total_timesteps 1760.
Path 174 | total_timesteps 1769.
Path 175 | total_timesteps 1780.
Path 176 | total_timesteps 1789.
Path 177 | total_timesteps 1799.
Path 178 | total_timesteps 1807.
Path 179 | total_timesteps 1817.
Path 180 | total_timesteps 1830.
Path 181 | total_timesteps 1842.
Path 182 | total_timesteps 1850.
Path 183 | total_timesteps 1864.
Path 184 | total_timesteps 1875.
Path 185 | total_timesteps 1886.
Path 186 | total_timesteps 1894.
Path 187 | total_timesteps 1905.
Path 188 | total_timesteps 1912.
Path 189 | total_timesteps 1920.
Path 190 | total_timesteps 1928.
Path 191 | total_timesteps 1941.
Path 192 | total_timesteps 1949.
Path 193 | total_timesteps 1963.
Path 194 | total_timesteps 1973.
Path 195 | total_timesteps 1982.
Path 196 | total_timesteps 1993.
Path 197 | total_timesteps 2001.
Path 198 | total_timesteps 2009.
Path 199 | total_timesteps 2021.
Path 200 | total_timesteps 2032.
Path 201 | total_timesteps 2046.
Path 202 | total_timesteps 2058.
Path 203 | total_timesteps 2067.
Path 204 | total_timesteps 2081.
Path 205 | total_timesteps 2094.
Path 206 | total_timesteps 2111.
Path 207 | total_timesteps 2123.
Path 208 | total_timesteps 2131.
Path 209 | total_timesteps 2139.
Path 210 | total_timesteps 2146.
Path 211 | total_timesteps 2154.
Path 212 | total_timesteps 2163.
Path 213 | total_timesteps 2175.
Path 214 | total_timesteps 2183.
Path 215 | total_timesteps 2193.
Path 216 | total_timesteps 2201.
Path 217 | total_timesteps 2209.
Path 218 | total_timesteps 2216.
Path 219 | total_timesteps 2226.
Path 220 | total_timesteps 2237.
Path 221 | total_timesteps 2247.
Path 222 | total_timesteps 2257.
Path 223 | total_timesteps 2268.
Path 224 | total_timesteps 2276.
Path 225 | total_timesteps 2283.
Path 226 | total_timesteps 2293.
Path 227 | total_timesteps 2304.
Path 228 | total_timesteps 2313.
Path 229 | total_timesteps 2323.
Path 230 | total_timesteps 2333.
Path 231 | total_timesteps 2344.
Path 232 | total_timesteps 2352.
Path 233 | total_timesteps 2363.
Path 234 | total_timesteps 2375.
Path 235 | total_timesteps 2383.
Path 236 | total_timesteps 2393.
Path 237 | total_timesteps 2401.
Path 238 | total_timesteps 2411.
Path 239 | total_timesteps 2419.
Path 240 | total_timesteps 2431.
Path 241 | total_timesteps 2441.
Path 242 | total_timesteps 2448.
Path 243 | total_timesteps 2457.
Path 244 | total_timesteps 2465.
Path 245 | total_timesteps 2484.
Path 246 | total_timesteps 2495.
Path 247 | total_timesteps 2506.
Path 248 | total_timesteps 2513.
Path 249 | total_timesteps 2525.
Path 250 | total_timesteps 2533.
Path 251 | total_timesteps 2546.
Path 252 | total_timesteps 2557.
Path 253 | total_timesteps 2564.
Path 254 | total_timesteps 2571.
Path 255 | total_timesteps 2583.
Path 256 | total_timesteps 2593.
Path 257 | total_timesteps 2604.
Path 258 | total_timesteps 2612.
Path 259 | total_timesteps 2622.
Path 260 | total_timesteps 2630.
Path 261 | total_timesteps 2636.
Path 262 | total_timesteps 2646.
Path 263 | total_timesteps 2654.
Path 264 | total_timesteps 2662.
Path 265 | total_timesteps 2671.
Path 266 | total_timesteps 2681.
Path 267 | total_timesteps 2690.
Path 268 | total_timesteps 2700.
Path 269 | total_timesteps 2708.
Path 270 | total_timesteps 2715.
Path 271 | total_timesteps 2725.
Path 272 | total_timesteps 2738.
Path 273 | total_timesteps 2748.
Path 274 | total_timesteps 2766.
Path 275 | total_timesteps 2776.
Path 276 | total_timesteps 2785.
Path 277 | total_timesteps 2805.
Path 278 | total_timesteps 2814.
Path 279 | total_timesteps 2823.
Path 280 | total_timesteps 2850.
Path 281 | total_timesteps 2858.
Path 282 | total_timesteps 2871.
Path 283 | total_timesteps 2881.
Path 284 | total_timesteps 2891.
Path 285 | total_timesteps 2901.
Path 286 | total_timesteps 2912.
Path 287 | total_timesteps 2922.
Path 288 | total_timesteps 2932.
Path 289 | total_timesteps 2941.
Path 290 | total_timesteps 2950.
Path 291 | total_timesteps 2957.
Path 292 | total_timesteps 2974.
Path 293 | total_timesteps 2980.
Path 294 | total_timesteps 2987.
Path 295 | total_timesteps 2999.
Path 296 | total_timesteps 3008.
Path 297 | total_timesteps 3020.
Path 298 | total_timesteps 3030.
Path 299 | total_timesteps 3040.
Path 300 | total_timesteps 3051.
Path 301 | total_timesteps 3058.
Path 302 | total_timesteps 3065.
Path 303 | total_timesteps 3073.
Path 304 | total_timesteps 3083.
Path 305 | total_timesteps 3090.
Path 306 | total_timesteps 3100.
Path 307 | total_timesteps 3109.
Path 308 | total_timesteps 3119.
Path 309 | total_timesteps 3127.
Path 310 | total_timesteps 3137.
Path 311 | total_timesteps 3147.
Path 312 | total_timesteps 3154.
Path 313 | total_timesteps 3165.
Path 314 | total_timesteps 3172.
Path 315 | total_timesteps 3179.
Path 316 | total_timesteps 3190.
Path 317 | total_timesteps 3202.
Path 318 | total_timesteps 3211.
Path 319 | total_timesteps 3220.
Path 320 | total_timesteps 3231.
Path 321 | total_timesteps 3243.
Path 322 | total_timesteps 3253.
Path 323 | total_timesteps 3263.
Path 324 | total_timesteps 3279.
Path 325 | total_timesteps 3289.
Path 326 | total_timesteps 3298.
Path 327 | total_timesteps 3307.
Path 328 | total_timesteps 3316.
Path 329 | total_timesteps 3327.
Path 330 | total_timesteps 3335.
Path 331 | total_timesteps 3344.
Path 332 | total_timesteps 3358.
Path 333 | total_timesteps 3368.
Path 334 | total_timesteps 3374.
Path 335 | total_timesteps 3384.
Path 336 | total_timesteps 3392.
Path 337 | total_timesteps 3401.
Path 338 | total_timesteps 3408.
Path 339 | total_timesteps 3416.
Path 340 | total_timesteps 3427.
Path 341 | total_timesteps 3434.
Path 342 | total_timesteps 3442.
Path 343 | total_timesteps 3455.
Path 344 | total_timesteps 3464.
Path 345 | total_timesteps 3474.
Path 346 | total_timesteps 3486.
Path 347 | total_timesteps 3496.
Path 348 | total_timesteps 3507.
Path 349 | total_timesteps 3515.
Path 350 | total_timesteps 3524.
Path 351 | total_timesteps 3531.
Path 352 | total_timesteps 3547.
Path 353 | total_timesteps 3559.
Path 354 | total_timesteps 3568.
Path 355 | total_timesteps 3579.
Path 356 | total_timesteps 3587.
Path 357 | total_timesteps 3599.
Path 358 | total_timesteps 3616.
Path 359 | total_timesteps 3625.
Path 360 | total_timesteps 3633.
Path 361 | total_timesteps 3645.
Path 362 | total_timesteps 3661.
Path 363 | total_timesteps 3672.
Path 364 | total_timesteps 3680.
Path 365 | total_timesteps 3688.
Path 366 | total_timesteps 3695.
Path 367 | total_timesteps 3707.
Path 368 | total_timesteps 3715.
Path 369 | total_timesteps 3740.
Path 370 | total_timesteps 3758.
Path 371 | total_timesteps 3768.
Path 372 | total_timesteps 3777.
Path 373 | total_timesteps 3789.
Path 374 | total_timesteps 3799.
Path 375 | total_timesteps 3815.
Path 376 | total_timesteps 3824.
Path 377 | total_timesteps 3836.
Path 378 | total_timesteps 3849.
Path 379 | total_timesteps 3857.
Path 380 | total_timesteps 3865.
Path 381 | total_timesteps 3874.
Path 382 | total_timesteps 3883.
Path 383 | total_timesteps 3892.
Path 384 | total_timesteps 3901.
Path 385 | total_timesteps 3913.
Path 386 | total_timesteps 3936.
Path 387 | total_timesteps 3943.
Path 388 | total_timesteps 3959.
Path 389 | total_timesteps 3973.
Path 390 | total_timesteps 3980.
Path 391 | total_timesteps 3988.
Path 392 | total_timesteps 4001.
Path 393 | total_timesteps 4016.
Path 394 | total_timesteps 4025.
Path 395 | total_timesteps 4039.
Path 396 | total_timesteps 4050.
Path 397 | total_timesteps 4061.
Path 398 | total_timesteps 4072.
Path 399 | total_timesteps 4084.
Path 400 | total_timesteps 4093.
Path 401 | total_timesteps 4102.
Path 402 | total_timesteps 4112.
Path 403 | total_timesteps 4120.
Path 404 | total_timesteps 4129.
Path 405 | total_timesteps 4137.
Path 406 | total_timesteps 4150.
Path 407 | total_timesteps 4158.
Path 408 | total_timesteps 4165.
Path 409 | total_timesteps 4175.
Path 410 | total_timesteps 4182.
Path 411 | total_timesteps 4191.
Path 412 | total_timesteps 4205.
Path 413 | total_timesteps 4219.
Path 414 | total_timesteps 4226.
Path 415 | total_timesteps 4235.
Path 416 | total_timesteps 4246.
Path 417 | total_timesteps 4256.
Path 418 | total_timesteps 4267.
Path 419 | total_timesteps 4277.
Path 420 | total_timesteps 4286.
Path 421 | total_timesteps 4294.
Path 422 | total_timesteps 4301.
Path 423 | total_timesteps 4314.
Path 424 | total_timesteps 4322.
Path 425 | total_timesteps 4331.
Path 426 | total_timesteps 4339.
Path 427 | total_timesteps 4350.
Path 428 | total_timesteps 4361.
Path 429 | total_timesteps 4372.
Path 430 | total_timesteps 4380.
Path 431 | total_timesteps 4390.
Path 432 | total_timesteps 4400.
Path 433 | total_timesteps 4410.
Path 434 | total_timesteps 4417.
Path 435 | total_timesteps 4435.
Path 436 | total_timesteps 4442.
Path 437 | total_timesteps 4450.
Path 438 | total_timesteps 4465.
Path 439 | total_timesteps 4473.
Path 440 | total_timesteps 4479.
Path 441 | total_timesteps 4486.
Path 442 | total_timesteps 4501.
Path 443 | total_timesteps 4509.
Path 444 | total_timesteps 4517.
Path 445 | total_timesteps 4532.
Path 446 | total_timesteps 4541.
Path 447 | total_timesteps 4548.
Path 448 | total_timesteps 4562.
Path 449 | total_timesteps 4571.
Path 450 | total_timesteps 4581.
Path 451 | total_timesteps 4594.
Path 452 | total_timesteps 4604.
Path 453 | total_timesteps 4616.
Path 454 | total_timesteps 4626.
Path 455 | total_timesteps 4638.
Path 456 | total_timesteps 4647.
Path 457 | total_timesteps 4663.
Path 458 | total_timesteps 4674.
Path 459 | total_timesteps 4694.
Path 460 | total_timesteps 4703.
Path 461 | total_timesteps 4713.
Path 462 | total_timesteps 4720.
Path 463 | total_timesteps 4728.
Path 464 | total_timesteps 4737.
Path 465 | total_timesteps 4748.
Path 466 | total_timesteps 4759.
Path 467 | total_timesteps 4771.
Path 468 | total_timesteps 4782.
Path 469 | total_timesteps 4792.
Path 470 | total_timesteps 4799.
Path 471 | total_timesteps 4807.
Path 472 | total_timesteps 4814.
Path 473 | total_timesteps 4821.
Path 474 | total_timesteps 4832.
Path 475 | total_timesteps 4841.
Path 476 | total_timesteps 4848.
Path 477 | total_timesteps 4857.
Path 478 | total_timesteps 4864.
Path 479 | total_timesteps 4873.
Path 480 | total_timesteps 4881.
Path 481 | total_timesteps 4891.
Path 482 | total_timesteps 4904.
Path 483 | total_timesteps 4916.
Path 484 | total_timesteps 4924.
Path 485 | total_timesteps 4932.
Path 486 | total_timesteps 4944.
Path 487 | total_timesteps 4959.
Path 488 | total_timesteps 4969.
Path 489 | total_timesteps 4982.
Path 490 | total_timesteps 4991.
Path 491 | total_timesteps 5002.
Path 492 | total_timesteps 5016.
Path 493 | total_timesteps 5029.
Path 494 | total_timesteps 5036.
Path 495 | total_timesteps 5047.
Path 496 | total_timesteps 5056.
Path 497 | total_timesteps 5066.
Path 498 | total_timesteps 5074.
Path 499 | total_timesteps 5082.
Path 500 | total_timesteps 5092.
Path 501 | total_timesteps 5103.
Path 502 | total_timesteps 5117.
Path 503 | total_timesteps 5132.
Path 504 | total_timesteps 5142.
Path 505 | total_timesteps 5149.
Path 506 | total_timesteps 5160.
Path 507 | total_timesteps 5166.
Path 508 | total_timesteps 5174.
Path 509 | total_timesteps 5183.
Path 510 | total_timesteps 5192.
Path 511 | total_timesteps 5199.
Path 512 | total_timesteps 5209.
Path 513 | total_timesteps 5217.
Path 514 | total_timesteps 5226.
Path 515 | total_timesteps 5234.
Path 516 | total_timesteps 5248.
Path 517 | total_timesteps 5265.
Path 518 | total_timesteps 5277.
Path 519 | total_timesteps 5285.
Path 520 | total_timesteps 5294.
Path 521 | total_timesteps 5305.
Path 522 | total_timesteps 5313.
Path 523 | total_timesteps 5327.
Path 524 | total_timesteps 5333.
Path 525 | total_timesteps 5344.
Path 526 | total_timesteps 5356.
Path 527 | total_timesteps 5368.
Path 528 | total_timesteps 5378.
Path 529 | total_timesteps 5387.
Path 530 | total_timesteps 5397.
Path 531 | total_timesteps 5404.
Path 532 | total_timesteps 5412.
Path 533 | total_timesteps 5424.
Path 534 | total_timesteps 5432.
Path 535 | total_timesteps 5441.
Path 536 | total_timesteps 5449.
Path 537 | total_timesteps 5456.
Path 538 | total_timesteps 5464.
Path 539 | total_timesteps 5473.
Path 540 | total_timesteps 5485.
Path 541 | total_timesteps 5496.
Path 542 | total_timesteps 5507.
Path 543 | total_timesteps 5515.
Path 544 | total_timesteps 5523.
Path 545 | total_timesteps 5535.
Path 546 | total_timesteps 5542.
Path 547 | total_timesteps 5556.
Path 548 | total_timesteps 5565.
Path 549 | total_timesteps 5573.
Path 550 | total_timesteps 5580.
Path 551 | total_timesteps 5589.
Path 552 | total_timesteps 5597.
Path 553 | total_timesteps 5605.
Path 554 | total_timesteps 5614.
Path 555 | total_timesteps 5622.
Path 556 | total_timesteps 5633.
Path 557 | total_timesteps 5639.
Path 558 | total_timesteps 5652.
Path 559 | total_timesteps 5661.
Path 560 | total_timesteps 5668.
Path 561 | total_timesteps 5677.
Path 562 | total_timesteps 5702.
Path 563 | total_timesteps 5709.
Path 564 | total_timesteps 5717.
Path 565 | total_timesteps 5730.
Path 566 | total_timesteps 5743.
Path 567 | total_timesteps 5750.
Path 568 | total_timesteps 5761.
Path 569 | total_timesteps 5771.
Path 570 | total_timesteps 5778.
Path 571 | total_timesteps 5787.
Path 572 | total_timesteps 5795.
Path 573 | total_timesteps 5804.
Path 574 | total_timesteps 5813.
Path 575 | total_timesteps 5824.
Path 576 | total_timesteps 5835.
Path 577 | total_timesteps 5842.
Path 578 | total_timesteps 5850.
Path 579 | total_timesteps 5860.
Path 580 | total_timesteps 5871.
Path 581 | total_timesteps 5879.
Path 582 | total_timesteps 5888.
Path 583 | total_timesteps 5898.
Path 584 | total_timesteps 5910.
Path 585 | total_timesteps 5922.
Path 586 | total_timesteps 5938.
Path 587 | total_timesteps 5946.
Path 588 | total_timesteps 5955.
Path 589 | total_timesteps 5965.
Path 590 | total_timesteps 5972.
Path 591 | total_timesteps 5983.
Path 592 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.7     |
| Iteration     | 5        |
| MaximumReturn | 1.84     |
| MinimumReturn | -19      |
| TotalSamples  | 28020    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.2898081839084625
Validation loss = 0.29483482241630554
Validation loss = 0.30214157700538635
Validation loss = 0.30118080973625183
Validation loss = 0.30242255330085754
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 80.
Path 9 | total_timesteps 89.
Path 10 | total_timesteps 97.
Path 11 | total_timesteps 108.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 132.
Path 14 | total_timesteps 140.
Path 15 | total_timesteps 157.
Path 16 | total_timesteps 167.
Path 17 | total_timesteps 176.
Path 18 | total_timesteps 184.
Path 19 | total_timesteps 196.
Path 20 | total_timesteps 204.
Path 21 | total_timesteps 213.
Path 22 | total_timesteps 228.
Path 23 | total_timesteps 238.
Path 24 | total_timesteps 248.
Path 25 | total_timesteps 254.
Path 26 | total_timesteps 264.
Path 27 | total_timesteps 272.
Path 28 | total_timesteps 283.
Path 29 | total_timesteps 292.
Path 30 | total_timesteps 301.
Path 31 | total_timesteps 309.
Path 32 | total_timesteps 317.
Path 33 | total_timesteps 327.
Path 34 | total_timesteps 334.
Path 35 | total_timesteps 341.
Path 36 | total_timesteps 351.
Path 37 | total_timesteps 361.
Path 38 | total_timesteps 368.
Path 39 | total_timesteps 377.
Path 40 | total_timesteps 385.
Path 41 | total_timesteps 395.
Path 42 | total_timesteps 405.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 431.
Path 45 | total_timesteps 441.
Path 46 | total_timesteps 455.
Path 47 | total_timesteps 462.
Path 48 | total_timesteps 470.
Path 49 | total_timesteps 480.
Path 50 | total_timesteps 487.
Path 51 | total_timesteps 496.
Path 52 | total_timesteps 507.
Path 53 | total_timesteps 514.
Path 54 | total_timesteps 522.
Path 55 | total_timesteps 532.
Path 56 | total_timesteps 542.
Path 57 | total_timesteps 552.
Path 58 | total_timesteps 560.
Path 59 | total_timesteps 574.
Path 60 | total_timesteps 584.
Path 61 | total_timesteps 595.
Path 62 | total_timesteps 606.
Path 63 | total_timesteps 618.
Path 64 | total_timesteps 631.
Path 65 | total_timesteps 639.
Path 66 | total_timesteps 648.
Path 67 | total_timesteps 662.
Path 68 | total_timesteps 670.
Path 69 | total_timesteps 681.
Path 70 | total_timesteps 688.
Path 71 | total_timesteps 696.
Path 72 | total_timesteps 703.
Path 73 | total_timesteps 713.
Path 74 | total_timesteps 725.
Path 75 | total_timesteps 733.
Path 76 | total_timesteps 746.
Path 77 | total_timesteps 762.
Path 78 | total_timesteps 770.
Path 79 | total_timesteps 785.
Path 80 | total_timesteps 794.
Path 81 | total_timesteps 801.
Path 82 | total_timesteps 808.
Path 83 | total_timesteps 820.
Path 84 | total_timesteps 833.
Path 85 | total_timesteps 845.
Path 86 | total_timesteps 859.
Path 87 | total_timesteps 876.
Path 88 | total_timesteps 884.
Path 89 | total_timesteps 893.
Path 90 | total_timesteps 900.
Path 91 | total_timesteps 908.
Path 92 | total_timesteps 918.
Path 93 | total_timesteps 930.
Path 94 | total_timesteps 937.
Path 95 | total_timesteps 945.
Path 96 | total_timesteps 955.
Path 97 | total_timesteps 963.
Path 98 | total_timesteps 971.
Path 99 | total_timesteps 983.
Path 100 | total_timesteps 990.
Path 101 | total_timesteps 999.
Path 102 | total_timesteps 1009.
Path 103 | total_timesteps 1020.
Path 104 | total_timesteps 1032.
Path 105 | total_timesteps 1041.
Path 106 | total_timesteps 1048.
Path 107 | total_timesteps 1057.
Path 108 | total_timesteps 1064.
Path 109 | total_timesteps 1072.
Path 110 | total_timesteps 1083.
Path 111 | total_timesteps 1092.
Path 112 | total_timesteps 1099.
Path 113 | total_timesteps 1107.
Path 114 | total_timesteps 1115.
Path 115 | total_timesteps 1125.
Path 116 | total_timesteps 1135.
Path 117 | total_timesteps 1147.
Path 118 | total_timesteps 1154.
Path 119 | total_timesteps 1162.
Path 120 | total_timesteps 1174.
Path 121 | total_timesteps 1183.
Path 122 | total_timesteps 1191.
Path 123 | total_timesteps 1201.
Path 124 | total_timesteps 1211.
Path 125 | total_timesteps 1219.
Path 126 | total_timesteps 1227.
Path 127 | total_timesteps 1236.
Path 128 | total_timesteps 1245.
Path 129 | total_timesteps 1254.
Path 130 | total_timesteps 1262.
Path 131 | total_timesteps 1269.
Path 132 | total_timesteps 1280.
Path 133 | total_timesteps 1291.
Path 134 | total_timesteps 1302.
Path 135 | total_timesteps 1312.
Path 136 | total_timesteps 1320.
Path 137 | total_timesteps 1332.
Path 138 | total_timesteps 1346.
Path 139 | total_timesteps 1355.
Path 140 | total_timesteps 1369.
Path 141 | total_timesteps 1379.
Path 142 | total_timesteps 1389.
Path 143 | total_timesteps 1398.
Path 144 | total_timesteps 1408.
Path 145 | total_timesteps 1417.
Path 146 | total_timesteps 1430.
Path 147 | total_timesteps 1440.
Path 148 | total_timesteps 1447.
Path 149 | total_timesteps 1457.
Path 150 | total_timesteps 1477.
Path 151 | total_timesteps 1487.
Path 152 | total_timesteps 1494.
Path 153 | total_timesteps 1503.
Path 154 | total_timesteps 1513.
Path 155 | total_timesteps 1530.
Path 156 | total_timesteps 1540.
Path 157 | total_timesteps 1549.
Path 158 | total_timesteps 1558.
Path 159 | total_timesteps 1569.
Path 160 | total_timesteps 1582.
Path 161 | total_timesteps 1592.
Path 162 | total_timesteps 1599.
Path 163 | total_timesteps 1608.
Path 164 | total_timesteps 1617.
Path 165 | total_timesteps 1627.
Path 166 | total_timesteps 1644.
Path 167 | total_timesteps 1653.
Path 168 | total_timesteps 1661.
Path 169 | total_timesteps 1674.
Path 170 | total_timesteps 1684.
Path 171 | total_timesteps 1694.
Path 172 | total_timesteps 1701.
Path 173 | total_timesteps 1709.
Path 174 | total_timesteps 1716.
Path 175 | total_timesteps 1731.
Path 176 | total_timesteps 1738.
Path 177 | total_timesteps 1746.
Path 178 | total_timesteps 1755.
Path 179 | total_timesteps 1763.
Path 180 | total_timesteps 1771.
Path 181 | total_timesteps 1783.
Path 182 | total_timesteps 1792.
Path 183 | total_timesteps 1800.
Path 184 | total_timesteps 1806.
Path 185 | total_timesteps 1815.
Path 186 | total_timesteps 1827.
Path 187 | total_timesteps 1836.
Path 188 | total_timesteps 1846.
Path 189 | total_timesteps 1859.
Path 190 | total_timesteps 1868.
Path 191 | total_timesteps 1878.
Path 192 | total_timesteps 1892.
Path 193 | total_timesteps 1901.
Path 194 | total_timesteps 1912.
Path 195 | total_timesteps 1924.
Path 196 | total_timesteps 1936.
Path 197 | total_timesteps 1946.
Path 198 | total_timesteps 1968.
Path 199 | total_timesteps 1979.
Path 200 | total_timesteps 1991.
Path 201 | total_timesteps 2002.
Path 202 | total_timesteps 2012.
Path 203 | total_timesteps 2020.
Path 204 | total_timesteps 2028.
Path 205 | total_timesteps 2039.
Path 206 | total_timesteps 2051.
Path 207 | total_timesteps 2059.
Path 208 | total_timesteps 2069.
Path 209 | total_timesteps 2081.
Path 210 | total_timesteps 2088.
Path 211 | total_timesteps 2100.
Path 212 | total_timesteps 2110.
Path 213 | total_timesteps 2118.
Path 214 | total_timesteps 2124.
Path 215 | total_timesteps 2139.
Path 216 | total_timesteps 2150.
Path 217 | total_timesteps 2160.
Path 218 | total_timesteps 2168.
Path 219 | total_timesteps 2177.
Path 220 | total_timesteps 2189.
Path 221 | total_timesteps 2200.
Path 222 | total_timesteps 2211.
Path 223 | total_timesteps 2225.
Path 224 | total_timesteps 2234.
Path 225 | total_timesteps 2241.
Path 226 | total_timesteps 2254.
Path 227 | total_timesteps 2262.
Path 228 | total_timesteps 2272.
Path 229 | total_timesteps 2282.
Path 230 | total_timesteps 2290.
Path 231 | total_timesteps 2299.
Path 232 | total_timesteps 2315.
Path 233 | total_timesteps 2326.
Path 234 | total_timesteps 2335.
Path 235 | total_timesteps 2348.
Path 236 | total_timesteps 2366.
Path 237 | total_timesteps 2375.
Path 238 | total_timesteps 2386.
Path 239 | total_timesteps 2394.
Path 240 | total_timesteps 2405.
Path 241 | total_timesteps 2415.
Path 242 | total_timesteps 2424.
Path 243 | total_timesteps 2436.
Path 244 | total_timesteps 2442.
Path 245 | total_timesteps 2454.
Path 246 | total_timesteps 2464.
Path 247 | total_timesteps 2477.
Path 248 | total_timesteps 2485.
Path 249 | total_timesteps 2494.
Path 250 | total_timesteps 2507.
Path 251 | total_timesteps 2516.
Path 252 | total_timesteps 2524.
Path 253 | total_timesteps 2535.
Path 254 | total_timesteps 2544.
Path 255 | total_timesteps 2552.
Path 256 | total_timesteps 2559.
Path 257 | total_timesteps 2565.
Path 258 | total_timesteps 2574.
Path 259 | total_timesteps 2584.
Path 260 | total_timesteps 2592.
Path 261 | total_timesteps 2601.
Path 262 | total_timesteps 2611.
Path 263 | total_timesteps 2622.
Path 264 | total_timesteps 2636.
Path 265 | total_timesteps 2645.
Path 266 | total_timesteps 2655.
Path 267 | total_timesteps 2664.
Path 268 | total_timesteps 2674.
Path 269 | total_timesteps 2690.
Path 270 | total_timesteps 2698.
Path 271 | total_timesteps 2710.
Path 272 | total_timesteps 2719.
Path 273 | total_timesteps 2726.
Path 274 | total_timesteps 2742.
Path 275 | total_timesteps 2749.
Path 276 | total_timesteps 2757.
Path 277 | total_timesteps 2770.
Path 278 | total_timesteps 2780.
Path 279 | total_timesteps 2789.
Path 280 | total_timesteps 2799.
Path 281 | total_timesteps 2813.
Path 282 | total_timesteps 2822.
Path 283 | total_timesteps 2831.
Path 284 | total_timesteps 2837.
Path 285 | total_timesteps 2852.
Path 286 | total_timesteps 2859.
Path 287 | total_timesteps 2868.
Path 288 | total_timesteps 2877.
Path 289 | total_timesteps 2883.
Path 290 | total_timesteps 2895.
Path 291 | total_timesteps 2904.
Path 292 | total_timesteps 2914.
Path 293 | total_timesteps 2922.
Path 294 | total_timesteps 2934.
Path 295 | total_timesteps 2942.
Path 296 | total_timesteps 2950.
Path 297 | total_timesteps 2962.
Path 298 | total_timesteps 2969.
Path 299 | total_timesteps 2982.
Path 300 | total_timesteps 2992.
Path 301 | total_timesteps 3000.
Path 302 | total_timesteps 3007.
Path 303 | total_timesteps 3018.
Path 304 | total_timesteps 3027.
Path 305 | total_timesteps 3037.
Path 306 | total_timesteps 3048.
Path 307 | total_timesteps 3060.
Path 308 | total_timesteps 3067.
Path 309 | total_timesteps 3076.
Path 310 | total_timesteps 3084.
Path 311 | total_timesteps 3094.
Path 312 | total_timesteps 3104.
Path 313 | total_timesteps 3110.
Path 314 | total_timesteps 3122.
Path 315 | total_timesteps 3134.
Path 316 | total_timesteps 3142.
Path 317 | total_timesteps 3153.
Path 318 | total_timesteps 3162.
Path 319 | total_timesteps 3171.
Path 320 | total_timesteps 3181.
Path 321 | total_timesteps 3188.
Path 322 | total_timesteps 3201.
Path 323 | total_timesteps 3210.
Path 324 | total_timesteps 3222.
Path 325 | total_timesteps 3232.
Path 326 | total_timesteps 3241.
Path 327 | total_timesteps 3248.
Path 328 | total_timesteps 3261.
Path 329 | total_timesteps 3269.
Path 330 | total_timesteps 3285.
Path 331 | total_timesteps 3298.
Path 332 | total_timesteps 3306.
Path 333 | total_timesteps 3320.
Path 334 | total_timesteps 3328.
Path 335 | total_timesteps 3337.
Path 336 | total_timesteps 3348.
Path 337 | total_timesteps 3356.
Path 338 | total_timesteps 3368.
Path 339 | total_timesteps 3375.
Path 340 | total_timesteps 3388.
Path 341 | total_timesteps 3396.
Path 342 | total_timesteps 3404.
Path 343 | total_timesteps 3415.
Path 344 | total_timesteps 3426.
Path 345 | total_timesteps 3433.
Path 346 | total_timesteps 3441.
Path 347 | total_timesteps 3452.
Path 348 | total_timesteps 3464.
Path 349 | total_timesteps 3477.
Path 350 | total_timesteps 3484.
Path 351 | total_timesteps 3492.
Path 352 | total_timesteps 3507.
Path 353 | total_timesteps 3517.
Path 354 | total_timesteps 3527.
Path 355 | total_timesteps 3537.
Path 356 | total_timesteps 3549.
Path 357 | total_timesteps 3558.
Path 358 | total_timesteps 3572.
Path 359 | total_timesteps 3585.
Path 360 | total_timesteps 3596.
Path 361 | total_timesteps 3606.
Path 362 | total_timesteps 3615.
Path 363 | total_timesteps 3623.
Path 364 | total_timesteps 3632.
Path 365 | total_timesteps 3646.
Path 366 | total_timesteps 3656.
Path 367 | total_timesteps 3664.
Path 368 | total_timesteps 3671.
Path 369 | total_timesteps 3684.
Path 370 | total_timesteps 3693.
Path 371 | total_timesteps 3704.
Path 372 | total_timesteps 3718.
Path 373 | total_timesteps 3733.
Path 374 | total_timesteps 3740.
Path 375 | total_timesteps 3747.
Path 376 | total_timesteps 3759.
Path 377 | total_timesteps 3768.
Path 378 | total_timesteps 3777.
Path 379 | total_timesteps 3785.
Path 380 | total_timesteps 3793.
Path 381 | total_timesteps 3801.
Path 382 | total_timesteps 3810.
Path 383 | total_timesteps 3818.
Path 384 | total_timesteps 3828.
Path 385 | total_timesteps 3838.
Path 386 | total_timesteps 3849.
Path 387 | total_timesteps 3857.
Path 388 | total_timesteps 3867.
Path 389 | total_timesteps 3878.
Path 390 | total_timesteps 3886.
Path 391 | total_timesteps 3895.
Path 392 | total_timesteps 3903.
Path 393 | total_timesteps 3914.
Path 394 | total_timesteps 3926.
Path 395 | total_timesteps 3936.
Path 396 | total_timesteps 3946.
Path 397 | total_timesteps 3954.
Path 398 | total_timesteps 3964.
Path 399 | total_timesteps 3972.
Path 400 | total_timesteps 3981.
Path 401 | total_timesteps 4000.
Path 402 | total_timesteps 4011.
Path 403 | total_timesteps 4020.
Path 404 | total_timesteps 4028.
Path 405 | total_timesteps 4038.
Path 406 | total_timesteps 4048.
Path 407 | total_timesteps 4056.
Path 408 | total_timesteps 4070.
Path 409 | total_timesteps 4077.
Path 410 | total_timesteps 4088.
Path 411 | total_timesteps 4108.
Path 412 | total_timesteps 4116.
Path 413 | total_timesteps 4125.
Path 414 | total_timesteps 4137.
Path 415 | total_timesteps 4145.
Path 416 | total_timesteps 4154.
Path 417 | total_timesteps 4168.
Path 418 | total_timesteps 4176.
Path 419 | total_timesteps 4186.
Path 420 | total_timesteps 4193.
Path 421 | total_timesteps 4204.
Path 422 | total_timesteps 4218.
Path 423 | total_timesteps 4231.
Path 424 | total_timesteps 4245.
Path 425 | total_timesteps 4255.
Path 426 | total_timesteps 4267.
Path 427 | total_timesteps 4278.
Path 428 | total_timesteps 4286.
Path 429 | total_timesteps 4297.
Path 430 | total_timesteps 4306.
Path 431 | total_timesteps 4321.
Path 432 | total_timesteps 4333.
Path 433 | total_timesteps 4345.
Path 434 | total_timesteps 4358.
Path 435 | total_timesteps 4369.
Path 436 | total_timesteps 4379.
Path 437 | total_timesteps 4389.
Path 438 | total_timesteps 4399.
Path 439 | total_timesteps 4407.
Path 440 | total_timesteps 4416.
Path 441 | total_timesteps 4433.
Path 442 | total_timesteps 4442.
Path 443 | total_timesteps 4451.
Path 444 | total_timesteps 4458.
Path 445 | total_timesteps 4468.
Path 446 | total_timesteps 4477.
Path 447 | total_timesteps 4495.
Path 448 | total_timesteps 4502.
Path 449 | total_timesteps 4511.
Path 450 | total_timesteps 4521.
Path 451 | total_timesteps 4529.
Path 452 | total_timesteps 4542.
Path 453 | total_timesteps 4550.
Path 454 | total_timesteps 4560.
Path 455 | total_timesteps 4569.
Path 456 | total_timesteps 4579.
Path 457 | total_timesteps 4591.
Path 458 | total_timesteps 4601.
Path 459 | total_timesteps 4609.
Path 460 | total_timesteps 4627.
Path 461 | total_timesteps 4641.
Path 462 | total_timesteps 4650.
Path 463 | total_timesteps 4658.
Path 464 | total_timesteps 4666.
Path 465 | total_timesteps 4673.
Path 466 | total_timesteps 4682.
Path 467 | total_timesteps 4690.
Path 468 | total_timesteps 4706.
Path 469 | total_timesteps 4717.
Path 470 | total_timesteps 4728.
Path 471 | total_timesteps 4739.
Path 472 | total_timesteps 4749.
Path 473 | total_timesteps 4757.
Path 474 | total_timesteps 4767.
Path 475 | total_timesteps 4775.
Path 476 | total_timesteps 4789.
Path 477 | total_timesteps 4798.
Path 478 | total_timesteps 4809.
Path 479 | total_timesteps 4821.
Path 480 | total_timesteps 4833.
Path 481 | total_timesteps 4845.
Path 482 | total_timesteps 4854.
Path 483 | total_timesteps 4863.
Path 484 | total_timesteps 4876.
Path 485 | total_timesteps 4886.
Path 486 | total_timesteps 4898.
Path 487 | total_timesteps 4905.
Path 488 | total_timesteps 4917.
Path 489 | total_timesteps 4931.
Path 490 | total_timesteps 4941.
Path 491 | total_timesteps 4952.
Path 492 | total_timesteps 4959.
Path 493 | total_timesteps 4966.
Path 494 | total_timesteps 4978.
Path 495 | total_timesteps 4987.
Path 496 | total_timesteps 4999.
Path 497 | total_timesteps 5008.
Path 498 | total_timesteps 5020.
Path 499 | total_timesteps 5035.
Path 500 | total_timesteps 5044.
Path 501 | total_timesteps 5057.
Path 502 | total_timesteps 5073.
Path 503 | total_timesteps 5083.
Path 504 | total_timesteps 5094.
Path 505 | total_timesteps 5105.
Path 506 | total_timesteps 5113.
Path 507 | total_timesteps 5123.
Path 508 | total_timesteps 5133.
Path 509 | total_timesteps 5146.
Path 510 | total_timesteps 5160.
Path 511 | total_timesteps 5170.
Path 512 | total_timesteps 5179.
Path 513 | total_timesteps 5192.
Path 514 | total_timesteps 5202.
Path 515 | total_timesteps 5214.
Path 516 | total_timesteps 5222.
Path 517 | total_timesteps 5232.
Path 518 | total_timesteps 5242.
Path 519 | total_timesteps 5253.
Path 520 | total_timesteps 5265.
Path 521 | total_timesteps 5276.
Path 522 | total_timesteps 5287.
Path 523 | total_timesteps 5295.
Path 524 | total_timesteps 5302.
Path 525 | total_timesteps 5309.
Path 526 | total_timesteps 5320.
Path 527 | total_timesteps 5329.
Path 528 | total_timesteps 5340.
Path 529 | total_timesteps 5351.
Path 530 | total_timesteps 5360.
Path 531 | total_timesteps 5371.
Path 532 | total_timesteps 5388.
Path 533 | total_timesteps 5396.
Path 534 | total_timesteps 5406.
Path 535 | total_timesteps 5412.
Path 536 | total_timesteps 5420.
Path 537 | total_timesteps 5429.
Path 538 | total_timesteps 5441.
Path 539 | total_timesteps 5457.
Path 540 | total_timesteps 5464.
Path 541 | total_timesteps 5475.
Path 542 | total_timesteps 5486.
Path 543 | total_timesteps 5494.
Path 544 | total_timesteps 5501.
Path 545 | total_timesteps 5510.
Path 546 | total_timesteps 5522.
Path 547 | total_timesteps 5534.
Path 548 | total_timesteps 5543.
Path 549 | total_timesteps 5553.
Path 550 | total_timesteps 5563.
Path 551 | total_timesteps 5576.
Path 552 | total_timesteps 5588.
Path 553 | total_timesteps 5596.
Path 554 | total_timesteps 5603.
Path 555 | total_timesteps 5612.
Path 556 | total_timesteps 5619.
Path 557 | total_timesteps 5629.
Path 558 | total_timesteps 5638.
Path 559 | total_timesteps 5652.
Path 560 | total_timesteps 5664.
Path 561 | total_timesteps 5672.
Path 562 | total_timesteps 5685.
Path 563 | total_timesteps 5697.
Path 564 | total_timesteps 5706.
Path 565 | total_timesteps 5717.
Path 566 | total_timesteps 5727.
Path 567 | total_timesteps 5738.
Path 568 | total_timesteps 5746.
Path 569 | total_timesteps 5756.
Path 570 | total_timesteps 5764.
Path 571 | total_timesteps 5775.
Path 572 | total_timesteps 5783.
Path 573 | total_timesteps 5794.
Path 574 | total_timesteps 5808.
Path 575 | total_timesteps 5818.
Path 576 | total_timesteps 5832.
Path 577 | total_timesteps 5841.
Path 578 | total_timesteps 5849.
Path 579 | total_timesteps 5855.
Path 580 | total_timesteps 5862.
Path 581 | total_timesteps 5870.
Path 582 | total_timesteps 5878.
Path 583 | total_timesteps 5890.
Path 584 | total_timesteps 5899.
Path 585 | total_timesteps 5909.
Path 586 | total_timesteps 5920.
Path 587 | total_timesteps 5930.
Path 588 | total_timesteps 5944.
Path 589 | total_timesteps 5951.
Path 590 | total_timesteps 5957.
Path 591 | total_timesteps 5972.
Path 592 | total_timesteps 5981.
Path 593 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.82    |
| Iteration     | 6        |
| MaximumReturn | 2.06     |
| MinimumReturn | -16.2    |
| TotalSamples  | 32020    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.2956983745098114
Validation loss = 0.29745322465896606
Validation loss = 0.30335789918899536
Validation loss = 0.3065776824951172
Validation loss = 0.31618791818618774
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 70.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 102.
Path 11 | total_timesteps 113.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 131.
Path 14 | total_timesteps 156.
Path 15 | total_timesteps 165.
Path 16 | total_timesteps 184.
Path 17 | total_timesteps 193.
Path 18 | total_timesteps 200.
Path 19 | total_timesteps 214.
Path 20 | total_timesteps 224.
Path 21 | total_timesteps 233.
Path 22 | total_timesteps 242.
Path 23 | total_timesteps 254.
Path 24 | total_timesteps 265.
Path 25 | total_timesteps 273.
Path 26 | total_timesteps 283.
Path 27 | total_timesteps 291.
Path 28 | total_timesteps 299.
Path 29 | total_timesteps 312.
Path 30 | total_timesteps 329.
Path 31 | total_timesteps 337.
Path 32 | total_timesteps 345.
Path 33 | total_timesteps 357.
Path 34 | total_timesteps 367.
Path 35 | total_timesteps 376.
Path 36 | total_timesteps 389.
Path 37 | total_timesteps 408.
Path 38 | total_timesteps 418.
Path 39 | total_timesteps 426.
Path 40 | total_timesteps 434.
Path 41 | total_timesteps 448.
Path 42 | total_timesteps 458.
Path 43 | total_timesteps 465.
Path 44 | total_timesteps 479.
Path 45 | total_timesteps 487.
Path 46 | total_timesteps 498.
Path 47 | total_timesteps 509.
Path 48 | total_timesteps 524.
Path 49 | total_timesteps 537.
Path 50 | total_timesteps 549.
Path 51 | total_timesteps 558.
Path 52 | total_timesteps 565.
Path 53 | total_timesteps 572.
Path 54 | total_timesteps 581.
Path 55 | total_timesteps 590.
Path 56 | total_timesteps 603.
Path 57 | total_timesteps 615.
Path 58 | total_timesteps 627.
Path 59 | total_timesteps 636.
Path 60 | total_timesteps 646.
Path 61 | total_timesteps 662.
Path 62 | total_timesteps 672.
Path 63 | total_timesteps 681.
Path 64 | total_timesteps 688.
Path 65 | total_timesteps 698.
Path 66 | total_timesteps 708.
Path 67 | total_timesteps 715.
Path 68 | total_timesteps 721.
Path 69 | total_timesteps 735.
Path 70 | total_timesteps 745.
Path 71 | total_timesteps 755.
Path 72 | total_timesteps 765.
Path 73 | total_timesteps 774.
Path 74 | total_timesteps 780.
Path 75 | total_timesteps 790.
Path 76 | total_timesteps 798.
Path 77 | total_timesteps 805.
Path 78 | total_timesteps 816.
Path 79 | total_timesteps 828.
Path 80 | total_timesteps 840.
Path 81 | total_timesteps 849.
Path 82 | total_timesteps 868.
Path 83 | total_timesteps 877.
Path 84 | total_timesteps 889.
Path 85 | total_timesteps 899.
Path 86 | total_timesteps 914.
Path 87 | total_timesteps 930.
Path 88 | total_timesteps 940.
Path 89 | total_timesteps 946.
Path 90 | total_timesteps 954.
Path 91 | total_timesteps 963.
Path 92 | total_timesteps 975.
Path 93 | total_timesteps 990.
Path 94 | total_timesteps 1006.
Path 95 | total_timesteps 1014.
Path 96 | total_timesteps 1025.
Path 97 | total_timesteps 1034.
Path 98 | total_timesteps 1050.
Path 99 | total_timesteps 1063.
Path 100 | total_timesteps 1072.
Path 101 | total_timesteps 1082.
Path 102 | total_timesteps 1093.
Path 103 | total_timesteps 1104.
Path 104 | total_timesteps 1114.
Path 105 | total_timesteps 1121.
Path 106 | total_timesteps 1132.
Path 107 | total_timesteps 1142.
Path 108 | total_timesteps 1149.
Path 109 | total_timesteps 1158.
Path 110 | total_timesteps 1168.
Path 111 | total_timesteps 1177.
Path 112 | total_timesteps 1187.
Path 113 | total_timesteps 1196.
Path 114 | total_timesteps 1205.
Path 115 | total_timesteps 1214.
Path 116 | total_timesteps 1226.
Path 117 | total_timesteps 1233.
Path 118 | total_timesteps 1239.
Path 119 | total_timesteps 1247.
Path 120 | total_timesteps 1256.
Path 121 | total_timesteps 1265.
Path 122 | total_timesteps 1277.
Path 123 | total_timesteps 1290.
Path 124 | total_timesteps 1305.
Path 125 | total_timesteps 1312.
Path 126 | total_timesteps 1320.
Path 127 | total_timesteps 1330.
Path 128 | total_timesteps 1340.
Path 129 | total_timesteps 1352.
Path 130 | total_timesteps 1361.
Path 131 | total_timesteps 1372.
Path 132 | total_timesteps 1382.
Path 133 | total_timesteps 1394.
Path 134 | total_timesteps 1405.
Path 135 | total_timesteps 1417.
Path 136 | total_timesteps 1425.
Path 137 | total_timesteps 1442.
Path 138 | total_timesteps 1454.
Path 139 | total_timesteps 1463.
Path 140 | total_timesteps 1470.
Path 141 | total_timesteps 1477.
Path 142 | total_timesteps 1485.
Path 143 | total_timesteps 1493.
Path 144 | total_timesteps 1501.
Path 145 | total_timesteps 1514.
Path 146 | total_timesteps 1527.
Path 147 | total_timesteps 1542.
Path 148 | total_timesteps 1553.
Path 149 | total_timesteps 1561.
Path 150 | total_timesteps 1572.
Path 151 | total_timesteps 1578.
Path 152 | total_timesteps 1586.
Path 153 | total_timesteps 1592.
Path 154 | total_timesteps 1606.
Path 155 | total_timesteps 1616.
Path 156 | total_timesteps 1625.
Path 157 | total_timesteps 1633.
Path 158 | total_timesteps 1641.
Path 159 | total_timesteps 1659.
Path 160 | total_timesteps 1666.
Path 161 | total_timesteps 1678.
Path 162 | total_timesteps 1690.
Path 163 | total_timesteps 1704.
Path 164 | total_timesteps 1717.
Path 165 | total_timesteps 1726.
Path 166 | total_timesteps 1734.
Path 167 | total_timesteps 1742.
Path 168 | total_timesteps 1749.
Path 169 | total_timesteps 1759.
Path 170 | total_timesteps 1768.
Path 171 | total_timesteps 1776.
Path 172 | total_timesteps 1790.
Path 173 | total_timesteps 1799.
Path 174 | total_timesteps 1806.
Path 175 | total_timesteps 1817.
Path 176 | total_timesteps 1826.
Path 177 | total_timesteps 1835.
Path 178 | total_timesteps 1841.
Path 179 | total_timesteps 1855.
Path 180 | total_timesteps 1866.
Path 181 | total_timesteps 1874.
Path 182 | total_timesteps 1883.
Path 183 | total_timesteps 1895.
Path 184 | total_timesteps 1903.
Path 185 | total_timesteps 1915.
Path 186 | total_timesteps 1928.
Path 187 | total_timesteps 1935.
Path 188 | total_timesteps 1943.
Path 189 | total_timesteps 1953.
Path 190 | total_timesteps 1965.
Path 191 | total_timesteps 1973.
Path 192 | total_timesteps 1981.
Path 193 | total_timesteps 1991.
Path 194 | total_timesteps 2002.
Path 195 | total_timesteps 2014.
Path 196 | total_timesteps 2026.
Path 197 | total_timesteps 2037.
Path 198 | total_timesteps 2048.
Path 199 | total_timesteps 2057.
Path 200 | total_timesteps 2066.
Path 201 | total_timesteps 2077.
Path 202 | total_timesteps 2086.
Path 203 | total_timesteps 2094.
Path 204 | total_timesteps 2101.
Path 205 | total_timesteps 2115.
Path 206 | total_timesteps 2127.
Path 207 | total_timesteps 2135.
Path 208 | total_timesteps 2146.
Path 209 | total_timesteps 2165.
Path 210 | total_timesteps 2174.
Path 211 | total_timesteps 2183.
Path 212 | total_timesteps 2195.
Path 213 | total_timesteps 2201.
Path 214 | total_timesteps 2213.
Path 215 | total_timesteps 2220.
Path 216 | total_timesteps 2231.
Path 217 | total_timesteps 2240.
Path 218 | total_timesteps 2249.
Path 219 | total_timesteps 2256.
Path 220 | total_timesteps 2265.
Path 221 | total_timesteps 2276.
Path 222 | total_timesteps 2292.
Path 223 | total_timesteps 2299.
Path 224 | total_timesteps 2307.
Path 225 | total_timesteps 2318.
Path 226 | total_timesteps 2325.
Path 227 | total_timesteps 2336.
Path 228 | total_timesteps 2344.
Path 229 | total_timesteps 2353.
Path 230 | total_timesteps 2360.
Path 231 | total_timesteps 2378.
Path 232 | total_timesteps 2389.
Path 233 | total_timesteps 2398.
Path 234 | total_timesteps 2406.
Path 235 | total_timesteps 2417.
Path 236 | total_timesteps 2431.
Path 237 | total_timesteps 2437.
Path 238 | total_timesteps 2448.
Path 239 | total_timesteps 2457.
Path 240 | total_timesteps 2466.
Path 241 | total_timesteps 2475.
Path 242 | total_timesteps 2484.
Path 243 | total_timesteps 2492.
Path 244 | total_timesteps 2505.
Path 245 | total_timesteps 2513.
Path 246 | total_timesteps 2523.
Path 247 | total_timesteps 2532.
Path 248 | total_timesteps 2545.
Path 249 | total_timesteps 2553.
Path 250 | total_timesteps 2563.
Path 251 | total_timesteps 2571.
Path 252 | total_timesteps 2584.
Path 253 | total_timesteps 2593.
Path 254 | total_timesteps 2600.
Path 255 | total_timesteps 2609.
Path 256 | total_timesteps 2618.
Path 257 | total_timesteps 2627.
Path 258 | total_timesteps 2635.
Path 259 | total_timesteps 2644.
Path 260 | total_timesteps 2654.
Path 261 | total_timesteps 2661.
Path 262 | total_timesteps 2671.
Path 263 | total_timesteps 2678.
Path 264 | total_timesteps 2688.
Path 265 | total_timesteps 2701.
Path 266 | total_timesteps 2712.
Path 267 | total_timesteps 2719.
Path 268 | total_timesteps 2728.
Path 269 | total_timesteps 2737.
Path 270 | total_timesteps 2744.
Path 271 | total_timesteps 2756.
Path 272 | total_timesteps 2766.
Path 273 | total_timesteps 2778.
Path 274 | total_timesteps 2784.
Path 275 | total_timesteps 2793.
Path 276 | total_timesteps 2807.
Path 277 | total_timesteps 2815.
Path 278 | total_timesteps 2825.
Path 279 | total_timesteps 2841.
Path 280 | total_timesteps 2850.
Path 281 | total_timesteps 2856.
Path 282 | total_timesteps 2867.
Path 283 | total_timesteps 2873.
Path 284 | total_timesteps 2883.
Path 285 | total_timesteps 2899.
Path 286 | total_timesteps 2910.
Path 287 | total_timesteps 2923.
Path 288 | total_timesteps 2934.
Path 289 | total_timesteps 2944.
Path 290 | total_timesteps 2952.
Path 291 | total_timesteps 2966.
Path 292 | total_timesteps 2975.
Path 293 | total_timesteps 2982.
Path 294 | total_timesteps 2995.
Path 295 | total_timesteps 3007.
Path 296 | total_timesteps 3018.
Path 297 | total_timesteps 3024.
Path 298 | total_timesteps 3033.
Path 299 | total_timesteps 3043.
Path 300 | total_timesteps 3052.
Path 301 | total_timesteps 3061.
Path 302 | total_timesteps 3072.
Path 303 | total_timesteps 3084.
Path 304 | total_timesteps 3093.
Path 305 | total_timesteps 3105.
Path 306 | total_timesteps 3119.
Path 307 | total_timesteps 3127.
Path 308 | total_timesteps 3137.
Path 309 | total_timesteps 3143.
Path 310 | total_timesteps 3154.
Path 311 | total_timesteps 3170.
Path 312 | total_timesteps 3178.
Path 313 | total_timesteps 3189.
Path 314 | total_timesteps 3197.
Path 315 | total_timesteps 3205.
Path 316 | total_timesteps 3216.
Path 317 | total_timesteps 3226.
Path 318 | total_timesteps 3235.
Path 319 | total_timesteps 3253.
Path 320 | total_timesteps 3262.
Path 321 | total_timesteps 3278.
Path 322 | total_timesteps 3287.
Path 323 | total_timesteps 3298.
Path 324 | total_timesteps 3311.
Path 325 | total_timesteps 3322.
Path 326 | total_timesteps 3332.
Path 327 | total_timesteps 3340.
Path 328 | total_timesteps 3352.
Path 329 | total_timesteps 3361.
Path 330 | total_timesteps 3370.
Path 331 | total_timesteps 3393.
Path 332 | total_timesteps 3403.
Path 333 | total_timesteps 3413.
Path 334 | total_timesteps 3422.
Path 335 | total_timesteps 3431.
Path 336 | total_timesteps 3441.
Path 337 | total_timesteps 3450.
Path 338 | total_timesteps 3458.
Path 339 | total_timesteps 3468.
Path 340 | total_timesteps 3480.
Path 341 | total_timesteps 3495.
Path 342 | total_timesteps 3504.
Path 343 | total_timesteps 3517.
Path 344 | total_timesteps 3527.
Path 345 | total_timesteps 3537.
Path 346 | total_timesteps 3548.
Path 347 | total_timesteps 3561.
Path 348 | total_timesteps 3574.
Path 349 | total_timesteps 3582.
Path 350 | total_timesteps 3592.
Path 351 | total_timesteps 3604.
Path 352 | total_timesteps 3614.
Path 353 | total_timesteps 3623.
Path 354 | total_timesteps 3633.
Path 355 | total_timesteps 3643.
Path 356 | total_timesteps 3651.
Path 357 | total_timesteps 3660.
Path 358 | total_timesteps 3667.
Path 359 | total_timesteps 3675.
Path 360 | total_timesteps 3687.
Path 361 | total_timesteps 3698.
Path 362 | total_timesteps 3710.
Path 363 | total_timesteps 3719.
Path 364 | total_timesteps 3728.
Path 365 | total_timesteps 3741.
Path 366 | total_timesteps 3753.
Path 367 | total_timesteps 3761.
Path 368 | total_timesteps 3771.
Path 369 | total_timesteps 3780.
Path 370 | total_timesteps 3788.
Path 371 | total_timesteps 3795.
Path 372 | total_timesteps 3802.
Path 373 | total_timesteps 3809.
Path 374 | total_timesteps 3822.
Path 375 | total_timesteps 3833.
Path 376 | total_timesteps 3842.
Path 377 | total_timesteps 3851.
Path 378 | total_timesteps 3859.
Path 379 | total_timesteps 3867.
Path 380 | total_timesteps 3875.
Path 381 | total_timesteps 3884.
Path 382 | total_timesteps 3894.
Path 383 | total_timesteps 3904.
Path 384 | total_timesteps 3919.
Path 385 | total_timesteps 3928.
Path 386 | total_timesteps 3939.
Path 387 | total_timesteps 3949.
Path 388 | total_timesteps 3955.
Path 389 | total_timesteps 3966.
Path 390 | total_timesteps 3976.
Path 391 | total_timesteps 3986.
Path 392 | total_timesteps 4010.
Path 393 | total_timesteps 4018.
Path 394 | total_timesteps 4034.
Path 395 | total_timesteps 4052.
Path 396 | total_timesteps 4061.
Path 397 | total_timesteps 4073.
Path 398 | total_timesteps 4082.
Path 399 | total_timesteps 4091.
Path 400 | total_timesteps 4103.
Path 401 | total_timesteps 4115.
Path 402 | total_timesteps 4124.
Path 403 | total_timesteps 4131.
Path 404 | total_timesteps 4146.
Path 405 | total_timesteps 4153.
Path 406 | total_timesteps 4165.
Path 407 | total_timesteps 4177.
Path 408 | total_timesteps 4189.
Path 409 | total_timesteps 4198.
Path 410 | total_timesteps 4205.
Path 411 | total_timesteps 4217.
Path 412 | total_timesteps 4230.
Path 413 | total_timesteps 4236.
Path 414 | total_timesteps 4246.
Path 415 | total_timesteps 4253.
Path 416 | total_timesteps 4263.
Path 417 | total_timesteps 4277.
Path 418 | total_timesteps 4287.
Path 419 | total_timesteps 4300.
Path 420 | total_timesteps 4308.
Path 421 | total_timesteps 4315.
Path 422 | total_timesteps 4326.
Path 423 | total_timesteps 4335.
Path 424 | total_timesteps 4344.
Path 425 | total_timesteps 4354.
Path 426 | total_timesteps 4364.
Path 427 | total_timesteps 4373.
Path 428 | total_timesteps 4383.
Path 429 | total_timesteps 4391.
Path 430 | total_timesteps 4403.
Path 431 | total_timesteps 4411.
Path 432 | total_timesteps 4420.
Path 433 | total_timesteps 4436.
Path 434 | total_timesteps 4446.
Path 435 | total_timesteps 4454.
Path 436 | total_timesteps 4462.
Path 437 | total_timesteps 4469.
Path 438 | total_timesteps 4485.
Path 439 | total_timesteps 4494.
Path 440 | total_timesteps 4512.
Path 441 | total_timesteps 4520.
Path 442 | total_timesteps 4528.
Path 443 | total_timesteps 4539.
Path 444 | total_timesteps 4550.
Path 445 | total_timesteps 4564.
Path 446 | total_timesteps 4576.
Path 447 | total_timesteps 4591.
Path 448 | total_timesteps 4601.
Path 449 | total_timesteps 4617.
Path 450 | total_timesteps 4627.
Path 451 | total_timesteps 4634.
Path 452 | total_timesteps 4640.
Path 453 | total_timesteps 4652.
Path 454 | total_timesteps 4659.
Path 455 | total_timesteps 4673.
Path 456 | total_timesteps 4682.
Path 457 | total_timesteps 4692.
Path 458 | total_timesteps 4703.
Path 459 | total_timesteps 4713.
Path 460 | total_timesteps 4724.
Path 461 | total_timesteps 4738.
Path 462 | total_timesteps 4746.
Path 463 | total_timesteps 4756.
Path 464 | total_timesteps 4767.
Path 465 | total_timesteps 4778.
Path 466 | total_timesteps 4785.
Path 467 | total_timesteps 4794.
Path 468 | total_timesteps 4805.
Path 469 | total_timesteps 4818.
Path 470 | total_timesteps 4830.
Path 471 | total_timesteps 4843.
Path 472 | total_timesteps 4852.
Path 473 | total_timesteps 4859.
Path 474 | total_timesteps 4869.
Path 475 | total_timesteps 4888.
Path 476 | total_timesteps 4897.
Path 477 | total_timesteps 4914.
Path 478 | total_timesteps 4922.
Path 479 | total_timesteps 4932.
Path 480 | total_timesteps 4942.
Path 481 | total_timesteps 4950.
Path 482 | total_timesteps 4963.
Path 483 | total_timesteps 4971.
Path 484 | total_timesteps 4979.
Path 485 | total_timesteps 4986.
Path 486 | total_timesteps 4998.
Path 487 | total_timesteps 5005.
Path 488 | total_timesteps 5015.
Path 489 | total_timesteps 5024.
Path 490 | total_timesteps 5032.
Path 491 | total_timesteps 5042.
Path 492 | total_timesteps 5052.
Path 493 | total_timesteps 5064.
Path 494 | total_timesteps 5071.
Path 495 | total_timesteps 5083.
Path 496 | total_timesteps 5091.
Path 497 | total_timesteps 5099.
Path 498 | total_timesteps 5108.
Path 499 | total_timesteps 5119.
Path 500 | total_timesteps 5128.
Path 501 | total_timesteps 5141.
Path 502 | total_timesteps 5153.
Path 503 | total_timesteps 5165.
Path 504 | total_timesteps 5172.
Path 505 | total_timesteps 5188.
Path 506 | total_timesteps 5201.
Path 507 | total_timesteps 5208.
Path 508 | total_timesteps 5215.
Path 509 | total_timesteps 5222.
Path 510 | total_timesteps 5233.
Path 511 | total_timesteps 5241.
Path 512 | total_timesteps 5250.
Path 513 | total_timesteps 5257.
Path 514 | total_timesteps 5265.
Path 515 | total_timesteps 5274.
Path 516 | total_timesteps 5284.
Path 517 | total_timesteps 5295.
Path 518 | total_timesteps 5304.
Path 519 | total_timesteps 5318.
Path 520 | total_timesteps 5328.
Path 521 | total_timesteps 5339.
Path 522 | total_timesteps 5356.
Path 523 | total_timesteps 5366.
Path 524 | total_timesteps 5378.
Path 525 | total_timesteps 5394.
Path 526 | total_timesteps 5403.
Path 527 | total_timesteps 5409.
Path 528 | total_timesteps 5420.
Path 529 | total_timesteps 5428.
Path 530 | total_timesteps 5443.
Path 531 | total_timesteps 5453.
Path 532 | total_timesteps 5462.
Path 533 | total_timesteps 5475.
Path 534 | total_timesteps 5481.
Path 535 | total_timesteps 5492.
Path 536 | total_timesteps 5504.
Path 537 | total_timesteps 5514.
Path 538 | total_timesteps 5521.
Path 539 | total_timesteps 5531.
Path 540 | total_timesteps 5541.
Path 541 | total_timesteps 5554.
Path 542 | total_timesteps 5562.
Path 543 | total_timesteps 5573.
Path 544 | total_timesteps 5584.
Path 545 | total_timesteps 5592.
Path 546 | total_timesteps 5601.
Path 547 | total_timesteps 5612.
Path 548 | total_timesteps 5630.
Path 549 | total_timesteps 5644.
Path 550 | total_timesteps 5655.
Path 551 | total_timesteps 5663.
Path 552 | total_timesteps 5674.
Path 553 | total_timesteps 5691.
Path 554 | total_timesteps 5702.
Path 555 | total_timesteps 5711.
Path 556 | total_timesteps 5720.
Path 557 | total_timesteps 5729.
Path 558 | total_timesteps 5735.
Path 559 | total_timesteps 5743.
Path 560 | total_timesteps 5753.
Path 561 | total_timesteps 5769.
Path 562 | total_timesteps 5780.
Path 563 | total_timesteps 5790.
Path 564 | total_timesteps 5804.
Path 565 | total_timesteps 5812.
Path 566 | total_timesteps 5823.
Path 567 | total_timesteps 5835.
Path 568 | total_timesteps 5846.
Path 569 | total_timesteps 5855.
Path 570 | total_timesteps 5868.
Path 571 | total_timesteps 5883.
Path 572 | total_timesteps 5893.
Path 573 | total_timesteps 5902.
Path 574 | total_timesteps 5909.
Path 575 | total_timesteps 5917.
Path 576 | total_timesteps 5929.
Path 577 | total_timesteps 5940.
Path 578 | total_timesteps 5949.
Path 579 | total_timesteps 5957.
Path 580 | total_timesteps 5969.
Path 581 | total_timesteps 5978.
Path 582 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.86    |
| Iteration     | 7        |
| MaximumReturn | 0.172    |
| MinimumReturn | -18.6    |
| TotalSamples  | 36020    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.3011381924152374
Validation loss = 0.3038583993911743
Validation loss = 0.3075394630432129
Validation loss = 0.32588884234428406
Validation loss = 0.32335636019706726
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 29.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 50.
Path 5 | total_timesteps 64.
Path 6 | total_timesteps 76.
Path 7 | total_timesteps 89.
Path 8 | total_timesteps 98.
Path 9 | total_timesteps 110.
Path 10 | total_timesteps 121.
Path 11 | total_timesteps 132.
Path 12 | total_timesteps 143.
Path 13 | total_timesteps 152.
Path 14 | total_timesteps 163.
Path 15 | total_timesteps 172.
Path 16 | total_timesteps 183.
Path 17 | total_timesteps 194.
Path 18 | total_timesteps 206.
Path 19 | total_timesteps 214.
Path 20 | total_timesteps 221.
Path 21 | total_timesteps 228.
Path 22 | total_timesteps 240.
Path 23 | total_timesteps 248.
Path 24 | total_timesteps 261.
Path 25 | total_timesteps 271.
Path 26 | total_timesteps 284.
Path 27 | total_timesteps 295.
Path 28 | total_timesteps 307.
Path 29 | total_timesteps 321.
Path 30 | total_timesteps 331.
Path 31 | total_timesteps 341.
Path 32 | total_timesteps 350.
Path 33 | total_timesteps 362.
Path 34 | total_timesteps 370.
Path 35 | total_timesteps 380.
Path 36 | total_timesteps 391.
Path 37 | total_timesteps 400.
Path 38 | total_timesteps 407.
Path 39 | total_timesteps 413.
Path 40 | total_timesteps 420.
Path 41 | total_timesteps 428.
Path 42 | total_timesteps 437.
Path 43 | total_timesteps 451.
Path 44 | total_timesteps 460.
Path 45 | total_timesteps 471.
Path 46 | total_timesteps 491.
Path 47 | total_timesteps 504.
Path 48 | total_timesteps 514.
Path 49 | total_timesteps 524.
Path 50 | total_timesteps 537.
Path 51 | total_timesteps 547.
Path 52 | total_timesteps 567.
Path 53 | total_timesteps 577.
Path 54 | total_timesteps 588.
Path 55 | total_timesteps 598.
Path 56 | total_timesteps 608.
Path 57 | total_timesteps 621.
Path 58 | total_timesteps 632.
Path 59 | total_timesteps 652.
Path 60 | total_timesteps 662.
Path 61 | total_timesteps 672.
Path 62 | total_timesteps 691.
Path 63 | total_timesteps 707.
Path 64 | total_timesteps 718.
Path 65 | total_timesteps 727.
Path 66 | total_timesteps 735.
Path 67 | total_timesteps 744.
Path 68 | total_timesteps 755.
Path 69 | total_timesteps 766.
Path 70 | total_timesteps 779.
Path 71 | total_timesteps 789.
Path 72 | total_timesteps 798.
Path 73 | total_timesteps 808.
Path 74 | total_timesteps 820.
Path 75 | total_timesteps 828.
Path 76 | total_timesteps 839.
Path 77 | total_timesteps 847.
Path 78 | total_timesteps 859.
Path 79 | total_timesteps 869.
Path 80 | total_timesteps 882.
Path 81 | total_timesteps 890.
Path 82 | total_timesteps 898.
Path 83 | total_timesteps 905.
Path 84 | total_timesteps 914.
Path 85 | total_timesteps 924.
Path 86 | total_timesteps 933.
Path 87 | total_timesteps 942.
Path 88 | total_timesteps 950.
Path 89 | total_timesteps 964.
Path 90 | total_timesteps 976.
Path 91 | total_timesteps 987.
Path 92 | total_timesteps 999.
Path 93 | total_timesteps 1010.
Path 94 | total_timesteps 1021.
Path 95 | total_timesteps 1032.
Path 96 | total_timesteps 1046.
Path 97 | total_timesteps 1060.
Path 98 | total_timesteps 1069.
Path 99 | total_timesteps 1079.
Path 100 | total_timesteps 1087.
Path 101 | total_timesteps 1094.
Path 102 | total_timesteps 1104.
Path 103 | total_timesteps 1113.
Path 104 | total_timesteps 1126.
Path 105 | total_timesteps 1137.
Path 106 | total_timesteps 1147.
Path 107 | total_timesteps 1159.
Path 108 | total_timesteps 1173.
Path 109 | total_timesteps 1186.
Path 110 | total_timesteps 1194.
Path 111 | total_timesteps 1202.
Path 112 | total_timesteps 1213.
Path 113 | total_timesteps 1224.
Path 114 | total_timesteps 1235.
Path 115 | total_timesteps 1244.
Path 116 | total_timesteps 1254.
Path 117 | total_timesteps 1265.
Path 118 | total_timesteps 1280.
Path 119 | total_timesteps 1290.
Path 120 | total_timesteps 1297.
Path 121 | total_timesteps 1305.
Path 122 | total_timesteps 1314.
Path 123 | total_timesteps 1323.
Path 124 | total_timesteps 1331.
Path 125 | total_timesteps 1343.
Path 126 | total_timesteps 1355.
Path 127 | total_timesteps 1368.
Path 128 | total_timesteps 1377.
Path 129 | total_timesteps 1399.
Path 130 | total_timesteps 1410.
Path 131 | total_timesteps 1427.
Path 132 | total_timesteps 1436.
Path 133 | total_timesteps 1451.
Path 134 | total_timesteps 1458.
Path 135 | total_timesteps 1469.
Path 136 | total_timesteps 1478.
Path 137 | total_timesteps 1486.
Path 138 | total_timesteps 1495.
Path 139 | total_timesteps 1505.
Path 140 | total_timesteps 1512.
Path 141 | total_timesteps 1520.
Path 142 | total_timesteps 1530.
Path 143 | total_timesteps 1539.
Path 144 | total_timesteps 1548.
Path 145 | total_timesteps 1558.
Path 146 | total_timesteps 1567.
Path 147 | total_timesteps 1575.
Path 148 | total_timesteps 1585.
Path 149 | total_timesteps 1594.
Path 150 | total_timesteps 1601.
Path 151 | total_timesteps 1610.
Path 152 | total_timesteps 1616.
Path 153 | total_timesteps 1624.
Path 154 | total_timesteps 1632.
Path 155 | total_timesteps 1641.
Path 156 | total_timesteps 1652.
Path 157 | total_timesteps 1661.
Path 158 | total_timesteps 1670.
Path 159 | total_timesteps 1680.
Path 160 | total_timesteps 1690.
Path 161 | total_timesteps 1700.
Path 162 | total_timesteps 1716.
Path 163 | total_timesteps 1726.
Path 164 | total_timesteps 1736.
Path 165 | total_timesteps 1742.
Path 166 | total_timesteps 1751.
Path 167 | total_timesteps 1759.
Path 168 | total_timesteps 1768.
Path 169 | total_timesteps 1778.
Path 170 | total_timesteps 1796.
Path 171 | total_timesteps 1808.
Path 172 | total_timesteps 1818.
Path 173 | total_timesteps 1825.
Path 174 | total_timesteps 1834.
Path 175 | total_timesteps 1845.
Path 176 | total_timesteps 1853.
Path 177 | total_timesteps 1862.
Path 178 | total_timesteps 1872.
Path 179 | total_timesteps 1881.
Path 180 | total_timesteps 1891.
Path 181 | total_timesteps 1898.
Path 182 | total_timesteps 1910.
Path 183 | total_timesteps 1917.
Path 184 | total_timesteps 1932.
Path 185 | total_timesteps 1942.
Path 186 | total_timesteps 1953.
Path 187 | total_timesteps 1963.
Path 188 | total_timesteps 1975.
Path 189 | total_timesteps 1995.
Path 190 | total_timesteps 2002.
Path 191 | total_timesteps 2010.
Path 192 | total_timesteps 2018.
Path 193 | total_timesteps 2027.
Path 194 | total_timesteps 2039.
Path 195 | total_timesteps 2048.
Path 196 | total_timesteps 2059.
Path 197 | total_timesteps 2066.
Path 198 | total_timesteps 2077.
Path 199 | total_timesteps 2090.
Path 200 | total_timesteps 2099.
Path 201 | total_timesteps 2107.
Path 202 | total_timesteps 2118.
Path 203 | total_timesteps 2126.
Path 204 | total_timesteps 2135.
Path 205 | total_timesteps 2143.
Path 206 | total_timesteps 2152.
Path 207 | total_timesteps 2160.
Path 208 | total_timesteps 2171.
Path 209 | total_timesteps 2188.
Path 210 | total_timesteps 2197.
Path 211 | total_timesteps 2208.
Path 212 | total_timesteps 2215.
Path 213 | total_timesteps 2224.
Path 214 | total_timesteps 2235.
Path 215 | total_timesteps 2248.
Path 216 | total_timesteps 2256.
Path 217 | total_timesteps 2265.
Path 218 | total_timesteps 2273.
Path 219 | total_timesteps 2282.
Path 220 | total_timesteps 2290.
Path 221 | total_timesteps 2306.
Path 222 | total_timesteps 2322.
Path 223 | total_timesteps 2331.
Path 224 | total_timesteps 2345.
Path 225 | total_timesteps 2352.
Path 226 | total_timesteps 2364.
Path 227 | total_timesteps 2372.
Path 228 | total_timesteps 2382.
Path 229 | total_timesteps 2392.
Path 230 | total_timesteps 2404.
Path 231 | total_timesteps 2416.
Path 232 | total_timesteps 2428.
Path 233 | total_timesteps 2443.
Path 234 | total_timesteps 2451.
Path 235 | total_timesteps 2465.
Path 236 | total_timesteps 2474.
Path 237 | total_timesteps 2485.
Path 238 | total_timesteps 2493.
Path 239 | total_timesteps 2505.
Path 240 | total_timesteps 2517.
Path 241 | total_timesteps 2532.
Path 242 | total_timesteps 2539.
Path 243 | total_timesteps 2546.
Path 244 | total_timesteps 2553.
Path 245 | total_timesteps 2565.
Path 246 | total_timesteps 2572.
Path 247 | total_timesteps 2582.
Path 248 | total_timesteps 2591.
Path 249 | total_timesteps 2600.
Path 250 | total_timesteps 2620.
Path 251 | total_timesteps 2636.
Path 252 | total_timesteps 2645.
Path 253 | total_timesteps 2652.
Path 254 | total_timesteps 2660.
Path 255 | total_timesteps 2668.
Path 256 | total_timesteps 2682.
Path 257 | total_timesteps 2692.
Path 258 | total_timesteps 2701.
Path 259 | total_timesteps 2712.
Path 260 | total_timesteps 2724.
Path 261 | total_timesteps 2737.
Path 262 | total_timesteps 2751.
Path 263 | total_timesteps 2760.
Path 264 | total_timesteps 2769.
Path 265 | total_timesteps 2780.
Path 266 | total_timesteps 2789.
Path 267 | total_timesteps 2799.
Path 268 | total_timesteps 2811.
Path 269 | total_timesteps 2824.
Path 270 | total_timesteps 2832.
Path 271 | total_timesteps 2840.
Path 272 | total_timesteps 2857.
Path 273 | total_timesteps 2867.
Path 274 | total_timesteps 2877.
Path 275 | total_timesteps 2884.
Path 276 | total_timesteps 2899.
Path 277 | total_timesteps 2918.
Path 278 | total_timesteps 2927.
Path 279 | total_timesteps 2946.
Path 280 | total_timesteps 2956.
Path 281 | total_timesteps 2965.
Path 282 | total_timesteps 2975.
Path 283 | total_timesteps 2989.
Path 284 | total_timesteps 3011.
Path 285 | total_timesteps 3020.
Path 286 | total_timesteps 3027.
Path 287 | total_timesteps 3041.
Path 288 | total_timesteps 3055.
Path 289 | total_timesteps 3067.
Path 290 | total_timesteps 3077.
Path 291 | total_timesteps 3086.
Path 292 | total_timesteps 3095.
Path 293 | total_timesteps 3105.
Path 294 | total_timesteps 3114.
Path 295 | total_timesteps 3123.
Path 296 | total_timesteps 3137.
Path 297 | total_timesteps 3151.
Path 298 | total_timesteps 3164.
Path 299 | total_timesteps 3174.
Path 300 | total_timesteps 3183.
Path 301 | total_timesteps 3191.
Path 302 | total_timesteps 3200.
Path 303 | total_timesteps 3217.
Path 304 | total_timesteps 3224.
Path 305 | total_timesteps 3238.
Path 306 | total_timesteps 3253.
Path 307 | total_timesteps 3268.
Path 308 | total_timesteps 3284.
Path 309 | total_timesteps 3291.
Path 310 | total_timesteps 3300.
Path 311 | total_timesteps 3309.
Path 312 | total_timesteps 3323.
Path 313 | total_timesteps 3335.
Path 314 | total_timesteps 3343.
Path 315 | total_timesteps 3355.
Path 316 | total_timesteps 3367.
Path 317 | total_timesteps 3377.
Path 318 | total_timesteps 3387.
Path 319 | total_timesteps 3410.
Path 320 | total_timesteps 3418.
Path 321 | total_timesteps 3430.
Path 322 | total_timesteps 3441.
Path 323 | total_timesteps 3448.
Path 324 | total_timesteps 3462.
Path 325 | total_timesteps 3474.
Path 326 | total_timesteps 3485.
Path 327 | total_timesteps 3498.
Path 328 | total_timesteps 3514.
Path 329 | total_timesteps 3523.
Path 330 | total_timesteps 3533.
Path 331 | total_timesteps 3541.
Path 332 | total_timesteps 3550.
Path 333 | total_timesteps 3562.
Path 334 | total_timesteps 3572.
Path 335 | total_timesteps 3584.
Path 336 | total_timesteps 3597.
Path 337 | total_timesteps 3610.
Path 338 | total_timesteps 3619.
Path 339 | total_timesteps 3629.
Path 340 | total_timesteps 3637.
Path 341 | total_timesteps 3647.
Path 342 | total_timesteps 3656.
Path 343 | total_timesteps 3667.
Path 344 | total_timesteps 3679.
Path 345 | total_timesteps 3691.
Path 346 | total_timesteps 3700.
Path 347 | total_timesteps 3709.
Path 348 | total_timesteps 3719.
Path 349 | total_timesteps 3731.
Path 350 | total_timesteps 3742.
Path 351 | total_timesteps 3755.
Path 352 | total_timesteps 3767.
Path 353 | total_timesteps 3775.
Path 354 | total_timesteps 3783.
Path 355 | total_timesteps 3792.
Path 356 | total_timesteps 3800.
Path 357 | total_timesteps 3808.
Path 358 | total_timesteps 3820.
Path 359 | total_timesteps 3828.
Path 360 | total_timesteps 3838.
Path 361 | total_timesteps 3847.
Path 362 | total_timesteps 3859.
Path 363 | total_timesteps 3867.
Path 364 | total_timesteps 3876.
Path 365 | total_timesteps 3887.
Path 366 | total_timesteps 3899.
Path 367 | total_timesteps 3907.
Path 368 | total_timesteps 3917.
Path 369 | total_timesteps 3927.
Path 370 | total_timesteps 3940.
Path 371 | total_timesteps 3949.
Path 372 | total_timesteps 3960.
Path 373 | total_timesteps 3983.
Path 374 | total_timesteps 3992.
Path 375 | total_timesteps 4001.
Path 376 | total_timesteps 4009.
Path 377 | total_timesteps 4019.
Path 378 | total_timesteps 4032.
Path 379 | total_timesteps 4040.
Path 380 | total_timesteps 4051.
Path 381 | total_timesteps 4060.
Path 382 | total_timesteps 4067.
Path 383 | total_timesteps 4077.
Path 384 | total_timesteps 4087.
Path 385 | total_timesteps 4096.
Path 386 | total_timesteps 4107.
Path 387 | total_timesteps 4124.
Path 388 | total_timesteps 4132.
Path 389 | total_timesteps 4145.
Path 390 | total_timesteps 4154.
Path 391 | total_timesteps 4164.
Path 392 | total_timesteps 4182.
Path 393 | total_timesteps 4190.
Path 394 | total_timesteps 4200.
Path 395 | total_timesteps 4213.
Path 396 | total_timesteps 4224.
Path 397 | total_timesteps 4239.
Path 398 | total_timesteps 4246.
Path 399 | total_timesteps 4258.
Path 400 | total_timesteps 4267.
Path 401 | total_timesteps 4277.
Path 402 | total_timesteps 4292.
Path 403 | total_timesteps 4304.
Path 404 | total_timesteps 4314.
Path 405 | total_timesteps 4324.
Path 406 | total_timesteps 4333.
Path 407 | total_timesteps 4340.
Path 408 | total_timesteps 4357.
Path 409 | total_timesteps 4371.
Path 410 | total_timesteps 4380.
Path 411 | total_timesteps 4389.
Path 412 | total_timesteps 4398.
Path 413 | total_timesteps 4407.
Path 414 | total_timesteps 4414.
Path 415 | total_timesteps 4424.
Path 416 | total_timesteps 4431.
Path 417 | total_timesteps 4441.
Path 418 | total_timesteps 4452.
Path 419 | total_timesteps 4471.
Path 420 | total_timesteps 4481.
Path 421 | total_timesteps 4492.
Path 422 | total_timesteps 4502.
Path 423 | total_timesteps 4512.
Path 424 | total_timesteps 4525.
Path 425 | total_timesteps 4535.
Path 426 | total_timesteps 4544.
Path 427 | total_timesteps 4554.
Path 428 | total_timesteps 4565.
Path 429 | total_timesteps 4574.
Path 430 | total_timesteps 4588.
Path 431 | total_timesteps 4595.
Path 432 | total_timesteps 4603.
Path 433 | total_timesteps 4612.
Path 434 | total_timesteps 4620.
Path 435 | total_timesteps 4629.
Path 436 | total_timesteps 4639.
Path 437 | total_timesteps 4647.
Path 438 | total_timesteps 4655.
Path 439 | total_timesteps 4662.
Path 440 | total_timesteps 4679.
Path 441 | total_timesteps 4687.
Path 442 | total_timesteps 4696.
Path 443 | total_timesteps 4706.
Path 444 | total_timesteps 4716.
Path 445 | total_timesteps 4725.
Path 446 | total_timesteps 4735.
Path 447 | total_timesteps 4744.
Path 448 | total_timesteps 4753.
Path 449 | total_timesteps 4766.
Path 450 | total_timesteps 4789.
Path 451 | total_timesteps 4799.
Path 452 | total_timesteps 4805.
Path 453 | total_timesteps 4824.
Path 454 | total_timesteps 4836.
Path 455 | total_timesteps 4848.
Path 456 | total_timesteps 4856.
Path 457 | total_timesteps 4865.
Path 458 | total_timesteps 4881.
Path 459 | total_timesteps 4894.
Path 460 | total_timesteps 4912.
Path 461 | total_timesteps 4922.
Path 462 | total_timesteps 4935.
Path 463 | total_timesteps 4941.
Path 464 | total_timesteps 4951.
Path 465 | total_timesteps 4965.
Path 466 | total_timesteps 4976.
Path 467 | total_timesteps 4988.
Path 468 | total_timesteps 5004.
Path 469 | total_timesteps 5017.
Path 470 | total_timesteps 5025.
Path 471 | total_timesteps 5036.
Path 472 | total_timesteps 5044.
Path 473 | total_timesteps 5059.
Path 474 | total_timesteps 5066.
Path 475 | total_timesteps 5075.
Path 476 | total_timesteps 5086.
Path 477 | total_timesteps 5094.
Path 478 | total_timesteps 5105.
Path 479 | total_timesteps 5120.
Path 480 | total_timesteps 5130.
Path 481 | total_timesteps 5137.
Path 482 | total_timesteps 5144.
Path 483 | total_timesteps 5154.
Path 484 | total_timesteps 5167.
Path 485 | total_timesteps 5180.
Path 486 | total_timesteps 5190.
Path 487 | total_timesteps 5201.
Path 488 | total_timesteps 5209.
Path 489 | total_timesteps 5216.
Path 490 | total_timesteps 5227.
Path 491 | total_timesteps 5244.
Path 492 | total_timesteps 5255.
Path 493 | total_timesteps 5265.
Path 494 | total_timesteps 5278.
Path 495 | total_timesteps 5287.
Path 496 | total_timesteps 5296.
Path 497 | total_timesteps 5312.
Path 498 | total_timesteps 5331.
Path 499 | total_timesteps 5343.
Path 500 | total_timesteps 5353.
Path 501 | total_timesteps 5361.
Path 502 | total_timesteps 5370.
Path 503 | total_timesteps 5376.
Path 504 | total_timesteps 5385.
Path 505 | total_timesteps 5396.
Path 506 | total_timesteps 5405.
Path 507 | total_timesteps 5419.
Path 508 | total_timesteps 5429.
Path 509 | total_timesteps 5438.
Path 510 | total_timesteps 5444.
Path 511 | total_timesteps 5453.
Path 512 | total_timesteps 5468.
Path 513 | total_timesteps 5479.
Path 514 | total_timesteps 5487.
Path 515 | total_timesteps 5501.
Path 516 | total_timesteps 5509.
Path 517 | total_timesteps 5519.
Path 518 | total_timesteps 5537.
Path 519 | total_timesteps 5553.
Path 520 | total_timesteps 5562.
Path 521 | total_timesteps 5573.
Path 522 | total_timesteps 5586.
Path 523 | total_timesteps 5595.
Path 524 | total_timesteps 5611.
Path 525 | total_timesteps 5625.
Path 526 | total_timesteps 5635.
Path 527 | total_timesteps 5646.
Path 528 | total_timesteps 5656.
Path 529 | total_timesteps 5665.
Path 530 | total_timesteps 5673.
Path 531 | total_timesteps 5681.
Path 532 | total_timesteps 5692.
Path 533 | total_timesteps 5701.
Path 534 | total_timesteps 5716.
Path 535 | total_timesteps 5725.
Path 536 | total_timesteps 5741.
Path 537 | total_timesteps 5747.
Path 538 | total_timesteps 5757.
Path 539 | total_timesteps 5763.
Path 540 | total_timesteps 5772.
Path 541 | total_timesteps 5781.
Path 542 | total_timesteps 5791.
Path 543 | total_timesteps 5803.
Path 544 | total_timesteps 5816.
Path 545 | total_timesteps 5823.
Path 546 | total_timesteps 5837.
Path 547 | total_timesteps 5851.
Path 548 | total_timesteps 5859.
Path 549 | total_timesteps 5870.
Path 550 | total_timesteps 5881.
Path 551 | total_timesteps 5890.
Path 552 | total_timesteps 5899.
Path 553 | total_timesteps 5905.
Path 554 | total_timesteps 5916.
Path 555 | total_timesteps 5928.
Path 556 | total_timesteps 5946.
Path 557 | total_timesteps 5954.
Path 558 | total_timesteps 5967.
Path 559 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.91    |
| Iteration     | 8        |
| MaximumReturn | 1.61     |
| MinimumReturn | -18.4    |
| TotalSamples  | 40020    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.3017313480377197
Validation loss = 0.307952880859375
Validation loss = 0.313886821269989
Validation loss = 0.32063204050064087
Validation loss = 0.31930720806121826
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 45.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 72.
Path 6 | total_timesteps 83.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 120.
Path 10 | total_timesteps 129.
Path 11 | total_timesteps 136.
Path 12 | total_timesteps 146.
Path 13 | total_timesteps 166.
Path 14 | total_timesteps 181.
Path 15 | total_timesteps 189.
Path 16 | total_timesteps 198.
Path 17 | total_timesteps 210.
Path 18 | total_timesteps 224.
Path 19 | total_timesteps 236.
Path 20 | total_timesteps 251.
Path 21 | total_timesteps 259.
Path 22 | total_timesteps 266.
Path 23 | total_timesteps 275.
Path 24 | total_timesteps 284.
Path 25 | total_timesteps 292.
Path 26 | total_timesteps 303.
Path 27 | total_timesteps 314.
Path 28 | total_timesteps 326.
Path 29 | total_timesteps 335.
Path 30 | total_timesteps 352.
Path 31 | total_timesteps 363.
Path 32 | total_timesteps 372.
Path 33 | total_timesteps 383.
Path 34 | total_timesteps 407.
Path 35 | total_timesteps 420.
Path 36 | total_timesteps 431.
Path 37 | total_timesteps 449.
Path 38 | total_timesteps 459.
Path 39 | total_timesteps 469.
Path 40 | total_timesteps 478.
Path 41 | total_timesteps 495.
Path 42 | total_timesteps 507.
Path 43 | total_timesteps 519.
Path 44 | total_timesteps 528.
Path 45 | total_timesteps 540.
Path 46 | total_timesteps 556.
Path 47 | total_timesteps 564.
Path 48 | total_timesteps 576.
Path 49 | total_timesteps 584.
Path 50 | total_timesteps 591.
Path 51 | total_timesteps 602.
Path 52 | total_timesteps 610.
Path 53 | total_timesteps 618.
Path 54 | total_timesteps 634.
Path 55 | total_timesteps 642.
Path 56 | total_timesteps 652.
Path 57 | total_timesteps 662.
Path 58 | total_timesteps 674.
Path 59 | total_timesteps 683.
Path 60 | total_timesteps 693.
Path 61 | total_timesteps 703.
Path 62 | total_timesteps 713.
Path 63 | total_timesteps 721.
Path 64 | total_timesteps 732.
Path 65 | total_timesteps 743.
Path 66 | total_timesteps 755.
Path 67 | total_timesteps 767.
Path 68 | total_timesteps 781.
Path 69 | total_timesteps 791.
Path 70 | total_timesteps 801.
Path 71 | total_timesteps 809.
Path 72 | total_timesteps 826.
Path 73 | total_timesteps 836.
Path 74 | total_timesteps 847.
Path 75 | total_timesteps 861.
Path 76 | total_timesteps 871.
Path 77 | total_timesteps 883.
Path 78 | total_timesteps 895.
Path 79 | total_timesteps 903.
Path 80 | total_timesteps 916.
Path 81 | total_timesteps 942.
Path 82 | total_timesteps 956.
Path 83 | total_timesteps 972.
Path 84 | total_timesteps 984.
Path 85 | total_timesteps 998.
Path 86 | total_timesteps 1010.
Path 87 | total_timesteps 1029.
Path 88 | total_timesteps 1039.
Path 89 | total_timesteps 1052.
Path 90 | total_timesteps 1065.
Path 91 | total_timesteps 1074.
Path 92 | total_timesteps 1087.
Path 93 | total_timesteps 1096.
Path 94 | total_timesteps 1105.
Path 95 | total_timesteps 1113.
Path 96 | total_timesteps 1121.
Path 97 | total_timesteps 1130.
Path 98 | total_timesteps 1140.
Path 99 | total_timesteps 1150.
Path 100 | total_timesteps 1159.
Path 101 | total_timesteps 1175.
Path 102 | total_timesteps 1188.
Path 103 | total_timesteps 1198.
Path 104 | total_timesteps 1205.
Path 105 | total_timesteps 1218.
Path 106 | total_timesteps 1226.
Path 107 | total_timesteps 1237.
Path 108 | total_timesteps 1247.
Path 109 | total_timesteps 1262.
Path 110 | total_timesteps 1274.
Path 111 | total_timesteps 1283.
Path 112 | total_timesteps 1292.
Path 113 | total_timesteps 1301.
Path 114 | total_timesteps 1314.
Path 115 | total_timesteps 1323.
Path 116 | total_timesteps 1336.
Path 117 | total_timesteps 1346.
Path 118 | total_timesteps 1354.
Path 119 | total_timesteps 1364.
Path 120 | total_timesteps 1373.
Path 121 | total_timesteps 1385.
Path 122 | total_timesteps 1392.
Path 123 | total_timesteps 1406.
Path 124 | total_timesteps 1417.
Path 125 | total_timesteps 1426.
Path 126 | total_timesteps 1439.
Path 127 | total_timesteps 1449.
Path 128 | total_timesteps 1460.
Path 129 | total_timesteps 1469.
Path 130 | total_timesteps 1477.
Path 131 | total_timesteps 1488.
Path 132 | total_timesteps 1498.
Path 133 | total_timesteps 1507.
Path 134 | total_timesteps 1517.
Path 135 | total_timesteps 1525.
Path 136 | total_timesteps 1534.
Path 137 | total_timesteps 1546.
Path 138 | total_timesteps 1554.
Path 139 | total_timesteps 1566.
Path 140 | total_timesteps 1582.
Path 141 | total_timesteps 1596.
Path 142 | total_timesteps 1608.
Path 143 | total_timesteps 1619.
Path 144 | total_timesteps 1630.
Path 145 | total_timesteps 1651.
Path 146 | total_timesteps 1662.
Path 147 | total_timesteps 1670.
Path 148 | total_timesteps 1683.
Path 149 | total_timesteps 1695.
Path 150 | total_timesteps 1705.
Path 151 | total_timesteps 1714.
Path 152 | total_timesteps 1723.
Path 153 | total_timesteps 1736.
Path 154 | total_timesteps 1748.
Path 155 | total_timesteps 1762.
Path 156 | total_timesteps 1769.
Path 157 | total_timesteps 1777.
Path 158 | total_timesteps 1785.
Path 159 | total_timesteps 1797.
Path 160 | total_timesteps 1808.
Path 161 | total_timesteps 1817.
Path 162 | total_timesteps 1834.
Path 163 | total_timesteps 1846.
Path 164 | total_timesteps 1853.
Path 165 | total_timesteps 1870.
Path 166 | total_timesteps 1884.
Path 167 | total_timesteps 1893.
Path 168 | total_timesteps 1905.
Path 169 | total_timesteps 1914.
Path 170 | total_timesteps 1924.
Path 171 | total_timesteps 1935.
Path 172 | total_timesteps 1946.
Path 173 | total_timesteps 1957.
Path 174 | total_timesteps 1966.
Path 175 | total_timesteps 1977.
Path 176 | total_timesteps 1989.
Path 177 | total_timesteps 1998.
Path 178 | total_timesteps 2009.
Path 179 | total_timesteps 2016.
Path 180 | total_timesteps 2030.
Path 181 | total_timesteps 2039.
Path 182 | total_timesteps 2052.
Path 183 | total_timesteps 2061.
Path 184 | total_timesteps 2072.
Path 185 | total_timesteps 2094.
Path 186 | total_timesteps 2106.
Path 187 | total_timesteps 2118.
Path 188 | total_timesteps 2126.
Path 189 | total_timesteps 2137.
Path 190 | total_timesteps 2148.
Path 191 | total_timesteps 2161.
Path 192 | total_timesteps 2171.
Path 193 | total_timesteps 2195.
Path 194 | total_timesteps 2204.
Path 195 | total_timesteps 2238.
Path 196 | total_timesteps 2247.
Path 197 | total_timesteps 2255.
Path 198 | total_timesteps 2266.
Path 199 | total_timesteps 2287.
Path 200 | total_timesteps 2299.
Path 201 | total_timesteps 2309.
Path 202 | total_timesteps 2319.
Path 203 | total_timesteps 2328.
Path 204 | total_timesteps 2338.
Path 205 | total_timesteps 2349.
Path 206 | total_timesteps 2358.
Path 207 | total_timesteps 2366.
Path 208 | total_timesteps 2375.
Path 209 | total_timesteps 2385.
Path 210 | total_timesteps 2401.
Path 211 | total_timesteps 2414.
Path 212 | total_timesteps 2427.
Path 213 | total_timesteps 2436.
Path 214 | total_timesteps 2447.
Path 215 | total_timesteps 2455.
Path 216 | total_timesteps 2465.
Path 217 | total_timesteps 2484.
Path 218 | total_timesteps 2496.
Path 219 | total_timesteps 2505.
Path 220 | total_timesteps 2516.
Path 221 | total_timesteps 2527.
Path 222 | total_timesteps 2539.
Path 223 | total_timesteps 2550.
Path 224 | total_timesteps 2561.
Path 225 | total_timesteps 2567.
Path 226 | total_timesteps 2582.
Path 227 | total_timesteps 2590.
Path 228 | total_timesteps 2598.
Path 229 | total_timesteps 2611.
Path 230 | total_timesteps 2621.
Path 231 | total_timesteps 2634.
Path 232 | total_timesteps 2646.
Path 233 | total_timesteps 2654.
Path 234 | total_timesteps 2666.
Path 235 | total_timesteps 2681.
Path 236 | total_timesteps 2696.
Path 237 | total_timesteps 2707.
Path 238 | total_timesteps 2715.
Path 239 | total_timesteps 2724.
Path 240 | total_timesteps 2732.
Path 241 | total_timesteps 2744.
Path 242 | total_timesteps 2753.
Path 243 | total_timesteps 2761.
Path 244 | total_timesteps 2773.
Path 245 | total_timesteps 2789.
Path 246 | total_timesteps 2798.
Path 247 | total_timesteps 2806.
Path 248 | total_timesteps 2813.
Path 249 | total_timesteps 2825.
Path 250 | total_timesteps 2840.
Path 251 | total_timesteps 2850.
Path 252 | total_timesteps 2861.
Path 253 | total_timesteps 2869.
Path 254 | total_timesteps 2876.
Path 255 | total_timesteps 2888.
Path 256 | total_timesteps 2902.
Path 257 | total_timesteps 2909.
Path 258 | total_timesteps 2917.
Path 259 | total_timesteps 2929.
Path 260 | total_timesteps 2937.
Path 261 | total_timesteps 2951.
Path 262 | total_timesteps 2961.
Path 263 | total_timesteps 2971.
Path 264 | total_timesteps 2993.
Path 265 | total_timesteps 3002.
Path 266 | total_timesteps 3012.
Path 267 | total_timesteps 3020.
Path 268 | total_timesteps 3028.
Path 269 | total_timesteps 3036.
Path 270 | total_timesteps 3047.
Path 271 | total_timesteps 3057.
Path 272 | total_timesteps 3067.
Path 273 | total_timesteps 3097.
Path 274 | total_timesteps 3115.
Path 275 | total_timesteps 3129.
Path 276 | total_timesteps 3137.
Path 277 | total_timesteps 3147.
Path 278 | total_timesteps 3156.
Path 279 | total_timesteps 3170.
Path 280 | total_timesteps 3180.
Path 281 | total_timesteps 3194.
Path 282 | total_timesteps 3205.
Path 283 | total_timesteps 3215.
Path 284 | total_timesteps 3224.
Path 285 | total_timesteps 3238.
Path 286 | total_timesteps 3254.
Path 287 | total_timesteps 3262.
Path 288 | total_timesteps 3274.
Path 289 | total_timesteps 3286.
Path 290 | total_timesteps 3305.
Path 291 | total_timesteps 3317.
Path 292 | total_timesteps 3327.
Path 293 | total_timesteps 3335.
Path 294 | total_timesteps 3345.
Path 295 | total_timesteps 3354.
Path 296 | total_timesteps 3364.
Path 297 | total_timesteps 3376.
Path 298 | total_timesteps 3393.
Path 299 | total_timesteps 3405.
Path 300 | total_timesteps 3416.
Path 301 | total_timesteps 3427.
Path 302 | total_timesteps 3434.
Path 303 | total_timesteps 3443.
Path 304 | total_timesteps 3453.
Path 305 | total_timesteps 3463.
Path 306 | total_timesteps 3472.
Path 307 | total_timesteps 3483.
Path 308 | total_timesteps 3492.
Path 309 | total_timesteps 3502.
Path 310 | total_timesteps 3509.
Path 311 | total_timesteps 3519.
Path 312 | total_timesteps 3530.
Path 313 | total_timesteps 3537.
Path 314 | total_timesteps 3550.
Path 315 | total_timesteps 3569.
Path 316 | total_timesteps 3578.
Path 317 | total_timesteps 3589.
Path 318 | total_timesteps 3612.
Path 319 | total_timesteps 3623.
Path 320 | total_timesteps 3633.
Path 321 | total_timesteps 3650.
Path 322 | total_timesteps 3658.
Path 323 | total_timesteps 3671.
Path 324 | total_timesteps 3679.
Path 325 | total_timesteps 3694.
Path 326 | total_timesteps 3704.
Path 327 | total_timesteps 3718.
Path 328 | total_timesteps 3727.
Path 329 | total_timesteps 3738.
Path 330 | total_timesteps 3753.
Path 331 | total_timesteps 3766.
Path 332 | total_timesteps 3775.
Path 333 | total_timesteps 3782.
Path 334 | total_timesteps 3791.
Path 335 | total_timesteps 3802.
Path 336 | total_timesteps 3813.
Path 337 | total_timesteps 3823.
Path 338 | total_timesteps 3837.
Path 339 | total_timesteps 3844.
Path 340 | total_timesteps 3858.
Path 341 | total_timesteps 3868.
Path 342 | total_timesteps 3880.
Path 343 | total_timesteps 3897.
Path 344 | total_timesteps 3906.
Path 345 | total_timesteps 3919.
Path 346 | total_timesteps 3929.
Path 347 | total_timesteps 3938.
Path 348 | total_timesteps 3956.
Path 349 | total_timesteps 3967.
Path 350 | total_timesteps 3978.
Path 351 | total_timesteps 3994.
Path 352 | total_timesteps 4002.
Path 353 | total_timesteps 4017.
Path 354 | total_timesteps 4025.
Path 355 | total_timesteps 4035.
Path 356 | total_timesteps 4043.
Path 357 | total_timesteps 4053.
Path 358 | total_timesteps 4065.
Path 359 | total_timesteps 4077.
Path 360 | total_timesteps 4089.
Path 361 | total_timesteps 4097.
Path 362 | total_timesteps 4110.
Path 363 | total_timesteps 4119.
Path 364 | total_timesteps 4126.
Path 365 | total_timesteps 4136.
Path 366 | total_timesteps 4147.
Path 367 | total_timesteps 4157.
Path 368 | total_timesteps 4165.
Path 369 | total_timesteps 4179.
Path 370 | total_timesteps 4189.
Path 371 | total_timesteps 4196.
Path 372 | total_timesteps 4208.
Path 373 | total_timesteps 4220.
Path 374 | total_timesteps 4234.
Path 375 | total_timesteps 4241.
Path 376 | total_timesteps 4255.
Path 377 | total_timesteps 4262.
Path 378 | total_timesteps 4275.
Path 379 | total_timesteps 4282.
Path 380 | total_timesteps 4291.
Path 381 | total_timesteps 4298.
Path 382 | total_timesteps 4321.
Path 383 | total_timesteps 4344.
Path 384 | total_timesteps 4357.
Path 385 | total_timesteps 4368.
Path 386 | total_timesteps 4383.
Path 387 | total_timesteps 4396.
Path 388 | total_timesteps 4408.
Path 389 | total_timesteps 4422.
Path 390 | total_timesteps 4433.
Path 391 | total_timesteps 4444.
Path 392 | total_timesteps 4452.
Path 393 | total_timesteps 4462.
Path 394 | total_timesteps 4471.
Path 395 | total_timesteps 4480.
Path 396 | total_timesteps 4487.
Path 397 | total_timesteps 4496.
Path 398 | total_timesteps 4520.
Path 399 | total_timesteps 4532.
Path 400 | total_timesteps 4540.
Path 401 | total_timesteps 4557.
Path 402 | total_timesteps 4574.
Path 403 | total_timesteps 4587.
Path 404 | total_timesteps 4604.
Path 405 | total_timesteps 4612.
Path 406 | total_timesteps 4621.
Path 407 | total_timesteps 4629.
Path 408 | total_timesteps 4643.
Path 409 | total_timesteps 4651.
Path 410 | total_timesteps 4659.
Path 411 | total_timesteps 4671.
Path 412 | total_timesteps 4684.
Path 413 | total_timesteps 4691.
Path 414 | total_timesteps 4700.
Path 415 | total_timesteps 4709.
Path 416 | total_timesteps 4717.
Path 417 | total_timesteps 4725.
Path 418 | total_timesteps 4737.
Path 419 | total_timesteps 4746.
Path 420 | total_timesteps 4760.
Path 421 | total_timesteps 4771.
Path 422 | total_timesteps 4781.
Path 423 | total_timesteps 4793.
Path 424 | total_timesteps 4811.
Path 425 | total_timesteps 4822.
Path 426 | total_timesteps 4835.
Path 427 | total_timesteps 4852.
Path 428 | total_timesteps 4860.
Path 429 | total_timesteps 4873.
Path 430 | total_timesteps 4886.
Path 431 | total_timesteps 4904.
Path 432 | total_timesteps 4912.
Path 433 | total_timesteps 4919.
Path 434 | total_timesteps 4929.
Path 435 | total_timesteps 4941.
Path 436 | total_timesteps 4956.
Path 437 | total_timesteps 4965.
Path 438 | total_timesteps 4977.
Path 439 | total_timesteps 4986.
Path 440 | total_timesteps 4996.
Path 441 | total_timesteps 5005.
Path 442 | total_timesteps 5015.
Path 443 | total_timesteps 5025.
Path 444 | total_timesteps 5040.
Path 445 | total_timesteps 5052.
Path 446 | total_timesteps 5075.
Path 447 | total_timesteps 5103.
Path 448 | total_timesteps 5116.
Path 449 | total_timesteps 5136.
Path 450 | total_timesteps 5149.
Path 451 | total_timesteps 5159.
Path 452 | total_timesteps 5177.
Path 453 | total_timesteps 5188.
Path 454 | total_timesteps 5197.
Path 455 | total_timesteps 5206.
Path 456 | total_timesteps 5214.
Path 457 | total_timesteps 5226.
Path 458 | total_timesteps 5241.
Path 459 | total_timesteps 5255.
Path 460 | total_timesteps 5265.
Path 461 | total_timesteps 5275.
Path 462 | total_timesteps 5289.
Path 463 | total_timesteps 5299.
Path 464 | total_timesteps 5310.
Path 465 | total_timesteps 5332.
Path 466 | total_timesteps 5347.
Path 467 | total_timesteps 5360.
Path 468 | total_timesteps 5368.
Path 469 | total_timesteps 5376.
Path 470 | total_timesteps 5387.
Path 471 | total_timesteps 5398.
Path 472 | total_timesteps 5408.
Path 473 | total_timesteps 5419.
Path 474 | total_timesteps 5430.
Path 475 | total_timesteps 5442.
Path 476 | total_timesteps 5452.
Path 477 | total_timesteps 5462.
Path 478 | total_timesteps 5474.
Path 479 | total_timesteps 5482.
Path 480 | total_timesteps 5502.
Path 481 | total_timesteps 5511.
Path 482 | total_timesteps 5523.
Path 483 | total_timesteps 5538.
Path 484 | total_timesteps 5546.
Path 485 | total_timesteps 5557.
Path 486 | total_timesteps 5568.
Path 487 | total_timesteps 5576.
Path 488 | total_timesteps 5589.
Path 489 | total_timesteps 5599.
Path 490 | total_timesteps 5607.
Path 491 | total_timesteps 5620.
Path 492 | total_timesteps 5632.
Path 493 | total_timesteps 5647.
Path 494 | total_timesteps 5657.
Path 495 | total_timesteps 5670.
Path 496 | total_timesteps 5684.
Path 497 | total_timesteps 5695.
Path 498 | total_timesteps 5707.
Path 499 | total_timesteps 5723.
Path 500 | total_timesteps 5732.
Path 501 | total_timesteps 5741.
Path 502 | total_timesteps 5758.
Path 503 | total_timesteps 5775.
Path 504 | total_timesteps 5783.
Path 505 | total_timesteps 5795.
Path 506 | total_timesteps 5808.
Path 507 | total_timesteps 5821.
Path 508 | total_timesteps 5832.
Path 509 | total_timesteps 5846.
Path 510 | total_timesteps 5858.
Path 511 | total_timesteps 5868.
Path 512 | total_timesteps 5876.
Path 513 | total_timesteps 5890.
Path 514 | total_timesteps 5900.
Path 515 | total_timesteps 5914.
Path 516 | total_timesteps 5925.
Path 517 | total_timesteps 5936.
Path 518 | total_timesteps 5948.
Path 519 | total_timesteps 5958.
Path 520 | total_timesteps 5969.
Path 521 | total_timesteps 5977.
Path 522 | total_timesteps 5984.
Path 523 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.87    |
| Iteration     | 9        |
| MaximumReturn | 2.87     |
| MinimumReturn | -20.3    |
| TotalSamples  | 44033    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.3058467209339142
Validation loss = 0.31245648860931396
Validation loss = 0.3170605003833771
Validation loss = 0.3194017708301544
Validation loss = 0.31960585713386536
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 46.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 65.
Path 6 | total_timesteps 78.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 102.
Path 9 | total_timesteps 108.
Path 10 | total_timesteps 115.
Path 11 | total_timesteps 139.
Path 12 | total_timesteps 151.
Path 13 | total_timesteps 162.
Path 14 | total_timesteps 176.
Path 15 | total_timesteps 189.
Path 16 | total_timesteps 197.
Path 17 | total_timesteps 206.
Path 18 | total_timesteps 217.
Path 19 | total_timesteps 230.
Path 20 | total_timesteps 244.
Path 21 | total_timesteps 256.
Path 22 | total_timesteps 271.
Path 23 | total_timesteps 280.
Path 24 | total_timesteps 294.
Path 25 | total_timesteps 308.
Path 26 | total_timesteps 317.
Path 27 | total_timesteps 332.
Path 28 | total_timesteps 348.
Path 29 | total_timesteps 355.
Path 30 | total_timesteps 367.
Path 31 | total_timesteps 377.
Path 32 | total_timesteps 389.
Path 33 | total_timesteps 402.
Path 34 | total_timesteps 411.
Path 35 | total_timesteps 419.
Path 36 | total_timesteps 428.
Path 37 | total_timesteps 439.
Path 38 | total_timesteps 448.
Path 39 | total_timesteps 454.
Path 40 | total_timesteps 465.
Path 41 | total_timesteps 475.
Path 42 | total_timesteps 490.
Path 43 | total_timesteps 502.
Path 44 | total_timesteps 512.
Path 45 | total_timesteps 519.
Path 46 | total_timesteps 527.
Path 47 | total_timesteps 536.
Path 48 | total_timesteps 550.
Path 49 | total_timesteps 563.
Path 50 | total_timesteps 574.
Path 51 | total_timesteps 588.
Path 52 | total_timesteps 597.
Path 53 | total_timesteps 610.
Path 54 | total_timesteps 624.
Path 55 | total_timesteps 633.
Path 56 | total_timesteps 641.
Path 57 | total_timesteps 650.
Path 58 | total_timesteps 661.
Path 59 | total_timesteps 679.
Path 60 | total_timesteps 694.
Path 61 | total_timesteps 706.
Path 62 | total_timesteps 720.
Path 63 | total_timesteps 733.
Path 64 | total_timesteps 743.
Path 65 | total_timesteps 754.
Path 66 | total_timesteps 769.
Path 67 | total_timesteps 779.
Path 68 | total_timesteps 788.
Path 69 | total_timesteps 797.
Path 70 | total_timesteps 809.
Path 71 | total_timesteps 823.
Path 72 | total_timesteps 837.
Path 73 | total_timesteps 847.
Path 74 | total_timesteps 857.
Path 75 | total_timesteps 870.
Path 76 | total_timesteps 880.
Path 77 | total_timesteps 889.
Path 78 | total_timesteps 900.
Path 79 | total_timesteps 910.
Path 80 | total_timesteps 917.
Path 81 | total_timesteps 934.
Path 82 | total_timesteps 950.
Path 83 | total_timesteps 960.
Path 84 | total_timesteps 972.
Path 85 | total_timesteps 981.
Path 86 | total_timesteps 995.
Path 87 | total_timesteps 1006.
Path 88 | total_timesteps 1018.
Path 89 | total_timesteps 1035.
Path 90 | total_timesteps 1046.
Path 91 | total_timesteps 1061.
Path 92 | total_timesteps 1076.
Path 93 | total_timesteps 1089.
Path 94 | total_timesteps 1101.
Path 95 | total_timesteps 1111.
Path 96 | total_timesteps 1119.
Path 97 | total_timesteps 1127.
Path 98 | total_timesteps 1140.
Path 99 | total_timesteps 1163.
Path 100 | total_timesteps 1174.
Path 101 | total_timesteps 1192.
Path 102 | total_timesteps 1206.
Path 103 | total_timesteps 1216.
Path 104 | total_timesteps 1225.
Path 105 | total_timesteps 1233.
Path 106 | total_timesteps 1253.
Path 107 | total_timesteps 1265.
Path 108 | total_timesteps 1276.
Path 109 | total_timesteps 1287.
Path 110 | total_timesteps 1296.
Path 111 | total_timesteps 1304.
Path 112 | total_timesteps 1318.
Path 113 | total_timesteps 1332.
Path 114 | total_timesteps 1343.
Path 115 | total_timesteps 1350.
Path 116 | total_timesteps 1357.
Path 117 | total_timesteps 1367.
Path 118 | total_timesteps 1377.
Path 119 | total_timesteps 1384.
Path 120 | total_timesteps 1392.
Path 121 | total_timesteps 1401.
Path 122 | total_timesteps 1424.
Path 123 | total_timesteps 1434.
Path 124 | total_timesteps 1443.
Path 125 | total_timesteps 1450.
Path 126 | total_timesteps 1460.
Path 127 | total_timesteps 1474.
Path 128 | total_timesteps 1489.
Path 129 | total_timesteps 1497.
Path 130 | total_timesteps 1513.
Path 131 | total_timesteps 1526.
Path 132 | total_timesteps 1536.
Path 133 | total_timesteps 1543.
Path 134 | total_timesteps 1552.
Path 135 | total_timesteps 1566.
Path 136 | total_timesteps 1575.
Path 137 | total_timesteps 1584.
Path 138 | total_timesteps 1599.
Path 139 | total_timesteps 1610.
Path 140 | total_timesteps 1618.
Path 141 | total_timesteps 1630.
Path 142 | total_timesteps 1640.
Path 143 | total_timesteps 1651.
Path 144 | total_timesteps 1661.
Path 145 | total_timesteps 1675.
Path 146 | total_timesteps 1684.
Path 147 | total_timesteps 1697.
Path 148 | total_timesteps 1707.
Path 149 | total_timesteps 1716.
Path 150 | total_timesteps 1725.
Path 151 | total_timesteps 1735.
Path 152 | total_timesteps 1744.
Path 153 | total_timesteps 1757.
Path 154 | total_timesteps 1769.
Path 155 | total_timesteps 1777.
Path 156 | total_timesteps 1789.
Path 157 | total_timesteps 1804.
Path 158 | total_timesteps 1813.
Path 159 | total_timesteps 1821.
Path 160 | total_timesteps 1837.
Path 161 | total_timesteps 1849.
Path 162 | total_timesteps 1869.
Path 163 | total_timesteps 1890.
Path 164 | total_timesteps 1898.
Path 165 | total_timesteps 1907.
Path 166 | total_timesteps 1919.
Path 167 | total_timesteps 1939.
Path 168 | total_timesteps 1949.
Path 169 | total_timesteps 1958.
Path 170 | total_timesteps 1971.
Path 171 | total_timesteps 1981.
Path 172 | total_timesteps 1994.
Path 173 | total_timesteps 2001.
Path 174 | total_timesteps 2013.
Path 175 | total_timesteps 2021.
Path 176 | total_timesteps 2031.
Path 177 | total_timesteps 2040.
Path 178 | total_timesteps 2049.
Path 179 | total_timesteps 2059.
Path 180 | total_timesteps 2071.
Path 181 | total_timesteps 2079.
Path 182 | total_timesteps 2090.
Path 183 | total_timesteps 2099.
Path 184 | total_timesteps 2111.
Path 185 | total_timesteps 2121.
Path 186 | total_timesteps 2134.
Path 187 | total_timesteps 2155.
Path 188 | total_timesteps 2174.
Path 189 | total_timesteps 2186.
Path 190 | total_timesteps 2197.
Path 191 | total_timesteps 2208.
Path 192 | total_timesteps 2215.
Path 193 | total_timesteps 2224.
Path 194 | total_timesteps 2238.
Path 195 | total_timesteps 2248.
Path 196 | total_timesteps 2261.
Path 197 | total_timesteps 2278.
Path 198 | total_timesteps 2286.
Path 199 | total_timesteps 2300.
Path 200 | total_timesteps 2308.
Path 201 | total_timesteps 2315.
Path 202 | total_timesteps 2327.
Path 203 | total_timesteps 2341.
Path 204 | total_timesteps 2351.
Path 205 | total_timesteps 2371.
Path 206 | total_timesteps 2382.
Path 207 | total_timesteps 2391.
Path 208 | total_timesteps 2406.
Path 209 | total_timesteps 2415.
Path 210 | total_timesteps 2425.
Path 211 | total_timesteps 2436.
Path 212 | total_timesteps 2449.
Path 213 | total_timesteps 2458.
Path 214 | total_timesteps 2477.
Path 215 | total_timesteps 2489.
Path 216 | total_timesteps 2498.
Path 217 | total_timesteps 2511.
Path 218 | total_timesteps 2521.
Path 219 | total_timesteps 2530.
Path 220 | total_timesteps 2538.
Path 221 | total_timesteps 2551.
Path 222 | total_timesteps 2562.
Path 223 | total_timesteps 2570.
Path 224 | total_timesteps 2585.
Path 225 | total_timesteps 2597.
Path 226 | total_timesteps 2611.
Path 227 | total_timesteps 2621.
Path 228 | total_timesteps 2629.
Path 229 | total_timesteps 2642.
Path 230 | total_timesteps 2651.
Path 231 | total_timesteps 2663.
Path 232 | total_timesteps 2678.
Path 233 | total_timesteps 2689.
Path 234 | total_timesteps 2704.
Path 235 | total_timesteps 2713.
Path 236 | total_timesteps 2723.
Path 237 | total_timesteps 2733.
Path 238 | total_timesteps 2745.
Path 239 | total_timesteps 2753.
Path 240 | total_timesteps 2762.
Path 241 | total_timesteps 2775.
Path 242 | total_timesteps 2788.
Path 243 | total_timesteps 2798.
Path 244 | total_timesteps 2809.
Path 245 | total_timesteps 2819.
Path 246 | total_timesteps 2832.
Path 247 | total_timesteps 2843.
Path 248 | total_timesteps 2852.
Path 249 | total_timesteps 2863.
Path 250 | total_timesteps 2872.
Path 251 | total_timesteps 2883.
Path 252 | total_timesteps 2896.
Path 253 | total_timesteps 2906.
Path 254 | total_timesteps 2917.
Path 255 | total_timesteps 2927.
Path 256 | total_timesteps 2936.
Path 257 | total_timesteps 2950.
Path 258 | total_timesteps 2960.
Path 259 | total_timesteps 2968.
Path 260 | total_timesteps 2980.
Path 261 | total_timesteps 2990.
Path 262 | total_timesteps 3004.
Path 263 | total_timesteps 3012.
Path 264 | total_timesteps 3024.
Path 265 | total_timesteps 3034.
Path 266 | total_timesteps 3046.
Path 267 | total_timesteps 3057.
Path 268 | total_timesteps 3068.
Path 269 | total_timesteps 3079.
Path 270 | total_timesteps 3095.
Path 271 | total_timesteps 3102.
Path 272 | total_timesteps 3113.
Path 273 | total_timesteps 3130.
Path 274 | total_timesteps 3139.
Path 275 | total_timesteps 3149.
Path 276 | total_timesteps 3156.
Path 277 | total_timesteps 3168.
Path 278 | total_timesteps 3176.
Path 279 | total_timesteps 3186.
Path 280 | total_timesteps 3196.
Path 281 | total_timesteps 3205.
Path 282 | total_timesteps 3214.
Path 283 | total_timesteps 3230.
Path 284 | total_timesteps 3238.
Path 285 | total_timesteps 3252.
Path 286 | total_timesteps 3263.
Path 287 | total_timesteps 3273.
Path 288 | total_timesteps 3293.
Path 289 | total_timesteps 3303.
Path 290 | total_timesteps 3313.
Path 291 | total_timesteps 3331.
Path 292 | total_timesteps 3341.
Path 293 | total_timesteps 3354.
Path 294 | total_timesteps 3364.
Path 295 | total_timesteps 3371.
Path 296 | total_timesteps 3383.
Path 297 | total_timesteps 3394.
Path 298 | total_timesteps 3407.
Path 299 | total_timesteps 3429.
Path 300 | total_timesteps 3441.
Path 301 | total_timesteps 3451.
Path 302 | total_timesteps 3463.
Path 303 | total_timesteps 3477.
Path 304 | total_timesteps 3489.
Path 305 | total_timesteps 3498.
Path 306 | total_timesteps 3506.
Path 307 | total_timesteps 3516.
Path 308 | total_timesteps 3528.
Path 309 | total_timesteps 3540.
Path 310 | total_timesteps 3549.
Path 311 | total_timesteps 3558.
Path 312 | total_timesteps 3567.
Path 313 | total_timesteps 3575.
Path 314 | total_timesteps 3586.
Path 315 | total_timesteps 3601.
Path 316 | total_timesteps 3610.
Path 317 | total_timesteps 3624.
Path 318 | total_timesteps 3633.
Path 319 | total_timesteps 3647.
Path 320 | total_timesteps 3659.
Path 321 | total_timesteps 3668.
Path 322 | total_timesteps 3675.
Path 323 | total_timesteps 3685.
Path 324 | total_timesteps 3691.
Path 325 | total_timesteps 3702.
Path 326 | total_timesteps 3714.
Path 327 | total_timesteps 3723.
Path 328 | total_timesteps 3734.
Path 329 | total_timesteps 3741.
Path 330 | total_timesteps 3753.
Path 331 | total_timesteps 3764.
Path 332 | total_timesteps 3774.
Path 333 | total_timesteps 3786.
Path 334 | total_timesteps 3793.
Path 335 | total_timesteps 3801.
Path 336 | total_timesteps 3808.
Path 337 | total_timesteps 3822.
Path 338 | total_timesteps 3839.
Path 339 | total_timesteps 3851.
Path 340 | total_timesteps 3860.
Path 341 | total_timesteps 3870.
Path 342 | total_timesteps 3880.
Path 343 | total_timesteps 3887.
Path 344 | total_timesteps 3897.
Path 345 | total_timesteps 3906.
Path 346 | total_timesteps 3917.
Path 347 | total_timesteps 3934.
Path 348 | total_timesteps 3943.
Path 349 | total_timesteps 3952.
Path 350 | total_timesteps 3962.
Path 351 | total_timesteps 3981.
Path 352 | total_timesteps 3989.
Path 353 | total_timesteps 3997.
Path 354 | total_timesteps 4004.
Path 355 | total_timesteps 4012.
Path 356 | total_timesteps 4021.
Path 357 | total_timesteps 4034.
Path 358 | total_timesteps 4045.
Path 359 | total_timesteps 4054.
Path 360 | total_timesteps 4062.
Path 361 | total_timesteps 4071.
Path 362 | total_timesteps 4080.
Path 363 | total_timesteps 4087.
Path 364 | total_timesteps 4095.
Path 365 | total_timesteps 4109.
Path 366 | total_timesteps 4118.
Path 367 | total_timesteps 4125.
Path 368 | total_timesteps 4135.
Path 369 | total_timesteps 4146.
Path 370 | total_timesteps 4157.
Path 371 | total_timesteps 4167.
Path 372 | total_timesteps 4178.
Path 373 | total_timesteps 4192.
Path 374 | total_timesteps 4208.
Path 375 | total_timesteps 4219.
Path 376 | total_timesteps 4229.
Path 377 | total_timesteps 4238.
Path 378 | total_timesteps 4250.
Path 379 | total_timesteps 4261.
Path 380 | total_timesteps 4275.
Path 381 | total_timesteps 4291.
Path 382 | total_timesteps 4306.
Path 383 | total_timesteps 4317.
Path 384 | total_timesteps 4328.
Path 385 | total_timesteps 4336.
Path 386 | total_timesteps 4345.
Path 387 | total_timesteps 4360.
Path 388 | total_timesteps 4370.
Path 389 | total_timesteps 4378.
Path 390 | total_timesteps 4388.
Path 391 | total_timesteps 4412.
Path 392 | total_timesteps 4424.
Path 393 | total_timesteps 4435.
Path 394 | total_timesteps 4445.
Path 395 | total_timesteps 4453.
Path 396 | total_timesteps 4467.
Path 397 | total_timesteps 4477.
Path 398 | total_timesteps 4493.
Path 399 | total_timesteps 4504.
Path 400 | total_timesteps 4516.
Path 401 | total_timesteps 4528.
Path 402 | total_timesteps 4536.
Path 403 | total_timesteps 4548.
Path 404 | total_timesteps 4558.
Path 405 | total_timesteps 4568.
Path 406 | total_timesteps 4580.
Path 407 | total_timesteps 4588.
Path 408 | total_timesteps 4598.
Path 409 | total_timesteps 4607.
Path 410 | total_timesteps 4617.
Path 411 | total_timesteps 4627.
Path 412 | total_timesteps 4641.
Path 413 | total_timesteps 4656.
Path 414 | total_timesteps 4668.
Path 415 | total_timesteps 4681.
Path 416 | total_timesteps 4690.
Path 417 | total_timesteps 4706.
Path 418 | total_timesteps 4714.
Path 419 | total_timesteps 4722.
Path 420 | total_timesteps 4733.
Path 421 | total_timesteps 4742.
Path 422 | total_timesteps 4755.
Path 423 | total_timesteps 4768.
Path 424 | total_timesteps 4778.
Path 425 | total_timesteps 4797.
Path 426 | total_timesteps 4810.
Path 427 | total_timesteps 4824.
Path 428 | total_timesteps 4836.
Path 429 | total_timesteps 4846.
Path 430 | total_timesteps 4854.
Path 431 | total_timesteps 4865.
Path 432 | total_timesteps 4881.
Path 433 | total_timesteps 4890.
Path 434 | total_timesteps 4902.
Path 435 | total_timesteps 4910.
Path 436 | total_timesteps 4920.
Path 437 | total_timesteps 4934.
Path 438 | total_timesteps 4942.
Path 439 | total_timesteps 4952.
Path 440 | total_timesteps 4961.
Path 441 | total_timesteps 4968.
Path 442 | total_timesteps 4975.
Path 443 | total_timesteps 4988.
Path 444 | total_timesteps 4996.
Path 445 | total_timesteps 5005.
Path 446 | total_timesteps 5020.
Path 447 | total_timesteps 5028.
Path 448 | total_timesteps 5041.
Path 449 | total_timesteps 5050.
Path 450 | total_timesteps 5067.
Path 451 | total_timesteps 5074.
Path 452 | total_timesteps 5086.
Path 453 | total_timesteps 5100.
Path 454 | total_timesteps 5107.
Path 455 | total_timesteps 5117.
Path 456 | total_timesteps 5131.
Path 457 | total_timesteps 5147.
Path 458 | total_timesteps 5158.
Path 459 | total_timesteps 5168.
Path 460 | total_timesteps 5181.
Path 461 | total_timesteps 5199.
Path 462 | total_timesteps 5209.
Path 463 | total_timesteps 5225.
Path 464 | total_timesteps 5236.
Path 465 | total_timesteps 5245.
Path 466 | total_timesteps 5253.
Path 467 | total_timesteps 5261.
Path 468 | total_timesteps 5271.
Path 469 | total_timesteps 5290.
Path 470 | total_timesteps 5303.
Path 471 | total_timesteps 5314.
Path 472 | total_timesteps 5324.
Path 473 | total_timesteps 5332.
Path 474 | total_timesteps 5346.
Path 475 | total_timesteps 5354.
Path 476 | total_timesteps 5364.
Path 477 | total_timesteps 5373.
Path 478 | total_timesteps 5382.
Path 479 | total_timesteps 5390.
Path 480 | total_timesteps 5401.
Path 481 | total_timesteps 5412.
Path 482 | total_timesteps 5421.
Path 483 | total_timesteps 5431.
Path 484 | total_timesteps 5449.
Path 485 | total_timesteps 5465.
Path 486 | total_timesteps 5475.
Path 487 | total_timesteps 5484.
Path 488 | total_timesteps 5492.
Path 489 | total_timesteps 5499.
Path 490 | total_timesteps 5510.
Path 491 | total_timesteps 5524.
Path 492 | total_timesteps 5535.
Path 493 | total_timesteps 5543.
Path 494 | total_timesteps 5552.
Path 495 | total_timesteps 5561.
Path 496 | total_timesteps 5581.
Path 497 | total_timesteps 5595.
Path 498 | total_timesteps 5604.
Path 499 | total_timesteps 5619.
Path 500 | total_timesteps 5632.
Path 501 | total_timesteps 5645.
Path 502 | total_timesteps 5659.
Path 503 | total_timesteps 5674.
Path 504 | total_timesteps 5681.
Path 505 | total_timesteps 5689.
Path 506 | total_timesteps 5696.
Path 507 | total_timesteps 5711.
Path 508 | total_timesteps 5722.
Path 509 | total_timesteps 5735.
Path 510 | total_timesteps 5744.
Path 511 | total_timesteps 5752.
Path 512 | total_timesteps 5766.
Path 513 | total_timesteps 5776.
Path 514 | total_timesteps 5782.
Path 515 | total_timesteps 5795.
Path 516 | total_timesteps 5804.
Path 517 | total_timesteps 5813.
Path 518 | total_timesteps 5831.
Path 519 | total_timesteps 5841.
Path 520 | total_timesteps 5853.
Path 521 | total_timesteps 5861.
Path 522 | total_timesteps 5873.
Path 523 | total_timesteps 5881.
Path 524 | total_timesteps 5891.
Path 525 | total_timesteps 5901.
Path 526 | total_timesteps 5915.
Path 527 | total_timesteps 5925.
Path 528 | total_timesteps 5933.
Path 529 | total_timesteps 5946.
Path 530 | total_timesteps 5957.
Path 531 | total_timesteps 5968.
Path 532 | total_timesteps 5978.
Path 533 | total_timesteps 5985.
Path 534 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.06    |
| Iteration     | 10       |
| MaximumReturn | 2.51     |
| MinimumReturn | -20.4    |
| TotalSamples  | 48034    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.3092567026615143
Validation loss = 0.31621941924095154
Validation loss = 0.31525909900665283
Validation loss = 0.32299619913101196
Validation loss = 0.31931373476982117
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 59.
Path 6 | total_timesteps 77.
Path 7 | total_timesteps 89.
Path 8 | total_timesteps 104.
Path 9 | total_timesteps 115.
Path 10 | total_timesteps 123.
Path 11 | total_timesteps 138.
Path 12 | total_timesteps 160.
Path 13 | total_timesteps 170.
Path 14 | total_timesteps 179.
Path 15 | total_timesteps 192.
Path 16 | total_timesteps 201.
Path 17 | total_timesteps 210.
Path 18 | total_timesteps 218.
Path 19 | total_timesteps 230.
Path 20 | total_timesteps 239.
Path 21 | total_timesteps 250.
Path 22 | total_timesteps 258.
Path 23 | total_timesteps 270.
Path 24 | total_timesteps 283.
Path 25 | total_timesteps 293.
Path 26 | total_timesteps 318.
Path 27 | total_timesteps 325.
Path 28 | total_timesteps 338.
Path 29 | total_timesteps 348.
Path 30 | total_timesteps 358.
Path 31 | total_timesteps 367.
Path 32 | total_timesteps 380.
Path 33 | total_timesteps 391.
Path 34 | total_timesteps 402.
Path 35 | total_timesteps 412.
Path 36 | total_timesteps 430.
Path 37 | total_timesteps 444.
Path 38 | total_timesteps 452.
Path 39 | total_timesteps 463.
Path 40 | total_timesteps 474.
Path 41 | total_timesteps 485.
Path 42 | total_timesteps 497.
Path 43 | total_timesteps 512.
Path 44 | total_timesteps 523.
Path 45 | total_timesteps 532.
Path 46 | total_timesteps 543.
Path 47 | total_timesteps 553.
Path 48 | total_timesteps 563.
Path 49 | total_timesteps 572.
Path 50 | total_timesteps 580.
Path 51 | total_timesteps 586.
Path 52 | total_timesteps 599.
Path 53 | total_timesteps 609.
Path 54 | total_timesteps 620.
Path 55 | total_timesteps 627.
Path 56 | total_timesteps 645.
Path 57 | total_timesteps 657.
Path 58 | total_timesteps 667.
Path 59 | total_timesteps 676.
Path 60 | total_timesteps 688.
Path 61 | total_timesteps 699.
Path 62 | total_timesteps 707.
Path 63 | total_timesteps 719.
Path 64 | total_timesteps 727.
Path 65 | total_timesteps 734.
Path 66 | total_timesteps 744.
Path 67 | total_timesteps 753.
Path 68 | total_timesteps 765.
Path 69 | total_timesteps 778.
Path 70 | total_timesteps 789.
Path 71 | total_timesteps 800.
Path 72 | total_timesteps 813.
Path 73 | total_timesteps 827.
Path 74 | total_timesteps 836.
Path 75 | total_timesteps 849.
Path 76 | total_timesteps 861.
Path 77 | total_timesteps 869.
Path 78 | total_timesteps 880.
Path 79 | total_timesteps 889.
Path 80 | total_timesteps 897.
Path 81 | total_timesteps 906.
Path 82 | total_timesteps 918.
Path 83 | total_timesteps 927.
Path 84 | total_timesteps 941.
Path 85 | total_timesteps 957.
Path 86 | total_timesteps 964.
Path 87 | total_timesteps 972.
Path 88 | total_timesteps 983.
Path 89 | total_timesteps 995.
Path 90 | total_timesteps 1007.
Path 91 | total_timesteps 1016.
Path 92 | total_timesteps 1027.
Path 93 | total_timesteps 1041.
Path 94 | total_timesteps 1050.
Path 95 | total_timesteps 1061.
Path 96 | total_timesteps 1073.
Path 97 | total_timesteps 1085.
Path 98 | total_timesteps 1097.
Path 99 | total_timesteps 1109.
Path 100 | total_timesteps 1119.
Path 101 | total_timesteps 1132.
Path 102 | total_timesteps 1140.
Path 103 | total_timesteps 1152.
Path 104 | total_timesteps 1169.
Path 105 | total_timesteps 1181.
Path 106 | total_timesteps 1196.
Path 107 | total_timesteps 1205.
Path 108 | total_timesteps 1212.
Path 109 | total_timesteps 1224.
Path 110 | total_timesteps 1239.
Path 111 | total_timesteps 1250.
Path 112 | total_timesteps 1262.
Path 113 | total_timesteps 1272.
Path 114 | total_timesteps 1288.
Path 115 | total_timesteps 1299.
Path 116 | total_timesteps 1306.
Path 117 | total_timesteps 1316.
Path 118 | total_timesteps 1324.
Path 119 | total_timesteps 1341.
Path 120 | total_timesteps 1351.
Path 121 | total_timesteps 1360.
Path 122 | total_timesteps 1369.
Path 123 | total_timesteps 1380.
Path 124 | total_timesteps 1389.
Path 125 | total_timesteps 1403.
Path 126 | total_timesteps 1412.
Path 127 | total_timesteps 1422.
Path 128 | total_timesteps 1431.
Path 129 | total_timesteps 1438.
Path 130 | total_timesteps 1448.
Path 131 | total_timesteps 1459.
Path 132 | total_timesteps 1474.
Path 133 | total_timesteps 1486.
Path 134 | total_timesteps 1503.
Path 135 | total_timesteps 1513.
Path 136 | total_timesteps 1527.
Path 137 | total_timesteps 1537.
Path 138 | total_timesteps 1547.
Path 139 | total_timesteps 1557.
Path 140 | total_timesteps 1567.
Path 141 | total_timesteps 1583.
Path 142 | total_timesteps 1593.
Path 143 | total_timesteps 1607.
Path 144 | total_timesteps 1615.
Path 145 | total_timesteps 1627.
Path 146 | total_timesteps 1639.
Path 147 | total_timesteps 1648.
Path 148 | total_timesteps 1662.
Path 149 | total_timesteps 1671.
Path 150 | total_timesteps 1680.
Path 151 | total_timesteps 1690.
Path 152 | total_timesteps 1700.
Path 153 | total_timesteps 1711.
Path 154 | total_timesteps 1722.
Path 155 | total_timesteps 1743.
Path 156 | total_timesteps 1762.
Path 157 | total_timesteps 1782.
Path 158 | total_timesteps 1793.
Path 159 | total_timesteps 1810.
Path 160 | total_timesteps 1820.
Path 161 | total_timesteps 1830.
Path 162 | total_timesteps 1845.
Path 163 | total_timesteps 1852.
Path 164 | total_timesteps 1861.
Path 165 | total_timesteps 1868.
Path 166 | total_timesteps 1883.
Path 167 | total_timesteps 1894.
Path 168 | total_timesteps 1905.
Path 169 | total_timesteps 1914.
Path 170 | total_timesteps 1924.
Path 171 | total_timesteps 1933.
Path 172 | total_timesteps 1948.
Path 173 | total_timesteps 1958.
Path 174 | total_timesteps 1969.
Path 175 | total_timesteps 1984.
Path 176 | total_timesteps 1992.
Path 177 | total_timesteps 2001.
Path 178 | total_timesteps 2012.
Path 179 | total_timesteps 2022.
Path 180 | total_timesteps 2033.
Path 181 | total_timesteps 2042.
Path 182 | total_timesteps 2053.
Path 183 | total_timesteps 2064.
Path 184 | total_timesteps 2082.
Path 185 | total_timesteps 2094.
Path 186 | total_timesteps 2111.
Path 187 | total_timesteps 2123.
Path 188 | total_timesteps 2137.
Path 189 | total_timesteps 2147.
Path 190 | total_timesteps 2171.
Path 191 | total_timesteps 2185.
Path 192 | total_timesteps 2194.
Path 193 | total_timesteps 2203.
Path 194 | total_timesteps 2216.
Path 195 | total_timesteps 2226.
Path 196 | total_timesteps 2237.
Path 197 | total_timesteps 2253.
Path 198 | total_timesteps 2263.
Path 199 | total_timesteps 2271.
Path 200 | total_timesteps 2278.
Path 201 | total_timesteps 2288.
Path 202 | total_timesteps 2296.
Path 203 | total_timesteps 2310.
Path 204 | total_timesteps 2322.
Path 205 | total_timesteps 2331.
Path 206 | total_timesteps 2339.
Path 207 | total_timesteps 2354.
Path 208 | total_timesteps 2369.
Path 209 | total_timesteps 2376.
Path 210 | total_timesteps 2387.
Path 211 | total_timesteps 2396.
Path 212 | total_timesteps 2408.
Path 213 | total_timesteps 2415.
Path 214 | total_timesteps 2426.
Path 215 | total_timesteps 2433.
Path 216 | total_timesteps 2443.
Path 217 | total_timesteps 2452.
Path 218 | total_timesteps 2465.
Path 219 | total_timesteps 2475.
Path 220 | total_timesteps 2486.
Path 221 | total_timesteps 2495.
Path 222 | total_timesteps 2503.
Path 223 | total_timesteps 2518.
Path 224 | total_timesteps 2525.
Path 225 | total_timesteps 2532.
Path 226 | total_timesteps 2543.
Path 227 | total_timesteps 2557.
Path 228 | total_timesteps 2566.
Path 229 | total_timesteps 2574.
Path 230 | total_timesteps 2591.
Path 231 | total_timesteps 2607.
Path 232 | total_timesteps 2618.
Path 233 | total_timesteps 2627.
Path 234 | total_timesteps 2636.
Path 235 | total_timesteps 2646.
Path 236 | total_timesteps 2657.
Path 237 | total_timesteps 2668.
Path 238 | total_timesteps 2679.
Path 239 | total_timesteps 2689.
Path 240 | total_timesteps 2706.
Path 241 | total_timesteps 2718.
Path 242 | total_timesteps 2728.
Path 243 | total_timesteps 2737.
Path 244 | total_timesteps 2750.
Path 245 | total_timesteps 2759.
Path 246 | total_timesteps 2767.
Path 247 | total_timesteps 2777.
Path 248 | total_timesteps 2789.
Path 249 | total_timesteps 2798.
Path 250 | total_timesteps 2806.
Path 251 | total_timesteps 2817.
Path 252 | total_timesteps 2825.
Path 253 | total_timesteps 2838.
Path 254 | total_timesteps 2850.
Path 255 | total_timesteps 2861.
Path 256 | total_timesteps 2870.
Path 257 | total_timesteps 2878.
Path 258 | total_timesteps 2886.
Path 259 | total_timesteps 2896.
Path 260 | total_timesteps 2905.
Path 261 | total_timesteps 2914.
Path 262 | total_timesteps 2921.
Path 263 | total_timesteps 2933.
Path 264 | total_timesteps 2946.
Path 265 | total_timesteps 2954.
Path 266 | total_timesteps 2967.
Path 267 | total_timesteps 2978.
Path 268 | total_timesteps 2991.
Path 269 | total_timesteps 3001.
Path 270 | total_timesteps 3010.
Path 271 | total_timesteps 3019.
Path 272 | total_timesteps 3030.
Path 273 | total_timesteps 3039.
Path 274 | total_timesteps 3048.
Path 275 | total_timesteps 3055.
Path 276 | total_timesteps 3065.
Path 277 | total_timesteps 3073.
Path 278 | total_timesteps 3081.
Path 279 | total_timesteps 3094.
Path 280 | total_timesteps 3104.
Path 281 | total_timesteps 3111.
Path 282 | total_timesteps 3118.
Path 283 | total_timesteps 3129.
Path 284 | total_timesteps 3136.
Path 285 | total_timesteps 3144.
Path 286 | total_timesteps 3158.
Path 287 | total_timesteps 3167.
Path 288 | total_timesteps 3185.
Path 289 | total_timesteps 3195.
Path 290 | total_timesteps 3203.
Path 291 | total_timesteps 3213.
Path 292 | total_timesteps 3224.
Path 293 | total_timesteps 3232.
Path 294 | total_timesteps 3244.
Path 295 | total_timesteps 3261.
Path 296 | total_timesteps 3273.
Path 297 | total_timesteps 3281.
Path 298 | total_timesteps 3298.
Path 299 | total_timesteps 3313.
Path 300 | total_timesteps 3327.
Path 301 | total_timesteps 3338.
Path 302 | total_timesteps 3345.
Path 303 | total_timesteps 3352.
Path 304 | total_timesteps 3360.
Path 305 | total_timesteps 3368.
Path 306 | total_timesteps 3377.
Path 307 | total_timesteps 3387.
Path 308 | total_timesteps 3394.
Path 309 | total_timesteps 3407.
Path 310 | total_timesteps 3417.
Path 311 | total_timesteps 3428.
Path 312 | total_timesteps 3437.
Path 313 | total_timesteps 3446.
Path 314 | total_timesteps 3460.
Path 315 | total_timesteps 3470.
Path 316 | total_timesteps 3478.
Path 317 | total_timesteps 3489.
Path 318 | total_timesteps 3503.
Path 319 | total_timesteps 3513.
Path 320 | total_timesteps 3521.
Path 321 | total_timesteps 3531.
Path 322 | total_timesteps 3543.
Path 323 | total_timesteps 3553.
Path 324 | total_timesteps 3564.
Path 325 | total_timesteps 3574.
Path 326 | total_timesteps 3588.
Path 327 | total_timesteps 3597.
Path 328 | total_timesteps 3608.
Path 329 | total_timesteps 3620.
Path 330 | total_timesteps 3630.
Path 331 | total_timesteps 3637.
Path 332 | total_timesteps 3647.
Path 333 | total_timesteps 3656.
Path 334 | total_timesteps 3668.
Path 335 | total_timesteps 3681.
Path 336 | total_timesteps 3689.
Path 337 | total_timesteps 3700.
Path 338 | total_timesteps 3711.
Path 339 | total_timesteps 3720.
Path 340 | total_timesteps 3731.
Path 341 | total_timesteps 3741.
Path 342 | total_timesteps 3753.
Path 343 | total_timesteps 3763.
Path 344 | total_timesteps 3773.
Path 345 | total_timesteps 3781.
Path 346 | total_timesteps 3789.
Path 347 | total_timesteps 3799.
Path 348 | total_timesteps 3808.
Path 349 | total_timesteps 3820.
Path 350 | total_timesteps 3830.
Path 351 | total_timesteps 3839.
Path 352 | total_timesteps 3851.
Path 353 | total_timesteps 3863.
Path 354 | total_timesteps 3879.
Path 355 | total_timesteps 3892.
Path 356 | total_timesteps 3900.
Path 357 | total_timesteps 3907.
Path 358 | total_timesteps 3922.
Path 359 | total_timesteps 3935.
Path 360 | total_timesteps 3945.
Path 361 | total_timesteps 3954.
Path 362 | total_timesteps 3960.
Path 363 | total_timesteps 3969.
Path 364 | total_timesteps 3978.
Path 365 | total_timesteps 3986.
Path 366 | total_timesteps 3998.
Path 367 | total_timesteps 4011.
Path 368 | total_timesteps 4022.
Path 369 | total_timesteps 4035.
Path 370 | total_timesteps 4047.
Path 371 | total_timesteps 4059.
Path 372 | total_timesteps 4068.
Path 373 | total_timesteps 4077.
Path 374 | total_timesteps 4088.
Path 375 | total_timesteps 4099.
Path 376 | total_timesteps 4107.
Path 377 | total_timesteps 4118.
Path 378 | total_timesteps 4129.
Path 379 | total_timesteps 4138.
Path 380 | total_timesteps 4149.
Path 381 | total_timesteps 4159.
Path 382 | total_timesteps 4169.
Path 383 | total_timesteps 4191.
Path 384 | total_timesteps 4199.
Path 385 | total_timesteps 4209.
Path 386 | total_timesteps 4221.
Path 387 | total_timesteps 4233.
Path 388 | total_timesteps 4242.
Path 389 | total_timesteps 4253.
Path 390 | total_timesteps 4267.
Path 391 | total_timesteps 4280.
Path 392 | total_timesteps 4289.
Path 393 | total_timesteps 4301.
Path 394 | total_timesteps 4316.
Path 395 | total_timesteps 4329.
Path 396 | total_timesteps 4339.
Path 397 | total_timesteps 4348.
Path 398 | total_timesteps 4361.
Path 399 | total_timesteps 4370.
Path 400 | total_timesteps 4380.
Path 401 | total_timesteps 4389.
Path 402 | total_timesteps 4397.
Path 403 | total_timesteps 4408.
Path 404 | total_timesteps 4416.
Path 405 | total_timesteps 4426.
Path 406 | total_timesteps 4453.
Path 407 | total_timesteps 4463.
Path 408 | total_timesteps 4473.
Path 409 | total_timesteps 4485.
Path 410 | total_timesteps 4499.
Path 411 | total_timesteps 4505.
Path 412 | total_timesteps 4518.
Path 413 | total_timesteps 4527.
Path 414 | total_timesteps 4540.
Path 415 | total_timesteps 4556.
Path 416 | total_timesteps 4565.
Path 417 | total_timesteps 4578.
Path 418 | total_timesteps 4591.
Path 419 | total_timesteps 4601.
Path 420 | total_timesteps 4609.
Path 421 | total_timesteps 4620.
Path 422 | total_timesteps 4630.
Path 423 | total_timesteps 4639.
Path 424 | total_timesteps 4659.
Path 425 | total_timesteps 4671.
Path 426 | total_timesteps 4684.
Path 427 | total_timesteps 4695.
Path 428 | total_timesteps 4703.
Path 429 | total_timesteps 4713.
Path 430 | total_timesteps 4723.
Path 431 | total_timesteps 4734.
Path 432 | total_timesteps 4747.
Path 433 | total_timesteps 4756.
Path 434 | total_timesteps 4765.
Path 435 | total_timesteps 4777.
Path 436 | total_timesteps 4787.
Path 437 | total_timesteps 4798.
Path 438 | total_timesteps 4811.
Path 439 | total_timesteps 4827.
Path 440 | total_timesteps 4835.
Path 441 | total_timesteps 4844.
Path 442 | total_timesteps 4854.
Path 443 | total_timesteps 4866.
Path 444 | total_timesteps 4878.
Path 445 | total_timesteps 4886.
Path 446 | total_timesteps 4898.
Path 447 | total_timesteps 4907.
Path 448 | total_timesteps 4924.
Path 449 | total_timesteps 4935.
Path 450 | total_timesteps 4950.
Path 451 | total_timesteps 4959.
Path 452 | total_timesteps 4968.
Path 453 | total_timesteps 4983.
Path 454 | total_timesteps 4993.
Path 455 | total_timesteps 5003.
Path 456 | total_timesteps 5011.
Path 457 | total_timesteps 5022.
Path 458 | total_timesteps 5036.
Path 459 | total_timesteps 5044.
Path 460 | total_timesteps 5052.
Path 461 | total_timesteps 5060.
Path 462 | total_timesteps 5072.
Path 463 | total_timesteps 5084.
Path 464 | total_timesteps 5094.
Path 465 | total_timesteps 5104.
Path 466 | total_timesteps 5112.
Path 467 | total_timesteps 5119.
Path 468 | total_timesteps 5134.
Path 469 | total_timesteps 5143.
Path 470 | total_timesteps 5152.
Path 471 | total_timesteps 5160.
Path 472 | total_timesteps 5167.
Path 473 | total_timesteps 5190.
Path 474 | total_timesteps 5199.
Path 475 | total_timesteps 5213.
Path 476 | total_timesteps 5226.
Path 477 | total_timesteps 5239.
Path 478 | total_timesteps 5252.
Path 479 | total_timesteps 5264.
Path 480 | total_timesteps 5274.
Path 481 | total_timesteps 5283.
Path 482 | total_timesteps 5294.
Path 483 | total_timesteps 5302.
Path 484 | total_timesteps 5315.
Path 485 | total_timesteps 5327.
Path 486 | total_timesteps 5337.
Path 487 | total_timesteps 5352.
Path 488 | total_timesteps 5364.
Path 489 | total_timesteps 5381.
Path 490 | total_timesteps 5387.
Path 491 | total_timesteps 5398.
Path 492 | total_timesteps 5409.
Path 493 | total_timesteps 5420.
Path 494 | total_timesteps 5438.
Path 495 | total_timesteps 5446.
Path 496 | total_timesteps 5454.
Path 497 | total_timesteps 5464.
Path 498 | total_timesteps 5473.
Path 499 | total_timesteps 5487.
Path 500 | total_timesteps 5500.
Path 501 | total_timesteps 5509.
Path 502 | total_timesteps 5522.
Path 503 | total_timesteps 5528.
Path 504 | total_timesteps 5541.
Path 505 | total_timesteps 5550.
Path 506 | total_timesteps 5565.
Path 507 | total_timesteps 5575.
Path 508 | total_timesteps 5582.
Path 509 | total_timesteps 5602.
Path 510 | total_timesteps 5616.
Path 511 | total_timesteps 5625.
Path 512 | total_timesteps 5638.
Path 513 | total_timesteps 5649.
Path 514 | total_timesteps 5664.
Path 515 | total_timesteps 5673.
Path 516 | total_timesteps 5682.
Path 517 | total_timesteps 5695.
Path 518 | total_timesteps 5710.
Path 519 | total_timesteps 5720.
Path 520 | total_timesteps 5731.
Path 521 | total_timesteps 5742.
Path 522 | total_timesteps 5760.
Path 523 | total_timesteps 5770.
Path 524 | total_timesteps 5783.
Path 525 | total_timesteps 5797.
Path 526 | total_timesteps 5805.
Path 527 | total_timesteps 5817.
Path 528 | total_timesteps 5827.
Path 529 | total_timesteps 5840.
Path 530 | total_timesteps 5850.
Path 531 | total_timesteps 5858.
Path 532 | total_timesteps 5871.
Path 533 | total_timesteps 5882.
Path 534 | total_timesteps 5889.
Path 535 | total_timesteps 5898.
Path 536 | total_timesteps 5915.
Path 537 | total_timesteps 5925.
Path 538 | total_timesteps 5936.
Path 539 | total_timesteps 5943.
Path 540 | total_timesteps 5958.
Path 541 | total_timesteps 5969.
Path 542 | total_timesteps 5984.
Path 543 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.89    |
| Iteration     | 11       |
| MaximumReturn | 2.42     |
| MinimumReturn | -20.1    |
| TotalSamples  | 52036    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.31237608194351196
Validation loss = 0.3147532343864441
Validation loss = 0.31953147053718567
Validation loss = 0.31911277770996094
Validation loss = 0.32231584191322327
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 35.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 91.
Path 8 | total_timesteps 101.
Path 9 | total_timesteps 112.
Path 10 | total_timesteps 120.
Path 11 | total_timesteps 129.
Path 12 | total_timesteps 141.
Path 13 | total_timesteps 156.
Path 14 | total_timesteps 166.
Path 15 | total_timesteps 176.
Path 16 | total_timesteps 183.
Path 17 | total_timesteps 192.
Path 18 | total_timesteps 204.
Path 19 | total_timesteps 215.
Path 20 | total_timesteps 234.
Path 21 | total_timesteps 251.
Path 22 | total_timesteps 263.
Path 23 | total_timesteps 276.
Path 24 | total_timesteps 286.
Path 25 | total_timesteps 297.
Path 26 | total_timesteps 312.
Path 27 | total_timesteps 330.
Path 28 | total_timesteps 340.
Path 29 | total_timesteps 348.
Path 30 | total_timesteps 359.
Path 31 | total_timesteps 373.
Path 32 | total_timesteps 381.
Path 33 | total_timesteps 393.
Path 34 | total_timesteps 401.
Path 35 | total_timesteps 410.
Path 36 | total_timesteps 421.
Path 37 | total_timesteps 431.
Path 38 | total_timesteps 449.
Path 39 | total_timesteps 466.
Path 40 | total_timesteps 474.
Path 41 | total_timesteps 482.
Path 42 | total_timesteps 495.
Path 43 | total_timesteps 508.
Path 44 | total_timesteps 521.
Path 45 | total_timesteps 528.
Path 46 | total_timesteps 541.
Path 47 | total_timesteps 566.
Path 48 | total_timesteps 580.
Path 49 | total_timesteps 587.
Path 50 | total_timesteps 598.
Path 51 | total_timesteps 606.
Path 52 | total_timesteps 613.
Path 53 | total_timesteps 626.
Path 54 | total_timesteps 634.
Path 55 | total_timesteps 645.
Path 56 | total_timesteps 657.
Path 57 | total_timesteps 666.
Path 58 | total_timesteps 675.
Path 59 | total_timesteps 683.
Path 60 | total_timesteps 694.
Path 61 | total_timesteps 703.
Path 62 | total_timesteps 709.
Path 63 | total_timesteps 719.
Path 64 | total_timesteps 733.
Path 65 | total_timesteps 742.
Path 66 | total_timesteps 750.
Path 67 | total_timesteps 768.
Path 68 | total_timesteps 781.
Path 69 | total_timesteps 793.
Path 70 | total_timesteps 804.
Path 71 | total_timesteps 814.
Path 72 | total_timesteps 823.
Path 73 | total_timesteps 834.
Path 74 | total_timesteps 849.
Path 75 | total_timesteps 859.
Path 76 | total_timesteps 871.
Path 77 | total_timesteps 883.
Path 78 | total_timesteps 892.
Path 79 | total_timesteps 902.
Path 80 | total_timesteps 923.
Path 81 | total_timesteps 935.
Path 82 | total_timesteps 947.
Path 83 | total_timesteps 956.
Path 84 | total_timesteps 965.
Path 85 | total_timesteps 984.
Path 86 | total_timesteps 994.
Path 87 | total_timesteps 1005.
Path 88 | total_timesteps 1012.
Path 89 | total_timesteps 1020.
Path 90 | total_timesteps 1028.
Path 91 | total_timesteps 1037.
Path 92 | total_timesteps 1047.
Path 93 | total_timesteps 1057.
Path 94 | total_timesteps 1075.
Path 95 | total_timesteps 1094.
Path 96 | total_timesteps 1103.
Path 97 | total_timesteps 1118.
Path 98 | total_timesteps 1137.
Path 99 | total_timesteps 1145.
Path 100 | total_timesteps 1156.
Path 101 | total_timesteps 1172.
Path 102 | total_timesteps 1187.
Path 103 | total_timesteps 1199.
Path 104 | total_timesteps 1212.
Path 105 | total_timesteps 1220.
Path 106 | total_timesteps 1240.
Path 107 | total_timesteps 1253.
Path 108 | total_timesteps 1263.
Path 109 | total_timesteps 1272.
Path 110 | total_timesteps 1286.
Path 111 | total_timesteps 1300.
Path 112 | total_timesteps 1310.
Path 113 | total_timesteps 1321.
Path 114 | total_timesteps 1355.
Path 115 | total_timesteps 1365.
Path 116 | total_timesteps 1376.
Path 117 | total_timesteps 1387.
Path 118 | total_timesteps 1398.
Path 119 | total_timesteps 1408.
Path 120 | total_timesteps 1420.
Path 121 | total_timesteps 1433.
Path 122 | total_timesteps 1444.
Path 123 | total_timesteps 1454.
Path 124 | total_timesteps 1461.
Path 125 | total_timesteps 1471.
Path 126 | total_timesteps 1479.
Path 127 | total_timesteps 1492.
Path 128 | total_timesteps 1501.
Path 129 | total_timesteps 1516.
Path 130 | total_timesteps 1524.
Path 131 | total_timesteps 1533.
Path 132 | total_timesteps 1543.
Path 133 | total_timesteps 1554.
Path 134 | total_timesteps 1569.
Path 135 | total_timesteps 1580.
Path 136 | total_timesteps 1593.
Path 137 | total_timesteps 1607.
Path 138 | total_timesteps 1617.
Path 139 | total_timesteps 1629.
Path 140 | total_timesteps 1636.
Path 141 | total_timesteps 1643.
Path 142 | total_timesteps 1657.
Path 143 | total_timesteps 1680.
Path 144 | total_timesteps 1692.
Path 145 | total_timesteps 1700.
Path 146 | total_timesteps 1713.
Path 147 | total_timesteps 1722.
Path 148 | total_timesteps 1733.
Path 149 | total_timesteps 1746.
Path 150 | total_timesteps 1758.
Path 151 | total_timesteps 1765.
Path 152 | total_timesteps 1777.
Path 153 | total_timesteps 1786.
Path 154 | total_timesteps 1796.
Path 155 | total_timesteps 1805.
Path 156 | total_timesteps 1818.
Path 157 | total_timesteps 1831.
Path 158 | total_timesteps 1840.
Path 159 | total_timesteps 1860.
Path 160 | total_timesteps 1870.
Path 161 | total_timesteps 1879.
Path 162 | total_timesteps 1897.
Path 163 | total_timesteps 1912.
Path 164 | total_timesteps 1922.
Path 165 | total_timesteps 1929.
Path 166 | total_timesteps 1940.
Path 167 | total_timesteps 1950.
Path 168 | total_timesteps 1958.
Path 169 | total_timesteps 1971.
Path 170 | total_timesteps 1980.
Path 171 | total_timesteps 1992.
Path 172 | total_timesteps 2002.
Path 173 | total_timesteps 2011.
Path 174 | total_timesteps 2022.
Path 175 | total_timesteps 2032.
Path 176 | total_timesteps 2040.
Path 177 | total_timesteps 2049.
Path 178 | total_timesteps 2061.
Path 179 | total_timesteps 2075.
Path 180 | total_timesteps 2081.
Path 181 | total_timesteps 2089.
Path 182 | total_timesteps 2107.
Path 183 | total_timesteps 2119.
Path 184 | total_timesteps 2131.
Path 185 | total_timesteps 2143.
Path 186 | total_timesteps 2151.
Path 187 | total_timesteps 2161.
Path 188 | total_timesteps 2172.
Path 189 | total_timesteps 2186.
Path 190 | total_timesteps 2199.
Path 191 | total_timesteps 2212.
Path 192 | total_timesteps 2223.
Path 193 | total_timesteps 2232.
Path 194 | total_timesteps 2242.
Path 195 | total_timesteps 2257.
Path 196 | total_timesteps 2272.
Path 197 | total_timesteps 2280.
Path 198 | total_timesteps 2288.
Path 199 | total_timesteps 2296.
Path 200 | total_timesteps 2306.
Path 201 | total_timesteps 2316.
Path 202 | total_timesteps 2324.
Path 203 | total_timesteps 2338.
Path 204 | total_timesteps 2352.
Path 205 | total_timesteps 2361.
Path 206 | total_timesteps 2374.
Path 207 | total_timesteps 2382.
Path 208 | total_timesteps 2393.
Path 209 | total_timesteps 2411.
Path 210 | total_timesteps 2432.
Path 211 | total_timesteps 2439.
Path 212 | total_timesteps 2449.
Path 213 | total_timesteps 2458.
Path 214 | total_timesteps 2466.
Path 215 | total_timesteps 2478.
Path 216 | total_timesteps 2490.
Path 217 | total_timesteps 2500.
Path 218 | total_timesteps 2509.
Path 219 | total_timesteps 2520.
Path 220 | total_timesteps 2537.
Path 221 | total_timesteps 2557.
Path 222 | total_timesteps 2566.
Path 223 | total_timesteps 2575.
Path 224 | total_timesteps 2583.
Path 225 | total_timesteps 2593.
Path 226 | total_timesteps 2604.
Path 227 | total_timesteps 2620.
Path 228 | total_timesteps 2632.
Path 229 | total_timesteps 2645.
Path 230 | total_timesteps 2663.
Path 231 | total_timesteps 2679.
Path 232 | total_timesteps 2689.
Path 233 | total_timesteps 2699.
Path 234 | total_timesteps 2719.
Path 235 | total_timesteps 2729.
Path 236 | total_timesteps 2740.
Path 237 | total_timesteps 2747.
Path 238 | total_timesteps 2758.
Path 239 | total_timesteps 2768.
Path 240 | total_timesteps 2779.
Path 241 | total_timesteps 2789.
Path 242 | total_timesteps 2798.
Path 243 | total_timesteps 2810.
Path 244 | total_timesteps 2819.
Path 245 | total_timesteps 2826.
Path 246 | total_timesteps 2843.
Path 247 | total_timesteps 2855.
Path 248 | total_timesteps 2864.
Path 249 | total_timesteps 2872.
Path 250 | total_timesteps 2880.
Path 251 | total_timesteps 2887.
Path 252 | total_timesteps 2894.
Path 253 | total_timesteps 2904.
Path 254 | total_timesteps 2916.
Path 255 | total_timesteps 2923.
Path 256 | total_timesteps 2932.
Path 257 | total_timesteps 2947.
Path 258 | total_timesteps 2958.
Path 259 | total_timesteps 2974.
Path 260 | total_timesteps 2983.
Path 261 | total_timesteps 2994.
Path 262 | total_timesteps 3008.
Path 263 | total_timesteps 3019.
Path 264 | total_timesteps 3030.
Path 265 | total_timesteps 3040.
Path 266 | total_timesteps 3055.
Path 267 | total_timesteps 3063.
Path 268 | total_timesteps 3073.
Path 269 | total_timesteps 3081.
Path 270 | total_timesteps 3094.
Path 271 | total_timesteps 3103.
Path 272 | total_timesteps 3113.
Path 273 | total_timesteps 3129.
Path 274 | total_timesteps 3139.
Path 275 | total_timesteps 3150.
Path 276 | total_timesteps 3156.
Path 277 | total_timesteps 3171.
Path 278 | total_timesteps 3181.
Path 279 | total_timesteps 3189.
Path 280 | total_timesteps 3201.
Path 281 | total_timesteps 3223.
Path 282 | total_timesteps 3231.
Path 283 | total_timesteps 3238.
Path 284 | total_timesteps 3246.
Path 285 | total_timesteps 3261.
Path 286 | total_timesteps 3271.
Path 287 | total_timesteps 3280.
Path 288 | total_timesteps 3289.
Path 289 | total_timesteps 3298.
Path 290 | total_timesteps 3308.
Path 291 | total_timesteps 3319.
Path 292 | total_timesteps 3329.
Path 293 | total_timesteps 3337.
Path 294 | total_timesteps 3346.
Path 295 | total_timesteps 3354.
Path 296 | total_timesteps 3363.
Path 297 | total_timesteps 3382.
Path 298 | total_timesteps 3389.
Path 299 | total_timesteps 3397.
Path 300 | total_timesteps 3407.
Path 301 | total_timesteps 3419.
Path 302 | total_timesteps 3431.
Path 303 | total_timesteps 3450.
Path 304 | total_timesteps 3464.
Path 305 | total_timesteps 3474.
Path 306 | total_timesteps 3483.
Path 307 | total_timesteps 3500.
Path 308 | total_timesteps 3507.
Path 309 | total_timesteps 3516.
Path 310 | total_timesteps 3526.
Path 311 | total_timesteps 3539.
Path 312 | total_timesteps 3547.
Path 313 | total_timesteps 3557.
Path 314 | total_timesteps 3567.
Path 315 | total_timesteps 3585.
Path 316 | total_timesteps 3594.
Path 317 | total_timesteps 3615.
Path 318 | total_timesteps 3626.
Path 319 | total_timesteps 3637.
Path 320 | total_timesteps 3644.
Path 321 | total_timesteps 3661.
Path 322 | total_timesteps 3678.
Path 323 | total_timesteps 3690.
Path 324 | total_timesteps 3702.
Path 325 | total_timesteps 3720.
Path 326 | total_timesteps 3738.
Path 327 | total_timesteps 3747.
Path 328 | total_timesteps 3761.
Path 329 | total_timesteps 3772.
Path 330 | total_timesteps 3780.
Path 331 | total_timesteps 3789.
Path 332 | total_timesteps 3797.
Path 333 | total_timesteps 3807.
Path 334 | total_timesteps 3817.
Path 335 | total_timesteps 3834.
Path 336 | total_timesteps 3843.
Path 337 | total_timesteps 3851.
Path 338 | total_timesteps 3860.
Path 339 | total_timesteps 3870.
Path 340 | total_timesteps 3881.
Path 341 | total_timesteps 3890.
Path 342 | total_timesteps 3898.
Path 343 | total_timesteps 3907.
Path 344 | total_timesteps 3919.
Path 345 | total_timesteps 3930.
Path 346 | total_timesteps 3945.
Path 347 | total_timesteps 3960.
Path 348 | total_timesteps 3969.
Path 349 | total_timesteps 3981.
Path 350 | total_timesteps 4001.
Path 351 | total_timesteps 4009.
Path 352 | total_timesteps 4026.
Path 353 | total_timesteps 4034.
Path 354 | total_timesteps 4043.
Path 355 | total_timesteps 4059.
Path 356 | total_timesteps 4070.
Path 357 | total_timesteps 4079.
Path 358 | total_timesteps 4088.
Path 359 | total_timesteps 4097.
Path 360 | total_timesteps 4108.
Path 361 | total_timesteps 4121.
Path 362 | total_timesteps 4129.
Path 363 | total_timesteps 4140.
Path 364 | total_timesteps 4148.
Path 365 | total_timesteps 4157.
Path 366 | total_timesteps 4170.
Path 367 | total_timesteps 4180.
Path 368 | total_timesteps 4201.
Path 369 | total_timesteps 4209.
Path 370 | total_timesteps 4222.
Path 371 | total_timesteps 4230.
Path 372 | total_timesteps 4244.
Path 373 | total_timesteps 4256.
Path 374 | total_timesteps 4263.
Path 375 | total_timesteps 4276.
Path 376 | total_timesteps 4285.
Path 377 | total_timesteps 4293.
Path 378 | total_timesteps 4300.
Path 379 | total_timesteps 4313.
Path 380 | total_timesteps 4328.
Path 381 | total_timesteps 4338.
Path 382 | total_timesteps 4350.
Path 383 | total_timesteps 4356.
Path 384 | total_timesteps 4372.
Path 385 | total_timesteps 4380.
Path 386 | total_timesteps 4392.
Path 387 | total_timesteps 4401.
Path 388 | total_timesteps 4410.
Path 389 | total_timesteps 4420.
Path 390 | total_timesteps 4432.
Path 391 | total_timesteps 4439.
Path 392 | total_timesteps 4451.
Path 393 | total_timesteps 4463.
Path 394 | total_timesteps 4475.
Path 395 | total_timesteps 4488.
Path 396 | total_timesteps 4503.
Path 397 | total_timesteps 4516.
Path 398 | total_timesteps 4530.
Path 399 | total_timesteps 4539.
Path 400 | total_timesteps 4546.
Path 401 | total_timesteps 4558.
Path 402 | total_timesteps 4571.
Path 403 | total_timesteps 4580.
Path 404 | total_timesteps 4589.
Path 405 | total_timesteps 4597.
Path 406 | total_timesteps 4608.
Path 407 | total_timesteps 4617.
Path 408 | total_timesteps 4625.
Path 409 | total_timesteps 4632.
Path 410 | total_timesteps 4646.
Path 411 | total_timesteps 4658.
Path 412 | total_timesteps 4667.
Path 413 | total_timesteps 4678.
Path 414 | total_timesteps 4687.
Path 415 | total_timesteps 4705.
Path 416 | total_timesteps 4717.
Path 417 | total_timesteps 4727.
Path 418 | total_timesteps 4737.
Path 419 | total_timesteps 4748.
Path 420 | total_timesteps 4758.
Path 421 | total_timesteps 4767.
Path 422 | total_timesteps 4779.
Path 423 | total_timesteps 4786.
Path 424 | total_timesteps 4804.
Path 425 | total_timesteps 4811.
Path 426 | total_timesteps 4821.
Path 427 | total_timesteps 4833.
Path 428 | total_timesteps 4846.
Path 429 | total_timesteps 4863.
Path 430 | total_timesteps 4880.
Path 431 | total_timesteps 4887.
Path 432 | total_timesteps 4901.
Path 433 | total_timesteps 4927.
Path 434 | total_timesteps 4937.
Path 435 | total_timesteps 4945.
Path 436 | total_timesteps 4954.
Path 437 | total_timesteps 4963.
Path 438 | total_timesteps 4974.
Path 439 | total_timesteps 4985.
Path 440 | total_timesteps 4993.
Path 441 | total_timesteps 5003.
Path 442 | total_timesteps 5014.
Path 443 | total_timesteps 5022.
Path 444 | total_timesteps 5030.
Path 445 | total_timesteps 5040.
Path 446 | total_timesteps 5049.
Path 447 | total_timesteps 5065.
Path 448 | total_timesteps 5073.
Path 449 | total_timesteps 5085.
Path 450 | total_timesteps 5098.
Path 451 | total_timesteps 5106.
Path 452 | total_timesteps 5118.
Path 453 | total_timesteps 5132.
Path 454 | total_timesteps 5142.
Path 455 | total_timesteps 5150.
Path 456 | total_timesteps 5166.
Path 457 | total_timesteps 5174.
Path 458 | total_timesteps 5204.
Path 459 | total_timesteps 5215.
Path 460 | total_timesteps 5225.
Path 461 | total_timesteps 5235.
Path 462 | total_timesteps 5245.
Path 463 | total_timesteps 5257.
Path 464 | total_timesteps 5268.
Path 465 | total_timesteps 5283.
Path 466 | total_timesteps 5291.
Path 467 | total_timesteps 5300.
Path 468 | total_timesteps 5308.
Path 469 | total_timesteps 5318.
Path 470 | total_timesteps 5330.
Path 471 | total_timesteps 5340.
Path 472 | total_timesteps 5349.
Path 473 | total_timesteps 5358.
Path 474 | total_timesteps 5368.
Path 475 | total_timesteps 5391.
Path 476 | total_timesteps 5405.
Path 477 | total_timesteps 5417.
Path 478 | total_timesteps 5425.
Path 479 | total_timesteps 5434.
Path 480 | total_timesteps 5446.
Path 481 | total_timesteps 5456.
Path 482 | total_timesteps 5465.
Path 483 | total_timesteps 5474.
Path 484 | total_timesteps 5481.
Path 485 | total_timesteps 5489.
Path 486 | total_timesteps 5498.
Path 487 | total_timesteps 5512.
Path 488 | total_timesteps 5527.
Path 489 | total_timesteps 5538.
Path 490 | total_timesteps 5547.
Path 491 | total_timesteps 5557.
Path 492 | total_timesteps 5568.
Path 493 | total_timesteps 5578.
Path 494 | total_timesteps 5592.
Path 495 | total_timesteps 5604.
Path 496 | total_timesteps 5614.
Path 497 | total_timesteps 5624.
Path 498 | total_timesteps 5632.
Path 499 | total_timesteps 5641.
Path 500 | total_timesteps 5650.
Path 501 | total_timesteps 5662.
Path 502 | total_timesteps 5675.
Path 503 | total_timesteps 5685.
Path 504 | total_timesteps 5694.
Path 505 | total_timesteps 5704.
Path 506 | total_timesteps 5715.
Path 507 | total_timesteps 5724.
Path 508 | total_timesteps 5738.
Path 509 | total_timesteps 5747.
Path 510 | total_timesteps 5759.
Path 511 | total_timesteps 5769.
Path 512 | total_timesteps 5777.
Path 513 | total_timesteps 5788.
Path 514 | total_timesteps 5802.
Path 515 | total_timesteps 5813.
Path 516 | total_timesteps 5824.
Path 517 | total_timesteps 5834.
Path 518 | total_timesteps 5846.
Path 519 | total_timesteps 5857.
Path 520 | total_timesteps 5864.
Path 521 | total_timesteps 5874.
Path 522 | total_timesteps 5881.
Path 523 | total_timesteps 5894.
Path 524 | total_timesteps 5901.
Path 525 | total_timesteps 5913.
Path 526 | total_timesteps 5927.
Path 527 | total_timesteps 5940.
Path 528 | total_timesteps 5948.
Path 529 | total_timesteps 5955.
Path 530 | total_timesteps 5965.
Path 531 | total_timesteps 5975.
Path 532 | total_timesteps 5986.
Path 533 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.94    |
| Iteration     | 12       |
| MaximumReturn | 3.6      |
| MinimumReturn | -19.1    |
| TotalSamples  | 56046    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.31510522961616516
Validation loss = 0.3172059655189514
Validation loss = 0.32057490944862366
Validation loss = 0.32593363523483276
Validation loss = 0.32429543137550354
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 67.
Path 7 | total_timesteps 78.
Path 8 | total_timesteps 90.
Path 9 | total_timesteps 103.
Path 10 | total_timesteps 113.
Path 11 | total_timesteps 124.
Path 12 | total_timesteps 141.
Path 13 | total_timesteps 150.
Path 14 | total_timesteps 157.
Path 15 | total_timesteps 167.
Path 16 | total_timesteps 176.
Path 17 | total_timesteps 194.
Path 18 | total_timesteps 204.
Path 19 | total_timesteps 218.
Path 20 | total_timesteps 230.
Path 21 | total_timesteps 240.
Path 22 | total_timesteps 250.
Path 23 | total_timesteps 262.
Path 24 | total_timesteps 273.
Path 25 | total_timesteps 284.
Path 26 | total_timesteps 295.
Path 27 | total_timesteps 304.
Path 28 | total_timesteps 315.
Path 29 | total_timesteps 322.
Path 30 | total_timesteps 330.
Path 31 | total_timesteps 340.
Path 32 | total_timesteps 349.
Path 33 | total_timesteps 366.
Path 34 | total_timesteps 387.
Path 35 | total_timesteps 396.
Path 36 | total_timesteps 404.
Path 37 | total_timesteps 414.
Path 38 | total_timesteps 425.
Path 39 | total_timesteps 433.
Path 40 | total_timesteps 453.
Path 41 | total_timesteps 464.
Path 42 | total_timesteps 474.
Path 43 | total_timesteps 483.
Path 44 | total_timesteps 496.
Path 45 | total_timesteps 505.
Path 46 | total_timesteps 517.
Path 47 | total_timesteps 525.
Path 48 | total_timesteps 534.
Path 49 | total_timesteps 547.
Path 50 | total_timesteps 557.
Path 51 | total_timesteps 566.
Path 52 | total_timesteps 573.
Path 53 | total_timesteps 586.
Path 54 | total_timesteps 601.
Path 55 | total_timesteps 612.
Path 56 | total_timesteps 620.
Path 57 | total_timesteps 630.
Path 58 | total_timesteps 639.
Path 59 | total_timesteps 648.
Path 60 | total_timesteps 660.
Path 61 | total_timesteps 672.
Path 62 | total_timesteps 683.
Path 63 | total_timesteps 694.
Path 64 | total_timesteps 704.
Path 65 | total_timesteps 712.
Path 66 | total_timesteps 722.
Path 67 | total_timesteps 736.
Path 68 | total_timesteps 744.
Path 69 | total_timesteps 752.
Path 70 | total_timesteps 765.
Path 71 | total_timesteps 779.
Path 72 | total_timesteps 787.
Path 73 | total_timesteps 799.
Path 74 | total_timesteps 807.
Path 75 | total_timesteps 816.
Path 76 | total_timesteps 825.
Path 77 | total_timesteps 833.
Path 78 | total_timesteps 843.
Path 79 | total_timesteps 866.
Path 80 | total_timesteps 879.
Path 81 | total_timesteps 888.
Path 82 | total_timesteps 899.
Path 83 | total_timesteps 911.
Path 84 | total_timesteps 925.
Path 85 | total_timesteps 936.
Path 86 | total_timesteps 949.
Path 87 | total_timesteps 960.
Path 88 | total_timesteps 973.
Path 89 | total_timesteps 980.
Path 90 | total_timesteps 993.
Path 91 | total_timesteps 1002.
Path 92 | total_timesteps 1011.
Path 93 | total_timesteps 1023.
Path 94 | total_timesteps 1034.
Path 95 | total_timesteps 1042.
Path 96 | total_timesteps 1055.
Path 97 | total_timesteps 1072.
Path 98 | total_timesteps 1079.
Path 99 | total_timesteps 1090.
Path 100 | total_timesteps 1100.
Path 101 | total_timesteps 1108.
Path 102 | total_timesteps 1120.
Path 103 | total_timesteps 1131.
Path 104 | total_timesteps 1144.
Path 105 | total_timesteps 1152.
Path 106 | total_timesteps 1161.
Path 107 | total_timesteps 1175.
Path 108 | total_timesteps 1186.
Path 109 | total_timesteps 1198.
Path 110 | total_timesteps 1209.
Path 111 | total_timesteps 1218.
Path 112 | total_timesteps 1230.
Path 113 | total_timesteps 1241.
Path 114 | total_timesteps 1251.
Path 115 | total_timesteps 1261.
Path 116 | total_timesteps 1269.
Path 117 | total_timesteps 1280.
Path 118 | total_timesteps 1288.
Path 119 | total_timesteps 1299.
Path 120 | total_timesteps 1316.
Path 121 | total_timesteps 1328.
Path 122 | total_timesteps 1340.
Path 123 | total_timesteps 1348.
Path 124 | total_timesteps 1361.
Path 125 | total_timesteps 1370.
Path 126 | total_timesteps 1382.
Path 127 | total_timesteps 1389.
Path 128 | total_timesteps 1398.
Path 129 | total_timesteps 1407.
Path 130 | total_timesteps 1414.
Path 131 | total_timesteps 1435.
Path 132 | total_timesteps 1443.
Path 133 | total_timesteps 1460.
Path 134 | total_timesteps 1472.
Path 135 | total_timesteps 1485.
Path 136 | total_timesteps 1496.
Path 137 | total_timesteps 1511.
Path 138 | total_timesteps 1523.
Path 139 | total_timesteps 1535.
Path 140 | total_timesteps 1545.
Path 141 | total_timesteps 1557.
Path 142 | total_timesteps 1567.
Path 143 | total_timesteps 1580.
Path 144 | total_timesteps 1588.
Path 145 | total_timesteps 1612.
Path 146 | total_timesteps 1622.
Path 147 | total_timesteps 1633.
Path 148 | total_timesteps 1640.
Path 149 | total_timesteps 1649.
Path 150 | total_timesteps 1661.
Path 151 | total_timesteps 1679.
Path 152 | total_timesteps 1691.
Path 153 | total_timesteps 1700.
Path 154 | total_timesteps 1714.
Path 155 | total_timesteps 1729.
Path 156 | total_timesteps 1739.
Path 157 | total_timesteps 1749.
Path 158 | total_timesteps 1760.
Path 159 | total_timesteps 1771.
Path 160 | total_timesteps 1781.
Path 161 | total_timesteps 1791.
Path 162 | total_timesteps 1801.
Path 163 | total_timesteps 1812.
Path 164 | total_timesteps 1824.
Path 165 | total_timesteps 1835.
Path 166 | total_timesteps 1849.
Path 167 | total_timesteps 1862.
Path 168 | total_timesteps 1870.
Path 169 | total_timesteps 1879.
Path 170 | total_timesteps 1886.
Path 171 | total_timesteps 1896.
Path 172 | total_timesteps 1903.
Path 173 | total_timesteps 1913.
Path 174 | total_timesteps 1924.
Path 175 | total_timesteps 1933.
Path 176 | total_timesteps 1944.
Path 177 | total_timesteps 1957.
Path 178 | total_timesteps 1964.
Path 179 | total_timesteps 1971.
Path 180 | total_timesteps 1980.
Path 181 | total_timesteps 1998.
Path 182 | total_timesteps 2005.
Path 183 | total_timesteps 2013.
Path 184 | total_timesteps 2024.
Path 185 | total_timesteps 2033.
Path 186 | total_timesteps 2043.
Path 187 | total_timesteps 2052.
Path 188 | total_timesteps 2061.
Path 189 | total_timesteps 2069.
Path 190 | total_timesteps 2078.
Path 191 | total_timesteps 2088.
Path 192 | total_timesteps 2099.
Path 193 | total_timesteps 2116.
Path 194 | total_timesteps 2126.
Path 195 | total_timesteps 2134.
Path 196 | total_timesteps 2145.
Path 197 | total_timesteps 2154.
Path 198 | total_timesteps 2164.
Path 199 | total_timesteps 2176.
Path 200 | total_timesteps 2184.
Path 201 | total_timesteps 2190.
Path 202 | total_timesteps 2203.
Path 203 | total_timesteps 2216.
Path 204 | total_timesteps 2226.
Path 205 | total_timesteps 2240.
Path 206 | total_timesteps 2254.
Path 207 | total_timesteps 2273.
Path 208 | total_timesteps 2286.
Path 209 | total_timesteps 2297.
Path 210 | total_timesteps 2306.
Path 211 | total_timesteps 2316.
Path 212 | total_timesteps 2325.
Path 213 | total_timesteps 2338.
Path 214 | total_timesteps 2347.
Path 215 | total_timesteps 2357.
Path 216 | total_timesteps 2368.
Path 217 | total_timesteps 2378.
Path 218 | total_timesteps 2388.
Path 219 | total_timesteps 2403.
Path 220 | total_timesteps 2412.
Path 221 | total_timesteps 2422.
Path 222 | total_timesteps 2435.
Path 223 | total_timesteps 2444.
Path 224 | total_timesteps 2454.
Path 225 | total_timesteps 2464.
Path 226 | total_timesteps 2475.
Path 227 | total_timesteps 2483.
Path 228 | total_timesteps 2494.
Path 229 | total_timesteps 2503.
Path 230 | total_timesteps 2511.
Path 231 | total_timesteps 2520.
Path 232 | total_timesteps 2527.
Path 233 | total_timesteps 2536.
Path 234 | total_timesteps 2547.
Path 235 | total_timesteps 2559.
Path 236 | total_timesteps 2574.
Path 237 | total_timesteps 2590.
Path 238 | total_timesteps 2600.
Path 239 | total_timesteps 2611.
Path 240 | total_timesteps 2619.
Path 241 | total_timesteps 2628.
Path 242 | total_timesteps 2634.
Path 243 | total_timesteps 2642.
Path 244 | total_timesteps 2650.
Path 245 | total_timesteps 2658.
Path 246 | total_timesteps 2670.
Path 247 | total_timesteps 2683.
Path 248 | total_timesteps 2696.
Path 249 | total_timesteps 2706.
Path 250 | total_timesteps 2717.
Path 251 | total_timesteps 2724.
Path 252 | total_timesteps 2740.
Path 253 | total_timesteps 2758.
Path 254 | total_timesteps 2768.
Path 255 | total_timesteps 2780.
Path 256 | total_timesteps 2792.
Path 257 | total_timesteps 2807.
Path 258 | total_timesteps 2817.
Path 259 | total_timesteps 2833.
Path 260 | total_timesteps 2841.
Path 261 | total_timesteps 2850.
Path 262 | total_timesteps 2858.
Path 263 | total_timesteps 2866.
Path 264 | total_timesteps 2876.
Path 265 | total_timesteps 2887.
Path 266 | total_timesteps 2898.
Path 267 | total_timesteps 2906.
Path 268 | total_timesteps 2917.
Path 269 | total_timesteps 2928.
Path 270 | total_timesteps 2937.
Path 271 | total_timesteps 2949.
Path 272 | total_timesteps 2959.
Path 273 | total_timesteps 2970.
Path 274 | total_timesteps 2986.
Path 275 | total_timesteps 3007.
Path 276 | total_timesteps 3015.
Path 277 | total_timesteps 3030.
Path 278 | total_timesteps 3042.
Path 279 | total_timesteps 3050.
Path 280 | total_timesteps 3058.
Path 281 | total_timesteps 3067.
Path 282 | total_timesteps 3077.
Path 283 | total_timesteps 3083.
Path 284 | total_timesteps 3093.
Path 285 | total_timesteps 3104.
Path 286 | total_timesteps 3120.
Path 287 | total_timesteps 3130.
Path 288 | total_timesteps 3147.
Path 289 | total_timesteps 3164.
Path 290 | total_timesteps 3176.
Path 291 | total_timesteps 3189.
Path 292 | total_timesteps 3198.
Path 293 | total_timesteps 3210.
Path 294 | total_timesteps 3221.
Path 295 | total_timesteps 3233.
Path 296 | total_timesteps 3245.
Path 297 | total_timesteps 3253.
Path 298 | total_timesteps 3266.
Path 299 | total_timesteps 3272.
Path 300 | total_timesteps 3289.
Path 301 | total_timesteps 3299.
Path 302 | total_timesteps 3328.
Path 303 | total_timesteps 3338.
Path 304 | total_timesteps 3350.
Path 305 | total_timesteps 3359.
Path 306 | total_timesteps 3369.
Path 307 | total_timesteps 3379.
Path 308 | total_timesteps 3387.
Path 309 | total_timesteps 3396.
Path 310 | total_timesteps 3405.
Path 311 | total_timesteps 3412.
Path 312 | total_timesteps 3422.
Path 313 | total_timesteps 3435.
Path 314 | total_timesteps 3442.
Path 315 | total_timesteps 3457.
Path 316 | total_timesteps 3466.
Path 317 | total_timesteps 3474.
Path 318 | total_timesteps 3484.
Path 319 | total_timesteps 3500.
Path 320 | total_timesteps 3508.
Path 321 | total_timesteps 3516.
Path 322 | total_timesteps 3535.
Path 323 | total_timesteps 3543.
Path 324 | total_timesteps 3559.
Path 325 | total_timesteps 3570.
Path 326 | total_timesteps 3579.
Path 327 | total_timesteps 3588.
Path 328 | total_timesteps 3605.
Path 329 | total_timesteps 3615.
Path 330 | total_timesteps 3630.
Path 331 | total_timesteps 3640.
Path 332 | total_timesteps 3651.
Path 333 | total_timesteps 3662.
Path 334 | total_timesteps 3672.
Path 335 | total_timesteps 3682.
Path 336 | total_timesteps 3691.
Path 337 | total_timesteps 3704.
Path 338 | total_timesteps 3715.
Path 339 | total_timesteps 3728.
Path 340 | total_timesteps 3735.
Path 341 | total_timesteps 3745.
Path 342 | total_timesteps 3754.
Path 343 | total_timesteps 3766.
Path 344 | total_timesteps 3780.
Path 345 | total_timesteps 3789.
Path 346 | total_timesteps 3800.
Path 347 | total_timesteps 3809.
Path 348 | total_timesteps 3819.
Path 349 | total_timesteps 3828.
Path 350 | total_timesteps 3838.
Path 351 | total_timesteps 3847.
Path 352 | total_timesteps 3857.
Path 353 | total_timesteps 3870.
Path 354 | total_timesteps 3877.
Path 355 | total_timesteps 3886.
Path 356 | total_timesteps 3896.
Path 357 | total_timesteps 3904.
Path 358 | total_timesteps 3915.
Path 359 | total_timesteps 3926.
Path 360 | total_timesteps 3934.
Path 361 | total_timesteps 3942.
Path 362 | total_timesteps 3953.
Path 363 | total_timesteps 3963.
Path 364 | total_timesteps 3977.
Path 365 | total_timesteps 3983.
Path 366 | total_timesteps 3989.
Path 367 | total_timesteps 4000.
Path 368 | total_timesteps 4008.
Path 369 | total_timesteps 4018.
Path 370 | total_timesteps 4029.
Path 371 | total_timesteps 4046.
Path 372 | total_timesteps 4054.
Path 373 | total_timesteps 4070.
Path 374 | total_timesteps 4079.
Path 375 | total_timesteps 4089.
Path 376 | total_timesteps 4101.
Path 377 | total_timesteps 4111.
Path 378 | total_timesteps 4121.
Path 379 | total_timesteps 4131.
Path 380 | total_timesteps 4142.
Path 381 | total_timesteps 4151.
Path 382 | total_timesteps 4165.
Path 383 | total_timesteps 4177.
Path 384 | total_timesteps 4189.
Path 385 | total_timesteps 4202.
Path 386 | total_timesteps 4211.
Path 387 | total_timesteps 4226.
Path 388 | total_timesteps 4240.
Path 389 | total_timesteps 4248.
Path 390 | total_timesteps 4262.
Path 391 | total_timesteps 4272.
Path 392 | total_timesteps 4291.
Path 393 | total_timesteps 4302.
Path 394 | total_timesteps 4315.
Path 395 | total_timesteps 4329.
Path 396 | total_timesteps 4340.
Path 397 | total_timesteps 4348.
Path 398 | total_timesteps 4356.
Path 399 | total_timesteps 4364.
Path 400 | total_timesteps 4372.
Path 401 | total_timesteps 4382.
Path 402 | total_timesteps 4400.
Path 403 | total_timesteps 4417.
Path 404 | total_timesteps 4439.
Path 405 | total_timesteps 4455.
Path 406 | total_timesteps 4463.
Path 407 | total_timesteps 4475.
Path 408 | total_timesteps 4483.
Path 409 | total_timesteps 4498.
Path 410 | total_timesteps 4510.
Path 411 | total_timesteps 4519.
Path 412 | total_timesteps 4526.
Path 413 | total_timesteps 4536.
Path 414 | total_timesteps 4547.
Path 415 | total_timesteps 4557.
Path 416 | total_timesteps 4569.
Path 417 | total_timesteps 4582.
Path 418 | total_timesteps 4593.
Path 419 | total_timesteps 4604.
Path 420 | total_timesteps 4612.
Path 421 | total_timesteps 4622.
Path 422 | total_timesteps 4634.
Path 423 | total_timesteps 4647.
Path 424 | total_timesteps 4663.
Path 425 | total_timesteps 4679.
Path 426 | total_timesteps 4691.
Path 427 | total_timesteps 4702.
Path 428 | total_timesteps 4711.
Path 429 | total_timesteps 4719.
Path 430 | total_timesteps 4728.
Path 431 | total_timesteps 4737.
Path 432 | total_timesteps 4748.
Path 433 | total_timesteps 4760.
Path 434 | total_timesteps 4769.
Path 435 | total_timesteps 4780.
Path 436 | total_timesteps 4789.
Path 437 | total_timesteps 4802.
Path 438 | total_timesteps 4814.
Path 439 | total_timesteps 4825.
Path 440 | total_timesteps 4837.
Path 441 | total_timesteps 4844.
Path 442 | total_timesteps 4859.
Path 443 | total_timesteps 4871.
Path 444 | total_timesteps 4880.
Path 445 | total_timesteps 4887.
Path 446 | total_timesteps 4911.
Path 447 | total_timesteps 4920.
Path 448 | total_timesteps 4932.
Path 449 | total_timesteps 4944.
Path 450 | total_timesteps 4952.
Path 451 | total_timesteps 4963.
Path 452 | total_timesteps 4971.
Path 453 | total_timesteps 4981.
Path 454 | total_timesteps 4993.
Path 455 | total_timesteps 5006.
Path 456 | total_timesteps 5013.
Path 457 | total_timesteps 5020.
Path 458 | total_timesteps 5032.
Path 459 | total_timesteps 5039.
Path 460 | total_timesteps 5049.
Path 461 | total_timesteps 5059.
Path 462 | total_timesteps 5069.
Path 463 | total_timesteps 5077.
Path 464 | total_timesteps 5088.
Path 465 | total_timesteps 5098.
Path 466 | total_timesteps 5107.
Path 467 | total_timesteps 5117.
Path 468 | total_timesteps 5126.
Path 469 | total_timesteps 5136.
Path 470 | total_timesteps 5151.
Path 471 | total_timesteps 5164.
Path 472 | total_timesteps 5173.
Path 473 | total_timesteps 5191.
Path 474 | total_timesteps 5199.
Path 475 | total_timesteps 5208.
Path 476 | total_timesteps 5216.
Path 477 | total_timesteps 5225.
Path 478 | total_timesteps 5234.
Path 479 | total_timesteps 5244.
Path 480 | total_timesteps 5262.
Path 481 | total_timesteps 5271.
Path 482 | total_timesteps 5284.
Path 483 | total_timesteps 5294.
Path 484 | total_timesteps 5305.
Path 485 | total_timesteps 5321.
Path 486 | total_timesteps 5337.
Path 487 | total_timesteps 5347.
Path 488 | total_timesteps 5358.
Path 489 | total_timesteps 5376.
Path 490 | total_timesteps 5384.
Path 491 | total_timesteps 5396.
Path 492 | total_timesteps 5404.
Path 493 | total_timesteps 5418.
Path 494 | total_timesteps 5430.
Path 495 | total_timesteps 5443.
Path 496 | total_timesteps 5452.
Path 497 | total_timesteps 5469.
Path 498 | total_timesteps 5482.
Path 499 | total_timesteps 5491.
Path 500 | total_timesteps 5503.
Path 501 | total_timesteps 5515.
Path 502 | total_timesteps 5525.
Path 503 | total_timesteps 5534.
Path 504 | total_timesteps 5554.
Path 505 | total_timesteps 5567.
Path 506 | total_timesteps 5579.
Path 507 | total_timesteps 5599.
Path 508 | total_timesteps 5608.
Path 509 | total_timesteps 5620.
Path 510 | total_timesteps 5635.
Path 511 | total_timesteps 5648.
Path 512 | total_timesteps 5655.
Path 513 | total_timesteps 5664.
Path 514 | total_timesteps 5674.
Path 515 | total_timesteps 5682.
Path 516 | total_timesteps 5689.
Path 517 | total_timesteps 5700.
Path 518 | total_timesteps 5710.
Path 519 | total_timesteps 5719.
Path 520 | total_timesteps 5728.
Path 521 | total_timesteps 5739.
Path 522 | total_timesteps 5754.
Path 523 | total_timesteps 5762.
Path 524 | total_timesteps 5771.
Path 525 | total_timesteps 5780.
Path 526 | total_timesteps 5790.
Path 527 | total_timesteps 5800.
Path 528 | total_timesteps 5811.
Path 529 | total_timesteps 5824.
Path 530 | total_timesteps 5835.
Path 531 | total_timesteps 5846.
Path 532 | total_timesteps 5856.
Path 533 | total_timesteps 5871.
Path 534 | total_timesteps 5882.
Path 535 | total_timesteps 5891.
Path 536 | total_timesteps 5900.
Path 537 | total_timesteps 5914.
Path 538 | total_timesteps 5927.
Path 539 | total_timesteps 5936.
Path 540 | total_timesteps 5951.
Path 541 | total_timesteps 5961.
Path 542 | total_timesteps 5971.
Path 543 | total_timesteps 5979.
Path 544 | total_timesteps 5985.
Path 545 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.71    |
| Iteration     | 13       |
| MaximumReturn | 1.7      |
| MinimumReturn | -19      |
| TotalSamples  | 60048    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.31942641735076904
Validation loss = 0.31573688983917236
Validation loss = 0.3212242126464844
Validation loss = 0.3249216079711914
Validation loss = 0.32405635714530945
Validation loss = 0.3260766267776489
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 41.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 63.
Path 6 | total_timesteps 80.
Path 7 | total_timesteps 92.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 114.
Path 10 | total_timesteps 124.
Path 11 | total_timesteps 148.
Path 12 | total_timesteps 157.
Path 13 | total_timesteps 170.
Path 14 | total_timesteps 180.
Path 15 | total_timesteps 190.
Path 16 | total_timesteps 208.
Path 17 | total_timesteps 216.
Path 18 | total_timesteps 233.
Path 19 | total_timesteps 242.
Path 20 | total_timesteps 250.
Path 21 | total_timesteps 257.
Path 22 | total_timesteps 264.
Path 23 | total_timesteps 272.
Path 24 | total_timesteps 290.
Path 25 | total_timesteps 300.
Path 26 | total_timesteps 308.
Path 27 | total_timesteps 318.
Path 28 | total_timesteps 327.
Path 29 | total_timesteps 336.
Path 30 | total_timesteps 343.
Path 31 | total_timesteps 351.
Path 32 | total_timesteps 359.
Path 33 | total_timesteps 373.
Path 34 | total_timesteps 384.
Path 35 | total_timesteps 393.
Path 36 | total_timesteps 402.
Path 37 | total_timesteps 410.
Path 38 | total_timesteps 420.
Path 39 | total_timesteps 430.
Path 40 | total_timesteps 440.
Path 41 | total_timesteps 454.
Path 42 | total_timesteps 466.
Path 43 | total_timesteps 480.
Path 44 | total_timesteps 491.
Path 45 | total_timesteps 502.
Path 46 | total_timesteps 513.
Path 47 | total_timesteps 525.
Path 48 | total_timesteps 534.
Path 49 | total_timesteps 543.
Path 50 | total_timesteps 562.
Path 51 | total_timesteps 569.
Path 52 | total_timesteps 582.
Path 53 | total_timesteps 589.
Path 54 | total_timesteps 600.
Path 55 | total_timesteps 608.
Path 56 | total_timesteps 627.
Path 57 | total_timesteps 638.
Path 58 | total_timesteps 649.
Path 59 | total_timesteps 658.
Path 60 | total_timesteps 669.
Path 61 | total_timesteps 678.
Path 62 | total_timesteps 690.
Path 63 | total_timesteps 700.
Path 64 | total_timesteps 710.
Path 65 | total_timesteps 725.
Path 66 | total_timesteps 738.
Path 67 | total_timesteps 753.
Path 68 | total_timesteps 761.
Path 69 | total_timesteps 773.
Path 70 | total_timesteps 785.
Path 71 | total_timesteps 794.
Path 72 | total_timesteps 804.
Path 73 | total_timesteps 812.
Path 74 | total_timesteps 821.
Path 75 | total_timesteps 830.
Path 76 | total_timesteps 838.
Path 77 | total_timesteps 850.
Path 78 | total_timesteps 867.
Path 79 | total_timesteps 875.
Path 80 | total_timesteps 885.
Path 81 | total_timesteps 897.
Path 82 | total_timesteps 905.
Path 83 | total_timesteps 929.
Path 84 | total_timesteps 937.
Path 85 | total_timesteps 944.
Path 86 | total_timesteps 951.
Path 87 | total_timesteps 958.
Path 88 | total_timesteps 969.
Path 89 | total_timesteps 980.
Path 90 | total_timesteps 990.
Path 91 | total_timesteps 1002.
Path 92 | total_timesteps 1014.
Path 93 | total_timesteps 1025.
Path 94 | total_timesteps 1034.
Path 95 | total_timesteps 1042.
Path 96 | total_timesteps 1053.
Path 97 | total_timesteps 1060.
Path 98 | total_timesteps 1069.
Path 99 | total_timesteps 1085.
Path 100 | total_timesteps 1099.
Path 101 | total_timesteps 1109.
Path 102 | total_timesteps 1117.
Path 103 | total_timesteps 1125.
Path 104 | total_timesteps 1135.
Path 105 | total_timesteps 1141.
Path 106 | total_timesteps 1150.
Path 107 | total_timesteps 1156.
Path 108 | total_timesteps 1163.
Path 109 | total_timesteps 1172.
Path 110 | total_timesteps 1181.
Path 111 | total_timesteps 1192.
Path 112 | total_timesteps 1204.
Path 113 | total_timesteps 1215.
Path 114 | total_timesteps 1232.
Path 115 | total_timesteps 1245.
Path 116 | total_timesteps 1258.
Path 117 | total_timesteps 1266.
Path 118 | total_timesteps 1286.
Path 119 | total_timesteps 1295.
Path 120 | total_timesteps 1304.
Path 121 | total_timesteps 1312.
Path 122 | total_timesteps 1324.
Path 123 | total_timesteps 1335.
Path 124 | total_timesteps 1342.
Path 125 | total_timesteps 1352.
Path 126 | total_timesteps 1363.
Path 127 | total_timesteps 1373.
Path 128 | total_timesteps 1381.
Path 129 | total_timesteps 1390.
Path 130 | total_timesteps 1400.
Path 131 | total_timesteps 1409.
Path 132 | total_timesteps 1423.
Path 133 | total_timesteps 1435.
Path 134 | total_timesteps 1443.
Path 135 | total_timesteps 1452.
Path 136 | total_timesteps 1463.
Path 137 | total_timesteps 1476.
Path 138 | total_timesteps 1483.
Path 139 | total_timesteps 1494.
Path 140 | total_timesteps 1505.
Path 141 | total_timesteps 1518.
Path 142 | total_timesteps 1537.
Path 143 | total_timesteps 1548.
Path 144 | total_timesteps 1558.
Path 145 | total_timesteps 1568.
Path 146 | total_timesteps 1576.
Path 147 | total_timesteps 1587.
Path 148 | total_timesteps 1596.
Path 149 | total_timesteps 1612.
Path 150 | total_timesteps 1638.
Path 151 | total_timesteps 1653.
Path 152 | total_timesteps 1664.
Path 153 | total_timesteps 1670.
Path 154 | total_timesteps 1680.
Path 155 | total_timesteps 1689.
Path 156 | total_timesteps 1697.
Path 157 | total_timesteps 1705.
Path 158 | total_timesteps 1715.
Path 159 | total_timesteps 1728.
Path 160 | total_timesteps 1737.
Path 161 | total_timesteps 1744.
Path 162 | total_timesteps 1750.
Path 163 | total_timesteps 1758.
Path 164 | total_timesteps 1769.
Path 165 | total_timesteps 1781.
Path 166 | total_timesteps 1789.
Path 167 | total_timesteps 1797.
Path 168 | total_timesteps 1804.
Path 169 | total_timesteps 1822.
Path 170 | total_timesteps 1840.
Path 171 | total_timesteps 1849.
Path 172 | total_timesteps 1858.
Path 173 | total_timesteps 1869.
Path 174 | total_timesteps 1881.
Path 175 | total_timesteps 1909.
Path 176 | total_timesteps 1920.
Path 177 | total_timesteps 1930.
Path 178 | total_timesteps 1941.
Path 179 | total_timesteps 1953.
Path 180 | total_timesteps 1964.
Path 181 | total_timesteps 1973.
Path 182 | total_timesteps 1982.
Path 183 | total_timesteps 1992.
Path 184 | total_timesteps 2001.
Path 185 | total_timesteps 2010.
Path 186 | total_timesteps 2025.
Path 187 | total_timesteps 2035.
Path 188 | total_timesteps 2048.
Path 189 | total_timesteps 2061.
Path 190 | total_timesteps 2072.
Path 191 | total_timesteps 2082.
Path 192 | total_timesteps 2094.
Path 193 | total_timesteps 2105.
Path 194 | total_timesteps 2121.
Path 195 | total_timesteps 2128.
Path 196 | total_timesteps 2136.
Path 197 | total_timesteps 2144.
Path 198 | total_timesteps 2157.
Path 199 | total_timesteps 2169.
Path 200 | total_timesteps 2178.
Path 201 | total_timesteps 2190.
Path 202 | total_timesteps 2203.
Path 203 | total_timesteps 2218.
Path 204 | total_timesteps 2236.
Path 205 | total_timesteps 2243.
Path 206 | total_timesteps 2252.
Path 207 | total_timesteps 2264.
Path 208 | total_timesteps 2298.
Path 209 | total_timesteps 2308.
Path 210 | total_timesteps 2317.
Path 211 | total_timesteps 2331.
Path 212 | total_timesteps 2344.
Path 213 | total_timesteps 2351.
Path 214 | total_timesteps 2359.
Path 215 | total_timesteps 2369.
Path 216 | total_timesteps 2377.
Path 217 | total_timesteps 2387.
Path 218 | total_timesteps 2397.
Path 219 | total_timesteps 2408.
Path 220 | total_timesteps 2419.
Path 221 | total_timesteps 2427.
Path 222 | total_timesteps 2436.
Path 223 | total_timesteps 2446.
Path 224 | total_timesteps 2458.
Path 225 | total_timesteps 2469.
Path 226 | total_timesteps 2478.
Path 227 | total_timesteps 2485.
Path 228 | total_timesteps 2498.
Path 229 | total_timesteps 2508.
Path 230 | total_timesteps 2517.
Path 231 | total_timesteps 2523.
Path 232 | total_timesteps 2532.
Path 233 | total_timesteps 2548.
Path 234 | total_timesteps 2558.
Path 235 | total_timesteps 2569.
Path 236 | total_timesteps 2577.
Path 237 | total_timesteps 2585.
Path 238 | total_timesteps 2594.
Path 239 | total_timesteps 2608.
Path 240 | total_timesteps 2615.
Path 241 | total_timesteps 2625.
Path 242 | total_timesteps 2636.
Path 243 | total_timesteps 2648.
Path 244 | total_timesteps 2655.
Path 245 | total_timesteps 2668.
Path 246 | total_timesteps 2681.
Path 247 | total_timesteps 2690.
Path 248 | total_timesteps 2698.
Path 249 | total_timesteps 2705.
Path 250 | total_timesteps 2721.
Path 251 | total_timesteps 2737.
Path 252 | total_timesteps 2748.
Path 253 | total_timesteps 2757.
Path 254 | total_timesteps 2768.
Path 255 | total_timesteps 2780.
Path 256 | total_timesteps 2790.
Path 257 | total_timesteps 2799.
Path 258 | total_timesteps 2818.
Path 259 | total_timesteps 2827.
Path 260 | total_timesteps 2839.
Path 261 | total_timesteps 2849.
Path 262 | total_timesteps 2857.
Path 263 | total_timesteps 2867.
Path 264 | total_timesteps 2879.
Path 265 | total_timesteps 2897.
Path 266 | total_timesteps 2906.
Path 267 | total_timesteps 2915.
Path 268 | total_timesteps 2926.
Path 269 | total_timesteps 2935.
Path 270 | total_timesteps 2945.
Path 271 | total_timesteps 2957.
Path 272 | total_timesteps 2966.
Path 273 | total_timesteps 2977.
Path 274 | total_timesteps 2989.
Path 275 | total_timesteps 3000.
Path 276 | total_timesteps 3014.
Path 277 | total_timesteps 3026.
Path 278 | total_timesteps 3036.
Path 279 | total_timesteps 3046.
Path 280 | total_timesteps 3054.
Path 281 | total_timesteps 3064.
Path 282 | total_timesteps 3076.
Path 283 | total_timesteps 3093.
Path 284 | total_timesteps 3104.
Path 285 | total_timesteps 3116.
Path 286 | total_timesteps 3125.
Path 287 | total_timesteps 3135.
Path 288 | total_timesteps 3145.
Path 289 | total_timesteps 3154.
Path 290 | total_timesteps 3165.
Path 291 | total_timesteps 3175.
Path 292 | total_timesteps 3185.
Path 293 | total_timesteps 3193.
Path 294 | total_timesteps 3200.
Path 295 | total_timesteps 3214.
Path 296 | total_timesteps 3226.
Path 297 | total_timesteps 3250.
Path 298 | total_timesteps 3262.
Path 299 | total_timesteps 3272.
Path 300 | total_timesteps 3281.
Path 301 | total_timesteps 3289.
Path 302 | total_timesteps 3297.
Path 303 | total_timesteps 3308.
Path 304 | total_timesteps 3319.
Path 305 | total_timesteps 3332.
Path 306 | total_timesteps 3342.
Path 307 | total_timesteps 3350.
Path 308 | total_timesteps 3359.
Path 309 | total_timesteps 3371.
Path 310 | total_timesteps 3385.
Path 311 | total_timesteps 3395.
Path 312 | total_timesteps 3403.
Path 313 | total_timesteps 3410.
Path 314 | total_timesteps 3422.
Path 315 | total_timesteps 3436.
Path 316 | total_timesteps 3449.
Path 317 | total_timesteps 3456.
Path 318 | total_timesteps 3468.
Path 319 | total_timesteps 3483.
Path 320 | total_timesteps 3493.
Path 321 | total_timesteps 3502.
Path 322 | total_timesteps 3511.
Path 323 | total_timesteps 3524.
Path 324 | total_timesteps 3536.
Path 325 | total_timesteps 3553.
Path 326 | total_timesteps 3563.
Path 327 | total_timesteps 3573.
Path 328 | total_timesteps 3586.
Path 329 | total_timesteps 3598.
Path 330 | total_timesteps 3609.
Path 331 | total_timesteps 3617.
Path 332 | total_timesteps 3624.
Path 333 | total_timesteps 3634.
Path 334 | total_timesteps 3645.
Path 335 | total_timesteps 3653.
Path 336 | total_timesteps 3663.
Path 337 | total_timesteps 3674.
Path 338 | total_timesteps 3682.
Path 339 | total_timesteps 3693.
Path 340 | total_timesteps 3700.
Path 341 | total_timesteps 3719.
Path 342 | total_timesteps 3736.
Path 343 | total_timesteps 3745.
Path 344 | total_timesteps 3755.
Path 345 | total_timesteps 3763.
Path 346 | total_timesteps 3777.
Path 347 | total_timesteps 3785.
Path 348 | total_timesteps 3794.
Path 349 | total_timesteps 3802.
Path 350 | total_timesteps 3810.
Path 351 | total_timesteps 3821.
Path 352 | total_timesteps 3829.
Path 353 | total_timesteps 3840.
Path 354 | total_timesteps 3849.
Path 355 | total_timesteps 3857.
Path 356 | total_timesteps 3866.
Path 357 | total_timesteps 3874.
Path 358 | total_timesteps 3883.
Path 359 | total_timesteps 3891.
Path 360 | total_timesteps 3902.
Path 361 | total_timesteps 3912.
Path 362 | total_timesteps 3923.
Path 363 | total_timesteps 3935.
Path 364 | total_timesteps 3947.
Path 365 | total_timesteps 3961.
Path 366 | total_timesteps 3973.
Path 367 | total_timesteps 4004.
Path 368 | total_timesteps 4015.
Path 369 | total_timesteps 4025.
Path 370 | total_timesteps 4036.
Path 371 | total_timesteps 4044.
Path 372 | total_timesteps 4055.
Path 373 | total_timesteps 4065.
Path 374 | total_timesteps 4073.
Path 375 | total_timesteps 4085.
Path 376 | total_timesteps 4093.
Path 377 | total_timesteps 4107.
Path 378 | total_timesteps 4122.
Path 379 | total_timesteps 4131.
Path 380 | total_timesteps 4141.
Path 381 | total_timesteps 4153.
Path 382 | total_timesteps 4161.
Path 383 | total_timesteps 4172.
Path 384 | total_timesteps 4186.
Path 385 | total_timesteps 4198.
Path 386 | total_timesteps 4209.
Path 387 | total_timesteps 4216.
Path 388 | total_timesteps 4226.
Path 389 | total_timesteps 4239.
Path 390 | total_timesteps 4248.
Path 391 | total_timesteps 4259.
Path 392 | total_timesteps 4269.
Path 393 | total_timesteps 4281.
Path 394 | total_timesteps 4289.
Path 395 | total_timesteps 4299.
Path 396 | total_timesteps 4316.
Path 397 | total_timesteps 4330.
Path 398 | total_timesteps 4339.
Path 399 | total_timesteps 4351.
Path 400 | total_timesteps 4360.
Path 401 | total_timesteps 4370.
Path 402 | total_timesteps 4384.
Path 403 | total_timesteps 4391.
Path 404 | total_timesteps 4398.
Path 405 | total_timesteps 4409.
Path 406 | total_timesteps 4420.
Path 407 | total_timesteps 4430.
Path 408 | total_timesteps 4436.
Path 409 | total_timesteps 4444.
Path 410 | total_timesteps 4451.
Path 411 | total_timesteps 4460.
Path 412 | total_timesteps 4468.
Path 413 | total_timesteps 4476.
Path 414 | total_timesteps 4486.
Path 415 | total_timesteps 4497.
Path 416 | total_timesteps 4505.
Path 417 | total_timesteps 4515.
Path 418 | total_timesteps 4529.
Path 419 | total_timesteps 4541.
Path 420 | total_timesteps 4549.
Path 421 | total_timesteps 4557.
Path 422 | total_timesteps 4569.
Path 423 | total_timesteps 4578.
Path 424 | total_timesteps 4590.
Path 425 | total_timesteps 4609.
Path 426 | total_timesteps 4618.
Path 427 | total_timesteps 4626.
Path 428 | total_timesteps 4636.
Path 429 | total_timesteps 4646.
Path 430 | total_timesteps 4658.
Path 431 | total_timesteps 4669.
Path 432 | total_timesteps 4683.
Path 433 | total_timesteps 4692.
Path 434 | total_timesteps 4703.
Path 435 | total_timesteps 4715.
Path 436 | total_timesteps 4731.
Path 437 | total_timesteps 4742.
Path 438 | total_timesteps 4751.
Path 439 | total_timesteps 4761.
Path 440 | total_timesteps 4771.
Path 441 | total_timesteps 4778.
Path 442 | total_timesteps 4789.
Path 443 | total_timesteps 4796.
Path 444 | total_timesteps 4806.
Path 445 | total_timesteps 4816.
Path 446 | total_timesteps 4829.
Path 447 | total_timesteps 4836.
Path 448 | total_timesteps 4850.
Path 449 | total_timesteps 4860.
Path 450 | total_timesteps 4867.
Path 451 | total_timesteps 4879.
Path 452 | total_timesteps 4889.
Path 453 | total_timesteps 4901.
Path 454 | total_timesteps 4911.
Path 455 | total_timesteps 4925.
Path 456 | total_timesteps 4935.
Path 457 | total_timesteps 4944.
Path 458 | total_timesteps 4954.
Path 459 | total_timesteps 4964.
Path 460 | total_timesteps 4978.
Path 461 | total_timesteps 4986.
Path 462 | total_timesteps 4996.
Path 463 | total_timesteps 5003.
Path 464 | total_timesteps 5013.
Path 465 | total_timesteps 5021.
Path 466 | total_timesteps 5031.
Path 467 | total_timesteps 5040.
Path 468 | total_timesteps 5049.
Path 469 | total_timesteps 5061.
Path 470 | total_timesteps 5070.
Path 471 | total_timesteps 5083.
Path 472 | total_timesteps 5090.
Path 473 | total_timesteps 5101.
Path 474 | total_timesteps 5112.
Path 475 | total_timesteps 5121.
Path 476 | total_timesteps 5129.
Path 477 | total_timesteps 5136.
Path 478 | total_timesteps 5145.
Path 479 | total_timesteps 5154.
Path 480 | total_timesteps 5164.
Path 481 | total_timesteps 5173.
Path 482 | total_timesteps 5182.
Path 483 | total_timesteps 5194.
Path 484 | total_timesteps 5210.
Path 485 | total_timesteps 5218.
Path 486 | total_timesteps 5227.
Path 487 | total_timesteps 5238.
Path 488 | total_timesteps 5247.
Path 489 | total_timesteps 5257.
Path 490 | total_timesteps 5278.
Path 491 | total_timesteps 5286.
Path 492 | total_timesteps 5296.
Path 493 | total_timesteps 5304.
Path 494 | total_timesteps 5312.
Path 495 | total_timesteps 5323.
Path 496 | total_timesteps 5335.
Path 497 | total_timesteps 5344.
Path 498 | total_timesteps 5354.
Path 499 | total_timesteps 5363.
Path 500 | total_timesteps 5371.
Path 501 | total_timesteps 5381.
Path 502 | total_timesteps 5406.
Path 503 | total_timesteps 5415.
Path 504 | total_timesteps 5430.
Path 505 | total_timesteps 5442.
Path 506 | total_timesteps 5452.
Path 507 | total_timesteps 5463.
Path 508 | total_timesteps 5478.
Path 509 | total_timesteps 5487.
Path 510 | total_timesteps 5500.
Path 511 | total_timesteps 5507.
Path 512 | total_timesteps 5515.
Path 513 | total_timesteps 5523.
Path 514 | total_timesteps 5537.
Path 515 | total_timesteps 5550.
Path 516 | total_timesteps 5562.
Path 517 | total_timesteps 5574.
Path 518 | total_timesteps 5582.
Path 519 | total_timesteps 5594.
Path 520 | total_timesteps 5603.
Path 521 | total_timesteps 5615.
Path 522 | total_timesteps 5628.
Path 523 | total_timesteps 5640.
Path 524 | total_timesteps 5649.
Path 525 | total_timesteps 5658.
Path 526 | total_timesteps 5669.
Path 527 | total_timesteps 5679.
Path 528 | total_timesteps 5688.
Path 529 | total_timesteps 5696.
Path 530 | total_timesteps 5703.
Path 531 | total_timesteps 5717.
Path 532 | total_timesteps 5725.
Path 533 | total_timesteps 5737.
Path 534 | total_timesteps 5745.
Path 535 | total_timesteps 5757.
Path 536 | total_timesteps 5767.
Path 537 | total_timesteps 5780.
Path 538 | total_timesteps 5795.
Path 539 | total_timesteps 5805.
Path 540 | total_timesteps 5814.
Path 541 | total_timesteps 5825.
Path 542 | total_timesteps 5835.
Path 543 | total_timesteps 5846.
Path 544 | total_timesteps 5853.
Path 545 | total_timesteps 5862.
Path 546 | total_timesteps 5869.
Path 547 | total_timesteps 5875.
Path 548 | total_timesteps 5888.
Path 549 | total_timesteps 5896.
Path 550 | total_timesteps 5906.
Path 551 | total_timesteps 5916.
Path 552 | total_timesteps 5932.
Path 553 | total_timesteps 5941.
Path 554 | total_timesteps 5948.
Path 555 | total_timesteps 5954.
Path 556 | total_timesteps 5984.
Path 557 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.06    |
| Iteration     | 14       |
| MaximumReturn | 3.73     |
| MinimumReturn | -19.7    |
| TotalSamples  | 64054    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.31358230113983154
Validation loss = 0.3164355456829071
Validation loss = 0.32179081439971924
Validation loss = 0.32142847776412964
Validation loss = 0.3205336332321167
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 40.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 67.
Path 7 | total_timesteps 79.
Path 8 | total_timesteps 89.
Path 9 | total_timesteps 101.
Path 10 | total_timesteps 113.
Path 11 | total_timesteps 123.
Path 12 | total_timesteps 134.
Path 13 | total_timesteps 146.
Path 14 | total_timesteps 156.
Path 15 | total_timesteps 165.
Path 16 | total_timesteps 172.
Path 17 | total_timesteps 182.
Path 18 | total_timesteps 194.
Path 19 | total_timesteps 201.
Path 20 | total_timesteps 213.
Path 21 | total_timesteps 222.
Path 22 | total_timesteps 230.
Path 23 | total_timesteps 239.
Path 24 | total_timesteps 249.
Path 25 | total_timesteps 258.
Path 26 | total_timesteps 267.
Path 27 | total_timesteps 274.
Path 28 | total_timesteps 286.
Path 29 | total_timesteps 297.
Path 30 | total_timesteps 312.
Path 31 | total_timesteps 322.
Path 32 | total_timesteps 331.
Path 33 | total_timesteps 341.
Path 34 | total_timesteps 348.
Path 35 | total_timesteps 359.
Path 36 | total_timesteps 369.
Path 37 | total_timesteps 379.
Path 38 | total_timesteps 389.
Path 39 | total_timesteps 396.
Path 40 | total_timesteps 406.
Path 41 | total_timesteps 419.
Path 42 | total_timesteps 433.
Path 43 | total_timesteps 445.
Path 44 | total_timesteps 453.
Path 45 | total_timesteps 463.
Path 46 | total_timesteps 473.
Path 47 | total_timesteps 484.
Path 48 | total_timesteps 493.
Path 49 | total_timesteps 502.
Path 50 | total_timesteps 511.
Path 51 | total_timesteps 524.
Path 52 | total_timesteps 545.
Path 53 | total_timesteps 556.
Path 54 | total_timesteps 568.
Path 55 | total_timesteps 583.
Path 56 | total_timesteps 592.
Path 57 | total_timesteps 603.
Path 58 | total_timesteps 614.
Path 59 | total_timesteps 623.
Path 60 | total_timesteps 634.
Path 61 | total_timesteps 645.
Path 62 | total_timesteps 654.
Path 63 | total_timesteps 664.
Path 64 | total_timesteps 679.
Path 65 | total_timesteps 688.
Path 66 | total_timesteps 705.
Path 67 | total_timesteps 715.
Path 68 | total_timesteps 726.
Path 69 | total_timesteps 738.
Path 70 | total_timesteps 751.
Path 71 | total_timesteps 765.
Path 72 | total_timesteps 776.
Path 73 | total_timesteps 786.
Path 74 | total_timesteps 795.
Path 75 | total_timesteps 806.
Path 76 | total_timesteps 814.
Path 77 | total_timesteps 830.
Path 78 | total_timesteps 839.
Path 79 | total_timesteps 852.
Path 80 | total_timesteps 866.
Path 81 | total_timesteps 875.
Path 82 | total_timesteps 885.
Path 83 | total_timesteps 896.
Path 84 | total_timesteps 914.
Path 85 | total_timesteps 922.
Path 86 | total_timesteps 938.
Path 87 | total_timesteps 950.
Path 88 | total_timesteps 959.
Path 89 | total_timesteps 974.
Path 90 | total_timesteps 990.
Path 91 | total_timesteps 999.
Path 92 | total_timesteps 1008.
Path 93 | total_timesteps 1020.
Path 94 | total_timesteps 1036.
Path 95 | total_timesteps 1045.
Path 96 | total_timesteps 1053.
Path 97 | total_timesteps 1063.
Path 98 | total_timesteps 1073.
Path 99 | total_timesteps 1080.
Path 100 | total_timesteps 1091.
Path 101 | total_timesteps 1098.
Path 102 | total_timesteps 1108.
Path 103 | total_timesteps 1121.
Path 104 | total_timesteps 1127.
Path 105 | total_timesteps 1141.
Path 106 | total_timesteps 1150.
Path 107 | total_timesteps 1156.
Path 108 | total_timesteps 1166.
Path 109 | total_timesteps 1181.
Path 110 | total_timesteps 1192.
Path 111 | total_timesteps 1201.
Path 112 | total_timesteps 1211.
Path 113 | total_timesteps 1220.
Path 114 | total_timesteps 1231.
Path 115 | total_timesteps 1242.
Path 116 | total_timesteps 1251.
Path 117 | total_timesteps 1260.
Path 118 | total_timesteps 1268.
Path 119 | total_timesteps 1278.
Path 120 | total_timesteps 1290.
Path 121 | total_timesteps 1306.
Path 122 | total_timesteps 1318.
Path 123 | total_timesteps 1328.
Path 124 | total_timesteps 1337.
Path 125 | total_timesteps 1345.
Path 126 | total_timesteps 1360.
Path 127 | total_timesteps 1372.
Path 128 | total_timesteps 1379.
Path 129 | total_timesteps 1386.
Path 130 | total_timesteps 1397.
Path 131 | total_timesteps 1404.
Path 132 | total_timesteps 1419.
Path 133 | total_timesteps 1427.
Path 134 | total_timesteps 1433.
Path 135 | total_timesteps 1443.
Path 136 | total_timesteps 1463.
Path 137 | total_timesteps 1476.
Path 138 | total_timesteps 1486.
Path 139 | total_timesteps 1502.
Path 140 | total_timesteps 1517.
Path 141 | total_timesteps 1534.
Path 142 | total_timesteps 1545.
Path 143 | total_timesteps 1555.
Path 144 | total_timesteps 1563.
Path 145 | total_timesteps 1572.
Path 146 | total_timesteps 1583.
Path 147 | total_timesteps 1592.
Path 148 | total_timesteps 1606.
Path 149 | total_timesteps 1617.
Path 150 | total_timesteps 1624.
Path 151 | total_timesteps 1631.
Path 152 | total_timesteps 1638.
Path 153 | total_timesteps 1647.
Path 154 | total_timesteps 1656.
Path 155 | total_timesteps 1665.
Path 156 | total_timesteps 1677.
Path 157 | total_timesteps 1686.
Path 158 | total_timesteps 1694.
Path 159 | total_timesteps 1707.
Path 160 | total_timesteps 1718.
Path 161 | total_timesteps 1728.
Path 162 | total_timesteps 1738.
Path 163 | total_timesteps 1746.
Path 164 | total_timesteps 1762.
Path 165 | total_timesteps 1770.
Path 166 | total_timesteps 1781.
Path 167 | total_timesteps 1790.
Path 168 | total_timesteps 1798.
Path 169 | total_timesteps 1811.
Path 170 | total_timesteps 1827.
Path 171 | total_timesteps 1837.
Path 172 | total_timesteps 1851.
Path 173 | total_timesteps 1861.
Path 174 | total_timesteps 1869.
Path 175 | total_timesteps 1878.
Path 176 | total_timesteps 1895.
Path 177 | total_timesteps 1905.
Path 178 | total_timesteps 1922.
Path 179 | total_timesteps 1931.
Path 180 | total_timesteps 1945.
Path 181 | total_timesteps 1952.
Path 182 | total_timesteps 1967.
Path 183 | total_timesteps 1978.
Path 184 | total_timesteps 1987.
Path 185 | total_timesteps 2001.
Path 186 | total_timesteps 2009.
Path 187 | total_timesteps 2020.
Path 188 | total_timesteps 2033.
Path 189 | total_timesteps 2053.
Path 190 | total_timesteps 2062.
Path 191 | total_timesteps 2072.
Path 192 | total_timesteps 2086.
Path 193 | total_timesteps 2094.
Path 194 | total_timesteps 2105.
Path 195 | total_timesteps 2112.
Path 196 | total_timesteps 2131.
Path 197 | total_timesteps 2149.
Path 198 | total_timesteps 2161.
Path 199 | total_timesteps 2169.
Path 200 | total_timesteps 2180.
Path 201 | total_timesteps 2190.
Path 202 | total_timesteps 2198.
Path 203 | total_timesteps 2209.
Path 204 | total_timesteps 2218.
Path 205 | total_timesteps 2228.
Path 206 | total_timesteps 2239.
Path 207 | total_timesteps 2248.
Path 208 | total_timesteps 2259.
Path 209 | total_timesteps 2266.
Path 210 | total_timesteps 2278.
Path 211 | total_timesteps 2287.
Path 212 | total_timesteps 2299.
Path 213 | total_timesteps 2311.
Path 214 | total_timesteps 2327.
Path 215 | total_timesteps 2336.
Path 216 | total_timesteps 2346.
Path 217 | total_timesteps 2352.
Path 218 | total_timesteps 2363.
Path 219 | total_timesteps 2375.
Path 220 | total_timesteps 2390.
Path 221 | total_timesteps 2408.
Path 222 | total_timesteps 2417.
Path 223 | total_timesteps 2429.
Path 224 | total_timesteps 2444.
Path 225 | total_timesteps 2452.
Path 226 | total_timesteps 2465.
Path 227 | total_timesteps 2479.
Path 228 | total_timesteps 2492.
Path 229 | total_timesteps 2503.
Path 230 | total_timesteps 2514.
Path 231 | total_timesteps 2522.
Path 232 | total_timesteps 2531.
Path 233 | total_timesteps 2541.
Path 234 | total_timesteps 2549.
Path 235 | total_timesteps 2559.
Path 236 | total_timesteps 2569.
Path 237 | total_timesteps 2583.
Path 238 | total_timesteps 2594.
Path 239 | total_timesteps 2605.
Path 240 | total_timesteps 2613.
Path 241 | total_timesteps 2627.
Path 242 | total_timesteps 2636.
Path 243 | total_timesteps 2654.
Path 244 | total_timesteps 2669.
Path 245 | total_timesteps 2680.
Path 246 | total_timesteps 2689.
Path 247 | total_timesteps 2699.
Path 248 | total_timesteps 2708.
Path 249 | total_timesteps 2718.
Path 250 | total_timesteps 2730.
Path 251 | total_timesteps 2743.
Path 252 | total_timesteps 2755.
Path 253 | total_timesteps 2765.
Path 254 | total_timesteps 2774.
Path 255 | total_timesteps 2784.
Path 256 | total_timesteps 2803.
Path 257 | total_timesteps 2814.
Path 258 | total_timesteps 2827.
Path 259 | total_timesteps 2835.
Path 260 | total_timesteps 2843.
Path 261 | total_timesteps 2857.
Path 262 | total_timesteps 2870.
Path 263 | total_timesteps 2877.
Path 264 | total_timesteps 2886.
Path 265 | total_timesteps 2895.
Path 266 | total_timesteps 2902.
Path 267 | total_timesteps 2912.
Path 268 | total_timesteps 2923.
Path 269 | total_timesteps 2931.
Path 270 | total_timesteps 2943.
Path 271 | total_timesteps 2954.
Path 272 | total_timesteps 2967.
Path 273 | total_timesteps 2983.
Path 274 | total_timesteps 2994.
Path 275 | total_timesteps 3013.
Path 276 | total_timesteps 3022.
Path 277 | total_timesteps 3032.
Path 278 | total_timesteps 3043.
Path 279 | total_timesteps 3056.
Path 280 | total_timesteps 3067.
Path 281 | total_timesteps 3082.
Path 282 | total_timesteps 3092.
Path 283 | total_timesteps 3103.
Path 284 | total_timesteps 3123.
Path 285 | total_timesteps 3132.
Path 286 | total_timesteps 3142.
Path 287 | total_timesteps 3151.
Path 288 | total_timesteps 3164.
Path 289 | total_timesteps 3178.
Path 290 | total_timesteps 3193.
Path 291 | total_timesteps 3200.
Path 292 | total_timesteps 3212.
Path 293 | total_timesteps 3222.
Path 294 | total_timesteps 3230.
Path 295 | total_timesteps 3239.
Path 296 | total_timesteps 3248.
Path 297 | total_timesteps 3261.
Path 298 | total_timesteps 3278.
Path 299 | total_timesteps 3289.
Path 300 | total_timesteps 3304.
Path 301 | total_timesteps 3321.
Path 302 | total_timesteps 3328.
Path 303 | total_timesteps 3341.
Path 304 | total_timesteps 3349.
Path 305 | total_timesteps 3362.
Path 306 | total_timesteps 3375.
Path 307 | total_timesteps 3385.
Path 308 | total_timesteps 3392.
Path 309 | total_timesteps 3402.
Path 310 | total_timesteps 3413.
Path 311 | total_timesteps 3421.
Path 312 | total_timesteps 3428.
Path 313 | total_timesteps 3435.
Path 314 | total_timesteps 3442.
Path 315 | total_timesteps 3452.
Path 316 | total_timesteps 3463.
Path 317 | total_timesteps 3481.
Path 318 | total_timesteps 3491.
Path 319 | total_timesteps 3502.
Path 320 | total_timesteps 3516.
Path 321 | total_timesteps 3527.
Path 322 | total_timesteps 3541.
Path 323 | total_timesteps 3551.
Path 324 | total_timesteps 3559.
Path 325 | total_timesteps 3568.
Path 326 | total_timesteps 3576.
Path 327 | total_timesteps 3585.
Path 328 | total_timesteps 3594.
Path 329 | total_timesteps 3605.
Path 330 | total_timesteps 3619.
Path 331 | total_timesteps 3629.
Path 332 | total_timesteps 3645.
Path 333 | total_timesteps 3656.
Path 334 | total_timesteps 3669.
Path 335 | total_timesteps 3679.
Path 336 | total_timesteps 3688.
Path 337 | total_timesteps 3700.
Path 338 | total_timesteps 3706.
Path 339 | total_timesteps 3716.
Path 340 | total_timesteps 3725.
Path 341 | total_timesteps 3737.
Path 342 | total_timesteps 3748.
Path 343 | total_timesteps 3757.
Path 344 | total_timesteps 3787.
Path 345 | total_timesteps 3794.
Path 346 | total_timesteps 3807.
Path 347 | total_timesteps 3818.
Path 348 | total_timesteps 3830.
Path 349 | total_timesteps 3841.
Path 350 | total_timesteps 3851.
Path 351 | total_timesteps 3867.
Path 352 | total_timesteps 3885.
Path 353 | total_timesteps 3892.
Path 354 | total_timesteps 3904.
Path 355 | total_timesteps 3918.
Path 356 | total_timesteps 3927.
Path 357 | total_timesteps 3938.
Path 358 | total_timesteps 3954.
Path 359 | total_timesteps 3967.
Path 360 | total_timesteps 3978.
Path 361 | total_timesteps 3987.
Path 362 | total_timesteps 4005.
Path 363 | total_timesteps 4018.
Path 364 | total_timesteps 4027.
Path 365 | total_timesteps 4033.
Path 366 | total_timesteps 4040.
Path 367 | total_timesteps 4052.
Path 368 | total_timesteps 4061.
Path 369 | total_timesteps 4072.
Path 370 | total_timesteps 4081.
Path 371 | total_timesteps 4090.
Path 372 | total_timesteps 4099.
Path 373 | total_timesteps 4113.
Path 374 | total_timesteps 4122.
Path 375 | total_timesteps 4132.
Path 376 | total_timesteps 4142.
Path 377 | total_timesteps 4152.
Path 378 | total_timesteps 4163.
Path 379 | total_timesteps 4171.
Path 380 | total_timesteps 4182.
Path 381 | total_timesteps 4199.
Path 382 | total_timesteps 4208.
Path 383 | total_timesteps 4219.
Path 384 | total_timesteps 4229.
Path 385 | total_timesteps 4238.
Path 386 | total_timesteps 4246.
Path 387 | total_timesteps 4259.
Path 388 | total_timesteps 4268.
Path 389 | total_timesteps 4278.
Path 390 | total_timesteps 4292.
Path 391 | total_timesteps 4301.
Path 392 | total_timesteps 4311.
Path 393 | total_timesteps 4320.
Path 394 | total_timesteps 4332.
Path 395 | total_timesteps 4340.
Path 396 | total_timesteps 4350.
Path 397 | total_timesteps 4361.
Path 398 | total_timesteps 4373.
Path 399 | total_timesteps 4382.
Path 400 | total_timesteps 4391.
Path 401 | total_timesteps 4402.
Path 402 | total_timesteps 4417.
Path 403 | total_timesteps 4426.
Path 404 | total_timesteps 4435.
Path 405 | total_timesteps 4448.
Path 406 | total_timesteps 4462.
Path 407 | total_timesteps 4471.
Path 408 | total_timesteps 4480.
Path 409 | total_timesteps 4493.
Path 410 | total_timesteps 4507.
Path 411 | total_timesteps 4517.
Path 412 | total_timesteps 4527.
Path 413 | total_timesteps 4535.
Path 414 | total_timesteps 4542.
Path 415 | total_timesteps 4558.
Path 416 | total_timesteps 4566.
Path 417 | total_timesteps 4576.
Path 418 | total_timesteps 4588.
Path 419 | total_timesteps 4595.
Path 420 | total_timesteps 4603.
Path 421 | total_timesteps 4613.
Path 422 | total_timesteps 4622.
Path 423 | total_timesteps 4633.
Path 424 | total_timesteps 4645.
Path 425 | total_timesteps 4663.
Path 426 | total_timesteps 4674.
Path 427 | total_timesteps 4683.
Path 428 | total_timesteps 4694.
Path 429 | total_timesteps 4709.
Path 430 | total_timesteps 4717.
Path 431 | total_timesteps 4729.
Path 432 | total_timesteps 4741.
Path 433 | total_timesteps 4751.
Path 434 | total_timesteps 4765.
Path 435 | total_timesteps 4775.
Path 436 | total_timesteps 4782.
Path 437 | total_timesteps 4792.
Path 438 | total_timesteps 4801.
Path 439 | total_timesteps 4813.
Path 440 | total_timesteps 4822.
Path 441 | total_timesteps 4831.
Path 442 | total_timesteps 4845.
Path 443 | total_timesteps 4858.
Path 444 | total_timesteps 4868.
Path 445 | total_timesteps 4886.
Path 446 | total_timesteps 4895.
Path 447 | total_timesteps 4904.
Path 448 | total_timesteps 4912.
Path 449 | total_timesteps 4923.
Path 450 | total_timesteps 4941.
Path 451 | total_timesteps 4951.
Path 452 | total_timesteps 4966.
Path 453 | total_timesteps 4975.
Path 454 | total_timesteps 4989.
Path 455 | total_timesteps 4999.
Path 456 | total_timesteps 5010.
Path 457 | total_timesteps 5026.
Path 458 | total_timesteps 5038.
Path 459 | total_timesteps 5049.
Path 460 | total_timesteps 5058.
Path 461 | total_timesteps 5068.
Path 462 | total_timesteps 5078.
Path 463 | total_timesteps 5087.
Path 464 | total_timesteps 5113.
Path 465 | total_timesteps 5129.
Path 466 | total_timesteps 5138.
Path 467 | total_timesteps 5151.
Path 468 | total_timesteps 5164.
Path 469 | total_timesteps 5173.
Path 470 | total_timesteps 5188.
Path 471 | total_timesteps 5196.
Path 472 | total_timesteps 5205.
Path 473 | total_timesteps 5216.
Path 474 | total_timesteps 5228.
Path 475 | total_timesteps 5238.
Path 476 | total_timesteps 5250.
Path 477 | total_timesteps 5267.
Path 478 | total_timesteps 5277.
Path 479 | total_timesteps 5287.
Path 480 | total_timesteps 5293.
Path 481 | total_timesteps 5306.
Path 482 | total_timesteps 5315.
Path 483 | total_timesteps 5324.
Path 484 | total_timesteps 5332.
Path 485 | total_timesteps 5342.
Path 486 | total_timesteps 5353.
Path 487 | total_timesteps 5365.
Path 488 | total_timesteps 5373.
Path 489 | total_timesteps 5382.
Path 490 | total_timesteps 5392.
Path 491 | total_timesteps 5403.
Path 492 | total_timesteps 5416.
Path 493 | total_timesteps 5431.
Path 494 | total_timesteps 5438.
Path 495 | total_timesteps 5451.
Path 496 | total_timesteps 5457.
Path 497 | total_timesteps 5468.
Path 498 | total_timesteps 5476.
Path 499 | total_timesteps 5489.
Path 500 | total_timesteps 5496.
Path 501 | total_timesteps 5505.
Path 502 | total_timesteps 5516.
Path 503 | total_timesteps 5526.
Path 504 | total_timesteps 5535.
Path 505 | total_timesteps 5545.
Path 506 | total_timesteps 5553.
Path 507 | total_timesteps 5561.
Path 508 | total_timesteps 5573.
Path 509 | total_timesteps 5581.
Path 510 | total_timesteps 5594.
Path 511 | total_timesteps 5605.
Path 512 | total_timesteps 5615.
Path 513 | total_timesteps 5626.
Path 514 | total_timesteps 5636.
Path 515 | total_timesteps 5643.
Path 516 | total_timesteps 5652.
Path 517 | total_timesteps 5662.
Path 518 | total_timesteps 5670.
Path 519 | total_timesteps 5680.
Path 520 | total_timesteps 5689.
Path 521 | total_timesteps 5698.
Path 522 | total_timesteps 5707.
Path 523 | total_timesteps 5715.
Path 524 | total_timesteps 5726.
Path 525 | total_timesteps 5733.
Path 526 | total_timesteps 5743.
Path 527 | total_timesteps 5755.
Path 528 | total_timesteps 5764.
Path 529 | total_timesteps 5776.
Path 530 | total_timesteps 5788.
Path 531 | total_timesteps 5800.
Path 532 | total_timesteps 5815.
Path 533 | total_timesteps 5829.
Path 534 | total_timesteps 5839.
Path 535 | total_timesteps 5847.
Path 536 | total_timesteps 5873.
Path 537 | total_timesteps 5888.
Path 538 | total_timesteps 5903.
Path 539 | total_timesteps 5914.
Path 540 | total_timesteps 5922.
Path 541 | total_timesteps 5931.
Path 542 | total_timesteps 5939.
Path 543 | total_timesteps 5949.
Path 544 | total_timesteps 5960.
Path 545 | total_timesteps 5968.
Path 546 | total_timesteps 5978.
Path 547 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.17    |
| Iteration     | 15       |
| MaximumReturn | 1.84     |
| MinimumReturn | -20.5    |
| TotalSamples  | 68055    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.3143295645713806
Validation loss = 0.3167078495025635
Validation loss = 0.319927841424942
Validation loss = 0.32018351554870605
Validation loss = 0.3237757384777069
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 65.
Path 5 | total_timesteps 73.
Path 6 | total_timesteps 84.
Path 7 | total_timesteps 93.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 114.
Path 10 | total_timesteps 122.
Path 11 | total_timesteps 134.
Path 12 | total_timesteps 143.
Path 13 | total_timesteps 152.
Path 14 | total_timesteps 160.
Path 15 | total_timesteps 174.
Path 16 | total_timesteps 184.
Path 17 | total_timesteps 194.
Path 18 | total_timesteps 202.
Path 19 | total_timesteps 212.
Path 20 | total_timesteps 228.
Path 21 | total_timesteps 243.
Path 22 | total_timesteps 256.
Path 23 | total_timesteps 267.
Path 24 | total_timesteps 279.
Path 25 | total_timesteps 289.
Path 26 | total_timesteps 297.
Path 27 | total_timesteps 310.
Path 28 | total_timesteps 318.
Path 29 | total_timesteps 331.
Path 30 | total_timesteps 339.
Path 31 | total_timesteps 346.
Path 32 | total_timesteps 360.
Path 33 | total_timesteps 368.
Path 34 | total_timesteps 380.
Path 35 | total_timesteps 390.
Path 36 | total_timesteps 401.
Path 37 | total_timesteps 410.
Path 38 | total_timesteps 432.
Path 39 | total_timesteps 444.
Path 40 | total_timesteps 454.
Path 41 | total_timesteps 470.
Path 42 | total_timesteps 482.
Path 43 | total_timesteps 491.
Path 44 | total_timesteps 501.
Path 45 | total_timesteps 512.
Path 46 | total_timesteps 522.
Path 47 | total_timesteps 536.
Path 48 | total_timesteps 543.
Path 49 | total_timesteps 551.
Path 50 | total_timesteps 567.
Path 51 | total_timesteps 578.
Path 52 | total_timesteps 590.
Path 53 | total_timesteps 597.
Path 54 | total_timesteps 609.
Path 55 | total_timesteps 624.
Path 56 | total_timesteps 635.
Path 57 | total_timesteps 653.
Path 58 | total_timesteps 666.
Path 59 | total_timesteps 679.
Path 60 | total_timesteps 691.
Path 61 | total_timesteps 701.
Path 62 | total_timesteps 708.
Path 63 | total_timesteps 716.
Path 64 | total_timesteps 723.
Path 65 | total_timesteps 731.
Path 66 | total_timesteps 743.
Path 67 | total_timesteps 751.
Path 68 | total_timesteps 761.
Path 69 | total_timesteps 770.
Path 70 | total_timesteps 779.
Path 71 | total_timesteps 788.
Path 72 | total_timesteps 803.
Path 73 | total_timesteps 814.
Path 74 | total_timesteps 821.
Path 75 | total_timesteps 830.
Path 76 | total_timesteps 839.
Path 77 | total_timesteps 850.
Path 78 | total_timesteps 865.
Path 79 | total_timesteps 876.
Path 80 | total_timesteps 886.
Path 81 | total_timesteps 895.
Path 82 | total_timesteps 905.
Path 83 | total_timesteps 915.
Path 84 | total_timesteps 926.
Path 85 | total_timesteps 938.
Path 86 | total_timesteps 947.
Path 87 | total_timesteps 956.
Path 88 | total_timesteps 964.
Path 89 | total_timesteps 972.
Path 90 | total_timesteps 981.
Path 91 | total_timesteps 994.
Path 92 | total_timesteps 1002.
Path 93 | total_timesteps 1012.
Path 94 | total_timesteps 1023.
Path 95 | total_timesteps 1038.
Path 96 | total_timesteps 1053.
Path 97 | total_timesteps 1063.
Path 98 | total_timesteps 1078.
Path 99 | total_timesteps 1088.
Path 100 | total_timesteps 1100.
Path 101 | total_timesteps 1110.
Path 102 | total_timesteps 1120.
Path 103 | total_timesteps 1135.
Path 104 | total_timesteps 1146.
Path 105 | total_timesteps 1156.
Path 106 | total_timesteps 1165.
Path 107 | total_timesteps 1174.
Path 108 | total_timesteps 1185.
Path 109 | total_timesteps 1193.
Path 110 | total_timesteps 1206.
Path 111 | total_timesteps 1217.
Path 112 | total_timesteps 1226.
Path 113 | total_timesteps 1236.
Path 114 | total_timesteps 1245.
Path 115 | total_timesteps 1252.
Path 116 | total_timesteps 1266.
Path 117 | total_timesteps 1275.
Path 118 | total_timesteps 1290.
Path 119 | total_timesteps 1300.
Path 120 | total_timesteps 1313.
Path 121 | total_timesteps 1323.
Path 122 | total_timesteps 1335.
Path 123 | total_timesteps 1342.
Path 124 | total_timesteps 1354.
Path 125 | total_timesteps 1362.
Path 126 | total_timesteps 1373.
Path 127 | total_timesteps 1383.
Path 128 | total_timesteps 1391.
Path 129 | total_timesteps 1404.
Path 130 | total_timesteps 1414.
Path 131 | total_timesteps 1437.
Path 132 | total_timesteps 1446.
Path 133 | total_timesteps 1460.
Path 134 | total_timesteps 1468.
Path 135 | total_timesteps 1475.
Path 136 | total_timesteps 1482.
Path 137 | total_timesteps 1493.
Path 138 | total_timesteps 1502.
Path 139 | total_timesteps 1517.
Path 140 | total_timesteps 1527.
Path 141 | total_timesteps 1536.
Path 142 | total_timesteps 1546.
Path 143 | total_timesteps 1556.
Path 144 | total_timesteps 1565.
Path 145 | total_timesteps 1574.
Path 146 | total_timesteps 1583.
Path 147 | total_timesteps 1598.
Path 148 | total_timesteps 1607.
Path 149 | total_timesteps 1618.
Path 150 | total_timesteps 1627.
Path 151 | total_timesteps 1640.
Path 152 | total_timesteps 1651.
Path 153 | total_timesteps 1658.
Path 154 | total_timesteps 1666.
Path 155 | total_timesteps 1677.
Path 156 | total_timesteps 1685.
Path 157 | total_timesteps 1697.
Path 158 | total_timesteps 1711.
Path 159 | total_timesteps 1723.
Path 160 | total_timesteps 1735.
Path 161 | total_timesteps 1750.
Path 162 | total_timesteps 1757.
Path 163 | total_timesteps 1767.
Path 164 | total_timesteps 1777.
Path 165 | total_timesteps 1788.
Path 166 | total_timesteps 1798.
Path 167 | total_timesteps 1809.
Path 168 | total_timesteps 1827.
Path 169 | total_timesteps 1836.
Path 170 | total_timesteps 1847.
Path 171 | total_timesteps 1857.
Path 172 | total_timesteps 1866.
Path 173 | total_timesteps 1873.
Path 174 | total_timesteps 1887.
Path 175 | total_timesteps 1898.
Path 176 | total_timesteps 1909.
Path 177 | total_timesteps 1921.
Path 178 | total_timesteps 1934.
Path 179 | total_timesteps 1956.
Path 180 | total_timesteps 1965.
Path 181 | total_timesteps 1973.
Path 182 | total_timesteps 1984.
Path 183 | total_timesteps 1995.
Path 184 | total_timesteps 2003.
Path 185 | total_timesteps 2010.
Path 186 | total_timesteps 2023.
Path 187 | total_timesteps 2035.
Path 188 | total_timesteps 2046.
Path 189 | total_timesteps 2055.
Path 190 | total_timesteps 2070.
Path 191 | total_timesteps 2080.
Path 192 | total_timesteps 2089.
Path 193 | total_timesteps 2100.
Path 194 | total_timesteps 2111.
Path 195 | total_timesteps 2122.
Path 196 | total_timesteps 2131.
Path 197 | total_timesteps 2146.
Path 198 | total_timesteps 2157.
Path 199 | total_timesteps 2165.
Path 200 | total_timesteps 2173.
Path 201 | total_timesteps 2183.
Path 202 | total_timesteps 2192.
Path 203 | total_timesteps 2200.
Path 204 | total_timesteps 2211.
Path 205 | total_timesteps 2219.
Path 206 | total_timesteps 2231.
Path 207 | total_timesteps 2241.
Path 208 | total_timesteps 2254.
Path 209 | total_timesteps 2266.
Path 210 | total_timesteps 2281.
Path 211 | total_timesteps 2288.
Path 212 | total_timesteps 2299.
Path 213 | total_timesteps 2306.
Path 214 | total_timesteps 2315.
Path 215 | total_timesteps 2322.
Path 216 | total_timesteps 2335.
Path 217 | total_timesteps 2352.
Path 218 | total_timesteps 2362.
Path 219 | total_timesteps 2372.
Path 220 | total_timesteps 2383.
Path 221 | total_timesteps 2391.
Path 222 | total_timesteps 2410.
Path 223 | total_timesteps 2419.
Path 224 | total_timesteps 2430.
Path 225 | total_timesteps 2444.
Path 226 | total_timesteps 2457.
Path 227 | total_timesteps 2480.
Path 228 | total_timesteps 2500.
Path 229 | total_timesteps 2513.
Path 230 | total_timesteps 2524.
Path 231 | total_timesteps 2532.
Path 232 | total_timesteps 2553.
Path 233 | total_timesteps 2566.
Path 234 | total_timesteps 2572.
Path 235 | total_timesteps 2585.
Path 236 | total_timesteps 2600.
Path 237 | total_timesteps 2608.
Path 238 | total_timesteps 2616.
Path 239 | total_timesteps 2623.
Path 240 | total_timesteps 2633.
Path 241 | total_timesteps 2645.
Path 242 | total_timesteps 2654.
Path 243 | total_timesteps 2662.
Path 244 | total_timesteps 2672.
Path 245 | total_timesteps 2684.
Path 246 | total_timesteps 2692.
Path 247 | total_timesteps 2703.
Path 248 | total_timesteps 2717.
Path 249 | total_timesteps 2734.
Path 250 | total_timesteps 2741.
Path 251 | total_timesteps 2754.
Path 252 | total_timesteps 2765.
Path 253 | total_timesteps 2777.
Path 254 | total_timesteps 2791.
Path 255 | total_timesteps 2802.
Path 256 | total_timesteps 2817.
Path 257 | total_timesteps 2825.
Path 258 | total_timesteps 2836.
Path 259 | total_timesteps 2855.
Path 260 | total_timesteps 2866.
Path 261 | total_timesteps 2879.
Path 262 | total_timesteps 2890.
Path 263 | total_timesteps 2899.
Path 264 | total_timesteps 2910.
Path 265 | total_timesteps 2919.
Path 266 | total_timesteps 2931.
Path 267 | total_timesteps 2941.
Path 268 | total_timesteps 2948.
Path 269 | total_timesteps 2959.
Path 270 | total_timesteps 2967.
Path 271 | total_timesteps 2978.
Path 272 | total_timesteps 2988.
Path 273 | total_timesteps 2996.
Path 274 | total_timesteps 3007.
Path 275 | total_timesteps 3015.
Path 276 | total_timesteps 3023.
Path 277 | total_timesteps 3032.
Path 278 | total_timesteps 3040.
Path 279 | total_timesteps 3049.
Path 280 | total_timesteps 3061.
Path 281 | total_timesteps 3081.
Path 282 | total_timesteps 3091.
Path 283 | total_timesteps 3101.
Path 284 | total_timesteps 3113.
Path 285 | total_timesteps 3125.
Path 286 | total_timesteps 3143.
Path 287 | total_timesteps 3151.
Path 288 | total_timesteps 3161.
Path 289 | total_timesteps 3173.
Path 290 | total_timesteps 3181.
Path 291 | total_timesteps 3189.
Path 292 | total_timesteps 3195.
Path 293 | total_timesteps 3203.
Path 294 | total_timesteps 3212.
Path 295 | total_timesteps 3232.
Path 296 | total_timesteps 3248.
Path 297 | total_timesteps 3260.
Path 298 | total_timesteps 3270.
Path 299 | total_timesteps 3279.
Path 300 | total_timesteps 3291.
Path 301 | total_timesteps 3302.
Path 302 | total_timesteps 3313.
Path 303 | total_timesteps 3322.
Path 304 | total_timesteps 3332.
Path 305 | total_timesteps 3341.
Path 306 | total_timesteps 3353.
Path 307 | total_timesteps 3364.
Path 308 | total_timesteps 3374.
Path 309 | total_timesteps 3381.
Path 310 | total_timesteps 3405.
Path 311 | total_timesteps 3414.
Path 312 | total_timesteps 3431.
Path 313 | total_timesteps 3441.
Path 314 | total_timesteps 3453.
Path 315 | total_timesteps 3474.
Path 316 | total_timesteps 3485.
Path 317 | total_timesteps 3497.
Path 318 | total_timesteps 3505.
Path 319 | total_timesteps 3543.
Path 320 | total_timesteps 3550.
Path 321 | total_timesteps 3558.
Path 322 | total_timesteps 3567.
Path 323 | total_timesteps 3590.
Path 324 | total_timesteps 3605.
Path 325 | total_timesteps 3623.
Path 326 | total_timesteps 3637.
Path 327 | total_timesteps 3659.
Path 328 | total_timesteps 3670.
Path 329 | total_timesteps 3686.
Path 330 | total_timesteps 3695.
Path 331 | total_timesteps 3706.
Path 332 | total_timesteps 3720.
Path 333 | total_timesteps 3729.
Path 334 | total_timesteps 3743.
Path 335 | total_timesteps 3753.
Path 336 | total_timesteps 3764.
Path 337 | total_timesteps 3774.
Path 338 | total_timesteps 3787.
Path 339 | total_timesteps 3796.
Path 340 | total_timesteps 3810.
Path 341 | total_timesteps 3822.
Path 342 | total_timesteps 3830.
Path 343 | total_timesteps 3840.
Path 344 | total_timesteps 3851.
Path 345 | total_timesteps 3860.
Path 346 | total_timesteps 3872.
Path 347 | total_timesteps 3883.
Path 348 | total_timesteps 3904.
Path 349 | total_timesteps 3913.
Path 350 | total_timesteps 3924.
Path 351 | total_timesteps 3934.
Path 352 | total_timesteps 3942.
Path 353 | total_timesteps 3951.
Path 354 | total_timesteps 3960.
Path 355 | total_timesteps 3972.
Path 356 | total_timesteps 3980.
Path 357 | total_timesteps 3993.
Path 358 | total_timesteps 4007.
Path 359 | total_timesteps 4017.
Path 360 | total_timesteps 4027.
Path 361 | total_timesteps 4038.
Path 362 | total_timesteps 4050.
Path 363 | total_timesteps 4059.
Path 364 | total_timesteps 4069.
Path 365 | total_timesteps 4079.
Path 366 | total_timesteps 4092.
Path 367 | total_timesteps 4103.
Path 368 | total_timesteps 4114.
Path 369 | total_timesteps 4128.
Path 370 | total_timesteps 4142.
Path 371 | total_timesteps 4153.
Path 372 | total_timesteps 4161.
Path 373 | total_timesteps 4169.
Path 374 | total_timesteps 4182.
Path 375 | total_timesteps 4192.
Path 376 | total_timesteps 4198.
Path 377 | total_timesteps 4206.
Path 378 | total_timesteps 4214.
Path 379 | total_timesteps 4220.
Path 380 | total_timesteps 4235.
Path 381 | total_timesteps 4243.
Path 382 | total_timesteps 4253.
Path 383 | total_timesteps 4262.
Path 384 | total_timesteps 4274.
Path 385 | total_timesteps 4283.
Path 386 | total_timesteps 4290.
Path 387 | total_timesteps 4302.
Path 388 | total_timesteps 4311.
Path 389 | total_timesteps 4324.
Path 390 | total_timesteps 4331.
Path 391 | total_timesteps 4340.
Path 392 | total_timesteps 4351.
Path 393 | total_timesteps 4362.
Path 394 | total_timesteps 4375.
Path 395 | total_timesteps 4384.
Path 396 | total_timesteps 4394.
Path 397 | total_timesteps 4417.
Path 398 | total_timesteps 4426.
Path 399 | total_timesteps 4436.
Path 400 | total_timesteps 4445.
Path 401 | total_timesteps 4455.
Path 402 | total_timesteps 4468.
Path 403 | total_timesteps 4480.
Path 404 | total_timesteps 4492.
Path 405 | total_timesteps 4500.
Path 406 | total_timesteps 4510.
Path 407 | total_timesteps 4517.
Path 408 | total_timesteps 4528.
Path 409 | total_timesteps 4536.
Path 410 | total_timesteps 4544.
Path 411 | total_timesteps 4555.
Path 412 | total_timesteps 4568.
Path 413 | total_timesteps 4580.
Path 414 | total_timesteps 4589.
Path 415 | total_timesteps 4595.
Path 416 | total_timesteps 4603.
Path 417 | total_timesteps 4615.
Path 418 | total_timesteps 4624.
Path 419 | total_timesteps 4635.
Path 420 | total_timesteps 4644.
Path 421 | total_timesteps 4654.
Path 422 | total_timesteps 4665.
Path 423 | total_timesteps 4679.
Path 424 | total_timesteps 4687.
Path 425 | total_timesteps 4696.
Path 426 | total_timesteps 4706.
Path 427 | total_timesteps 4718.
Path 428 | total_timesteps 4728.
Path 429 | total_timesteps 4750.
Path 430 | total_timesteps 4763.
Path 431 | total_timesteps 4779.
Path 432 | total_timesteps 4789.
Path 433 | total_timesteps 4807.
Path 434 | total_timesteps 4818.
Path 435 | total_timesteps 4828.
Path 436 | total_timesteps 4838.
Path 437 | total_timesteps 4847.
Path 438 | total_timesteps 4858.
Path 439 | total_timesteps 4865.
Path 440 | total_timesteps 4887.
Path 441 | total_timesteps 4897.
Path 442 | total_timesteps 4905.
Path 443 | total_timesteps 4913.
Path 444 | total_timesteps 4923.
Path 445 | total_timesteps 4936.
Path 446 | total_timesteps 4945.
Path 447 | total_timesteps 4955.
Path 448 | total_timesteps 4965.
Path 449 | total_timesteps 4978.
Path 450 | total_timesteps 4986.
Path 451 | total_timesteps 4995.
Path 452 | total_timesteps 5014.
Path 453 | total_timesteps 5023.
Path 454 | total_timesteps 5032.
Path 455 | total_timesteps 5041.
Path 456 | total_timesteps 5051.
Path 457 | total_timesteps 5065.
Path 458 | total_timesteps 5073.
Path 459 | total_timesteps 5085.
Path 460 | total_timesteps 5093.
Path 461 | total_timesteps 5104.
Path 462 | total_timesteps 5113.
Path 463 | total_timesteps 5122.
Path 464 | total_timesteps 5134.
Path 465 | total_timesteps 5144.
Path 466 | total_timesteps 5156.
Path 467 | total_timesteps 5165.
Path 468 | total_timesteps 5172.
Path 469 | total_timesteps 5180.
Path 470 | total_timesteps 5189.
Path 471 | total_timesteps 5201.
Path 472 | total_timesteps 5210.
Path 473 | total_timesteps 5220.
Path 474 | total_timesteps 5232.
Path 475 | total_timesteps 5243.
Path 476 | total_timesteps 5254.
Path 477 | total_timesteps 5264.
Path 478 | total_timesteps 5273.
Path 479 | total_timesteps 5280.
Path 480 | total_timesteps 5288.
Path 481 | total_timesteps 5299.
Path 482 | total_timesteps 5308.
Path 483 | total_timesteps 5316.
Path 484 | total_timesteps 5325.
Path 485 | total_timesteps 5335.
Path 486 | total_timesteps 5349.
Path 487 | total_timesteps 5358.
Path 488 | total_timesteps 5368.
Path 489 | total_timesteps 5378.
Path 490 | total_timesteps 5392.
Path 491 | total_timesteps 5402.
Path 492 | total_timesteps 5415.
Path 493 | total_timesteps 5431.
Path 494 | total_timesteps 5445.
Path 495 | total_timesteps 5457.
Path 496 | total_timesteps 5469.
Path 497 | total_timesteps 5479.
Path 498 | total_timesteps 5488.
Path 499 | total_timesteps 5500.
Path 500 | total_timesteps 5514.
Path 501 | total_timesteps 5527.
Path 502 | total_timesteps 5535.
Path 503 | total_timesteps 5546.
Path 504 | total_timesteps 5556.
Path 505 | total_timesteps 5574.
Path 506 | total_timesteps 5584.
Path 507 | total_timesteps 5599.
Path 508 | total_timesteps 5609.
Path 509 | total_timesteps 5620.
Path 510 | total_timesteps 5631.
Path 511 | total_timesteps 5637.
Path 512 | total_timesteps 5644.
Path 513 | total_timesteps 5653.
Path 514 | total_timesteps 5661.
Path 515 | total_timesteps 5672.
Path 516 | total_timesteps 5681.
Path 517 | total_timesteps 5692.
Path 518 | total_timesteps 5700.
Path 519 | total_timesteps 5710.
Path 520 | total_timesteps 5726.
Path 521 | total_timesteps 5737.
Path 522 | total_timesteps 5749.
Path 523 | total_timesteps 5758.
Path 524 | total_timesteps 5771.
Path 525 | total_timesteps 5780.
Path 526 | total_timesteps 5791.
Path 527 | total_timesteps 5804.
Path 528 | total_timesteps 5813.
Path 529 | total_timesteps 5823.
Path 530 | total_timesteps 5837.
Path 531 | total_timesteps 5845.
Path 532 | total_timesteps 5862.
Path 533 | total_timesteps 5880.
Path 534 | total_timesteps 5888.
Path 535 | total_timesteps 5897.
Path 536 | total_timesteps 5904.
Path 537 | total_timesteps 5912.
Path 538 | total_timesteps 5926.
Path 539 | total_timesteps 5938.
Path 540 | total_timesteps 5955.
Path 541 | total_timesteps 5964.
Path 542 | total_timesteps 5973.
Path 543 | total_timesteps 5981.
Path 544 | total_timesteps 5990.
Path 545 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.12    |
| Iteration     | 16       |
| MaximumReturn | 3.16     |
| MinimumReturn | -16.1    |
| TotalSamples  | 72061    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.3181787133216858
Validation loss = 0.31780296564102173
Validation loss = 0.3207017183303833
Validation loss = 0.32060959935188293
Validation loss = 0.32442718744277954
Validation loss = 0.32178574800491333
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 60.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 96.
Path 10 | total_timesteps 117.
Path 11 | total_timesteps 129.
Path 12 | total_timesteps 148.
Path 13 | total_timesteps 157.
Path 14 | total_timesteps 167.
Path 15 | total_timesteps 176.
Path 16 | total_timesteps 189.
Path 17 | total_timesteps 202.
Path 18 | total_timesteps 212.
Path 19 | total_timesteps 225.
Path 20 | total_timesteps 233.
Path 21 | total_timesteps 242.
Path 22 | total_timesteps 251.
Path 23 | total_timesteps 265.
Path 24 | total_timesteps 278.
Path 25 | total_timesteps 288.
Path 26 | total_timesteps 297.
Path 27 | total_timesteps 305.
Path 28 | total_timesteps 315.
Path 29 | total_timesteps 327.
Path 30 | total_timesteps 338.
Path 31 | total_timesteps 349.
Path 32 | total_timesteps 360.
Path 33 | total_timesteps 372.
Path 34 | total_timesteps 381.
Path 35 | total_timesteps 394.
Path 36 | total_timesteps 410.
Path 37 | total_timesteps 421.
Path 38 | total_timesteps 434.
Path 39 | total_timesteps 442.
Path 40 | total_timesteps 452.
Path 41 | total_timesteps 465.
Path 42 | total_timesteps 472.
Path 43 | total_timesteps 486.
Path 44 | total_timesteps 500.
Path 45 | total_timesteps 507.
Path 46 | total_timesteps 521.
Path 47 | total_timesteps 532.
Path 48 | total_timesteps 544.
Path 49 | total_timesteps 554.
Path 50 | total_timesteps 565.
Path 51 | total_timesteps 575.
Path 52 | total_timesteps 588.
Path 53 | total_timesteps 601.
Path 54 | total_timesteps 608.
Path 55 | total_timesteps 623.
Path 56 | total_timesteps 632.
Path 57 | total_timesteps 646.
Path 58 | total_timesteps 661.
Path 59 | total_timesteps 675.
Path 60 | total_timesteps 686.
Path 61 | total_timesteps 696.
Path 62 | total_timesteps 704.
Path 63 | total_timesteps 717.
Path 64 | total_timesteps 735.
Path 65 | total_timesteps 745.
Path 66 | total_timesteps 766.
Path 67 | total_timesteps 777.
Path 68 | total_timesteps 790.
Path 69 | total_timesteps 803.
Path 70 | total_timesteps 815.
Path 71 | total_timesteps 823.
Path 72 | total_timesteps 831.
Path 73 | total_timesteps 839.
Path 74 | total_timesteps 850.
Path 75 | total_timesteps 865.
Path 76 | total_timesteps 875.
Path 77 | total_timesteps 884.
Path 78 | total_timesteps 891.
Path 79 | total_timesteps 902.
Path 80 | total_timesteps 914.
Path 81 | total_timesteps 923.
Path 82 | total_timesteps 936.
Path 83 | total_timesteps 946.
Path 84 | total_timesteps 960.
Path 85 | total_timesteps 974.
Path 86 | total_timesteps 984.
Path 87 | total_timesteps 993.
Path 88 | total_timesteps 1003.
Path 89 | total_timesteps 1014.
Path 90 | total_timesteps 1024.
Path 91 | total_timesteps 1040.
Path 92 | total_timesteps 1049.
Path 93 | total_timesteps 1059.
Path 94 | total_timesteps 1071.
Path 95 | total_timesteps 1080.
Path 96 | total_timesteps 1103.
Path 97 | total_timesteps 1119.
Path 98 | total_timesteps 1127.
Path 99 | total_timesteps 1139.
Path 100 | total_timesteps 1149.
Path 101 | total_timesteps 1157.
Path 102 | total_timesteps 1170.
Path 103 | total_timesteps 1181.
Path 104 | total_timesteps 1190.
Path 105 | total_timesteps 1197.
Path 106 | total_timesteps 1205.
Path 107 | total_timesteps 1213.
Path 108 | total_timesteps 1221.
Path 109 | total_timesteps 1233.
Path 110 | total_timesteps 1245.
Path 111 | total_timesteps 1259.
Path 112 | total_timesteps 1265.
Path 113 | total_timesteps 1273.
Path 114 | total_timesteps 1284.
Path 115 | total_timesteps 1293.
Path 116 | total_timesteps 1302.
Path 117 | total_timesteps 1319.
Path 118 | total_timesteps 1328.
Path 119 | total_timesteps 1338.
Path 120 | total_timesteps 1355.
Path 121 | total_timesteps 1364.
Path 122 | total_timesteps 1373.
Path 123 | total_timesteps 1381.
Path 124 | total_timesteps 1389.
Path 125 | total_timesteps 1398.
Path 126 | total_timesteps 1405.
Path 127 | total_timesteps 1420.
Path 128 | total_timesteps 1429.
Path 129 | total_timesteps 1438.
Path 130 | total_timesteps 1445.
Path 131 | total_timesteps 1453.
Path 132 | total_timesteps 1464.
Path 133 | total_timesteps 1473.
Path 134 | total_timesteps 1486.
Path 135 | total_timesteps 1499.
Path 136 | total_timesteps 1507.
Path 137 | total_timesteps 1516.
Path 138 | total_timesteps 1527.
Path 139 | total_timesteps 1534.
Path 140 | total_timesteps 1547.
Path 141 | total_timesteps 1554.
Path 142 | total_timesteps 1565.
Path 143 | total_timesteps 1578.
Path 144 | total_timesteps 1587.
Path 145 | total_timesteps 1600.
Path 146 | total_timesteps 1611.
Path 147 | total_timesteps 1619.
Path 148 | total_timesteps 1629.
Path 149 | total_timesteps 1636.
Path 150 | total_timesteps 1644.
Path 151 | total_timesteps 1655.
Path 152 | total_timesteps 1664.
Path 153 | total_timesteps 1671.
Path 154 | total_timesteps 1683.
Path 155 | total_timesteps 1691.
Path 156 | total_timesteps 1698.
Path 157 | total_timesteps 1706.
Path 158 | total_timesteps 1716.
Path 159 | total_timesteps 1725.
Path 160 | total_timesteps 1734.
Path 161 | total_timesteps 1744.
Path 162 | total_timesteps 1754.
Path 163 | total_timesteps 1763.
Path 164 | total_timesteps 1774.
Path 165 | total_timesteps 1782.
Path 166 | total_timesteps 1790.
Path 167 | total_timesteps 1798.
Path 168 | total_timesteps 1810.
Path 169 | total_timesteps 1821.
Path 170 | total_timesteps 1841.
Path 171 | total_timesteps 1851.
Path 172 | total_timesteps 1862.
Path 173 | total_timesteps 1872.
Path 174 | total_timesteps 1886.
Path 175 | total_timesteps 1899.
Path 176 | total_timesteps 1908.
Path 177 | total_timesteps 1919.
Path 178 | total_timesteps 1931.
Path 179 | total_timesteps 1941.
Path 180 | total_timesteps 1948.
Path 181 | total_timesteps 1960.
Path 182 | total_timesteps 1970.
Path 183 | total_timesteps 1978.
Path 184 | total_timesteps 1989.
Path 185 | total_timesteps 1996.
Path 186 | total_timesteps 2008.
Path 187 | total_timesteps 2017.
Path 188 | total_timesteps 2026.
Path 189 | total_timesteps 2033.
Path 190 | total_timesteps 2049.
Path 191 | total_timesteps 2060.
Path 192 | total_timesteps 2071.
Path 193 | total_timesteps 2089.
Path 194 | total_timesteps 2100.
Path 195 | total_timesteps 2108.
Path 196 | total_timesteps 2120.
Path 197 | total_timesteps 2131.
Path 198 | total_timesteps 2142.
Path 199 | total_timesteps 2156.
Path 200 | total_timesteps 2169.
Path 201 | total_timesteps 2179.
Path 202 | total_timesteps 2191.
Path 203 | total_timesteps 2204.
Path 204 | total_timesteps 2219.
Path 205 | total_timesteps 2232.
Path 206 | total_timesteps 2242.
Path 207 | total_timesteps 2261.
Path 208 | total_timesteps 2273.
Path 209 | total_timesteps 2284.
Path 210 | total_timesteps 2293.
Path 211 | total_timesteps 2302.
Path 212 | total_timesteps 2315.
Path 213 | total_timesteps 2324.
Path 214 | total_timesteps 2334.
Path 215 | total_timesteps 2353.
Path 216 | total_timesteps 2362.
Path 217 | total_timesteps 2373.
Path 218 | total_timesteps 2384.
Path 219 | total_timesteps 2395.
Path 220 | total_timesteps 2402.
Path 221 | total_timesteps 2410.
Path 222 | total_timesteps 2421.
Path 223 | total_timesteps 2432.
Path 224 | total_timesteps 2445.
Path 225 | total_timesteps 2456.
Path 226 | total_timesteps 2467.
Path 227 | total_timesteps 2485.
Path 228 | total_timesteps 2495.
Path 229 | total_timesteps 2512.
Path 230 | total_timesteps 2525.
Path 231 | total_timesteps 2538.
Path 232 | total_timesteps 2560.
Path 233 | total_timesteps 2569.
Path 234 | total_timesteps 2581.
Path 235 | total_timesteps 2589.
Path 236 | total_timesteps 2597.
Path 237 | total_timesteps 2605.
Path 238 | total_timesteps 2616.
Path 239 | total_timesteps 2631.
Path 240 | total_timesteps 2647.
Path 241 | total_timesteps 2660.
Path 242 | total_timesteps 2672.
Path 243 | total_timesteps 2682.
Path 244 | total_timesteps 2691.
Path 245 | total_timesteps 2700.
Path 246 | total_timesteps 2716.
Path 247 | total_timesteps 2725.
Path 248 | total_timesteps 2741.
Path 249 | total_timesteps 2748.
Path 250 | total_timesteps 2761.
Path 251 | total_timesteps 2768.
Path 252 | total_timesteps 2776.
Path 253 | total_timesteps 2787.
Path 254 | total_timesteps 2797.
Path 255 | total_timesteps 2806.
Path 256 | total_timesteps 2815.
Path 257 | total_timesteps 2829.
Path 258 | total_timesteps 2847.
Path 259 | total_timesteps 2867.
Path 260 | total_timesteps 2877.
Path 261 | total_timesteps 2886.
Path 262 | total_timesteps 2896.
Path 263 | total_timesteps 2909.
Path 264 | total_timesteps 2921.
Path 265 | total_timesteps 2931.
Path 266 | total_timesteps 2942.
Path 267 | total_timesteps 2952.
Path 268 | total_timesteps 2959.
Path 269 | total_timesteps 2972.
Path 270 | total_timesteps 2985.
Path 271 | total_timesteps 2993.
Path 272 | total_timesteps 3001.
Path 273 | total_timesteps 3013.
Path 274 | total_timesteps 3029.
Path 275 | total_timesteps 3038.
Path 276 | total_timesteps 3047.
Path 277 | total_timesteps 3059.
Path 278 | total_timesteps 3067.
Path 279 | total_timesteps 3081.
Path 280 | total_timesteps 3090.
Path 281 | total_timesteps 3102.
Path 282 | total_timesteps 3112.
Path 283 | total_timesteps 3123.
Path 284 | total_timesteps 3132.
Path 285 | total_timesteps 3141.
Path 286 | total_timesteps 3154.
Path 287 | total_timesteps 3167.
Path 288 | total_timesteps 3174.
Path 289 | total_timesteps 3189.
Path 290 | total_timesteps 3199.
Path 291 | total_timesteps 3210.
Path 292 | total_timesteps 3220.
Path 293 | total_timesteps 3236.
Path 294 | total_timesteps 3248.
Path 295 | total_timesteps 3258.
Path 296 | total_timesteps 3271.
Path 297 | total_timesteps 3278.
Path 298 | total_timesteps 3286.
Path 299 | total_timesteps 3293.
Path 300 | total_timesteps 3307.
Path 301 | total_timesteps 3319.
Path 302 | total_timesteps 3328.
Path 303 | total_timesteps 3345.
Path 304 | total_timesteps 3356.
Path 305 | total_timesteps 3368.
Path 306 | total_timesteps 3377.
Path 307 | total_timesteps 3385.
Path 308 | total_timesteps 3401.
Path 309 | total_timesteps 3410.
Path 310 | total_timesteps 3429.
Path 311 | total_timesteps 3439.
Path 312 | total_timesteps 3449.
Path 313 | total_timesteps 3459.
Path 314 | total_timesteps 3471.
Path 315 | total_timesteps 3491.
Path 316 | total_timesteps 3501.
Path 317 | total_timesteps 3514.
Path 318 | total_timesteps 3523.
Path 319 | total_timesteps 3531.
Path 320 | total_timesteps 3541.
Path 321 | total_timesteps 3550.
Path 322 | total_timesteps 3558.
Path 323 | total_timesteps 3580.
Path 324 | total_timesteps 3592.
Path 325 | total_timesteps 3604.
Path 326 | total_timesteps 3616.
Path 327 | total_timesteps 3624.
Path 328 | total_timesteps 3635.
Path 329 | total_timesteps 3647.
Path 330 | total_timesteps 3660.
Path 331 | total_timesteps 3674.
Path 332 | total_timesteps 3685.
Path 333 | total_timesteps 3701.
Path 334 | total_timesteps 3711.
Path 335 | total_timesteps 3719.
Path 336 | total_timesteps 3726.
Path 337 | total_timesteps 3736.
Path 338 | total_timesteps 3746.
Path 339 | total_timesteps 3756.
Path 340 | total_timesteps 3770.
Path 341 | total_timesteps 3793.
Path 342 | total_timesteps 3801.
Path 343 | total_timesteps 3810.
Path 344 | total_timesteps 3819.
Path 345 | total_timesteps 3829.
Path 346 | total_timesteps 3849.
Path 347 | total_timesteps 3858.
Path 348 | total_timesteps 3869.
Path 349 | total_timesteps 3876.
Path 350 | total_timesteps 3893.
Path 351 | total_timesteps 3901.
Path 352 | total_timesteps 3913.
Path 353 | total_timesteps 3927.
Path 354 | total_timesteps 3937.
Path 355 | total_timesteps 3947.
Path 356 | total_timesteps 3954.
Path 357 | total_timesteps 3965.
Path 358 | total_timesteps 3979.
Path 359 | total_timesteps 3990.
Path 360 | total_timesteps 3998.
Path 361 | total_timesteps 4007.
Path 362 | total_timesteps 4023.
Path 363 | total_timesteps 4031.
Path 364 | total_timesteps 4041.
Path 365 | total_timesteps 4051.
Path 366 | total_timesteps 4061.
Path 367 | total_timesteps 4074.
Path 368 | total_timesteps 4082.
Path 369 | total_timesteps 4096.
Path 370 | total_timesteps 4105.
Path 371 | total_timesteps 4117.
Path 372 | total_timesteps 4126.
Path 373 | total_timesteps 4141.
Path 374 | total_timesteps 4147.
Path 375 | total_timesteps 4157.
Path 376 | total_timesteps 4169.
Path 377 | total_timesteps 4177.
Path 378 | total_timesteps 4194.
Path 379 | total_timesteps 4206.
Path 380 | total_timesteps 4215.
Path 381 | total_timesteps 4226.
Path 382 | total_timesteps 4240.
Path 383 | total_timesteps 4248.
Path 384 | total_timesteps 4259.
Path 385 | total_timesteps 4270.
Path 386 | total_timesteps 4281.
Path 387 | total_timesteps 4293.
Path 388 | total_timesteps 4301.
Path 389 | total_timesteps 4311.
Path 390 | total_timesteps 4328.
Path 391 | total_timesteps 4341.
Path 392 | total_timesteps 4349.
Path 393 | total_timesteps 4360.
Path 394 | total_timesteps 4369.
Path 395 | total_timesteps 4386.
Path 396 | total_timesteps 4394.
Path 397 | total_timesteps 4410.
Path 398 | total_timesteps 4423.
Path 399 | total_timesteps 4432.
Path 400 | total_timesteps 4443.
Path 401 | total_timesteps 4457.
Path 402 | total_timesteps 4465.
Path 403 | total_timesteps 4472.
Path 404 | total_timesteps 4484.
Path 405 | total_timesteps 4494.
Path 406 | total_timesteps 4504.
Path 407 | total_timesteps 4513.
Path 408 | total_timesteps 4525.
Path 409 | total_timesteps 4536.
Path 410 | total_timesteps 4548.
Path 411 | total_timesteps 4558.
Path 412 | total_timesteps 4569.
Path 413 | total_timesteps 4588.
Path 414 | total_timesteps 4601.
Path 415 | total_timesteps 4609.
Path 416 | total_timesteps 4625.
Path 417 | total_timesteps 4635.
Path 418 | total_timesteps 4647.
Path 419 | total_timesteps 4654.
Path 420 | total_timesteps 4668.
Path 421 | total_timesteps 4683.
Path 422 | total_timesteps 4693.
Path 423 | total_timesteps 4705.
Path 424 | total_timesteps 4714.
Path 425 | total_timesteps 4741.
Path 426 | total_timesteps 4753.
Path 427 | total_timesteps 4764.
Path 428 | total_timesteps 4773.
Path 429 | total_timesteps 4788.
Path 430 | total_timesteps 4796.
Path 431 | total_timesteps 4806.
Path 432 | total_timesteps 4815.
Path 433 | total_timesteps 4828.
Path 434 | total_timesteps 4836.
Path 435 | total_timesteps 4845.
Path 436 | total_timesteps 4855.
Path 437 | total_timesteps 4864.
Path 438 | total_timesteps 4875.
Path 439 | total_timesteps 4896.
Path 440 | total_timesteps 4913.
Path 441 | total_timesteps 4927.
Path 442 | total_timesteps 4935.
Path 443 | total_timesteps 4945.
Path 444 | total_timesteps 4956.
Path 445 | total_timesteps 4976.
Path 446 | total_timesteps 4986.
Path 447 | total_timesteps 5005.
Path 448 | total_timesteps 5014.
Path 449 | total_timesteps 5022.
Path 450 | total_timesteps 5030.
Path 451 | total_timesteps 5037.
Path 452 | total_timesteps 5045.
Path 453 | total_timesteps 5055.
Path 454 | total_timesteps 5067.
Path 455 | total_timesteps 5074.
Path 456 | total_timesteps 5088.
Path 457 | total_timesteps 5097.
Path 458 | total_timesteps 5108.
Path 459 | total_timesteps 5118.
Path 460 | total_timesteps 5128.
Path 461 | total_timesteps 5137.
Path 462 | total_timesteps 5149.
Path 463 | total_timesteps 5162.
Path 464 | total_timesteps 5173.
Path 465 | total_timesteps 5185.
Path 466 | total_timesteps 5193.
Path 467 | total_timesteps 5200.
Path 468 | total_timesteps 5210.
Path 469 | total_timesteps 5219.
Path 470 | total_timesteps 5226.
Path 471 | total_timesteps 5234.
Path 472 | total_timesteps 5243.
Path 473 | total_timesteps 5253.
Path 474 | total_timesteps 5263.
Path 475 | total_timesteps 5272.
Path 476 | total_timesteps 5279.
Path 477 | total_timesteps 5286.
Path 478 | total_timesteps 5307.
Path 479 | total_timesteps 5317.
Path 480 | total_timesteps 5326.
Path 481 | total_timesteps 5334.
Path 482 | total_timesteps 5345.
Path 483 | total_timesteps 5359.
Path 484 | total_timesteps 5371.
Path 485 | total_timesteps 5381.
Path 486 | total_timesteps 5392.
Path 487 | total_timesteps 5401.
Path 488 | total_timesteps 5410.
Path 489 | total_timesteps 5418.
Path 490 | total_timesteps 5434.
Path 491 | total_timesteps 5444.
Path 492 | total_timesteps 5454.
Path 493 | total_timesteps 5463.
Path 494 | total_timesteps 5478.
Path 495 | total_timesteps 5488.
Path 496 | total_timesteps 5498.
Path 497 | total_timesteps 5510.
Path 498 | total_timesteps 5518.
Path 499 | total_timesteps 5529.
Path 500 | total_timesteps 5539.
Path 501 | total_timesteps 5547.
Path 502 | total_timesteps 5560.
Path 503 | total_timesteps 5572.
Path 504 | total_timesteps 5581.
Path 505 | total_timesteps 5599.
Path 506 | total_timesteps 5607.
Path 507 | total_timesteps 5625.
Path 508 | total_timesteps 5647.
Path 509 | total_timesteps 5661.
Path 510 | total_timesteps 5668.
Path 511 | total_timesteps 5679.
Path 512 | total_timesteps 5694.
Path 513 | total_timesteps 5703.
Path 514 | total_timesteps 5715.
Path 515 | total_timesteps 5724.
Path 516 | total_timesteps 5733.
Path 517 | total_timesteps 5741.
Path 518 | total_timesteps 5754.
Path 519 | total_timesteps 5762.
Path 520 | total_timesteps 5771.
Path 521 | total_timesteps 5779.
Path 522 | total_timesteps 5786.
Path 523 | total_timesteps 5796.
Path 524 | total_timesteps 5809.
Path 525 | total_timesteps 5819.
Path 526 | total_timesteps 5832.
Path 527 | total_timesteps 5841.
Path 528 | total_timesteps 5854.
Path 529 | total_timesteps 5865.
Path 530 | total_timesteps 5881.
Path 531 | total_timesteps 5895.
Path 532 | total_timesteps 5905.
Path 533 | total_timesteps 5913.
Path 534 | total_timesteps 5922.
Path 535 | total_timesteps 5934.
Path 536 | total_timesteps 5949.
Path 537 | total_timesteps 5960.
Path 538 | total_timesteps 5968.
Path 539 | total_timesteps 5975.
Path 540 | total_timesteps 5987.
Path 541 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.13    |
| Iteration     | 17       |
| MaximumReturn | -0.262   |
| MinimumReturn | -19.6    |
| TotalSamples  | 76069    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.31457388401031494
Validation loss = 0.31921476125717163
Validation loss = 0.32157906889915466
Validation loss = 0.3220857083797455
Validation loss = 0.3249022960662842
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 37.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 59.
Path 6 | total_timesteps 70.
Path 7 | total_timesteps 78.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 95.
Path 10 | total_timesteps 108.
Path 11 | total_timesteps 119.
Path 12 | total_timesteps 128.
Path 13 | total_timesteps 141.
Path 14 | total_timesteps 150.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 171.
Path 17 | total_timesteps 178.
Path 18 | total_timesteps 186.
Path 19 | total_timesteps 199.
Path 20 | total_timesteps 211.
Path 21 | total_timesteps 222.
Path 22 | total_timesteps 229.
Path 23 | total_timesteps 236.
Path 24 | total_timesteps 251.
Path 25 | total_timesteps 263.
Path 26 | total_timesteps 275.
Path 27 | total_timesteps 286.
Path 28 | total_timesteps 295.
Path 29 | total_timesteps 309.
Path 30 | total_timesteps 321.
Path 31 | total_timesteps 335.
Path 32 | total_timesteps 347.
Path 33 | total_timesteps 360.
Path 34 | total_timesteps 372.
Path 35 | total_timesteps 382.
Path 36 | total_timesteps 404.
Path 37 | total_timesteps 414.
Path 38 | total_timesteps 426.
Path 39 | total_timesteps 437.
Path 40 | total_timesteps 449.
Path 41 | total_timesteps 456.
Path 42 | total_timesteps 469.
Path 43 | total_timesteps 478.
Path 44 | total_timesteps 492.
Path 45 | total_timesteps 499.
Path 46 | total_timesteps 508.
Path 47 | total_timesteps 517.
Path 48 | total_timesteps 529.
Path 49 | total_timesteps 538.
Path 50 | total_timesteps 554.
Path 51 | total_timesteps 565.
Path 52 | total_timesteps 573.
Path 53 | total_timesteps 583.
Path 54 | total_timesteps 595.
Path 55 | total_timesteps 601.
Path 56 | total_timesteps 613.
Path 57 | total_timesteps 627.
Path 58 | total_timesteps 656.
Path 59 | total_timesteps 665.
Path 60 | total_timesteps 674.
Path 61 | total_timesteps 692.
Path 62 | total_timesteps 712.
Path 63 | total_timesteps 719.
Path 64 | total_timesteps 730.
Path 65 | total_timesteps 742.
Path 66 | total_timesteps 753.
Path 67 | total_timesteps 765.
Path 68 | total_timesteps 773.
Path 69 | total_timesteps 795.
Path 70 | total_timesteps 804.
Path 71 | total_timesteps 811.
Path 72 | total_timesteps 819.
Path 73 | total_timesteps 827.
Path 74 | total_timesteps 836.
Path 75 | total_timesteps 847.
Path 76 | total_timesteps 859.
Path 77 | total_timesteps 868.
Path 78 | total_timesteps 875.
Path 79 | total_timesteps 890.
Path 80 | total_timesteps 905.
Path 81 | total_timesteps 914.
Path 82 | total_timesteps 926.
Path 83 | total_timesteps 940.
Path 84 | total_timesteps 958.
Path 85 | total_timesteps 970.
Path 86 | total_timesteps 977.
Path 87 | total_timesteps 987.
Path 88 | total_timesteps 998.
Path 89 | total_timesteps 1012.
Path 90 | total_timesteps 1024.
Path 91 | total_timesteps 1033.
Path 92 | total_timesteps 1048.
Path 93 | total_timesteps 1062.
Path 94 | total_timesteps 1072.
Path 95 | total_timesteps 1081.
Path 96 | total_timesteps 1089.
Path 97 | total_timesteps 1100.
Path 98 | total_timesteps 1113.
Path 99 | total_timesteps 1123.
Path 100 | total_timesteps 1137.
Path 101 | total_timesteps 1148.
Path 102 | total_timesteps 1159.
Path 103 | total_timesteps 1179.
Path 104 | total_timesteps 1187.
Path 105 | total_timesteps 1196.
Path 106 | total_timesteps 1204.
Path 107 | total_timesteps 1213.
Path 108 | total_timesteps 1222.
Path 109 | total_timesteps 1229.
Path 110 | total_timesteps 1245.
Path 111 | total_timesteps 1273.
Path 112 | total_timesteps 1281.
Path 113 | total_timesteps 1291.
Path 114 | total_timesteps 1301.
Path 115 | total_timesteps 1309.
Path 116 | total_timesteps 1317.
Path 117 | total_timesteps 1326.
Path 118 | total_timesteps 1335.
Path 119 | total_timesteps 1349.
Path 120 | total_timesteps 1356.
Path 121 | total_timesteps 1368.
Path 122 | total_timesteps 1379.
Path 123 | total_timesteps 1389.
Path 124 | total_timesteps 1401.
Path 125 | total_timesteps 1408.
Path 126 | total_timesteps 1417.
Path 127 | total_timesteps 1428.
Path 128 | total_timesteps 1436.
Path 129 | total_timesteps 1444.
Path 130 | total_timesteps 1455.
Path 131 | total_timesteps 1464.
Path 132 | total_timesteps 1473.
Path 133 | total_timesteps 1484.
Path 134 | total_timesteps 1495.
Path 135 | total_timesteps 1508.
Path 136 | total_timesteps 1518.
Path 137 | total_timesteps 1526.
Path 138 | total_timesteps 1536.
Path 139 | total_timesteps 1548.
Path 140 | total_timesteps 1559.
Path 141 | total_timesteps 1567.
Path 142 | total_timesteps 1575.
Path 143 | total_timesteps 1587.
Path 144 | total_timesteps 1601.
Path 145 | total_timesteps 1615.
Path 146 | total_timesteps 1633.
Path 147 | total_timesteps 1642.
Path 148 | total_timesteps 1654.
Path 149 | total_timesteps 1663.
Path 150 | total_timesteps 1674.
Path 151 | total_timesteps 1682.
Path 152 | total_timesteps 1691.
Path 153 | total_timesteps 1703.
Path 154 | total_timesteps 1711.
Path 155 | total_timesteps 1720.
Path 156 | total_timesteps 1731.
Path 157 | total_timesteps 1751.
Path 158 | total_timesteps 1759.
Path 159 | total_timesteps 1773.
Path 160 | total_timesteps 1794.
Path 161 | total_timesteps 1803.
Path 162 | total_timesteps 1812.
Path 163 | total_timesteps 1819.
Path 164 | total_timesteps 1827.
Path 165 | total_timesteps 1835.
Path 166 | total_timesteps 1848.
Path 167 | total_timesteps 1857.
Path 168 | total_timesteps 1872.
Path 169 | total_timesteps 1889.
Path 170 | total_timesteps 1897.
Path 171 | total_timesteps 1907.
Path 172 | total_timesteps 1920.
Path 173 | total_timesteps 1937.
Path 174 | total_timesteps 1945.
Path 175 | total_timesteps 1958.
Path 176 | total_timesteps 1972.
Path 177 | total_timesteps 1980.
Path 178 | total_timesteps 1992.
Path 179 | total_timesteps 2003.
Path 180 | total_timesteps 2015.
Path 181 | total_timesteps 2024.
Path 182 | total_timesteps 2032.
Path 183 | total_timesteps 2040.
Path 184 | total_timesteps 2048.
Path 185 | total_timesteps 2057.
Path 186 | total_timesteps 2065.
Path 187 | total_timesteps 2075.
Path 188 | total_timesteps 2087.
Path 189 | total_timesteps 2094.
Path 190 | total_timesteps 2102.
Path 191 | total_timesteps 2111.
Path 192 | total_timesteps 2120.
Path 193 | total_timesteps 2128.
Path 194 | total_timesteps 2136.
Path 195 | total_timesteps 2143.
Path 196 | total_timesteps 2153.
Path 197 | total_timesteps 2162.
Path 198 | total_timesteps 2169.
Path 199 | total_timesteps 2178.
Path 200 | total_timesteps 2193.
Path 201 | total_timesteps 2205.
Path 202 | total_timesteps 2217.
Path 203 | total_timesteps 2228.
Path 204 | total_timesteps 2242.
Path 205 | total_timesteps 2251.
Path 206 | total_timesteps 2263.
Path 207 | total_timesteps 2275.
Path 208 | total_timesteps 2283.
Path 209 | total_timesteps 2290.
Path 210 | total_timesteps 2298.
Path 211 | total_timesteps 2306.
Path 212 | total_timesteps 2317.
Path 213 | total_timesteps 2329.
Path 214 | total_timesteps 2345.
Path 215 | total_timesteps 2357.
Path 216 | total_timesteps 2372.
Path 217 | total_timesteps 2386.
Path 218 | total_timesteps 2399.
Path 219 | total_timesteps 2409.
Path 220 | total_timesteps 2420.
Path 221 | total_timesteps 2429.
Path 222 | total_timesteps 2440.
Path 223 | total_timesteps 2449.
Path 224 | total_timesteps 2458.
Path 225 | total_timesteps 2468.
Path 226 | total_timesteps 2477.
Path 227 | total_timesteps 2491.
Path 228 | total_timesteps 2501.
Path 229 | total_timesteps 2510.
Path 230 | total_timesteps 2523.
Path 231 | total_timesteps 2532.
Path 232 | total_timesteps 2542.
Path 233 | total_timesteps 2555.
Path 234 | total_timesteps 2573.
Path 235 | total_timesteps 2583.
Path 236 | total_timesteps 2590.
Path 237 | total_timesteps 2600.
Path 238 | total_timesteps 2610.
Path 239 | total_timesteps 2617.
Path 240 | total_timesteps 2624.
Path 241 | total_timesteps 2634.
Path 242 | total_timesteps 2641.
Path 243 | total_timesteps 2662.
Path 244 | total_timesteps 2674.
Path 245 | total_timesteps 2686.
Path 246 | total_timesteps 2692.
Path 247 | total_timesteps 2705.
Path 248 | total_timesteps 2717.
Path 249 | total_timesteps 2726.
Path 250 | total_timesteps 2735.
Path 251 | total_timesteps 2743.
Path 252 | total_timesteps 2752.
Path 253 | total_timesteps 2762.
Path 254 | total_timesteps 2777.
Path 255 | total_timesteps 2787.
Path 256 | total_timesteps 2800.
Path 257 | total_timesteps 2811.
Path 258 | total_timesteps 2821.
Path 259 | total_timesteps 2834.
Path 260 | total_timesteps 2847.
Path 261 | total_timesteps 2859.
Path 262 | total_timesteps 2873.
Path 263 | total_timesteps 2885.
Path 264 | total_timesteps 2903.
Path 265 | total_timesteps 2920.
Path 266 | total_timesteps 2930.
Path 267 | total_timesteps 2940.
Path 268 | total_timesteps 2950.
Path 269 | total_timesteps 2965.
Path 270 | total_timesteps 2976.
Path 271 | total_timesteps 2984.
Path 272 | total_timesteps 2992.
Path 273 | total_timesteps 3004.
Path 274 | total_timesteps 3014.
Path 275 | total_timesteps 3024.
Path 276 | total_timesteps 3037.
Path 277 | total_timesteps 3046.
Path 278 | total_timesteps 3059.
Path 279 | total_timesteps 3067.
Path 280 | total_timesteps 3074.
Path 281 | total_timesteps 3084.
Path 282 | total_timesteps 3095.
Path 283 | total_timesteps 3104.
Path 284 | total_timesteps 3117.
Path 285 | total_timesteps 3125.
Path 286 | total_timesteps 3135.
Path 287 | total_timesteps 3145.
Path 288 | total_timesteps 3158.
Path 289 | total_timesteps 3168.
Path 290 | total_timesteps 3176.
Path 291 | total_timesteps 3184.
Path 292 | total_timesteps 3193.
Path 293 | total_timesteps 3204.
Path 294 | total_timesteps 3216.
Path 295 | total_timesteps 3225.
Path 296 | total_timesteps 3234.
Path 297 | total_timesteps 3245.
Path 298 | total_timesteps 3260.
Path 299 | total_timesteps 3285.
Path 300 | total_timesteps 3295.
Path 301 | total_timesteps 3306.
Path 302 | total_timesteps 3316.
Path 303 | total_timesteps 3325.
Path 304 | total_timesteps 3338.
Path 305 | total_timesteps 3349.
Path 306 | total_timesteps 3359.
Path 307 | total_timesteps 3371.
Path 308 | total_timesteps 3385.
Path 309 | total_timesteps 3396.
Path 310 | total_timesteps 3403.
Path 311 | total_timesteps 3415.
Path 312 | total_timesteps 3424.
Path 313 | total_timesteps 3432.
Path 314 | total_timesteps 3442.
Path 315 | total_timesteps 3452.
Path 316 | total_timesteps 3462.
Path 317 | total_timesteps 3471.
Path 318 | total_timesteps 3483.
Path 319 | total_timesteps 3495.
Path 320 | total_timesteps 3508.
Path 321 | total_timesteps 3518.
Path 322 | total_timesteps 3540.
Path 323 | total_timesteps 3555.
Path 324 | total_timesteps 3565.
Path 325 | total_timesteps 3580.
Path 326 | total_timesteps 3589.
Path 327 | total_timesteps 3601.
Path 328 | total_timesteps 3614.
Path 329 | total_timesteps 3626.
Path 330 | total_timesteps 3637.
Path 331 | total_timesteps 3652.
Path 332 | total_timesteps 3662.
Path 333 | total_timesteps 3670.
Path 334 | total_timesteps 3681.
Path 335 | total_timesteps 3691.
Path 336 | total_timesteps 3701.
Path 337 | total_timesteps 3712.
Path 338 | total_timesteps 3723.
Path 339 | total_timesteps 3730.
Path 340 | total_timesteps 3745.
Path 341 | total_timesteps 3752.
Path 342 | total_timesteps 3760.
Path 343 | total_timesteps 3767.
Path 344 | total_timesteps 3775.
Path 345 | total_timesteps 3787.
Path 346 | total_timesteps 3798.
Path 347 | total_timesteps 3805.
Path 348 | total_timesteps 3813.
Path 349 | total_timesteps 3822.
Path 350 | total_timesteps 3832.
Path 351 | total_timesteps 3841.
Path 352 | total_timesteps 3852.
Path 353 | total_timesteps 3865.
Path 354 | total_timesteps 3874.
Path 355 | total_timesteps 3886.
Path 356 | total_timesteps 3895.
Path 357 | total_timesteps 3906.
Path 358 | total_timesteps 3917.
Path 359 | total_timesteps 3926.
Path 360 | total_timesteps 3937.
Path 361 | total_timesteps 3947.
Path 362 | total_timesteps 3957.
Path 363 | total_timesteps 3970.
Path 364 | total_timesteps 3981.
Path 365 | total_timesteps 3994.
Path 366 | total_timesteps 4003.
Path 367 | total_timesteps 4021.
Path 368 | total_timesteps 4032.
Path 369 | total_timesteps 4040.
Path 370 | total_timesteps 4048.
Path 371 | total_timesteps 4059.
Path 372 | total_timesteps 4070.
Path 373 | total_timesteps 4082.
Path 374 | total_timesteps 4093.
Path 375 | total_timesteps 4104.
Path 376 | total_timesteps 4114.
Path 377 | total_timesteps 4122.
Path 378 | total_timesteps 4130.
Path 379 | total_timesteps 4146.
Path 380 | total_timesteps 4158.
Path 381 | total_timesteps 4165.
Path 382 | total_timesteps 4176.
Path 383 | total_timesteps 4187.
Path 384 | total_timesteps 4197.
Path 385 | total_timesteps 4215.
Path 386 | total_timesteps 4222.
Path 387 | total_timesteps 4240.
Path 388 | total_timesteps 4252.
Path 389 | total_timesteps 4264.
Path 390 | total_timesteps 4277.
Path 391 | total_timesteps 4286.
Path 392 | total_timesteps 4297.
Path 393 | total_timesteps 4305.
Path 394 | total_timesteps 4319.
Path 395 | total_timesteps 4326.
Path 396 | total_timesteps 4333.
Path 397 | total_timesteps 4341.
Path 398 | total_timesteps 4354.
Path 399 | total_timesteps 4365.
Path 400 | total_timesteps 4380.
Path 401 | total_timesteps 4393.
Path 402 | total_timesteps 4402.
Path 403 | total_timesteps 4410.
Path 404 | total_timesteps 4419.
Path 405 | total_timesteps 4427.
Path 406 | total_timesteps 4435.
Path 407 | total_timesteps 4444.
Path 408 | total_timesteps 4453.
Path 409 | total_timesteps 4462.
Path 410 | total_timesteps 4475.
Path 411 | total_timesteps 4485.
Path 412 | total_timesteps 4501.
Path 413 | total_timesteps 4511.
Path 414 | total_timesteps 4520.
Path 415 | total_timesteps 4528.
Path 416 | total_timesteps 4536.
Path 417 | total_timesteps 4553.
Path 418 | total_timesteps 4561.
Path 419 | total_timesteps 4571.
Path 420 | total_timesteps 4581.
Path 421 | total_timesteps 4596.
Path 422 | total_timesteps 4604.
Path 423 | total_timesteps 4612.
Path 424 | total_timesteps 4622.
Path 425 | total_timesteps 4635.
Path 426 | total_timesteps 4645.
Path 427 | total_timesteps 4659.
Path 428 | total_timesteps 4671.
Path 429 | total_timesteps 4682.
Path 430 | total_timesteps 4696.
Path 431 | total_timesteps 4708.
Path 432 | total_timesteps 4719.
Path 433 | total_timesteps 4731.
Path 434 | total_timesteps 4740.
Path 435 | total_timesteps 4756.
Path 436 | total_timesteps 4766.
Path 437 | total_timesteps 4776.
Path 438 | total_timesteps 4785.
Path 439 | total_timesteps 4791.
Path 440 | total_timesteps 4802.
Path 441 | total_timesteps 4809.
Path 442 | total_timesteps 4821.
Path 443 | total_timesteps 4830.
Path 444 | total_timesteps 4839.
Path 445 | total_timesteps 4849.
Path 446 | total_timesteps 4856.
Path 447 | total_timesteps 4865.
Path 448 | total_timesteps 4874.
Path 449 | total_timesteps 4892.
Path 450 | total_timesteps 4900.
Path 451 | total_timesteps 4913.
Path 452 | total_timesteps 4920.
Path 453 | total_timesteps 4928.
Path 454 | total_timesteps 4939.
Path 455 | total_timesteps 4954.
Path 456 | total_timesteps 4963.
Path 457 | total_timesteps 4974.
Path 458 | total_timesteps 4982.
Path 459 | total_timesteps 4990.
Path 460 | total_timesteps 5002.
Path 461 | total_timesteps 5009.
Path 462 | total_timesteps 5019.
Path 463 | total_timesteps 5032.
Path 464 | total_timesteps 5041.
Path 465 | total_timesteps 5049.
Path 466 | total_timesteps 5058.
Path 467 | total_timesteps 5070.
Path 468 | total_timesteps 5081.
Path 469 | total_timesteps 5089.
Path 470 | total_timesteps 5099.
Path 471 | total_timesteps 5108.
Path 472 | total_timesteps 5120.
Path 473 | total_timesteps 5139.
Path 474 | total_timesteps 5149.
Path 475 | total_timesteps 5157.
Path 476 | total_timesteps 5168.
Path 477 | total_timesteps 5179.
Path 478 | total_timesteps 5195.
Path 479 | total_timesteps 5210.
Path 480 | total_timesteps 5218.
Path 481 | total_timesteps 5233.
Path 482 | total_timesteps 5246.
Path 483 | total_timesteps 5259.
Path 484 | total_timesteps 5269.
Path 485 | total_timesteps 5282.
Path 486 | total_timesteps 5290.
Path 487 | total_timesteps 5300.
Path 488 | total_timesteps 5309.
Path 489 | total_timesteps 5319.
Path 490 | total_timesteps 5328.
Path 491 | total_timesteps 5339.
Path 492 | total_timesteps 5347.
Path 493 | total_timesteps 5357.
Path 494 | total_timesteps 5365.
Path 495 | total_timesteps 5381.
Path 496 | total_timesteps 5394.
Path 497 | total_timesteps 5407.
Path 498 | total_timesteps 5416.
Path 499 | total_timesteps 5425.
Path 500 | total_timesteps 5437.
Path 501 | total_timesteps 5446.
Path 502 | total_timesteps 5473.
Path 503 | total_timesteps 5486.
Path 504 | total_timesteps 5495.
Path 505 | total_timesteps 5505.
Path 506 | total_timesteps 5515.
Path 507 | total_timesteps 5527.
Path 508 | total_timesteps 5537.
Path 509 | total_timesteps 5552.
Path 510 | total_timesteps 5559.
Path 511 | total_timesteps 5568.
Path 512 | total_timesteps 5583.
Path 513 | total_timesteps 5591.
Path 514 | total_timesteps 5599.
Path 515 | total_timesteps 5611.
Path 516 | total_timesteps 5621.
Path 517 | total_timesteps 5629.
Path 518 | total_timesteps 5638.
Path 519 | total_timesteps 5649.
Path 520 | total_timesteps 5660.
Path 521 | total_timesteps 5669.
Path 522 | total_timesteps 5675.
Path 523 | total_timesteps 5682.
Path 524 | total_timesteps 5700.
Path 525 | total_timesteps 5711.
Path 526 | total_timesteps 5720.
Path 527 | total_timesteps 5730.
Path 528 | total_timesteps 5744.
Path 529 | total_timesteps 5752.
Path 530 | total_timesteps 5762.
Path 531 | total_timesteps 5773.
Path 532 | total_timesteps 5783.
Path 533 | total_timesteps 5802.
Path 534 | total_timesteps 5811.
Path 535 | total_timesteps 5819.
Path 536 | total_timesteps 5840.
Path 537 | total_timesteps 5850.
Path 538 | total_timesteps 5867.
Path 539 | total_timesteps 5876.
Path 540 | total_timesteps 5887.
Path 541 | total_timesteps 5897.
Path 542 | total_timesteps 5910.
Path 543 | total_timesteps 5921.
Path 544 | total_timesteps 5936.
Path 545 | total_timesteps 5944.
Path 546 | total_timesteps 5954.
Path 547 | total_timesteps 5963.
Path 548 | total_timesteps 5970.
Path 549 | total_timesteps 5977.
Path 550 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.86    |
| Iteration     | 18       |
| MaximumReturn | 3.68     |
| MinimumReturn | -20.7    |
| TotalSamples  | 80075    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.3160029351711273
Validation loss = 0.3178556561470032
Validation loss = 0.31965598464012146
Validation loss = 0.32391852140426636
Validation loss = 0.32516542077064514
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 52.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 71.
Path 7 | total_timesteps 86.
Path 8 | total_timesteps 96.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 116.
Path 11 | total_timesteps 129.
Path 12 | total_timesteps 144.
Path 13 | total_timesteps 153.
Path 14 | total_timesteps 163.
Path 15 | total_timesteps 174.
Path 16 | total_timesteps 182.
Path 17 | total_timesteps 191.
Path 18 | total_timesteps 202.
Path 19 | total_timesteps 214.
Path 20 | total_timesteps 221.
Path 21 | total_timesteps 230.
Path 22 | total_timesteps 245.
Path 23 | total_timesteps 255.
Path 24 | total_timesteps 266.
Path 25 | total_timesteps 274.
Path 26 | total_timesteps 283.
Path 27 | total_timesteps 292.
Path 28 | total_timesteps 312.
Path 29 | total_timesteps 329.
Path 30 | total_timesteps 339.
Path 31 | total_timesteps 348.
Path 32 | total_timesteps 357.
Path 33 | total_timesteps 366.
Path 34 | total_timesteps 377.
Path 35 | total_timesteps 383.
Path 36 | total_timesteps 395.
Path 37 | total_timesteps 405.
Path 38 | total_timesteps 413.
Path 39 | total_timesteps 421.
Path 40 | total_timesteps 431.
Path 41 | total_timesteps 439.
Path 42 | total_timesteps 449.
Path 43 | total_timesteps 463.
Path 44 | total_timesteps 478.
Path 45 | total_timesteps 486.
Path 46 | total_timesteps 499.
Path 47 | total_timesteps 509.
Path 48 | total_timesteps 519.
Path 49 | total_timesteps 529.
Path 50 | total_timesteps 540.
Path 51 | total_timesteps 548.
Path 52 | total_timesteps 557.
Path 53 | total_timesteps 565.
Path 54 | total_timesteps 578.
Path 55 | total_timesteps 587.
Path 56 | total_timesteps 598.
Path 57 | total_timesteps 608.
Path 58 | total_timesteps 619.
Path 59 | total_timesteps 627.
Path 60 | total_timesteps 635.
Path 61 | total_timesteps 644.
Path 62 | total_timesteps 652.
Path 63 | total_timesteps 662.
Path 64 | total_timesteps 669.
Path 65 | total_timesteps 684.
Path 66 | total_timesteps 691.
Path 67 | total_timesteps 703.
Path 68 | total_timesteps 715.
Path 69 | total_timesteps 724.
Path 70 | total_timesteps 733.
Path 71 | total_timesteps 740.
Path 72 | total_timesteps 748.
Path 73 | total_timesteps 760.
Path 74 | total_timesteps 768.
Path 75 | total_timesteps 778.
Path 76 | total_timesteps 789.
Path 77 | total_timesteps 799.
Path 78 | total_timesteps 808.
Path 79 | total_timesteps 817.
Path 80 | total_timesteps 825.
Path 81 | total_timesteps 837.
Path 82 | total_timesteps 848.
Path 83 | total_timesteps 858.
Path 84 | total_timesteps 869.
Path 85 | total_timesteps 877.
Path 86 | total_timesteps 886.
Path 87 | total_timesteps 902.
Path 88 | total_timesteps 914.
Path 89 | total_timesteps 920.
Path 90 | total_timesteps 927.
Path 91 | total_timesteps 937.
Path 92 | total_timesteps 949.
Path 93 | total_timesteps 960.
Path 94 | total_timesteps 971.
Path 95 | total_timesteps 983.
Path 96 | total_timesteps 993.
Path 97 | total_timesteps 1004.
Path 98 | total_timesteps 1015.
Path 99 | total_timesteps 1025.
Path 100 | total_timesteps 1033.
Path 101 | total_timesteps 1041.
Path 102 | total_timesteps 1049.
Path 103 | total_timesteps 1061.
Path 104 | total_timesteps 1074.
Path 105 | total_timesteps 1088.
Path 106 | total_timesteps 1099.
Path 107 | total_timesteps 1109.
Path 108 | total_timesteps 1120.
Path 109 | total_timesteps 1129.
Path 110 | total_timesteps 1140.
Path 111 | total_timesteps 1149.
Path 112 | total_timesteps 1158.
Path 113 | total_timesteps 1170.
Path 114 | total_timesteps 1180.
Path 115 | total_timesteps 1189.
Path 116 | total_timesteps 1198.
Path 117 | total_timesteps 1215.
Path 118 | total_timesteps 1223.
Path 119 | total_timesteps 1234.
Path 120 | total_timesteps 1244.
Path 121 | total_timesteps 1255.
Path 122 | total_timesteps 1265.
Path 123 | total_timesteps 1277.
Path 124 | total_timesteps 1287.
Path 125 | total_timesteps 1294.
Path 126 | total_timesteps 1304.
Path 127 | total_timesteps 1311.
Path 128 | total_timesteps 1319.
Path 129 | total_timesteps 1332.
Path 130 | total_timesteps 1340.
Path 131 | total_timesteps 1350.
Path 132 | total_timesteps 1362.
Path 133 | total_timesteps 1372.
Path 134 | total_timesteps 1380.
Path 135 | total_timesteps 1386.
Path 136 | total_timesteps 1396.
Path 137 | total_timesteps 1406.
Path 138 | total_timesteps 1419.
Path 139 | total_timesteps 1428.
Path 140 | total_timesteps 1435.
Path 141 | total_timesteps 1443.
Path 142 | total_timesteps 1456.
Path 143 | total_timesteps 1467.
Path 144 | total_timesteps 1477.
Path 145 | total_timesteps 1488.
Path 146 | total_timesteps 1500.
Path 147 | total_timesteps 1513.
Path 148 | total_timesteps 1524.
Path 149 | total_timesteps 1534.
Path 150 | total_timesteps 1542.
Path 151 | total_timesteps 1554.
Path 152 | total_timesteps 1571.
Path 153 | total_timesteps 1579.
Path 154 | total_timesteps 1591.
Path 155 | total_timesteps 1604.
Path 156 | total_timesteps 1614.
Path 157 | total_timesteps 1622.
Path 158 | total_timesteps 1630.
Path 159 | total_timesteps 1641.
Path 160 | total_timesteps 1650.
Path 161 | total_timesteps 1666.
Path 162 | total_timesteps 1677.
Path 163 | total_timesteps 1685.
Path 164 | total_timesteps 1692.
Path 165 | total_timesteps 1706.
Path 166 | total_timesteps 1718.
Path 167 | total_timesteps 1728.
Path 168 | total_timesteps 1737.
Path 169 | total_timesteps 1746.
Path 170 | total_timesteps 1755.
Path 171 | total_timesteps 1769.
Path 172 | total_timesteps 1778.
Path 173 | total_timesteps 1790.
Path 174 | total_timesteps 1804.
Path 175 | total_timesteps 1818.
Path 176 | total_timesteps 1827.
Path 177 | total_timesteps 1843.
Path 178 | total_timesteps 1854.
Path 179 | total_timesteps 1863.
Path 180 | total_timesteps 1872.
Path 181 | total_timesteps 1884.
Path 182 | total_timesteps 1893.
Path 183 | total_timesteps 1901.
Path 184 | total_timesteps 1910.
Path 185 | total_timesteps 1920.
Path 186 | total_timesteps 1929.
Path 187 | total_timesteps 1942.
Path 188 | total_timesteps 1950.
Path 189 | total_timesteps 1957.
Path 190 | total_timesteps 1966.
Path 191 | total_timesteps 1983.
Path 192 | total_timesteps 1997.
Path 193 | total_timesteps 2009.
Path 194 | total_timesteps 2019.
Path 195 | total_timesteps 2028.
Path 196 | total_timesteps 2039.
Path 197 | total_timesteps 2048.
Path 198 | total_timesteps 2060.
Path 199 | total_timesteps 2069.
Path 200 | total_timesteps 2076.
Path 201 | total_timesteps 2090.
Path 202 | total_timesteps 2105.
Path 203 | total_timesteps 2118.
Path 204 | total_timesteps 2130.
Path 205 | total_timesteps 2142.
Path 206 | total_timesteps 2150.
Path 207 | total_timesteps 2158.
Path 208 | total_timesteps 2167.
Path 209 | total_timesteps 2176.
Path 210 | total_timesteps 2189.
Path 211 | total_timesteps 2198.
Path 212 | total_timesteps 2209.
Path 213 | total_timesteps 2217.
Path 214 | total_timesteps 2226.
Path 215 | total_timesteps 2233.
Path 216 | total_timesteps 2244.
Path 217 | total_timesteps 2254.
Path 218 | total_timesteps 2262.
Path 219 | total_timesteps 2273.
Path 220 | total_timesteps 2286.
Path 221 | total_timesteps 2296.
Path 222 | total_timesteps 2306.
Path 223 | total_timesteps 2315.
Path 224 | total_timesteps 2326.
Path 225 | total_timesteps 2337.
Path 226 | total_timesteps 2347.
Path 227 | total_timesteps 2355.
Path 228 | total_timesteps 2364.
Path 229 | total_timesteps 2372.
Path 230 | total_timesteps 2382.
Path 231 | total_timesteps 2390.
Path 232 | total_timesteps 2398.
Path 233 | total_timesteps 2410.
Path 234 | total_timesteps 2419.
Path 235 | total_timesteps 2427.
Path 236 | total_timesteps 2434.
Path 237 | total_timesteps 2446.
Path 238 | total_timesteps 2457.
Path 239 | total_timesteps 2471.
Path 240 | total_timesteps 2482.
Path 241 | total_timesteps 2496.
Path 242 | total_timesteps 2507.
Path 243 | total_timesteps 2514.
Path 244 | total_timesteps 2527.
Path 245 | total_timesteps 2536.
Path 246 | total_timesteps 2549.
Path 247 | total_timesteps 2558.
Path 248 | total_timesteps 2565.
Path 249 | total_timesteps 2573.
Path 250 | total_timesteps 2583.
Path 251 | total_timesteps 2590.
Path 252 | total_timesteps 2601.
Path 253 | total_timesteps 2608.
Path 254 | total_timesteps 2615.
Path 255 | total_timesteps 2622.
Path 256 | total_timesteps 2630.
Path 257 | total_timesteps 2640.
Path 258 | total_timesteps 2654.
Path 259 | total_timesteps 2667.
Path 260 | total_timesteps 2676.
Path 261 | total_timesteps 2684.
Path 262 | total_timesteps 2693.
Path 263 | total_timesteps 2708.
Path 264 | total_timesteps 2718.
Path 265 | total_timesteps 2726.
Path 266 | total_timesteps 2737.
Path 267 | total_timesteps 2745.
Path 268 | total_timesteps 2755.
Path 269 | total_timesteps 2764.
Path 270 | total_timesteps 2774.
Path 271 | total_timesteps 2782.
Path 272 | total_timesteps 2790.
Path 273 | total_timesteps 2801.
Path 274 | total_timesteps 2812.
Path 275 | total_timesteps 2820.
Path 276 | total_timesteps 2828.
Path 277 | total_timesteps 2836.
Path 278 | total_timesteps 2844.
Path 279 | total_timesteps 2851.
Path 280 | total_timesteps 2860.
Path 281 | total_timesteps 2867.
Path 282 | total_timesteps 2878.
Path 283 | total_timesteps 2889.
Path 284 | total_timesteps 2901.
Path 285 | total_timesteps 2910.
Path 286 | total_timesteps 2920.
Path 287 | total_timesteps 2932.
Path 288 | total_timesteps 2940.
Path 289 | total_timesteps 2951.
Path 290 | total_timesteps 2963.
Path 291 | total_timesteps 2976.
Path 292 | total_timesteps 2987.
Path 293 | total_timesteps 2999.
Path 294 | total_timesteps 3013.
Path 295 | total_timesteps 3022.
Path 296 | total_timesteps 3033.
Path 297 | total_timesteps 3042.
Path 298 | total_timesteps 3051.
Path 299 | total_timesteps 3065.
Path 300 | total_timesteps 3080.
Path 301 | total_timesteps 3087.
Path 302 | total_timesteps 3097.
Path 303 | total_timesteps 3105.
Path 304 | total_timesteps 3114.
Path 305 | total_timesteps 3121.
Path 306 | total_timesteps 3131.
Path 307 | total_timesteps 3139.
Path 308 | total_timesteps 3150.
Path 309 | total_timesteps 3159.
Path 310 | total_timesteps 3171.
Path 311 | total_timesteps 3185.
Path 312 | total_timesteps 3194.
Path 313 | total_timesteps 3206.
Path 314 | total_timesteps 3223.
Path 315 | total_timesteps 3237.
Path 316 | total_timesteps 3244.
Path 317 | total_timesteps 3256.
Path 318 | total_timesteps 3266.
Path 319 | total_timesteps 3274.
Path 320 | total_timesteps 3282.
Path 321 | total_timesteps 3290.
Path 322 | total_timesteps 3302.
Path 323 | total_timesteps 3312.
Path 324 | total_timesteps 3322.
Path 325 | total_timesteps 3330.
Path 326 | total_timesteps 3340.
Path 327 | total_timesteps 3347.
Path 328 | total_timesteps 3357.
Path 329 | total_timesteps 3366.
Path 330 | total_timesteps 3374.
Path 331 | total_timesteps 3400.
Path 332 | total_timesteps 3408.
Path 333 | total_timesteps 3416.
Path 334 | total_timesteps 3423.
Path 335 | total_timesteps 3435.
Path 336 | total_timesteps 3444.
Path 337 | total_timesteps 3456.
Path 338 | total_timesteps 3467.
Path 339 | total_timesteps 3475.
Path 340 | total_timesteps 3485.
Path 341 | total_timesteps 3497.
Path 342 | total_timesteps 3505.
Path 343 | total_timesteps 3514.
Path 344 | total_timesteps 3528.
Path 345 | total_timesteps 3542.
Path 346 | total_timesteps 3555.
Path 347 | total_timesteps 3567.
Path 348 | total_timesteps 3576.
Path 349 | total_timesteps 3586.
Path 350 | total_timesteps 3605.
Path 351 | total_timesteps 3613.
Path 352 | total_timesteps 3624.
Path 353 | total_timesteps 3643.
Path 354 | total_timesteps 3651.
Path 355 | total_timesteps 3659.
Path 356 | total_timesteps 3670.
Path 357 | total_timesteps 3686.
Path 358 | total_timesteps 3696.
Path 359 | total_timesteps 3707.
Path 360 | total_timesteps 3718.
Path 361 | total_timesteps 3724.
Path 362 | total_timesteps 3736.
Path 363 | total_timesteps 3745.
Path 364 | total_timesteps 3754.
Path 365 | total_timesteps 3764.
Path 366 | total_timesteps 3776.
Path 367 | total_timesteps 3785.
Path 368 | total_timesteps 3797.
Path 369 | total_timesteps 3807.
Path 370 | total_timesteps 3816.
Path 371 | total_timesteps 3829.
Path 372 | total_timesteps 3839.
Path 373 | total_timesteps 3859.
Path 374 | total_timesteps 3876.
Path 375 | total_timesteps 3888.
Path 376 | total_timesteps 3897.
Path 377 | total_timesteps 3908.
Path 378 | total_timesteps 3919.
Path 379 | total_timesteps 3930.
Path 380 | total_timesteps 3946.
Path 381 | total_timesteps 3954.
Path 382 | total_timesteps 3962.
Path 383 | total_timesteps 3969.
Path 384 | total_timesteps 3982.
Path 385 | total_timesteps 3993.
Path 386 | total_timesteps 4002.
Path 387 | total_timesteps 4014.
Path 388 | total_timesteps 4023.
Path 389 | total_timesteps 4032.
Path 390 | total_timesteps 4042.
Path 391 | total_timesteps 4052.
Path 392 | total_timesteps 4062.
Path 393 | total_timesteps 4079.
Path 394 | total_timesteps 4091.
Path 395 | total_timesteps 4110.
Path 396 | total_timesteps 4120.
Path 397 | total_timesteps 4128.
Path 398 | total_timesteps 4136.
Path 399 | total_timesteps 4144.
Path 400 | total_timesteps 4154.
Path 401 | total_timesteps 4161.
Path 402 | total_timesteps 4173.
Path 403 | total_timesteps 4189.
Path 404 | total_timesteps 4197.
Path 405 | total_timesteps 4208.
Path 406 | total_timesteps 4219.
Path 407 | total_timesteps 4226.
Path 408 | total_timesteps 4236.
Path 409 | total_timesteps 4249.
Path 410 | total_timesteps 4262.
Path 411 | total_timesteps 4271.
Path 412 | total_timesteps 4280.
Path 413 | total_timesteps 4290.
Path 414 | total_timesteps 4304.
Path 415 | total_timesteps 4313.
Path 416 | total_timesteps 4321.
Path 417 | total_timesteps 4330.
Path 418 | total_timesteps 4340.
Path 419 | total_timesteps 4355.
Path 420 | total_timesteps 4365.
Path 421 | total_timesteps 4372.
Path 422 | total_timesteps 4380.
Path 423 | total_timesteps 4388.
Path 424 | total_timesteps 4401.
Path 425 | total_timesteps 4411.
Path 426 | total_timesteps 4420.
Path 427 | total_timesteps 4435.
Path 428 | total_timesteps 4444.
Path 429 | total_timesteps 4458.
Path 430 | total_timesteps 4469.
Path 431 | total_timesteps 4478.
Path 432 | total_timesteps 4495.
Path 433 | total_timesteps 4504.
Path 434 | total_timesteps 4515.
Path 435 | total_timesteps 4524.
Path 436 | total_timesteps 4538.
Path 437 | total_timesteps 4549.
Path 438 | total_timesteps 4559.
Path 439 | total_timesteps 4566.
Path 440 | total_timesteps 4575.
Path 441 | total_timesteps 4587.
Path 442 | total_timesteps 4598.
Path 443 | total_timesteps 4608.
Path 444 | total_timesteps 4617.
Path 445 | total_timesteps 4627.
Path 446 | total_timesteps 4636.
Path 447 | total_timesteps 4650.
Path 448 | total_timesteps 4659.
Path 449 | total_timesteps 4668.
Path 450 | total_timesteps 4675.
Path 451 | total_timesteps 4688.
Path 452 | total_timesteps 4696.
Path 453 | total_timesteps 4707.
Path 454 | total_timesteps 4720.
Path 455 | total_timesteps 4736.
Path 456 | total_timesteps 4746.
Path 457 | total_timesteps 4754.
Path 458 | total_timesteps 4765.
Path 459 | total_timesteps 4775.
Path 460 | total_timesteps 4786.
Path 461 | total_timesteps 4793.
Path 462 | total_timesteps 4807.
Path 463 | total_timesteps 4817.
Path 464 | total_timesteps 4824.
Path 465 | total_timesteps 4835.
Path 466 | total_timesteps 4846.
Path 467 | total_timesteps 4854.
Path 468 | total_timesteps 4861.
Path 469 | total_timesteps 4877.
Path 470 | total_timesteps 4883.
Path 471 | total_timesteps 4890.
Path 472 | total_timesteps 4904.
Path 473 | total_timesteps 4913.
Path 474 | total_timesteps 4931.
Path 475 | total_timesteps 4946.
Path 476 | total_timesteps 4955.
Path 477 | total_timesteps 4964.
Path 478 | total_timesteps 4973.
Path 479 | total_timesteps 4982.
Path 480 | total_timesteps 4995.
Path 481 | total_timesteps 5013.
Path 482 | total_timesteps 5023.
Path 483 | total_timesteps 5036.
Path 484 | total_timesteps 5049.
Path 485 | total_timesteps 5059.
Path 486 | total_timesteps 5071.
Path 487 | total_timesteps 5085.
Path 488 | total_timesteps 5092.
Path 489 | total_timesteps 5101.
Path 490 | total_timesteps 5110.
Path 491 | total_timesteps 5121.
Path 492 | total_timesteps 5133.
Path 493 | total_timesteps 5144.
Path 494 | total_timesteps 5154.
Path 495 | total_timesteps 5164.
Path 496 | total_timesteps 5179.
Path 497 | total_timesteps 5200.
Path 498 | total_timesteps 5212.
Path 499 | total_timesteps 5224.
Path 500 | total_timesteps 5232.
Path 501 | total_timesteps 5245.
Path 502 | total_timesteps 5258.
Path 503 | total_timesteps 5267.
Path 504 | total_timesteps 5277.
Path 505 | total_timesteps 5292.
Path 506 | total_timesteps 5301.
Path 507 | total_timesteps 5315.
Path 508 | total_timesteps 5324.
Path 509 | total_timesteps 5334.
Path 510 | total_timesteps 5348.
Path 511 | total_timesteps 5356.
Path 512 | total_timesteps 5368.
Path 513 | total_timesteps 5377.
Path 514 | total_timesteps 5384.
Path 515 | total_timesteps 5398.
Path 516 | total_timesteps 5406.
Path 517 | total_timesteps 5413.
Path 518 | total_timesteps 5434.
Path 519 | total_timesteps 5444.
Path 520 | total_timesteps 5454.
Path 521 | total_timesteps 5465.
Path 522 | total_timesteps 5474.
Path 523 | total_timesteps 5480.
Path 524 | total_timesteps 5492.
Path 525 | total_timesteps 5500.
Path 526 | total_timesteps 5510.
Path 527 | total_timesteps 5518.
Path 528 | total_timesteps 5525.
Path 529 | total_timesteps 5533.
Path 530 | total_timesteps 5542.
Path 531 | total_timesteps 5551.
Path 532 | total_timesteps 5561.
Path 533 | total_timesteps 5571.
Path 534 | total_timesteps 5579.
Path 535 | total_timesteps 5585.
Path 536 | total_timesteps 5594.
Path 537 | total_timesteps 5611.
Path 538 | total_timesteps 5627.
Path 539 | total_timesteps 5640.
Path 540 | total_timesteps 5653.
Path 541 | total_timesteps 5665.
Path 542 | total_timesteps 5674.
Path 543 | total_timesteps 5682.
Path 544 | total_timesteps 5694.
Path 545 | total_timesteps 5704.
Path 546 | total_timesteps 5713.
Path 547 | total_timesteps 5722.
Path 548 | total_timesteps 5731.
Path 549 | total_timesteps 5739.
Path 550 | total_timesteps 5747.
Path 551 | total_timesteps 5758.
Path 552 | total_timesteps 5765.
Path 553 | total_timesteps 5775.
Path 554 | total_timesteps 5784.
Path 555 | total_timesteps 5795.
Path 556 | total_timesteps 5808.
Path 557 | total_timesteps 5821.
Path 558 | total_timesteps 5833.
Path 559 | total_timesteps 5840.
Path 560 | total_timesteps 5848.
Path 561 | total_timesteps 5862.
Path 562 | total_timesteps 5881.
Path 563 | total_timesteps 5890.
Path 564 | total_timesteps 5899.
Path 565 | total_timesteps 5907.
Path 566 | total_timesteps 5920.
Path 567 | total_timesteps 5928.
Path 568 | total_timesteps 5936.
Path 569 | total_timesteps 5946.
Path 570 | total_timesteps 5957.
Path 571 | total_timesteps 5967.
Path 572 | total_timesteps 5985.
Path 573 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.72    |
| Iteration     | 19       |
| MaximumReturn | 3.01     |
| MinimumReturn | -19.2    |
| TotalSamples  | 84083    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.31612178683280945
Validation loss = 0.31820690631866455
Validation loss = 0.3235543966293335
Validation loss = 0.32287290692329407
Validation loss = 0.32434341311454773
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 99.
Path 11 | total_timesteps 110.
Path 12 | total_timesteps 121.
Path 13 | total_timesteps 135.
Path 14 | total_timesteps 153.
Path 15 | total_timesteps 166.
Path 16 | total_timesteps 176.
Path 17 | total_timesteps 189.
Path 18 | total_timesteps 197.
Path 19 | total_timesteps 207.
Path 20 | total_timesteps 216.
Path 21 | total_timesteps 227.
Path 22 | total_timesteps 236.
Path 23 | total_timesteps 246.
Path 24 | total_timesteps 255.
Path 25 | total_timesteps 263.
Path 26 | total_timesteps 269.
Path 27 | total_timesteps 280.
Path 28 | total_timesteps 291.
Path 29 | total_timesteps 299.
Path 30 | total_timesteps 307.
Path 31 | total_timesteps 320.
Path 32 | total_timesteps 331.
Path 33 | total_timesteps 339.
Path 34 | total_timesteps 351.
Path 35 | total_timesteps 364.
Path 36 | total_timesteps 384.
Path 37 | total_timesteps 392.
Path 38 | total_timesteps 400.
Path 39 | total_timesteps 409.
Path 40 | total_timesteps 420.
Path 41 | total_timesteps 430.
Path 42 | total_timesteps 439.
Path 43 | total_timesteps 447.
Path 44 | total_timesteps 456.
Path 45 | total_timesteps 470.
Path 46 | total_timesteps 480.
Path 47 | total_timesteps 490.
Path 48 | total_timesteps 500.
Path 49 | total_timesteps 510.
Path 50 | total_timesteps 519.
Path 51 | total_timesteps 527.
Path 52 | total_timesteps 539.
Path 53 | total_timesteps 548.
Path 54 | total_timesteps 557.
Path 55 | total_timesteps 570.
Path 56 | total_timesteps 581.
Path 57 | total_timesteps 594.
Path 58 | total_timesteps 602.
Path 59 | total_timesteps 618.
Path 60 | total_timesteps 628.
Path 61 | total_timesteps 636.
Path 62 | total_timesteps 643.
Path 63 | total_timesteps 653.
Path 64 | total_timesteps 661.
Path 65 | total_timesteps 671.
Path 66 | total_timesteps 680.
Path 67 | total_timesteps 687.
Path 68 | total_timesteps 695.
Path 69 | total_timesteps 708.
Path 70 | total_timesteps 716.
Path 71 | total_timesteps 725.
Path 72 | total_timesteps 738.
Path 73 | total_timesteps 747.
Path 74 | total_timesteps 755.
Path 75 | total_timesteps 764.
Path 76 | total_timesteps 774.
Path 77 | total_timesteps 791.
Path 78 | total_timesteps 816.
Path 79 | total_timesteps 827.
Path 80 | total_timesteps 834.
Path 81 | total_timesteps 845.
Path 82 | total_timesteps 853.
Path 83 | total_timesteps 864.
Path 84 | total_timesteps 876.
Path 85 | total_timesteps 888.
Path 86 | total_timesteps 895.
Path 87 | total_timesteps 906.
Path 88 | total_timesteps 920.
Path 89 | total_timesteps 930.
Path 90 | total_timesteps 937.
Path 91 | total_timesteps 948.
Path 92 | total_timesteps 957.
Path 93 | total_timesteps 968.
Path 94 | total_timesteps 984.
Path 95 | total_timesteps 1003.
Path 96 | total_timesteps 1011.
Path 97 | total_timesteps 1021.
Path 98 | total_timesteps 1031.
Path 99 | total_timesteps 1040.
Path 100 | total_timesteps 1047.
Path 101 | total_timesteps 1058.
Path 102 | total_timesteps 1066.
Path 103 | total_timesteps 1075.
Path 104 | total_timesteps 1090.
Path 105 | total_timesteps 1100.
Path 106 | total_timesteps 1111.
Path 107 | total_timesteps 1119.
Path 108 | total_timesteps 1129.
Path 109 | total_timesteps 1142.
Path 110 | total_timesteps 1165.
Path 111 | total_timesteps 1175.
Path 112 | total_timesteps 1182.
Path 113 | total_timesteps 1191.
Path 114 | total_timesteps 1203.
Path 115 | total_timesteps 1211.
Path 116 | total_timesteps 1219.
Path 117 | total_timesteps 1234.
Path 118 | total_timesteps 1244.
Path 119 | total_timesteps 1256.
Path 120 | total_timesteps 1265.
Path 121 | total_timesteps 1274.
Path 122 | total_timesteps 1286.
Path 123 | total_timesteps 1296.
Path 124 | total_timesteps 1305.
Path 125 | total_timesteps 1315.
Path 126 | total_timesteps 1324.
Path 127 | total_timesteps 1335.
Path 128 | total_timesteps 1342.
Path 129 | total_timesteps 1357.
Path 130 | total_timesteps 1368.
Path 131 | total_timesteps 1380.
Path 132 | total_timesteps 1387.
Path 133 | total_timesteps 1396.
Path 134 | total_timesteps 1405.
Path 135 | total_timesteps 1419.
Path 136 | total_timesteps 1430.
Path 137 | total_timesteps 1438.
Path 138 | total_timesteps 1447.
Path 139 | total_timesteps 1456.
Path 140 | total_timesteps 1464.
Path 141 | total_timesteps 1478.
Path 142 | total_timesteps 1486.
Path 143 | total_timesteps 1499.
Path 144 | total_timesteps 1507.
Path 145 | total_timesteps 1519.
Path 146 | total_timesteps 1527.
Path 147 | total_timesteps 1535.
Path 148 | total_timesteps 1544.
Path 149 | total_timesteps 1554.
Path 150 | total_timesteps 1561.
Path 151 | total_timesteps 1569.
Path 152 | total_timesteps 1578.
Path 153 | total_timesteps 1584.
Path 154 | total_timesteps 1596.
Path 155 | total_timesteps 1605.
Path 156 | total_timesteps 1621.
Path 157 | total_timesteps 1629.
Path 158 | total_timesteps 1639.
Path 159 | total_timesteps 1650.
Path 160 | total_timesteps 1659.
Path 161 | total_timesteps 1670.
Path 162 | total_timesteps 1678.
Path 163 | total_timesteps 1687.
Path 164 | total_timesteps 1700.
Path 165 | total_timesteps 1708.
Path 166 | total_timesteps 1721.
Path 167 | total_timesteps 1739.
Path 168 | total_timesteps 1747.
Path 169 | total_timesteps 1754.
Path 170 | total_timesteps 1766.
Path 171 | total_timesteps 1776.
Path 172 | total_timesteps 1784.
Path 173 | total_timesteps 1796.
Path 174 | total_timesteps 1804.
Path 175 | total_timesteps 1818.
Path 176 | total_timesteps 1825.
Path 177 | total_timesteps 1835.
Path 178 | total_timesteps 1858.
Path 179 | total_timesteps 1866.
Path 180 | total_timesteps 1875.
Path 181 | total_timesteps 1886.
Path 182 | total_timesteps 1898.
Path 183 | total_timesteps 1908.
Path 184 | total_timesteps 1918.
Path 185 | total_timesteps 1930.
Path 186 | total_timesteps 1938.
Path 187 | total_timesteps 1949.
Path 188 | total_timesteps 1960.
Path 189 | total_timesteps 1972.
Path 190 | total_timesteps 1978.
Path 191 | total_timesteps 1986.
Path 192 | total_timesteps 1993.
Path 193 | total_timesteps 2005.
Path 194 | total_timesteps 2021.
Path 195 | total_timesteps 2030.
Path 196 | total_timesteps 2040.
Path 197 | total_timesteps 2051.
Path 198 | total_timesteps 2062.
Path 199 | total_timesteps 2071.
Path 200 | total_timesteps 2079.
Path 201 | total_timesteps 2090.
Path 202 | total_timesteps 2097.
Path 203 | total_timesteps 2105.
Path 204 | total_timesteps 2113.
Path 205 | total_timesteps 2122.
Path 206 | total_timesteps 2134.
Path 207 | total_timesteps 2143.
Path 208 | total_timesteps 2150.
Path 209 | total_timesteps 2160.
Path 210 | total_timesteps 2167.
Path 211 | total_timesteps 2178.
Path 212 | total_timesteps 2187.
Path 213 | total_timesteps 2199.
Path 214 | total_timesteps 2209.
Path 215 | total_timesteps 2219.
Path 216 | total_timesteps 2232.
Path 217 | total_timesteps 2243.
Path 218 | total_timesteps 2255.
Path 219 | total_timesteps 2266.
Path 220 | total_timesteps 2275.
Path 221 | total_timesteps 2286.
Path 222 | total_timesteps 2297.
Path 223 | total_timesteps 2308.
Path 224 | total_timesteps 2320.
Path 225 | total_timesteps 2330.
Path 226 | total_timesteps 2337.
Path 227 | total_timesteps 2349.
Path 228 | total_timesteps 2358.
Path 229 | total_timesteps 2370.
Path 230 | total_timesteps 2382.
Path 231 | total_timesteps 2391.
Path 232 | total_timesteps 2401.
Path 233 | total_timesteps 2411.
Path 234 | total_timesteps 2417.
Path 235 | total_timesteps 2425.
Path 236 | total_timesteps 2435.
Path 237 | total_timesteps 2447.
Path 238 | total_timesteps 2455.
Path 239 | total_timesteps 2465.
Path 240 | total_timesteps 2476.
Path 241 | total_timesteps 2484.
Path 242 | total_timesteps 2491.
Path 243 | total_timesteps 2501.
Path 244 | total_timesteps 2510.
Path 245 | total_timesteps 2526.
Path 246 | total_timesteps 2535.
Path 247 | total_timesteps 2544.
Path 248 | total_timesteps 2551.
Path 249 | total_timesteps 2563.
Path 250 | total_timesteps 2570.
Path 251 | total_timesteps 2578.
Path 252 | total_timesteps 2586.
Path 253 | total_timesteps 2597.
Path 254 | total_timesteps 2605.
Path 255 | total_timesteps 2618.
Path 256 | total_timesteps 2627.
Path 257 | total_timesteps 2636.
Path 258 | total_timesteps 2643.
Path 259 | total_timesteps 2651.
Path 260 | total_timesteps 2663.
Path 261 | total_timesteps 2673.
Path 262 | total_timesteps 2684.
Path 263 | total_timesteps 2690.
Path 264 | total_timesteps 2700.
Path 265 | total_timesteps 2706.
Path 266 | total_timesteps 2715.
Path 267 | total_timesteps 2726.
Path 268 | total_timesteps 2735.
Path 269 | total_timesteps 2744.
Path 270 | total_timesteps 2754.
Path 271 | total_timesteps 2761.
Path 272 | total_timesteps 2775.
Path 273 | total_timesteps 2785.
Path 274 | total_timesteps 2793.
Path 275 | total_timesteps 2803.
Path 276 | total_timesteps 2815.
Path 277 | total_timesteps 2825.
Path 278 | total_timesteps 2837.
Path 279 | total_timesteps 2846.
Path 280 | total_timesteps 2855.
Path 281 | total_timesteps 2864.
Path 282 | total_timesteps 2874.
Path 283 | total_timesteps 2882.
Path 284 | total_timesteps 2889.
Path 285 | total_timesteps 2898.
Path 286 | total_timesteps 2908.
Path 287 | total_timesteps 2917.
Path 288 | total_timesteps 2930.
Path 289 | total_timesteps 2943.
Path 290 | total_timesteps 2950.
Path 291 | total_timesteps 2959.
Path 292 | total_timesteps 2971.
Path 293 | total_timesteps 2984.
Path 294 | total_timesteps 2997.
Path 295 | total_timesteps 3009.
Path 296 | total_timesteps 3017.
Path 297 | total_timesteps 3032.
Path 298 | total_timesteps 3047.
Path 299 | total_timesteps 3056.
Path 300 | total_timesteps 3069.
Path 301 | total_timesteps 3076.
Path 302 | total_timesteps 3085.
Path 303 | total_timesteps 3095.
Path 304 | total_timesteps 3106.
Path 305 | total_timesteps 3116.
Path 306 | total_timesteps 3125.
Path 307 | total_timesteps 3133.
Path 308 | total_timesteps 3142.
Path 309 | total_timesteps 3152.
Path 310 | total_timesteps 3161.
Path 311 | total_timesteps 3172.
Path 312 | total_timesteps 3182.
Path 313 | total_timesteps 3190.
Path 314 | total_timesteps 3197.
Path 315 | total_timesteps 3205.
Path 316 | total_timesteps 3218.
Path 317 | total_timesteps 3228.
Path 318 | total_timesteps 3240.
Path 319 | total_timesteps 3250.
Path 320 | total_timesteps 3264.
Path 321 | total_timesteps 3274.
Path 322 | total_timesteps 3284.
Path 323 | total_timesteps 3293.
Path 324 | total_timesteps 3301.
Path 325 | total_timesteps 3311.
Path 326 | total_timesteps 3323.
Path 327 | total_timesteps 3334.
Path 328 | total_timesteps 3347.
Path 329 | total_timesteps 3359.
Path 330 | total_timesteps 3367.
Path 331 | total_timesteps 3374.
Path 332 | total_timesteps 3385.
Path 333 | total_timesteps 3393.
Path 334 | total_timesteps 3401.
Path 335 | total_timesteps 3415.
Path 336 | total_timesteps 3427.
Path 337 | total_timesteps 3434.
Path 338 | total_timesteps 3444.
Path 339 | total_timesteps 3454.
Path 340 | total_timesteps 3464.
Path 341 | total_timesteps 3471.
Path 342 | total_timesteps 3484.
Path 343 | total_timesteps 3495.
Path 344 | total_timesteps 3507.
Path 345 | total_timesteps 3515.
Path 346 | total_timesteps 3524.
Path 347 | total_timesteps 3536.
Path 348 | total_timesteps 3544.
Path 349 | total_timesteps 3551.
Path 350 | total_timesteps 3564.
Path 351 | total_timesteps 3573.
Path 352 | total_timesteps 3584.
Path 353 | total_timesteps 3595.
Path 354 | total_timesteps 3605.
Path 355 | total_timesteps 3614.
Path 356 | total_timesteps 3624.
Path 357 | total_timesteps 3632.
Path 358 | total_timesteps 3639.
Path 359 | total_timesteps 3647.
Path 360 | total_timesteps 3659.
Path 361 | total_timesteps 3667.
Path 362 | total_timesteps 3677.
Path 363 | total_timesteps 3685.
Path 364 | total_timesteps 3693.
Path 365 | total_timesteps 3702.
Path 366 | total_timesteps 3713.
Path 367 | total_timesteps 3722.
Path 368 | total_timesteps 3734.
Path 369 | total_timesteps 3741.
Path 370 | total_timesteps 3748.
Path 371 | total_timesteps 3758.
Path 372 | total_timesteps 3767.
Path 373 | total_timesteps 3779.
Path 374 | total_timesteps 3793.
Path 375 | total_timesteps 3816.
Path 376 | total_timesteps 3829.
Path 377 | total_timesteps 3842.
Path 378 | total_timesteps 3856.
Path 379 | total_timesteps 3866.
Path 380 | total_timesteps 3874.
Path 381 | total_timesteps 3881.
Path 382 | total_timesteps 3896.
Path 383 | total_timesteps 3903.
Path 384 | total_timesteps 3913.
Path 385 | total_timesteps 3922.
Path 386 | total_timesteps 3934.
Path 387 | total_timesteps 3942.
Path 388 | total_timesteps 3953.
Path 389 | total_timesteps 3963.
Path 390 | total_timesteps 3974.
Path 391 | total_timesteps 3983.
Path 392 | total_timesteps 3997.
Path 393 | total_timesteps 4004.
Path 394 | total_timesteps 4013.
Path 395 | total_timesteps 4024.
Path 396 | total_timesteps 4038.
Path 397 | total_timesteps 4053.
Path 398 | total_timesteps 4069.
Path 399 | total_timesteps 4080.
Path 400 | total_timesteps 4087.
Path 401 | total_timesteps 4098.
Path 402 | total_timesteps 4111.
Path 403 | total_timesteps 4117.
Path 404 | total_timesteps 4127.
Path 405 | total_timesteps 4136.
Path 406 | total_timesteps 4145.
Path 407 | total_timesteps 4158.
Path 408 | total_timesteps 4168.
Path 409 | total_timesteps 4179.
Path 410 | total_timesteps 4191.
Path 411 | total_timesteps 4203.
Path 412 | total_timesteps 4212.
Path 413 | total_timesteps 4223.
Path 414 | total_timesteps 4233.
Path 415 | total_timesteps 4243.
Path 416 | total_timesteps 4254.
Path 417 | total_timesteps 4260.
Path 418 | total_timesteps 4267.
Path 419 | total_timesteps 4278.
Path 420 | total_timesteps 4285.
Path 421 | total_timesteps 4298.
Path 422 | total_timesteps 4310.
Path 423 | total_timesteps 4324.
Path 424 | total_timesteps 4340.
Path 425 | total_timesteps 4347.
Path 426 | total_timesteps 4360.
Path 427 | total_timesteps 4372.
Path 428 | total_timesteps 4381.
Path 429 | total_timesteps 4400.
Path 430 | total_timesteps 4408.
Path 431 | total_timesteps 4415.
Path 432 | total_timesteps 4433.
Path 433 | total_timesteps 4440.
Path 434 | total_timesteps 4448.
Path 435 | total_timesteps 4461.
Path 436 | total_timesteps 4468.
Path 437 | total_timesteps 4485.
Path 438 | total_timesteps 4495.
Path 439 | total_timesteps 4502.
Path 440 | total_timesteps 4511.
Path 441 | total_timesteps 4522.
Path 442 | total_timesteps 4532.
Path 443 | total_timesteps 4544.
Path 444 | total_timesteps 4556.
Path 445 | total_timesteps 4565.
Path 446 | total_timesteps 4582.
Path 447 | total_timesteps 4601.
Path 448 | total_timesteps 4609.
Path 449 | total_timesteps 4617.
Path 450 | total_timesteps 4626.
Path 451 | total_timesteps 4634.
Path 452 | total_timesteps 4644.
Path 453 | total_timesteps 4654.
Path 454 | total_timesteps 4667.
Path 455 | total_timesteps 4679.
Path 456 | total_timesteps 4691.
Path 457 | total_timesteps 4707.
Path 458 | total_timesteps 4716.
Path 459 | total_timesteps 4726.
Path 460 | total_timesteps 4736.
Path 461 | total_timesteps 4744.
Path 462 | total_timesteps 4760.
Path 463 | total_timesteps 4775.
Path 464 | total_timesteps 4788.
Path 465 | total_timesteps 4797.
Path 466 | total_timesteps 4806.
Path 467 | total_timesteps 4813.
Path 468 | total_timesteps 4824.
Path 469 | total_timesteps 4836.
Path 470 | total_timesteps 4843.
Path 471 | total_timesteps 4854.
Path 472 | total_timesteps 4863.
Path 473 | total_timesteps 4871.
Path 474 | total_timesteps 4881.
Path 475 | total_timesteps 4890.
Path 476 | total_timesteps 4897.
Path 477 | total_timesteps 4909.
Path 478 | total_timesteps 4924.
Path 479 | total_timesteps 4935.
Path 480 | total_timesteps 4943.
Path 481 | total_timesteps 4951.
Path 482 | total_timesteps 4959.
Path 483 | total_timesteps 4971.
Path 484 | total_timesteps 4981.
Path 485 | total_timesteps 4993.
Path 486 | total_timesteps 5003.
Path 487 | total_timesteps 5020.
Path 488 | total_timesteps 5030.
Path 489 | total_timesteps 5038.
Path 490 | total_timesteps 5047.
Path 491 | total_timesteps 5060.
Path 492 | total_timesteps 5069.
Path 493 | total_timesteps 5082.
Path 494 | total_timesteps 5089.
Path 495 | total_timesteps 5099.
Path 496 | total_timesteps 5113.
Path 497 | total_timesteps 5121.
Path 498 | total_timesteps 5133.
Path 499 | total_timesteps 5140.
Path 500 | total_timesteps 5149.
Path 501 | total_timesteps 5156.
Path 502 | total_timesteps 5164.
Path 503 | total_timesteps 5175.
Path 504 | total_timesteps 5187.
Path 505 | total_timesteps 5200.
Path 506 | total_timesteps 5211.
Path 507 | total_timesteps 5225.
Path 508 | total_timesteps 5234.
Path 509 | total_timesteps 5252.
Path 510 | total_timesteps 5259.
Path 511 | total_timesteps 5273.
Path 512 | total_timesteps 5286.
Path 513 | total_timesteps 5297.
Path 514 | total_timesteps 5305.
Path 515 | total_timesteps 5316.
Path 516 | total_timesteps 5326.
Path 517 | total_timesteps 5333.
Path 518 | total_timesteps 5342.
Path 519 | total_timesteps 5354.
Path 520 | total_timesteps 5370.
Path 521 | total_timesteps 5377.
Path 522 | total_timesteps 5386.
Path 523 | total_timesteps 5397.
Path 524 | total_timesteps 5410.
Path 525 | total_timesteps 5419.
Path 526 | total_timesteps 5428.
Path 527 | total_timesteps 5437.
Path 528 | total_timesteps 5444.
Path 529 | total_timesteps 5456.
Path 530 | total_timesteps 5464.
Path 531 | total_timesteps 5474.
Path 532 | total_timesteps 5488.
Path 533 | total_timesteps 5496.
Path 534 | total_timesteps 5505.
Path 535 | total_timesteps 5516.
Path 536 | total_timesteps 5534.
Path 537 | total_timesteps 5544.
Path 538 | total_timesteps 5554.
Path 539 | total_timesteps 5566.
Path 540 | total_timesteps 5578.
Path 541 | total_timesteps 5588.
Path 542 | total_timesteps 5599.
Path 543 | total_timesteps 5610.
Path 544 | total_timesteps 5617.
Path 545 | total_timesteps 5626.
Path 546 | total_timesteps 5637.
Path 547 | total_timesteps 5644.
Path 548 | total_timesteps 5663.
Path 549 | total_timesteps 5672.
Path 550 | total_timesteps 5682.
Path 551 | total_timesteps 5695.
Path 552 | total_timesteps 5704.
Path 553 | total_timesteps 5712.
Path 554 | total_timesteps 5720.
Path 555 | total_timesteps 5730.
Path 556 | total_timesteps 5738.
Path 557 | total_timesteps 5747.
Path 558 | total_timesteps 5755.
Path 559 | total_timesteps 5764.
Path 560 | total_timesteps 5775.
Path 561 | total_timesteps 5786.
Path 562 | total_timesteps 5797.
Path 563 | total_timesteps 5806.
Path 564 | total_timesteps 5821.
Path 565 | total_timesteps 5830.
Path 566 | total_timesteps 5839.
Path 567 | total_timesteps 5847.
Path 568 | total_timesteps 5855.
Path 569 | total_timesteps 5866.
Path 570 | total_timesteps 5876.
Path 571 | total_timesteps 5887.
Path 572 | total_timesteps 5899.
Path 573 | total_timesteps 5910.
Path 574 | total_timesteps 5920.
Path 575 | total_timesteps 5928.
Path 576 | total_timesteps 5936.
Path 577 | total_timesteps 5945.
Path 578 | total_timesteps 5954.
Path 579 | total_timesteps 5965.
Path 580 | total_timesteps 5975.
Path 581 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.84    |
| Iteration     | 20       |
| MaximumReturn | 0.506    |
| MinimumReturn | -18.8    |
| TotalSamples  | 88085    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.3137262165546417
Validation loss = 0.3190325200557709
Validation loss = 0.3206014931201935
Validation loss = 0.3240021765232086
Validation loss = 0.3234398066997528
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 78.
Path 9 | total_timesteps 94.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 114.
Path 12 | total_timesteps 123.
Path 13 | total_timesteps 130.
Path 14 | total_timesteps 142.
Path 15 | total_timesteps 151.
Path 16 | total_timesteps 162.
Path 17 | total_timesteps 175.
Path 18 | total_timesteps 187.
Path 19 | total_timesteps 204.
Path 20 | total_timesteps 216.
Path 21 | total_timesteps 224.
Path 22 | total_timesteps 238.
Path 23 | total_timesteps 244.
Path 24 | total_timesteps 256.
Path 25 | total_timesteps 267.
Path 26 | total_timesteps 273.
Path 27 | total_timesteps 281.
Path 28 | total_timesteps 292.
Path 29 | total_timesteps 303.
Path 30 | total_timesteps 314.
Path 31 | total_timesteps 325.
Path 32 | total_timesteps 338.
Path 33 | total_timesteps 350.
Path 34 | total_timesteps 358.
Path 35 | total_timesteps 366.
Path 36 | total_timesteps 376.
Path 37 | total_timesteps 383.
Path 38 | total_timesteps 404.
Path 39 | total_timesteps 415.
Path 40 | total_timesteps 422.
Path 41 | total_timesteps 434.
Path 42 | total_timesteps 444.
Path 43 | total_timesteps 454.
Path 44 | total_timesteps 468.
Path 45 | total_timesteps 476.
Path 46 | total_timesteps 486.
Path 47 | total_timesteps 496.
Path 48 | total_timesteps 507.
Path 49 | total_timesteps 516.
Path 50 | total_timesteps 525.
Path 51 | total_timesteps 534.
Path 52 | total_timesteps 543.
Path 53 | total_timesteps 554.
Path 54 | total_timesteps 564.
Path 55 | total_timesteps 576.
Path 56 | total_timesteps 590.
Path 57 | total_timesteps 600.
Path 58 | total_timesteps 616.
Path 59 | total_timesteps 625.
Path 60 | total_timesteps 634.
Path 61 | total_timesteps 643.
Path 62 | total_timesteps 652.
Path 63 | total_timesteps 660.
Path 64 | total_timesteps 672.
Path 65 | total_timesteps 682.
Path 66 | total_timesteps 693.
Path 67 | total_timesteps 703.
Path 68 | total_timesteps 712.
Path 69 | total_timesteps 724.
Path 70 | total_timesteps 734.
Path 71 | total_timesteps 742.
Path 72 | total_timesteps 752.
Path 73 | total_timesteps 760.
Path 74 | total_timesteps 771.
Path 75 | total_timesteps 779.
Path 76 | total_timesteps 790.
Path 77 | total_timesteps 802.
Path 78 | total_timesteps 811.
Path 79 | total_timesteps 822.
Path 80 | total_timesteps 833.
Path 81 | total_timesteps 846.
Path 82 | total_timesteps 857.
Path 83 | total_timesteps 864.
Path 84 | total_timesteps 874.
Path 85 | total_timesteps 886.
Path 86 | total_timesteps 897.
Path 87 | total_timesteps 907.
Path 88 | total_timesteps 918.
Path 89 | total_timesteps 927.
Path 90 | total_timesteps 937.
Path 91 | total_timesteps 944.
Path 92 | total_timesteps 950.
Path 93 | total_timesteps 962.
Path 94 | total_timesteps 970.
Path 95 | total_timesteps 981.
Path 96 | total_timesteps 992.
Path 97 | total_timesteps 1001.
Path 98 | total_timesteps 1008.
Path 99 | total_timesteps 1021.
Path 100 | total_timesteps 1028.
Path 101 | total_timesteps 1035.
Path 102 | total_timesteps 1050.
Path 103 | total_timesteps 1060.
Path 104 | total_timesteps 1073.
Path 105 | total_timesteps 1090.
Path 106 | total_timesteps 1099.
Path 107 | total_timesteps 1107.
Path 108 | total_timesteps 1116.
Path 109 | total_timesteps 1125.
Path 110 | total_timesteps 1136.
Path 111 | total_timesteps 1150.
Path 112 | total_timesteps 1161.
Path 113 | total_timesteps 1171.
Path 114 | total_timesteps 1179.
Path 115 | total_timesteps 1190.
Path 116 | total_timesteps 1201.
Path 117 | total_timesteps 1210.
Path 118 | total_timesteps 1218.
Path 119 | total_timesteps 1226.
Path 120 | total_timesteps 1237.
Path 121 | total_timesteps 1249.
Path 122 | total_timesteps 1259.
Path 123 | total_timesteps 1276.
Path 124 | total_timesteps 1285.
Path 125 | total_timesteps 1296.
Path 126 | total_timesteps 1306.
Path 127 | total_timesteps 1318.
Path 128 | total_timesteps 1327.
Path 129 | total_timesteps 1337.
Path 130 | total_timesteps 1350.
Path 131 | total_timesteps 1360.
Path 132 | total_timesteps 1368.
Path 133 | total_timesteps 1378.
Path 134 | total_timesteps 1395.
Path 135 | total_timesteps 1403.
Path 136 | total_timesteps 1413.
Path 137 | total_timesteps 1419.
Path 138 | total_timesteps 1435.
Path 139 | total_timesteps 1449.
Path 140 | total_timesteps 1458.
Path 141 | total_timesteps 1469.
Path 142 | total_timesteps 1491.
Path 143 | total_timesteps 1501.
Path 144 | total_timesteps 1512.
Path 145 | total_timesteps 1521.
Path 146 | total_timesteps 1538.
Path 147 | total_timesteps 1552.
Path 148 | total_timesteps 1562.
Path 149 | total_timesteps 1572.
Path 150 | total_timesteps 1585.
Path 151 | total_timesteps 1597.
Path 152 | total_timesteps 1606.
Path 153 | total_timesteps 1616.
Path 154 | total_timesteps 1627.
Path 155 | total_timesteps 1636.
Path 156 | total_timesteps 1642.
Path 157 | total_timesteps 1651.
Path 158 | total_timesteps 1658.
Path 159 | total_timesteps 1671.
Path 160 | total_timesteps 1681.
Path 161 | total_timesteps 1692.
Path 162 | total_timesteps 1703.
Path 163 | total_timesteps 1713.
Path 164 | total_timesteps 1724.
Path 165 | total_timesteps 1746.
Path 166 | total_timesteps 1755.
Path 167 | total_timesteps 1768.
Path 168 | total_timesteps 1779.
Path 169 | total_timesteps 1793.
Path 170 | total_timesteps 1802.
Path 171 | total_timesteps 1808.
Path 172 | total_timesteps 1815.
Path 173 | total_timesteps 1822.
Path 174 | total_timesteps 1828.
Path 175 | total_timesteps 1837.
Path 176 | total_timesteps 1848.
Path 177 | total_timesteps 1861.
Path 178 | total_timesteps 1874.
Path 179 | total_timesteps 1883.
Path 180 | total_timesteps 1896.
Path 181 | total_timesteps 1910.
Path 182 | total_timesteps 1927.
Path 183 | total_timesteps 1935.
Path 184 | total_timesteps 1945.
Path 185 | total_timesteps 1962.
Path 186 | total_timesteps 1976.
Path 187 | total_timesteps 1985.
Path 188 | total_timesteps 1998.
Path 189 | total_timesteps 2013.
Path 190 | total_timesteps 2023.
Path 191 | total_timesteps 2030.
Path 192 | total_timesteps 2041.
Path 193 | total_timesteps 2049.
Path 194 | total_timesteps 2059.
Path 195 | total_timesteps 2067.
Path 196 | total_timesteps 2079.
Path 197 | total_timesteps 2087.
Path 198 | total_timesteps 2102.
Path 199 | total_timesteps 2110.
Path 200 | total_timesteps 2119.
Path 201 | total_timesteps 2131.
Path 202 | total_timesteps 2138.
Path 203 | total_timesteps 2148.
Path 204 | total_timesteps 2159.
Path 205 | total_timesteps 2168.
Path 206 | total_timesteps 2175.
Path 207 | total_timesteps 2196.
Path 208 | total_timesteps 2207.
Path 209 | total_timesteps 2224.
Path 210 | total_timesteps 2232.
Path 211 | total_timesteps 2241.
Path 212 | total_timesteps 2251.
Path 213 | total_timesteps 2264.
Path 214 | total_timesteps 2273.
Path 215 | total_timesteps 2282.
Path 216 | total_timesteps 2292.
Path 217 | total_timesteps 2301.
Path 218 | total_timesteps 2310.
Path 219 | total_timesteps 2321.
Path 220 | total_timesteps 2333.
Path 221 | total_timesteps 2342.
Path 222 | total_timesteps 2350.
Path 223 | total_timesteps 2365.
Path 224 | total_timesteps 2376.
Path 225 | total_timesteps 2383.
Path 226 | total_timesteps 2392.
Path 227 | total_timesteps 2404.
Path 228 | total_timesteps 2413.
Path 229 | total_timesteps 2436.
Path 230 | total_timesteps 2447.
Path 231 | total_timesteps 2458.
Path 232 | total_timesteps 2466.
Path 233 | total_timesteps 2475.
Path 234 | total_timesteps 2484.
Path 235 | total_timesteps 2497.
Path 236 | total_timesteps 2508.
Path 237 | total_timesteps 2515.
Path 238 | total_timesteps 2524.
Path 239 | total_timesteps 2539.
Path 240 | total_timesteps 2553.
Path 241 | total_timesteps 2572.
Path 242 | total_timesteps 2592.
Path 243 | total_timesteps 2600.
Path 244 | total_timesteps 2620.
Path 245 | total_timesteps 2634.
Path 246 | total_timesteps 2642.
Path 247 | total_timesteps 2653.
Path 248 | total_timesteps 2661.
Path 249 | total_timesteps 2670.
Path 250 | total_timesteps 2679.
Path 251 | total_timesteps 2687.
Path 252 | total_timesteps 2702.
Path 253 | total_timesteps 2716.
Path 254 | total_timesteps 2725.
Path 255 | total_timesteps 2735.
Path 256 | total_timesteps 2743.
Path 257 | total_timesteps 2756.
Path 258 | total_timesteps 2765.
Path 259 | total_timesteps 2776.
Path 260 | total_timesteps 2782.
Path 261 | total_timesteps 2789.
Path 262 | total_timesteps 2806.
Path 263 | total_timesteps 2817.
Path 264 | total_timesteps 2831.
Path 265 | total_timesteps 2842.
Path 266 | total_timesteps 2850.
Path 267 | total_timesteps 2860.
Path 268 | total_timesteps 2872.
Path 269 | total_timesteps 2883.
Path 270 | total_timesteps 2893.
Path 271 | total_timesteps 2902.
Path 272 | total_timesteps 2910.
Path 273 | total_timesteps 2917.
Path 274 | total_timesteps 2928.
Path 275 | total_timesteps 2939.
Path 276 | total_timesteps 2949.
Path 277 | total_timesteps 2957.
Path 278 | total_timesteps 2971.
Path 279 | total_timesteps 2988.
Path 280 | total_timesteps 2997.
Path 281 | total_timesteps 3006.
Path 282 | total_timesteps 3015.
Path 283 | total_timesteps 3025.
Path 284 | total_timesteps 3032.
Path 285 | total_timesteps 3044.
Path 286 | total_timesteps 3055.
Path 287 | total_timesteps 3067.
Path 288 | total_timesteps 3076.
Path 289 | total_timesteps 3088.
Path 290 | total_timesteps 3096.
Path 291 | total_timesteps 3111.
Path 292 | total_timesteps 3118.
Path 293 | total_timesteps 3128.
Path 294 | total_timesteps 3137.
Path 295 | total_timesteps 3149.
Path 296 | total_timesteps 3157.
Path 297 | total_timesteps 3174.
Path 298 | total_timesteps 3184.
Path 299 | total_timesteps 3193.
Path 300 | total_timesteps 3203.
Path 301 | total_timesteps 3214.
Path 302 | total_timesteps 3227.
Path 303 | total_timesteps 3236.
Path 304 | total_timesteps 3246.
Path 305 | total_timesteps 3255.
Path 306 | total_timesteps 3268.
Path 307 | total_timesteps 3281.
Path 308 | total_timesteps 3298.
Path 309 | total_timesteps 3309.
Path 310 | total_timesteps 3323.
Path 311 | total_timesteps 3332.
Path 312 | total_timesteps 3341.
Path 313 | total_timesteps 3355.
Path 314 | total_timesteps 3369.
Path 315 | total_timesteps 3378.
Path 316 | total_timesteps 3389.
Path 317 | total_timesteps 3397.
Path 318 | total_timesteps 3408.
Path 319 | total_timesteps 3417.
Path 320 | total_timesteps 3427.
Path 321 | total_timesteps 3435.
Path 322 | total_timesteps 3443.
Path 323 | total_timesteps 3451.
Path 324 | total_timesteps 3459.
Path 325 | total_timesteps 3470.
Path 326 | total_timesteps 3479.
Path 327 | total_timesteps 3487.
Path 328 | total_timesteps 3496.
Path 329 | total_timesteps 3504.
Path 330 | total_timesteps 3511.
Path 331 | total_timesteps 3519.
Path 332 | total_timesteps 3526.
Path 333 | total_timesteps 3535.
Path 334 | total_timesteps 3542.
Path 335 | total_timesteps 3552.
Path 336 | total_timesteps 3561.
Path 337 | total_timesteps 3570.
Path 338 | total_timesteps 3583.
Path 339 | total_timesteps 3595.
Path 340 | total_timesteps 3605.
Path 341 | total_timesteps 3615.
Path 342 | total_timesteps 3628.
Path 343 | total_timesteps 3640.
Path 344 | total_timesteps 3652.
Path 345 | total_timesteps 3663.
Path 346 | total_timesteps 3677.
Path 347 | total_timesteps 3684.
Path 348 | total_timesteps 3691.
Path 349 | total_timesteps 3701.
Path 350 | total_timesteps 3711.
Path 351 | total_timesteps 3722.
Path 352 | total_timesteps 3733.
Path 353 | total_timesteps 3744.
Path 354 | total_timesteps 3756.
Path 355 | total_timesteps 3763.
Path 356 | total_timesteps 3779.
Path 357 | total_timesteps 3788.
Path 358 | total_timesteps 3799.
Path 359 | total_timesteps 3807.
Path 360 | total_timesteps 3815.
Path 361 | total_timesteps 3823.
Path 362 | total_timesteps 3837.
Path 363 | total_timesteps 3845.
Path 364 | total_timesteps 3855.
Path 365 | total_timesteps 3867.
Path 366 | total_timesteps 3874.
Path 367 | total_timesteps 3891.
Path 368 | total_timesteps 3902.
Path 369 | total_timesteps 3911.
Path 370 | total_timesteps 3922.
Path 371 | total_timesteps 3932.
Path 372 | total_timesteps 3945.
Path 373 | total_timesteps 3955.
Path 374 | total_timesteps 3967.
Path 375 | total_timesteps 3980.
Path 376 | total_timesteps 3991.
Path 377 | total_timesteps 3998.
Path 378 | total_timesteps 4011.
Path 379 | total_timesteps 4020.
Path 380 | total_timesteps 4029.
Path 381 | total_timesteps 4038.
Path 382 | total_timesteps 4053.
Path 383 | total_timesteps 4061.
Path 384 | total_timesteps 4071.
Path 385 | total_timesteps 4086.
Path 386 | total_timesteps 4092.
Path 387 | total_timesteps 4112.
Path 388 | total_timesteps 4120.
Path 389 | total_timesteps 4129.
Path 390 | total_timesteps 4140.
Path 391 | total_timesteps 4151.
Path 392 | total_timesteps 4159.
Path 393 | total_timesteps 4174.
Path 394 | total_timesteps 4185.
Path 395 | total_timesteps 4193.
Path 396 | total_timesteps 4202.
Path 397 | total_timesteps 4216.
Path 398 | total_timesteps 4224.
Path 399 | total_timesteps 4234.
Path 400 | total_timesteps 4243.
Path 401 | total_timesteps 4251.
Path 402 | total_timesteps 4263.
Path 403 | total_timesteps 4274.
Path 404 | total_timesteps 4285.
Path 405 | total_timesteps 4300.
Path 406 | total_timesteps 4310.
Path 407 | total_timesteps 4317.
Path 408 | total_timesteps 4330.
Path 409 | total_timesteps 4339.
Path 410 | total_timesteps 4349.
Path 411 | total_timesteps 4357.
Path 412 | total_timesteps 4371.
Path 413 | total_timesteps 4381.
Path 414 | total_timesteps 4394.
Path 415 | total_timesteps 4404.
Path 416 | total_timesteps 4416.
Path 417 | total_timesteps 4428.
Path 418 | total_timesteps 4439.
Path 419 | total_timesteps 4455.
Path 420 | total_timesteps 4466.
Path 421 | total_timesteps 4475.
Path 422 | total_timesteps 4485.
Path 423 | total_timesteps 4493.
Path 424 | total_timesteps 4504.
Path 425 | total_timesteps 4514.
Path 426 | total_timesteps 4523.
Path 427 | total_timesteps 4529.
Path 428 | total_timesteps 4540.
Path 429 | total_timesteps 4548.
Path 430 | total_timesteps 4561.
Path 431 | total_timesteps 4573.
Path 432 | total_timesteps 4585.
Path 433 | total_timesteps 4599.
Path 434 | total_timesteps 4611.
Path 435 | total_timesteps 4618.
Path 436 | total_timesteps 4628.
Path 437 | total_timesteps 4635.
Path 438 | total_timesteps 4645.
Path 439 | total_timesteps 4661.
Path 440 | total_timesteps 4672.
Path 441 | total_timesteps 4678.
Path 442 | total_timesteps 4688.
Path 443 | total_timesteps 4697.
Path 444 | total_timesteps 4705.
Path 445 | total_timesteps 4716.
Path 446 | total_timesteps 4724.
Path 447 | total_timesteps 4735.
Path 448 | total_timesteps 4742.
Path 449 | total_timesteps 4754.
Path 450 | total_timesteps 4766.
Path 451 | total_timesteps 4773.
Path 452 | total_timesteps 4783.
Path 453 | total_timesteps 4794.
Path 454 | total_timesteps 4810.
Path 455 | total_timesteps 4821.
Path 456 | total_timesteps 4833.
Path 457 | total_timesteps 4844.
Path 458 | total_timesteps 4859.
Path 459 | total_timesteps 4870.
Path 460 | total_timesteps 4889.
Path 461 | total_timesteps 4896.
Path 462 | total_timesteps 4912.
Path 463 | total_timesteps 4925.
Path 464 | total_timesteps 4947.
Path 465 | total_timesteps 4956.
Path 466 | total_timesteps 4966.
Path 467 | total_timesteps 4981.
Path 468 | total_timesteps 4992.
Path 469 | total_timesteps 5000.
Path 470 | total_timesteps 5008.
Path 471 | total_timesteps 5017.
Path 472 | total_timesteps 5027.
Path 473 | total_timesteps 5035.
Path 474 | total_timesteps 5046.
Path 475 | total_timesteps 5054.
Path 476 | total_timesteps 5066.
Path 477 | total_timesteps 5073.
Path 478 | total_timesteps 5084.
Path 479 | total_timesteps 5095.
Path 480 | total_timesteps 5102.
Path 481 | total_timesteps 5111.
Path 482 | total_timesteps 5119.
Path 483 | total_timesteps 5132.
Path 484 | total_timesteps 5138.
Path 485 | total_timesteps 5147.
Path 486 | total_timesteps 5159.
Path 487 | total_timesteps 5168.
Path 488 | total_timesteps 5177.
Path 489 | total_timesteps 5192.
Path 490 | total_timesteps 5202.
Path 491 | total_timesteps 5215.
Path 492 | total_timesteps 5223.
Path 493 | total_timesteps 5230.
Path 494 | total_timesteps 5243.
Path 495 | total_timesteps 5251.
Path 496 | total_timesteps 5264.
Path 497 | total_timesteps 5275.
Path 498 | total_timesteps 5284.
Path 499 | total_timesteps 5293.
Path 500 | total_timesteps 5300.
Path 501 | total_timesteps 5309.
Path 502 | total_timesteps 5317.
Path 503 | total_timesteps 5328.
Path 504 | total_timesteps 5338.
Path 505 | total_timesteps 5350.
Path 506 | total_timesteps 5357.
Path 507 | total_timesteps 5367.
Path 508 | total_timesteps 5378.
Path 509 | total_timesteps 5387.
Path 510 | total_timesteps 5398.
Path 511 | total_timesteps 5411.
Path 512 | total_timesteps 5427.
Path 513 | total_timesteps 5436.
Path 514 | total_timesteps 5449.
Path 515 | total_timesteps 5460.
Path 516 | total_timesteps 5473.
Path 517 | total_timesteps 5483.
Path 518 | total_timesteps 5493.
Path 519 | total_timesteps 5499.
Path 520 | total_timesteps 5515.
Path 521 | total_timesteps 5523.
Path 522 | total_timesteps 5532.
Path 523 | total_timesteps 5540.
Path 524 | total_timesteps 5547.
Path 525 | total_timesteps 5556.
Path 526 | total_timesteps 5571.
Path 527 | total_timesteps 5579.
Path 528 | total_timesteps 5590.
Path 529 | total_timesteps 5603.
Path 530 | total_timesteps 5613.
Path 531 | total_timesteps 5621.
Path 532 | total_timesteps 5632.
Path 533 | total_timesteps 5644.
Path 534 | total_timesteps 5655.
Path 535 | total_timesteps 5665.
Path 536 | total_timesteps 5677.
Path 537 | total_timesteps 5689.
Path 538 | total_timesteps 5712.
Path 539 | total_timesteps 5732.
Path 540 | total_timesteps 5741.
Path 541 | total_timesteps 5749.
Path 542 | total_timesteps 5759.
Path 543 | total_timesteps 5770.
Path 544 | total_timesteps 5782.
Path 545 | total_timesteps 5793.
Path 546 | total_timesteps 5808.
Path 547 | total_timesteps 5818.
Path 548 | total_timesteps 5827.
Path 549 | total_timesteps 5841.
Path 550 | total_timesteps 5849.
Path 551 | total_timesteps 5860.
Path 552 | total_timesteps 5869.
Path 553 | total_timesteps 5879.
Path 554 | total_timesteps 5887.
Path 555 | total_timesteps 5898.
Path 556 | total_timesteps 5910.
Path 557 | total_timesteps 5919.
Path 558 | total_timesteps 5928.
Path 559 | total_timesteps 5941.
Path 560 | total_timesteps 5954.
Path 561 | total_timesteps 5961.
Path 562 | total_timesteps 5971.
Path 563 | total_timesteps 5979.
Path 564 | total_timesteps 5986.
Path 565 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.09    |
| Iteration     | 21       |
| MaximumReturn | 1.38     |
| MinimumReturn | -20.6    |
| TotalSamples  | 92089    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.3146231472492218
Validation loss = 0.32011881470680237
Validation loss = 0.32076364755630493
Validation loss = 0.3221116364002228
Validation loss = 0.3215543329715729
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 44.
Path 4 | total_timesteps 54.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 86.
Path 8 | total_timesteps 94.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 115.
Path 11 | total_timesteps 127.
Path 12 | total_timesteps 137.
Path 13 | total_timesteps 144.
Path 14 | total_timesteps 155.
Path 15 | total_timesteps 164.
Path 16 | total_timesteps 171.
Path 17 | total_timesteps 182.
Path 18 | total_timesteps 191.
Path 19 | total_timesteps 199.
Path 20 | total_timesteps 207.
Path 21 | total_timesteps 215.
Path 22 | total_timesteps 226.
Path 23 | total_timesteps 235.
Path 24 | total_timesteps 247.
Path 25 | total_timesteps 254.
Path 26 | total_timesteps 262.
Path 27 | total_timesteps 280.
Path 28 | total_timesteps 288.
Path 29 | total_timesteps 294.
Path 30 | total_timesteps 303.
Path 31 | total_timesteps 312.
Path 32 | total_timesteps 320.
Path 33 | total_timesteps 328.
Path 34 | total_timesteps 336.
Path 35 | total_timesteps 349.
Path 36 | total_timesteps 362.
Path 37 | total_timesteps 371.
Path 38 | total_timesteps 381.
Path 39 | total_timesteps 389.
Path 40 | total_timesteps 400.
Path 41 | total_timesteps 406.
Path 42 | total_timesteps 415.
Path 43 | total_timesteps 428.
Path 44 | total_timesteps 436.
Path 45 | total_timesteps 448.
Path 46 | total_timesteps 458.
Path 47 | total_timesteps 473.
Path 48 | total_timesteps 482.
Path 49 | total_timesteps 492.
Path 50 | total_timesteps 502.
Path 51 | total_timesteps 515.
Path 52 | total_timesteps 522.
Path 53 | total_timesteps 532.
Path 54 | total_timesteps 541.
Path 55 | total_timesteps 553.
Path 56 | total_timesteps 561.
Path 57 | total_timesteps 573.
Path 58 | total_timesteps 584.
Path 59 | total_timesteps 595.
Path 60 | total_timesteps 610.
Path 61 | total_timesteps 619.
Path 62 | total_timesteps 626.
Path 63 | total_timesteps 638.
Path 64 | total_timesteps 644.
Path 65 | total_timesteps 655.
Path 66 | total_timesteps 663.
Path 67 | total_timesteps 671.
Path 68 | total_timesteps 679.
Path 69 | total_timesteps 695.
Path 70 | total_timesteps 705.
Path 71 | total_timesteps 714.
Path 72 | total_timesteps 720.
Path 73 | total_timesteps 731.
Path 74 | total_timesteps 739.
Path 75 | total_timesteps 762.
Path 76 | total_timesteps 771.
Path 77 | total_timesteps 783.
Path 78 | total_timesteps 795.
Path 79 | total_timesteps 804.
Path 80 | total_timesteps 816.
Path 81 | total_timesteps 823.
Path 82 | total_timesteps 831.
Path 83 | total_timesteps 845.
Path 84 | total_timesteps 855.
Path 85 | total_timesteps 864.
Path 86 | total_timesteps 873.
Path 87 | total_timesteps 885.
Path 88 | total_timesteps 902.
Path 89 | total_timesteps 913.
Path 90 | total_timesteps 921.
Path 91 | total_timesteps 932.
Path 92 | total_timesteps 940.
Path 93 | total_timesteps 954.
Path 94 | total_timesteps 964.
Path 95 | total_timesteps 980.
Path 96 | total_timesteps 989.
Path 97 | total_timesteps 1000.
Path 98 | total_timesteps 1012.
Path 99 | total_timesteps 1026.
Path 100 | total_timesteps 1040.
Path 101 | total_timesteps 1051.
Path 102 | total_timesteps 1072.
Path 103 | total_timesteps 1086.
Path 104 | total_timesteps 1093.
Path 105 | total_timesteps 1104.
Path 106 | total_timesteps 1119.
Path 107 | total_timesteps 1128.
Path 108 | total_timesteps 1143.
Path 109 | total_timesteps 1150.
Path 110 | total_timesteps 1159.
Path 111 | total_timesteps 1168.
Path 112 | total_timesteps 1178.
Path 113 | total_timesteps 1186.
Path 114 | total_timesteps 1197.
Path 115 | total_timesteps 1211.
Path 116 | total_timesteps 1223.
Path 117 | total_timesteps 1233.
Path 118 | total_timesteps 1242.
Path 119 | total_timesteps 1249.
Path 120 | total_timesteps 1266.
Path 121 | total_timesteps 1279.
Path 122 | total_timesteps 1291.
Path 123 | total_timesteps 1302.
Path 124 | total_timesteps 1308.
Path 125 | total_timesteps 1315.
Path 126 | total_timesteps 1326.
Path 127 | total_timesteps 1337.
Path 128 | total_timesteps 1346.
Path 129 | total_timesteps 1353.
Path 130 | total_timesteps 1361.
Path 131 | total_timesteps 1371.
Path 132 | total_timesteps 1379.
Path 133 | total_timesteps 1389.
Path 134 | total_timesteps 1404.
Path 135 | total_timesteps 1416.
Path 136 | total_timesteps 1429.
Path 137 | total_timesteps 1439.
Path 138 | total_timesteps 1446.
Path 139 | total_timesteps 1459.
Path 140 | total_timesteps 1467.
Path 141 | total_timesteps 1478.
Path 142 | total_timesteps 1494.
Path 143 | total_timesteps 1501.
Path 144 | total_timesteps 1510.
Path 145 | total_timesteps 1520.
Path 146 | total_timesteps 1530.
Path 147 | total_timesteps 1540.
Path 148 | total_timesteps 1551.
Path 149 | total_timesteps 1564.
Path 150 | total_timesteps 1573.
Path 151 | total_timesteps 1582.
Path 152 | total_timesteps 1597.
Path 153 | total_timesteps 1605.
Path 154 | total_timesteps 1617.
Path 155 | total_timesteps 1629.
Path 156 | total_timesteps 1637.
Path 157 | total_timesteps 1646.
Path 158 | total_timesteps 1656.
Path 159 | total_timesteps 1664.
Path 160 | total_timesteps 1672.
Path 161 | total_timesteps 1680.
Path 162 | total_timesteps 1689.
Path 163 | total_timesteps 1699.
Path 164 | total_timesteps 1709.
Path 165 | total_timesteps 1720.
Path 166 | total_timesteps 1730.
Path 167 | total_timesteps 1745.
Path 168 | total_timesteps 1756.
Path 169 | total_timesteps 1768.
Path 170 | total_timesteps 1779.
Path 171 | total_timesteps 1796.
Path 172 | total_timesteps 1807.
Path 173 | total_timesteps 1818.
Path 174 | total_timesteps 1828.
Path 175 | total_timesteps 1835.
Path 176 | total_timesteps 1844.
Path 177 | total_timesteps 1854.
Path 178 | total_timesteps 1863.
Path 179 | total_timesteps 1873.
Path 180 | total_timesteps 1880.
Path 181 | total_timesteps 1892.
Path 182 | total_timesteps 1908.
Path 183 | total_timesteps 1918.
Path 184 | total_timesteps 1928.
Path 185 | total_timesteps 1947.
Path 186 | total_timesteps 1963.
Path 187 | total_timesteps 1976.
Path 188 | total_timesteps 1986.
Path 189 | total_timesteps 1996.
Path 190 | total_timesteps 2005.
Path 191 | total_timesteps 2015.
Path 192 | total_timesteps 2030.
Path 193 | total_timesteps 2037.
Path 194 | total_timesteps 2056.
Path 195 | total_timesteps 2063.
Path 196 | total_timesteps 2072.
Path 197 | total_timesteps 2080.
Path 198 | total_timesteps 2089.
Path 199 | total_timesteps 2099.
Path 200 | total_timesteps 2108.
Path 201 | total_timesteps 2117.
Path 202 | total_timesteps 2135.
Path 203 | total_timesteps 2142.
Path 204 | total_timesteps 2149.
Path 205 | total_timesteps 2159.
Path 206 | total_timesteps 2171.
Path 207 | total_timesteps 2181.
Path 208 | total_timesteps 2189.
Path 209 | total_timesteps 2199.
Path 210 | total_timesteps 2212.
Path 211 | total_timesteps 2222.
Path 212 | total_timesteps 2232.
Path 213 | total_timesteps 2241.
Path 214 | total_timesteps 2250.
Path 215 | total_timesteps 2261.
Path 216 | total_timesteps 2275.
Path 217 | total_timesteps 2284.
Path 218 | total_timesteps 2295.
Path 219 | total_timesteps 2304.
Path 220 | total_timesteps 2317.
Path 221 | total_timesteps 2329.
Path 222 | total_timesteps 2336.
Path 223 | total_timesteps 2347.
Path 224 | total_timesteps 2356.
Path 225 | total_timesteps 2365.
Path 226 | total_timesteps 2374.
Path 227 | total_timesteps 2385.
Path 228 | total_timesteps 2395.
Path 229 | total_timesteps 2408.
Path 230 | total_timesteps 2421.
Path 231 | total_timesteps 2430.
Path 232 | total_timesteps 2441.
Path 233 | total_timesteps 2454.
Path 234 | total_timesteps 2463.
Path 235 | total_timesteps 2471.
Path 236 | total_timesteps 2478.
Path 237 | total_timesteps 2488.
Path 238 | total_timesteps 2498.
Path 239 | total_timesteps 2506.
Path 240 | total_timesteps 2517.
Path 241 | total_timesteps 2525.
Path 242 | total_timesteps 2538.
Path 243 | total_timesteps 2551.
Path 244 | total_timesteps 2559.
Path 245 | total_timesteps 2571.
Path 246 | total_timesteps 2578.
Path 247 | total_timesteps 2590.
Path 248 | total_timesteps 2598.
Path 249 | total_timesteps 2611.
Path 250 | total_timesteps 2634.
Path 251 | total_timesteps 2643.
Path 252 | total_timesteps 2655.
Path 253 | total_timesteps 2665.
Path 254 | total_timesteps 2674.
Path 255 | total_timesteps 2682.
Path 256 | total_timesteps 2692.
Path 257 | total_timesteps 2712.
Path 258 | total_timesteps 2723.
Path 259 | total_timesteps 2734.
Path 260 | total_timesteps 2743.
Path 261 | total_timesteps 2752.
Path 262 | total_timesteps 2760.
Path 263 | total_timesteps 2773.
Path 264 | total_timesteps 2780.
Path 265 | total_timesteps 2788.
Path 266 | total_timesteps 2801.
Path 267 | total_timesteps 2814.
Path 268 | total_timesteps 2823.
Path 269 | total_timesteps 2831.
Path 270 | total_timesteps 2841.
Path 271 | total_timesteps 2849.
Path 272 | total_timesteps 2863.
Path 273 | total_timesteps 2871.
Path 274 | total_timesteps 2882.
Path 275 | total_timesteps 2890.
Path 276 | total_timesteps 2898.
Path 277 | total_timesteps 2905.
Path 278 | total_timesteps 2927.
Path 279 | total_timesteps 2942.
Path 280 | total_timesteps 2954.
Path 281 | total_timesteps 2967.
Path 282 | total_timesteps 2977.
Path 283 | total_timesteps 2987.
Path 284 | total_timesteps 3000.
Path 285 | total_timesteps 3012.
Path 286 | total_timesteps 3023.
Path 287 | total_timesteps 3033.
Path 288 | total_timesteps 3044.
Path 289 | total_timesteps 3053.
Path 290 | total_timesteps 3060.
Path 291 | total_timesteps 3074.
Path 292 | total_timesteps 3084.
Path 293 | total_timesteps 3094.
Path 294 | total_timesteps 3108.
Path 295 | total_timesteps 3116.
Path 296 | total_timesteps 3125.
Path 297 | total_timesteps 3135.
Path 298 | total_timesteps 3149.
Path 299 | total_timesteps 3159.
Path 300 | total_timesteps 3171.
Path 301 | total_timesteps 3180.
Path 302 | total_timesteps 3189.
Path 303 | total_timesteps 3202.
Path 304 | total_timesteps 3211.
Path 305 | total_timesteps 3220.
Path 306 | total_timesteps 3227.
Path 307 | total_timesteps 3237.
Path 308 | total_timesteps 3248.
Path 309 | total_timesteps 3256.
Path 310 | total_timesteps 3268.
Path 311 | total_timesteps 3281.
Path 312 | total_timesteps 3288.
Path 313 | total_timesteps 3299.
Path 314 | total_timesteps 3311.
Path 315 | total_timesteps 3319.
Path 316 | total_timesteps 3335.
Path 317 | total_timesteps 3344.
Path 318 | total_timesteps 3357.
Path 319 | total_timesteps 3367.
Path 320 | total_timesteps 3379.
Path 321 | total_timesteps 3392.
Path 322 | total_timesteps 3403.
Path 323 | total_timesteps 3413.
Path 324 | total_timesteps 3421.
Path 325 | total_timesteps 3436.
Path 326 | total_timesteps 3445.
Path 327 | total_timesteps 3452.
Path 328 | total_timesteps 3464.
Path 329 | total_timesteps 3474.
Path 330 | total_timesteps 3485.
Path 331 | total_timesteps 3493.
Path 332 | total_timesteps 3502.
Path 333 | total_timesteps 3512.
Path 334 | total_timesteps 3520.
Path 335 | total_timesteps 3531.
Path 336 | total_timesteps 3540.
Path 337 | total_timesteps 3548.
Path 338 | total_timesteps 3560.
Path 339 | total_timesteps 3573.
Path 340 | total_timesteps 3583.
Path 341 | total_timesteps 3599.
Path 342 | total_timesteps 3618.
Path 343 | total_timesteps 3628.
Path 344 | total_timesteps 3642.
Path 345 | total_timesteps 3665.
Path 346 | total_timesteps 3674.
Path 347 | total_timesteps 3700.
Path 348 | total_timesteps 3713.
Path 349 | total_timesteps 3733.
Path 350 | total_timesteps 3743.
Path 351 | total_timesteps 3753.
Path 352 | total_timesteps 3765.
Path 353 | total_timesteps 3775.
Path 354 | total_timesteps 3786.
Path 355 | total_timesteps 3799.
Path 356 | total_timesteps 3808.
Path 357 | total_timesteps 3823.
Path 358 | total_timesteps 3832.
Path 359 | total_timesteps 3839.
Path 360 | total_timesteps 3846.
Path 361 | total_timesteps 3854.
Path 362 | total_timesteps 3867.
Path 363 | total_timesteps 3877.
Path 364 | total_timesteps 3888.
Path 365 | total_timesteps 3906.
Path 366 | total_timesteps 3916.
Path 367 | total_timesteps 3925.
Path 368 | total_timesteps 3937.
Path 369 | total_timesteps 3947.
Path 370 | total_timesteps 3957.
Path 371 | total_timesteps 3966.
Path 372 | total_timesteps 3974.
Path 373 | total_timesteps 3984.
Path 374 | total_timesteps 3994.
Path 375 | total_timesteps 4002.
Path 376 | total_timesteps 4011.
Path 377 | total_timesteps 4019.
Path 378 | total_timesteps 4035.
Path 379 | total_timesteps 4053.
Path 380 | total_timesteps 4060.
Path 381 | total_timesteps 4080.
Path 382 | total_timesteps 4092.
Path 383 | total_timesteps 4100.
Path 384 | total_timesteps 4107.
Path 385 | total_timesteps 4117.
Path 386 | total_timesteps 4141.
Path 387 | total_timesteps 4153.
Path 388 | total_timesteps 4161.
Path 389 | total_timesteps 4174.
Path 390 | total_timesteps 4182.
Path 391 | total_timesteps 4191.
Path 392 | total_timesteps 4207.
Path 393 | total_timesteps 4216.
Path 394 | total_timesteps 4223.
Path 395 | total_timesteps 4235.
Path 396 | total_timesteps 4242.
Path 397 | total_timesteps 4253.
Path 398 | total_timesteps 4267.
Path 399 | total_timesteps 4278.
Path 400 | total_timesteps 4289.
Path 401 | total_timesteps 4300.
Path 402 | total_timesteps 4312.
Path 403 | total_timesteps 4320.
Path 404 | total_timesteps 4341.
Path 405 | total_timesteps 4352.
Path 406 | total_timesteps 4363.
Path 407 | total_timesteps 4375.
Path 408 | total_timesteps 4385.
Path 409 | total_timesteps 4395.
Path 410 | total_timesteps 4403.
Path 411 | total_timesteps 4414.
Path 412 | total_timesteps 4426.
Path 413 | total_timesteps 4435.
Path 414 | total_timesteps 4447.
Path 415 | total_timesteps 4459.
Path 416 | total_timesteps 4467.
Path 417 | total_timesteps 4477.
Path 418 | total_timesteps 4489.
Path 419 | total_timesteps 4497.
Path 420 | total_timesteps 4513.
Path 421 | total_timesteps 4522.
Path 422 | total_timesteps 4533.
Path 423 | total_timesteps 4540.
Path 424 | total_timesteps 4550.
Path 425 | total_timesteps 4568.
Path 426 | total_timesteps 4578.
Path 427 | total_timesteps 4588.
Path 428 | total_timesteps 4602.
Path 429 | total_timesteps 4610.
Path 430 | total_timesteps 4618.
Path 431 | total_timesteps 4629.
Path 432 | total_timesteps 4637.
Path 433 | total_timesteps 4649.
Path 434 | total_timesteps 4656.
Path 435 | total_timesteps 4665.
Path 436 | total_timesteps 4674.
Path 437 | total_timesteps 4683.
Path 438 | total_timesteps 4692.
Path 439 | total_timesteps 4701.
Path 440 | total_timesteps 4711.
Path 441 | total_timesteps 4722.
Path 442 | total_timesteps 4735.
Path 443 | total_timesteps 4745.
Path 444 | total_timesteps 4753.
Path 445 | total_timesteps 4762.
Path 446 | total_timesteps 4771.
Path 447 | total_timesteps 4781.
Path 448 | total_timesteps 4791.
Path 449 | total_timesteps 4801.
Path 450 | total_timesteps 4812.
Path 451 | total_timesteps 4820.
Path 452 | total_timesteps 4839.
Path 453 | total_timesteps 4849.
Path 454 | total_timesteps 4864.
Path 455 | total_timesteps 4875.
Path 456 | total_timesteps 4900.
Path 457 | total_timesteps 4910.
Path 458 | total_timesteps 4919.
Path 459 | total_timesteps 4933.
Path 460 | total_timesteps 4944.
Path 461 | total_timesteps 4952.
Path 462 | total_timesteps 4962.
Path 463 | total_timesteps 4969.
Path 464 | total_timesteps 4981.
Path 465 | total_timesteps 4990.
Path 466 | total_timesteps 5002.
Path 467 | total_timesteps 5008.
Path 468 | total_timesteps 5017.
Path 469 | total_timesteps 5028.
Path 470 | total_timesteps 5037.
Path 471 | total_timesteps 5044.
Path 472 | total_timesteps 5051.
Path 473 | total_timesteps 5065.
Path 474 | total_timesteps 5077.
Path 475 | total_timesteps 5085.
Path 476 | total_timesteps 5100.
Path 477 | total_timesteps 5116.
Path 478 | total_timesteps 5127.
Path 479 | total_timesteps 5134.
Path 480 | total_timesteps 5145.
Path 481 | total_timesteps 5154.
Path 482 | total_timesteps 5169.
Path 483 | total_timesteps 5182.
Path 484 | total_timesteps 5198.
Path 485 | total_timesteps 5204.
Path 486 | total_timesteps 5212.
Path 487 | total_timesteps 5222.
Path 488 | total_timesteps 5233.
Path 489 | total_timesteps 5240.
Path 490 | total_timesteps 5255.
Path 491 | total_timesteps 5264.
Path 492 | total_timesteps 5282.
Path 493 | total_timesteps 5296.
Path 494 | total_timesteps 5315.
Path 495 | total_timesteps 5325.
Path 496 | total_timesteps 5339.
Path 497 | total_timesteps 5345.
Path 498 | total_timesteps 5356.
Path 499 | total_timesteps 5364.
Path 500 | total_timesteps 5379.
Path 501 | total_timesteps 5388.
Path 502 | total_timesteps 5396.
Path 503 | total_timesteps 5406.
Path 504 | total_timesteps 5416.
Path 505 | total_timesteps 5426.
Path 506 | total_timesteps 5433.
Path 507 | total_timesteps 5447.
Path 508 | total_timesteps 5463.
Path 509 | total_timesteps 5472.
Path 510 | total_timesteps 5480.
Path 511 | total_timesteps 5491.
Path 512 | total_timesteps 5499.
Path 513 | total_timesteps 5512.
Path 514 | total_timesteps 5523.
Path 515 | total_timesteps 5532.
Path 516 | total_timesteps 5542.
Path 517 | total_timesteps 5551.
Path 518 | total_timesteps 5559.
Path 519 | total_timesteps 5568.
Path 520 | total_timesteps 5577.
Path 521 | total_timesteps 5586.
Path 522 | total_timesteps 5595.
Path 523 | total_timesteps 5608.
Path 524 | total_timesteps 5618.
Path 525 | total_timesteps 5627.
Path 526 | total_timesteps 5642.
Path 527 | total_timesteps 5654.
Path 528 | total_timesteps 5664.
Path 529 | total_timesteps 5672.
Path 530 | total_timesteps 5682.
Path 531 | total_timesteps 5692.
Path 532 | total_timesteps 5704.
Path 533 | total_timesteps 5712.
Path 534 | total_timesteps 5720.
Path 535 | total_timesteps 5730.
Path 536 | total_timesteps 5739.
Path 537 | total_timesteps 5750.
Path 538 | total_timesteps 5763.
Path 539 | total_timesteps 5776.
Path 540 | total_timesteps 5786.
Path 541 | total_timesteps 5795.
Path 542 | total_timesteps 5810.
Path 543 | total_timesteps 5819.
Path 544 | total_timesteps 5831.
Path 545 | total_timesteps 5840.
Path 546 | total_timesteps 5848.
Path 547 | total_timesteps 5856.
Path 548 | total_timesteps 5877.
Path 549 | total_timesteps 5890.
Path 550 | total_timesteps 5901.
Path 551 | total_timesteps 5911.
Path 552 | total_timesteps 5922.
Path 553 | total_timesteps 5937.
Path 554 | total_timesteps 5947.
Path 555 | total_timesteps 5962.
Path 556 | total_timesteps 5970.
Path 557 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.94    |
| Iteration     | 22       |
| MaximumReturn | 1.92     |
| MinimumReturn | -20.6    |
| TotalSamples  | 96090    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.31870946288108826
Validation loss = 0.3201946020126343
Validation loss = 0.3197809159755707
Validation loss = 0.32417547702789307
Validation loss = 0.3223641812801361
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 45.
Path 5 | total_timesteps 61.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 85.
Path 8 | total_timesteps 99.
Path 9 | total_timesteps 107.
Path 10 | total_timesteps 115.
Path 11 | total_timesteps 124.
Path 12 | total_timesteps 133.
Path 13 | total_timesteps 142.
Path 14 | total_timesteps 151.
Path 15 | total_timesteps 161.
Path 16 | total_timesteps 172.
Path 17 | total_timesteps 181.
Path 18 | total_timesteps 195.
Path 19 | total_timesteps 211.
Path 20 | total_timesteps 220.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 245.
Path 23 | total_timesteps 258.
Path 24 | total_timesteps 267.
Path 25 | total_timesteps 277.
Path 26 | total_timesteps 284.
Path 27 | total_timesteps 294.
Path 28 | total_timesteps 305.
Path 29 | total_timesteps 314.
Path 30 | total_timesteps 322.
Path 31 | total_timesteps 332.
Path 32 | total_timesteps 342.
Path 33 | total_timesteps 354.
Path 34 | total_timesteps 364.
Path 35 | total_timesteps 372.
Path 36 | total_timesteps 382.
Path 37 | total_timesteps 392.
Path 38 | total_timesteps 403.
Path 39 | total_timesteps 412.
Path 40 | total_timesteps 422.
Path 41 | total_timesteps 430.
Path 42 | total_timesteps 441.
Path 43 | total_timesteps 450.
Path 44 | total_timesteps 462.
Path 45 | total_timesteps 470.
Path 46 | total_timesteps 482.
Path 47 | total_timesteps 495.
Path 48 | total_timesteps 505.
Path 49 | total_timesteps 514.
Path 50 | total_timesteps 523.
Path 51 | total_timesteps 533.
Path 52 | total_timesteps 552.
Path 53 | total_timesteps 570.
Path 54 | total_timesteps 581.
Path 55 | total_timesteps 603.
Path 56 | total_timesteps 615.
Path 57 | total_timesteps 623.
Path 58 | total_timesteps 633.
Path 59 | total_timesteps 643.
Path 60 | total_timesteps 655.
Path 61 | total_timesteps 667.
Path 62 | total_timesteps 675.
Path 63 | total_timesteps 685.
Path 64 | total_timesteps 697.
Path 65 | total_timesteps 708.
Path 66 | total_timesteps 719.
Path 67 | total_timesteps 727.
Path 68 | total_timesteps 734.
Path 69 | total_timesteps 750.
Path 70 | total_timesteps 758.
Path 71 | total_timesteps 768.
Path 72 | total_timesteps 778.
Path 73 | total_timesteps 788.
Path 74 | total_timesteps 802.
Path 75 | total_timesteps 811.
Path 76 | total_timesteps 821.
Path 77 | total_timesteps 834.
Path 78 | total_timesteps 848.
Path 79 | total_timesteps 855.
Path 80 | total_timesteps 865.
Path 81 | total_timesteps 877.
Path 82 | total_timesteps 885.
Path 83 | total_timesteps 895.
Path 84 | total_timesteps 910.
Path 85 | total_timesteps 917.
Path 86 | total_timesteps 924.
Path 87 | total_timesteps 934.
Path 88 | total_timesteps 944.
Path 89 | total_timesteps 951.
Path 90 | total_timesteps 961.
Path 91 | total_timesteps 971.
Path 92 | total_timesteps 980.
Path 93 | total_timesteps 990.
Path 94 | total_timesteps 998.
Path 95 | total_timesteps 1006.
Path 96 | total_timesteps 1014.
Path 97 | total_timesteps 1022.
Path 98 | total_timesteps 1029.
Path 99 | total_timesteps 1047.
Path 100 | total_timesteps 1058.
Path 101 | total_timesteps 1068.
Path 102 | total_timesteps 1080.
Path 103 | total_timesteps 1092.
Path 104 | total_timesteps 1099.
Path 105 | total_timesteps 1112.
Path 106 | total_timesteps 1120.
Path 107 | total_timesteps 1128.
Path 108 | total_timesteps 1139.
Path 109 | total_timesteps 1147.
Path 110 | total_timesteps 1157.
Path 111 | total_timesteps 1164.
Path 112 | total_timesteps 1174.
Path 113 | total_timesteps 1182.
Path 114 | total_timesteps 1193.
Path 115 | total_timesteps 1204.
Path 116 | total_timesteps 1211.
Path 117 | total_timesteps 1219.
Path 118 | total_timesteps 1228.
Path 119 | total_timesteps 1238.
Path 120 | total_timesteps 1247.
Path 121 | total_timesteps 1256.
Path 122 | total_timesteps 1267.
Path 123 | total_timesteps 1278.
Path 124 | total_timesteps 1286.
Path 125 | total_timesteps 1307.
Path 126 | total_timesteps 1317.
Path 127 | total_timesteps 1330.
Path 128 | total_timesteps 1337.
Path 129 | total_timesteps 1347.
Path 130 | total_timesteps 1355.
Path 131 | total_timesteps 1363.
Path 132 | total_timesteps 1374.
Path 133 | total_timesteps 1383.
Path 134 | total_timesteps 1390.
Path 135 | total_timesteps 1406.
Path 136 | total_timesteps 1416.
Path 137 | total_timesteps 1427.
Path 138 | total_timesteps 1440.
Path 139 | total_timesteps 1447.
Path 140 | total_timesteps 1462.
Path 141 | total_timesteps 1471.
Path 142 | total_timesteps 1480.
Path 143 | total_timesteps 1490.
Path 144 | total_timesteps 1502.
Path 145 | total_timesteps 1515.
Path 146 | total_timesteps 1523.
Path 147 | total_timesteps 1530.
Path 148 | total_timesteps 1541.
Path 149 | total_timesteps 1551.
Path 150 | total_timesteps 1560.
Path 151 | total_timesteps 1572.
Path 152 | total_timesteps 1579.
Path 153 | total_timesteps 1589.
Path 154 | total_timesteps 1602.
Path 155 | total_timesteps 1612.
Path 156 | total_timesteps 1620.
Path 157 | total_timesteps 1635.
Path 158 | total_timesteps 1647.
Path 159 | total_timesteps 1655.
Path 160 | total_timesteps 1663.
Path 161 | total_timesteps 1672.
Path 162 | total_timesteps 1680.
Path 163 | total_timesteps 1688.
Path 164 | total_timesteps 1702.
Path 165 | total_timesteps 1711.
Path 166 | total_timesteps 1722.
Path 167 | total_timesteps 1731.
Path 168 | total_timesteps 1743.
Path 169 | total_timesteps 1750.
Path 170 | total_timesteps 1761.
Path 171 | total_timesteps 1769.
Path 172 | total_timesteps 1778.
Path 173 | total_timesteps 1788.
Path 174 | total_timesteps 1799.
Path 175 | total_timesteps 1809.
Path 176 | total_timesteps 1825.
Path 177 | total_timesteps 1835.
Path 178 | total_timesteps 1845.
Path 179 | total_timesteps 1851.
Path 180 | total_timesteps 1863.
Path 181 | total_timesteps 1880.
Path 182 | total_timesteps 1888.
Path 183 | total_timesteps 1898.
Path 184 | total_timesteps 1908.
Path 185 | total_timesteps 1924.
Path 186 | total_timesteps 1932.
Path 187 | total_timesteps 1942.
Path 188 | total_timesteps 1952.
Path 189 | total_timesteps 1965.
Path 190 | total_timesteps 1976.
Path 191 | total_timesteps 1989.
Path 192 | total_timesteps 2002.
Path 193 | total_timesteps 2009.
Path 194 | total_timesteps 2017.
Path 195 | total_timesteps 2029.
Path 196 | total_timesteps 2037.
Path 197 | total_timesteps 2052.
Path 198 | total_timesteps 2061.
Path 199 | total_timesteps 2082.
Path 200 | total_timesteps 2094.
Path 201 | total_timesteps 2104.
Path 202 | total_timesteps 2112.
Path 203 | total_timesteps 2131.
Path 204 | total_timesteps 2140.
Path 205 | total_timesteps 2147.
Path 206 | total_timesteps 2155.
Path 207 | total_timesteps 2164.
Path 208 | total_timesteps 2174.
Path 209 | total_timesteps 2185.
Path 210 | total_timesteps 2192.
Path 211 | total_timesteps 2198.
Path 212 | total_timesteps 2212.
Path 213 | total_timesteps 2222.
Path 214 | total_timesteps 2229.
Path 215 | total_timesteps 2238.
Path 216 | total_timesteps 2250.
Path 217 | total_timesteps 2260.
Path 218 | total_timesteps 2270.
Path 219 | total_timesteps 2279.
Path 220 | total_timesteps 2296.
Path 221 | total_timesteps 2312.
Path 222 | total_timesteps 2326.
Path 223 | total_timesteps 2337.
Path 224 | total_timesteps 2347.
Path 225 | total_timesteps 2356.
Path 226 | total_timesteps 2364.
Path 227 | total_timesteps 2371.
Path 228 | total_timesteps 2382.
Path 229 | total_timesteps 2395.
Path 230 | total_timesteps 2405.
Path 231 | total_timesteps 2419.
Path 232 | total_timesteps 2429.
Path 233 | total_timesteps 2437.
Path 234 | total_timesteps 2445.
Path 235 | total_timesteps 2455.
Path 236 | total_timesteps 2464.
Path 237 | total_timesteps 2475.
Path 238 | total_timesteps 2485.
Path 239 | total_timesteps 2495.
Path 240 | total_timesteps 2504.
Path 241 | total_timesteps 2513.
Path 242 | total_timesteps 2522.
Path 243 | total_timesteps 2531.
Path 244 | total_timesteps 2538.
Path 245 | total_timesteps 2548.
Path 246 | total_timesteps 2556.
Path 247 | total_timesteps 2570.
Path 248 | total_timesteps 2578.
Path 249 | total_timesteps 2591.
Path 250 | total_timesteps 2610.
Path 251 | total_timesteps 2617.
Path 252 | total_timesteps 2624.
Path 253 | total_timesteps 2636.
Path 254 | total_timesteps 2644.
Path 255 | total_timesteps 2655.
Path 256 | total_timesteps 2668.
Path 257 | total_timesteps 2679.
Path 258 | total_timesteps 2686.
Path 259 | total_timesteps 2700.
Path 260 | total_timesteps 2714.
Path 261 | total_timesteps 2724.
Path 262 | total_timesteps 2735.
Path 263 | total_timesteps 2748.
Path 264 | total_timesteps 2758.
Path 265 | total_timesteps 2773.
Path 266 | total_timesteps 2784.
Path 267 | total_timesteps 2800.
Path 268 | total_timesteps 2811.
Path 269 | total_timesteps 2824.
Path 270 | total_timesteps 2832.
Path 271 | total_timesteps 2839.
Path 272 | total_timesteps 2848.
Path 273 | total_timesteps 2862.
Path 274 | total_timesteps 2870.
Path 275 | total_timesteps 2880.
Path 276 | total_timesteps 2896.
Path 277 | total_timesteps 2907.
Path 278 | total_timesteps 2916.
Path 279 | total_timesteps 2924.
Path 280 | total_timesteps 2933.
Path 281 | total_timesteps 2941.
Path 282 | total_timesteps 2951.
Path 283 | total_timesteps 2962.
Path 284 | total_timesteps 2976.
Path 285 | total_timesteps 2984.
Path 286 | total_timesteps 2995.
Path 287 | total_timesteps 3003.
Path 288 | total_timesteps 3014.
Path 289 | total_timesteps 3024.
Path 290 | total_timesteps 3035.
Path 291 | total_timesteps 3045.
Path 292 | total_timesteps 3057.
Path 293 | total_timesteps 3066.
Path 294 | total_timesteps 3075.
Path 295 | total_timesteps 3082.
Path 296 | total_timesteps 3091.
Path 297 | total_timesteps 3100.
Path 298 | total_timesteps 3110.
Path 299 | total_timesteps 3124.
Path 300 | total_timesteps 3133.
Path 301 | total_timesteps 3143.
Path 302 | total_timesteps 3152.
Path 303 | total_timesteps 3162.
Path 304 | total_timesteps 3171.
Path 305 | total_timesteps 3179.
Path 306 | total_timesteps 3189.
Path 307 | total_timesteps 3197.
Path 308 | total_timesteps 3208.
Path 309 | total_timesteps 3221.
Path 310 | total_timesteps 3232.
Path 311 | total_timesteps 3243.
Path 312 | total_timesteps 3254.
Path 313 | total_timesteps 3269.
Path 314 | total_timesteps 3280.
Path 315 | total_timesteps 3292.
Path 316 | total_timesteps 3303.
Path 317 | total_timesteps 3316.
Path 318 | total_timesteps 3323.
Path 319 | total_timesteps 3333.
Path 320 | total_timesteps 3341.
Path 321 | total_timesteps 3350.
Path 322 | total_timesteps 3360.
Path 323 | total_timesteps 3371.
Path 324 | total_timesteps 3379.
Path 325 | total_timesteps 3389.
Path 326 | total_timesteps 3401.
Path 327 | total_timesteps 3413.
Path 328 | total_timesteps 3421.
Path 329 | total_timesteps 3429.
Path 330 | total_timesteps 3436.
Path 331 | total_timesteps 3446.
Path 332 | total_timesteps 3456.
Path 333 | total_timesteps 3466.
Path 334 | total_timesteps 3484.
Path 335 | total_timesteps 3495.
Path 336 | total_timesteps 3503.
Path 337 | total_timesteps 3513.
Path 338 | total_timesteps 3522.
Path 339 | total_timesteps 3533.
Path 340 | total_timesteps 3540.
Path 341 | total_timesteps 3555.
Path 342 | total_timesteps 3562.
Path 343 | total_timesteps 3572.
Path 344 | total_timesteps 3581.
Path 345 | total_timesteps 3594.
Path 346 | total_timesteps 3601.
Path 347 | total_timesteps 3611.
Path 348 | total_timesteps 3621.
Path 349 | total_timesteps 3631.
Path 350 | total_timesteps 3639.
Path 351 | total_timesteps 3647.
Path 352 | total_timesteps 3657.
Path 353 | total_timesteps 3666.
Path 354 | total_timesteps 3676.
Path 355 | total_timesteps 3687.
Path 356 | total_timesteps 3697.
Path 357 | total_timesteps 3708.
Path 358 | total_timesteps 3719.
Path 359 | total_timesteps 3731.
Path 360 | total_timesteps 3740.
Path 361 | total_timesteps 3753.
Path 362 | total_timesteps 3763.
Path 363 | total_timesteps 3770.
Path 364 | total_timesteps 3780.
Path 365 | total_timesteps 3787.
Path 366 | total_timesteps 3795.
Path 367 | total_timesteps 3803.
Path 368 | total_timesteps 3815.
Path 369 | total_timesteps 3824.
Path 370 | total_timesteps 3835.
Path 371 | total_timesteps 3842.
Path 372 | total_timesteps 3849.
Path 373 | total_timesteps 3856.
Path 374 | total_timesteps 3868.
Path 375 | total_timesteps 3880.
Path 376 | total_timesteps 3891.
Path 377 | total_timesteps 3901.
Path 378 | total_timesteps 3913.
Path 379 | total_timesteps 3924.
Path 380 | total_timesteps 3937.
Path 381 | total_timesteps 3952.
Path 382 | total_timesteps 3964.
Path 383 | total_timesteps 3972.
Path 384 | total_timesteps 3979.
Path 385 | total_timesteps 3995.
Path 386 | total_timesteps 4005.
Path 387 | total_timesteps 4014.
Path 388 | total_timesteps 4025.
Path 389 | total_timesteps 4038.
Path 390 | total_timesteps 4050.
Path 391 | total_timesteps 4063.
Path 392 | total_timesteps 4075.
Path 393 | total_timesteps 4084.
Path 394 | total_timesteps 4097.
Path 395 | total_timesteps 4109.
Path 396 | total_timesteps 4118.
Path 397 | total_timesteps 4128.
Path 398 | total_timesteps 4139.
Path 399 | total_timesteps 4148.
Path 400 | total_timesteps 4155.
Path 401 | total_timesteps 4166.
Path 402 | total_timesteps 4177.
Path 403 | total_timesteps 4192.
Path 404 | total_timesteps 4204.
Path 405 | total_timesteps 4236.
Path 406 | total_timesteps 4247.
Path 407 | total_timesteps 4258.
Path 408 | total_timesteps 4267.
Path 409 | total_timesteps 4279.
Path 410 | total_timesteps 4293.
Path 411 | total_timesteps 4311.
Path 412 | total_timesteps 4333.
Path 413 | total_timesteps 4343.
Path 414 | total_timesteps 4359.
Path 415 | total_timesteps 4371.
Path 416 | total_timesteps 4380.
Path 417 | total_timesteps 4388.
Path 418 | total_timesteps 4397.
Path 419 | total_timesteps 4408.
Path 420 | total_timesteps 4420.
Path 421 | total_timesteps 4433.
Path 422 | total_timesteps 4446.
Path 423 | total_timesteps 4455.
Path 424 | total_timesteps 4466.
Path 425 | total_timesteps 4475.
Path 426 | total_timesteps 4485.
Path 427 | total_timesteps 4494.
Path 428 | total_timesteps 4510.
Path 429 | total_timesteps 4520.
Path 430 | total_timesteps 4534.
Path 431 | total_timesteps 4546.
Path 432 | total_timesteps 4559.
Path 433 | total_timesteps 4570.
Path 434 | total_timesteps 4579.
Path 435 | total_timesteps 4589.
Path 436 | total_timesteps 4595.
Path 437 | total_timesteps 4605.
Path 438 | total_timesteps 4632.
Path 439 | total_timesteps 4643.
Path 440 | total_timesteps 4654.
Path 441 | total_timesteps 4661.
Path 442 | total_timesteps 4670.
Path 443 | total_timesteps 4683.
Path 444 | total_timesteps 4695.
Path 445 | total_timesteps 4703.
Path 446 | total_timesteps 4711.
Path 447 | total_timesteps 4722.
Path 448 | total_timesteps 4733.
Path 449 | total_timesteps 4742.
Path 450 | total_timesteps 4751.
Path 451 | total_timesteps 4759.
Path 452 | total_timesteps 4767.
Path 453 | total_timesteps 4781.
Path 454 | total_timesteps 4793.
Path 455 | total_timesteps 4802.
Path 456 | total_timesteps 4814.
Path 457 | total_timesteps 4822.
Path 458 | total_timesteps 4832.
Path 459 | total_timesteps 4848.
Path 460 | total_timesteps 4855.
Path 461 | total_timesteps 4864.
Path 462 | total_timesteps 4872.
Path 463 | total_timesteps 4884.
Path 464 | total_timesteps 4893.
Path 465 | total_timesteps 4903.
Path 466 | total_timesteps 4911.
Path 467 | total_timesteps 4926.
Path 468 | total_timesteps 4935.
Path 469 | total_timesteps 4944.
Path 470 | total_timesteps 4954.
Path 471 | total_timesteps 4966.
Path 472 | total_timesteps 4978.
Path 473 | total_timesteps 4985.
Path 474 | total_timesteps 4993.
Path 475 | total_timesteps 5004.
Path 476 | total_timesteps 5017.
Path 477 | total_timesteps 5032.
Path 478 | total_timesteps 5044.
Path 479 | total_timesteps 5053.
Path 480 | total_timesteps 5063.
Path 481 | total_timesteps 5071.
Path 482 | total_timesteps 5083.
Path 483 | total_timesteps 5091.
Path 484 | total_timesteps 5100.
Path 485 | total_timesteps 5113.
Path 486 | total_timesteps 5121.
Path 487 | total_timesteps 5132.
Path 488 | total_timesteps 5139.
Path 489 | total_timesteps 5152.
Path 490 | total_timesteps 5160.
Path 491 | total_timesteps 5170.
Path 492 | total_timesteps 5178.
Path 493 | total_timesteps 5185.
Path 494 | total_timesteps 5195.
Path 495 | total_timesteps 5203.
Path 496 | total_timesteps 5214.
Path 497 | total_timesteps 5224.
Path 498 | total_timesteps 5233.
Path 499 | total_timesteps 5245.
Path 500 | total_timesteps 5252.
Path 501 | total_timesteps 5260.
Path 502 | total_timesteps 5269.
Path 503 | total_timesteps 5283.
Path 504 | total_timesteps 5291.
Path 505 | total_timesteps 5301.
Path 506 | total_timesteps 5309.
Path 507 | total_timesteps 5319.
Path 508 | total_timesteps 5326.
Path 509 | total_timesteps 5339.
Path 510 | total_timesteps 5347.
Path 511 | total_timesteps 5357.
Path 512 | total_timesteps 5369.
Path 513 | total_timesteps 5375.
Path 514 | total_timesteps 5391.
Path 515 | total_timesteps 5405.
Path 516 | total_timesteps 5414.
Path 517 | total_timesteps 5432.
Path 518 | total_timesteps 5443.
Path 519 | total_timesteps 5450.
Path 520 | total_timesteps 5458.
Path 521 | total_timesteps 5467.
Path 522 | total_timesteps 5475.
Path 523 | total_timesteps 5490.
Path 524 | total_timesteps 5505.
Path 525 | total_timesteps 5517.
Path 526 | total_timesteps 5535.
Path 527 | total_timesteps 5542.
Path 528 | total_timesteps 5552.
Path 529 | total_timesteps 5561.
Path 530 | total_timesteps 5574.
Path 531 | total_timesteps 5591.
Path 532 | total_timesteps 5600.
Path 533 | total_timesteps 5610.
Path 534 | total_timesteps 5618.
Path 535 | total_timesteps 5626.
Path 536 | total_timesteps 5633.
Path 537 | total_timesteps 5652.
Path 538 | total_timesteps 5662.
Path 539 | total_timesteps 5675.
Path 540 | total_timesteps 5687.
Path 541 | total_timesteps 5696.
Path 542 | total_timesteps 5708.
Path 543 | total_timesteps 5719.
Path 544 | total_timesteps 5727.
Path 545 | total_timesteps 5736.
Path 546 | total_timesteps 5743.
Path 547 | total_timesteps 5749.
Path 548 | total_timesteps 5759.
Path 549 | total_timesteps 5767.
Path 550 | total_timesteps 5775.
Path 551 | total_timesteps 5789.
Path 552 | total_timesteps 5799.
Path 553 | total_timesteps 5821.
Path 554 | total_timesteps 5829.
Path 555 | total_timesteps 5837.
Path 556 | total_timesteps 5847.
Path 557 | total_timesteps 5863.
Path 558 | total_timesteps 5874.
Path 559 | total_timesteps 5887.
Path 560 | total_timesteps 5901.
Path 561 | total_timesteps 5912.
Path 562 | total_timesteps 5922.
Path 563 | total_timesteps 5934.
Path 564 | total_timesteps 5944.
Path 565 | total_timesteps 5953.
Path 566 | total_timesteps 5961.
Path 567 | total_timesteps 5969.
Path 568 | total_timesteps 5979.
Path 569 | total_timesteps 5985.
Path 570 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.79    |
| Iteration     | 23       |
| MaximumReturn | 0.985    |
| MinimumReturn | -17.5    |
| TotalSamples  | 100095   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.314910888671875
Validation loss = 0.31862759590148926
Validation loss = 0.3213937282562256
Validation loss = 0.32283657789230347
Validation loss = 0.3224465847015381
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 32.
Path 5 | total_timesteps 40.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 74.
Path 9 | total_timesteps 81.
Path 10 | total_timesteps 91.
Path 11 | total_timesteps 98.
Path 12 | total_timesteps 107.
Path 13 | total_timesteps 117.
Path 14 | total_timesteps 128.
Path 15 | total_timesteps 140.
Path 16 | total_timesteps 151.
Path 17 | total_timesteps 159.
Path 18 | total_timesteps 173.
Path 19 | total_timesteps 183.
Path 20 | total_timesteps 196.
Path 21 | total_timesteps 203.
Path 22 | total_timesteps 213.
Path 23 | total_timesteps 225.
Path 24 | total_timesteps 233.
Path 25 | total_timesteps 242.
Path 26 | total_timesteps 250.
Path 27 | total_timesteps 258.
Path 28 | total_timesteps 268.
Path 29 | total_timesteps 276.
Path 30 | total_timesteps 287.
Path 31 | total_timesteps 295.
Path 32 | total_timesteps 307.
Path 33 | total_timesteps 315.
Path 34 | total_timesteps 322.
Path 35 | total_timesteps 331.
Path 36 | total_timesteps 349.
Path 37 | total_timesteps 363.
Path 38 | total_timesteps 371.
Path 39 | total_timesteps 386.
Path 40 | total_timesteps 400.
Path 41 | total_timesteps 412.
Path 42 | total_timesteps 422.
Path 43 | total_timesteps 429.
Path 44 | total_timesteps 437.
Path 45 | total_timesteps 445.
Path 46 | total_timesteps 453.
Path 47 | total_timesteps 464.
Path 48 | total_timesteps 475.
Path 49 | total_timesteps 491.
Path 50 | total_timesteps 502.
Path 51 | total_timesteps 512.
Path 52 | total_timesteps 520.
Path 53 | total_timesteps 529.
Path 54 | total_timesteps 539.
Path 55 | total_timesteps 545.
Path 56 | total_timesteps 556.
Path 57 | total_timesteps 569.
Path 58 | total_timesteps 578.
Path 59 | total_timesteps 589.
Path 60 | total_timesteps 601.
Path 61 | total_timesteps 609.
Path 62 | total_timesteps 616.
Path 63 | total_timesteps 627.
Path 64 | total_timesteps 640.
Path 65 | total_timesteps 650.
Path 66 | total_timesteps 657.
Path 67 | total_timesteps 674.
Path 68 | total_timesteps 688.
Path 69 | total_timesteps 699.
Path 70 | total_timesteps 709.
Path 71 | total_timesteps 720.
Path 72 | total_timesteps 730.
Path 73 | total_timesteps 738.
Path 74 | total_timesteps 752.
Path 75 | total_timesteps 765.
Path 76 | total_timesteps 775.
Path 77 | total_timesteps 783.
Path 78 | total_timesteps 794.
Path 79 | total_timesteps 807.
Path 80 | total_timesteps 821.
Path 81 | total_timesteps 842.
Path 82 | total_timesteps 850.
Path 83 | total_timesteps 865.
Path 84 | total_timesteps 875.
Path 85 | total_timesteps 893.
Path 86 | total_timesteps 902.
Path 87 | total_timesteps 912.
Path 88 | total_timesteps 920.
Path 89 | total_timesteps 932.
Path 90 | total_timesteps 939.
Path 91 | total_timesteps 946.
Path 92 | total_timesteps 954.
Path 93 | total_timesteps 961.
Path 94 | total_timesteps 970.
Path 95 | total_timesteps 978.
Path 96 | total_timesteps 993.
Path 97 | total_timesteps 1001.
Path 98 | total_timesteps 1011.
Path 99 | total_timesteps 1022.
Path 100 | total_timesteps 1031.
Path 101 | total_timesteps 1045.
Path 102 | total_timesteps 1052.
Path 103 | total_timesteps 1067.
Path 104 | total_timesteps 1075.
Path 105 | total_timesteps 1087.
Path 106 | total_timesteps 1099.
Path 107 | total_timesteps 1107.
Path 108 | total_timesteps 1123.
Path 109 | total_timesteps 1132.
Path 110 | total_timesteps 1141.
Path 111 | total_timesteps 1153.
Path 112 | total_timesteps 1166.
Path 113 | total_timesteps 1178.
Path 114 | total_timesteps 1187.
Path 115 | total_timesteps 1197.
Path 116 | total_timesteps 1205.
Path 117 | total_timesteps 1214.
Path 118 | total_timesteps 1227.
Path 119 | total_timesteps 1237.
Path 120 | total_timesteps 1250.
Path 121 | total_timesteps 1259.
Path 122 | total_timesteps 1267.
Path 123 | total_timesteps 1275.
Path 124 | total_timesteps 1285.
Path 125 | total_timesteps 1297.
Path 126 | total_timesteps 1313.
Path 127 | total_timesteps 1322.
Path 128 | total_timesteps 1333.
Path 129 | total_timesteps 1343.
Path 130 | total_timesteps 1351.
Path 131 | total_timesteps 1360.
Path 132 | total_timesteps 1371.
Path 133 | total_timesteps 1383.
Path 134 | total_timesteps 1394.
Path 135 | total_timesteps 1406.
Path 136 | total_timesteps 1417.
Path 137 | total_timesteps 1425.
Path 138 | total_timesteps 1435.
Path 139 | total_timesteps 1445.
Path 140 | total_timesteps 1453.
Path 141 | total_timesteps 1460.
Path 142 | total_timesteps 1472.
Path 143 | total_timesteps 1485.
Path 144 | total_timesteps 1498.
Path 145 | total_timesteps 1507.
Path 146 | total_timesteps 1513.
Path 147 | total_timesteps 1522.
Path 148 | total_timesteps 1534.
Path 149 | total_timesteps 1542.
Path 150 | total_timesteps 1555.
Path 151 | total_timesteps 1568.
Path 152 | total_timesteps 1578.
Path 153 | total_timesteps 1590.
Path 154 | total_timesteps 1599.
Path 155 | total_timesteps 1608.
Path 156 | total_timesteps 1619.
Path 157 | total_timesteps 1628.
Path 158 | total_timesteps 1635.
Path 159 | total_timesteps 1644.
Path 160 | total_timesteps 1658.
Path 161 | total_timesteps 1667.
Path 162 | total_timesteps 1677.
Path 163 | total_timesteps 1693.
Path 164 | total_timesteps 1704.
Path 165 | total_timesteps 1718.
Path 166 | total_timesteps 1729.
Path 167 | total_timesteps 1743.
Path 168 | total_timesteps 1760.
Path 169 | total_timesteps 1768.
Path 170 | total_timesteps 1781.
Path 171 | total_timesteps 1793.
Path 172 | total_timesteps 1809.
Path 173 | total_timesteps 1822.
Path 174 | total_timesteps 1833.
Path 175 | total_timesteps 1844.
Path 176 | total_timesteps 1855.
Path 177 | total_timesteps 1866.
Path 178 | total_timesteps 1879.
Path 179 | total_timesteps 1885.
Path 180 | total_timesteps 1902.
Path 181 | total_timesteps 1916.
Path 182 | total_timesteps 1927.
Path 183 | total_timesteps 1938.
Path 184 | total_timesteps 1951.
Path 185 | total_timesteps 1960.
Path 186 | total_timesteps 1973.
Path 187 | total_timesteps 1984.
Path 188 | total_timesteps 1999.
Path 189 | total_timesteps 2007.
Path 190 | total_timesteps 2022.
Path 191 | total_timesteps 2040.
Path 192 | total_timesteps 2050.
Path 193 | total_timesteps 2063.
Path 194 | total_timesteps 2073.
Path 195 | total_timesteps 2081.
Path 196 | total_timesteps 2094.
Path 197 | total_timesteps 2109.
Path 198 | total_timesteps 2124.
Path 199 | total_timesteps 2135.
Path 200 | total_timesteps 2149.
Path 201 | total_timesteps 2166.
Path 202 | total_timesteps 2175.
Path 203 | total_timesteps 2183.
Path 204 | total_timesteps 2192.
Path 205 | total_timesteps 2201.
Path 206 | total_timesteps 2212.
Path 207 | total_timesteps 2221.
Path 208 | total_timesteps 2228.
Path 209 | total_timesteps 2238.
Path 210 | total_timesteps 2245.
Path 211 | total_timesteps 2255.
Path 212 | total_timesteps 2269.
Path 213 | total_timesteps 2281.
Path 214 | total_timesteps 2289.
Path 215 | total_timesteps 2300.
Path 216 | total_timesteps 2316.
Path 217 | total_timesteps 2325.
Path 218 | total_timesteps 2334.
Path 219 | total_timesteps 2346.
Path 220 | total_timesteps 2353.
Path 221 | total_timesteps 2364.
Path 222 | total_timesteps 2373.
Path 223 | total_timesteps 2384.
Path 224 | total_timesteps 2395.
Path 225 | total_timesteps 2406.
Path 226 | total_timesteps 2416.
Path 227 | total_timesteps 2428.
Path 228 | total_timesteps 2437.
Path 229 | total_timesteps 2455.
Path 230 | total_timesteps 2466.
Path 231 | total_timesteps 2475.
Path 232 | total_timesteps 2485.
Path 233 | total_timesteps 2495.
Path 234 | total_timesteps 2505.
Path 235 | total_timesteps 2515.
Path 236 | total_timesteps 2533.
Path 237 | total_timesteps 2544.
Path 238 | total_timesteps 2554.
Path 239 | total_timesteps 2562.
Path 240 | total_timesteps 2576.
Path 241 | total_timesteps 2592.
Path 242 | total_timesteps 2604.
Path 243 | total_timesteps 2615.
Path 244 | total_timesteps 2625.
Path 245 | total_timesteps 2635.
Path 246 | total_timesteps 2644.
Path 247 | total_timesteps 2656.
Path 248 | total_timesteps 2667.
Path 249 | total_timesteps 2681.
Path 250 | total_timesteps 2690.
Path 251 | total_timesteps 2699.
Path 252 | total_timesteps 2707.
Path 253 | total_timesteps 2716.
Path 254 | total_timesteps 2726.
Path 255 | total_timesteps 2737.
Path 256 | total_timesteps 2753.
Path 257 | total_timesteps 2764.
Path 258 | total_timesteps 2776.
Path 259 | total_timesteps 2788.
Path 260 | total_timesteps 2800.
Path 261 | total_timesteps 2807.
Path 262 | total_timesteps 2824.
Path 263 | total_timesteps 2833.
Path 264 | total_timesteps 2849.
Path 265 | total_timesteps 2858.
Path 266 | total_timesteps 2870.
Path 267 | total_timesteps 2884.
Path 268 | total_timesteps 2894.
Path 269 | total_timesteps 2903.
Path 270 | total_timesteps 2911.
Path 271 | total_timesteps 2921.
Path 272 | total_timesteps 2933.
Path 273 | total_timesteps 2941.
Path 274 | total_timesteps 2957.
Path 275 | total_timesteps 2969.
Path 276 | total_timesteps 2982.
Path 277 | total_timesteps 2990.
Path 278 | total_timesteps 3001.
Path 279 | total_timesteps 3019.
Path 280 | total_timesteps 3029.
Path 281 | total_timesteps 3042.
Path 282 | total_timesteps 3050.
Path 283 | total_timesteps 3058.
Path 284 | total_timesteps 3069.
Path 285 | total_timesteps 3077.
Path 286 | total_timesteps 3087.
Path 287 | total_timesteps 3098.
Path 288 | total_timesteps 3109.
Path 289 | total_timesteps 3134.
Path 290 | total_timesteps 3142.
Path 291 | total_timesteps 3156.
Path 292 | total_timesteps 3169.
Path 293 | total_timesteps 3185.
Path 294 | total_timesteps 3196.
Path 295 | total_timesteps 3207.
Path 296 | total_timesteps 3217.
Path 297 | total_timesteps 3226.
Path 298 | total_timesteps 3235.
Path 299 | total_timesteps 3246.
Path 300 | total_timesteps 3254.
Path 301 | total_timesteps 3267.
Path 302 | total_timesteps 3275.
Path 303 | total_timesteps 3284.
Path 304 | total_timesteps 3291.
Path 305 | total_timesteps 3298.
Path 306 | total_timesteps 3306.
Path 307 | total_timesteps 3314.
Path 308 | total_timesteps 3324.
Path 309 | total_timesteps 3334.
Path 310 | total_timesteps 3342.
Path 311 | total_timesteps 3351.
Path 312 | total_timesteps 3361.
Path 313 | total_timesteps 3370.
Path 314 | total_timesteps 3385.
Path 315 | total_timesteps 3397.
Path 316 | total_timesteps 3408.
Path 317 | total_timesteps 3422.
Path 318 | total_timesteps 3430.
Path 319 | total_timesteps 3438.
Path 320 | total_timesteps 3449.
Path 321 | total_timesteps 3462.
Path 322 | total_timesteps 3474.
Path 323 | total_timesteps 3483.
Path 324 | total_timesteps 3498.
Path 325 | total_timesteps 3509.
Path 326 | total_timesteps 3520.
Path 327 | total_timesteps 3530.
Path 328 | total_timesteps 3539.
Path 329 | total_timesteps 3547.
Path 330 | total_timesteps 3558.
Path 331 | total_timesteps 3565.
Path 332 | total_timesteps 3577.
Path 333 | total_timesteps 3589.
Path 334 | total_timesteps 3605.
Path 335 | total_timesteps 3616.
Path 336 | total_timesteps 3637.
Path 337 | total_timesteps 3650.
Path 338 | total_timesteps 3658.
Path 339 | total_timesteps 3670.
Path 340 | total_timesteps 3689.
Path 341 | total_timesteps 3703.
Path 342 | total_timesteps 3712.
Path 343 | total_timesteps 3724.
Path 344 | total_timesteps 3732.
Path 345 | total_timesteps 3738.
Path 346 | total_timesteps 3746.
Path 347 | total_timesteps 3754.
Path 348 | total_timesteps 3764.
Path 349 | total_timesteps 3775.
Path 350 | total_timesteps 3786.
Path 351 | total_timesteps 3795.
Path 352 | total_timesteps 3806.
Path 353 | total_timesteps 3817.
Path 354 | total_timesteps 3840.
Path 355 | total_timesteps 3855.
Path 356 | total_timesteps 3865.
Path 357 | total_timesteps 3874.
Path 358 | total_timesteps 3888.
Path 359 | total_timesteps 3895.
Path 360 | total_timesteps 3904.
Path 361 | total_timesteps 3920.
Path 362 | total_timesteps 3927.
Path 363 | total_timesteps 3935.
Path 364 | total_timesteps 3942.
Path 365 | total_timesteps 3949.
Path 366 | total_timesteps 3959.
Path 367 | total_timesteps 3981.
Path 368 | total_timesteps 3989.
Path 369 | total_timesteps 4000.
Path 370 | total_timesteps 4008.
Path 371 | total_timesteps 4018.
Path 372 | total_timesteps 4026.
Path 373 | total_timesteps 4038.
Path 374 | total_timesteps 4048.
Path 375 | total_timesteps 4060.
Path 376 | total_timesteps 4070.
Path 377 | total_timesteps 4078.
Path 378 | total_timesteps 4088.
Path 379 | total_timesteps 4098.
Path 380 | total_timesteps 4106.
Path 381 | total_timesteps 4115.
Path 382 | total_timesteps 4125.
Path 383 | total_timesteps 4135.
Path 384 | total_timesteps 4143.
Path 385 | total_timesteps 4154.
Path 386 | total_timesteps 4167.
Path 387 | total_timesteps 4178.
Path 388 | total_timesteps 4188.
Path 389 | total_timesteps 4198.
Path 390 | total_timesteps 4205.
Path 391 | total_timesteps 4217.
Path 392 | total_timesteps 4226.
Path 393 | total_timesteps 4239.
Path 394 | total_timesteps 4253.
Path 395 | total_timesteps 4265.
Path 396 | total_timesteps 4273.
Path 397 | total_timesteps 4284.
Path 398 | total_timesteps 4294.
Path 399 | total_timesteps 4306.
Path 400 | total_timesteps 4317.
Path 401 | total_timesteps 4327.
Path 402 | total_timesteps 4336.
Path 403 | total_timesteps 4345.
Path 404 | total_timesteps 4357.
Path 405 | total_timesteps 4365.
Path 406 | total_timesteps 4373.
Path 407 | total_timesteps 4381.
Path 408 | total_timesteps 4397.
Path 409 | total_timesteps 4406.
Path 410 | total_timesteps 4418.
Path 411 | total_timesteps 4426.
Path 412 | total_timesteps 4439.
Path 413 | total_timesteps 4453.
Path 414 | total_timesteps 4464.
Path 415 | total_timesteps 4473.
Path 416 | total_timesteps 4480.
Path 417 | total_timesteps 4488.
Path 418 | total_timesteps 4496.
Path 419 | total_timesteps 4507.
Path 420 | total_timesteps 4518.
Path 421 | total_timesteps 4526.
Path 422 | total_timesteps 4536.
Path 423 | total_timesteps 4545.
Path 424 | total_timesteps 4553.
Path 425 | total_timesteps 4562.
Path 426 | total_timesteps 4577.
Path 427 | total_timesteps 4587.
Path 428 | total_timesteps 4597.
Path 429 | total_timesteps 4609.
Path 430 | total_timesteps 4619.
Path 431 | total_timesteps 4632.
Path 432 | total_timesteps 4643.
Path 433 | total_timesteps 4651.
Path 434 | total_timesteps 4664.
Path 435 | total_timesteps 4671.
Path 436 | total_timesteps 4698.
Path 437 | total_timesteps 4708.
Path 438 | total_timesteps 4722.
Path 439 | total_timesteps 4738.
Path 440 | total_timesteps 4748.
Path 441 | total_timesteps 4761.
Path 442 | total_timesteps 4769.
Path 443 | total_timesteps 4781.
Path 444 | total_timesteps 4793.
Path 445 | total_timesteps 4804.
Path 446 | total_timesteps 4815.
Path 447 | total_timesteps 4826.
Path 448 | total_timesteps 4839.
Path 449 | total_timesteps 4850.
Path 450 | total_timesteps 4866.
Path 451 | total_timesteps 4877.
Path 452 | total_timesteps 4886.
Path 453 | total_timesteps 4896.
Path 454 | total_timesteps 4906.
Path 455 | total_timesteps 4920.
Path 456 | total_timesteps 4928.
Path 457 | total_timesteps 4939.
Path 458 | total_timesteps 4949.
Path 459 | total_timesteps 4957.
Path 460 | total_timesteps 4966.
Path 461 | total_timesteps 4979.
Path 462 | total_timesteps 4985.
Path 463 | total_timesteps 4994.
Path 464 | total_timesteps 5005.
Path 465 | total_timesteps 5021.
Path 466 | total_timesteps 5029.
Path 467 | total_timesteps 5040.
Path 468 | total_timesteps 5050.
Path 469 | total_timesteps 5058.
Path 470 | total_timesteps 5071.
Path 471 | total_timesteps 5079.
Path 472 | total_timesteps 5092.
Path 473 | total_timesteps 5105.
Path 474 | total_timesteps 5114.
Path 475 | total_timesteps 5124.
Path 476 | total_timesteps 5137.
Path 477 | total_timesteps 5147.
Path 478 | total_timesteps 5158.
Path 479 | total_timesteps 5165.
Path 480 | total_timesteps 5178.
Path 481 | total_timesteps 5186.
Path 482 | total_timesteps 5203.
Path 483 | total_timesteps 5224.
Path 484 | total_timesteps 5232.
Path 485 | total_timesteps 5243.
Path 486 | total_timesteps 5254.
Path 487 | total_timesteps 5267.
Path 488 | total_timesteps 5280.
Path 489 | total_timesteps 5288.
Path 490 | total_timesteps 5300.
Path 491 | total_timesteps 5310.
Path 492 | total_timesteps 5319.
Path 493 | total_timesteps 5328.
Path 494 | total_timesteps 5340.
Path 495 | total_timesteps 5351.
Path 496 | total_timesteps 5364.
Path 497 | total_timesteps 5373.
Path 498 | total_timesteps 5380.
Path 499 | total_timesteps 5389.
Path 500 | total_timesteps 5400.
Path 501 | total_timesteps 5429.
Path 502 | total_timesteps 5444.
Path 503 | total_timesteps 5456.
Path 504 | total_timesteps 5466.
Path 505 | total_timesteps 5475.
Path 506 | total_timesteps 5486.
Path 507 | total_timesteps 5495.
Path 508 | total_timesteps 5509.
Path 509 | total_timesteps 5515.
Path 510 | total_timesteps 5527.
Path 511 | total_timesteps 5543.
Path 512 | total_timesteps 5555.
Path 513 | total_timesteps 5564.
Path 514 | total_timesteps 5574.
Path 515 | total_timesteps 5582.
Path 516 | total_timesteps 5593.
Path 517 | total_timesteps 5603.
Path 518 | total_timesteps 5613.
Path 519 | total_timesteps 5625.
Path 520 | total_timesteps 5635.
Path 521 | total_timesteps 5647.
Path 522 | total_timesteps 5667.
Path 523 | total_timesteps 5676.
Path 524 | total_timesteps 5691.
Path 525 | total_timesteps 5698.
Path 526 | total_timesteps 5709.
Path 527 | total_timesteps 5718.
Path 528 | total_timesteps 5726.
Path 529 | total_timesteps 5738.
Path 530 | total_timesteps 5746.
Path 531 | total_timesteps 5760.
Path 532 | total_timesteps 5768.
Path 533 | total_timesteps 5782.
Path 534 | total_timesteps 5793.
Path 535 | total_timesteps 5800.
Path 536 | total_timesteps 5811.
Path 537 | total_timesteps 5821.
Path 538 | total_timesteps 5833.
Path 539 | total_timesteps 5849.
Path 540 | total_timesteps 5859.
Path 541 | total_timesteps 5872.
Path 542 | total_timesteps 5889.
Path 543 | total_timesteps 5901.
Path 544 | total_timesteps 5911.
Path 545 | total_timesteps 5918.
Path 546 | total_timesteps 5927.
Path 547 | total_timesteps 5940.
Path 548 | total_timesteps 5950.
Path 549 | total_timesteps 5968.
Path 550 | total_timesteps 5985.
Path 551 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.92    |
| Iteration     | 24       |
| MaximumReturn | 1.74     |
| MinimumReturn | -18.1    |
| TotalSamples  | 104099   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.3167794346809387
Validation loss = 0.3196341097354889
Validation loss = 0.3229810297489166
Validation loss = 0.31978586316108704
Validation loss = 0.3229261040687561
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 76.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 93.
Path 10 | total_timesteps 106.
Path 11 | total_timesteps 125.
Path 12 | total_timesteps 136.
Path 13 | total_timesteps 147.
Path 14 | total_timesteps 156.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 175.
Path 17 | total_timesteps 182.
Path 18 | total_timesteps 196.
Path 19 | total_timesteps 210.
Path 20 | total_timesteps 226.
Path 21 | total_timesteps 234.
Path 22 | total_timesteps 241.
Path 23 | total_timesteps 252.
Path 24 | total_timesteps 261.
Path 25 | total_timesteps 278.
Path 26 | total_timesteps 294.
Path 27 | total_timesteps 303.
Path 28 | total_timesteps 318.
Path 29 | total_timesteps 335.
Path 30 | total_timesteps 351.
Path 31 | total_timesteps 363.
Path 32 | total_timesteps 375.
Path 33 | total_timesteps 383.
Path 34 | total_timesteps 393.
Path 35 | total_timesteps 409.
Path 36 | total_timesteps 416.
Path 37 | total_timesteps 426.
Path 38 | total_timesteps 436.
Path 39 | total_timesteps 444.
Path 40 | total_timesteps 455.
Path 41 | total_timesteps 467.
Path 42 | total_timesteps 479.
Path 43 | total_timesteps 489.
Path 44 | total_timesteps 502.
Path 45 | total_timesteps 512.
Path 46 | total_timesteps 523.
Path 47 | total_timesteps 531.
Path 48 | total_timesteps 554.
Path 49 | total_timesteps 569.
Path 50 | total_timesteps 582.
Path 51 | total_timesteps 593.
Path 52 | total_timesteps 602.
Path 53 | total_timesteps 611.
Path 54 | total_timesteps 629.
Path 55 | total_timesteps 640.
Path 56 | total_timesteps 650.
Path 57 | total_timesteps 658.
Path 58 | total_timesteps 668.
Path 59 | total_timesteps 677.
Path 60 | total_timesteps 692.
Path 61 | total_timesteps 708.
Path 62 | total_timesteps 718.
Path 63 | total_timesteps 732.
Path 64 | total_timesteps 742.
Path 65 | total_timesteps 749.
Path 66 | total_timesteps 760.
Path 67 | total_timesteps 774.
Path 68 | total_timesteps 780.
Path 69 | total_timesteps 794.
Path 70 | total_timesteps 809.
Path 71 | total_timesteps 820.
Path 72 | total_timesteps 828.
Path 73 | total_timesteps 840.
Path 74 | total_timesteps 849.
Path 75 | total_timesteps 860.
Path 76 | total_timesteps 871.
Path 77 | total_timesteps 883.
Path 78 | total_timesteps 891.
Path 79 | total_timesteps 897.
Path 80 | total_timesteps 908.
Path 81 | total_timesteps 922.
Path 82 | total_timesteps 933.
Path 83 | total_timesteps 947.
Path 84 | total_timesteps 961.
Path 85 | total_timesteps 972.
Path 86 | total_timesteps 981.
Path 87 | total_timesteps 988.
Path 88 | total_timesteps 998.
Path 89 | total_timesteps 1007.
Path 90 | total_timesteps 1019.
Path 91 | total_timesteps 1028.
Path 92 | total_timesteps 1038.
Path 93 | total_timesteps 1054.
Path 94 | total_timesteps 1065.
Path 95 | total_timesteps 1076.
Path 96 | total_timesteps 1087.
Path 97 | total_timesteps 1098.
Path 98 | total_timesteps 1107.
Path 99 | total_timesteps 1115.
Path 100 | total_timesteps 1124.
Path 101 | total_timesteps 1134.
Path 102 | total_timesteps 1147.
Path 103 | total_timesteps 1157.
Path 104 | total_timesteps 1169.
Path 105 | total_timesteps 1182.
Path 106 | total_timesteps 1193.
Path 107 | total_timesteps 1201.
Path 108 | total_timesteps 1214.
Path 109 | total_timesteps 1225.
Path 110 | total_timesteps 1240.
Path 111 | total_timesteps 1249.
Path 112 | total_timesteps 1262.
Path 113 | total_timesteps 1270.
Path 114 | total_timesteps 1281.
Path 115 | total_timesteps 1294.
Path 116 | total_timesteps 1302.
Path 117 | total_timesteps 1312.
Path 118 | total_timesteps 1320.
Path 119 | total_timesteps 1330.
Path 120 | total_timesteps 1347.
Path 121 | total_timesteps 1358.
Path 122 | total_timesteps 1367.
Path 123 | total_timesteps 1376.
Path 124 | total_timesteps 1385.
Path 125 | total_timesteps 1396.
Path 126 | total_timesteps 1404.
Path 127 | total_timesteps 1414.
Path 128 | total_timesteps 1426.
Path 129 | total_timesteps 1434.
Path 130 | total_timesteps 1443.
Path 131 | total_timesteps 1455.
Path 132 | total_timesteps 1465.
Path 133 | total_timesteps 1472.
Path 134 | total_timesteps 1482.
Path 135 | total_timesteps 1493.
Path 136 | total_timesteps 1506.
Path 137 | total_timesteps 1521.
Path 138 | total_timesteps 1529.
Path 139 | total_timesteps 1539.
Path 140 | total_timesteps 1553.
Path 141 | total_timesteps 1566.
Path 142 | total_timesteps 1575.
Path 143 | total_timesteps 1591.
Path 144 | total_timesteps 1597.
Path 145 | total_timesteps 1606.
Path 146 | total_timesteps 1625.
Path 147 | total_timesteps 1643.
Path 148 | total_timesteps 1660.
Path 149 | total_timesteps 1669.
Path 150 | total_timesteps 1680.
Path 151 | total_timesteps 1692.
Path 152 | total_timesteps 1699.
Path 153 | total_timesteps 1705.
Path 154 | total_timesteps 1713.
Path 155 | total_timesteps 1720.
Path 156 | total_timesteps 1728.
Path 157 | total_timesteps 1736.
Path 158 | total_timesteps 1742.
Path 159 | total_timesteps 1751.
Path 160 | total_timesteps 1760.
Path 161 | total_timesteps 1772.
Path 162 | total_timesteps 1783.
Path 163 | total_timesteps 1793.
Path 164 | total_timesteps 1803.
Path 165 | total_timesteps 1815.
Path 166 | total_timesteps 1825.
Path 167 | total_timesteps 1837.
Path 168 | total_timesteps 1847.
Path 169 | total_timesteps 1857.
Path 170 | total_timesteps 1870.
Path 171 | total_timesteps 1879.
Path 172 | total_timesteps 1889.
Path 173 | total_timesteps 1898.
Path 174 | total_timesteps 1905.
Path 175 | total_timesteps 1915.
Path 176 | total_timesteps 1924.
Path 177 | total_timesteps 1938.
Path 178 | total_timesteps 1951.
Path 179 | total_timesteps 1963.
Path 180 | total_timesteps 1974.
Path 181 | total_timesteps 1984.
Path 182 | total_timesteps 1993.
Path 183 | total_timesteps 2003.
Path 184 | total_timesteps 2014.
Path 185 | total_timesteps 2031.
Path 186 | total_timesteps 2042.
Path 187 | total_timesteps 2050.
Path 188 | total_timesteps 2061.
Path 189 | total_timesteps 2075.
Path 190 | total_timesteps 2085.
Path 191 | total_timesteps 2094.
Path 192 | total_timesteps 2102.
Path 193 | total_timesteps 2111.
Path 194 | total_timesteps 2123.
Path 195 | total_timesteps 2132.
Path 196 | total_timesteps 2141.
Path 197 | total_timesteps 2152.
Path 198 | total_timesteps 2163.
Path 199 | total_timesteps 2171.
Path 200 | total_timesteps 2181.
Path 201 | total_timesteps 2190.
Path 202 | total_timesteps 2200.
Path 203 | total_timesteps 2212.
Path 204 | total_timesteps 2225.
Path 205 | total_timesteps 2233.
Path 206 | total_timesteps 2242.
Path 207 | total_timesteps 2250.
Path 208 | total_timesteps 2262.
Path 209 | total_timesteps 2271.
Path 210 | total_timesteps 2283.
Path 211 | total_timesteps 2292.
Path 212 | total_timesteps 2299.
Path 213 | total_timesteps 2310.
Path 214 | total_timesteps 2325.
Path 215 | total_timesteps 2334.
Path 216 | total_timesteps 2343.
Path 217 | total_timesteps 2360.
Path 218 | total_timesteps 2372.
Path 219 | total_timesteps 2389.
Path 220 | total_timesteps 2403.
Path 221 | total_timesteps 2410.
Path 222 | total_timesteps 2421.
Path 223 | total_timesteps 2431.
Path 224 | total_timesteps 2440.
Path 225 | total_timesteps 2452.
Path 226 | total_timesteps 2461.
Path 227 | total_timesteps 2469.
Path 228 | total_timesteps 2484.
Path 229 | total_timesteps 2493.
Path 230 | total_timesteps 2501.
Path 231 | total_timesteps 2517.
Path 232 | total_timesteps 2528.
Path 233 | total_timesteps 2550.
Path 234 | total_timesteps 2558.
Path 235 | total_timesteps 2567.
Path 236 | total_timesteps 2574.
Path 237 | total_timesteps 2584.
Path 238 | total_timesteps 2595.
Path 239 | total_timesteps 2604.
Path 240 | total_timesteps 2615.
Path 241 | total_timesteps 2624.
Path 242 | total_timesteps 2632.
Path 243 | total_timesteps 2640.
Path 244 | total_timesteps 2649.
Path 245 | total_timesteps 2656.
Path 246 | total_timesteps 2674.
Path 247 | total_timesteps 2682.
Path 248 | total_timesteps 2692.
Path 249 | total_timesteps 2701.
Path 250 | total_timesteps 2712.
Path 251 | total_timesteps 2724.
Path 252 | total_timesteps 2739.
Path 253 | total_timesteps 2748.
Path 254 | total_timesteps 2759.
Path 255 | total_timesteps 2767.
Path 256 | total_timesteps 2777.
Path 257 | total_timesteps 2788.
Path 258 | total_timesteps 2802.
Path 259 | total_timesteps 2810.
Path 260 | total_timesteps 2822.
Path 261 | total_timesteps 2831.
Path 262 | total_timesteps 2841.
Path 263 | total_timesteps 2852.
Path 264 | total_timesteps 2859.
Path 265 | total_timesteps 2869.
Path 266 | total_timesteps 2876.
Path 267 | total_timesteps 2885.
Path 268 | total_timesteps 2896.
Path 269 | total_timesteps 2908.
Path 270 | total_timesteps 2916.
Path 271 | total_timesteps 2927.
Path 272 | total_timesteps 2939.
Path 273 | total_timesteps 2949.
Path 274 | total_timesteps 2962.
Path 275 | total_timesteps 2973.
Path 276 | total_timesteps 2981.
Path 277 | total_timesteps 2996.
Path 278 | total_timesteps 3009.
Path 279 | total_timesteps 3027.
Path 280 | total_timesteps 3034.
Path 281 | total_timesteps 3042.
Path 282 | total_timesteps 3049.
Path 283 | total_timesteps 3060.
Path 284 | total_timesteps 3068.
Path 285 | total_timesteps 3075.
Path 286 | total_timesteps 3084.
Path 287 | total_timesteps 3093.
Path 288 | total_timesteps 3101.
Path 289 | total_timesteps 3111.
Path 290 | total_timesteps 3125.
Path 291 | total_timesteps 3137.
Path 292 | total_timesteps 3145.
Path 293 | total_timesteps 3155.
Path 294 | total_timesteps 3162.
Path 295 | total_timesteps 3170.
Path 296 | total_timesteps 3181.
Path 297 | total_timesteps 3192.
Path 298 | total_timesteps 3198.
Path 299 | total_timesteps 3208.
Path 300 | total_timesteps 3218.
Path 301 | total_timesteps 3228.
Path 302 | total_timesteps 3239.
Path 303 | total_timesteps 3247.
Path 304 | total_timesteps 3258.
Path 305 | total_timesteps 3267.
Path 306 | total_timesteps 3276.
Path 307 | total_timesteps 3286.
Path 308 | total_timesteps 3293.
Path 309 | total_timesteps 3301.
Path 310 | total_timesteps 3315.
Path 311 | total_timesteps 3327.
Path 312 | total_timesteps 3341.
Path 313 | total_timesteps 3355.
Path 314 | total_timesteps 3362.
Path 315 | total_timesteps 3373.
Path 316 | total_timesteps 3383.
Path 317 | total_timesteps 3394.
Path 318 | total_timesteps 3403.
Path 319 | total_timesteps 3413.
Path 320 | total_timesteps 3423.
Path 321 | total_timesteps 3434.
Path 322 | total_timesteps 3443.
Path 323 | total_timesteps 3465.
Path 324 | total_timesteps 3474.
Path 325 | total_timesteps 3482.
Path 326 | total_timesteps 3494.
Path 327 | total_timesteps 3505.
Path 328 | total_timesteps 3513.
Path 329 | total_timesteps 3520.
Path 330 | total_timesteps 3528.
Path 331 | total_timesteps 3538.
Path 332 | total_timesteps 3547.
Path 333 | total_timesteps 3556.
Path 334 | total_timesteps 3567.
Path 335 | total_timesteps 3577.
Path 336 | total_timesteps 3586.
Path 337 | total_timesteps 3594.
Path 338 | total_timesteps 3602.
Path 339 | total_timesteps 3616.
Path 340 | total_timesteps 3627.
Path 341 | total_timesteps 3638.
Path 342 | total_timesteps 3647.
Path 343 | total_timesteps 3655.
Path 344 | total_timesteps 3663.
Path 345 | total_timesteps 3672.
Path 346 | total_timesteps 3680.
Path 347 | total_timesteps 3692.
Path 348 | total_timesteps 3702.
Path 349 | total_timesteps 3711.
Path 350 | total_timesteps 3724.
Path 351 | total_timesteps 3733.
Path 352 | total_timesteps 3744.
Path 353 | total_timesteps 3753.
Path 354 | total_timesteps 3763.
Path 355 | total_timesteps 3773.
Path 356 | total_timesteps 3780.
Path 357 | total_timesteps 3789.
Path 358 | total_timesteps 3795.
Path 359 | total_timesteps 3810.
Path 360 | total_timesteps 3822.
Path 361 | total_timesteps 3836.
Path 362 | total_timesteps 3848.
Path 363 | total_timesteps 3860.
Path 364 | total_timesteps 3868.
Path 365 | total_timesteps 3878.
Path 366 | total_timesteps 3895.
Path 367 | total_timesteps 3905.
Path 368 | total_timesteps 3916.
Path 369 | total_timesteps 3926.
Path 370 | total_timesteps 3936.
Path 371 | total_timesteps 3961.
Path 372 | total_timesteps 3971.
Path 373 | total_timesteps 3980.
Path 374 | total_timesteps 3989.
Path 375 | total_timesteps 4000.
Path 376 | total_timesteps 4010.
Path 377 | total_timesteps 4019.
Path 378 | total_timesteps 4030.
Path 379 | total_timesteps 4041.
Path 380 | total_timesteps 4063.
Path 381 | total_timesteps 4070.
Path 382 | total_timesteps 4082.
Path 383 | total_timesteps 4088.
Path 384 | total_timesteps 4102.
Path 385 | total_timesteps 4116.
Path 386 | total_timesteps 4128.
Path 387 | total_timesteps 4135.
Path 388 | total_timesteps 4145.
Path 389 | total_timesteps 4154.
Path 390 | total_timesteps 4165.
Path 391 | total_timesteps 4181.
Path 392 | total_timesteps 4191.
Path 393 | total_timesteps 4201.
Path 394 | total_timesteps 4209.
Path 395 | total_timesteps 4219.
Path 396 | total_timesteps 4233.
Path 397 | total_timesteps 4242.
Path 398 | total_timesteps 4256.
Path 399 | total_timesteps 4270.
Path 400 | total_timesteps 4278.
Path 401 | total_timesteps 4289.
Path 402 | total_timesteps 4296.
Path 403 | total_timesteps 4306.
Path 404 | total_timesteps 4314.
Path 405 | total_timesteps 4326.
Path 406 | total_timesteps 4340.
Path 407 | total_timesteps 4351.
Path 408 | total_timesteps 4359.
Path 409 | total_timesteps 4368.
Path 410 | total_timesteps 4377.
Path 411 | total_timesteps 4388.
Path 412 | total_timesteps 4400.
Path 413 | total_timesteps 4408.
Path 414 | total_timesteps 4423.
Path 415 | total_timesteps 4432.
Path 416 | total_timesteps 4446.
Path 417 | total_timesteps 4455.
Path 418 | total_timesteps 4466.
Path 419 | total_timesteps 4481.
Path 420 | total_timesteps 4497.
Path 421 | total_timesteps 4511.
Path 422 | total_timesteps 4519.
Path 423 | total_timesteps 4537.
Path 424 | total_timesteps 4549.
Path 425 | total_timesteps 4558.
Path 426 | total_timesteps 4566.
Path 427 | total_timesteps 4574.
Path 428 | total_timesteps 4585.
Path 429 | total_timesteps 4593.
Path 430 | total_timesteps 4611.
Path 431 | total_timesteps 4619.
Path 432 | total_timesteps 4629.
Path 433 | total_timesteps 4637.
Path 434 | total_timesteps 4649.
Path 435 | total_timesteps 4659.
Path 436 | total_timesteps 4677.
Path 437 | total_timesteps 4699.
Path 438 | total_timesteps 4713.
Path 439 | total_timesteps 4721.
Path 440 | total_timesteps 4729.
Path 441 | total_timesteps 4738.
Path 442 | total_timesteps 4761.
Path 443 | total_timesteps 4773.
Path 444 | total_timesteps 4785.
Path 445 | total_timesteps 4797.
Path 446 | total_timesteps 4812.
Path 447 | total_timesteps 4827.
Path 448 | total_timesteps 4837.
Path 449 | total_timesteps 4849.
Path 450 | total_timesteps 4859.
Path 451 | total_timesteps 4868.
Path 452 | total_timesteps 4876.
Path 453 | total_timesteps 4887.
Path 454 | total_timesteps 4898.
Path 455 | total_timesteps 4906.
Path 456 | total_timesteps 4916.
Path 457 | total_timesteps 4926.
Path 458 | total_timesteps 4943.
Path 459 | total_timesteps 4955.
Path 460 | total_timesteps 4967.
Path 461 | total_timesteps 4976.
Path 462 | total_timesteps 4986.
Path 463 | total_timesteps 4995.
Path 464 | total_timesteps 5003.
Path 465 | total_timesteps 5013.
Path 466 | total_timesteps 5023.
Path 467 | total_timesteps 5034.
Path 468 | total_timesteps 5045.
Path 469 | total_timesteps 5053.
Path 470 | total_timesteps 5066.
Path 471 | total_timesteps 5072.
Path 472 | total_timesteps 5080.
Path 473 | total_timesteps 5089.
Path 474 | total_timesteps 5096.
Path 475 | total_timesteps 5104.
Path 476 | total_timesteps 5113.
Path 477 | total_timesteps 5122.
Path 478 | total_timesteps 5131.
Path 479 | total_timesteps 5145.
Path 480 | total_timesteps 5153.
Path 481 | total_timesteps 5163.
Path 482 | total_timesteps 5172.
Path 483 | total_timesteps 5189.
Path 484 | total_timesteps 5199.
Path 485 | total_timesteps 5208.
Path 486 | total_timesteps 5215.
Path 487 | total_timesteps 5226.
Path 488 | total_timesteps 5238.
Path 489 | total_timesteps 5249.
Path 490 | total_timesteps 5259.
Path 491 | total_timesteps 5269.
Path 492 | total_timesteps 5275.
Path 493 | total_timesteps 5287.
Path 494 | total_timesteps 5305.
Path 495 | total_timesteps 5313.
Path 496 | total_timesteps 5323.
Path 497 | total_timesteps 5332.
Path 498 | total_timesteps 5341.
Path 499 | total_timesteps 5352.
Path 500 | total_timesteps 5359.
Path 501 | total_timesteps 5370.
Path 502 | total_timesteps 5380.
Path 503 | total_timesteps 5393.
Path 504 | total_timesteps 5402.
Path 505 | total_timesteps 5410.
Path 506 | total_timesteps 5419.
Path 507 | total_timesteps 5430.
Path 508 | total_timesteps 5444.
Path 509 | total_timesteps 5450.
Path 510 | total_timesteps 5462.
Path 511 | total_timesteps 5470.
Path 512 | total_timesteps 5483.
Path 513 | total_timesteps 5499.
Path 514 | total_timesteps 5509.
Path 515 | total_timesteps 5523.
Path 516 | total_timesteps 5530.
Path 517 | total_timesteps 5538.
Path 518 | total_timesteps 5547.
Path 519 | total_timesteps 5556.
Path 520 | total_timesteps 5569.
Path 521 | total_timesteps 5581.
Path 522 | total_timesteps 5589.
Path 523 | total_timesteps 5600.
Path 524 | total_timesteps 5608.
Path 525 | total_timesteps 5625.
Path 526 | total_timesteps 5636.
Path 527 | total_timesteps 5644.
Path 528 | total_timesteps 5661.
Path 529 | total_timesteps 5671.
Path 530 | total_timesteps 5679.
Path 531 | total_timesteps 5689.
Path 532 | total_timesteps 5704.
Path 533 | total_timesteps 5712.
Path 534 | total_timesteps 5719.
Path 535 | total_timesteps 5728.
Path 536 | total_timesteps 5741.
Path 537 | total_timesteps 5756.
Path 538 | total_timesteps 5767.
Path 539 | total_timesteps 5776.
Path 540 | total_timesteps 5788.
Path 541 | total_timesteps 5798.
Path 542 | total_timesteps 5811.
Path 543 | total_timesteps 5820.
Path 544 | total_timesteps 5835.
Path 545 | total_timesteps 5841.
Path 546 | total_timesteps 5852.
Path 547 | total_timesteps 5861.
Path 548 | total_timesteps 5868.
Path 549 | total_timesteps 5875.
Path 550 | total_timesteps 5883.
Path 551 | total_timesteps 5892.
Path 552 | total_timesteps 5901.
Path 553 | total_timesteps 5909.
Path 554 | total_timesteps 5918.
Path 555 | total_timesteps 5928.
Path 556 | total_timesteps 5940.
Path 557 | total_timesteps 5950.
Path 558 | total_timesteps 5965.
Path 559 | total_timesteps 5976.
Path 560 | total_timesteps 5985.
Path 561 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.15    |
| Iteration     | 25       |
| MaximumReturn | 1.14     |
| MinimumReturn | -23      |
| TotalSamples  | 108102   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.31561416387557983
Validation loss = 0.3183629810810089
Validation loss = 0.32088130712509155
Validation loss = 0.32197004556655884
Validation loss = 0.3204510509967804
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 47.
Path 6 | total_timesteps 68.
Path 7 | total_timesteps 77.
Path 8 | total_timesteps 85.
Path 9 | total_timesteps 98.
Path 10 | total_timesteps 109.
Path 11 | total_timesteps 120.
Path 12 | total_timesteps 129.
Path 13 | total_timesteps 138.
Path 14 | total_timesteps 157.
Path 15 | total_timesteps 165.
Path 16 | total_timesteps 173.
Path 17 | total_timesteps 186.
Path 18 | total_timesteps 194.
Path 19 | total_timesteps 207.
Path 20 | total_timesteps 218.
Path 21 | total_timesteps 228.
Path 22 | total_timesteps 239.
Path 23 | total_timesteps 248.
Path 24 | total_timesteps 255.
Path 25 | total_timesteps 265.
Path 26 | total_timesteps 274.
Path 27 | total_timesteps 284.
Path 28 | total_timesteps 295.
Path 29 | total_timesteps 305.
Path 30 | total_timesteps 315.
Path 31 | total_timesteps 325.
Path 32 | total_timesteps 339.
Path 33 | total_timesteps 348.
Path 34 | total_timesteps 359.
Path 35 | total_timesteps 370.
Path 36 | total_timesteps 378.
Path 37 | total_timesteps 390.
Path 38 | total_timesteps 398.
Path 39 | total_timesteps 406.
Path 40 | total_timesteps 415.
Path 41 | total_timesteps 423.
Path 42 | total_timesteps 433.
Path 43 | total_timesteps 446.
Path 44 | total_timesteps 456.
Path 45 | total_timesteps 468.
Path 46 | total_timesteps 476.
Path 47 | total_timesteps 486.
Path 48 | total_timesteps 509.
Path 49 | total_timesteps 523.
Path 50 | total_timesteps 532.
Path 51 | total_timesteps 545.
Path 52 | total_timesteps 558.
Path 53 | total_timesteps 565.
Path 54 | total_timesteps 572.
Path 55 | total_timesteps 580.
Path 56 | total_timesteps 596.
Path 57 | total_timesteps 608.
Path 58 | total_timesteps 618.
Path 59 | total_timesteps 631.
Path 60 | total_timesteps 640.
Path 61 | total_timesteps 650.
Path 62 | total_timesteps 662.
Path 63 | total_timesteps 682.
Path 64 | total_timesteps 696.
Path 65 | total_timesteps 706.
Path 66 | total_timesteps 717.
Path 67 | total_timesteps 728.
Path 68 | total_timesteps 738.
Path 69 | total_timesteps 757.
Path 70 | total_timesteps 767.
Path 71 | total_timesteps 775.
Path 72 | total_timesteps 784.
Path 73 | total_timesteps 793.
Path 74 | total_timesteps 804.
Path 75 | total_timesteps 819.
Path 76 | total_timesteps 829.
Path 77 | total_timesteps 839.
Path 78 | total_timesteps 848.
Path 79 | total_timesteps 858.
Path 80 | total_timesteps 869.
Path 81 | total_timesteps 877.
Path 82 | total_timesteps 888.
Path 83 | total_timesteps 895.
Path 84 | total_timesteps 908.
Path 85 | total_timesteps 918.
Path 86 | total_timesteps 926.
Path 87 | total_timesteps 936.
Path 88 | total_timesteps 950.
Path 89 | total_timesteps 961.
Path 90 | total_timesteps 973.
Path 91 | total_timesteps 981.
Path 92 | total_timesteps 989.
Path 93 | total_timesteps 1001.
Path 94 | total_timesteps 1015.
Path 95 | total_timesteps 1028.
Path 96 | total_timesteps 1037.
Path 97 | total_timesteps 1049.
Path 98 | total_timesteps 1060.
Path 99 | total_timesteps 1068.
Path 100 | total_timesteps 1083.
Path 101 | total_timesteps 1092.
Path 102 | total_timesteps 1104.
Path 103 | total_timesteps 1113.
Path 104 | total_timesteps 1126.
Path 105 | total_timesteps 1137.
Path 106 | total_timesteps 1148.
Path 107 | total_timesteps 1163.
Path 108 | total_timesteps 1172.
Path 109 | total_timesteps 1182.
Path 110 | total_timesteps 1194.
Path 111 | total_timesteps 1203.
Path 112 | total_timesteps 1213.
Path 113 | total_timesteps 1220.
Path 114 | total_timesteps 1232.
Path 115 | total_timesteps 1239.
Path 116 | total_timesteps 1247.
Path 117 | total_timesteps 1257.
Path 118 | total_timesteps 1267.
Path 119 | total_timesteps 1275.
Path 120 | total_timesteps 1285.
Path 121 | total_timesteps 1295.
Path 122 | total_timesteps 1304.
Path 123 | total_timesteps 1314.
Path 124 | total_timesteps 1333.
Path 125 | total_timesteps 1346.
Path 126 | total_timesteps 1356.
Path 127 | total_timesteps 1364.
Path 128 | total_timesteps 1376.
Path 129 | total_timesteps 1392.
Path 130 | total_timesteps 1402.
Path 131 | total_timesteps 1411.
Path 132 | total_timesteps 1421.
Path 133 | total_timesteps 1431.
Path 134 | total_timesteps 1444.
Path 135 | total_timesteps 1453.
Path 136 | total_timesteps 1461.
Path 137 | total_timesteps 1471.
Path 138 | total_timesteps 1483.
Path 139 | total_timesteps 1497.
Path 140 | total_timesteps 1507.
Path 141 | total_timesteps 1521.
Path 142 | total_timesteps 1531.
Path 143 | total_timesteps 1543.
Path 144 | total_timesteps 1553.
Path 145 | total_timesteps 1562.
Path 146 | total_timesteps 1573.
Path 147 | total_timesteps 1585.
Path 148 | total_timesteps 1592.
Path 149 | total_timesteps 1602.
Path 150 | total_timesteps 1612.
Path 151 | total_timesteps 1629.
Path 152 | total_timesteps 1639.
Path 153 | total_timesteps 1652.
Path 154 | total_timesteps 1659.
Path 155 | total_timesteps 1670.
Path 156 | total_timesteps 1681.
Path 157 | total_timesteps 1692.
Path 158 | total_timesteps 1708.
Path 159 | total_timesteps 1719.
Path 160 | total_timesteps 1735.
Path 161 | total_timesteps 1744.
Path 162 | total_timesteps 1752.
Path 163 | total_timesteps 1763.
Path 164 | total_timesteps 1772.
Path 165 | total_timesteps 1781.
Path 166 | total_timesteps 1793.
Path 167 | total_timesteps 1805.
Path 168 | total_timesteps 1816.
Path 169 | total_timesteps 1830.
Path 170 | total_timesteps 1840.
Path 171 | total_timesteps 1848.
Path 172 | total_timesteps 1855.
Path 173 | total_timesteps 1871.
Path 174 | total_timesteps 1885.
Path 175 | total_timesteps 1893.
Path 176 | total_timesteps 1906.
Path 177 | total_timesteps 1918.
Path 178 | total_timesteps 1929.
Path 179 | total_timesteps 1942.
Path 180 | total_timesteps 1954.
Path 181 | total_timesteps 1963.
Path 182 | total_timesteps 1971.
Path 183 | total_timesteps 1981.
Path 184 | total_timesteps 1999.
Path 185 | total_timesteps 2007.
Path 186 | total_timesteps 2014.
Path 187 | total_timesteps 2027.
Path 188 | total_timesteps 2036.
Path 189 | total_timesteps 2042.
Path 190 | total_timesteps 2055.
Path 191 | total_timesteps 2063.
Path 192 | total_timesteps 2076.
Path 193 | total_timesteps 2083.
Path 194 | total_timesteps 2105.
Path 195 | total_timesteps 2112.
Path 196 | total_timesteps 2125.
Path 197 | total_timesteps 2134.
Path 198 | total_timesteps 2148.
Path 199 | total_timesteps 2156.
Path 200 | total_timesteps 2168.
Path 201 | total_timesteps 2178.
Path 202 | total_timesteps 2189.
Path 203 | total_timesteps 2196.
Path 204 | total_timesteps 2207.
Path 205 | total_timesteps 2216.
Path 206 | total_timesteps 2224.
Path 207 | total_timesteps 2235.
Path 208 | total_timesteps 2244.
Path 209 | total_timesteps 2251.
Path 210 | total_timesteps 2259.
Path 211 | total_timesteps 2268.
Path 212 | total_timesteps 2283.
Path 213 | total_timesteps 2291.
Path 214 | total_timesteps 2302.
Path 215 | total_timesteps 2310.
Path 216 | total_timesteps 2321.
Path 217 | total_timesteps 2332.
Path 218 | total_timesteps 2343.
Path 219 | total_timesteps 2355.
Path 220 | total_timesteps 2365.
Path 221 | total_timesteps 2373.
Path 222 | total_timesteps 2391.
Path 223 | total_timesteps 2399.
Path 224 | total_timesteps 2407.
Path 225 | total_timesteps 2416.
Path 226 | total_timesteps 2423.
Path 227 | total_timesteps 2432.
Path 228 | total_timesteps 2449.
Path 229 | total_timesteps 2464.
Path 230 | total_timesteps 2472.
Path 231 | total_timesteps 2484.
Path 232 | total_timesteps 2496.
Path 233 | total_timesteps 2509.
Path 234 | total_timesteps 2522.
Path 235 | total_timesteps 2531.
Path 236 | total_timesteps 2541.
Path 237 | total_timesteps 2552.
Path 238 | total_timesteps 2562.
Path 239 | total_timesteps 2569.
Path 240 | total_timesteps 2578.
Path 241 | total_timesteps 2592.
Path 242 | total_timesteps 2601.
Path 243 | total_timesteps 2614.
Path 244 | total_timesteps 2624.
Path 245 | total_timesteps 2634.
Path 246 | total_timesteps 2649.
Path 247 | total_timesteps 2657.
Path 248 | total_timesteps 2667.
Path 249 | total_timesteps 2677.
Path 250 | total_timesteps 2686.
Path 251 | total_timesteps 2702.
Path 252 | total_timesteps 2711.
Path 253 | total_timesteps 2719.
Path 254 | total_timesteps 2726.
Path 255 | total_timesteps 2740.
Path 256 | total_timesteps 2754.
Path 257 | total_timesteps 2765.
Path 258 | total_timesteps 2775.
Path 259 | total_timesteps 2783.
Path 260 | total_timesteps 2792.
Path 261 | total_timesteps 2802.
Path 262 | total_timesteps 2818.
Path 263 | total_timesteps 2826.
Path 264 | total_timesteps 2837.
Path 265 | total_timesteps 2848.
Path 266 | total_timesteps 2863.
Path 267 | total_timesteps 2871.
Path 268 | total_timesteps 2884.
Path 269 | total_timesteps 2897.
Path 270 | total_timesteps 2906.
Path 271 | total_timesteps 2918.
Path 272 | total_timesteps 2928.
Path 273 | total_timesteps 2948.
Path 274 | total_timesteps 2958.
Path 275 | total_timesteps 2966.
Path 276 | total_timesteps 2977.
Path 277 | total_timesteps 2986.
Path 278 | total_timesteps 2997.
Path 279 | total_timesteps 3006.
Path 280 | total_timesteps 3017.
Path 281 | total_timesteps 3039.
Path 282 | total_timesteps 3050.
Path 283 | total_timesteps 3059.
Path 284 | total_timesteps 3066.
Path 285 | total_timesteps 3073.
Path 286 | total_timesteps 3082.
Path 287 | total_timesteps 3094.
Path 288 | total_timesteps 3102.
Path 289 | total_timesteps 3113.
Path 290 | total_timesteps 3119.
Path 291 | total_timesteps 3127.
Path 292 | total_timesteps 3138.
Path 293 | total_timesteps 3155.
Path 294 | total_timesteps 3170.
Path 295 | total_timesteps 3186.
Path 296 | total_timesteps 3200.
Path 297 | total_timesteps 3210.
Path 298 | total_timesteps 3224.
Path 299 | total_timesteps 3239.
Path 300 | total_timesteps 3253.
Path 301 | total_timesteps 3260.
Path 302 | total_timesteps 3272.
Path 303 | total_timesteps 3278.
Path 304 | total_timesteps 3291.
Path 305 | total_timesteps 3305.
Path 306 | total_timesteps 3314.
Path 307 | total_timesteps 3338.
Path 308 | total_timesteps 3346.
Path 309 | total_timesteps 3356.
Path 310 | total_timesteps 3366.
Path 311 | total_timesteps 3376.
Path 312 | total_timesteps 3394.
Path 313 | total_timesteps 3402.
Path 314 | total_timesteps 3415.
Path 315 | total_timesteps 3424.
Path 316 | total_timesteps 3435.
Path 317 | total_timesteps 3456.
Path 318 | total_timesteps 3465.
Path 319 | total_timesteps 3476.
Path 320 | total_timesteps 3484.
Path 321 | total_timesteps 3495.
Path 322 | total_timesteps 3504.
Path 323 | total_timesteps 3517.
Path 324 | total_timesteps 3528.
Path 325 | total_timesteps 3538.
Path 326 | total_timesteps 3548.
Path 327 | total_timesteps 3555.
Path 328 | total_timesteps 3569.
Path 329 | total_timesteps 3582.
Path 330 | total_timesteps 3597.
Path 331 | total_timesteps 3613.
Path 332 | total_timesteps 3622.
Path 333 | total_timesteps 3632.
Path 334 | total_timesteps 3644.
Path 335 | total_timesteps 3651.
Path 336 | total_timesteps 3662.
Path 337 | total_timesteps 3669.
Path 338 | total_timesteps 3679.
Path 339 | total_timesteps 3693.
Path 340 | total_timesteps 3704.
Path 341 | total_timesteps 3719.
Path 342 | total_timesteps 3727.
Path 343 | total_timesteps 3737.
Path 344 | total_timesteps 3745.
Path 345 | total_timesteps 3762.
Path 346 | total_timesteps 3773.
Path 347 | total_timesteps 3782.
Path 348 | total_timesteps 3789.
Path 349 | total_timesteps 3800.
Path 350 | total_timesteps 3809.
Path 351 | total_timesteps 3822.
Path 352 | total_timesteps 3831.
Path 353 | total_timesteps 3838.
Path 354 | total_timesteps 3845.
Path 355 | total_timesteps 3854.
Path 356 | total_timesteps 3867.
Path 357 | total_timesteps 3879.
Path 358 | total_timesteps 3890.
Path 359 | total_timesteps 3899.
Path 360 | total_timesteps 3906.
Path 361 | total_timesteps 3918.
Path 362 | total_timesteps 3925.
Path 363 | total_timesteps 3940.
Path 364 | total_timesteps 3948.
Path 365 | total_timesteps 3959.
Path 366 | total_timesteps 3967.
Path 367 | total_timesteps 3977.
Path 368 | total_timesteps 3987.
Path 369 | total_timesteps 3996.
Path 370 | total_timesteps 4009.
Path 371 | total_timesteps 4018.
Path 372 | total_timesteps 4025.
Path 373 | total_timesteps 4033.
Path 374 | total_timesteps 4045.
Path 375 | total_timesteps 4055.
Path 376 | total_timesteps 4065.
Path 377 | total_timesteps 4075.
Path 378 | total_timesteps 4085.
Path 379 | total_timesteps 4093.
Path 380 | total_timesteps 4105.
Path 381 | total_timesteps 4112.
Path 382 | total_timesteps 4121.
Path 383 | total_timesteps 4130.
Path 384 | total_timesteps 4140.
Path 385 | total_timesteps 4149.
Path 386 | total_timesteps 4159.
Path 387 | total_timesteps 4176.
Path 388 | total_timesteps 4186.
Path 389 | total_timesteps 4197.
Path 390 | total_timesteps 4205.
Path 391 | total_timesteps 4216.
Path 392 | total_timesteps 4225.
Path 393 | total_timesteps 4235.
Path 394 | total_timesteps 4248.
Path 395 | total_timesteps 4257.
Path 396 | total_timesteps 4266.
Path 397 | total_timesteps 4275.
Path 398 | total_timesteps 4285.
Path 399 | total_timesteps 4293.
Path 400 | total_timesteps 4302.
Path 401 | total_timesteps 4311.
Path 402 | total_timesteps 4320.
Path 403 | total_timesteps 4329.
Path 404 | total_timesteps 4338.
Path 405 | total_timesteps 4346.
Path 406 | total_timesteps 4355.
Path 407 | total_timesteps 4363.
Path 408 | total_timesteps 4374.
Path 409 | total_timesteps 4387.
Path 410 | total_timesteps 4399.
Path 411 | total_timesteps 4409.
Path 412 | total_timesteps 4421.
Path 413 | total_timesteps 4431.
Path 414 | total_timesteps 4439.
Path 415 | total_timesteps 4446.
Path 416 | total_timesteps 4457.
Path 417 | total_timesteps 4467.
Path 418 | total_timesteps 4476.
Path 419 | total_timesteps 4482.
Path 420 | total_timesteps 4492.
Path 421 | total_timesteps 4502.
Path 422 | total_timesteps 4512.
Path 423 | total_timesteps 4522.
Path 424 | total_timesteps 4533.
Path 425 | total_timesteps 4543.
Path 426 | total_timesteps 4553.
Path 427 | total_timesteps 4564.
Path 428 | total_timesteps 4578.
Path 429 | total_timesteps 4590.
Path 430 | total_timesteps 4598.
Path 431 | total_timesteps 4607.
Path 432 | total_timesteps 4623.
Path 433 | total_timesteps 4638.
Path 434 | total_timesteps 4646.
Path 435 | total_timesteps 4655.
Path 436 | total_timesteps 4667.
Path 437 | total_timesteps 4678.
Path 438 | total_timesteps 4685.
Path 439 | total_timesteps 4697.
Path 440 | total_timesteps 4708.
Path 441 | total_timesteps 4717.
Path 442 | total_timesteps 4727.
Path 443 | total_timesteps 4735.
Path 444 | total_timesteps 4744.
Path 445 | total_timesteps 4752.
Path 446 | total_timesteps 4769.
Path 447 | total_timesteps 4779.
Path 448 | total_timesteps 4792.
Path 449 | total_timesteps 4798.
Path 450 | total_timesteps 4810.
Path 451 | total_timesteps 4818.
Path 452 | total_timesteps 4829.
Path 453 | total_timesteps 4836.
Path 454 | total_timesteps 4848.
Path 455 | total_timesteps 4858.
Path 456 | total_timesteps 4870.
Path 457 | total_timesteps 4878.
Path 458 | total_timesteps 4889.
Path 459 | total_timesteps 4898.
Path 460 | total_timesteps 4908.
Path 461 | total_timesteps 4924.
Path 462 | total_timesteps 4935.
Path 463 | total_timesteps 4953.
Path 464 | total_timesteps 4961.
Path 465 | total_timesteps 4976.
Path 466 | total_timesteps 4990.
Path 467 | total_timesteps 4998.
Path 468 | total_timesteps 5011.
Path 469 | total_timesteps 5021.
Path 470 | total_timesteps 5034.
Path 471 | total_timesteps 5053.
Path 472 | total_timesteps 5064.
Path 473 | total_timesteps 5072.
Path 474 | total_timesteps 5081.
Path 475 | total_timesteps 5092.
Path 476 | total_timesteps 5101.
Path 477 | total_timesteps 5109.
Path 478 | total_timesteps 5118.
Path 479 | total_timesteps 5129.
Path 480 | total_timesteps 5137.
Path 481 | total_timesteps 5147.
Path 482 | total_timesteps 5155.
Path 483 | total_timesteps 5168.
Path 484 | total_timesteps 5182.
Path 485 | total_timesteps 5193.
Path 486 | total_timesteps 5204.
Path 487 | total_timesteps 5214.
Path 488 | total_timesteps 5221.
Path 489 | total_timesteps 5230.
Path 490 | total_timesteps 5238.
Path 491 | total_timesteps 5248.
Path 492 | total_timesteps 5258.
Path 493 | total_timesteps 5273.
Path 494 | total_timesteps 5282.
Path 495 | total_timesteps 5294.
Path 496 | total_timesteps 5302.
Path 497 | total_timesteps 5310.
Path 498 | total_timesteps 5321.
Path 499 | total_timesteps 5335.
Path 500 | total_timesteps 5351.
Path 501 | total_timesteps 5359.
Path 502 | total_timesteps 5370.
Path 503 | total_timesteps 5381.
Path 504 | total_timesteps 5401.
Path 505 | total_timesteps 5409.
Path 506 | total_timesteps 5416.
Path 507 | total_timesteps 5426.
Path 508 | total_timesteps 5439.
Path 509 | total_timesteps 5448.
Path 510 | total_timesteps 5464.
Path 511 | total_timesteps 5472.
Path 512 | total_timesteps 5484.
Path 513 | total_timesteps 5494.
Path 514 | total_timesteps 5503.
Path 515 | total_timesteps 5513.
Path 516 | total_timesteps 5521.
Path 517 | total_timesteps 5533.
Path 518 | total_timesteps 5542.
Path 519 | total_timesteps 5555.
Path 520 | total_timesteps 5564.
Path 521 | total_timesteps 5574.
Path 522 | total_timesteps 5583.
Path 523 | total_timesteps 5589.
Path 524 | total_timesteps 5599.
Path 525 | total_timesteps 5611.
Path 526 | total_timesteps 5624.
Path 527 | total_timesteps 5635.
Path 528 | total_timesteps 5645.
Path 529 | total_timesteps 5655.
Path 530 | total_timesteps 5674.
Path 531 | total_timesteps 5684.
Path 532 | total_timesteps 5703.
Path 533 | total_timesteps 5716.
Path 534 | total_timesteps 5731.
Path 535 | total_timesteps 5738.
Path 536 | total_timesteps 5747.
Path 537 | total_timesteps 5759.
Path 538 | total_timesteps 5769.
Path 539 | total_timesteps 5778.
Path 540 | total_timesteps 5793.
Path 541 | total_timesteps 5800.
Path 542 | total_timesteps 5817.
Path 543 | total_timesteps 5826.
Path 544 | total_timesteps 5840.
Path 545 | total_timesteps 5851.
Path 546 | total_timesteps 5858.
Path 547 | total_timesteps 5865.
Path 548 | total_timesteps 5876.
Path 549 | total_timesteps 5885.
Path 550 | total_timesteps 5898.
Path 551 | total_timesteps 5912.
Path 552 | total_timesteps 5921.
Path 553 | total_timesteps 5932.
Path 554 | total_timesteps 5939.
Path 555 | total_timesteps 5948.
Path 556 | total_timesteps 5969.
Path 557 | total_timesteps 5987.
Path 558 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.37    |
| Iteration     | 26       |
| MaximumReturn | 0.727    |
| MinimumReturn | -21.1    |
| TotalSamples  | 112106   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.3166937828063965
Validation loss = 0.32188600301742554
Validation loss = 0.32195720076560974
Validation loss = 0.32070401310920715
Validation loss = 0.320937842130661
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 61.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 79.
Path 9 | total_timesteps 98.
Path 10 | total_timesteps 108.
Path 11 | total_timesteps 117.
Path 12 | total_timesteps 136.
Path 13 | total_timesteps 144.
Path 14 | total_timesteps 152.
Path 15 | total_timesteps 160.
Path 16 | total_timesteps 170.
Path 17 | total_timesteps 177.
Path 18 | total_timesteps 185.
Path 19 | total_timesteps 193.
Path 20 | total_timesteps 204.
Path 21 | total_timesteps 211.
Path 22 | total_timesteps 224.
Path 23 | total_timesteps 232.
Path 24 | total_timesteps 243.
Path 25 | total_timesteps 253.
Path 26 | total_timesteps 263.
Path 27 | total_timesteps 269.
Path 28 | total_timesteps 283.
Path 29 | total_timesteps 292.
Path 30 | total_timesteps 300.
Path 31 | total_timesteps 309.
Path 32 | total_timesteps 322.
Path 33 | total_timesteps 335.
Path 34 | total_timesteps 344.
Path 35 | total_timesteps 355.
Path 36 | total_timesteps 363.
Path 37 | total_timesteps 375.
Path 38 | total_timesteps 389.
Path 39 | total_timesteps 398.
Path 40 | total_timesteps 411.
Path 41 | total_timesteps 419.
Path 42 | total_timesteps 431.
Path 43 | total_timesteps 443.
Path 44 | total_timesteps 460.
Path 45 | total_timesteps 468.
Path 46 | total_timesteps 477.
Path 47 | total_timesteps 488.
Path 48 | total_timesteps 496.
Path 49 | total_timesteps 508.
Path 50 | total_timesteps 516.
Path 51 | total_timesteps 524.
Path 52 | total_timesteps 537.
Path 53 | total_timesteps 553.
Path 54 | total_timesteps 569.
Path 55 | total_timesteps 577.
Path 56 | total_timesteps 587.
Path 57 | total_timesteps 599.
Path 58 | total_timesteps 608.
Path 59 | total_timesteps 619.
Path 60 | total_timesteps 631.
Path 61 | total_timesteps 640.
Path 62 | total_timesteps 650.
Path 63 | total_timesteps 661.
Path 64 | total_timesteps 671.
Path 65 | total_timesteps 683.
Path 66 | total_timesteps 691.
Path 67 | total_timesteps 710.
Path 68 | total_timesteps 718.
Path 69 | total_timesteps 732.
Path 70 | total_timesteps 746.
Path 71 | total_timesteps 761.
Path 72 | total_timesteps 768.
Path 73 | total_timesteps 775.
Path 74 | total_timesteps 789.
Path 75 | total_timesteps 804.
Path 76 | total_timesteps 813.
Path 77 | total_timesteps 821.
Path 78 | total_timesteps 837.
Path 79 | total_timesteps 847.
Path 80 | total_timesteps 862.
Path 81 | total_timesteps 873.
Path 82 | total_timesteps 889.
Path 83 | total_timesteps 898.
Path 84 | total_timesteps 910.
Path 85 | total_timesteps 918.
Path 86 | total_timesteps 926.
Path 87 | total_timesteps 936.
Path 88 | total_timesteps 946.
Path 89 | total_timesteps 952.
Path 90 | total_timesteps 960.
Path 91 | total_timesteps 972.
Path 92 | total_timesteps 984.
Path 93 | total_timesteps 991.
Path 94 | total_timesteps 1000.
Path 95 | total_timesteps 1012.
Path 96 | total_timesteps 1027.
Path 97 | total_timesteps 1037.
Path 98 | total_timesteps 1048.
Path 99 | total_timesteps 1056.
Path 100 | total_timesteps 1065.
Path 101 | total_timesteps 1074.
Path 102 | total_timesteps 1085.
Path 103 | total_timesteps 1100.
Path 104 | total_timesteps 1108.
Path 105 | total_timesteps 1118.
Path 106 | total_timesteps 1129.
Path 107 | total_timesteps 1137.
Path 108 | total_timesteps 1146.
Path 109 | total_timesteps 1155.
Path 110 | total_timesteps 1165.
Path 111 | total_timesteps 1175.
Path 112 | total_timesteps 1184.
Path 113 | total_timesteps 1195.
Path 114 | total_timesteps 1210.
Path 115 | total_timesteps 1222.
Path 116 | total_timesteps 1232.
Path 117 | total_timesteps 1246.
Path 118 | total_timesteps 1255.
Path 119 | total_timesteps 1264.
Path 120 | total_timesteps 1277.
Path 121 | total_timesteps 1290.
Path 122 | total_timesteps 1302.
Path 123 | total_timesteps 1310.
Path 124 | total_timesteps 1320.
Path 125 | total_timesteps 1328.
Path 126 | total_timesteps 1339.
Path 127 | total_timesteps 1347.
Path 128 | total_timesteps 1361.
Path 129 | total_timesteps 1372.
Path 130 | total_timesteps 1384.
Path 131 | total_timesteps 1401.
Path 132 | total_timesteps 1411.
Path 133 | total_timesteps 1420.
Path 134 | total_timesteps 1426.
Path 135 | total_timesteps 1436.
Path 136 | total_timesteps 1443.
Path 137 | total_timesteps 1453.
Path 138 | total_timesteps 1462.
Path 139 | total_timesteps 1472.
Path 140 | total_timesteps 1485.
Path 141 | total_timesteps 1494.
Path 142 | total_timesteps 1507.
Path 143 | total_timesteps 1521.
Path 144 | total_timesteps 1529.
Path 145 | total_timesteps 1536.
Path 146 | total_timesteps 1545.
Path 147 | total_timesteps 1555.
Path 148 | total_timesteps 1564.
Path 149 | total_timesteps 1574.
Path 150 | total_timesteps 1584.
Path 151 | total_timesteps 1592.
Path 152 | total_timesteps 1603.
Path 153 | total_timesteps 1612.
Path 154 | total_timesteps 1627.
Path 155 | total_timesteps 1636.
Path 156 | total_timesteps 1645.
Path 157 | total_timesteps 1657.
Path 158 | total_timesteps 1669.
Path 159 | total_timesteps 1680.
Path 160 | total_timesteps 1687.
Path 161 | total_timesteps 1701.
Path 162 | total_timesteps 1714.
Path 163 | total_timesteps 1727.
Path 164 | total_timesteps 1744.
Path 165 | total_timesteps 1753.
Path 166 | total_timesteps 1766.
Path 167 | total_timesteps 1777.
Path 168 | total_timesteps 1786.
Path 169 | total_timesteps 1797.
Path 170 | total_timesteps 1808.
Path 171 | total_timesteps 1820.
Path 172 | total_timesteps 1828.
Path 173 | total_timesteps 1838.
Path 174 | total_timesteps 1850.
Path 175 | total_timesteps 1863.
Path 176 | total_timesteps 1872.
Path 177 | total_timesteps 1880.
Path 178 | total_timesteps 1891.
Path 179 | total_timesteps 1901.
Path 180 | total_timesteps 1913.
Path 181 | total_timesteps 1927.
Path 182 | total_timesteps 1934.
Path 183 | total_timesteps 1940.
Path 184 | total_timesteps 1953.
Path 185 | total_timesteps 1966.
Path 186 | total_timesteps 1975.
Path 187 | total_timesteps 1986.
Path 188 | total_timesteps 1995.
Path 189 | total_timesteps 2006.
Path 190 | total_timesteps 2015.
Path 191 | total_timesteps 2022.
Path 192 | total_timesteps 2031.
Path 193 | total_timesteps 2040.
Path 194 | total_timesteps 2050.
Path 195 | total_timesteps 2059.
Path 196 | total_timesteps 2068.
Path 197 | total_timesteps 2076.
Path 198 | total_timesteps 2086.
Path 199 | total_timesteps 2096.
Path 200 | total_timesteps 2104.
Path 201 | total_timesteps 2113.
Path 202 | total_timesteps 2124.
Path 203 | total_timesteps 2134.
Path 204 | total_timesteps 2145.
Path 205 | total_timesteps 2152.
Path 206 | total_timesteps 2163.
Path 207 | total_timesteps 2172.
Path 208 | total_timesteps 2183.
Path 209 | total_timesteps 2193.
Path 210 | total_timesteps 2202.
Path 211 | total_timesteps 2215.
Path 212 | total_timesteps 2227.
Path 213 | total_timesteps 2238.
Path 214 | total_timesteps 2246.
Path 215 | total_timesteps 2261.
Path 216 | total_timesteps 2271.
Path 217 | total_timesteps 2281.
Path 218 | total_timesteps 2290.
Path 219 | total_timesteps 2300.
Path 220 | total_timesteps 2312.
Path 221 | total_timesteps 2321.
Path 222 | total_timesteps 2329.
Path 223 | total_timesteps 2336.
Path 224 | total_timesteps 2345.
Path 225 | total_timesteps 2353.
Path 226 | total_timesteps 2362.
Path 227 | total_timesteps 2371.
Path 228 | total_timesteps 2382.
Path 229 | total_timesteps 2395.
Path 230 | total_timesteps 2408.
Path 231 | total_timesteps 2418.
Path 232 | total_timesteps 2433.
Path 233 | total_timesteps 2442.
Path 234 | total_timesteps 2451.
Path 235 | total_timesteps 2458.
Path 236 | total_timesteps 2467.
Path 237 | total_timesteps 2479.
Path 238 | total_timesteps 2490.
Path 239 | total_timesteps 2498.
Path 240 | total_timesteps 2509.
Path 241 | total_timesteps 2518.
Path 242 | total_timesteps 2529.
Path 243 | total_timesteps 2539.
Path 244 | total_timesteps 2547.
Path 245 | total_timesteps 2561.
Path 246 | total_timesteps 2571.
Path 247 | total_timesteps 2582.
Path 248 | total_timesteps 2590.
Path 249 | total_timesteps 2598.
Path 250 | total_timesteps 2608.
Path 251 | total_timesteps 2617.
Path 252 | total_timesteps 2629.
Path 253 | total_timesteps 2638.
Path 254 | total_timesteps 2652.
Path 255 | total_timesteps 2666.
Path 256 | total_timesteps 2677.
Path 257 | total_timesteps 2686.
Path 258 | total_timesteps 2693.
Path 259 | total_timesteps 2701.
Path 260 | total_timesteps 2709.
Path 261 | total_timesteps 2717.
Path 262 | total_timesteps 2728.
Path 263 | total_timesteps 2739.
Path 264 | total_timesteps 2748.
Path 265 | total_timesteps 2757.
Path 266 | total_timesteps 2769.
Path 267 | total_timesteps 2786.
Path 268 | total_timesteps 2801.
Path 269 | total_timesteps 2810.
Path 270 | total_timesteps 2819.
Path 271 | total_timesteps 2829.
Path 272 | total_timesteps 2837.
Path 273 | total_timesteps 2845.
Path 274 | total_timesteps 2854.
Path 275 | total_timesteps 2863.
Path 276 | total_timesteps 2877.
Path 277 | total_timesteps 2891.
Path 278 | total_timesteps 2900.
Path 279 | total_timesteps 2914.
Path 280 | total_timesteps 2920.
Path 281 | total_timesteps 2931.
Path 282 | total_timesteps 2940.
Path 283 | total_timesteps 2950.
Path 284 | total_timesteps 2959.
Path 285 | total_timesteps 2968.
Path 286 | total_timesteps 2988.
Path 287 | total_timesteps 2997.
Path 288 | total_timesteps 3006.
Path 289 | total_timesteps 3020.
Path 290 | total_timesteps 3031.
Path 291 | total_timesteps 3044.
Path 292 | total_timesteps 3057.
Path 293 | total_timesteps 3075.
Path 294 | total_timesteps 3086.
Path 295 | total_timesteps 3096.
Path 296 | total_timesteps 3105.
Path 297 | total_timesteps 3112.
Path 298 | total_timesteps 3121.
Path 299 | total_timesteps 3131.
Path 300 | total_timesteps 3141.
Path 301 | total_timesteps 3155.
Path 302 | total_timesteps 3167.
Path 303 | total_timesteps 3177.
Path 304 | total_timesteps 3188.
Path 305 | total_timesteps 3198.
Path 306 | total_timesteps 3206.
Path 307 | total_timesteps 3212.
Path 308 | total_timesteps 3224.
Path 309 | total_timesteps 3236.
Path 310 | total_timesteps 3244.
Path 311 | total_timesteps 3258.
Path 312 | total_timesteps 3272.
Path 313 | total_timesteps 3284.
Path 314 | total_timesteps 3295.
Path 315 | total_timesteps 3306.
Path 316 | total_timesteps 3316.
Path 317 | total_timesteps 3327.
Path 318 | total_timesteps 3337.
Path 319 | total_timesteps 3345.
Path 320 | total_timesteps 3354.
Path 321 | total_timesteps 3366.
Path 322 | total_timesteps 3374.
Path 323 | total_timesteps 3392.
Path 324 | total_timesteps 3402.
Path 325 | total_timesteps 3413.
Path 326 | total_timesteps 3423.
Path 327 | total_timesteps 3441.
Path 328 | total_timesteps 3450.
Path 329 | total_timesteps 3458.
Path 330 | total_timesteps 3466.
Path 331 | total_timesteps 3473.
Path 332 | total_timesteps 3491.
Path 333 | total_timesteps 3502.
Path 334 | total_timesteps 3521.
Path 335 | total_timesteps 3531.
Path 336 | total_timesteps 3544.
Path 337 | total_timesteps 3556.
Path 338 | total_timesteps 3574.
Path 339 | total_timesteps 3582.
Path 340 | total_timesteps 3592.
Path 341 | total_timesteps 3602.
Path 342 | total_timesteps 3612.
Path 343 | total_timesteps 3623.
Path 344 | total_timesteps 3631.
Path 345 | total_timesteps 3639.
Path 346 | total_timesteps 3651.
Path 347 | total_timesteps 3662.
Path 348 | total_timesteps 3672.
Path 349 | total_timesteps 3679.
Path 350 | total_timesteps 3694.
Path 351 | total_timesteps 3705.
Path 352 | total_timesteps 3715.
Path 353 | total_timesteps 3738.
Path 354 | total_timesteps 3745.
Path 355 | total_timesteps 3756.
Path 356 | total_timesteps 3765.
Path 357 | total_timesteps 3774.
Path 358 | total_timesteps 3782.
Path 359 | total_timesteps 3790.
Path 360 | total_timesteps 3800.
Path 361 | total_timesteps 3808.
Path 362 | total_timesteps 3818.
Path 363 | total_timesteps 3829.
Path 364 | total_timesteps 3839.
Path 365 | total_timesteps 3846.
Path 366 | total_timesteps 3856.
Path 367 | total_timesteps 3866.
Path 368 | total_timesteps 3879.
Path 369 | total_timesteps 3888.
Path 370 | total_timesteps 3900.
Path 371 | total_timesteps 3912.
Path 372 | total_timesteps 3919.
Path 373 | total_timesteps 3933.
Path 374 | total_timesteps 3940.
Path 375 | total_timesteps 3951.
Path 376 | total_timesteps 3961.
Path 377 | total_timesteps 3970.
Path 378 | total_timesteps 3982.
Path 379 | total_timesteps 3993.
Path 380 | total_timesteps 4002.
Path 381 | total_timesteps 4016.
Path 382 | total_timesteps 4027.
Path 383 | total_timesteps 4036.
Path 384 | total_timesteps 4048.
Path 385 | total_timesteps 4061.
Path 386 | total_timesteps 4069.
Path 387 | total_timesteps 4077.
Path 388 | total_timesteps 4087.
Path 389 | total_timesteps 4098.
Path 390 | total_timesteps 4114.
Path 391 | total_timesteps 4126.
Path 392 | total_timesteps 4134.
Path 393 | total_timesteps 4142.
Path 394 | total_timesteps 4153.
Path 395 | total_timesteps 4161.
Path 396 | total_timesteps 4170.
Path 397 | total_timesteps 4179.
Path 398 | total_timesteps 4189.
Path 399 | total_timesteps 4199.
Path 400 | total_timesteps 4206.
Path 401 | total_timesteps 4214.
Path 402 | total_timesteps 4223.
Path 403 | total_timesteps 4230.
Path 404 | total_timesteps 4240.
Path 405 | total_timesteps 4257.
Path 406 | total_timesteps 4269.
Path 407 | total_timesteps 4280.
Path 408 | total_timesteps 4298.
Path 409 | total_timesteps 4307.
Path 410 | total_timesteps 4318.
Path 411 | total_timesteps 4325.
Path 412 | total_timesteps 4335.
Path 413 | total_timesteps 4342.
Path 414 | total_timesteps 4353.
Path 415 | total_timesteps 4363.
Path 416 | total_timesteps 4371.
Path 417 | total_timesteps 4381.
Path 418 | total_timesteps 4392.
Path 419 | total_timesteps 4406.
Path 420 | total_timesteps 4419.
Path 421 | total_timesteps 4427.
Path 422 | total_timesteps 4435.
Path 423 | total_timesteps 4448.
Path 424 | total_timesteps 4458.
Path 425 | total_timesteps 4465.
Path 426 | total_timesteps 4477.
Path 427 | total_timesteps 4487.
Path 428 | total_timesteps 4497.
Path 429 | total_timesteps 4504.
Path 430 | total_timesteps 4515.
Path 431 | total_timesteps 4524.
Path 432 | total_timesteps 4534.
Path 433 | total_timesteps 4541.
Path 434 | total_timesteps 4549.
Path 435 | total_timesteps 4556.
Path 436 | total_timesteps 4566.
Path 437 | total_timesteps 4575.
Path 438 | total_timesteps 4585.
Path 439 | total_timesteps 4594.
Path 440 | total_timesteps 4604.
Path 441 | total_timesteps 4613.
Path 442 | total_timesteps 4621.
Path 443 | total_timesteps 4639.
Path 444 | total_timesteps 4649.
Path 445 | total_timesteps 4660.
Path 446 | total_timesteps 4671.
Path 447 | total_timesteps 4677.
Path 448 | total_timesteps 4689.
Path 449 | total_timesteps 4702.
Path 450 | total_timesteps 4717.
Path 451 | total_timesteps 4727.
Path 452 | total_timesteps 4737.
Path 453 | total_timesteps 4748.
Path 454 | total_timesteps 4761.
Path 455 | total_timesteps 4772.
Path 456 | total_timesteps 4782.
Path 457 | total_timesteps 4793.
Path 458 | total_timesteps 4800.
Path 459 | total_timesteps 4809.
Path 460 | total_timesteps 4818.
Path 461 | total_timesteps 4828.
Path 462 | total_timesteps 4838.
Path 463 | total_timesteps 4849.
Path 464 | total_timesteps 4860.
Path 465 | total_timesteps 4868.
Path 466 | total_timesteps 4878.
Path 467 | total_timesteps 4885.
Path 468 | total_timesteps 4902.
Path 469 | total_timesteps 4911.
Path 470 | total_timesteps 4921.
Path 471 | total_timesteps 4934.
Path 472 | total_timesteps 4944.
Path 473 | total_timesteps 4958.
Path 474 | total_timesteps 4969.
Path 475 | total_timesteps 4978.
Path 476 | total_timesteps 4986.
Path 477 | total_timesteps 5003.
Path 478 | total_timesteps 5015.
Path 479 | total_timesteps 5027.
Path 480 | total_timesteps 5040.
Path 481 | total_timesteps 5050.
Path 482 | total_timesteps 5058.
Path 483 | total_timesteps 5065.
Path 484 | total_timesteps 5083.
Path 485 | total_timesteps 5092.
Path 486 | total_timesteps 5101.
Path 487 | total_timesteps 5111.
Path 488 | total_timesteps 5117.
Path 489 | total_timesteps 5131.
Path 490 | total_timesteps 5139.
Path 491 | total_timesteps 5152.
Path 492 | total_timesteps 5163.
Path 493 | total_timesteps 5174.
Path 494 | total_timesteps 5183.
Path 495 | total_timesteps 5197.
Path 496 | total_timesteps 5210.
Path 497 | total_timesteps 5219.
Path 498 | total_timesteps 5230.
Path 499 | total_timesteps 5243.
Path 500 | total_timesteps 5259.
Path 501 | total_timesteps 5267.
Path 502 | total_timesteps 5279.
Path 503 | total_timesteps 5293.
Path 504 | total_timesteps 5300.
Path 505 | total_timesteps 5310.
Path 506 | total_timesteps 5317.
Path 507 | total_timesteps 5325.
Path 508 | total_timesteps 5334.
Path 509 | total_timesteps 5342.
Path 510 | total_timesteps 5351.
Path 511 | total_timesteps 5360.
Path 512 | total_timesteps 5370.
Path 513 | total_timesteps 5379.
Path 514 | total_timesteps 5390.
Path 515 | total_timesteps 5400.
Path 516 | total_timesteps 5410.
Path 517 | total_timesteps 5419.
Path 518 | total_timesteps 5428.
Path 519 | total_timesteps 5438.
Path 520 | total_timesteps 5454.
Path 521 | total_timesteps 5462.
Path 522 | total_timesteps 5475.
Path 523 | total_timesteps 5485.
Path 524 | total_timesteps 5494.
Path 525 | total_timesteps 5502.
Path 526 | total_timesteps 5509.
Path 527 | total_timesteps 5520.
Path 528 | total_timesteps 5529.
Path 529 | total_timesteps 5539.
Path 530 | total_timesteps 5548.
Path 531 | total_timesteps 5557.
Path 532 | total_timesteps 5567.
Path 533 | total_timesteps 5579.
Path 534 | total_timesteps 5589.
Path 535 | total_timesteps 5601.
Path 536 | total_timesteps 5609.
Path 537 | total_timesteps 5617.
Path 538 | total_timesteps 5630.
Path 539 | total_timesteps 5637.
Path 540 | total_timesteps 5646.
Path 541 | total_timesteps 5654.
Path 542 | total_timesteps 5661.
Path 543 | total_timesteps 5682.
Path 544 | total_timesteps 5690.
Path 545 | total_timesteps 5699.
Path 546 | total_timesteps 5710.
Path 547 | total_timesteps 5728.
Path 548 | total_timesteps 5734.
Path 549 | total_timesteps 5747.
Path 550 | total_timesteps 5764.
Path 551 | total_timesteps 5771.
Path 552 | total_timesteps 5781.
Path 553 | total_timesteps 5795.
Path 554 | total_timesteps 5804.
Path 555 | total_timesteps 5813.
Path 556 | total_timesteps 5823.
Path 557 | total_timesteps 5830.
Path 558 | total_timesteps 5844.
Path 559 | total_timesteps 5855.
Path 560 | total_timesteps 5867.
Path 561 | total_timesteps 5877.
Path 562 | total_timesteps 5891.
Path 563 | total_timesteps 5897.
Path 564 | total_timesteps 5909.
Path 565 | total_timesteps 5920.
Path 566 | total_timesteps 5929.
Path 567 | total_timesteps 5939.
Path 568 | total_timesteps 5948.
Path 569 | total_timesteps 5958.
Path 570 | total_timesteps 5972.
Path 571 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.13    |
| Iteration     | 27       |
| MaximumReturn | 0.361    |
| MinimumReturn | -19      |
| TotalSamples  | 116108   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.31405165791511536
Validation loss = 0.3183000683784485
Validation loss = 0.31938987970352173
Validation loss = 0.32347482442855835
Validation loss = 0.3206949532032013
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 76.
Path 8 | total_timesteps 84.
Path 9 | total_timesteps 94.
Path 10 | total_timesteps 110.
Path 11 | total_timesteps 125.
Path 12 | total_timesteps 135.
Path 13 | total_timesteps 142.
Path 14 | total_timesteps 154.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 172.
Path 17 | total_timesteps 180.
Path 18 | total_timesteps 187.
Path 19 | total_timesteps 197.
Path 20 | total_timesteps 208.
Path 21 | total_timesteps 218.
Path 22 | total_timesteps 233.
Path 23 | total_timesteps 239.
Path 24 | total_timesteps 246.
Path 25 | total_timesteps 256.
Path 26 | total_timesteps 268.
Path 27 | total_timesteps 280.
Path 28 | total_timesteps 288.
Path 29 | total_timesteps 297.
Path 30 | total_timesteps 310.
Path 31 | total_timesteps 320.
Path 32 | total_timesteps 328.
Path 33 | total_timesteps 339.
Path 34 | total_timesteps 350.
Path 35 | total_timesteps 363.
Path 36 | total_timesteps 372.
Path 37 | total_timesteps 386.
Path 38 | total_timesteps 399.
Path 39 | total_timesteps 408.
Path 40 | total_timesteps 415.
Path 41 | total_timesteps 425.
Path 42 | total_timesteps 432.
Path 43 | total_timesteps 445.
Path 44 | total_timesteps 453.
Path 45 | total_timesteps 461.
Path 46 | total_timesteps 475.
Path 47 | total_timesteps 484.
Path 48 | total_timesteps 496.
Path 49 | total_timesteps 507.
Path 50 | total_timesteps 522.
Path 51 | total_timesteps 538.
Path 52 | total_timesteps 549.
Path 53 | total_timesteps 559.
Path 54 | total_timesteps 568.
Path 55 | total_timesteps 578.
Path 56 | total_timesteps 588.
Path 57 | total_timesteps 598.
Path 58 | total_timesteps 612.
Path 59 | total_timesteps 620.
Path 60 | total_timesteps 631.
Path 61 | total_timesteps 642.
Path 62 | total_timesteps 657.
Path 63 | total_timesteps 667.
Path 64 | total_timesteps 677.
Path 65 | total_timesteps 689.
Path 66 | total_timesteps 698.
Path 67 | total_timesteps 707.
Path 68 | total_timesteps 734.
Path 69 | total_timesteps 742.
Path 70 | total_timesteps 754.
Path 71 | total_timesteps 765.
Path 72 | total_timesteps 776.
Path 73 | total_timesteps 784.
Path 74 | total_timesteps 793.
Path 75 | total_timesteps 804.
Path 76 | total_timesteps 814.
Path 77 | total_timesteps 830.
Path 78 | total_timesteps 838.
Path 79 | total_timesteps 856.
Path 80 | total_timesteps 865.
Path 81 | total_timesteps 876.
Path 82 | total_timesteps 892.
Path 83 | total_timesteps 903.
Path 84 | total_timesteps 911.
Path 85 | total_timesteps 919.
Path 86 | total_timesteps 927.
Path 87 | total_timesteps 935.
Path 88 | total_timesteps 945.
Path 89 | total_timesteps 953.
Path 90 | total_timesteps 961.
Path 91 | total_timesteps 975.
Path 92 | total_timesteps 986.
Path 93 | total_timesteps 994.
Path 94 | total_timesteps 1001.
Path 95 | total_timesteps 1008.
Path 96 | total_timesteps 1018.
Path 97 | total_timesteps 1027.
Path 98 | total_timesteps 1036.
Path 99 | total_timesteps 1048.
Path 100 | total_timesteps 1059.
Path 101 | total_timesteps 1066.
Path 102 | total_timesteps 1075.
Path 103 | total_timesteps 1083.
Path 104 | total_timesteps 1094.
Path 105 | total_timesteps 1107.
Path 106 | total_timesteps 1121.
Path 107 | total_timesteps 1130.
Path 108 | total_timesteps 1144.
Path 109 | total_timesteps 1156.
Path 110 | total_timesteps 1166.
Path 111 | total_timesteps 1179.
Path 112 | total_timesteps 1189.
Path 113 | total_timesteps 1201.
Path 114 | total_timesteps 1209.
Path 115 | total_timesteps 1218.
Path 116 | total_timesteps 1225.
Path 117 | total_timesteps 1237.
Path 118 | total_timesteps 1251.
Path 119 | total_timesteps 1270.
Path 120 | total_timesteps 1280.
Path 121 | total_timesteps 1288.
Path 122 | total_timesteps 1295.
Path 123 | total_timesteps 1304.
Path 124 | total_timesteps 1316.
Path 125 | total_timesteps 1335.
Path 126 | total_timesteps 1343.
Path 127 | total_timesteps 1356.
Path 128 | total_timesteps 1370.
Path 129 | total_timesteps 1380.
Path 130 | total_timesteps 1386.
Path 131 | total_timesteps 1394.
Path 132 | total_timesteps 1407.
Path 133 | total_timesteps 1414.
Path 134 | total_timesteps 1424.
Path 135 | total_timesteps 1432.
Path 136 | total_timesteps 1443.
Path 137 | total_timesteps 1452.
Path 138 | total_timesteps 1459.
Path 139 | total_timesteps 1467.
Path 140 | total_timesteps 1483.
Path 141 | total_timesteps 1495.
Path 142 | total_timesteps 1511.
Path 143 | total_timesteps 1521.
Path 144 | total_timesteps 1532.
Path 145 | total_timesteps 1544.
Path 146 | total_timesteps 1553.
Path 147 | total_timesteps 1560.
Path 148 | total_timesteps 1575.
Path 149 | total_timesteps 1584.
Path 150 | total_timesteps 1596.
Path 151 | total_timesteps 1605.
Path 152 | total_timesteps 1619.
Path 153 | total_timesteps 1634.
Path 154 | total_timesteps 1641.
Path 155 | total_timesteps 1651.
Path 156 | total_timesteps 1663.
Path 157 | total_timesteps 1676.
Path 158 | total_timesteps 1683.
Path 159 | total_timesteps 1701.
Path 160 | total_timesteps 1712.
Path 161 | total_timesteps 1721.
Path 162 | total_timesteps 1731.
Path 163 | total_timesteps 1742.
Path 164 | total_timesteps 1755.
Path 165 | total_timesteps 1765.
Path 166 | total_timesteps 1773.
Path 167 | total_timesteps 1783.
Path 168 | total_timesteps 1795.
Path 169 | total_timesteps 1809.
Path 170 | total_timesteps 1825.
Path 171 | total_timesteps 1835.
Path 172 | total_timesteps 1851.
Path 173 | total_timesteps 1861.
Path 174 | total_timesteps 1871.
Path 175 | total_timesteps 1884.
Path 176 | total_timesteps 1892.
Path 177 | total_timesteps 1906.
Path 178 | total_timesteps 1919.
Path 179 | total_timesteps 1932.
Path 180 | total_timesteps 1943.
Path 181 | total_timesteps 1957.
Path 182 | total_timesteps 1966.
Path 183 | total_timesteps 1976.
Path 184 | total_timesteps 1985.
Path 185 | total_timesteps 1997.
Path 186 | total_timesteps 2004.
Path 187 | total_timesteps 2011.
Path 188 | total_timesteps 2018.
Path 189 | total_timesteps 2025.
Path 190 | total_timesteps 2033.
Path 191 | total_timesteps 2046.
Path 192 | total_timesteps 2057.
Path 193 | total_timesteps 2072.
Path 194 | total_timesteps 2087.
Path 195 | total_timesteps 2097.
Path 196 | total_timesteps 2114.
Path 197 | total_timesteps 2123.
Path 198 | total_timesteps 2134.
Path 199 | total_timesteps 2142.
Path 200 | total_timesteps 2153.
Path 201 | total_timesteps 2160.
Path 202 | total_timesteps 2169.
Path 203 | total_timesteps 2179.
Path 204 | total_timesteps 2187.
Path 205 | total_timesteps 2200.
Path 206 | total_timesteps 2219.
Path 207 | total_timesteps 2229.
Path 208 | total_timesteps 2237.
Path 209 | total_timesteps 2249.
Path 210 | total_timesteps 2260.
Path 211 | total_timesteps 2268.
Path 212 | total_timesteps 2276.
Path 213 | total_timesteps 2286.
Path 214 | total_timesteps 2297.
Path 215 | total_timesteps 2305.
Path 216 | total_timesteps 2322.
Path 217 | total_timesteps 2332.
Path 218 | total_timesteps 2341.
Path 219 | total_timesteps 2351.
Path 220 | total_timesteps 2362.
Path 221 | total_timesteps 2372.
Path 222 | total_timesteps 2380.
Path 223 | total_timesteps 2393.
Path 224 | total_timesteps 2403.
Path 225 | total_timesteps 2410.
Path 226 | total_timesteps 2416.
Path 227 | total_timesteps 2425.
Path 228 | total_timesteps 2438.
Path 229 | total_timesteps 2451.
Path 230 | total_timesteps 2462.
Path 231 | total_timesteps 2471.
Path 232 | total_timesteps 2485.
Path 233 | total_timesteps 2494.
Path 234 | total_timesteps 2502.
Path 235 | total_timesteps 2511.
Path 236 | total_timesteps 2522.
Path 237 | total_timesteps 2531.
Path 238 | total_timesteps 2542.
Path 239 | total_timesteps 2551.
Path 240 | total_timesteps 2566.
Path 241 | total_timesteps 2574.
Path 242 | total_timesteps 2581.
Path 243 | total_timesteps 2591.
Path 244 | total_timesteps 2602.
Path 245 | total_timesteps 2610.
Path 246 | total_timesteps 2620.
Path 247 | total_timesteps 2629.
Path 248 | total_timesteps 2640.
Path 249 | total_timesteps 2657.
Path 250 | total_timesteps 2664.
Path 251 | total_timesteps 2675.
Path 252 | total_timesteps 2685.
Path 253 | total_timesteps 2698.
Path 254 | total_timesteps 2706.
Path 255 | total_timesteps 2720.
Path 256 | total_timesteps 2731.
Path 257 | total_timesteps 2740.
Path 258 | total_timesteps 2750.
Path 259 | total_timesteps 2759.
Path 260 | total_timesteps 2774.
Path 261 | total_timesteps 2788.
Path 262 | total_timesteps 2797.
Path 263 | total_timesteps 2805.
Path 264 | total_timesteps 2819.
Path 265 | total_timesteps 2827.
Path 266 | total_timesteps 2841.
Path 267 | total_timesteps 2849.
Path 268 | total_timesteps 2861.
Path 269 | total_timesteps 2868.
Path 270 | total_timesteps 2879.
Path 271 | total_timesteps 2893.
Path 272 | total_timesteps 2900.
Path 273 | total_timesteps 2909.
Path 274 | total_timesteps 2919.
Path 275 | total_timesteps 2928.
Path 276 | total_timesteps 2937.
Path 277 | total_timesteps 2943.
Path 278 | total_timesteps 2953.
Path 279 | total_timesteps 2962.
Path 280 | total_timesteps 2975.
Path 281 | total_timesteps 2987.
Path 282 | total_timesteps 2998.
Path 283 | total_timesteps 3010.
Path 284 | total_timesteps 3019.
Path 285 | total_timesteps 3027.
Path 286 | total_timesteps 3038.
Path 287 | total_timesteps 3046.
Path 288 | total_timesteps 3055.
Path 289 | total_timesteps 3068.
Path 290 | total_timesteps 3080.
Path 291 | total_timesteps 3088.
Path 292 | total_timesteps 3096.
Path 293 | total_timesteps 3103.
Path 294 | total_timesteps 3114.
Path 295 | total_timesteps 3123.
Path 296 | total_timesteps 3134.
Path 297 | total_timesteps 3142.
Path 298 | total_timesteps 3154.
Path 299 | total_timesteps 3163.
Path 300 | total_timesteps 3173.
Path 301 | total_timesteps 3184.
Path 302 | total_timesteps 3192.
Path 303 | total_timesteps 3202.
Path 304 | total_timesteps 3209.
Path 305 | total_timesteps 3217.
Path 306 | total_timesteps 3228.
Path 307 | total_timesteps 3248.
Path 308 | total_timesteps 3260.
Path 309 | total_timesteps 3273.
Path 310 | total_timesteps 3283.
Path 311 | total_timesteps 3292.
Path 312 | total_timesteps 3300.
Path 313 | total_timesteps 3310.
Path 314 | total_timesteps 3320.
Path 315 | total_timesteps 3328.
Path 316 | total_timesteps 3337.
Path 317 | total_timesteps 3346.
Path 318 | total_timesteps 3357.
Path 319 | total_timesteps 3364.
Path 320 | total_timesteps 3372.
Path 321 | total_timesteps 3382.
Path 322 | total_timesteps 3390.
Path 323 | total_timesteps 3398.
Path 324 | total_timesteps 3406.
Path 325 | total_timesteps 3414.
Path 326 | total_timesteps 3422.
Path 327 | total_timesteps 3438.
Path 328 | total_timesteps 3450.
Path 329 | total_timesteps 3457.
Path 330 | total_timesteps 3465.
Path 331 | total_timesteps 3481.
Path 332 | total_timesteps 3490.
Path 333 | total_timesteps 3503.
Path 334 | total_timesteps 3519.
Path 335 | total_timesteps 3530.
Path 336 | total_timesteps 3539.
Path 337 | total_timesteps 3549.
Path 338 | total_timesteps 3558.
Path 339 | total_timesteps 3571.
Path 340 | total_timesteps 3580.
Path 341 | total_timesteps 3591.
Path 342 | total_timesteps 3599.
Path 343 | total_timesteps 3611.
Path 344 | total_timesteps 3618.
Path 345 | total_timesteps 3633.
Path 346 | total_timesteps 3639.
Path 347 | total_timesteps 3646.
Path 348 | total_timesteps 3657.
Path 349 | total_timesteps 3665.
Path 350 | total_timesteps 3680.
Path 351 | total_timesteps 3691.
Path 352 | total_timesteps 3700.
Path 353 | total_timesteps 3708.
Path 354 | total_timesteps 3718.
Path 355 | total_timesteps 3734.
Path 356 | total_timesteps 3745.
Path 357 | total_timesteps 3753.
Path 358 | total_timesteps 3773.
Path 359 | total_timesteps 3784.
Path 360 | total_timesteps 3791.
Path 361 | total_timesteps 3800.
Path 362 | total_timesteps 3807.
Path 363 | total_timesteps 3817.
Path 364 | total_timesteps 3826.
Path 365 | total_timesteps 3839.
Path 366 | total_timesteps 3855.
Path 367 | total_timesteps 3867.
Path 368 | total_timesteps 3875.
Path 369 | total_timesteps 3885.
Path 370 | total_timesteps 3895.
Path 371 | total_timesteps 3904.
Path 372 | total_timesteps 3912.
Path 373 | total_timesteps 3923.
Path 374 | total_timesteps 3934.
Path 375 | total_timesteps 3945.
Path 376 | total_timesteps 3956.
Path 377 | total_timesteps 3968.
Path 378 | total_timesteps 3987.
Path 379 | total_timesteps 3998.
Path 380 | total_timesteps 4007.
Path 381 | total_timesteps 4022.
Path 382 | total_timesteps 4029.
Path 383 | total_timesteps 4038.
Path 384 | total_timesteps 4047.
Path 385 | total_timesteps 4056.
Path 386 | total_timesteps 4066.
Path 387 | total_timesteps 4076.
Path 388 | total_timesteps 4085.
Path 389 | total_timesteps 4096.
Path 390 | total_timesteps 4106.
Path 391 | total_timesteps 4115.
Path 392 | total_timesteps 4124.
Path 393 | total_timesteps 4134.
Path 394 | total_timesteps 4142.
Path 395 | total_timesteps 4152.
Path 396 | total_timesteps 4168.
Path 397 | total_timesteps 4178.
Path 398 | total_timesteps 4186.
Path 399 | total_timesteps 4194.
Path 400 | total_timesteps 4201.
Path 401 | total_timesteps 4208.
Path 402 | total_timesteps 4215.
Path 403 | total_timesteps 4231.
Path 404 | total_timesteps 4239.
Path 405 | total_timesteps 4248.
Path 406 | total_timesteps 4257.
Path 407 | total_timesteps 4268.
Path 408 | total_timesteps 4275.
Path 409 | total_timesteps 4287.
Path 410 | total_timesteps 4297.
Path 411 | total_timesteps 4309.
Path 412 | total_timesteps 4320.
Path 413 | total_timesteps 4331.
Path 414 | total_timesteps 4340.
Path 415 | total_timesteps 4347.
Path 416 | total_timesteps 4355.
Path 417 | total_timesteps 4367.
Path 418 | total_timesteps 4375.
Path 419 | total_timesteps 4382.
Path 420 | total_timesteps 4393.
Path 421 | total_timesteps 4402.
Path 422 | total_timesteps 4409.
Path 423 | total_timesteps 4420.
Path 424 | total_timesteps 4428.
Path 425 | total_timesteps 4439.
Path 426 | total_timesteps 4447.
Path 427 | total_timesteps 4459.
Path 428 | total_timesteps 4471.
Path 429 | total_timesteps 4481.
Path 430 | total_timesteps 4489.
Path 431 | total_timesteps 4498.
Path 432 | total_timesteps 4507.
Path 433 | total_timesteps 4518.
Path 434 | total_timesteps 4528.
Path 435 | total_timesteps 4537.
Path 436 | total_timesteps 4547.
Path 437 | total_timesteps 4555.
Path 438 | total_timesteps 4564.
Path 439 | total_timesteps 4583.
Path 440 | total_timesteps 4591.
Path 441 | total_timesteps 4601.
Path 442 | total_timesteps 4612.
Path 443 | total_timesteps 4623.
Path 444 | total_timesteps 4635.
Path 445 | total_timesteps 4649.
Path 446 | total_timesteps 4661.
Path 447 | total_timesteps 4670.
Path 448 | total_timesteps 4679.
Path 449 | total_timesteps 4688.
Path 450 | total_timesteps 4696.
Path 451 | total_timesteps 4705.
Path 452 | total_timesteps 4716.
Path 453 | total_timesteps 4726.
Path 454 | total_timesteps 4742.
Path 455 | total_timesteps 4752.
Path 456 | total_timesteps 4765.
Path 457 | total_timesteps 4774.
Path 458 | total_timesteps 4784.
Path 459 | total_timesteps 4794.
Path 460 | total_timesteps 4802.
Path 461 | total_timesteps 4809.
Path 462 | total_timesteps 4822.
Path 463 | total_timesteps 4832.
Path 464 | total_timesteps 4841.
Path 465 | total_timesteps 4852.
Path 466 | total_timesteps 4865.
Path 467 | total_timesteps 4882.
Path 468 | total_timesteps 4890.
Path 469 | total_timesteps 4903.
Path 470 | total_timesteps 4913.
Path 471 | total_timesteps 4925.
Path 472 | total_timesteps 4936.
Path 473 | total_timesteps 4945.
Path 474 | total_timesteps 4953.
Path 475 | total_timesteps 4964.
Path 476 | total_timesteps 4972.
Path 477 | total_timesteps 4982.
Path 478 | total_timesteps 4991.
Path 479 | total_timesteps 5001.
Path 480 | total_timesteps 5009.
Path 481 | total_timesteps 5019.
Path 482 | total_timesteps 5033.
Path 483 | total_timesteps 5043.
Path 484 | total_timesteps 5050.
Path 485 | total_timesteps 5066.
Path 486 | total_timesteps 5084.
Path 487 | total_timesteps 5091.
Path 488 | total_timesteps 5099.
Path 489 | total_timesteps 5107.
Path 490 | total_timesteps 5116.
Path 491 | total_timesteps 5127.
Path 492 | total_timesteps 5134.
Path 493 | total_timesteps 5142.
Path 494 | total_timesteps 5151.
Path 495 | total_timesteps 5159.
Path 496 | total_timesteps 5169.
Path 497 | total_timesteps 5176.
Path 498 | total_timesteps 5189.
Path 499 | total_timesteps 5201.
Path 500 | total_timesteps 5213.
Path 501 | total_timesteps 5220.
Path 502 | total_timesteps 5228.
Path 503 | total_timesteps 5238.
Path 504 | total_timesteps 5246.
Path 505 | total_timesteps 5256.
Path 506 | total_timesteps 5266.
Path 507 | total_timesteps 5275.
Path 508 | total_timesteps 5286.
Path 509 | total_timesteps 5294.
Path 510 | total_timesteps 5303.
Path 511 | total_timesteps 5312.
Path 512 | total_timesteps 5326.
Path 513 | total_timesteps 5338.
Path 514 | total_timesteps 5349.
Path 515 | total_timesteps 5359.
Path 516 | total_timesteps 5369.
Path 517 | total_timesteps 5380.
Path 518 | total_timesteps 5391.
Path 519 | total_timesteps 5404.
Path 520 | total_timesteps 5416.
Path 521 | total_timesteps 5422.
Path 522 | total_timesteps 5431.
Path 523 | total_timesteps 5444.
Path 524 | total_timesteps 5452.
Path 525 | total_timesteps 5459.
Path 526 | total_timesteps 5469.
Path 527 | total_timesteps 5478.
Path 528 | total_timesteps 5488.
Path 529 | total_timesteps 5497.
Path 530 | total_timesteps 5512.
Path 531 | total_timesteps 5520.
Path 532 | total_timesteps 5532.
Path 533 | total_timesteps 5539.
Path 534 | total_timesteps 5547.
Path 535 | total_timesteps 5556.
Path 536 | total_timesteps 5566.
Path 537 | total_timesteps 5579.
Path 538 | total_timesteps 5589.
Path 539 | total_timesteps 5599.
Path 540 | total_timesteps 5607.
Path 541 | total_timesteps 5617.
Path 542 | total_timesteps 5628.
Path 543 | total_timesteps 5640.
Path 544 | total_timesteps 5652.
Path 545 | total_timesteps 5669.
Path 546 | total_timesteps 5677.
Path 547 | total_timesteps 5685.
Path 548 | total_timesteps 5700.
Path 549 | total_timesteps 5708.
Path 550 | total_timesteps 5717.
Path 551 | total_timesteps 5737.
Path 552 | total_timesteps 5749.
Path 553 | total_timesteps 5758.
Path 554 | total_timesteps 5767.
Path 555 | total_timesteps 5777.
Path 556 | total_timesteps 5789.
Path 557 | total_timesteps 5798.
Path 558 | total_timesteps 5807.
Path 559 | total_timesteps 5818.
Path 560 | total_timesteps 5836.
Path 561 | total_timesteps 5847.
Path 562 | total_timesteps 5854.
Path 563 | total_timesteps 5862.
Path 564 | total_timesteps 5870.
Path 565 | total_timesteps 5878.
Path 566 | total_timesteps 5889.
Path 567 | total_timesteps 5904.
Path 568 | total_timesteps 5913.
Path 569 | total_timesteps 5930.
Path 570 | total_timesteps 5937.
Path 571 | total_timesteps 5950.
Path 572 | total_timesteps 5961.
Path 573 | total_timesteps 5972.
Path 574 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.01    |
| Iteration     | 28       |
| MaximumReturn | 1.74     |
| MinimumReturn | -17.8    |
| TotalSamples  | 120108   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.31672433018684387
Validation loss = 0.31768035888671875
Validation loss = 0.32108160853385925
Validation loss = 0.3204096257686615
Validation loss = 0.32033073902130127
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 43.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 76.
Path 8 | total_timesteps 82.
Path 9 | total_timesteps 93.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 128.
Path 13 | total_timesteps 139.
Path 14 | total_timesteps 154.
Path 15 | total_timesteps 163.
Path 16 | total_timesteps 177.
Path 17 | total_timesteps 185.
Path 18 | total_timesteps 193.
Path 19 | total_timesteps 201.
Path 20 | total_timesteps 211.
Path 21 | total_timesteps 219.
Path 22 | total_timesteps 230.
Path 23 | total_timesteps 241.
Path 24 | total_timesteps 248.
Path 25 | total_timesteps 257.
Path 26 | total_timesteps 266.
Path 27 | total_timesteps 277.
Path 28 | total_timesteps 284.
Path 29 | total_timesteps 292.
Path 30 | total_timesteps 303.
Path 31 | total_timesteps 317.
Path 32 | total_timesteps 324.
Path 33 | total_timesteps 331.
Path 34 | total_timesteps 351.
Path 35 | total_timesteps 360.
Path 36 | total_timesteps 375.
Path 37 | total_timesteps 384.
Path 38 | total_timesteps 392.
Path 39 | total_timesteps 402.
Path 40 | total_timesteps 415.
Path 41 | total_timesteps 424.
Path 42 | total_timesteps 433.
Path 43 | total_timesteps 448.
Path 44 | total_timesteps 456.
Path 45 | total_timesteps 465.
Path 46 | total_timesteps 476.
Path 47 | total_timesteps 487.
Path 48 | total_timesteps 495.
Path 49 | total_timesteps 508.
Path 50 | total_timesteps 522.
Path 51 | total_timesteps 532.
Path 52 | total_timesteps 545.
Path 53 | total_timesteps 551.
Path 54 | total_timesteps 564.
Path 55 | total_timesteps 573.
Path 56 | total_timesteps 580.
Path 57 | total_timesteps 587.
Path 58 | total_timesteps 597.
Path 59 | total_timesteps 609.
Path 60 | total_timesteps 621.
Path 61 | total_timesteps 629.
Path 62 | total_timesteps 638.
Path 63 | total_timesteps 647.
Path 64 | total_timesteps 660.
Path 65 | total_timesteps 669.
Path 66 | total_timesteps 678.
Path 67 | total_timesteps 687.
Path 68 | total_timesteps 700.
Path 69 | total_timesteps 709.
Path 70 | total_timesteps 715.
Path 71 | total_timesteps 726.
Path 72 | total_timesteps 739.
Path 73 | total_timesteps 747.
Path 74 | total_timesteps 758.
Path 75 | total_timesteps 767.
Path 76 | total_timesteps 778.
Path 77 | total_timesteps 793.
Path 78 | total_timesteps 804.
Path 79 | total_timesteps 812.
Path 80 | total_timesteps 825.
Path 81 | total_timesteps 835.
Path 82 | total_timesteps 845.
Path 83 | total_timesteps 860.
Path 84 | total_timesteps 868.
Path 85 | total_timesteps 875.
Path 86 | total_timesteps 886.
Path 87 | total_timesteps 898.
Path 88 | total_timesteps 907.
Path 89 | total_timesteps 915.
Path 90 | total_timesteps 924.
Path 91 | total_timesteps 931.
Path 92 | total_timesteps 941.
Path 93 | total_timesteps 948.
Path 94 | total_timesteps 958.
Path 95 | total_timesteps 970.
Path 96 | total_timesteps 978.
Path 97 | total_timesteps 986.
Path 98 | total_timesteps 993.
Path 99 | total_timesteps 1007.
Path 100 | total_timesteps 1015.
Path 101 | total_timesteps 1026.
Path 102 | total_timesteps 1035.
Path 103 | total_timesteps 1042.
Path 104 | total_timesteps 1054.
Path 105 | total_timesteps 1064.
Path 106 | total_timesteps 1073.
Path 107 | total_timesteps 1082.
Path 108 | total_timesteps 1090.
Path 109 | total_timesteps 1102.
Path 110 | total_timesteps 1113.
Path 111 | total_timesteps 1120.
Path 112 | total_timesteps 1131.
Path 113 | total_timesteps 1142.
Path 114 | total_timesteps 1160.
Path 115 | total_timesteps 1169.
Path 116 | total_timesteps 1177.
Path 117 | total_timesteps 1193.
Path 118 | total_timesteps 1201.
Path 119 | total_timesteps 1209.
Path 120 | total_timesteps 1216.
Path 121 | total_timesteps 1230.
Path 122 | total_timesteps 1239.
Path 123 | total_timesteps 1247.
Path 124 | total_timesteps 1256.
Path 125 | total_timesteps 1266.
Path 126 | total_timesteps 1273.
Path 127 | total_timesteps 1282.
Path 128 | total_timesteps 1290.
Path 129 | total_timesteps 1299.
Path 130 | total_timesteps 1308.
Path 131 | total_timesteps 1327.
Path 132 | total_timesteps 1337.
Path 133 | total_timesteps 1350.
Path 134 | total_timesteps 1366.
Path 135 | total_timesteps 1376.
Path 136 | total_timesteps 1385.
Path 137 | total_timesteps 1404.
Path 138 | total_timesteps 1414.
Path 139 | total_timesteps 1425.
Path 140 | total_timesteps 1438.
Path 141 | total_timesteps 1457.
Path 142 | total_timesteps 1465.
Path 143 | total_timesteps 1474.
Path 144 | total_timesteps 1482.
Path 145 | total_timesteps 1489.
Path 146 | total_timesteps 1498.
Path 147 | total_timesteps 1510.
Path 148 | total_timesteps 1522.
Path 149 | total_timesteps 1530.
Path 150 | total_timesteps 1538.
Path 151 | total_timesteps 1551.
Path 152 | total_timesteps 1562.
Path 153 | total_timesteps 1577.
Path 154 | total_timesteps 1591.
Path 155 | total_timesteps 1599.
Path 156 | total_timesteps 1608.
Path 157 | total_timesteps 1617.
Path 158 | total_timesteps 1628.
Path 159 | total_timesteps 1642.
Path 160 | total_timesteps 1651.
Path 161 | total_timesteps 1664.
Path 162 | total_timesteps 1673.
Path 163 | total_timesteps 1683.
Path 164 | total_timesteps 1691.
Path 165 | total_timesteps 1703.
Path 166 | total_timesteps 1712.
Path 167 | total_timesteps 1724.
Path 168 | total_timesteps 1736.
Path 169 | total_timesteps 1743.
Path 170 | total_timesteps 1751.
Path 171 | total_timesteps 1763.
Path 172 | total_timesteps 1781.
Path 173 | total_timesteps 1791.
Path 174 | total_timesteps 1804.
Path 175 | total_timesteps 1814.
Path 176 | total_timesteps 1825.
Path 177 | total_timesteps 1836.
Path 178 | total_timesteps 1845.
Path 179 | total_timesteps 1856.
Path 180 | total_timesteps 1866.
Path 181 | total_timesteps 1874.
Path 182 | total_timesteps 1884.
Path 183 | total_timesteps 1895.
Path 184 | total_timesteps 1911.
Path 185 | total_timesteps 1920.
Path 186 | total_timesteps 1932.
Path 187 | total_timesteps 1941.
Path 188 | total_timesteps 1949.
Path 189 | total_timesteps 1961.
Path 190 | total_timesteps 1974.
Path 191 | total_timesteps 1982.
Path 192 | total_timesteps 1990.
Path 193 | total_timesteps 1998.
Path 194 | total_timesteps 2007.
Path 195 | total_timesteps 2026.
Path 196 | total_timesteps 2036.
Path 197 | total_timesteps 2045.
Path 198 | total_timesteps 2060.
Path 199 | total_timesteps 2069.
Path 200 | total_timesteps 2080.
Path 201 | total_timesteps 2087.
Path 202 | total_timesteps 2095.
Path 203 | total_timesteps 2103.
Path 204 | total_timesteps 2112.
Path 205 | total_timesteps 2121.
Path 206 | total_timesteps 2131.
Path 207 | total_timesteps 2143.
Path 208 | total_timesteps 2154.
Path 209 | total_timesteps 2167.
Path 210 | total_timesteps 2177.
Path 211 | total_timesteps 2186.
Path 212 | total_timesteps 2194.
Path 213 | total_timesteps 2203.
Path 214 | total_timesteps 2215.
Path 215 | total_timesteps 2224.
Path 216 | total_timesteps 2232.
Path 217 | total_timesteps 2241.
Path 218 | total_timesteps 2250.
Path 219 | total_timesteps 2259.
Path 220 | total_timesteps 2269.
Path 221 | total_timesteps 2281.
Path 222 | total_timesteps 2289.
Path 223 | total_timesteps 2299.
Path 224 | total_timesteps 2306.
Path 225 | total_timesteps 2312.
Path 226 | total_timesteps 2322.
Path 227 | total_timesteps 2334.
Path 228 | total_timesteps 2341.
Path 229 | total_timesteps 2352.
Path 230 | total_timesteps 2364.
Path 231 | total_timesteps 2372.
Path 232 | total_timesteps 2382.
Path 233 | total_timesteps 2389.
Path 234 | total_timesteps 2400.
Path 235 | total_timesteps 2412.
Path 236 | total_timesteps 2421.
Path 237 | total_timesteps 2433.
Path 238 | total_timesteps 2442.
Path 239 | total_timesteps 2454.
Path 240 | total_timesteps 2464.
Path 241 | total_timesteps 2472.
Path 242 | total_timesteps 2483.
Path 243 | total_timesteps 2493.
Path 244 | total_timesteps 2504.
Path 245 | total_timesteps 2512.
Path 246 | total_timesteps 2521.
Path 247 | total_timesteps 2534.
Path 248 | total_timesteps 2544.
Path 249 | total_timesteps 2555.
Path 250 | total_timesteps 2565.
Path 251 | total_timesteps 2574.
Path 252 | total_timesteps 2583.
Path 253 | total_timesteps 2591.
Path 254 | total_timesteps 2602.
Path 255 | total_timesteps 2608.
Path 256 | total_timesteps 2621.
Path 257 | total_timesteps 2632.
Path 258 | total_timesteps 2642.
Path 259 | total_timesteps 2654.
Path 260 | total_timesteps 2670.
Path 261 | total_timesteps 2679.
Path 262 | total_timesteps 2689.
Path 263 | total_timesteps 2698.
Path 264 | total_timesteps 2710.
Path 265 | total_timesteps 2721.
Path 266 | total_timesteps 2732.
Path 267 | total_timesteps 2742.
Path 268 | total_timesteps 2760.
Path 269 | total_timesteps 2769.
Path 270 | total_timesteps 2776.
Path 271 | total_timesteps 2784.
Path 272 | total_timesteps 2791.
Path 273 | total_timesteps 2799.
Path 274 | total_timesteps 2807.
Path 275 | total_timesteps 2817.
Path 276 | total_timesteps 2825.
Path 277 | total_timesteps 2835.
Path 278 | total_timesteps 2843.
Path 279 | total_timesteps 2856.
Path 280 | total_timesteps 2864.
Path 281 | total_timesteps 2871.
Path 282 | total_timesteps 2881.
Path 283 | total_timesteps 2890.
Path 284 | total_timesteps 2901.
Path 285 | total_timesteps 2911.
Path 286 | total_timesteps 2922.
Path 287 | total_timesteps 2931.
Path 288 | total_timesteps 2940.
Path 289 | total_timesteps 2950.
Path 290 | total_timesteps 2959.
Path 291 | total_timesteps 2968.
Path 292 | total_timesteps 2984.
Path 293 | total_timesteps 2993.
Path 294 | total_timesteps 3002.
Path 295 | total_timesteps 3013.
Path 296 | total_timesteps 3021.
Path 297 | total_timesteps 3031.
Path 298 | total_timesteps 3041.
Path 299 | total_timesteps 3061.
Path 300 | total_timesteps 3077.
Path 301 | total_timesteps 3092.
Path 302 | total_timesteps 3099.
Path 303 | total_timesteps 3111.
Path 304 | total_timesteps 3119.
Path 305 | total_timesteps 3134.
Path 306 | total_timesteps 3145.
Path 307 | total_timesteps 3159.
Path 308 | total_timesteps 3168.
Path 309 | total_timesteps 3182.
Path 310 | total_timesteps 3192.
Path 311 | total_timesteps 3202.
Path 312 | total_timesteps 3212.
Path 313 | total_timesteps 3221.
Path 314 | total_timesteps 3230.
Path 315 | total_timesteps 3241.
Path 316 | total_timesteps 3252.
Path 317 | total_timesteps 3266.
Path 318 | total_timesteps 3278.
Path 319 | total_timesteps 3288.
Path 320 | total_timesteps 3297.
Path 321 | total_timesteps 3308.
Path 322 | total_timesteps 3318.
Path 323 | total_timesteps 3334.
Path 324 | total_timesteps 3347.
Path 325 | total_timesteps 3359.
Path 326 | total_timesteps 3371.
Path 327 | total_timesteps 3383.
Path 328 | total_timesteps 3396.
Path 329 | total_timesteps 3405.
Path 330 | total_timesteps 3413.
Path 331 | total_timesteps 3424.
Path 332 | total_timesteps 3432.
Path 333 | total_timesteps 3444.
Path 334 | total_timesteps 3454.
Path 335 | total_timesteps 3465.
Path 336 | total_timesteps 3485.
Path 337 | total_timesteps 3503.
Path 338 | total_timesteps 3516.
Path 339 | total_timesteps 3526.
Path 340 | total_timesteps 3532.
Path 341 | total_timesteps 3547.
Path 342 | total_timesteps 3556.
Path 343 | total_timesteps 3570.
Path 344 | total_timesteps 3577.
Path 345 | total_timesteps 3584.
Path 346 | total_timesteps 3596.
Path 347 | total_timesteps 3608.
Path 348 | total_timesteps 3621.
Path 349 | total_timesteps 3630.
Path 350 | total_timesteps 3642.
Path 351 | total_timesteps 3653.
Path 352 | total_timesteps 3661.
Path 353 | total_timesteps 3670.
Path 354 | total_timesteps 3680.
Path 355 | total_timesteps 3690.
Path 356 | total_timesteps 3703.
Path 357 | total_timesteps 3713.
Path 358 | total_timesteps 3724.
Path 359 | total_timesteps 3735.
Path 360 | total_timesteps 3743.
Path 361 | total_timesteps 3753.
Path 362 | total_timesteps 3764.
Path 363 | total_timesteps 3774.
Path 364 | total_timesteps 3786.
Path 365 | total_timesteps 3796.
Path 366 | total_timesteps 3805.
Path 367 | total_timesteps 3814.
Path 368 | total_timesteps 3824.
Path 369 | total_timesteps 3835.
Path 370 | total_timesteps 3847.
Path 371 | total_timesteps 3858.
Path 372 | total_timesteps 3867.
Path 373 | total_timesteps 3875.
Path 374 | total_timesteps 3888.
Path 375 | total_timesteps 3900.
Path 376 | total_timesteps 3908.
Path 377 | total_timesteps 3921.
Path 378 | total_timesteps 3932.
Path 379 | total_timesteps 3945.
Path 380 | total_timesteps 3953.
Path 381 | total_timesteps 3967.
Path 382 | total_timesteps 3978.
Path 383 | total_timesteps 3986.
Path 384 | total_timesteps 4000.
Path 385 | total_timesteps 4010.
Path 386 | total_timesteps 4025.
Path 387 | total_timesteps 4035.
Path 388 | total_timesteps 4046.
Path 389 | total_timesteps 4056.
Path 390 | total_timesteps 4074.
Path 391 | total_timesteps 4086.
Path 392 | total_timesteps 4097.
Path 393 | total_timesteps 4112.
Path 394 | total_timesteps 4126.
Path 395 | total_timesteps 4138.
Path 396 | total_timesteps 4148.
Path 397 | total_timesteps 4156.
Path 398 | total_timesteps 4169.
Path 399 | total_timesteps 4180.
Path 400 | total_timesteps 4192.
Path 401 | total_timesteps 4201.
Path 402 | total_timesteps 4210.
Path 403 | total_timesteps 4217.
Path 404 | total_timesteps 4230.
Path 405 | total_timesteps 4242.
Path 406 | total_timesteps 4251.
Path 407 | total_timesteps 4259.
Path 408 | total_timesteps 4269.
Path 409 | total_timesteps 4277.
Path 410 | total_timesteps 4289.
Path 411 | total_timesteps 4301.
Path 412 | total_timesteps 4312.
Path 413 | total_timesteps 4324.
Path 414 | total_timesteps 4334.
Path 415 | total_timesteps 4345.
Path 416 | total_timesteps 4356.
Path 417 | total_timesteps 4366.
Path 418 | total_timesteps 4377.
Path 419 | total_timesteps 4385.
Path 420 | total_timesteps 4395.
Path 421 | total_timesteps 4402.
Path 422 | total_timesteps 4414.
Path 423 | total_timesteps 4425.
Path 424 | total_timesteps 4434.
Path 425 | total_timesteps 4445.
Path 426 | total_timesteps 4452.
Path 427 | total_timesteps 4467.
Path 428 | total_timesteps 4479.
Path 429 | total_timesteps 4489.
Path 430 | total_timesteps 4499.
Path 431 | total_timesteps 4511.
Path 432 | total_timesteps 4520.
Path 433 | total_timesteps 4528.
Path 434 | total_timesteps 4538.
Path 435 | total_timesteps 4549.
Path 436 | total_timesteps 4557.
Path 437 | total_timesteps 4566.
Path 438 | total_timesteps 4578.
Path 439 | total_timesteps 4589.
Path 440 | total_timesteps 4597.
Path 441 | total_timesteps 4605.
Path 442 | total_timesteps 4613.
Path 443 | total_timesteps 4621.
Path 444 | total_timesteps 4628.
Path 445 | total_timesteps 4639.
Path 446 | total_timesteps 4654.
Path 447 | total_timesteps 4664.
Path 448 | total_timesteps 4674.
Path 449 | total_timesteps 4681.
Path 450 | total_timesteps 4691.
Path 451 | total_timesteps 4703.
Path 452 | total_timesteps 4714.
Path 453 | total_timesteps 4724.
Path 454 | total_timesteps 4736.
Path 455 | total_timesteps 4750.
Path 456 | total_timesteps 4758.
Path 457 | total_timesteps 4766.
Path 458 | total_timesteps 4775.
Path 459 | total_timesteps 4782.
Path 460 | total_timesteps 4791.
Path 461 | total_timesteps 4800.
Path 462 | total_timesteps 4813.
Path 463 | total_timesteps 4820.
Path 464 | total_timesteps 4832.
Path 465 | total_timesteps 4841.
Path 466 | total_timesteps 4851.
Path 467 | total_timesteps 4861.
Path 468 | total_timesteps 4873.
Path 469 | total_timesteps 4882.
Path 470 | total_timesteps 4893.
Path 471 | total_timesteps 4901.
Path 472 | total_timesteps 4913.
Path 473 | total_timesteps 4921.
Path 474 | total_timesteps 4937.
Path 475 | total_timesteps 4946.
Path 476 | total_timesteps 4955.
Path 477 | total_timesteps 4968.
Path 478 | total_timesteps 4975.
Path 479 | total_timesteps 4984.
Path 480 | total_timesteps 4993.
Path 481 | total_timesteps 5005.
Path 482 | total_timesteps 5016.
Path 483 | total_timesteps 5025.
Path 484 | total_timesteps 5033.
Path 485 | total_timesteps 5043.
Path 486 | total_timesteps 5055.
Path 487 | total_timesteps 5065.
Path 488 | total_timesteps 5079.
Path 489 | total_timesteps 5087.
Path 490 | total_timesteps 5099.
Path 491 | total_timesteps 5108.
Path 492 | total_timesteps 5121.
Path 493 | total_timesteps 5130.
Path 494 | total_timesteps 5138.
Path 495 | total_timesteps 5146.
Path 496 | total_timesteps 5159.
Path 497 | total_timesteps 5169.
Path 498 | total_timesteps 5178.
Path 499 | total_timesteps 5185.
Path 500 | total_timesteps 5194.
Path 501 | total_timesteps 5206.
Path 502 | total_timesteps 5213.
Path 503 | total_timesteps 5223.
Path 504 | total_timesteps 5237.
Path 505 | total_timesteps 5249.
Path 506 | total_timesteps 5265.
Path 507 | total_timesteps 5279.
Path 508 | total_timesteps 5296.
Path 509 | total_timesteps 5305.
Path 510 | total_timesteps 5315.
Path 511 | total_timesteps 5324.
Path 512 | total_timesteps 5335.
Path 513 | total_timesteps 5346.
Path 514 | total_timesteps 5367.
Path 515 | total_timesteps 5377.
Path 516 | total_timesteps 5395.
Path 517 | total_timesteps 5406.
Path 518 | total_timesteps 5415.
Path 519 | total_timesteps 5431.
Path 520 | total_timesteps 5441.
Path 521 | total_timesteps 5451.
Path 522 | total_timesteps 5461.
Path 523 | total_timesteps 5474.
Path 524 | total_timesteps 5487.
Path 525 | total_timesteps 5495.
Path 526 | total_timesteps 5504.
Path 527 | total_timesteps 5511.
Path 528 | total_timesteps 5521.
Path 529 | total_timesteps 5536.
Path 530 | total_timesteps 5544.
Path 531 | total_timesteps 5553.
Path 532 | total_timesteps 5563.
Path 533 | total_timesteps 5572.
Path 534 | total_timesteps 5582.
Path 535 | total_timesteps 5596.
Path 536 | total_timesteps 5603.
Path 537 | total_timesteps 5616.
Path 538 | total_timesteps 5628.
Path 539 | total_timesteps 5637.
Path 540 | total_timesteps 5649.
Path 541 | total_timesteps 5661.
Path 542 | total_timesteps 5671.
Path 543 | total_timesteps 5683.
Path 544 | total_timesteps 5693.
Path 545 | total_timesteps 5702.
Path 546 | total_timesteps 5715.
Path 547 | total_timesteps 5727.
Path 548 | total_timesteps 5739.
Path 549 | total_timesteps 5751.
Path 550 | total_timesteps 5769.
Path 551 | total_timesteps 5778.
Path 552 | total_timesteps 5793.
Path 553 | total_timesteps 5803.
Path 554 | total_timesteps 5812.
Path 555 | total_timesteps 5821.
Path 556 | total_timesteps 5830.
Path 557 | total_timesteps 5838.
Path 558 | total_timesteps 5849.
Path 559 | total_timesteps 5863.
Path 560 | total_timesteps 5873.
Path 561 | total_timesteps 5883.
Path 562 | total_timesteps 5892.
Path 563 | total_timesteps 5902.
Path 564 | total_timesteps 5913.
Path 565 | total_timesteps 5925.
Path 566 | total_timesteps 5933.
Path 567 | total_timesteps 5944.
Path 568 | total_timesteps 5956.
Path 569 | total_timesteps 5965.
Path 570 | total_timesteps 5975.
Path 571 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.89    |
| Iteration     | 29       |
| MaximumReturn | 1.84     |
| MinimumReturn | -19      |
| TotalSamples  | 124108   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.3151022493839264
Validation loss = 0.31813353300094604
Validation loss = 0.3195250332355499
Validation loss = 0.31856292486190796
Validation loss = 0.31926825642585754
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 66.
Path 8 | total_timesteps 75.
Path 9 | total_timesteps 85.
Path 10 | total_timesteps 92.
Path 11 | total_timesteps 102.
Path 12 | total_timesteps 115.
Path 13 | total_timesteps 124.
Path 14 | total_timesteps 136.
Path 15 | total_timesteps 146.
Path 16 | total_timesteps 157.
Path 17 | total_timesteps 167.
Path 18 | total_timesteps 183.
Path 19 | total_timesteps 193.
Path 20 | total_timesteps 200.
Path 21 | total_timesteps 211.
Path 22 | total_timesteps 220.
Path 23 | total_timesteps 229.
Path 24 | total_timesteps 240.
Path 25 | total_timesteps 249.
Path 26 | total_timesteps 257.
Path 27 | total_timesteps 265.
Path 28 | total_timesteps 276.
Path 29 | total_timesteps 292.
Path 30 | total_timesteps 304.
Path 31 | total_timesteps 317.
Path 32 | total_timesteps 324.
Path 33 | total_timesteps 334.
Path 34 | total_timesteps 344.
Path 35 | total_timesteps 353.
Path 36 | total_timesteps 362.
Path 37 | total_timesteps 372.
Path 38 | total_timesteps 387.
Path 39 | total_timesteps 399.
Path 40 | total_timesteps 408.
Path 41 | total_timesteps 416.
Path 42 | total_timesteps 427.
Path 43 | total_timesteps 443.
Path 44 | total_timesteps 453.
Path 45 | total_timesteps 467.
Path 46 | total_timesteps 477.
Path 47 | total_timesteps 489.
Path 48 | total_timesteps 498.
Path 49 | total_timesteps 506.
Path 50 | total_timesteps 527.
Path 51 | total_timesteps 537.
Path 52 | total_timesteps 546.
Path 53 | total_timesteps 555.
Path 54 | total_timesteps 566.
Path 55 | total_timesteps 577.
Path 56 | total_timesteps 585.
Path 57 | total_timesteps 594.
Path 58 | total_timesteps 607.
Path 59 | total_timesteps 618.
Path 60 | total_timesteps 638.
Path 61 | total_timesteps 652.
Path 62 | total_timesteps 665.
Path 63 | total_timesteps 685.
Path 64 | total_timesteps 695.
Path 65 | total_timesteps 703.
Path 66 | total_timesteps 716.
Path 67 | total_timesteps 724.
Path 68 | total_timesteps 737.
Path 69 | total_timesteps 763.
Path 70 | total_timesteps 772.
Path 71 | total_timesteps 784.
Path 72 | total_timesteps 793.
Path 73 | total_timesteps 801.
Path 74 | total_timesteps 810.
Path 75 | total_timesteps 819.
Path 76 | total_timesteps 830.
Path 77 | total_timesteps 838.
Path 78 | total_timesteps 848.
Path 79 | total_timesteps 859.
Path 80 | total_timesteps 872.
Path 81 | total_timesteps 885.
Path 82 | total_timesteps 900.
Path 83 | total_timesteps 912.
Path 84 | total_timesteps 925.
Path 85 | total_timesteps 935.
Path 86 | total_timesteps 944.
Path 87 | total_timesteps 952.
Path 88 | total_timesteps 964.
Path 89 | total_timesteps 976.
Path 90 | total_timesteps 987.
Path 91 | total_timesteps 996.
Path 92 | total_timesteps 1004.
Path 93 | total_timesteps 1018.
Path 94 | total_timesteps 1028.
Path 95 | total_timesteps 1044.
Path 96 | total_timesteps 1053.
Path 97 | total_timesteps 1060.
Path 98 | total_timesteps 1075.
Path 99 | total_timesteps 1085.
Path 100 | total_timesteps 1095.
Path 101 | total_timesteps 1104.
Path 102 | total_timesteps 1113.
Path 103 | total_timesteps 1135.
Path 104 | total_timesteps 1144.
Path 105 | total_timesteps 1154.
Path 106 | total_timesteps 1162.
Path 107 | total_timesteps 1172.
Path 108 | total_timesteps 1179.
Path 109 | total_timesteps 1188.
Path 110 | total_timesteps 1198.
Path 111 | total_timesteps 1207.
Path 112 | total_timesteps 1217.
Path 113 | total_timesteps 1230.
Path 114 | total_timesteps 1238.
Path 115 | total_timesteps 1246.
Path 116 | total_timesteps 1252.
Path 117 | total_timesteps 1263.
Path 118 | total_timesteps 1272.
Path 119 | total_timesteps 1282.
Path 120 | total_timesteps 1291.
Path 121 | total_timesteps 1299.
Path 122 | total_timesteps 1307.
Path 123 | total_timesteps 1315.
Path 124 | total_timesteps 1323.
Path 125 | total_timesteps 1336.
Path 126 | total_timesteps 1349.
Path 127 | total_timesteps 1358.
Path 128 | total_timesteps 1371.
Path 129 | total_timesteps 1381.
Path 130 | total_timesteps 1394.
Path 131 | total_timesteps 1401.
Path 132 | total_timesteps 1409.
Path 133 | total_timesteps 1421.
Path 134 | total_timesteps 1436.
Path 135 | total_timesteps 1447.
Path 136 | total_timesteps 1456.
Path 137 | total_timesteps 1465.
Path 138 | total_timesteps 1475.
Path 139 | total_timesteps 1489.
Path 140 | total_timesteps 1500.
Path 141 | total_timesteps 1509.
Path 142 | total_timesteps 1528.
Path 143 | total_timesteps 1537.
Path 144 | total_timesteps 1547.
Path 145 | total_timesteps 1555.
Path 146 | total_timesteps 1565.
Path 147 | total_timesteps 1581.
Path 148 | total_timesteps 1596.
Path 149 | total_timesteps 1607.
Path 150 | total_timesteps 1618.
Path 151 | total_timesteps 1626.
Path 152 | total_timesteps 1635.
Path 153 | total_timesteps 1644.
Path 154 | total_timesteps 1653.
Path 155 | total_timesteps 1668.
Path 156 | total_timesteps 1676.
Path 157 | total_timesteps 1684.
Path 158 | total_timesteps 1696.
Path 159 | total_timesteps 1704.
Path 160 | total_timesteps 1716.
Path 161 | total_timesteps 1731.
Path 162 | total_timesteps 1743.
Path 163 | total_timesteps 1752.
Path 164 | total_timesteps 1763.
Path 165 | total_timesteps 1779.
Path 166 | total_timesteps 1786.
Path 167 | total_timesteps 1800.
Path 168 | total_timesteps 1807.
Path 169 | total_timesteps 1815.
Path 170 | total_timesteps 1825.
Path 171 | total_timesteps 1833.
Path 172 | total_timesteps 1843.
Path 173 | total_timesteps 1859.
Path 174 | total_timesteps 1880.
Path 175 | total_timesteps 1889.
Path 176 | total_timesteps 1900.
Path 177 | total_timesteps 1916.
Path 178 | total_timesteps 1930.
Path 179 | total_timesteps 1940.
Path 180 | total_timesteps 1949.
Path 181 | total_timesteps 1958.
Path 182 | total_timesteps 1966.
Path 183 | total_timesteps 1979.
Path 184 | total_timesteps 1988.
Path 185 | total_timesteps 1997.
Path 186 | total_timesteps 2011.
Path 187 | total_timesteps 2026.
Path 188 | total_timesteps 2035.
Path 189 | total_timesteps 2044.
Path 190 | total_timesteps 2057.
Path 191 | total_timesteps 2066.
Path 192 | total_timesteps 2086.
Path 193 | total_timesteps 2093.
Path 194 | total_timesteps 2113.
Path 195 | total_timesteps 2123.
Path 196 | total_timesteps 2133.
Path 197 | total_timesteps 2143.
Path 198 | total_timesteps 2162.
Path 199 | total_timesteps 2172.
Path 200 | total_timesteps 2182.
Path 201 | total_timesteps 2194.
Path 202 | total_timesteps 2204.
Path 203 | total_timesteps 2213.
Path 204 | total_timesteps 2223.
Path 205 | total_timesteps 2233.
Path 206 | total_timesteps 2241.
Path 207 | total_timesteps 2251.
Path 208 | total_timesteps 2259.
Path 209 | total_timesteps 2267.
Path 210 | total_timesteps 2280.
Path 211 | total_timesteps 2290.
Path 212 | total_timesteps 2297.
Path 213 | total_timesteps 2305.
Path 214 | total_timesteps 2315.
Path 215 | total_timesteps 2322.
Path 216 | total_timesteps 2330.
Path 217 | total_timesteps 2337.
Path 218 | total_timesteps 2345.
Path 219 | total_timesteps 2353.
Path 220 | total_timesteps 2364.
Path 221 | total_timesteps 2378.
Path 222 | total_timesteps 2385.
Path 223 | total_timesteps 2398.
Path 224 | total_timesteps 2405.
Path 225 | total_timesteps 2414.
Path 226 | total_timesteps 2422.
Path 227 | total_timesteps 2434.
Path 228 | total_timesteps 2446.
Path 229 | total_timesteps 2453.
Path 230 | total_timesteps 2469.
Path 231 | total_timesteps 2476.
Path 232 | total_timesteps 2484.
Path 233 | total_timesteps 2493.
Path 234 | total_timesteps 2504.
Path 235 | total_timesteps 2514.
Path 236 | total_timesteps 2521.
Path 237 | total_timesteps 2530.
Path 238 | total_timesteps 2540.
Path 239 | total_timesteps 2550.
Path 240 | total_timesteps 2560.
Path 241 | total_timesteps 2570.
Path 242 | total_timesteps 2578.
Path 243 | total_timesteps 2587.
Path 244 | total_timesteps 2596.
Path 245 | total_timesteps 2607.
Path 246 | total_timesteps 2614.
Path 247 | total_timesteps 2626.
Path 248 | total_timesteps 2635.
Path 249 | total_timesteps 2652.
Path 250 | total_timesteps 2661.
Path 251 | total_timesteps 2669.
Path 252 | total_timesteps 2687.
Path 253 | total_timesteps 2695.
Path 254 | total_timesteps 2706.
Path 255 | total_timesteps 2715.
Path 256 | total_timesteps 2724.
Path 257 | total_timesteps 2733.
Path 258 | total_timesteps 2740.
Path 259 | total_timesteps 2752.
Path 260 | total_timesteps 2763.
Path 261 | total_timesteps 2773.
Path 262 | total_timesteps 2782.
Path 263 | total_timesteps 2790.
Path 264 | total_timesteps 2802.
Path 265 | total_timesteps 2813.
Path 266 | total_timesteps 2822.
Path 267 | total_timesteps 2837.
Path 268 | total_timesteps 2847.
Path 269 | total_timesteps 2862.
Path 270 | total_timesteps 2875.
Path 271 | total_timesteps 2884.
Path 272 | total_timesteps 2896.
Path 273 | total_timesteps 2908.
Path 274 | total_timesteps 2918.
Path 275 | total_timesteps 2926.
Path 276 | total_timesteps 2936.
Path 277 | total_timesteps 2947.
Path 278 | total_timesteps 2955.
Path 279 | total_timesteps 2964.
Path 280 | total_timesteps 2973.
Path 281 | total_timesteps 2980.
Path 282 | total_timesteps 2991.
Path 283 | total_timesteps 2999.
Path 284 | total_timesteps 3014.
Path 285 | total_timesteps 3025.
Path 286 | total_timesteps 3035.
Path 287 | total_timesteps 3046.
Path 288 | total_timesteps 3055.
Path 289 | total_timesteps 3066.
Path 290 | total_timesteps 3076.
Path 291 | total_timesteps 3089.
Path 292 | total_timesteps 3108.
Path 293 | total_timesteps 3119.
Path 294 | total_timesteps 3130.
Path 295 | total_timesteps 3140.
Path 296 | total_timesteps 3147.
Path 297 | total_timesteps 3155.
Path 298 | total_timesteps 3166.
Path 299 | total_timesteps 3176.
Path 300 | total_timesteps 3186.
Path 301 | total_timesteps 3194.
Path 302 | total_timesteps 3203.
Path 303 | total_timesteps 3220.
Path 304 | total_timesteps 3232.
Path 305 | total_timesteps 3248.
Path 306 | total_timesteps 3257.
Path 307 | total_timesteps 3265.
Path 308 | total_timesteps 3273.
Path 309 | total_timesteps 3281.
Path 310 | total_timesteps 3293.
Path 311 | total_timesteps 3302.
Path 312 | total_timesteps 3312.
Path 313 | total_timesteps 3336.
Path 314 | total_timesteps 3347.
Path 315 | total_timesteps 3364.
Path 316 | total_timesteps 3375.
Path 317 | total_timesteps 3383.
Path 318 | total_timesteps 3394.
Path 319 | total_timesteps 3405.
Path 320 | total_timesteps 3427.
Path 321 | total_timesteps 3436.
Path 322 | total_timesteps 3448.
Path 323 | total_timesteps 3458.
Path 324 | total_timesteps 3467.
Path 325 | total_timesteps 3475.
Path 326 | total_timesteps 3485.
Path 327 | total_timesteps 3494.
Path 328 | total_timesteps 3507.
Path 329 | total_timesteps 3517.
Path 330 | total_timesteps 3524.
Path 331 | total_timesteps 3532.
Path 332 | total_timesteps 3540.
Path 333 | total_timesteps 3551.
Path 334 | total_timesteps 3563.
Path 335 | total_timesteps 3573.
Path 336 | total_timesteps 3586.
Path 337 | total_timesteps 3598.
Path 338 | total_timesteps 3609.
Path 339 | total_timesteps 3615.
Path 340 | total_timesteps 3628.
Path 341 | total_timesteps 3639.
Path 342 | total_timesteps 3652.
Path 343 | total_timesteps 3664.
Path 344 | total_timesteps 3673.
Path 345 | total_timesteps 3683.
Path 346 | total_timesteps 3697.
Path 347 | total_timesteps 3713.
Path 348 | total_timesteps 3724.
Path 349 | total_timesteps 3731.
Path 350 | total_timesteps 3740.
Path 351 | total_timesteps 3751.
Path 352 | total_timesteps 3762.
Path 353 | total_timesteps 3773.
Path 354 | total_timesteps 3784.
Path 355 | total_timesteps 3795.
Path 356 | total_timesteps 3806.
Path 357 | total_timesteps 3820.
Path 358 | total_timesteps 3837.
Path 359 | total_timesteps 3849.
Path 360 | total_timesteps 3860.
Path 361 | total_timesteps 3873.
Path 362 | total_timesteps 3885.
Path 363 | total_timesteps 3894.
Path 364 | total_timesteps 3904.
Path 365 | total_timesteps 3914.
Path 366 | total_timesteps 3923.
Path 367 | total_timesteps 3933.
Path 368 | total_timesteps 3941.
Path 369 | total_timesteps 3950.
Path 370 | total_timesteps 3963.
Path 371 | total_timesteps 3969.
Path 372 | total_timesteps 3982.
Path 373 | total_timesteps 3989.
Path 374 | total_timesteps 3998.
Path 375 | total_timesteps 4010.
Path 376 | total_timesteps 4022.
Path 377 | total_timesteps 4032.
Path 378 | total_timesteps 4043.
Path 379 | total_timesteps 4055.
Path 380 | total_timesteps 4066.
Path 381 | total_timesteps 4075.
Path 382 | total_timesteps 4087.
Path 383 | total_timesteps 4097.
Path 384 | total_timesteps 4108.
Path 385 | total_timesteps 4116.
Path 386 | total_timesteps 4132.
Path 387 | total_timesteps 4145.
Path 388 | total_timesteps 4155.
Path 389 | total_timesteps 4164.
Path 390 | total_timesteps 4180.
Path 391 | total_timesteps 4191.
Path 392 | total_timesteps 4202.
Path 393 | total_timesteps 4209.
Path 394 | total_timesteps 4218.
Path 395 | total_timesteps 4225.
Path 396 | total_timesteps 4240.
Path 397 | total_timesteps 4250.
Path 398 | total_timesteps 4263.
Path 399 | total_timesteps 4274.
Path 400 | total_timesteps 4283.
Path 401 | total_timesteps 4296.
Path 402 | total_timesteps 4305.
Path 403 | total_timesteps 4315.
Path 404 | total_timesteps 4322.
Path 405 | total_timesteps 4332.
Path 406 | total_timesteps 4342.
Path 407 | total_timesteps 4351.
Path 408 | total_timesteps 4361.
Path 409 | total_timesteps 4370.
Path 410 | total_timesteps 4380.
Path 411 | total_timesteps 4391.
Path 412 | total_timesteps 4404.
Path 413 | total_timesteps 4414.
Path 414 | total_timesteps 4423.
Path 415 | total_timesteps 4430.
Path 416 | total_timesteps 4446.
Path 417 | total_timesteps 4457.
Path 418 | total_timesteps 4465.
Path 419 | total_timesteps 4474.
Path 420 | total_timesteps 4485.
Path 421 | total_timesteps 4498.
Path 422 | total_timesteps 4509.
Path 423 | total_timesteps 4522.
Path 424 | total_timesteps 4530.
Path 425 | total_timesteps 4545.
Path 426 | total_timesteps 4557.
Path 427 | total_timesteps 4568.
Path 428 | total_timesteps 4577.
Path 429 | total_timesteps 4588.
Path 430 | total_timesteps 4595.
Path 431 | total_timesteps 4608.
Path 432 | total_timesteps 4617.
Path 433 | total_timesteps 4627.
Path 434 | total_timesteps 4636.
Path 435 | total_timesteps 4652.
Path 436 | total_timesteps 4668.
Path 437 | total_timesteps 4677.
Path 438 | total_timesteps 4687.
Path 439 | total_timesteps 4695.
Path 440 | total_timesteps 4707.
Path 441 | total_timesteps 4716.
Path 442 | total_timesteps 4726.
Path 443 | total_timesteps 4737.
Path 444 | total_timesteps 4747.
Path 445 | total_timesteps 4756.
Path 446 | total_timesteps 4766.
Path 447 | total_timesteps 4781.
Path 448 | total_timesteps 4789.
Path 449 | total_timesteps 4803.
Path 450 | total_timesteps 4812.
Path 451 | total_timesteps 4823.
Path 452 | total_timesteps 4834.
Path 453 | total_timesteps 4847.
Path 454 | total_timesteps 4856.
Path 455 | total_timesteps 4864.
Path 456 | total_timesteps 4877.
Path 457 | total_timesteps 4885.
Path 458 | total_timesteps 4894.
Path 459 | total_timesteps 4904.
Path 460 | total_timesteps 4912.
Path 461 | total_timesteps 4930.
Path 462 | total_timesteps 4939.
Path 463 | total_timesteps 4949.
Path 464 | total_timesteps 4961.
Path 465 | total_timesteps 4970.
Path 466 | total_timesteps 4996.
Path 467 | total_timesteps 5008.
Path 468 | total_timesteps 5018.
Path 469 | total_timesteps 5033.
Path 470 | total_timesteps 5048.
Path 471 | total_timesteps 5056.
Path 472 | total_timesteps 5066.
Path 473 | total_timesteps 5082.
Path 474 | total_timesteps 5099.
Path 475 | total_timesteps 5108.
Path 476 | total_timesteps 5118.
Path 477 | total_timesteps 5146.
Path 478 | total_timesteps 5160.
Path 479 | total_timesteps 5170.
Path 480 | total_timesteps 5179.
Path 481 | total_timesteps 5188.
Path 482 | total_timesteps 5203.
Path 483 | total_timesteps 5212.
Path 484 | total_timesteps 5221.
Path 485 | total_timesteps 5234.
Path 486 | total_timesteps 5244.
Path 487 | total_timesteps 5253.
Path 488 | total_timesteps 5263.
Path 489 | total_timesteps 5272.
Path 490 | total_timesteps 5279.
Path 491 | total_timesteps 5289.
Path 492 | total_timesteps 5299.
Path 493 | total_timesteps 5307.
Path 494 | total_timesteps 5317.
Path 495 | total_timesteps 5325.
Path 496 | total_timesteps 5332.
Path 497 | total_timesteps 5345.
Path 498 | total_timesteps 5362.
Path 499 | total_timesteps 5373.
Path 500 | total_timesteps 5383.
Path 501 | total_timesteps 5396.
Path 502 | total_timesteps 5405.
Path 503 | total_timesteps 5416.
Path 504 | total_timesteps 5428.
Path 505 | total_timesteps 5443.
Path 506 | total_timesteps 5452.
Path 507 | total_timesteps 5463.
Path 508 | total_timesteps 5475.
Path 509 | total_timesteps 5483.
Path 510 | total_timesteps 5492.
Path 511 | total_timesteps 5500.
Path 512 | total_timesteps 5507.
Path 513 | total_timesteps 5518.
Path 514 | total_timesteps 5530.
Path 515 | total_timesteps 5544.
Path 516 | total_timesteps 5552.
Path 517 | total_timesteps 5568.
Path 518 | total_timesteps 5583.
Path 519 | total_timesteps 5590.
Path 520 | total_timesteps 5599.
Path 521 | total_timesteps 5609.
Path 522 | total_timesteps 5621.
Path 523 | total_timesteps 5631.
Path 524 | total_timesteps 5640.
Path 525 | total_timesteps 5653.
Path 526 | total_timesteps 5663.
Path 527 | total_timesteps 5678.
Path 528 | total_timesteps 5689.
Path 529 | total_timesteps 5698.
Path 530 | total_timesteps 5716.
Path 531 | total_timesteps 5724.
Path 532 | total_timesteps 5731.
Path 533 | total_timesteps 5749.
Path 534 | total_timesteps 5757.
Path 535 | total_timesteps 5770.
Path 536 | total_timesteps 5781.
Path 537 | total_timesteps 5792.
Path 538 | total_timesteps 5803.
Path 539 | total_timesteps 5810.
Path 540 | total_timesteps 5826.
Path 541 | total_timesteps 5834.
Path 542 | total_timesteps 5843.
Path 543 | total_timesteps 5849.
Path 544 | total_timesteps 5858.
Path 545 | total_timesteps 5866.
Path 546 | total_timesteps 5882.
Path 547 | total_timesteps 5897.
Path 548 | total_timesteps 5905.
Path 549 | total_timesteps 5913.
Path 550 | total_timesteps 5923.
Path 551 | total_timesteps 5931.
Path 552 | total_timesteps 5940.
Path 553 | total_timesteps 5947.
Path 554 | total_timesteps 5957.
Path 555 | total_timesteps 5964.
Path 556 | total_timesteps 5973.
Path 557 | total_timesteps 5984.
Path 558 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.92    |
| Iteration     | 30       |
| MaximumReturn | 3.65     |
| MinimumReturn | -18.4    |
| TotalSamples  | 128112   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.3178686499595642
Validation loss = 0.3189826011657715
Validation loss = 0.31855684518814087
Validation loss = 0.3172260522842407
Validation loss = 0.31836989521980286
Validation loss = 0.3184267282485962
Validation loss = 0.3188972771167755
Validation loss = 0.31877487897872925
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 44.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 70.
Path 7 | total_timesteps 79.
Path 8 | total_timesteps 95.
Path 9 | total_timesteps 106.
Path 10 | total_timesteps 114.
Path 11 | total_timesteps 131.
Path 12 | total_timesteps 142.
Path 13 | total_timesteps 153.
Path 14 | total_timesteps 186.
Path 15 | total_timesteps 204.
Path 16 | total_timesteps 214.
Path 17 | total_timesteps 227.
Path 18 | total_timesteps 238.
Path 19 | total_timesteps 247.
Path 20 | total_timesteps 259.
Path 21 | total_timesteps 269.
Path 22 | total_timesteps 279.
Path 23 | total_timesteps 288.
Path 24 | total_timesteps 297.
Path 25 | total_timesteps 305.
Path 26 | total_timesteps 319.
Path 27 | total_timesteps 326.
Path 28 | total_timesteps 337.
Path 29 | total_timesteps 348.
Path 30 | total_timesteps 361.
Path 31 | total_timesteps 373.
Path 32 | total_timesteps 384.
Path 33 | total_timesteps 394.
Path 34 | total_timesteps 403.
Path 35 | total_timesteps 412.
Path 36 | total_timesteps 422.
Path 37 | total_timesteps 432.
Path 38 | total_timesteps 440.
Path 39 | total_timesteps 450.
Path 40 | total_timesteps 467.
Path 41 | total_timesteps 484.
Path 42 | total_timesteps 492.
Path 43 | total_timesteps 505.
Path 44 | total_timesteps 515.
Path 45 | total_timesteps 525.
Path 46 | total_timesteps 534.
Path 47 | total_timesteps 557.
Path 48 | total_timesteps 567.
Path 49 | total_timesteps 581.
Path 50 | total_timesteps 591.
Path 51 | total_timesteps 603.
Path 52 | total_timesteps 613.
Path 53 | total_timesteps 623.
Path 54 | total_timesteps 635.
Path 55 | total_timesteps 651.
Path 56 | total_timesteps 660.
Path 57 | total_timesteps 670.
Path 58 | total_timesteps 682.
Path 59 | total_timesteps 688.
Path 60 | total_timesteps 696.
Path 61 | total_timesteps 704.
Path 62 | total_timesteps 714.
Path 63 | total_timesteps 723.
Path 64 | total_timesteps 733.
Path 65 | total_timesteps 740.
Path 66 | total_timesteps 755.
Path 67 | total_timesteps 767.
Path 68 | total_timesteps 776.
Path 69 | total_timesteps 790.
Path 70 | total_timesteps 798.
Path 71 | total_timesteps 812.
Path 72 | total_timesteps 820.
Path 73 | total_timesteps 840.
Path 74 | total_timesteps 852.
Path 75 | total_timesteps 860.
Path 76 | total_timesteps 874.
Path 77 | total_timesteps 884.
Path 78 | total_timesteps 894.
Path 79 | total_timesteps 919.
Path 80 | total_timesteps 932.
Path 81 | total_timesteps 941.
Path 82 | total_timesteps 950.
Path 83 | total_timesteps 964.
Path 84 | total_timesteps 973.
Path 85 | total_timesteps 983.
Path 86 | total_timesteps 996.
Path 87 | total_timesteps 1008.
Path 88 | total_timesteps 1019.
Path 89 | total_timesteps 1032.
Path 90 | total_timesteps 1046.
Path 91 | total_timesteps 1053.
Path 92 | total_timesteps 1061.
Path 93 | total_timesteps 1072.
Path 94 | total_timesteps 1081.
Path 95 | total_timesteps 1094.
Path 96 | total_timesteps 1102.
Path 97 | total_timesteps 1118.
Path 98 | total_timesteps 1126.
Path 99 | total_timesteps 1133.
Path 100 | total_timesteps 1142.
Path 101 | total_timesteps 1157.
Path 102 | total_timesteps 1166.
Path 103 | total_timesteps 1177.
Path 104 | total_timesteps 1185.
Path 105 | total_timesteps 1193.
Path 106 | total_timesteps 1203.
Path 107 | total_timesteps 1210.
Path 108 | total_timesteps 1223.
Path 109 | total_timesteps 1233.
Path 110 | total_timesteps 1240.
Path 111 | total_timesteps 1255.
Path 112 | total_timesteps 1262.
Path 113 | total_timesteps 1273.
Path 114 | total_timesteps 1286.
Path 115 | total_timesteps 1294.
Path 116 | total_timesteps 1301.
Path 117 | total_timesteps 1310.
Path 118 | total_timesteps 1318.
Path 119 | total_timesteps 1328.
Path 120 | total_timesteps 1343.
Path 121 | total_timesteps 1353.
Path 122 | total_timesteps 1359.
Path 123 | total_timesteps 1367.
Path 124 | total_timesteps 1376.
Path 125 | total_timesteps 1388.
Path 126 | total_timesteps 1397.
Path 127 | total_timesteps 1405.
Path 128 | total_timesteps 1414.
Path 129 | total_timesteps 1421.
Path 130 | total_timesteps 1430.
Path 131 | total_timesteps 1458.
Path 132 | total_timesteps 1469.
Path 133 | total_timesteps 1480.
Path 134 | total_timesteps 1498.
Path 135 | total_timesteps 1509.
Path 136 | total_timesteps 1524.
Path 137 | total_timesteps 1543.
Path 138 | total_timesteps 1557.
Path 139 | total_timesteps 1566.
Path 140 | total_timesteps 1580.
Path 141 | total_timesteps 1592.
Path 142 | total_timesteps 1600.
Path 143 | total_timesteps 1608.
Path 144 | total_timesteps 1615.
Path 145 | total_timesteps 1628.
Path 146 | total_timesteps 1635.
Path 147 | total_timesteps 1643.
Path 148 | total_timesteps 1654.
Path 149 | total_timesteps 1662.
Path 150 | total_timesteps 1670.
Path 151 | total_timesteps 1678.
Path 152 | total_timesteps 1688.
Path 153 | total_timesteps 1698.
Path 154 | total_timesteps 1708.
Path 155 | total_timesteps 1720.
Path 156 | total_timesteps 1732.
Path 157 | total_timesteps 1742.
Path 158 | total_timesteps 1757.
Path 159 | total_timesteps 1766.
Path 160 | total_timesteps 1777.
Path 161 | total_timesteps 1788.
Path 162 | total_timesteps 1799.
Path 163 | total_timesteps 1814.
Path 164 | total_timesteps 1823.
Path 165 | total_timesteps 1834.
Path 166 | total_timesteps 1845.
Path 167 | total_timesteps 1855.
Path 168 | total_timesteps 1867.
Path 169 | total_timesteps 1875.
Path 170 | total_timesteps 1890.
Path 171 | total_timesteps 1900.
Path 172 | total_timesteps 1909.
Path 173 | total_timesteps 1917.
Path 174 | total_timesteps 1926.
Path 175 | total_timesteps 1941.
Path 176 | total_timesteps 1951.
Path 177 | total_timesteps 1958.
Path 178 | total_timesteps 1966.
Path 179 | total_timesteps 1974.
Path 180 | total_timesteps 1985.
Path 181 | total_timesteps 1994.
Path 182 | total_timesteps 2004.
Path 183 | total_timesteps 2019.
Path 184 | total_timesteps 2028.
Path 185 | total_timesteps 2038.
Path 186 | total_timesteps 2047.
Path 187 | total_timesteps 2056.
Path 188 | total_timesteps 2067.
Path 189 | total_timesteps 2074.
Path 190 | total_timesteps 2087.
Path 191 | total_timesteps 2096.
Path 192 | total_timesteps 2105.
Path 193 | total_timesteps 2121.
Path 194 | total_timesteps 2130.
Path 195 | total_timesteps 2145.
Path 196 | total_timesteps 2152.
Path 197 | total_timesteps 2162.
Path 198 | total_timesteps 2170.
Path 199 | total_timesteps 2188.
Path 200 | total_timesteps 2201.
Path 201 | total_timesteps 2209.
Path 202 | total_timesteps 2217.
Path 203 | total_timesteps 2228.
Path 204 | total_timesteps 2235.
Path 205 | total_timesteps 2246.
Path 206 | total_timesteps 2257.
Path 207 | total_timesteps 2266.
Path 208 | total_timesteps 2280.
Path 209 | total_timesteps 2289.
Path 210 | total_timesteps 2297.
Path 211 | total_timesteps 2306.
Path 212 | total_timesteps 2320.
Path 213 | total_timesteps 2333.
Path 214 | total_timesteps 2343.
Path 215 | total_timesteps 2353.
Path 216 | total_timesteps 2362.
Path 217 | total_timesteps 2372.
Path 218 | total_timesteps 2383.
Path 219 | total_timesteps 2393.
Path 220 | total_timesteps 2402.
Path 221 | total_timesteps 2414.
Path 222 | total_timesteps 2424.
Path 223 | total_timesteps 2433.
Path 224 | total_timesteps 2446.
Path 225 | total_timesteps 2465.
Path 226 | total_timesteps 2476.
Path 227 | total_timesteps 2486.
Path 228 | total_timesteps 2499.
Path 229 | total_timesteps 2507.
Path 230 | total_timesteps 2520.
Path 231 | total_timesteps 2529.
Path 232 | total_timesteps 2546.
Path 233 | total_timesteps 2556.
Path 234 | total_timesteps 2567.
Path 235 | total_timesteps 2575.
Path 236 | total_timesteps 2582.
Path 237 | total_timesteps 2595.
Path 238 | total_timesteps 2603.
Path 239 | total_timesteps 2618.
Path 240 | total_timesteps 2628.
Path 241 | total_timesteps 2642.
Path 242 | total_timesteps 2655.
Path 243 | total_timesteps 2666.
Path 244 | total_timesteps 2677.
Path 245 | total_timesteps 2684.
Path 246 | total_timesteps 2692.
Path 247 | total_timesteps 2702.
Path 248 | total_timesteps 2714.
Path 249 | total_timesteps 2722.
Path 250 | total_timesteps 2733.
Path 251 | total_timesteps 2746.
Path 252 | total_timesteps 2759.
Path 253 | total_timesteps 2772.
Path 254 | total_timesteps 2779.
Path 255 | total_timesteps 2787.
Path 256 | total_timesteps 2799.
Path 257 | total_timesteps 2809.
Path 258 | total_timesteps 2819.
Path 259 | total_timesteps 2826.
Path 260 | total_timesteps 2834.
Path 261 | total_timesteps 2842.
Path 262 | total_timesteps 2852.
Path 263 | total_timesteps 2860.
Path 264 | total_timesteps 2869.
Path 265 | total_timesteps 2880.
Path 266 | total_timesteps 2891.
Path 267 | total_timesteps 2900.
Path 268 | total_timesteps 2909.
Path 269 | total_timesteps 2917.
Path 270 | total_timesteps 2930.
Path 271 | total_timesteps 2947.
Path 272 | total_timesteps 2957.
Path 273 | total_timesteps 2966.
Path 274 | total_timesteps 2974.
Path 275 | total_timesteps 2987.
Path 276 | total_timesteps 2996.
Path 277 | total_timesteps 3010.
Path 278 | total_timesteps 3021.
Path 279 | total_timesteps 3029.
Path 280 | total_timesteps 3036.
Path 281 | total_timesteps 3048.
Path 282 | total_timesteps 3061.
Path 283 | total_timesteps 3080.
Path 284 | total_timesteps 3089.
Path 285 | total_timesteps 3102.
Path 286 | total_timesteps 3111.
Path 287 | total_timesteps 3123.
Path 288 | total_timesteps 3131.
Path 289 | total_timesteps 3142.
Path 290 | total_timesteps 3152.
Path 291 | total_timesteps 3161.
Path 292 | total_timesteps 3172.
Path 293 | total_timesteps 3183.
Path 294 | total_timesteps 3195.
Path 295 | total_timesteps 3216.
Path 296 | total_timesteps 3226.
Path 297 | total_timesteps 3238.
Path 298 | total_timesteps 3254.
Path 299 | total_timesteps 3267.
Path 300 | total_timesteps 3282.
Path 301 | total_timesteps 3293.
Path 302 | total_timesteps 3307.
Path 303 | total_timesteps 3315.
Path 304 | total_timesteps 3326.
Path 305 | total_timesteps 3336.
Path 306 | total_timesteps 3348.
Path 307 | total_timesteps 3360.
Path 308 | total_timesteps 3373.
Path 309 | total_timesteps 3383.
Path 310 | total_timesteps 3391.
Path 311 | total_timesteps 3401.
Path 312 | total_timesteps 3414.
Path 313 | total_timesteps 3422.
Path 314 | total_timesteps 3429.
Path 315 | total_timesteps 3445.
Path 316 | total_timesteps 3453.
Path 317 | total_timesteps 3463.
Path 318 | total_timesteps 3474.
Path 319 | total_timesteps 3483.
Path 320 | total_timesteps 3497.
Path 321 | total_timesteps 3507.
Path 322 | total_timesteps 3514.
Path 323 | total_timesteps 3522.
Path 324 | total_timesteps 3536.
Path 325 | total_timesteps 3545.
Path 326 | total_timesteps 3555.
Path 327 | total_timesteps 3564.
Path 328 | total_timesteps 3575.
Path 329 | total_timesteps 3587.
Path 330 | total_timesteps 3598.
Path 331 | total_timesteps 3608.
Path 332 | total_timesteps 3615.
Path 333 | total_timesteps 3630.
Path 334 | total_timesteps 3641.
Path 335 | total_timesteps 3649.
Path 336 | total_timesteps 3660.
Path 337 | total_timesteps 3667.
Path 338 | total_timesteps 3676.
Path 339 | total_timesteps 3689.
Path 340 | total_timesteps 3703.
Path 341 | total_timesteps 3712.
Path 342 | total_timesteps 3727.
Path 343 | total_timesteps 3734.
Path 344 | total_timesteps 3747.
Path 345 | total_timesteps 3756.
Path 346 | total_timesteps 3767.
Path 347 | total_timesteps 3778.
Path 348 | total_timesteps 3787.
Path 349 | total_timesteps 3797.
Path 350 | total_timesteps 3809.
Path 351 | total_timesteps 3818.
Path 352 | total_timesteps 3826.
Path 353 | total_timesteps 3836.
Path 354 | total_timesteps 3846.
Path 355 | total_timesteps 3854.
Path 356 | total_timesteps 3861.
Path 357 | total_timesteps 3873.
Path 358 | total_timesteps 3880.
Path 359 | total_timesteps 3890.
Path 360 | total_timesteps 3897.
Path 361 | total_timesteps 3911.
Path 362 | total_timesteps 3919.
Path 363 | total_timesteps 3928.
Path 364 | total_timesteps 3938.
Path 365 | total_timesteps 3948.
Path 366 | total_timesteps 3960.
Path 367 | total_timesteps 3975.
Path 368 | total_timesteps 3991.
Path 369 | total_timesteps 3997.
Path 370 | total_timesteps 4006.
Path 371 | total_timesteps 4014.
Path 372 | total_timesteps 4023.
Path 373 | total_timesteps 4034.
Path 374 | total_timesteps 4043.
Path 375 | total_timesteps 4050.
Path 376 | total_timesteps 4062.
Path 377 | total_timesteps 4074.
Path 378 | total_timesteps 4084.
Path 379 | total_timesteps 4091.
Path 380 | total_timesteps 4099.
Path 381 | total_timesteps 4107.
Path 382 | total_timesteps 4119.
Path 383 | total_timesteps 4126.
Path 384 | total_timesteps 4136.
Path 385 | total_timesteps 4143.
Path 386 | total_timesteps 4158.
Path 387 | total_timesteps 4171.
Path 388 | total_timesteps 4182.
Path 389 | total_timesteps 4196.
Path 390 | total_timesteps 4209.
Path 391 | total_timesteps 4220.
Path 392 | total_timesteps 4227.
Path 393 | total_timesteps 4237.
Path 394 | total_timesteps 4248.
Path 395 | total_timesteps 4258.
Path 396 | total_timesteps 4267.
Path 397 | total_timesteps 4276.
Path 398 | total_timesteps 4295.
Path 399 | total_timesteps 4309.
Path 400 | total_timesteps 4318.
Path 401 | total_timesteps 4330.
Path 402 | total_timesteps 4337.
Path 403 | total_timesteps 4346.
Path 404 | total_timesteps 4358.
Path 405 | total_timesteps 4366.
Path 406 | total_timesteps 4376.
Path 407 | total_timesteps 4385.
Path 408 | total_timesteps 4397.
Path 409 | total_timesteps 4403.
Path 410 | total_timesteps 4414.
Path 411 | total_timesteps 4422.
Path 412 | total_timesteps 4431.
Path 413 | total_timesteps 4438.
Path 414 | total_timesteps 4449.
Path 415 | total_timesteps 4460.
Path 416 | total_timesteps 4468.
Path 417 | total_timesteps 4478.
Path 418 | total_timesteps 4485.
Path 419 | total_timesteps 4497.
Path 420 | total_timesteps 4511.
Path 421 | total_timesteps 4527.
Path 422 | total_timesteps 4536.
Path 423 | total_timesteps 4551.
Path 424 | total_timesteps 4560.
Path 425 | total_timesteps 4571.
Path 426 | total_timesteps 4581.
Path 427 | total_timesteps 4592.
Path 428 | total_timesteps 4603.
Path 429 | total_timesteps 4614.
Path 430 | total_timesteps 4623.
Path 431 | total_timesteps 4633.
Path 432 | total_timesteps 4641.
Path 433 | total_timesteps 4654.
Path 434 | total_timesteps 4663.
Path 435 | total_timesteps 4674.
Path 436 | total_timesteps 4685.
Path 437 | total_timesteps 4693.
Path 438 | total_timesteps 4703.
Path 439 | total_timesteps 4711.
Path 440 | total_timesteps 4720.
Path 441 | total_timesteps 4729.
Path 442 | total_timesteps 4736.
Path 443 | total_timesteps 4750.
Path 444 | total_timesteps 4757.
Path 445 | total_timesteps 4765.
Path 446 | total_timesteps 4774.
Path 447 | total_timesteps 4787.
Path 448 | total_timesteps 4795.
Path 449 | total_timesteps 4803.
Path 450 | total_timesteps 4815.
Path 451 | total_timesteps 4827.
Path 452 | total_timesteps 4839.
Path 453 | total_timesteps 4847.
Path 454 | total_timesteps 4854.
Path 455 | total_timesteps 4864.
Path 456 | total_timesteps 4871.
Path 457 | total_timesteps 4884.
Path 458 | total_timesteps 4903.
Path 459 | total_timesteps 4915.
Path 460 | total_timesteps 4926.
Path 461 | total_timesteps 4939.
Path 462 | total_timesteps 4951.
Path 463 | total_timesteps 4964.
Path 464 | total_timesteps 4971.
Path 465 | total_timesteps 4980.
Path 466 | total_timesteps 4994.
Path 467 | total_timesteps 5003.
Path 468 | total_timesteps 5015.
Path 469 | total_timesteps 5030.
Path 470 | total_timesteps 5039.
Path 471 | total_timesteps 5046.
Path 472 | total_timesteps 5054.
Path 473 | total_timesteps 5062.
Path 474 | total_timesteps 5073.
Path 475 | total_timesteps 5084.
Path 476 | total_timesteps 5094.
Path 477 | total_timesteps 5104.
Path 478 | total_timesteps 5113.
Path 479 | total_timesteps 5123.
Path 480 | total_timesteps 5133.
Path 481 | total_timesteps 5141.
Path 482 | total_timesteps 5151.
Path 483 | total_timesteps 5160.
Path 484 | total_timesteps 5170.
Path 485 | total_timesteps 5179.
Path 486 | total_timesteps 5188.
Path 487 | total_timesteps 5195.
Path 488 | total_timesteps 5204.
Path 489 | total_timesteps 5210.
Path 490 | total_timesteps 5220.
Path 491 | total_timesteps 5232.
Path 492 | total_timesteps 5240.
Path 493 | total_timesteps 5250.
Path 494 | total_timesteps 5257.
Path 495 | total_timesteps 5271.
Path 496 | total_timesteps 5283.
Path 497 | total_timesteps 5293.
Path 498 | total_timesteps 5303.
Path 499 | total_timesteps 5312.
Path 500 | total_timesteps 5323.
Path 501 | total_timesteps 5331.
Path 502 | total_timesteps 5340.
Path 503 | total_timesteps 5354.
Path 504 | total_timesteps 5363.
Path 505 | total_timesteps 5378.
Path 506 | total_timesteps 5387.
Path 507 | total_timesteps 5398.
Path 508 | total_timesteps 5408.
Path 509 | total_timesteps 5418.
Path 510 | total_timesteps 5426.
Path 511 | total_timesteps 5437.
Path 512 | total_timesteps 5450.
Path 513 | total_timesteps 5457.
Path 514 | total_timesteps 5471.
Path 515 | total_timesteps 5479.
Path 516 | total_timesteps 5489.
Path 517 | total_timesteps 5500.
Path 518 | total_timesteps 5509.
Path 519 | total_timesteps 5522.
Path 520 | total_timesteps 5537.
Path 521 | total_timesteps 5544.
Path 522 | total_timesteps 5551.
Path 523 | total_timesteps 5567.
Path 524 | total_timesteps 5578.
Path 525 | total_timesteps 5588.
Path 526 | total_timesteps 5597.
Path 527 | total_timesteps 5611.
Path 528 | total_timesteps 5626.
Path 529 | total_timesteps 5634.
Path 530 | total_timesteps 5641.
Path 531 | total_timesteps 5650.
Path 532 | total_timesteps 5657.
Path 533 | total_timesteps 5665.
Path 534 | total_timesteps 5676.
Path 535 | total_timesteps 5688.
Path 536 | total_timesteps 5694.
Path 537 | total_timesteps 5705.
Path 538 | total_timesteps 5714.
Path 539 | total_timesteps 5724.
Path 540 | total_timesteps 5732.
Path 541 | total_timesteps 5742.
Path 542 | total_timesteps 5750.
Path 543 | total_timesteps 5760.
Path 544 | total_timesteps 5769.
Path 545 | total_timesteps 5781.
Path 546 | total_timesteps 5793.
Path 547 | total_timesteps 5805.
Path 548 | total_timesteps 5811.
Path 549 | total_timesteps 5825.
Path 550 | total_timesteps 5836.
Path 551 | total_timesteps 5845.
Path 552 | total_timesteps 5854.
Path 553 | total_timesteps 5868.
Path 554 | total_timesteps 5880.
Path 555 | total_timesteps 5897.
Path 556 | total_timesteps 5904.
Path 557 | total_timesteps 5912.
Path 558 | total_timesteps 5923.
Path 559 | total_timesteps 5933.
Path 560 | total_timesteps 5944.
Path 561 | total_timesteps 5954.
Path 562 | total_timesteps 5961.
Path 563 | total_timesteps 5970.
Path 564 | total_timesteps 5984.
Path 565 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.01    |
| Iteration     | 31       |
| MaximumReturn | 2.4      |
| MinimumReturn | -20      |
| TotalSamples  | 132118   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.3152202367782593
Validation loss = 0.3185565173625946
Validation loss = 0.3171204924583435
Validation loss = 0.31799355149269104
Validation loss = 0.3179835081100464
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 84.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 102.
Path 11 | total_timesteps 119.
Path 12 | total_timesteps 126.
Path 13 | total_timesteps 134.
Path 14 | total_timesteps 146.
Path 15 | total_timesteps 154.
Path 16 | total_timesteps 170.
Path 17 | total_timesteps 179.
Path 18 | total_timesteps 188.
Path 19 | total_timesteps 197.
Path 20 | total_timesteps 207.
Path 21 | total_timesteps 215.
Path 22 | total_timesteps 227.
Path 23 | total_timesteps 235.
Path 24 | total_timesteps 246.
Path 25 | total_timesteps 254.
Path 26 | total_timesteps 266.
Path 27 | total_timesteps 278.
Path 28 | total_timesteps 286.
Path 29 | total_timesteps 295.
Path 30 | total_timesteps 305.
Path 31 | total_timesteps 317.
Path 32 | total_timesteps 326.
Path 33 | total_timesteps 336.
Path 34 | total_timesteps 347.
Path 35 | total_timesteps 356.
Path 36 | total_timesteps 367.
Path 37 | total_timesteps 387.
Path 38 | total_timesteps 400.
Path 39 | total_timesteps 409.
Path 40 | total_timesteps 420.
Path 41 | total_timesteps 431.
Path 42 | total_timesteps 440.
Path 43 | total_timesteps 454.
Path 44 | total_timesteps 467.
Path 45 | total_timesteps 476.
Path 46 | total_timesteps 483.
Path 47 | total_timesteps 496.
Path 48 | total_timesteps 515.
Path 49 | total_timesteps 526.
Path 50 | total_timesteps 536.
Path 51 | total_timesteps 548.
Path 52 | total_timesteps 559.
Path 53 | total_timesteps 567.
Path 54 | total_timesteps 580.
Path 55 | total_timesteps 590.
Path 56 | total_timesteps 603.
Path 57 | total_timesteps 611.
Path 58 | total_timesteps 619.
Path 59 | total_timesteps 628.
Path 60 | total_timesteps 640.
Path 61 | total_timesteps 652.
Path 62 | total_timesteps 661.
Path 63 | total_timesteps 667.
Path 64 | total_timesteps 677.
Path 65 | total_timesteps 684.
Path 66 | total_timesteps 691.
Path 67 | total_timesteps 703.
Path 68 | total_timesteps 714.
Path 69 | total_timesteps 724.
Path 70 | total_timesteps 738.
Path 71 | total_timesteps 748.
Path 72 | total_timesteps 760.
Path 73 | total_timesteps 775.
Path 74 | total_timesteps 795.
Path 75 | total_timesteps 805.
Path 76 | total_timesteps 814.
Path 77 | total_timesteps 826.
Path 78 | total_timesteps 839.
Path 79 | total_timesteps 847.
Path 80 | total_timesteps 856.
Path 81 | total_timesteps 866.
Path 82 | total_timesteps 881.
Path 83 | total_timesteps 889.
Path 84 | total_timesteps 911.
Path 85 | total_timesteps 920.
Path 86 | total_timesteps 932.
Path 87 | total_timesteps 942.
Path 88 | total_timesteps 958.
Path 89 | total_timesteps 967.
Path 90 | total_timesteps 980.
Path 91 | total_timesteps 993.
Path 92 | total_timesteps 1004.
Path 93 | total_timesteps 1019.
Path 94 | total_timesteps 1032.
Path 95 | total_timesteps 1041.
Path 96 | total_timesteps 1050.
Path 97 | total_timesteps 1057.
Path 98 | total_timesteps 1065.
Path 99 | total_timesteps 1077.
Path 100 | total_timesteps 1087.
Path 101 | total_timesteps 1098.
Path 102 | total_timesteps 1108.
Path 103 | total_timesteps 1117.
Path 104 | total_timesteps 1126.
Path 105 | total_timesteps 1139.
Path 106 | total_timesteps 1150.
Path 107 | total_timesteps 1162.
Path 108 | total_timesteps 1168.
Path 109 | total_timesteps 1176.
Path 110 | total_timesteps 1190.
Path 111 | total_timesteps 1198.
Path 112 | total_timesteps 1210.
Path 113 | total_timesteps 1222.
Path 114 | total_timesteps 1235.
Path 115 | total_timesteps 1244.
Path 116 | total_timesteps 1255.
Path 117 | total_timesteps 1267.
Path 118 | total_timesteps 1276.
Path 119 | total_timesteps 1284.
Path 120 | total_timesteps 1293.
Path 121 | total_timesteps 1302.
Path 122 | total_timesteps 1310.
Path 123 | total_timesteps 1318.
Path 124 | total_timesteps 1331.
Path 125 | total_timesteps 1341.
Path 126 | total_timesteps 1355.
Path 127 | total_timesteps 1368.
Path 128 | total_timesteps 1380.
Path 129 | total_timesteps 1391.
Path 130 | total_timesteps 1401.
Path 131 | total_timesteps 1409.
Path 132 | total_timesteps 1420.
Path 133 | total_timesteps 1431.
Path 134 | total_timesteps 1440.
Path 135 | total_timesteps 1453.
Path 136 | total_timesteps 1463.
Path 137 | total_timesteps 1481.
Path 138 | total_timesteps 1491.
Path 139 | total_timesteps 1501.
Path 140 | total_timesteps 1513.
Path 141 | total_timesteps 1521.
Path 142 | total_timesteps 1530.
Path 143 | total_timesteps 1539.
Path 144 | total_timesteps 1549.
Path 145 | total_timesteps 1557.
Path 146 | total_timesteps 1571.
Path 147 | total_timesteps 1579.
Path 148 | total_timesteps 1590.
Path 149 | total_timesteps 1602.
Path 150 | total_timesteps 1611.
Path 151 | total_timesteps 1627.
Path 152 | total_timesteps 1635.
Path 153 | total_timesteps 1651.
Path 154 | total_timesteps 1660.
Path 155 | total_timesteps 1670.
Path 156 | total_timesteps 1683.
Path 157 | total_timesteps 1701.
Path 158 | total_timesteps 1712.
Path 159 | total_timesteps 1723.
Path 160 | total_timesteps 1734.
Path 161 | total_timesteps 1742.
Path 162 | total_timesteps 1753.
Path 163 | total_timesteps 1766.
Path 164 | total_timesteps 1777.
Path 165 | total_timesteps 1787.
Path 166 | total_timesteps 1804.
Path 167 | total_timesteps 1814.
Path 168 | total_timesteps 1823.
Path 169 | total_timesteps 1843.
Path 170 | total_timesteps 1854.
Path 171 | total_timesteps 1862.
Path 172 | total_timesteps 1870.
Path 173 | total_timesteps 1882.
Path 174 | total_timesteps 1893.
Path 175 | total_timesteps 1901.
Path 176 | total_timesteps 1911.
Path 177 | total_timesteps 1923.
Path 178 | total_timesteps 1932.
Path 179 | total_timesteps 1943.
Path 180 | total_timesteps 1953.
Path 181 | total_timesteps 1966.
Path 182 | total_timesteps 1978.
Path 183 | total_timesteps 1986.
Path 184 | total_timesteps 1997.
Path 185 | total_timesteps 2005.
Path 186 | total_timesteps 2014.
Path 187 | total_timesteps 2023.
Path 188 | total_timesteps 2032.
Path 189 | total_timesteps 2042.
Path 190 | total_timesteps 2052.
Path 191 | total_timesteps 2060.
Path 192 | total_timesteps 2083.
Path 193 | total_timesteps 2094.
Path 194 | total_timesteps 2103.
Path 195 | total_timesteps 2110.
Path 196 | total_timesteps 2120.
Path 197 | total_timesteps 2132.
Path 198 | total_timesteps 2142.
Path 199 | total_timesteps 2151.
Path 200 | total_timesteps 2158.
Path 201 | total_timesteps 2166.
Path 202 | total_timesteps 2177.
Path 203 | total_timesteps 2189.
Path 204 | total_timesteps 2204.
Path 205 | total_timesteps 2212.
Path 206 | total_timesteps 2220.
Path 207 | total_timesteps 2229.
Path 208 | total_timesteps 2242.
Path 209 | total_timesteps 2248.
Path 210 | total_timesteps 2259.
Path 211 | total_timesteps 2273.
Path 212 | total_timesteps 2283.
Path 213 | total_timesteps 2290.
Path 214 | total_timesteps 2299.
Path 215 | total_timesteps 2305.
Path 216 | total_timesteps 2313.
Path 217 | total_timesteps 2323.
Path 218 | total_timesteps 2337.
Path 219 | total_timesteps 2344.
Path 220 | total_timesteps 2357.
Path 221 | total_timesteps 2370.
Path 222 | total_timesteps 2382.
Path 223 | total_timesteps 2391.
Path 224 | total_timesteps 2401.
Path 225 | total_timesteps 2411.
Path 226 | total_timesteps 2423.
Path 227 | total_timesteps 2431.
Path 228 | total_timesteps 2440.
Path 229 | total_timesteps 2450.
Path 230 | total_timesteps 2464.
Path 231 | total_timesteps 2474.
Path 232 | total_timesteps 2486.
Path 233 | total_timesteps 2494.
Path 234 | total_timesteps 2503.
Path 235 | total_timesteps 2512.
Path 236 | total_timesteps 2524.
Path 237 | total_timesteps 2542.
Path 238 | total_timesteps 2549.
Path 239 | total_timesteps 2563.
Path 240 | total_timesteps 2578.
Path 241 | total_timesteps 2587.
Path 242 | total_timesteps 2598.
Path 243 | total_timesteps 2608.
Path 244 | total_timesteps 2621.
Path 245 | total_timesteps 2630.
Path 246 | total_timesteps 2637.
Path 247 | total_timesteps 2647.
Path 248 | total_timesteps 2656.
Path 249 | total_timesteps 2668.
Path 250 | total_timesteps 2678.
Path 251 | total_timesteps 2688.
Path 252 | total_timesteps 2700.
Path 253 | total_timesteps 2719.
Path 254 | total_timesteps 2730.
Path 255 | total_timesteps 2741.
Path 256 | total_timesteps 2750.
Path 257 | total_timesteps 2759.
Path 258 | total_timesteps 2767.
Path 259 | total_timesteps 2784.
Path 260 | total_timesteps 2793.
Path 261 | total_timesteps 2813.
Path 262 | total_timesteps 2825.
Path 263 | total_timesteps 2834.
Path 264 | total_timesteps 2843.
Path 265 | total_timesteps 2856.
Path 266 | total_timesteps 2865.
Path 267 | total_timesteps 2876.
Path 268 | total_timesteps 2889.
Path 269 | total_timesteps 2897.
Path 270 | total_timesteps 2906.
Path 271 | total_timesteps 2918.
Path 272 | total_timesteps 2928.
Path 273 | total_timesteps 2938.
Path 274 | total_timesteps 2947.
Path 275 | total_timesteps 2960.
Path 276 | total_timesteps 2969.
Path 277 | total_timesteps 2976.
Path 278 | total_timesteps 2988.
Path 279 | total_timesteps 3000.
Path 280 | total_timesteps 3012.
Path 281 | total_timesteps 3019.
Path 282 | total_timesteps 3030.
Path 283 | total_timesteps 3043.
Path 284 | total_timesteps 3054.
Path 285 | total_timesteps 3068.
Path 286 | total_timesteps 3077.
Path 287 | total_timesteps 3088.
Path 288 | total_timesteps 3098.
Path 289 | total_timesteps 3105.
Path 290 | total_timesteps 3116.
Path 291 | total_timesteps 3123.
Path 292 | total_timesteps 3132.
Path 293 | total_timesteps 3141.
Path 294 | total_timesteps 3153.
Path 295 | total_timesteps 3164.
Path 296 | total_timesteps 3178.
Path 297 | total_timesteps 3193.
Path 298 | total_timesteps 3202.
Path 299 | total_timesteps 3213.
Path 300 | total_timesteps 3221.
Path 301 | total_timesteps 3229.
Path 302 | total_timesteps 3237.
Path 303 | total_timesteps 3247.
Path 304 | total_timesteps 3259.
Path 305 | total_timesteps 3268.
Path 306 | total_timesteps 3286.
Path 307 | total_timesteps 3299.
Path 308 | total_timesteps 3310.
Path 309 | total_timesteps 3324.
Path 310 | total_timesteps 3336.
Path 311 | total_timesteps 3348.
Path 312 | total_timesteps 3357.
Path 313 | total_timesteps 3367.
Path 314 | total_timesteps 3375.
Path 315 | total_timesteps 3386.
Path 316 | total_timesteps 3395.
Path 317 | total_timesteps 3403.
Path 318 | total_timesteps 3412.
Path 319 | total_timesteps 3423.
Path 320 | total_timesteps 3434.
Path 321 | total_timesteps 3447.
Path 322 | total_timesteps 3457.
Path 323 | total_timesteps 3468.
Path 324 | total_timesteps 3480.
Path 325 | total_timesteps 3492.
Path 326 | total_timesteps 3503.
Path 327 | total_timesteps 3517.
Path 328 | total_timesteps 3529.
Path 329 | total_timesteps 3541.
Path 330 | total_timesteps 3549.
Path 331 | total_timesteps 3560.
Path 332 | total_timesteps 3570.
Path 333 | total_timesteps 3578.
Path 334 | total_timesteps 3589.
Path 335 | total_timesteps 3597.
Path 336 | total_timesteps 3608.
Path 337 | total_timesteps 3617.
Path 338 | total_timesteps 3625.
Path 339 | total_timesteps 3635.
Path 340 | total_timesteps 3642.
Path 341 | total_timesteps 3650.
Path 342 | total_timesteps 3661.
Path 343 | total_timesteps 3674.
Path 344 | total_timesteps 3682.
Path 345 | total_timesteps 3694.
Path 346 | total_timesteps 3703.
Path 347 | total_timesteps 3714.
Path 348 | total_timesteps 3724.
Path 349 | total_timesteps 3735.
Path 350 | total_timesteps 3744.
Path 351 | total_timesteps 3754.
Path 352 | total_timesteps 3762.
Path 353 | total_timesteps 3770.
Path 354 | total_timesteps 3780.
Path 355 | total_timesteps 3795.
Path 356 | total_timesteps 3807.
Path 357 | total_timesteps 3819.
Path 358 | total_timesteps 3827.
Path 359 | total_timesteps 3836.
Path 360 | total_timesteps 3848.
Path 361 | total_timesteps 3865.
Path 362 | total_timesteps 3875.
Path 363 | total_timesteps 3887.
Path 364 | total_timesteps 3895.
Path 365 | total_timesteps 3905.
Path 366 | total_timesteps 3914.
Path 367 | total_timesteps 3922.
Path 368 | total_timesteps 3932.
Path 369 | total_timesteps 3942.
Path 370 | total_timesteps 3952.
Path 371 | total_timesteps 3961.
Path 372 | total_timesteps 3974.
Path 373 | total_timesteps 3985.
Path 374 | total_timesteps 4001.
Path 375 | total_timesteps 4008.
Path 376 | total_timesteps 4016.
Path 377 | total_timesteps 4027.
Path 378 | total_timesteps 4036.
Path 379 | total_timesteps 4046.
Path 380 | total_timesteps 4055.
Path 381 | total_timesteps 4062.
Path 382 | total_timesteps 4080.
Path 383 | total_timesteps 4088.
Path 384 | total_timesteps 4098.
Path 385 | total_timesteps 4107.
Path 386 | total_timesteps 4118.
Path 387 | total_timesteps 4130.
Path 388 | total_timesteps 4140.
Path 389 | total_timesteps 4155.
Path 390 | total_timesteps 4164.
Path 391 | total_timesteps 4179.
Path 392 | total_timesteps 4187.
Path 393 | total_timesteps 4202.
Path 394 | total_timesteps 4211.
Path 395 | total_timesteps 4222.
Path 396 | total_timesteps 4233.
Path 397 | total_timesteps 4243.
Path 398 | total_timesteps 4253.
Path 399 | total_timesteps 4262.
Path 400 | total_timesteps 4275.
Path 401 | total_timesteps 4285.
Path 402 | total_timesteps 4294.
Path 403 | total_timesteps 4304.
Path 404 | total_timesteps 4313.
Path 405 | total_timesteps 4322.
Path 406 | total_timesteps 4330.
Path 407 | total_timesteps 4343.
Path 408 | total_timesteps 4351.
Path 409 | total_timesteps 4368.
Path 410 | total_timesteps 4378.
Path 411 | total_timesteps 4388.
Path 412 | total_timesteps 4395.
Path 413 | total_timesteps 4405.
Path 414 | total_timesteps 4418.
Path 415 | total_timesteps 4426.
Path 416 | total_timesteps 4444.
Path 417 | total_timesteps 4462.
Path 418 | total_timesteps 4475.
Path 419 | total_timesteps 4483.
Path 420 | total_timesteps 4493.
Path 421 | total_timesteps 4503.
Path 422 | total_timesteps 4517.
Path 423 | total_timesteps 4528.
Path 424 | total_timesteps 4537.
Path 425 | total_timesteps 4550.
Path 426 | total_timesteps 4560.
Path 427 | total_timesteps 4567.
Path 428 | total_timesteps 4579.
Path 429 | total_timesteps 4591.
Path 430 | total_timesteps 4600.
Path 431 | total_timesteps 4607.
Path 432 | total_timesteps 4617.
Path 433 | total_timesteps 4630.
Path 434 | total_timesteps 4645.
Path 435 | total_timesteps 4653.
Path 436 | total_timesteps 4661.
Path 437 | total_timesteps 4670.
Path 438 | total_timesteps 4683.
Path 439 | total_timesteps 4701.
Path 440 | total_timesteps 4710.
Path 441 | total_timesteps 4720.
Path 442 | total_timesteps 4729.
Path 443 | total_timesteps 4738.
Path 444 | total_timesteps 4749.
Path 445 | total_timesteps 4757.
Path 446 | total_timesteps 4763.
Path 447 | total_timesteps 4770.
Path 448 | total_timesteps 4779.
Path 449 | total_timesteps 4789.
Path 450 | total_timesteps 4803.
Path 451 | total_timesteps 4812.
Path 452 | total_timesteps 4819.
Path 453 | total_timesteps 4828.
Path 454 | total_timesteps 4836.
Path 455 | total_timesteps 4843.
Path 456 | total_timesteps 4852.
Path 457 | total_timesteps 4865.
Path 458 | total_timesteps 4877.
Path 459 | total_timesteps 4887.
Path 460 | total_timesteps 4897.
Path 461 | total_timesteps 4906.
Path 462 | total_timesteps 4920.
Path 463 | total_timesteps 4931.
Path 464 | total_timesteps 4946.
Path 465 | total_timesteps 4960.
Path 466 | total_timesteps 4968.
Path 467 | total_timesteps 4978.
Path 468 | total_timesteps 4986.
Path 469 | total_timesteps 4995.
Path 470 | total_timesteps 5005.
Path 471 | total_timesteps 5017.
Path 472 | total_timesteps 5029.
Path 473 | total_timesteps 5037.
Path 474 | total_timesteps 5055.
Path 475 | total_timesteps 5065.
Path 476 | total_timesteps 5076.
Path 477 | total_timesteps 5084.
Path 478 | total_timesteps 5091.
Path 479 | total_timesteps 5106.
Path 480 | total_timesteps 5127.
Path 481 | total_timesteps 5138.
Path 482 | total_timesteps 5151.
Path 483 | total_timesteps 5160.
Path 484 | total_timesteps 5169.
Path 485 | total_timesteps 5179.
Path 486 | total_timesteps 5188.
Path 487 | total_timesteps 5198.
Path 488 | total_timesteps 5207.
Path 489 | total_timesteps 5222.
Path 490 | total_timesteps 5235.
Path 491 | total_timesteps 5243.
Path 492 | total_timesteps 5251.
Path 493 | total_timesteps 5266.
Path 494 | total_timesteps 5276.
Path 495 | total_timesteps 5290.
Path 496 | total_timesteps 5300.
Path 497 | total_timesteps 5310.
Path 498 | total_timesteps 5323.
Path 499 | total_timesteps 5337.
Path 500 | total_timesteps 5345.
Path 501 | total_timesteps 5358.
Path 502 | total_timesteps 5373.
Path 503 | total_timesteps 5380.
Path 504 | total_timesteps 5397.
Path 505 | total_timesteps 5409.
Path 506 | total_timesteps 5423.
Path 507 | total_timesteps 5437.
Path 508 | total_timesteps 5451.
Path 509 | total_timesteps 5460.
Path 510 | total_timesteps 5470.
Path 511 | total_timesteps 5488.
Path 512 | total_timesteps 5496.
Path 513 | total_timesteps 5503.
Path 514 | total_timesteps 5510.
Path 515 | total_timesteps 5522.
Path 516 | total_timesteps 5529.
Path 517 | total_timesteps 5539.
Path 518 | total_timesteps 5550.
Path 519 | total_timesteps 5563.
Path 520 | total_timesteps 5570.
Path 521 | total_timesteps 5579.
Path 522 | total_timesteps 5594.
Path 523 | total_timesteps 5602.
Path 524 | total_timesteps 5612.
Path 525 | total_timesteps 5626.
Path 526 | total_timesteps 5635.
Path 527 | total_timesteps 5647.
Path 528 | total_timesteps 5657.
Path 529 | total_timesteps 5667.
Path 530 | total_timesteps 5675.
Path 531 | total_timesteps 5685.
Path 532 | total_timesteps 5696.
Path 533 | total_timesteps 5706.
Path 534 | total_timesteps 5718.
Path 535 | total_timesteps 5731.
Path 536 | total_timesteps 5750.
Path 537 | total_timesteps 5760.
Path 538 | total_timesteps 5772.
Path 539 | total_timesteps 5780.
Path 540 | total_timesteps 5793.
Path 541 | total_timesteps 5803.
Path 542 | total_timesteps 5812.
Path 543 | total_timesteps 5820.
Path 544 | total_timesteps 5837.
Path 545 | total_timesteps 5850.
Path 546 | total_timesteps 5857.
Path 547 | total_timesteps 5868.
Path 548 | total_timesteps 5884.
Path 549 | total_timesteps 5892.
Path 550 | total_timesteps 5906.
Path 551 | total_timesteps 5916.
Path 552 | total_timesteps 5928.
Path 553 | total_timesteps 5937.
Path 554 | total_timesteps 5955.
Path 555 | total_timesteps 5965.
Path 556 | total_timesteps 5976.
Path 557 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -5.75    |
| Iteration     | 32       |
| MaximumReturn | 1.86     |
| MinimumReturn | -20.1    |
| TotalSamples  | 136121   |
----------------------------
