Logging to experiments/gym_fwalker2d/W/Mon-07-Nov-2022-10-28-41-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2431
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 36.
Path 3 | total_timesteps 52.
Path 4 | total_timesteps 78.
Path 5 | total_timesteps 99.
Path 6 | total_timesteps 145.
Path 7 | total_timesteps 157.
Path 8 | total_timesteps 183.
Path 9 | total_timesteps 200.
Path 10 | total_timesteps 213.
Path 11 | total_timesteps 231.
Path 12 | total_timesteps 247.
Path 13 | total_timesteps 256.
Path 14 | total_timesteps 270.
Path 15 | total_timesteps 280.
Path 16 | total_timesteps 296.
Path 17 | total_timesteps 318.
Path 18 | total_timesteps 333.
Path 19 | total_timesteps 344.
Path 20 | total_timesteps 356.
Path 21 | total_timesteps 376.
Path 22 | total_timesteps 390.
Path 23 | total_timesteps 422.
Path 24 | total_timesteps 434.
Path 25 | total_timesteps 476.
Path 26 | total_timesteps 503.
Path 27 | total_timesteps 519.
Path 28 | total_timesteps 549.
Path 29 | total_timesteps 562.
Path 30 | total_timesteps 587.
Path 31 | total_timesteps 600.
Path 32 | total_timesteps 613.
Path 33 | total_timesteps 650.
Path 34 | total_timesteps 673.
Path 35 | total_timesteps 712.
Path 36 | total_timesteps 733.
Path 37 | total_timesteps 763.
Path 38 | total_timesteps 787.
Path 39 | total_timesteps 821.
Path 40 | total_timesteps 833.
Path 41 | total_timesteps 845.
Path 42 | total_timesteps 871.
Path 43 | total_timesteps 897.
Path 44 | total_timesteps 918.
Path 45 | total_timesteps 937.
Path 46 | total_timesteps 957.
Path 47 | total_timesteps 976.
Path 48 | total_timesteps 997.
Path 49 | total_timesteps 1010.
Path 50 | total_timesteps 1025.
Path 51 | total_timesteps 1053.
Path 52 | total_timesteps 1067.
Path 53 | total_timesteps 1084.
Path 54 | total_timesteps 1100.
Path 55 | total_timesteps 1118.
Path 56 | total_timesteps 1135.
Path 57 | total_timesteps 1166.
Path 58 | total_timesteps 1176.
Path 59 | total_timesteps 1195.
Path 60 | total_timesteps 1204.
Path 61 | total_timesteps 1226.
Path 62 | total_timesteps 1252.
Path 63 | total_timesteps 1262.
Path 64 | total_timesteps 1280.
Path 65 | total_timesteps 1295.
Path 66 | total_timesteps 1307.
Path 67 | total_timesteps 1329.
Path 68 | total_timesteps 1346.
Path 69 | total_timesteps 1359.
Path 70 | total_timesteps 1372.
Path 71 | total_timesteps 1384.
Path 72 | total_timesteps 1405.
Path 73 | total_timesteps 1440.
Path 74 | total_timesteps 1454.
Path 75 | total_timesteps 1464.
Path 76 | total_timesteps 1487.
Path 77 | total_timesteps 1497.
Path 78 | total_timesteps 1509.
Path 79 | total_timesteps 1562.
Path 80 | total_timesteps 1576.
Path 81 | total_timesteps 1588.
Path 82 | total_timesteps 1597.
Path 83 | total_timesteps 1622.
Path 84 | total_timesteps 1637.
Path 85 | total_timesteps 1668.
Path 86 | total_timesteps 1680.
Path 87 | total_timesteps 1710.
Path 88 | total_timesteps 1730.
Path 89 | total_timesteps 1742.
Path 90 | total_timesteps 1755.
Path 91 | total_timesteps 1766.
Path 92 | total_timesteps 1781.
Path 93 | total_timesteps 1799.
Path 94 | total_timesteps 1823.
Path 95 | total_timesteps 1835.
Path 96 | total_timesteps 1852.
Path 97 | total_timesteps 1873.
Path 98 | total_timesteps 1883.
Path 99 | total_timesteps 1908.
Path 100 | total_timesteps 1917.
Path 101 | total_timesteps 1934.
Path 102 | total_timesteps 1969.
Path 103 | total_timesteps 1983.
Path 104 | total_timesteps 2001.
Path 105 | total_timesteps 2035.
Path 106 | total_timesteps 2050.
Path 107 | total_timesteps 2062.
Path 108 | total_timesteps 2084.
Path 109 | total_timesteps 2103.
Path 110 | total_timesteps 2120.
Path 111 | total_timesteps 2153.
Path 112 | total_timesteps 2165.
Path 113 | total_timesteps 2176.
Path 114 | total_timesteps 2192.
Path 115 | total_timesteps 2206.
Path 116 | total_timesteps 2223.
Path 117 | total_timesteps 2254.
Path 118 | total_timesteps 2289.
Path 119 | total_timesteps 2314.
Path 120 | total_timesteps 2329.
Path 121 | total_timesteps 2360.
Path 122 | total_timesteps 2387.
Path 123 | total_timesteps 2418.
Path 124 | total_timesteps 2431.
Path 125 | total_timesteps 2454.
Path 126 | total_timesteps 2472.
Path 127 | total_timesteps 2508.
Path 128 | total_timesteps 2550.
Path 129 | total_timesteps 2567.
Path 130 | total_timesteps 2598.
Path 131 | total_timesteps 2613.
Path 132 | total_timesteps 2627.
Path 133 | total_timesteps 2648.
Path 134 | total_timesteps 2661.
Path 135 | total_timesteps 2676.
Path 136 | total_timesteps 2691.
Path 137 | total_timesteps 2707.
Path 138 | total_timesteps 2719.
Path 139 | total_timesteps 2732.
Path 140 | total_timesteps 2748.
Path 141 | total_timesteps 2763.
Path 142 | total_timesteps 2776.
Path 143 | total_timesteps 2790.
Path 144 | total_timesteps 2806.
Path 145 | total_timesteps 2821.
Path 146 | total_timesteps 2833.
Path 147 | total_timesteps 2866.
Path 148 | total_timesteps 2893.
Path 149 | total_timesteps 2906.
Path 150 | total_timesteps 2932.
Path 151 | total_timesteps 2944.
Path 152 | total_timesteps 2962.
Path 153 | total_timesteps 2973.
Path 154 | total_timesteps 2985.
Path 155 | total_timesteps 2999.
Path 156 | total_timesteps 3009.
Path 157 | total_timesteps 3024.
Path 158 | total_timesteps 3043.
Path 159 | total_timesteps 3076.
Path 160 | total_timesteps 3105.
Path 161 | total_timesteps 3119.
Path 162 | total_timesteps 3137.
Path 163 | total_timesteps 3148.
Path 164 | total_timesteps 3165.
Path 165 | total_timesteps 3199.
Path 166 | total_timesteps 3213.
Path 167 | total_timesteps 3236.
Path 168 | total_timesteps 3251.
Path 169 | total_timesteps 3269.
Path 170 | total_timesteps 3292.
Path 171 | total_timesteps 3319.
Path 172 | total_timesteps 3341.
Path 173 | total_timesteps 3374.
Path 174 | total_timesteps 3398.
Path 175 | total_timesteps 3409.
Path 176 | total_timesteps 3427.
Path 177 | total_timesteps 3465.
Path 178 | total_timesteps 3477.
Path 179 | total_timesteps 3518.
Path 180 | total_timesteps 3533.
Path 181 | total_timesteps 3567.
Path 182 | total_timesteps 3586.
Path 183 | total_timesteps 3594.
Path 184 | total_timesteps 3634.
Path 185 | total_timesteps 3656.
Path 186 | total_timesteps 3682.
Path 187 | total_timesteps 3713.
Path 188 | total_timesteps 3734.
Path 189 | total_timesteps 3750.
Path 190 | total_timesteps 3764.
Path 191 | total_timesteps 3787.
Path 192 | total_timesteps 3814.
Path 193 | total_timesteps 3825.
Path 194 | total_timesteps 3845.
Path 195 | total_timesteps 3864.
Path 196 | total_timesteps 3876.
Path 197 | total_timesteps 3916.
Path 198 | total_timesteps 3927.
Path 199 | total_timesteps 3938.
Path 200 | total_timesteps 3950.
Path 201 | total_timesteps 3989.
Path 202 | total_timesteps 4001.
Path 203 | total_timesteps 4029.
Path 204 | total_timesteps 4049.
Path 205 | total_timesteps 4059.
Path 206 | total_timesteps 4083.
Path 207 | total_timesteps 4094.
Path 208 | total_timesteps 4114.
Path 209 | total_timesteps 4131.
Path 210 | total_timesteps 4151.
Path 211 | total_timesteps 4161.
Path 212 | total_timesteps 4206.
Path 213 | total_timesteps 4221.
Path 214 | total_timesteps 4235.
Path 215 | total_timesteps 4249.
Path 216 | total_timesteps 4262.
Path 217 | total_timesteps 4275.
Path 218 | total_timesteps 4294.
Path 219 | total_timesteps 4336.
Path 220 | total_timesteps 4357.
Path 221 | total_timesteps 4388.
Path 222 | total_timesteps 4401.
Path 223 | total_timesteps 4413.
Path 224 | total_timesteps 4442.
Path 225 | total_timesteps 4455.
Path 226 | total_timesteps 4466.
Path 227 | total_timesteps 4481.
Path 228 | total_timesteps 4513.
Path 229 | total_timesteps 4524.
Path 230 | total_timesteps 4537.
Path 231 | total_timesteps 4565.
Path 232 | total_timesteps 4583.
Path 233 | total_timesteps 4603.
Path 234 | total_timesteps 4618.
Path 235 | total_timesteps 4635.
Path 236 | total_timesteps 4650.
Path 237 | total_timesteps 4681.
Path 238 | total_timesteps 4697.
Path 239 | total_timesteps 4716.
Path 240 | total_timesteps 4735.
Path 241 | total_timesteps 4748.
Path 242 | total_timesteps 4757.
Path 243 | total_timesteps 4783.
Path 244 | total_timesteps 4797.
Path 245 | total_timesteps 4820.
Path 246 | total_timesteps 4840.
Path 247 | total_timesteps 4862.
Path 248 | total_timesteps 4880.
Path 249 | total_timesteps 4892.
Path 250 | total_timesteps 4914.
Path 251 | total_timesteps 4934.
Path 252 | total_timesteps 4949.
Path 253 | total_timesteps 4970.
Path 254 | total_timesteps 4997.
Path 255 | total_timesteps 5023.
Path 256 | total_timesteps 5075.
Path 257 | total_timesteps 5085.
Path 258 | total_timesteps 5106.
Path 259 | total_timesteps 5123.
Path 260 | total_timesteps 5156.
Path 261 | total_timesteps 5184.
Path 262 | total_timesteps 5204.
Path 263 | total_timesteps 5227.
Path 264 | total_timesteps 5258.
Path 265 | total_timesteps 5271.
Path 266 | total_timesteps 5301.
Path 267 | total_timesteps 5322.
Path 268 | total_timesteps 5341.
Path 269 | total_timesteps 5355.
Path 270 | total_timesteps 5376.
Path 271 | total_timesteps 5393.
Path 272 | total_timesteps 5420.
Path 273 | total_timesteps 5437.
Path 274 | total_timesteps 5464.
Path 275 | total_timesteps 5478.
Path 276 | total_timesteps 5492.
Path 277 | total_timesteps 5522.
Path 278 | total_timesteps 5540.
Path 279 | total_timesteps 5558.
Path 280 | total_timesteps 5587.
Path 281 | total_timesteps 5616.
Path 282 | total_timesteps 5629.
Path 283 | total_timesteps 5659.
Path 284 | total_timesteps 5679.
Path 285 | total_timesteps 5700.
Path 286 | total_timesteps 5723.
Path 287 | total_timesteps 5737.
Path 288 | total_timesteps 5756.
Path 289 | total_timesteps 5772.
Path 290 | total_timesteps 5793.
Path 291 | total_timesteps 5809.
Path 292 | total_timesteps 5842.
Path 293 | total_timesteps 5862.
Path 294 | total_timesteps 5890.
Path 295 | total_timesteps 5904.
Path 296 | total_timesteps 5914.
Path 297 | total_timesteps 5937.
Path 298 | total_timesteps 5975.
Path 299 | total_timesteps 5997.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.41791313886642456
Validation loss = 0.13282744586467743
Validation loss = 0.10138633847236633
Validation loss = 0.08623705804347992
Validation loss = 0.07949818670749664
Validation loss = 0.07224518060684204
Validation loss = 0.06998300552368164
Validation loss = 0.06155596673488617
Validation loss = 0.06441110372543335
Validation loss = 0.05502587929368019
Validation loss = 0.057907890528440475
Validation loss = 0.052010465413331985
Validation loss = 0.05303063988685608
Validation loss = 0.059594035148620605
Validation loss = 0.05017832666635513
Validation loss = 0.05179467052221298
Validation loss = 0.050203531980514526
Validation loss = 0.04695159196853638
Validation loss = 0.04813884198665619
Validation loss = 0.04682491719722748
Validation loss = 0.05322851240634918
Validation loss = 0.047127123922109604
Validation loss = 0.05315571278333664
Validation loss = 0.047271668910980225
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 46.
Path 2 | total_timesteps 80.
Path 3 | total_timesteps 106.
Path 4 | total_timesteps 142.
Path 5 | total_timesteps 169.
Path 6 | total_timesteps 190.
Path 7 | total_timesteps 220.
Path 8 | total_timesteps 242.
Path 9 | total_timesteps 260.
Path 10 | total_timesteps 273.
Path 11 | total_timesteps 297.
Path 12 | total_timesteps 333.
Path 13 | total_timesteps 352.
Path 14 | total_timesteps 372.
Path 15 | total_timesteps 401.
Path 16 | total_timesteps 414.
Path 17 | total_timesteps 435.
Path 18 | total_timesteps 457.
Path 19 | total_timesteps 478.
Path 20 | total_timesteps 502.
Path 21 | total_timesteps 510.
Path 22 | total_timesteps 523.
Path 23 | total_timesteps 535.
Path 24 | total_timesteps 568.
Path 25 | total_timesteps 594.
Path 26 | total_timesteps 610.
Path 27 | total_timesteps 646.
Path 28 | total_timesteps 664.
Path 29 | total_timesteps 684.
Path 30 | total_timesteps 699.
Path 31 | total_timesteps 712.
Path 32 | total_timesteps 723.
Path 33 | total_timesteps 772.
Path 34 | total_timesteps 798.
Path 35 | total_timesteps 828.
Path 36 | total_timesteps 861.
Path 37 | total_timesteps 883.
Path 38 | total_timesteps 898.
Path 39 | total_timesteps 917.
Path 40 | total_timesteps 950.
Path 41 | total_timesteps 983.
Path 42 | total_timesteps 1013.
Path 43 | total_timesteps 1035.
Path 44 | total_timesteps 1072.
Path 45 | total_timesteps 1092.
Path 46 | total_timesteps 1122.
Path 47 | total_timesteps 1147.
Path 48 | total_timesteps 1167.
Path 49 | total_timesteps 1201.
Path 50 | total_timesteps 1216.
Path 51 | total_timesteps 1255.
Path 52 | total_timesteps 1282.
Path 53 | total_timesteps 1317.
Path 54 | total_timesteps 1347.
Path 55 | total_timesteps 1364.
Path 56 | total_timesteps 1386.
Path 57 | total_timesteps 1409.
Path 58 | total_timesteps 1420.
Path 59 | total_timesteps 1434.
Path 60 | total_timesteps 1456.
Path 61 | total_timesteps 1478.
Path 62 | total_timesteps 1508.
Path 63 | total_timesteps 1552.
Path 64 | total_timesteps 1590.
Path 65 | total_timesteps 1614.
Path 66 | total_timesteps 1629.
Path 67 | total_timesteps 1663.
Path 68 | total_timesteps 1677.
Path 69 | total_timesteps 1753.
Path 70 | total_timesteps 1777.
Path 71 | total_timesteps 1808.
Path 72 | total_timesteps 1839.
Path 73 | total_timesteps 1865.
Path 74 | total_timesteps 1891.
Path 75 | total_timesteps 1921.
Path 76 | total_timesteps 1943.
Path 77 | total_timesteps 1973.
Path 78 | total_timesteps 2002.
Path 79 | total_timesteps 2035.
Path 80 | total_timesteps 2061.
Path 81 | total_timesteps 2083.
Path 82 | total_timesteps 2109.
Path 83 | total_timesteps 2124.
Path 84 | total_timesteps 2193.
Path 85 | total_timesteps 2218.
Path 86 | total_timesteps 2255.
Path 87 | total_timesteps 2282.
Path 88 | total_timesteps 2325.
Path 89 | total_timesteps 2348.
Path 90 | total_timesteps 2367.
Path 91 | total_timesteps 2381.
Path 92 | total_timesteps 2410.
Path 93 | total_timesteps 2440.
Path 94 | total_timesteps 2457.
Path 95 | total_timesteps 2515.
Path 96 | total_timesteps 2555.
Path 97 | total_timesteps 2562.
Path 98 | total_timesteps 2586.
Path 99 | total_timesteps 2616.
Path 100 | total_timesteps 2630.
Path 101 | total_timesteps 2663.
Path 102 | total_timesteps 2687.
Path 103 | total_timesteps 2726.
Path 104 | total_timesteps 2761.
Path 105 | total_timesteps 2789.
Path 106 | total_timesteps 2812.
Path 107 | total_timesteps 2830.
Path 108 | total_timesteps 2864.
Path 109 | total_timesteps 2879.
Path 110 | total_timesteps 2913.
Path 111 | total_timesteps 2942.
Path 112 | total_timesteps 2956.
Path 113 | total_timesteps 2984.
Path 114 | total_timesteps 3002.
Path 115 | total_timesteps 3050.
Path 116 | total_timesteps 3092.
Path 117 | total_timesteps 3127.
Path 118 | total_timesteps 3160.
Path 119 | total_timesteps 3182.
Path 120 | total_timesteps 3206.
Path 121 | total_timesteps 3224.
Path 122 | total_timesteps 3235.
Path 123 | total_timesteps 3266.
Path 124 | total_timesteps 3279.
Path 125 | total_timesteps 3315.
Path 126 | total_timesteps 3337.
Path 127 | total_timesteps 3357.
Path 128 | total_timesteps 3407.
Path 129 | total_timesteps 3440.
Path 130 | total_timesteps 3459.
Path 131 | total_timesteps 3486.
Path 132 | total_timesteps 3506.
Path 133 | total_timesteps 3550.
Path 134 | total_timesteps 3595.
Path 135 | total_timesteps 3621.
Path 136 | total_timesteps 3669.
Path 137 | total_timesteps 3694.
Path 138 | total_timesteps 3712.
Path 139 | total_timesteps 3730.
Path 140 | total_timesteps 3773.
Path 141 | total_timesteps 3788.
Path 142 | total_timesteps 3828.
Path 143 | total_timesteps 3858.
Path 144 | total_timesteps 3871.
Path 145 | total_timesteps 3910.
Path 146 | total_timesteps 3928.
Path 147 | total_timesteps 3944.
Path 148 | total_timesteps 3957.
Path 149 | total_timesteps 3981.
Path 150 | total_timesteps 4020.
Path 151 | total_timesteps 4048.
Path 152 | total_timesteps 4070.
Path 153 | total_timesteps 4082.
Path 154 | total_timesteps 4095.
Path 155 | total_timesteps 4117.
Path 156 | total_timesteps 4168.
Path 157 | total_timesteps 4235.
Path 158 | total_timesteps 4246.
Path 159 | total_timesteps 4261.
Path 160 | total_timesteps 4280.
Path 161 | total_timesteps 4306.
Path 162 | total_timesteps 4329.
Path 163 | total_timesteps 4367.
Path 164 | total_timesteps 4390.
Path 165 | total_timesteps 4415.
Path 166 | total_timesteps 4439.
Path 167 | total_timesteps 4467.
Path 168 | total_timesteps 4485.
Path 169 | total_timesteps 4503.
Path 170 | total_timesteps 4520.
Path 171 | total_timesteps 4543.
Path 172 | total_timesteps 4566.
Path 173 | total_timesteps 4580.
Path 174 | total_timesteps 4616.
Path 175 | total_timesteps 4631.
Path 176 | total_timesteps 4660.
Path 177 | total_timesteps 4680.
Path 178 | total_timesteps 4693.
Path 179 | total_timesteps 4719.
Path 180 | total_timesteps 4759.
Path 181 | total_timesteps 4779.
Path 182 | total_timesteps 4797.
Path 183 | total_timesteps 4813.
Path 184 | total_timesteps 4832.
Path 185 | total_timesteps 4849.
Path 186 | total_timesteps 4866.
Path 187 | total_timesteps 4889.
Path 188 | total_timesteps 4913.
Path 189 | total_timesteps 4942.
Path 190 | total_timesteps 4951.
Path 191 | total_timesteps 4978.
Path 192 | total_timesteps 5004.
Path 193 | total_timesteps 5023.
Path 194 | total_timesteps 5061.
Path 195 | total_timesteps 5071.
Path 196 | total_timesteps 5084.
Path 197 | total_timesteps 5111.
Path 198 | total_timesteps 5139.
Path 199 | total_timesteps 5154.
Path 200 | total_timesteps 5172.
Path 201 | total_timesteps 5198.
Path 202 | total_timesteps 5222.
Path 203 | total_timesteps 5235.
Path 204 | total_timesteps 5277.
Path 205 | total_timesteps 5317.
Path 206 | total_timesteps 5345.
Path 207 | total_timesteps 5367.
Path 208 | total_timesteps 5391.
Path 209 | total_timesteps 5413.
Path 210 | total_timesteps 5428.
Path 211 | total_timesteps 5465.
Path 212 | total_timesteps 5493.
Path 213 | total_timesteps 5504.
Path 214 | total_timesteps 5534.
Path 215 | total_timesteps 5559.
Path 216 | total_timesteps 5568.
Path 217 | total_timesteps 5591.
Path 218 | total_timesteps 5609.
Path 219 | total_timesteps 5632.
Path 220 | total_timesteps 5678.
Path 221 | total_timesteps 5690.
Path 222 | total_timesteps 5715.
Path 223 | total_timesteps 5745.
Path 224 | total_timesteps 5759.
Path 225 | total_timesteps 5795.
Path 226 | total_timesteps 5821.
Path 227 | total_timesteps 5842.
Path 228 | total_timesteps 5856.
Path 229 | total_timesteps 5875.
Path 230 | total_timesteps 5901.
Path 231 | total_timesteps 5931.
Path 232 | total_timesteps 5957.
Path 233 | total_timesteps 5982.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.61    |
| Iteration     | 0        |
| MaximumReturn | 10.1     |
| MinimumReturn | -21.1    |
| TotalSamples  | 8018     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.09315649420022964
Validation loss = 0.07171794772148132
Validation loss = 0.06160394102334976
Validation loss = 0.04740510880947113
Validation loss = 0.04817066714167595
Validation loss = 0.04253459721803665
Validation loss = 0.04285194352269173
Validation loss = 0.04159104451537132
Validation loss = 0.043399639427661896
Validation loss = 0.03926148638129234
Validation loss = 0.039488669484853745
Validation loss = 0.04176247864961624
Validation loss = 0.041881121695041656
Validation loss = 0.03969857096672058
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 20.
Path 2 | total_timesteps 37.
Path 3 | total_timesteps 56.
Path 4 | total_timesteps 78.
Path 5 | total_timesteps 108.
Path 6 | total_timesteps 140.
Path 7 | total_timesteps 153.
Path 8 | total_timesteps 180.
Path 9 | total_timesteps 199.
Path 10 | total_timesteps 234.
Path 11 | total_timesteps 287.
Path 12 | total_timesteps 342.
Path 13 | total_timesteps 354.
Path 14 | total_timesteps 364.
Path 15 | total_timesteps 396.
Path 16 | total_timesteps 452.
Path 17 | total_timesteps 482.
Path 18 | total_timesteps 506.
Path 19 | total_timesteps 534.
Path 20 | total_timesteps 564.
Path 21 | total_timesteps 585.
Path 22 | total_timesteps 596.
Path 23 | total_timesteps 627.
Path 24 | total_timesteps 675.
Path 25 | total_timesteps 694.
Path 26 | total_timesteps 710.
Path 27 | total_timesteps 742.
Path 28 | total_timesteps 761.
Path 29 | total_timesteps 796.
Path 30 | total_timesteps 823.
Path 31 | total_timesteps 845.
Path 32 | total_timesteps 877.
Path 33 | total_timesteps 919.
Path 34 | total_timesteps 942.
Path 35 | total_timesteps 964.
Path 36 | total_timesteps 984.
Path 37 | total_timesteps 1008.
Path 38 | total_timesteps 1047.
Path 39 | total_timesteps 1074.
Path 40 | total_timesteps 1104.
Path 41 | total_timesteps 1139.
Path 42 | total_timesteps 1204.
Path 43 | total_timesteps 1216.
Path 44 | total_timesteps 1232.
Path 45 | total_timesteps 1244.
Path 46 | total_timesteps 1276.
Path 47 | total_timesteps 1311.
Path 48 | total_timesteps 1344.
Path 49 | total_timesteps 1384.
Path 50 | total_timesteps 1407.
Path 51 | total_timesteps 1464.
Path 52 | total_timesteps 1490.
Path 53 | total_timesteps 1537.
Path 54 | total_timesteps 1577.
Path 55 | total_timesteps 1627.
Path 56 | total_timesteps 1639.
Path 57 | total_timesteps 1690.
Path 58 | total_timesteps 1724.
Path 59 | total_timesteps 1761.
Path 60 | total_timesteps 1776.
Path 61 | total_timesteps 1799.
Path 62 | total_timesteps 1815.
Path 63 | total_timesteps 1841.
Path 64 | total_timesteps 1862.
Path 65 | total_timesteps 1900.
Path 66 | total_timesteps 1914.
Path 67 | total_timesteps 1966.
Path 68 | total_timesteps 1988.
Path 69 | total_timesteps 2039.
Path 70 | total_timesteps 2054.
Path 71 | total_timesteps 2083.
Path 72 | total_timesteps 2098.
Path 73 | total_timesteps 2140.
Path 74 | total_timesteps 2170.
Path 75 | total_timesteps 2186.
Path 76 | total_timesteps 2214.
Path 77 | total_timesteps 2238.
Path 78 | total_timesteps 2255.
Path 79 | total_timesteps 2291.
Path 80 | total_timesteps 2314.
Path 81 | total_timesteps 2359.
Path 82 | total_timesteps 2380.
Path 83 | total_timesteps 2405.
Path 84 | total_timesteps 2419.
Path 85 | total_timesteps 2449.
Path 86 | total_timesteps 2473.
Path 87 | total_timesteps 2489.
Path 88 | total_timesteps 2513.
Path 89 | total_timesteps 2537.
Path 90 | total_timesteps 2572.
Path 91 | total_timesteps 2634.
Path 92 | total_timesteps 2671.
Path 93 | total_timesteps 2716.
Path 94 | total_timesteps 2738.
Path 95 | total_timesteps 2779.
Path 96 | total_timesteps 2810.
Path 97 | total_timesteps 2843.
Path 98 | total_timesteps 2883.
Path 99 | total_timesteps 2910.
Path 100 | total_timesteps 2979.
Path 101 | total_timesteps 3003.
Path 102 | total_timesteps 3031.
Path 103 | total_timesteps 3060.
Path 104 | total_timesteps 3086.
Path 105 | total_timesteps 3106.
Path 106 | total_timesteps 3133.
Path 107 | total_timesteps 3183.
Path 108 | total_timesteps 3209.
Path 109 | total_timesteps 3221.
Path 110 | total_timesteps 3267.
Path 111 | total_timesteps 3299.
Path 112 | total_timesteps 3320.
Path 113 | total_timesteps 3355.
Path 114 | total_timesteps 3378.
Path 115 | total_timesteps 3405.
Path 116 | total_timesteps 3437.
Path 117 | total_timesteps 3465.
Path 118 | total_timesteps 3491.
Path 119 | total_timesteps 3519.
Path 120 | total_timesteps 3531.
Path 121 | total_timesteps 3557.
Path 122 | total_timesteps 3574.
Path 123 | total_timesteps 3600.
Path 124 | total_timesteps 3624.
Path 125 | total_timesteps 3635.
Path 126 | total_timesteps 3657.
Path 127 | total_timesteps 3666.
Path 128 | total_timesteps 3703.
Path 129 | total_timesteps 3728.
Path 130 | total_timesteps 3778.
Path 131 | total_timesteps 3803.
Path 132 | total_timesteps 3831.
Path 133 | total_timesteps 3854.
Path 134 | total_timesteps 3876.
Path 135 | total_timesteps 3894.
Path 136 | total_timesteps 3939.
Path 137 | total_timesteps 3961.
Path 138 | total_timesteps 3999.
Path 139 | total_timesteps 4041.
Path 140 | total_timesteps 4077.
Path 141 | total_timesteps 4094.
Path 142 | total_timesteps 4110.
Path 143 | total_timesteps 4142.
Path 144 | total_timesteps 4168.
Path 145 | total_timesteps 4184.
Path 146 | total_timesteps 4203.
Path 147 | total_timesteps 4225.
Path 148 | total_timesteps 4255.
Path 149 | total_timesteps 4277.
Path 150 | total_timesteps 4299.
Path 151 | total_timesteps 4322.
Path 152 | total_timesteps 4339.
Path 153 | total_timesteps 4363.
Path 154 | total_timesteps 4383.
Path 155 | total_timesteps 4420.
Path 156 | total_timesteps 4455.
Path 157 | total_timesteps 4506.
Path 158 | total_timesteps 4518.
Path 159 | total_timesteps 4559.
Path 160 | total_timesteps 4575.
Path 161 | total_timesteps 4620.
Path 162 | total_timesteps 4645.
Path 163 | total_timesteps 4674.
Path 164 | total_timesteps 4693.
Path 165 | total_timesteps 4710.
Path 166 | total_timesteps 4728.
Path 167 | total_timesteps 4748.
Path 168 | total_timesteps 4774.
Path 169 | total_timesteps 4797.
Path 170 | total_timesteps 4814.
Path 171 | total_timesteps 4841.
Path 172 | total_timesteps 4874.
Path 173 | total_timesteps 4895.
Path 174 | total_timesteps 4923.
Path 175 | total_timesteps 4986.
Path 176 | total_timesteps 5031.
Path 177 | total_timesteps 5057.
Path 178 | total_timesteps 5076.
Path 179 | total_timesteps 5096.
Path 180 | total_timesteps 5117.
Path 181 | total_timesteps 5137.
Path 182 | total_timesteps 5176.
Path 183 | total_timesteps 5216.
Path 184 | total_timesteps 5251.
Path 185 | total_timesteps 5274.
Path 186 | total_timesteps 5286.
Path 187 | total_timesteps 5310.
Path 188 | total_timesteps 5340.
Path 189 | total_timesteps 5374.
Path 190 | total_timesteps 5423.
Path 191 | total_timesteps 5455.
Path 192 | total_timesteps 5464.
Path 193 | total_timesteps 5505.
Path 194 | total_timesteps 5528.
Path 195 | total_timesteps 5550.
Path 196 | total_timesteps 5583.
Path 197 | total_timesteps 5630.
Path 198 | total_timesteps 5650.
Path 199 | total_timesteps 5708.
Path 200 | total_timesteps 5735.
Path 201 | total_timesteps 5755.
Path 202 | total_timesteps 5787.
Path 203 | total_timesteps 5797.
Path 204 | total_timesteps 5822.
Path 205 | total_timesteps 5842.
Path 206 | total_timesteps 5863.
Path 207 | total_timesteps 5893.
Path 208 | total_timesteps 5952.
Path 209 | total_timesteps 5977.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.99    |
| Iteration     | 1        |
| MaximumReturn | 23.6     |
| MinimumReturn | -27.1    |
| TotalSamples  | 12025    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.05269767716526985
Validation loss = 0.03507217392325401
Validation loss = 0.037705112248659134
Validation loss = 0.032448478043079376
Validation loss = 0.03328753635287285
Validation loss = 0.044812459498643875
Validation loss = 0.030944397673010826
Validation loss = 0.03243853151798248
Validation loss = 0.03031393140554428
Validation loss = 0.0307539701461792
Validation loss = 0.03255082666873932
Validation loss = 0.030124744400382042
Validation loss = 0.02815396897494793
Validation loss = 0.029042840003967285
Validation loss = 0.027741998434066772
Validation loss = 0.02901257760822773
Validation loss = 0.027050137519836426
Validation loss = 0.02667538821697235
Validation loss = 0.035994891077280045
Validation loss = 0.05379394814372063
Validation loss = 0.0264158695936203
Validation loss = 0.032152142375707626
Validation loss = 0.02934553660452366
Validation loss = 0.028974361717700958
Validation loss = 0.02866947464644909
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 68.
Path 4 | total_timesteps 97.
Path 5 | total_timesteps 169.
Path 6 | total_timesteps 189.
Path 7 | total_timesteps 240.
Path 8 | total_timesteps 288.
Path 9 | total_timesteps 309.
Path 10 | total_timesteps 341.
Path 11 | total_timesteps 369.
Path 12 | total_timesteps 391.
Path 13 | total_timesteps 409.
Path 14 | total_timesteps 427.
Path 15 | total_timesteps 450.
Path 16 | total_timesteps 464.
Path 17 | total_timesteps 499.
Path 18 | total_timesteps 517.
Path 19 | total_timesteps 542.
Path 20 | total_timesteps 560.
Path 21 | total_timesteps 601.
Path 22 | total_timesteps 634.
Path 23 | total_timesteps 647.
Path 24 | total_timesteps 677.
Path 25 | total_timesteps 691.
Path 26 | total_timesteps 711.
Path 27 | total_timesteps 751.
Path 28 | total_timesteps 777.
Path 29 | total_timesteps 801.
Path 30 | total_timesteps 833.
Path 31 | total_timesteps 881.
Path 32 | total_timesteps 910.
Path 33 | total_timesteps 969.
Path 34 | total_timesteps 1002.
Path 35 | total_timesteps 1015.
Path 36 | total_timesteps 1027.
Path 37 | total_timesteps 1051.
Path 38 | total_timesteps 1079.
Path 39 | total_timesteps 1099.
Path 40 | total_timesteps 1109.
Path 41 | total_timesteps 1140.
Path 42 | total_timesteps 1202.
Path 43 | total_timesteps 1231.
Path 44 | total_timesteps 1249.
Path 45 | total_timesteps 1285.
Path 46 | total_timesteps 1318.
Path 47 | total_timesteps 1349.
Path 48 | total_timesteps 1384.
Path 49 | total_timesteps 1405.
Path 50 | total_timesteps 1416.
Path 51 | total_timesteps 1446.
Path 52 | total_timesteps 1461.
Path 53 | total_timesteps 1494.
Path 54 | total_timesteps 1522.
Path 55 | total_timesteps 1554.
Path 56 | total_timesteps 1569.
Path 57 | total_timesteps 1591.
Path 58 | total_timesteps 1622.
Path 59 | total_timesteps 1648.
Path 60 | total_timesteps 1669.
Path 61 | total_timesteps 1696.
Path 62 | total_timesteps 1724.
Path 63 | total_timesteps 1750.
Path 64 | total_timesteps 1804.
Path 65 | total_timesteps 1841.
Path 66 | total_timesteps 1858.
Path 67 | total_timesteps 1878.
Path 68 | total_timesteps 1917.
Path 69 | total_timesteps 1939.
Path 70 | total_timesteps 1962.
Path 71 | total_timesteps 1992.
Path 72 | total_timesteps 2018.
Path 73 | total_timesteps 2041.
Path 74 | total_timesteps 2103.
Path 75 | total_timesteps 2134.
Path 76 | total_timesteps 2150.
Path 77 | total_timesteps 2176.
Path 78 | total_timesteps 2212.
Path 79 | total_timesteps 2254.
Path 80 | total_timesteps 2278.
Path 81 | total_timesteps 2314.
Path 82 | total_timesteps 2341.
Path 83 | total_timesteps 2369.
Path 84 | total_timesteps 2384.
Path 85 | total_timesteps 2425.
Path 86 | total_timesteps 2449.
Path 87 | total_timesteps 2465.
Path 88 | total_timesteps 2499.
Path 89 | total_timesteps 2534.
Path 90 | total_timesteps 2557.
Path 91 | total_timesteps 2575.
Path 92 | total_timesteps 2594.
Path 93 | total_timesteps 2624.
Path 94 | total_timesteps 2640.
Path 95 | total_timesteps 2666.
Path 96 | total_timesteps 2687.
Path 97 | total_timesteps 2695.
Path 98 | total_timesteps 2728.
Path 99 | total_timesteps 2749.
Path 100 | total_timesteps 2784.
Path 101 | total_timesteps 2811.
Path 102 | total_timesteps 2832.
Path 103 | total_timesteps 2851.
Path 104 | total_timesteps 2870.
Path 105 | total_timesteps 2893.
Path 106 | total_timesteps 2911.
Path 107 | total_timesteps 2942.
Path 108 | total_timesteps 2960.
Path 109 | total_timesteps 3019.
Path 110 | total_timesteps 3047.
Path 111 | total_timesteps 3059.
Path 112 | total_timesteps 3086.
Path 113 | total_timesteps 3099.
Path 114 | total_timesteps 3130.
Path 115 | total_timesteps 3177.
Path 116 | total_timesteps 3195.
Path 117 | total_timesteps 3223.
Path 118 | total_timesteps 3246.
Path 119 | total_timesteps 3310.
Path 120 | total_timesteps 3334.
Path 121 | total_timesteps 3343.
Path 122 | total_timesteps 3367.
Path 123 | total_timesteps 3391.
Path 124 | total_timesteps 3411.
Path 125 | total_timesteps 3431.
Path 126 | total_timesteps 3456.
Path 127 | total_timesteps 3468.
Path 128 | total_timesteps 3512.
Path 129 | total_timesteps 3535.
Path 130 | total_timesteps 3562.
Path 131 | total_timesteps 3579.
Path 132 | total_timesteps 3597.
Path 133 | total_timesteps 3618.
Path 134 | total_timesteps 3635.
Path 135 | total_timesteps 3653.
Path 136 | total_timesteps 3684.
Path 137 | total_timesteps 3730.
Path 138 | total_timesteps 3760.
Path 139 | total_timesteps 3781.
Path 140 | total_timesteps 3805.
Path 141 | total_timesteps 3830.
Path 142 | total_timesteps 3842.
Path 143 | total_timesteps 3866.
Path 144 | total_timesteps 3881.
Path 145 | total_timesteps 3910.
Path 146 | total_timesteps 3947.
Path 147 | total_timesteps 3975.
Path 148 | total_timesteps 3986.
Path 149 | total_timesteps 4032.
Path 150 | total_timesteps 4049.
Path 151 | total_timesteps 4061.
Path 152 | total_timesteps 4091.
Path 153 | total_timesteps 4136.
Path 154 | total_timesteps 4180.
Path 155 | total_timesteps 4204.
Path 156 | total_timesteps 4227.
Path 157 | total_timesteps 4249.
Path 158 | total_timesteps 4267.
Path 159 | total_timesteps 4296.
Path 160 | total_timesteps 4313.
Path 161 | total_timesteps 4335.
Path 162 | total_timesteps 4355.
Path 163 | total_timesteps 4386.
Path 164 | total_timesteps 4407.
Path 165 | total_timesteps 4431.
Path 166 | total_timesteps 4470.
Path 167 | total_timesteps 4489.
Path 168 | total_timesteps 4521.
Path 169 | total_timesteps 4540.
Path 170 | total_timesteps 4557.
Path 171 | total_timesteps 4583.
Path 172 | total_timesteps 4622.
Path 173 | total_timesteps 4651.
Path 174 | total_timesteps 4673.
Path 175 | total_timesteps 4697.
Path 176 | total_timesteps 4739.
Path 177 | total_timesteps 4763.
Path 178 | total_timesteps 4778.
Path 179 | total_timesteps 4799.
Path 180 | total_timesteps 4850.
Path 181 | total_timesteps 4881.
Path 182 | total_timesteps 4916.
Path 183 | total_timesteps 4969.
Path 184 | total_timesteps 5024.
Path 185 | total_timesteps 5067.
Path 186 | total_timesteps 5086.
Path 187 | total_timesteps 5105.
Path 188 | total_timesteps 5167.
Path 189 | total_timesteps 5197.
Path 190 | total_timesteps 5212.
Path 191 | total_timesteps 5225.
Path 192 | total_timesteps 5250.
Path 193 | total_timesteps 5269.
Path 194 | total_timesteps 5293.
Path 195 | total_timesteps 5311.
Path 196 | total_timesteps 5330.
Path 197 | total_timesteps 5352.
Path 198 | total_timesteps 5376.
Path 199 | total_timesteps 5399.
Path 200 | total_timesteps 5422.
Path 201 | total_timesteps 5444.
Path 202 | total_timesteps 5470.
Path 203 | total_timesteps 5518.
Path 204 | total_timesteps 5534.
Path 205 | total_timesteps 5556.
Path 206 | total_timesteps 5586.
Path 207 | total_timesteps 5622.
Path 208 | total_timesteps 5637.
Path 209 | total_timesteps 5674.
Path 210 | total_timesteps 5693.
Path 211 | total_timesteps 5715.
Path 212 | total_timesteps 5729.
Path 213 | total_timesteps 5755.
Path 214 | total_timesteps 5797.
Path 215 | total_timesteps 5808.
Path 216 | total_timesteps 5831.
Path 217 | total_timesteps 5850.
Path 218 | total_timesteps 5890.
Path 219 | total_timesteps 5938.
Path 220 | total_timesteps 5953.
Path 221 | total_timesteps 5976.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.95    |
| Iteration     | 2        |
| MaximumReturn | 27.5     |
| MinimumReturn | -26.4    |
| TotalSamples  | 16033    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.03334622457623482
Validation loss = 0.025563491508364677
Validation loss = 0.02400152012705803
Validation loss = 0.02303590066730976
Validation loss = 0.03014884516596794
Validation loss = 0.02230958268046379
Validation loss = 0.021964112296700478
Validation loss = 0.02398025617003441
Validation loss = 0.022465063259005547
Validation loss = 0.0222206749022007
Validation loss = 0.023336581885814667
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 39.
Path 4 | total_timesteps 69.
Path 5 | total_timesteps 93.
Path 6 | total_timesteps 163.
Path 7 | total_timesteps 181.
Path 8 | total_timesteps 199.
Path 9 | total_timesteps 249.
Path 10 | total_timesteps 280.
Path 11 | total_timesteps 312.
Path 12 | total_timesteps 337.
Path 13 | total_timesteps 361.
Path 14 | total_timesteps 374.
Path 15 | total_timesteps 403.
Path 16 | total_timesteps 427.
Path 17 | total_timesteps 460.
Path 18 | total_timesteps 479.
Path 19 | total_timesteps 495.
Path 20 | total_timesteps 523.
Path 21 | total_timesteps 553.
Path 22 | total_timesteps 569.
Path 23 | total_timesteps 592.
Path 24 | total_timesteps 641.
Path 25 | total_timesteps 668.
Path 26 | total_timesteps 684.
Path 27 | total_timesteps 695.
Path 28 | total_timesteps 727.
Path 29 | total_timesteps 744.
Path 30 | total_timesteps 766.
Path 31 | total_timesteps 792.
Path 32 | total_timesteps 809.
Path 33 | total_timesteps 834.
Path 34 | total_timesteps 847.
Path 35 | total_timesteps 878.
Path 36 | total_timesteps 897.
Path 37 | total_timesteps 931.
Path 38 | total_timesteps 961.
Path 39 | total_timesteps 987.
Path 40 | total_timesteps 1003.
Path 41 | total_timesteps 1021.
Path 42 | total_timesteps 1044.
Path 43 | total_timesteps 1068.
Path 44 | total_timesteps 1082.
Path 45 | total_timesteps 1096.
Path 46 | total_timesteps 1118.
Path 47 | total_timesteps 1131.
Path 48 | total_timesteps 1155.
Path 49 | total_timesteps 1176.
Path 50 | total_timesteps 1189.
Path 51 | total_timesteps 1211.
Path 52 | total_timesteps 1233.
Path 53 | total_timesteps 1271.
Path 54 | total_timesteps 1297.
Path 55 | total_timesteps 1322.
Path 56 | total_timesteps 1345.
Path 57 | total_timesteps 1379.
Path 58 | total_timesteps 1394.
Path 59 | total_timesteps 1420.
Path 60 | total_timesteps 1448.
Path 61 | total_timesteps 1486.
Path 62 | total_timesteps 1503.
Path 63 | total_timesteps 1542.
Path 64 | total_timesteps 1563.
Path 65 | total_timesteps 1603.
Path 66 | total_timesteps 1624.
Path 67 | total_timesteps 1640.
Path 68 | total_timesteps 1665.
Path 69 | total_timesteps 1679.
Path 70 | total_timesteps 1708.
Path 71 | total_timesteps 1727.
Path 72 | total_timesteps 1750.
Path 73 | total_timesteps 1776.
Path 74 | total_timesteps 1804.
Path 75 | total_timesteps 1817.
Path 76 | total_timesteps 1836.
Path 77 | total_timesteps 1863.
Path 78 | total_timesteps 1873.
Path 79 | total_timesteps 1894.
Path 80 | total_timesteps 1907.
Path 81 | total_timesteps 1920.
Path 82 | total_timesteps 1934.
Path 83 | total_timesteps 1957.
Path 84 | total_timesteps 1994.
Path 85 | total_timesteps 2015.
Path 86 | total_timesteps 2032.
Path 87 | total_timesteps 2062.
Path 88 | total_timesteps 2088.
Path 89 | total_timesteps 2102.
Path 90 | total_timesteps 2115.
Path 91 | total_timesteps 2133.
Path 92 | total_timesteps 2156.
Path 93 | total_timesteps 2182.
Path 94 | total_timesteps 2231.
Path 95 | total_timesteps 2262.
Path 96 | total_timesteps 2289.
Path 97 | total_timesteps 2309.
Path 98 | total_timesteps 2338.
Path 99 | total_timesteps 2360.
Path 100 | total_timesteps 2378.
Path 101 | total_timesteps 2413.
Path 102 | total_timesteps 2449.
Path 103 | total_timesteps 2493.
Path 104 | total_timesteps 2526.
Path 105 | total_timesteps 2552.
Path 106 | total_timesteps 2574.
Path 107 | total_timesteps 2589.
Path 108 | total_timesteps 2611.
Path 109 | total_timesteps 2640.
Path 110 | total_timesteps 2671.
Path 111 | total_timesteps 2689.
Path 112 | total_timesteps 2710.
Path 113 | total_timesteps 2727.
Path 114 | total_timesteps 2759.
Path 115 | total_timesteps 2771.
Path 116 | total_timesteps 2793.
Path 117 | total_timesteps 2816.
Path 118 | total_timesteps 2828.
Path 119 | total_timesteps 2846.
Path 120 | total_timesteps 2868.
Path 121 | total_timesteps 2900.
Path 122 | total_timesteps 2924.
Path 123 | total_timesteps 2958.
Path 124 | total_timesteps 2976.
Path 125 | total_timesteps 3002.
Path 126 | total_timesteps 3015.
Path 127 | total_timesteps 3049.
Path 128 | total_timesteps 3067.
Path 129 | total_timesteps 3099.
Path 130 | total_timesteps 3124.
Path 131 | total_timesteps 3159.
Path 132 | total_timesteps 3184.
Path 133 | total_timesteps 3203.
Path 134 | total_timesteps 3227.
Path 135 | total_timesteps 3248.
Path 136 | total_timesteps 3305.
Path 137 | total_timesteps 3370.
Path 138 | total_timesteps 3384.
Path 139 | total_timesteps 3401.
Path 140 | total_timesteps 3415.
Path 141 | total_timesteps 3465.
Path 142 | total_timesteps 3510.
Path 143 | total_timesteps 3531.
Path 144 | total_timesteps 3562.
Path 145 | total_timesteps 3580.
Path 146 | total_timesteps 3625.
Path 147 | total_timesteps 3669.
Path 148 | total_timesteps 3692.
Path 149 | total_timesteps 3718.
Path 150 | total_timesteps 3742.
Path 151 | total_timesteps 3769.
Path 152 | total_timesteps 3786.
Path 153 | total_timesteps 3802.
Path 154 | total_timesteps 3826.
Path 155 | total_timesteps 3866.
Path 156 | total_timesteps 3892.
Path 157 | total_timesteps 3914.
Path 158 | total_timesteps 3930.
Path 159 | total_timesteps 3949.
Path 160 | total_timesteps 3957.
Path 161 | total_timesteps 3978.
Path 162 | total_timesteps 4006.
Path 163 | total_timesteps 4025.
Path 164 | total_timesteps 4047.
Path 165 | total_timesteps 4074.
Path 166 | total_timesteps 4097.
Path 167 | total_timesteps 4134.
Path 168 | total_timesteps 4167.
Path 169 | total_timesteps 4195.
Path 170 | total_timesteps 4257.
Path 171 | total_timesteps 4287.
Path 172 | total_timesteps 4342.
Path 173 | total_timesteps 4359.
Path 174 | total_timesteps 4388.
Path 175 | total_timesteps 4420.
Path 176 | total_timesteps 4433.
Path 177 | total_timesteps 4466.
Path 178 | total_timesteps 4492.
Path 179 | total_timesteps 4513.
Path 180 | total_timesteps 4538.
Path 181 | total_timesteps 4554.
Path 182 | total_timesteps 4570.
Path 183 | total_timesteps 4582.
Path 184 | total_timesteps 4598.
Path 185 | total_timesteps 4617.
Path 186 | total_timesteps 4649.
Path 187 | total_timesteps 4669.
Path 188 | total_timesteps 4688.
Path 189 | total_timesteps 4710.
Path 190 | total_timesteps 4724.
Path 191 | total_timesteps 4739.
Path 192 | total_timesteps 4757.
Path 193 | total_timesteps 4777.
Path 194 | total_timesteps 4805.
Path 195 | total_timesteps 4858.
Path 196 | total_timesteps 4876.
Path 197 | total_timesteps 4904.
Path 198 | total_timesteps 4951.
Path 199 | total_timesteps 4970.
Path 200 | total_timesteps 5011.
Path 201 | total_timesteps 5038.
Path 202 | total_timesteps 5063.
Path 203 | total_timesteps 5087.
Path 204 | total_timesteps 5121.
Path 205 | total_timesteps 5136.
Path 206 | total_timesteps 5213.
Path 207 | total_timesteps 5253.
Path 208 | total_timesteps 5269.
Path 209 | total_timesteps 5289.
Path 210 | total_timesteps 5305.
Path 211 | total_timesteps 5335.
Path 212 | total_timesteps 5372.
Path 213 | total_timesteps 5418.
Path 214 | total_timesteps 5446.
Path 215 | total_timesteps 5468.
Path 216 | total_timesteps 5500.
Path 217 | total_timesteps 5519.
Path 218 | total_timesteps 5535.
Path 219 | total_timesteps 5547.
Path 220 | total_timesteps 5572.
Path 221 | total_timesteps 5598.
Path 222 | total_timesteps 5617.
Path 223 | total_timesteps 5645.
Path 224 | total_timesteps 5662.
Path 225 | total_timesteps 5680.
Path 226 | total_timesteps 5717.
Path 227 | total_timesteps 5731.
Path 228 | total_timesteps 5753.
Path 229 | total_timesteps 5779.
Path 230 | total_timesteps 5803.
Path 231 | total_timesteps 5830.
Path 232 | total_timesteps 5848.
Path 233 | total_timesteps 5876.
Path 234 | total_timesteps 5901.
Path 235 | total_timesteps 5916.
Path 236 | total_timesteps 5946.
Path 237 | total_timesteps 5979.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.38    |
| Iteration     | 3        |
| MaximumReturn | 31.8     |
| MinimumReturn | -22.3    |
| TotalSamples  | 20039    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.025205060839653015
Validation loss = 0.02173839695751667
Validation loss = 0.020206410437822342
Validation loss = 0.019307952374219894
Validation loss = 0.019130345433950424
Validation loss = 0.018815573304891586
Validation loss = 0.01791664958000183
Validation loss = 0.018932297825813293
Validation loss = 0.018385406583547592
Validation loss = 0.019155625253915787
Validation loss = 0.019321955740451813
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 24.
Path 2 | total_timesteps 50.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 89.
Path 5 | total_timesteps 106.
Path 6 | total_timesteps 126.
Path 7 | total_timesteps 186.
Path 8 | total_timesteps 204.
Path 9 | total_timesteps 217.
Path 10 | total_timesteps 236.
Path 11 | total_timesteps 268.
Path 12 | total_timesteps 297.
Path 13 | total_timesteps 332.
Path 14 | total_timesteps 356.
Path 15 | total_timesteps 378.
Path 16 | total_timesteps 395.
Path 17 | total_timesteps 428.
Path 18 | total_timesteps 454.
Path 19 | total_timesteps 478.
Path 20 | total_timesteps 491.
Path 21 | total_timesteps 511.
Path 22 | total_timesteps 542.
Path 23 | total_timesteps 559.
Path 24 | total_timesteps 584.
Path 25 | total_timesteps 600.
Path 26 | total_timesteps 652.
Path 27 | total_timesteps 674.
Path 28 | total_timesteps 696.
Path 29 | total_timesteps 723.
Path 30 | total_timesteps 754.
Path 31 | total_timesteps 783.
Path 32 | total_timesteps 807.
Path 33 | total_timesteps 852.
Path 34 | total_timesteps 889.
Path 35 | total_timesteps 909.
Path 36 | total_timesteps 933.
Path 37 | total_timesteps 944.
Path 38 | total_timesteps 972.
Path 39 | total_timesteps 1013.
Path 40 | total_timesteps 1024.
Path 41 | total_timesteps 1040.
Path 42 | total_timesteps 1068.
Path 43 | total_timesteps 1085.
Path 44 | total_timesteps 1104.
Path 45 | total_timesteps 1132.
Path 46 | total_timesteps 1150.
Path 47 | total_timesteps 1211.
Path 48 | total_timesteps 1236.
Path 49 | total_timesteps 1253.
Path 50 | total_timesteps 1272.
Path 51 | total_timesteps 1302.
Path 52 | total_timesteps 1329.
Path 53 | total_timesteps 1351.
Path 54 | total_timesteps 1371.
Path 55 | total_timesteps 1388.
Path 56 | total_timesteps 1441.
Path 57 | total_timesteps 1457.
Path 58 | total_timesteps 1492.
Path 59 | total_timesteps 1514.
Path 60 | total_timesteps 1530.
Path 61 | total_timesteps 1549.
Path 62 | total_timesteps 1577.
Path 63 | total_timesteps 1587.
Path 64 | total_timesteps 1599.
Path 65 | total_timesteps 1607.
Path 66 | total_timesteps 1625.
Path 67 | total_timesteps 1642.
Path 68 | total_timesteps 1662.
Path 69 | total_timesteps 1685.
Path 70 | total_timesteps 1708.
Path 71 | total_timesteps 1728.
Path 72 | total_timesteps 1758.
Path 73 | total_timesteps 1780.
Path 74 | total_timesteps 1791.
Path 75 | total_timesteps 1810.
Path 76 | total_timesteps 1861.
Path 77 | total_timesteps 1893.
Path 78 | total_timesteps 1929.
Path 79 | total_timesteps 1965.
Path 80 | total_timesteps 1989.
Path 81 | total_timesteps 2039.
Path 82 | total_timesteps 2056.
Path 83 | total_timesteps 2087.
Path 84 | total_timesteps 2104.
Path 85 | total_timesteps 2122.
Path 86 | total_timesteps 2143.
Path 87 | total_timesteps 2167.
Path 88 | total_timesteps 2188.
Path 89 | total_timesteps 2207.
Path 90 | total_timesteps 2219.
Path 91 | total_timesteps 2247.
Path 92 | total_timesteps 2274.
Path 93 | total_timesteps 2309.
Path 94 | total_timesteps 2329.
Path 95 | total_timesteps 2356.
Path 96 | total_timesteps 2374.
Path 97 | total_timesteps 2393.
Path 98 | total_timesteps 2415.
Path 99 | total_timesteps 2427.
Path 100 | total_timesteps 2446.
Path 101 | total_timesteps 2469.
Path 102 | total_timesteps 2487.
Path 103 | total_timesteps 2502.
Path 104 | total_timesteps 2515.
Path 105 | total_timesteps 2525.
Path 106 | total_timesteps 2557.
Path 107 | total_timesteps 2569.
Path 108 | total_timesteps 2592.
Path 109 | total_timesteps 2610.
Path 110 | total_timesteps 2634.
Path 111 | total_timesteps 2647.
Path 112 | total_timesteps 2678.
Path 113 | total_timesteps 2694.
Path 114 | total_timesteps 2711.
Path 115 | total_timesteps 2734.
Path 116 | total_timesteps 2763.
Path 117 | total_timesteps 2816.
Path 118 | total_timesteps 2840.
Path 119 | total_timesteps 2874.
Path 120 | total_timesteps 2911.
Path 121 | total_timesteps 2926.
Path 122 | total_timesteps 2936.
Path 123 | total_timesteps 2982.
Path 124 | total_timesteps 3008.
Path 125 | total_timesteps 3042.
Path 126 | total_timesteps 3098.
Path 127 | total_timesteps 3126.
Path 128 | total_timesteps 3148.
Path 129 | total_timesteps 3211.
Path 130 | total_timesteps 3228.
Path 131 | total_timesteps 3243.
Path 132 | total_timesteps 3287.
Path 133 | total_timesteps 3312.
Path 134 | total_timesteps 3332.
Path 135 | total_timesteps 3353.
Path 136 | total_timesteps 3368.
Path 137 | total_timesteps 3399.
Path 138 | total_timesteps 3420.
Path 139 | total_timesteps 3450.
Path 140 | total_timesteps 3471.
Path 141 | total_timesteps 3483.
Path 142 | total_timesteps 3502.
Path 143 | total_timesteps 3522.
Path 144 | total_timesteps 3552.
Path 145 | total_timesteps 3566.
Path 146 | total_timesteps 3585.
Path 147 | total_timesteps 3601.
Path 148 | total_timesteps 3616.
Path 149 | total_timesteps 3634.
Path 150 | total_timesteps 3670.
Path 151 | total_timesteps 3716.
Path 152 | total_timesteps 3734.
Path 153 | total_timesteps 3757.
Path 154 | total_timesteps 3774.
Path 155 | total_timesteps 3801.
Path 156 | total_timesteps 3830.
Path 157 | total_timesteps 3851.
Path 158 | total_timesteps 3894.
Path 159 | total_timesteps 3912.
Path 160 | total_timesteps 3934.
Path 161 | total_timesteps 3946.
Path 162 | total_timesteps 3969.
Path 163 | total_timesteps 3997.
Path 164 | total_timesteps 4053.
Path 165 | total_timesteps 4075.
Path 166 | total_timesteps 4090.
Path 167 | total_timesteps 4107.
Path 168 | total_timesteps 4135.
Path 169 | total_timesteps 4183.
Path 170 | total_timesteps 4196.
Path 171 | total_timesteps 4207.
Path 172 | total_timesteps 4230.
Path 173 | total_timesteps 4243.
Path 174 | total_timesteps 4269.
Path 175 | total_timesteps 4291.
Path 176 | total_timesteps 4325.
Path 177 | total_timesteps 4346.
Path 178 | total_timesteps 4360.
Path 179 | total_timesteps 4382.
Path 180 | total_timesteps 4392.
Path 181 | total_timesteps 4409.
Path 182 | total_timesteps 4427.
Path 183 | total_timesteps 4461.
Path 184 | total_timesteps 4489.
Path 185 | total_timesteps 4510.
Path 186 | total_timesteps 4530.
Path 187 | total_timesteps 4557.
Path 188 | total_timesteps 4597.
Path 189 | total_timesteps 4613.
Path 190 | total_timesteps 4633.
Path 191 | total_timesteps 4655.
Path 192 | total_timesteps 4697.
Path 193 | total_timesteps 4710.
Path 194 | total_timesteps 4732.
Path 195 | total_timesteps 4761.
Path 196 | total_timesteps 4776.
Path 197 | total_timesteps 4792.
Path 198 | total_timesteps 4815.
Path 199 | total_timesteps 4844.
Path 200 | total_timesteps 4856.
Path 201 | total_timesteps 4890.
Path 202 | total_timesteps 4920.
Path 203 | total_timesteps 4944.
Path 204 | total_timesteps 4971.
Path 205 | total_timesteps 4991.
Path 206 | total_timesteps 5021.
Path 207 | total_timesteps 5054.
Path 208 | total_timesteps 5074.
Path 209 | total_timesteps 5115.
Path 210 | total_timesteps 5135.
Path 211 | total_timesteps 5165.
Path 212 | total_timesteps 5177.
Path 213 | total_timesteps 5197.
Path 214 | total_timesteps 5213.
Path 215 | total_timesteps 5230.
Path 216 | total_timesteps 5266.
Path 217 | total_timesteps 5284.
Path 218 | total_timesteps 5301.
Path 219 | total_timesteps 5316.
Path 220 | total_timesteps 5343.
Path 221 | total_timesteps 5367.
Path 222 | total_timesteps 5389.
Path 223 | total_timesteps 5418.
Path 224 | total_timesteps 5436.
Path 225 | total_timesteps 5448.
Path 226 | total_timesteps 5466.
Path 227 | total_timesteps 5486.
Path 228 | total_timesteps 5506.
Path 229 | total_timesteps 5532.
Path 230 | total_timesteps 5587.
Path 231 | total_timesteps 5610.
Path 232 | total_timesteps 5646.
Path 233 | total_timesteps 5677.
Path 234 | total_timesteps 5698.
Path 235 | total_timesteps 5731.
Path 236 | total_timesteps 5756.
Path 237 | total_timesteps 5774.
Path 238 | total_timesteps 5798.
Path 239 | total_timesteps 5819.
Path 240 | total_timesteps 5839.
Path 241 | total_timesteps 5860.
Path 242 | total_timesteps 5880.
Path 243 | total_timesteps 5897.
Path 244 | total_timesteps 5915.
Path 245 | total_timesteps 5941.
Path 246 | total_timesteps 5951.
Path 247 | total_timesteps 5965.
Path 248 | total_timesteps 5987.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.76    |
| Iteration     | 4        |
| MaximumReturn | 15.9     |
| MinimumReturn | -26.5    |
| TotalSamples  | 24040    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.02111865021288395
Validation loss = 0.017549898475408554
Validation loss = 0.01827917993068695
Validation loss = 0.016452165320515633
Validation loss = 0.016628002747893333
Validation loss = 0.015650812536478043
Validation loss = 0.01608211360871792
Validation loss = 0.017310447990894318
Validation loss = 0.015028475783765316
Validation loss = 0.01636113040149212
Validation loss = 0.017047179862856865
Validation loss = 0.015275758691132069
Validation loss = 0.01511041447520256
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 24.
Path 2 | total_timesteps 63.
Path 3 | total_timesteps 80.
Path 4 | total_timesteps 92.
Path 5 | total_timesteps 110.
Path 6 | total_timesteps 144.
Path 7 | total_timesteps 166.
Path 8 | total_timesteps 216.
Path 9 | total_timesteps 250.
Path 10 | total_timesteps 265.
Path 11 | total_timesteps 276.
Path 12 | total_timesteps 284.
Path 13 | total_timesteps 309.
Path 14 | total_timesteps 332.
Path 15 | total_timesteps 353.
Path 16 | total_timesteps 372.
Path 17 | total_timesteps 386.
Path 18 | total_timesteps 418.
Path 19 | total_timesteps 429.
Path 20 | total_timesteps 462.
Path 21 | total_timesteps 486.
Path 22 | total_timesteps 507.
Path 23 | total_timesteps 520.
Path 24 | total_timesteps 542.
Path 25 | total_timesteps 565.
Path 26 | total_timesteps 594.
Path 27 | total_timesteps 610.
Path 28 | total_timesteps 636.
Path 29 | total_timesteps 649.
Path 30 | total_timesteps 674.
Path 31 | total_timesteps 684.
Path 32 | total_timesteps 720.
Path 33 | total_timesteps 747.
Path 34 | total_timesteps 774.
Path 35 | total_timesteps 812.
Path 36 | total_timesteps 824.
Path 37 | total_timesteps 848.
Path 38 | total_timesteps 865.
Path 39 | total_timesteps 881.
Path 40 | total_timesteps 897.
Path 41 | total_timesteps 906.
Path 42 | total_timesteps 926.
Path 43 | total_timesteps 962.
Path 44 | total_timesteps 986.
Path 45 | total_timesteps 1020.
Path 46 | total_timesteps 1053.
Path 47 | total_timesteps 1080.
Path 48 | total_timesteps 1102.
Path 49 | total_timesteps 1121.
Path 50 | total_timesteps 1141.
Path 51 | total_timesteps 1155.
Path 52 | total_timesteps 1183.
Path 53 | total_timesteps 1193.
Path 54 | total_timesteps 1214.
Path 55 | total_timesteps 1238.
Path 56 | total_timesteps 1250.
Path 57 | total_timesteps 1262.
Path 58 | total_timesteps 1307.
Path 59 | total_timesteps 1327.
Path 60 | total_timesteps 1362.
Path 61 | total_timesteps 1385.
Path 62 | total_timesteps 1415.
Path 63 | total_timesteps 1439.
Path 64 | total_timesteps 1494.
Path 65 | total_timesteps 1516.
Path 66 | total_timesteps 1532.
Path 67 | total_timesteps 1554.
Path 68 | total_timesteps 1573.
Path 69 | total_timesteps 1613.
Path 70 | total_timesteps 1650.
Path 71 | total_timesteps 1672.
Path 72 | total_timesteps 1715.
Path 73 | total_timesteps 1732.
Path 74 | total_timesteps 1747.
Path 75 | total_timesteps 1774.
Path 76 | total_timesteps 1789.
Path 77 | total_timesteps 1818.
Path 78 | total_timesteps 1844.
Path 79 | total_timesteps 1855.
Path 80 | total_timesteps 1888.
Path 81 | total_timesteps 1913.
Path 82 | total_timesteps 1935.
Path 83 | total_timesteps 1951.
Path 84 | total_timesteps 1966.
Path 85 | total_timesteps 1982.
Path 86 | total_timesteps 2009.
Path 87 | total_timesteps 2033.
Path 88 | total_timesteps 2041.
Path 89 | total_timesteps 2064.
Path 90 | total_timesteps 2076.
Path 91 | total_timesteps 2095.
Path 92 | total_timesteps 2120.
Path 93 | total_timesteps 2172.
Path 94 | total_timesteps 2193.
Path 95 | total_timesteps 2205.
Path 96 | total_timesteps 2220.
Path 97 | total_timesteps 2241.
Path 98 | total_timesteps 2268.
Path 99 | total_timesteps 2282.
Path 100 | total_timesteps 2306.
Path 101 | total_timesteps 2330.
Path 102 | total_timesteps 2349.
Path 103 | total_timesteps 2364.
Path 104 | total_timesteps 2381.
Path 105 | total_timesteps 2391.
Path 106 | total_timesteps 2405.
Path 107 | total_timesteps 2421.
Path 108 | total_timesteps 2441.
Path 109 | total_timesteps 2471.
Path 110 | total_timesteps 2485.
Path 111 | total_timesteps 2519.
Path 112 | total_timesteps 2531.
Path 113 | total_timesteps 2562.
Path 114 | total_timesteps 2576.
Path 115 | total_timesteps 2590.
Path 116 | total_timesteps 2601.
Path 117 | total_timesteps 2639.
Path 118 | total_timesteps 2654.
Path 119 | total_timesteps 2676.
Path 120 | total_timesteps 2699.
Path 121 | total_timesteps 2712.
Path 122 | total_timesteps 2735.
Path 123 | total_timesteps 2755.
Path 124 | total_timesteps 2767.
Path 125 | total_timesteps 2793.
Path 126 | total_timesteps 2824.
Path 127 | total_timesteps 2836.
Path 128 | total_timesteps 2854.
Path 129 | total_timesteps 2878.
Path 130 | total_timesteps 2902.
Path 131 | total_timesteps 2912.
Path 132 | total_timesteps 2941.
Path 133 | total_timesteps 2958.
Path 134 | total_timesteps 2969.
Path 135 | total_timesteps 2992.
Path 136 | total_timesteps 3007.
Path 137 | total_timesteps 3032.
Path 138 | total_timesteps 3091.
Path 139 | total_timesteps 3115.
Path 140 | total_timesteps 3129.
Path 141 | total_timesteps 3153.
Path 142 | total_timesteps 3182.
Path 143 | total_timesteps 3218.
Path 144 | total_timesteps 3236.
Path 145 | total_timesteps 3264.
Path 146 | total_timesteps 3281.
Path 147 | total_timesteps 3318.
Path 148 | total_timesteps 3350.
Path 149 | total_timesteps 3375.
Path 150 | total_timesteps 3395.
Path 151 | total_timesteps 3413.
Path 152 | total_timesteps 3433.
Path 153 | total_timesteps 3446.
Path 154 | total_timesteps 3463.
Path 155 | total_timesteps 3489.
Path 156 | total_timesteps 3508.
Path 157 | total_timesteps 3523.
Path 158 | total_timesteps 3534.
Path 159 | total_timesteps 3556.
Path 160 | total_timesteps 3607.
Path 161 | total_timesteps 3624.
Path 162 | total_timesteps 3650.
Path 163 | total_timesteps 3671.
Path 164 | total_timesteps 3699.
Path 165 | total_timesteps 3722.
Path 166 | total_timesteps 3737.
Path 167 | total_timesteps 3753.
Path 168 | total_timesteps 3767.
Path 169 | total_timesteps 3789.
Path 170 | total_timesteps 3816.
Path 171 | total_timesteps 3831.
Path 172 | total_timesteps 3861.
Path 173 | total_timesteps 3899.
Path 174 | total_timesteps 3918.
Path 175 | total_timesteps 3933.
Path 176 | total_timesteps 3951.
Path 177 | total_timesteps 3971.
Path 178 | total_timesteps 3992.
Path 179 | total_timesteps 4005.
Path 180 | total_timesteps 4015.
Path 181 | total_timesteps 4031.
Path 182 | total_timesteps 4042.
Path 183 | total_timesteps 4056.
Path 184 | total_timesteps 4083.
Path 185 | total_timesteps 4109.
Path 186 | total_timesteps 4134.
Path 187 | total_timesteps 4153.
Path 188 | total_timesteps 4170.
Path 189 | total_timesteps 4185.
Path 190 | total_timesteps 4223.
Path 191 | total_timesteps 4252.
Path 192 | total_timesteps 4298.
Path 193 | total_timesteps 4309.
Path 194 | total_timesteps 4321.
Path 195 | total_timesteps 4332.
Path 196 | total_timesteps 4359.
Path 197 | total_timesteps 4406.
Path 198 | total_timesteps 4432.
Path 199 | total_timesteps 4454.
Path 200 | total_timesteps 4483.
Path 201 | total_timesteps 4537.
Path 202 | total_timesteps 4564.
Path 203 | total_timesteps 4591.
Path 204 | total_timesteps 4616.
Path 205 | total_timesteps 4631.
Path 206 | total_timesteps 4648.
Path 207 | total_timesteps 4664.
Path 208 | total_timesteps 4680.
Path 209 | total_timesteps 4700.
Path 210 | total_timesteps 4720.
Path 211 | total_timesteps 4731.
Path 212 | total_timesteps 4747.
Path 213 | total_timesteps 4778.
Path 214 | total_timesteps 4788.
Path 215 | total_timesteps 4814.
Path 216 | total_timesteps 4829.
Path 217 | total_timesteps 4856.
Path 218 | total_timesteps 4879.
Path 219 | total_timesteps 4900.
Path 220 | total_timesteps 4916.
Path 221 | total_timesteps 4932.
Path 222 | total_timesteps 4965.
Path 223 | total_timesteps 4997.
Path 224 | total_timesteps 5024.
Path 225 | total_timesteps 5035.
Path 226 | total_timesteps 5056.
Path 227 | total_timesteps 5077.
Path 228 | total_timesteps 5089.
Path 229 | total_timesteps 5099.
Path 230 | total_timesteps 5126.
Path 231 | total_timesteps 5146.
Path 232 | total_timesteps 5194.
Path 233 | total_timesteps 5215.
Path 234 | total_timesteps 5227.
Path 235 | total_timesteps 5258.
Path 236 | total_timesteps 5275.
Path 237 | total_timesteps 5294.
Path 238 | total_timesteps 5316.
Path 239 | total_timesteps 5335.
Path 240 | total_timesteps 5374.
Path 241 | total_timesteps 5389.
Path 242 | total_timesteps 5412.
Path 243 | total_timesteps 5425.
Path 244 | total_timesteps 5481.
Path 245 | total_timesteps 5519.
Path 246 | total_timesteps 5545.
Path 247 | total_timesteps 5567.
Path 248 | total_timesteps 5578.
Path 249 | total_timesteps 5596.
Path 250 | total_timesteps 5622.
Path 251 | total_timesteps 5639.
Path 252 | total_timesteps 5659.
Path 253 | total_timesteps 5695.
Path 254 | total_timesteps 5743.
Path 255 | total_timesteps 5757.
Path 256 | total_timesteps 5767.
Path 257 | total_timesteps 5790.
Path 258 | total_timesteps 5805.
Path 259 | total_timesteps 5823.
Path 260 | total_timesteps 5840.
Path 261 | total_timesteps 5866.
Path 262 | total_timesteps 5886.
Path 263 | total_timesteps 5916.
Path 264 | total_timesteps 5929.
Path 265 | total_timesteps 5961.
Path 266 | total_timesteps 5976.
Path 267 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.45    |
| Iteration     | 5        |
| MaximumReturn | 15.4     |
| MinimumReturn | -22.6    |
| TotalSamples  | 28055    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.018401023000478745
Validation loss = 0.013981162570416927
Validation loss = 0.014415149576961994
Validation loss = 0.013608694076538086
Validation loss = 0.014513621106743813
Validation loss = 0.014272132888436317
Validation loss = 0.014302619732916355
Validation loss = 0.013789207674562931
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 60.
Path 5 | total_timesteps 81.
Path 6 | total_timesteps 97.
Path 7 | total_timesteps 138.
Path 8 | total_timesteps 146.
Path 9 | total_timesteps 181.
Path 10 | total_timesteps 235.
Path 11 | total_timesteps 257.
Path 12 | total_timesteps 279.
Path 13 | total_timesteps 289.
Path 14 | total_timesteps 311.
Path 15 | total_timesteps 342.
Path 16 | total_timesteps 373.
Path 17 | total_timesteps 393.
Path 18 | total_timesteps 409.
Path 19 | total_timesteps 424.
Path 20 | total_timesteps 455.
Path 21 | total_timesteps 469.
Path 22 | total_timesteps 506.
Path 23 | total_timesteps 535.
Path 24 | total_timesteps 554.
Path 25 | total_timesteps 567.
Path 26 | total_timesteps 583.
Path 27 | total_timesteps 641.
Path 28 | total_timesteps 660.
Path 29 | total_timesteps 681.
Path 30 | total_timesteps 697.
Path 31 | total_timesteps 721.
Path 32 | total_timesteps 739.
Path 33 | total_timesteps 786.
Path 34 | total_timesteps 801.
Path 35 | total_timesteps 839.
Path 36 | total_timesteps 853.
Path 37 | total_timesteps 874.
Path 38 | total_timesteps 886.
Path 39 | total_timesteps 915.
Path 40 | total_timesteps 948.
Path 41 | total_timesteps 961.
Path 42 | total_timesteps 988.
Path 43 | total_timesteps 1004.
Path 44 | total_timesteps 1030.
Path 45 | total_timesteps 1068.
Path 46 | total_timesteps 1097.
Path 47 | total_timesteps 1118.
Path 48 | total_timesteps 1133.
Path 49 | total_timesteps 1159.
Path 50 | total_timesteps 1169.
Path 51 | total_timesteps 1191.
Path 52 | total_timesteps 1237.
Path 53 | total_timesteps 1249.
Path 54 | total_timesteps 1275.
Path 55 | total_timesteps 1305.
Path 56 | total_timesteps 1332.
Path 57 | total_timesteps 1345.
Path 58 | total_timesteps 1386.
Path 59 | total_timesteps 1405.
Path 60 | total_timesteps 1428.
Path 61 | total_timesteps 1463.
Path 62 | total_timesteps 1488.
Path 63 | total_timesteps 1505.
Path 64 | total_timesteps 1523.
Path 65 | total_timesteps 1552.
Path 66 | total_timesteps 1563.
Path 67 | total_timesteps 1584.
Path 68 | total_timesteps 1609.
Path 69 | total_timesteps 1623.
Path 70 | total_timesteps 1642.
Path 71 | total_timesteps 1689.
Path 72 | total_timesteps 1711.
Path 73 | total_timesteps 1734.
Path 74 | total_timesteps 1754.
Path 75 | total_timesteps 1775.
Path 76 | total_timesteps 1801.
Path 77 | total_timesteps 1812.
Path 78 | total_timesteps 1823.
Path 79 | total_timesteps 1839.
Path 80 | total_timesteps 1864.
Path 81 | total_timesteps 1887.
Path 82 | total_timesteps 1903.
Path 83 | total_timesteps 1922.
Path 84 | total_timesteps 1953.
Path 85 | total_timesteps 1981.
Path 86 | total_timesteps 2003.
Path 87 | total_timesteps 2016.
Path 88 | total_timesteps 2033.
Path 89 | total_timesteps 2047.
Path 90 | total_timesteps 2078.
Path 91 | total_timesteps 2093.
Path 92 | total_timesteps 2126.
Path 93 | total_timesteps 2157.
Path 94 | total_timesteps 2177.
Path 95 | total_timesteps 2190.
Path 96 | total_timesteps 2211.
Path 97 | total_timesteps 2234.
Path 98 | total_timesteps 2252.
Path 99 | total_timesteps 2263.
Path 100 | total_timesteps 2280.
Path 101 | total_timesteps 2308.
Path 102 | total_timesteps 2320.
Path 103 | total_timesteps 2332.
Path 104 | total_timesteps 2368.
Path 105 | total_timesteps 2419.
Path 106 | total_timesteps 2443.
Path 107 | total_timesteps 2470.
Path 108 | total_timesteps 2493.
Path 109 | total_timesteps 2508.
Path 110 | total_timesteps 2533.
Path 111 | total_timesteps 2551.
Path 112 | total_timesteps 2563.
Path 113 | total_timesteps 2586.
Path 114 | total_timesteps 2602.
Path 115 | total_timesteps 2621.
Path 116 | total_timesteps 2646.
Path 117 | total_timesteps 2667.
Path 118 | total_timesteps 2679.
Path 119 | total_timesteps 2694.
Path 120 | total_timesteps 2720.
Path 121 | total_timesteps 2730.
Path 122 | total_timesteps 2745.
Path 123 | total_timesteps 2755.
Path 124 | total_timesteps 2775.
Path 125 | total_timesteps 2808.
Path 126 | total_timesteps 2823.
Path 127 | total_timesteps 2844.
Path 128 | total_timesteps 2893.
Path 129 | total_timesteps 2938.
Path 130 | total_timesteps 2959.
Path 131 | total_timesteps 2986.
Path 132 | total_timesteps 3022.
Path 133 | total_timesteps 3044.
Path 134 | total_timesteps 3067.
Path 135 | total_timesteps 3083.
Path 136 | total_timesteps 3106.
Path 137 | total_timesteps 3122.
Path 138 | total_timesteps 3137.
Path 139 | total_timesteps 3156.
Path 140 | total_timesteps 3166.
Path 141 | total_timesteps 3203.
Path 142 | total_timesteps 3214.
Path 143 | total_timesteps 3226.
Path 144 | total_timesteps 3247.
Path 145 | total_timesteps 3271.
Path 146 | total_timesteps 3288.
Path 147 | total_timesteps 3318.
Path 148 | total_timesteps 3359.
Path 149 | total_timesteps 3378.
Path 150 | total_timesteps 3400.
Path 151 | total_timesteps 3425.
Path 152 | total_timesteps 3447.
Path 153 | total_timesteps 3469.
Path 154 | total_timesteps 3488.
Path 155 | total_timesteps 3511.
Path 156 | total_timesteps 3531.
Path 157 | total_timesteps 3555.
Path 158 | total_timesteps 3578.
Path 159 | total_timesteps 3595.
Path 160 | total_timesteps 3611.
Path 161 | total_timesteps 3633.
Path 162 | total_timesteps 3650.
Path 163 | total_timesteps 3666.
Path 164 | total_timesteps 3679.
Path 165 | total_timesteps 3690.
Path 166 | total_timesteps 3712.
Path 167 | total_timesteps 3741.
Path 168 | total_timesteps 3762.
Path 169 | total_timesteps 3789.
Path 170 | total_timesteps 3822.
Path 171 | total_timesteps 3841.
Path 172 | total_timesteps 3852.
Path 173 | total_timesteps 3880.
Path 174 | total_timesteps 3899.
Path 175 | total_timesteps 3921.
Path 176 | total_timesteps 3946.
Path 177 | total_timesteps 3956.
Path 178 | total_timesteps 3979.
Path 179 | total_timesteps 3993.
Path 180 | total_timesteps 4014.
Path 181 | total_timesteps 4049.
Path 182 | total_timesteps 4071.
Path 183 | total_timesteps 4086.
Path 184 | total_timesteps 4096.
Path 185 | total_timesteps 4121.
Path 186 | total_timesteps 4131.
Path 187 | total_timesteps 4143.
Path 188 | total_timesteps 4161.
Path 189 | total_timesteps 4181.
Path 190 | total_timesteps 4213.
Path 191 | total_timesteps 4237.
Path 192 | total_timesteps 4254.
Path 193 | total_timesteps 4271.
Path 194 | total_timesteps 4305.
Path 195 | total_timesteps 4328.
Path 196 | total_timesteps 4363.
Path 197 | total_timesteps 4380.
Path 198 | total_timesteps 4402.
Path 199 | total_timesteps 4423.
Path 200 | total_timesteps 4448.
Path 201 | total_timesteps 4480.
Path 202 | total_timesteps 4519.
Path 203 | total_timesteps 4562.
Path 204 | total_timesteps 4577.
Path 205 | total_timesteps 4601.
Path 206 | total_timesteps 4611.
Path 207 | total_timesteps 4627.
Path 208 | total_timesteps 4642.
Path 209 | total_timesteps 4662.
Path 210 | total_timesteps 4688.
Path 211 | total_timesteps 4739.
Path 212 | total_timesteps 4758.
Path 213 | total_timesteps 4775.
Path 214 | total_timesteps 4796.
Path 215 | total_timesteps 4815.
Path 216 | total_timesteps 4843.
Path 217 | total_timesteps 4857.
Path 218 | total_timesteps 4871.
Path 219 | total_timesteps 4887.
Path 220 | total_timesteps 4916.
Path 221 | total_timesteps 4932.
Path 222 | total_timesteps 4952.
Path 223 | total_timesteps 4967.
Path 224 | total_timesteps 4986.
Path 225 | total_timesteps 5004.
Path 226 | total_timesteps 5026.
Path 227 | total_timesteps 5038.
Path 228 | total_timesteps 5056.
Path 229 | total_timesteps 5071.
Path 230 | total_timesteps 5089.
Path 231 | total_timesteps 5103.
Path 232 | total_timesteps 5134.
Path 233 | total_timesteps 5175.
Path 234 | total_timesteps 5202.
Path 235 | total_timesteps 5218.
Path 236 | total_timesteps 5232.
Path 237 | total_timesteps 5254.
Path 238 | total_timesteps 5267.
Path 239 | total_timesteps 5301.
Path 240 | total_timesteps 5327.
Path 241 | total_timesteps 5337.
Path 242 | total_timesteps 5358.
Path 243 | total_timesteps 5382.
Path 244 | total_timesteps 5402.
Path 245 | total_timesteps 5419.
Path 246 | total_timesteps 5431.
Path 247 | total_timesteps 5447.
Path 248 | total_timesteps 5465.
Path 249 | total_timesteps 5490.
Path 250 | total_timesteps 5508.
Path 251 | total_timesteps 5528.
Path 252 | total_timesteps 5576.
Path 253 | total_timesteps 5601.
Path 254 | total_timesteps 5614.
Path 255 | total_timesteps 5641.
Path 256 | total_timesteps 5656.
Path 257 | total_timesteps 5677.
Path 258 | total_timesteps 5691.
Path 259 | total_timesteps 5716.
Path 260 | total_timesteps 5746.
Path 261 | total_timesteps 5767.
Path 262 | total_timesteps 5788.
Path 263 | total_timesteps 5833.
Path 264 | total_timesteps 5850.
Path 265 | total_timesteps 5867.
Path 266 | total_timesteps 5881.
Path 267 | total_timesteps 5898.
Path 268 | total_timesteps 5930.
Path 269 | total_timesteps 5952.
Path 270 | total_timesteps 5974.
Path 271 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.08    |
| Iteration     | 6        |
| MaximumReturn | 21.9     |
| MinimumReturn | -25.5    |
| TotalSamples  | 32055    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.01439425628632307
Validation loss = 0.013181052170693874
Validation loss = 0.014167513698339462
Validation loss = 0.012887530028820038
Validation loss = 0.012623021379113197
Validation loss = 0.011842937208712101
Validation loss = 0.013274245895445347
Validation loss = 0.011646928265690804
Validation loss = 0.012295716442167759
Validation loss = 0.011717068031430244
Validation loss = 0.011829204857349396
Validation loss = 0.012970959767699242
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 41.
Path 3 | total_timesteps 53.
Path 4 | total_timesteps 81.
Path 5 | total_timesteps 106.
Path 6 | total_timesteps 132.
Path 7 | total_timesteps 151.
Path 8 | total_timesteps 166.
Path 9 | total_timesteps 197.
Path 10 | total_timesteps 248.
Path 11 | total_timesteps 260.
Path 12 | total_timesteps 291.
Path 13 | total_timesteps 309.
Path 14 | total_timesteps 330.
Path 15 | total_timesteps 342.
Path 16 | total_timesteps 357.
Path 17 | total_timesteps 383.
Path 18 | total_timesteps 410.
Path 19 | total_timesteps 431.
Path 20 | total_timesteps 448.
Path 21 | total_timesteps 462.
Path 22 | total_timesteps 484.
Path 23 | total_timesteps 526.
Path 24 | total_timesteps 566.
Path 25 | total_timesteps 580.
Path 26 | total_timesteps 595.
Path 27 | total_timesteps 613.
Path 28 | total_timesteps 649.
Path 29 | total_timesteps 668.
Path 30 | total_timesteps 681.
Path 31 | total_timesteps 712.
Path 32 | total_timesteps 727.
Path 33 | total_timesteps 746.
Path 34 | total_timesteps 781.
Path 35 | total_timesteps 798.
Path 36 | total_timesteps 812.
Path 37 | total_timesteps 829.
Path 38 | total_timesteps 878.
Path 39 | total_timesteps 897.
Path 40 | total_timesteps 921.
Path 41 | total_timesteps 945.
Path 42 | total_timesteps 973.
Path 43 | total_timesteps 1000.
Path 44 | total_timesteps 1029.
Path 45 | total_timesteps 1057.
Path 46 | total_timesteps 1081.
Path 47 | total_timesteps 1095.
Path 48 | total_timesteps 1115.
Path 49 | total_timesteps 1128.
Path 50 | total_timesteps 1143.
Path 51 | total_timesteps 1186.
Path 52 | total_timesteps 1218.
Path 53 | total_timesteps 1258.
Path 54 | total_timesteps 1273.
Path 55 | total_timesteps 1293.
Path 56 | total_timesteps 1312.
Path 57 | total_timesteps 1342.
Path 58 | total_timesteps 1354.
Path 59 | total_timesteps 1373.
Path 60 | total_timesteps 1388.
Path 61 | total_timesteps 1421.
Path 62 | total_timesteps 1446.
Path 63 | total_timesteps 1462.
Path 64 | total_timesteps 1491.
Path 65 | total_timesteps 1509.
Path 66 | total_timesteps 1528.
Path 67 | total_timesteps 1563.
Path 68 | total_timesteps 1584.
Path 69 | total_timesteps 1598.
Path 70 | total_timesteps 1625.
Path 71 | total_timesteps 1657.
Path 72 | total_timesteps 1678.
Path 73 | total_timesteps 1697.
Path 74 | total_timesteps 1741.
Path 75 | total_timesteps 1766.
Path 76 | total_timesteps 1790.
Path 77 | total_timesteps 1827.
Path 78 | total_timesteps 1854.
Path 79 | total_timesteps 1876.
Path 80 | total_timesteps 1902.
Path 81 | total_timesteps 1916.
Path 82 | total_timesteps 1932.
Path 83 | total_timesteps 1953.
Path 84 | total_timesteps 1967.
Path 85 | total_timesteps 1995.
Path 86 | total_timesteps 2009.
Path 87 | total_timesteps 2023.
Path 88 | total_timesteps 2040.
Path 89 | total_timesteps 2075.
Path 90 | total_timesteps 2093.
Path 91 | total_timesteps 2120.
Path 92 | total_timesteps 2130.
Path 93 | total_timesteps 2155.
Path 94 | total_timesteps 2171.
Path 95 | total_timesteps 2199.
Path 96 | total_timesteps 2230.
Path 97 | total_timesteps 2240.
Path 98 | total_timesteps 2271.
Path 99 | total_timesteps 2290.
Path 100 | total_timesteps 2348.
Path 101 | total_timesteps 2366.
Path 102 | total_timesteps 2388.
Path 103 | total_timesteps 2405.
Path 104 | total_timesteps 2417.
Path 105 | total_timesteps 2428.
Path 106 | total_timesteps 2453.
Path 107 | total_timesteps 2476.
Path 108 | total_timesteps 2499.
Path 109 | total_timesteps 2529.
Path 110 | total_timesteps 2573.
Path 111 | total_timesteps 2600.
Path 112 | total_timesteps 2625.
Path 113 | total_timesteps 2642.
Path 114 | total_timesteps 2663.
Path 115 | total_timesteps 2677.
Path 116 | total_timesteps 2697.
Path 117 | total_timesteps 2718.
Path 118 | total_timesteps 2736.
Path 119 | total_timesteps 2756.
Path 120 | total_timesteps 2776.
Path 121 | total_timesteps 2803.
Path 122 | total_timesteps 2813.
Path 123 | total_timesteps 2836.
Path 124 | total_timesteps 2856.
Path 125 | total_timesteps 2879.
Path 126 | total_timesteps 2896.
Path 127 | total_timesteps 2906.
Path 128 | total_timesteps 2928.
Path 129 | total_timesteps 2955.
Path 130 | total_timesteps 2976.
Path 131 | total_timesteps 2992.
Path 132 | total_timesteps 3011.
Path 133 | total_timesteps 3029.
Path 134 | total_timesteps 3046.
Path 135 | total_timesteps 3065.
Path 136 | total_timesteps 3078.
Path 137 | total_timesteps 3095.
Path 138 | total_timesteps 3112.
Path 139 | total_timesteps 3123.
Path 140 | total_timesteps 3138.
Path 141 | total_timesteps 3162.
Path 142 | total_timesteps 3186.
Path 143 | total_timesteps 3206.
Path 144 | total_timesteps 3220.
Path 145 | total_timesteps 3246.
Path 146 | total_timesteps 3268.
Path 147 | total_timesteps 3287.
Path 148 | total_timesteps 3299.
Path 149 | total_timesteps 3313.
Path 150 | total_timesteps 3336.
Path 151 | total_timesteps 3362.
Path 152 | total_timesteps 3382.
Path 153 | total_timesteps 3398.
Path 154 | total_timesteps 3416.
Path 155 | total_timesteps 3433.
Path 156 | total_timesteps 3452.
Path 157 | total_timesteps 3484.
Path 158 | total_timesteps 3498.
Path 159 | total_timesteps 3521.
Path 160 | total_timesteps 3528.
Path 161 | total_timesteps 3565.
Path 162 | total_timesteps 3579.
Path 163 | total_timesteps 3598.
Path 164 | total_timesteps 3623.
Path 165 | total_timesteps 3636.
Path 166 | total_timesteps 3645.
Path 167 | total_timesteps 3657.
Path 168 | total_timesteps 3683.
Path 169 | total_timesteps 3699.
Path 170 | total_timesteps 3720.
Path 171 | total_timesteps 3734.
Path 172 | total_timesteps 3754.
Path 173 | total_timesteps 3776.
Path 174 | total_timesteps 3800.
Path 175 | total_timesteps 3828.
Path 176 | total_timesteps 3849.
Path 177 | total_timesteps 3869.
Path 178 | total_timesteps 3897.
Path 179 | total_timesteps 3914.
Path 180 | total_timesteps 3939.
Path 181 | total_timesteps 3960.
Path 182 | total_timesteps 3981.
Path 183 | total_timesteps 3996.
Path 184 | total_timesteps 4082.
Path 185 | total_timesteps 4104.
Path 186 | total_timesteps 4134.
Path 187 | total_timesteps 4157.
Path 188 | total_timesteps 4182.
Path 189 | total_timesteps 4206.
Path 190 | total_timesteps 4223.
Path 191 | total_timesteps 4248.
Path 192 | total_timesteps 4271.
Path 193 | total_timesteps 4287.
Path 194 | total_timesteps 4322.
Path 195 | total_timesteps 4337.
Path 196 | total_timesteps 4363.
Path 197 | total_timesteps 4393.
Path 198 | total_timesteps 4407.
Path 199 | total_timesteps 4438.
Path 200 | total_timesteps 4460.
Path 201 | total_timesteps 4494.
Path 202 | total_timesteps 4515.
Path 203 | total_timesteps 4541.
Path 204 | total_timesteps 4565.
Path 205 | total_timesteps 4583.
Path 206 | total_timesteps 4594.
Path 207 | total_timesteps 4625.
Path 208 | total_timesteps 4648.
Path 209 | total_timesteps 4672.
Path 210 | total_timesteps 4687.
Path 211 | total_timesteps 4700.
Path 212 | total_timesteps 4726.
Path 213 | total_timesteps 4747.
Path 214 | total_timesteps 4767.
Path 215 | total_timesteps 4789.
Path 216 | total_timesteps 4812.
Path 217 | total_timesteps 4831.
Path 218 | total_timesteps 4880.
Path 219 | total_timesteps 4896.
Path 220 | total_timesteps 4926.
Path 221 | total_timesteps 4949.
Path 222 | total_timesteps 4959.
Path 223 | total_timesteps 4981.
Path 224 | total_timesteps 4998.
Path 225 | total_timesteps 5019.
Path 226 | total_timesteps 5039.
Path 227 | total_timesteps 5054.
Path 228 | total_timesteps 5076.
Path 229 | total_timesteps 5096.
Path 230 | total_timesteps 5125.
Path 231 | total_timesteps 5145.
Path 232 | total_timesteps 5174.
Path 233 | total_timesteps 5184.
Path 234 | total_timesteps 5210.
Path 235 | total_timesteps 5230.
Path 236 | total_timesteps 5245.
Path 237 | total_timesteps 5262.
Path 238 | total_timesteps 5279.
Path 239 | total_timesteps 5294.
Path 240 | total_timesteps 5305.
Path 241 | total_timesteps 5318.
Path 242 | total_timesteps 5328.
Path 243 | total_timesteps 5373.
Path 244 | total_timesteps 5397.
Path 245 | total_timesteps 5420.
Path 246 | total_timesteps 5447.
Path 247 | total_timesteps 5469.
Path 248 | total_timesteps 5493.
Path 249 | total_timesteps 5523.
Path 250 | total_timesteps 5541.
Path 251 | total_timesteps 5558.
Path 252 | total_timesteps 5577.
Path 253 | total_timesteps 5597.
Path 254 | total_timesteps 5618.
Path 255 | total_timesteps 5644.
Path 256 | total_timesteps 5669.
Path 257 | total_timesteps 5687.
Path 258 | total_timesteps 5702.
Path 259 | total_timesteps 5715.
Path 260 | total_timesteps 5741.
Path 261 | total_timesteps 5764.
Path 262 | total_timesteps 5782.
Path 263 | total_timesteps 5810.
Path 264 | total_timesteps 5821.
Path 265 | total_timesteps 5836.
Path 266 | total_timesteps 5860.
Path 267 | total_timesteps 5871.
Path 268 | total_timesteps 5916.
Path 269 | total_timesteps 5936.
Path 270 | total_timesteps 5954.
Path 271 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.92    |
| Iteration     | 7        |
| MaximumReturn | 18       |
| MinimumReturn | -22.1    |
| TotalSamples  | 36066    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.01441301591694355
Validation loss = 0.011982325464487076
Validation loss = 0.011099427938461304
Validation loss = 0.01292448304593563
Validation loss = 0.010238072834908962
Validation loss = 0.010571054182946682
Validation loss = 0.011122744530439377
Validation loss = 0.010774997062981129
Validation loss = 0.010611016303300858
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 33.
Path 2 | total_timesteps 46.
Path 3 | total_timesteps 69.
Path 4 | total_timesteps 91.
Path 5 | total_timesteps 116.
Path 6 | total_timesteps 132.
Path 7 | total_timesteps 159.
Path 8 | total_timesteps 184.
Path 9 | total_timesteps 208.
Path 10 | total_timesteps 232.
Path 11 | total_timesteps 248.
Path 12 | total_timesteps 281.
Path 13 | total_timesteps 297.
Path 14 | total_timesteps 315.
Path 15 | total_timesteps 333.
Path 16 | total_timesteps 353.
Path 17 | total_timesteps 362.
Path 18 | total_timesteps 385.
Path 19 | total_timesteps 399.
Path 20 | total_timesteps 412.
Path 21 | total_timesteps 423.
Path 22 | total_timesteps 436.
Path 23 | total_timesteps 458.
Path 24 | total_timesteps 470.
Path 25 | total_timesteps 481.
Path 26 | total_timesteps 509.
Path 27 | total_timesteps 540.
Path 28 | total_timesteps 547.
Path 29 | total_timesteps 559.
Path 30 | total_timesteps 585.
Path 31 | total_timesteps 606.
Path 32 | total_timesteps 622.
Path 33 | total_timesteps 638.
Path 34 | total_timesteps 679.
Path 35 | total_timesteps 691.
Path 36 | total_timesteps 723.
Path 37 | total_timesteps 759.
Path 38 | total_timesteps 790.
Path 39 | total_timesteps 815.
Path 40 | total_timesteps 844.
Path 41 | total_timesteps 858.
Path 42 | total_timesteps 876.
Path 43 | total_timesteps 899.
Path 44 | total_timesteps 931.
Path 45 | total_timesteps 949.
Path 46 | total_timesteps 962.
Path 47 | total_timesteps 977.
Path 48 | total_timesteps 990.
Path 49 | total_timesteps 1029.
Path 50 | total_timesteps 1043.
Path 51 | total_timesteps 1054.
Path 52 | total_timesteps 1068.
Path 53 | total_timesteps 1088.
Path 54 | total_timesteps 1118.
Path 55 | total_timesteps 1145.
Path 56 | total_timesteps 1181.
Path 57 | total_timesteps 1213.
Path 58 | total_timesteps 1245.
Path 59 | total_timesteps 1275.
Path 60 | total_timesteps 1288.
Path 61 | total_timesteps 1303.
Path 62 | total_timesteps 1323.
Path 63 | total_timesteps 1338.
Path 64 | total_timesteps 1364.
Path 65 | total_timesteps 1374.
Path 66 | total_timesteps 1410.
Path 67 | total_timesteps 1426.
Path 68 | total_timesteps 1455.
Path 69 | total_timesteps 1471.
Path 70 | total_timesteps 1501.
Path 71 | total_timesteps 1515.
Path 72 | total_timesteps 1527.
Path 73 | total_timesteps 1572.
Path 74 | total_timesteps 1585.
Path 75 | total_timesteps 1604.
Path 76 | total_timesteps 1634.
Path 77 | total_timesteps 1648.
Path 78 | total_timesteps 1666.
Path 79 | total_timesteps 1675.
Path 80 | total_timesteps 1696.
Path 81 | total_timesteps 1708.
Path 82 | total_timesteps 1738.
Path 83 | total_timesteps 1764.
Path 84 | total_timesteps 1804.
Path 85 | total_timesteps 1827.
Path 86 | total_timesteps 1838.
Path 87 | total_timesteps 1847.
Path 88 | total_timesteps 1872.
Path 89 | total_timesteps 1890.
Path 90 | total_timesteps 1917.
Path 91 | total_timesteps 1931.
Path 92 | total_timesteps 1951.
Path 93 | total_timesteps 1964.
Path 94 | total_timesteps 1983.
Path 95 | total_timesteps 2010.
Path 96 | total_timesteps 2029.
Path 97 | total_timesteps 2046.
Path 98 | total_timesteps 2057.
Path 99 | total_timesteps 2073.
Path 100 | total_timesteps 2085.
Path 101 | total_timesteps 2116.
Path 102 | total_timesteps 2140.
Path 103 | total_timesteps 2156.
Path 104 | total_timesteps 2179.
Path 105 | total_timesteps 2204.
Path 106 | total_timesteps 2234.
Path 107 | total_timesteps 2247.
Path 108 | total_timesteps 2282.
Path 109 | total_timesteps 2297.
Path 110 | total_timesteps 2320.
Path 111 | total_timesteps 2338.
Path 112 | total_timesteps 2349.
Path 113 | total_timesteps 2370.
Path 114 | total_timesteps 2396.
Path 115 | total_timesteps 2414.
Path 116 | total_timesteps 2432.
Path 117 | total_timesteps 2452.
Path 118 | total_timesteps 2488.
Path 119 | total_timesteps 2498.
Path 120 | total_timesteps 2520.
Path 121 | total_timesteps 2531.
Path 122 | total_timesteps 2553.
Path 123 | total_timesteps 2566.
Path 124 | total_timesteps 2594.
Path 125 | total_timesteps 2616.
Path 126 | total_timesteps 2631.
Path 127 | total_timesteps 2653.
Path 128 | total_timesteps 2684.
Path 129 | total_timesteps 2703.
Path 130 | total_timesteps 2720.
Path 131 | total_timesteps 2732.
Path 132 | total_timesteps 2751.
Path 133 | total_timesteps 2772.
Path 134 | total_timesteps 2799.
Path 135 | total_timesteps 2839.
Path 136 | total_timesteps 2858.
Path 137 | total_timesteps 2874.
Path 138 | total_timesteps 2897.
Path 139 | total_timesteps 2909.
Path 140 | total_timesteps 2928.
Path 141 | total_timesteps 2940.
Path 142 | total_timesteps 2956.
Path 143 | total_timesteps 2969.
Path 144 | total_timesteps 2997.
Path 145 | total_timesteps 3019.
Path 146 | total_timesteps 3036.
Path 147 | total_timesteps 3060.
Path 148 | total_timesteps 3074.
Path 149 | total_timesteps 3110.
Path 150 | total_timesteps 3124.
Path 151 | total_timesteps 3137.
Path 152 | total_timesteps 3154.
Path 153 | total_timesteps 3169.
Path 154 | total_timesteps 3185.
Path 155 | total_timesteps 3201.
Path 156 | total_timesteps 3232.
Path 157 | total_timesteps 3256.
Path 158 | total_timesteps 3287.
Path 159 | total_timesteps 3299.
Path 160 | total_timesteps 3336.
Path 161 | total_timesteps 3360.
Path 162 | total_timesteps 3371.
Path 163 | total_timesteps 3392.
Path 164 | total_timesteps 3447.
Path 165 | total_timesteps 3467.
Path 166 | total_timesteps 3487.
Path 167 | total_timesteps 3501.
Path 168 | total_timesteps 3515.
Path 169 | total_timesteps 3537.
Path 170 | total_timesteps 3557.
Path 171 | total_timesteps 3575.
Path 172 | total_timesteps 3594.
Path 173 | total_timesteps 3608.
Path 174 | total_timesteps 3620.
Path 175 | total_timesteps 3649.
Path 176 | total_timesteps 3674.
Path 177 | total_timesteps 3692.
Path 178 | total_timesteps 3709.
Path 179 | total_timesteps 3742.
Path 180 | total_timesteps 3778.
Path 181 | total_timesteps 3790.
Path 182 | total_timesteps 3815.
Path 183 | total_timesteps 3834.
Path 184 | total_timesteps 3866.
Path 185 | total_timesteps 3874.
Path 186 | total_timesteps 3894.
Path 187 | total_timesteps 3910.
Path 188 | total_timesteps 3922.
Path 189 | total_timesteps 3936.
Path 190 | total_timesteps 3950.
Path 191 | total_timesteps 3960.
Path 192 | total_timesteps 3972.
Path 193 | total_timesteps 3991.
Path 194 | total_timesteps 4045.
Path 195 | total_timesteps 4066.
Path 196 | total_timesteps 4085.
Path 197 | total_timesteps 4112.
Path 198 | total_timesteps 4149.
Path 199 | total_timesteps 4165.
Path 200 | total_timesteps 4187.
Path 201 | total_timesteps 4213.
Path 202 | total_timesteps 4226.
Path 203 | total_timesteps 4262.
Path 204 | total_timesteps 4291.
Path 205 | total_timesteps 4304.
Path 206 | total_timesteps 4317.
Path 207 | total_timesteps 4332.
Path 208 | total_timesteps 4364.
Path 209 | total_timesteps 4385.
Path 210 | total_timesteps 4404.
Path 211 | total_timesteps 4413.
Path 212 | total_timesteps 4424.
Path 213 | total_timesteps 4455.
Path 214 | total_timesteps 4478.
Path 215 | total_timesteps 4506.
Path 216 | total_timesteps 4532.
Path 217 | total_timesteps 4566.
Path 218 | total_timesteps 4583.
Path 219 | total_timesteps 4606.
Path 220 | total_timesteps 4637.
Path 221 | total_timesteps 4657.
Path 222 | total_timesteps 4672.
Path 223 | total_timesteps 4691.
Path 224 | total_timesteps 4713.
Path 225 | total_timesteps 4733.
Path 226 | total_timesteps 4756.
Path 227 | total_timesteps 4769.
Path 228 | total_timesteps 4793.
Path 229 | total_timesteps 4815.
Path 230 | total_timesteps 4835.
Path 231 | total_timesteps 4847.
Path 232 | total_timesteps 4865.
Path 233 | total_timesteps 4881.
Path 234 | total_timesteps 4892.
Path 235 | total_timesteps 4902.
Path 236 | total_timesteps 4918.
Path 237 | total_timesteps 4938.
Path 238 | total_timesteps 4962.
Path 239 | total_timesteps 4978.
Path 240 | total_timesteps 5001.
Path 241 | total_timesteps 5013.
Path 242 | total_timesteps 5046.
Path 243 | total_timesteps 5073.
Path 244 | total_timesteps 5095.
Path 245 | total_timesteps 5117.
Path 246 | total_timesteps 5134.
Path 247 | total_timesteps 5161.
Path 248 | total_timesteps 5193.
Path 249 | total_timesteps 5218.
Path 250 | total_timesteps 5263.
Path 251 | total_timesteps 5276.
Path 252 | total_timesteps 5307.
Path 253 | total_timesteps 5318.
Path 254 | total_timesteps 5337.
Path 255 | total_timesteps 5347.
Path 256 | total_timesteps 5359.
Path 257 | total_timesteps 5371.
Path 258 | total_timesteps 5399.
Path 259 | total_timesteps 5415.
Path 260 | total_timesteps 5439.
Path 261 | total_timesteps 5454.
Path 262 | total_timesteps 5471.
Path 263 | total_timesteps 5488.
Path 264 | total_timesteps 5499.
Path 265 | total_timesteps 5532.
Path 266 | total_timesteps 5597.
Path 267 | total_timesteps 5655.
Path 268 | total_timesteps 5669.
Path 269 | total_timesteps 5686.
Path 270 | total_timesteps 5700.
Path 271 | total_timesteps 5716.
Path 272 | total_timesteps 5733.
Path 273 | total_timesteps 5760.
Path 274 | total_timesteps 5774.
Path 275 | total_timesteps 5807.
Path 276 | total_timesteps 5822.
Path 277 | total_timesteps 5837.
Path 278 | total_timesteps 5859.
Path 279 | total_timesteps 5872.
Path 280 | total_timesteps 5897.
Path 281 | total_timesteps 5909.
Path 282 | total_timesteps 5929.
Path 283 | total_timesteps 5940.
Path 284 | total_timesteps 5960.
Path 285 | total_timesteps 5984.
Path 286 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.64    |
| Iteration     | 8        |
| MaximumReturn | 11.4     |
| MinimumReturn | -25.6    |
| TotalSamples  | 40078    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.010728835128247738
Validation loss = 0.010020194575190544
Validation loss = 0.01001555286347866
Validation loss = 0.010985266417264938
Validation loss = 0.009384245611727238
Validation loss = 0.010038476437330246
Validation loss = 0.009157282300293446
Validation loss = 0.009938429109752178
Validation loss = 0.009536155499517918
Validation loss = 0.009139949455857277
Validation loss = 0.009148756042122841
Validation loss = 0.010390492156147957
Validation loss = 0.009311525151133537
Validation loss = 0.009195900522172451
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 27.
Path 3 | total_timesteps 44.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 74.
Path 6 | total_timesteps 95.
Path 7 | total_timesteps 117.
Path 8 | total_timesteps 135.
Path 9 | total_timesteps 171.
Path 10 | total_timesteps 191.
Path 11 | total_timesteps 213.
Path 12 | total_timesteps 246.
Path 13 | total_timesteps 266.
Path 14 | total_timesteps 287.
Path 15 | total_timesteps 296.
Path 16 | total_timesteps 311.
Path 17 | total_timesteps 327.
Path 18 | total_timesteps 339.
Path 19 | total_timesteps 368.
Path 20 | total_timesteps 378.
Path 21 | total_timesteps 410.
Path 22 | total_timesteps 420.
Path 23 | total_timesteps 440.
Path 24 | total_timesteps 464.
Path 25 | total_timesteps 485.
Path 26 | total_timesteps 501.
Path 27 | total_timesteps 524.
Path 28 | total_timesteps 538.
Path 29 | total_timesteps 548.
Path 30 | total_timesteps 578.
Path 31 | total_timesteps 603.
Path 32 | total_timesteps 619.
Path 33 | total_timesteps 637.
Path 34 | total_timesteps 651.
Path 35 | total_timesteps 675.
Path 36 | total_timesteps 693.
Path 37 | total_timesteps 706.
Path 38 | total_timesteps 724.
Path 39 | total_timesteps 735.
Path 40 | total_timesteps 767.
Path 41 | total_timesteps 787.
Path 42 | total_timesteps 798.
Path 43 | total_timesteps 831.
Path 44 | total_timesteps 844.
Path 45 | total_timesteps 869.
Path 46 | total_timesteps 914.
Path 47 | total_timesteps 928.
Path 48 | total_timesteps 944.
Path 49 | total_timesteps 964.
Path 50 | total_timesteps 985.
Path 51 | total_timesteps 1002.
Path 52 | total_timesteps 1011.
Path 53 | total_timesteps 1031.
Path 54 | total_timesteps 1044.
Path 55 | total_timesteps 1055.
Path 56 | total_timesteps 1073.
Path 57 | total_timesteps 1091.
Path 58 | total_timesteps 1119.
Path 59 | total_timesteps 1148.
Path 60 | total_timesteps 1172.
Path 61 | total_timesteps 1181.
Path 62 | total_timesteps 1193.
Path 63 | total_timesteps 1215.
Path 64 | total_timesteps 1237.
Path 65 | total_timesteps 1256.
Path 66 | total_timesteps 1276.
Path 67 | total_timesteps 1300.
Path 68 | total_timesteps 1325.
Path 69 | total_timesteps 1337.
Path 70 | total_timesteps 1354.
Path 71 | total_timesteps 1369.
Path 72 | total_timesteps 1394.
Path 73 | total_timesteps 1416.
Path 74 | total_timesteps 1447.
Path 75 | total_timesteps 1457.
Path 76 | total_timesteps 1469.
Path 77 | total_timesteps 1482.
Path 78 | total_timesteps 1498.
Path 79 | total_timesteps 1516.
Path 80 | total_timesteps 1539.
Path 81 | total_timesteps 1559.
Path 82 | total_timesteps 1571.
Path 83 | total_timesteps 1590.
Path 84 | total_timesteps 1607.
Path 85 | total_timesteps 1622.
Path 86 | total_timesteps 1663.
Path 87 | total_timesteps 1677.
Path 88 | total_timesteps 1688.
Path 89 | total_timesteps 1705.
Path 90 | total_timesteps 1734.
Path 91 | total_timesteps 1749.
Path 92 | total_timesteps 1766.
Path 93 | total_timesteps 1787.
Path 94 | total_timesteps 1824.
Path 95 | total_timesteps 1838.
Path 96 | total_timesteps 1858.
Path 97 | total_timesteps 1897.
Path 98 | total_timesteps 1908.
Path 99 | total_timesteps 1928.
Path 100 | total_timesteps 1952.
Path 101 | total_timesteps 1966.
Path 102 | total_timesteps 1976.
Path 103 | total_timesteps 1998.
Path 104 | total_timesteps 2023.
Path 105 | total_timesteps 2050.
Path 106 | total_timesteps 2067.
Path 107 | total_timesteps 2080.
Path 108 | total_timesteps 2088.
Path 109 | total_timesteps 2147.
Path 110 | total_timesteps 2161.
Path 111 | total_timesteps 2172.
Path 112 | total_timesteps 2200.
Path 113 | total_timesteps 2222.
Path 114 | total_timesteps 2247.
Path 115 | total_timesteps 2269.
Path 116 | total_timesteps 2282.
Path 117 | total_timesteps 2292.
Path 118 | total_timesteps 2304.
Path 119 | total_timesteps 2315.
Path 120 | total_timesteps 2327.
Path 121 | total_timesteps 2342.
Path 122 | total_timesteps 2357.
Path 123 | total_timesteps 2371.
Path 124 | total_timesteps 2393.
Path 125 | total_timesteps 2415.
Path 126 | total_timesteps 2427.
Path 127 | total_timesteps 2439.
Path 128 | total_timesteps 2452.
Path 129 | total_timesteps 2469.
Path 130 | total_timesteps 2488.
Path 131 | total_timesteps 2510.
Path 132 | total_timesteps 2522.
Path 133 | total_timesteps 2559.
Path 134 | total_timesteps 2572.
Path 135 | total_timesteps 2585.
Path 136 | total_timesteps 2612.
Path 137 | total_timesteps 2627.
Path 138 | total_timesteps 2651.
Path 139 | total_timesteps 2685.
Path 140 | total_timesteps 2708.
Path 141 | total_timesteps 2731.
Path 142 | total_timesteps 2742.
Path 143 | total_timesteps 2753.
Path 144 | total_timesteps 2767.
Path 145 | total_timesteps 2777.
Path 146 | total_timesteps 2814.
Path 147 | total_timesteps 2826.
Path 148 | total_timesteps 2853.
Path 149 | total_timesteps 2871.
Path 150 | total_timesteps 2883.
Path 151 | total_timesteps 2915.
Path 152 | total_timesteps 2931.
Path 153 | total_timesteps 2960.
Path 154 | total_timesteps 2980.
Path 155 | total_timesteps 3004.
Path 156 | total_timesteps 3030.
Path 157 | total_timesteps 3040.
Path 158 | total_timesteps 3053.
Path 159 | total_timesteps 3067.
Path 160 | total_timesteps 3093.
Path 161 | total_timesteps 3112.
Path 162 | total_timesteps 3139.
Path 163 | total_timesteps 3151.
Path 164 | total_timesteps 3177.
Path 165 | total_timesteps 3221.
Path 166 | total_timesteps 3243.
Path 167 | total_timesteps 3257.
Path 168 | total_timesteps 3270.
Path 169 | total_timesteps 3285.
Path 170 | total_timesteps 3308.
Path 171 | total_timesteps 3345.
Path 172 | total_timesteps 3357.
Path 173 | total_timesteps 3376.
Path 174 | total_timesteps 3393.
Path 175 | total_timesteps 3418.
Path 176 | total_timesteps 3438.
Path 177 | total_timesteps 3465.
Path 178 | total_timesteps 3482.
Path 179 | total_timesteps 3502.
Path 180 | total_timesteps 3525.
Path 181 | total_timesteps 3559.
Path 182 | total_timesteps 3574.
Path 183 | total_timesteps 3603.
Path 184 | total_timesteps 3617.
Path 185 | total_timesteps 3637.
Path 186 | total_timesteps 3656.
Path 187 | total_timesteps 3669.
Path 188 | total_timesteps 3687.
Path 189 | total_timesteps 3697.
Path 190 | total_timesteps 3716.
Path 191 | total_timesteps 3737.
Path 192 | total_timesteps 3753.
Path 193 | total_timesteps 3763.
Path 194 | total_timesteps 3785.
Path 195 | total_timesteps 3800.
Path 196 | total_timesteps 3820.
Path 197 | total_timesteps 3844.
Path 198 | total_timesteps 3866.
Path 199 | total_timesteps 3880.
Path 200 | total_timesteps 3896.
Path 201 | total_timesteps 3921.
Path 202 | total_timesteps 3951.
Path 203 | total_timesteps 3966.
Path 204 | total_timesteps 3986.
Path 205 | total_timesteps 4011.
Path 206 | total_timesteps 4021.
Path 207 | total_timesteps 4045.
Path 208 | total_timesteps 4085.
Path 209 | total_timesteps 4098.
Path 210 | total_timesteps 4109.
Path 211 | total_timesteps 4120.
Path 212 | total_timesteps 4154.
Path 213 | total_timesteps 4180.
Path 214 | total_timesteps 4229.
Path 215 | total_timesteps 4240.
Path 216 | total_timesteps 4251.
Path 217 | total_timesteps 4265.
Path 218 | total_timesteps 4308.
Path 219 | total_timesteps 4341.
Path 220 | total_timesteps 4356.
Path 221 | total_timesteps 4384.
Path 222 | total_timesteps 4398.
Path 223 | total_timesteps 4414.
Path 224 | total_timesteps 4430.
Path 225 | total_timesteps 4450.
Path 226 | total_timesteps 4465.
Path 227 | total_timesteps 4478.
Path 228 | total_timesteps 4496.
Path 229 | total_timesteps 4520.
Path 230 | total_timesteps 4539.
Path 231 | total_timesteps 4551.
Path 232 | total_timesteps 4569.
Path 233 | total_timesteps 4597.
Path 234 | total_timesteps 4626.
Path 235 | total_timesteps 4653.
Path 236 | total_timesteps 4677.
Path 237 | total_timesteps 4698.
Path 238 | total_timesteps 4712.
Path 239 | total_timesteps 4730.
Path 240 | total_timesteps 4757.
Path 241 | total_timesteps 4778.
Path 242 | total_timesteps 4805.
Path 243 | total_timesteps 4815.
Path 244 | total_timesteps 4847.
Path 245 | total_timesteps 4867.
Path 246 | total_timesteps 4896.
Path 247 | total_timesteps 4915.
Path 248 | total_timesteps 4940.
Path 249 | total_timesteps 4963.
Path 250 | total_timesteps 4974.
Path 251 | total_timesteps 5004.
Path 252 | total_timesteps 5017.
Path 253 | total_timesteps 5033.
Path 254 | total_timesteps 5046.
Path 255 | total_timesteps 5083.
Path 256 | total_timesteps 5099.
Path 257 | total_timesteps 5118.
Path 258 | total_timesteps 5137.
Path 259 | total_timesteps 5160.
Path 260 | total_timesteps 5177.
Path 261 | total_timesteps 5186.
Path 262 | total_timesteps 5205.
Path 263 | total_timesteps 5227.
Path 264 | total_timesteps 5243.
Path 265 | total_timesteps 5287.
Path 266 | total_timesteps 5302.
Path 267 | total_timesteps 5323.
Path 268 | total_timesteps 5352.
Path 269 | total_timesteps 5368.
Path 270 | total_timesteps 5391.
Path 271 | total_timesteps 5406.
Path 272 | total_timesteps 5417.
Path 273 | total_timesteps 5432.
Path 274 | total_timesteps 5449.
Path 275 | total_timesteps 5468.
Path 276 | total_timesteps 5495.
Path 277 | total_timesteps 5511.
Path 278 | total_timesteps 5531.
Path 279 | total_timesteps 5555.
Path 280 | total_timesteps 5569.
Path 281 | total_timesteps 5587.
Path 282 | total_timesteps 5609.
Path 283 | total_timesteps 5624.
Path 284 | total_timesteps 5673.
Path 285 | total_timesteps 5691.
Path 286 | total_timesteps 5715.
Path 287 | total_timesteps 5737.
Path 288 | total_timesteps 5759.
Path 289 | total_timesteps 5774.
Path 290 | total_timesteps 5787.
Path 291 | total_timesteps 5804.
Path 292 | total_timesteps 5814.
Path 293 | total_timesteps 5828.
Path 294 | total_timesteps 5841.
Path 295 | total_timesteps 5857.
Path 296 | total_timesteps 5872.
Path 297 | total_timesteps 5889.
Path 298 | total_timesteps 5905.
Path 299 | total_timesteps 5918.
Path 300 | total_timesteps 5935.
Path 301 | total_timesteps 5956.
Path 302 | total_timesteps 5982.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.84    |
| Iteration     | 9        |
| MaximumReturn | 15.2     |
| MinimumReturn | -20.4    |
| TotalSamples  | 44079    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.009463861584663391
Validation loss = 0.00877232477068901
Validation loss = 0.009065143764019012
Validation loss = 0.009635296650230885
Validation loss = 0.008853581734001637
Validation loss = 0.009251951240003109
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 50.
Path 4 | total_timesteps 67.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 109.
Path 7 | total_timesteps 139.
Path 8 | total_timesteps 165.
Path 9 | total_timesteps 195.
Path 10 | total_timesteps 222.
Path 11 | total_timesteps 238.
Path 12 | total_timesteps 280.
Path 13 | total_timesteps 295.
Path 14 | total_timesteps 317.
Path 15 | total_timesteps 331.
Path 16 | total_timesteps 346.
Path 17 | total_timesteps 360.
Path 18 | total_timesteps 382.
Path 19 | total_timesteps 410.
Path 20 | total_timesteps 431.
Path 21 | total_timesteps 452.
Path 22 | total_timesteps 483.
Path 23 | total_timesteps 501.
Path 24 | total_timesteps 544.
Path 25 | total_timesteps 561.
Path 26 | total_timesteps 585.
Path 27 | total_timesteps 603.
Path 28 | total_timesteps 614.
Path 29 | total_timesteps 629.
Path 30 | total_timesteps 645.
Path 31 | total_timesteps 661.
Path 32 | total_timesteps 674.
Path 33 | total_timesteps 699.
Path 34 | total_timesteps 711.
Path 35 | total_timesteps 730.
Path 36 | total_timesteps 744.
Path 37 | total_timesteps 767.
Path 38 | total_timesteps 801.
Path 39 | total_timesteps 818.
Path 40 | total_timesteps 837.
Path 41 | total_timesteps 866.
Path 42 | total_timesteps 903.
Path 43 | total_timesteps 932.
Path 44 | total_timesteps 949.
Path 45 | total_timesteps 962.
Path 46 | total_timesteps 979.
Path 47 | total_timesteps 1007.
Path 48 | total_timesteps 1018.
Path 49 | total_timesteps 1038.
Path 50 | total_timesteps 1053.
Path 51 | total_timesteps 1069.
Path 52 | total_timesteps 1095.
Path 53 | total_timesteps 1117.
Path 54 | total_timesteps 1143.
Path 55 | total_timesteps 1154.
Path 56 | total_timesteps 1170.
Path 57 | total_timesteps 1182.
Path 58 | total_timesteps 1191.
Path 59 | total_timesteps 1202.
Path 60 | total_timesteps 1221.
Path 61 | total_timesteps 1239.
Path 62 | total_timesteps 1252.
Path 63 | total_timesteps 1269.
Path 64 | total_timesteps 1283.
Path 65 | total_timesteps 1305.
Path 66 | total_timesteps 1329.
Path 67 | total_timesteps 1342.
Path 68 | total_timesteps 1354.
Path 69 | total_timesteps 1364.
Path 70 | total_timesteps 1372.
Path 71 | total_timesteps 1398.
Path 72 | total_timesteps 1421.
Path 73 | total_timesteps 1442.
Path 74 | total_timesteps 1458.
Path 75 | total_timesteps 1473.
Path 76 | total_timesteps 1491.
Path 77 | total_timesteps 1505.
Path 78 | total_timesteps 1518.
Path 79 | total_timesteps 1531.
Path 80 | total_timesteps 1558.
Path 81 | total_timesteps 1576.
Path 82 | total_timesteps 1590.
Path 83 | total_timesteps 1613.
Path 84 | total_timesteps 1629.
Path 85 | total_timesteps 1647.
Path 86 | total_timesteps 1676.
Path 87 | total_timesteps 1704.
Path 88 | total_timesteps 1727.
Path 89 | total_timesteps 1744.
Path 90 | total_timesteps 1775.
Path 91 | total_timesteps 1797.
Path 92 | total_timesteps 1814.
Path 93 | total_timesteps 1826.
Path 94 | total_timesteps 1855.
Path 95 | total_timesteps 1871.
Path 96 | total_timesteps 1886.
Path 97 | total_timesteps 1935.
Path 98 | total_timesteps 1945.
Path 99 | total_timesteps 1965.
Path 100 | total_timesteps 1995.
Path 101 | total_timesteps 2009.
Path 102 | total_timesteps 2024.
Path 103 | total_timesteps 2032.
Path 104 | total_timesteps 2052.
Path 105 | total_timesteps 2060.
Path 106 | total_timesteps 2083.
Path 107 | total_timesteps 2094.
Path 108 | total_timesteps 2108.
Path 109 | total_timesteps 2126.
Path 110 | total_timesteps 2149.
Path 111 | total_timesteps 2167.
Path 112 | total_timesteps 2202.
Path 113 | total_timesteps 2217.
Path 114 | total_timesteps 2229.
Path 115 | total_timesteps 2248.
Path 116 | total_timesteps 2261.
Path 117 | total_timesteps 2274.
Path 118 | total_timesteps 2293.
Path 119 | total_timesteps 2309.
Path 120 | total_timesteps 2321.
Path 121 | total_timesteps 2335.
Path 122 | total_timesteps 2357.
Path 123 | total_timesteps 2374.
Path 124 | total_timesteps 2403.
Path 125 | total_timesteps 2419.
Path 126 | total_timesteps 2442.
Path 127 | total_timesteps 2477.
Path 128 | total_timesteps 2494.
Path 129 | total_timesteps 2520.
Path 130 | total_timesteps 2545.
Path 131 | total_timesteps 2555.
Path 132 | total_timesteps 2565.
Path 133 | total_timesteps 2584.
Path 134 | total_timesteps 2616.
Path 135 | total_timesteps 2632.
Path 136 | total_timesteps 2649.
Path 137 | total_timesteps 2670.
Path 138 | total_timesteps 2691.
Path 139 | total_timesteps 2722.
Path 140 | total_timesteps 2745.
Path 141 | total_timesteps 2756.
Path 142 | total_timesteps 2799.
Path 143 | total_timesteps 2814.
Path 144 | total_timesteps 2844.
Path 145 | total_timesteps 2866.
Path 146 | total_timesteps 2881.
Path 147 | total_timesteps 2895.
Path 148 | total_timesteps 2913.
Path 149 | total_timesteps 2937.
Path 150 | total_timesteps 2951.
Path 151 | total_timesteps 2975.
Path 152 | total_timesteps 2987.
Path 153 | total_timesteps 2997.
Path 154 | total_timesteps 3008.
Path 155 | total_timesteps 3019.
Path 156 | total_timesteps 3029.
Path 157 | total_timesteps 3041.
Path 158 | total_timesteps 3059.
Path 159 | total_timesteps 3083.
Path 160 | total_timesteps 3096.
Path 161 | total_timesteps 3121.
Path 162 | total_timesteps 3145.
Path 163 | total_timesteps 3169.
Path 164 | total_timesteps 3190.
Path 165 | total_timesteps 3214.
Path 166 | total_timesteps 3251.
Path 167 | total_timesteps 3259.
Path 168 | total_timesteps 3271.
Path 169 | total_timesteps 3291.
Path 170 | total_timesteps 3305.
Path 171 | total_timesteps 3325.
Path 172 | total_timesteps 3342.
Path 173 | total_timesteps 3360.
Path 174 | total_timesteps 3385.
Path 175 | total_timesteps 3410.
Path 176 | total_timesteps 3422.
Path 177 | total_timesteps 3431.
Path 178 | total_timesteps 3440.
Path 179 | total_timesteps 3459.
Path 180 | total_timesteps 3488.
Path 181 | total_timesteps 3538.
Path 182 | total_timesteps 3556.
Path 183 | total_timesteps 3573.
Path 184 | total_timesteps 3585.
Path 185 | total_timesteps 3603.
Path 186 | total_timesteps 3627.
Path 187 | total_timesteps 3643.
Path 188 | total_timesteps 3655.
Path 189 | total_timesteps 3712.
Path 190 | total_timesteps 3741.
Path 191 | total_timesteps 3762.
Path 192 | total_timesteps 3778.
Path 193 | total_timesteps 3799.
Path 194 | total_timesteps 3823.
Path 195 | total_timesteps 3850.
Path 196 | total_timesteps 3864.
Path 197 | total_timesteps 3879.
Path 198 | total_timesteps 3896.
Path 199 | total_timesteps 3921.
Path 200 | total_timesteps 3942.
Path 201 | total_timesteps 3957.
Path 202 | total_timesteps 3970.
Path 203 | total_timesteps 3991.
Path 204 | total_timesteps 4024.
Path 205 | total_timesteps 4033.
Path 206 | total_timesteps 4055.
Path 207 | total_timesteps 4073.
Path 208 | total_timesteps 4091.
Path 209 | total_timesteps 4108.
Path 210 | total_timesteps 4122.
Path 211 | total_timesteps 4144.
Path 212 | total_timesteps 4170.
Path 213 | total_timesteps 4193.
Path 214 | total_timesteps 4204.
Path 215 | total_timesteps 4226.
Path 216 | total_timesteps 4237.
Path 217 | total_timesteps 4252.
Path 218 | total_timesteps 4273.
Path 219 | total_timesteps 4304.
Path 220 | total_timesteps 4314.
Path 221 | total_timesteps 4344.
Path 222 | total_timesteps 4358.
Path 223 | total_timesteps 4379.
Path 224 | total_timesteps 4396.
Path 225 | total_timesteps 4428.
Path 226 | total_timesteps 4441.
Path 227 | total_timesteps 4461.
Path 228 | total_timesteps 4474.
Path 229 | total_timesteps 4489.
Path 230 | total_timesteps 4513.
Path 231 | total_timesteps 4523.
Path 232 | total_timesteps 4532.
Path 233 | total_timesteps 4550.
Path 234 | total_timesteps 4577.
Path 235 | total_timesteps 4595.
Path 236 | total_timesteps 4611.
Path 237 | total_timesteps 4625.
Path 238 | total_timesteps 4650.
Path 239 | total_timesteps 4673.
Path 240 | total_timesteps 4686.
Path 241 | total_timesteps 4710.
Path 242 | total_timesteps 4727.
Path 243 | total_timesteps 4779.
Path 244 | total_timesteps 4796.
Path 245 | total_timesteps 4819.
Path 246 | total_timesteps 4838.
Path 247 | total_timesteps 4849.
Path 248 | total_timesteps 4880.
Path 249 | total_timesteps 4888.
Path 250 | total_timesteps 4920.
Path 251 | total_timesteps 4933.
Path 252 | total_timesteps 4981.
Path 253 | total_timesteps 5027.
Path 254 | total_timesteps 5043.
Path 255 | total_timesteps 5058.
Path 256 | total_timesteps 5073.
Path 257 | total_timesteps 5087.
Path 258 | total_timesteps 5115.
Path 259 | total_timesteps 5127.
Path 260 | total_timesteps 5149.
Path 261 | total_timesteps 5164.
Path 262 | total_timesteps 5182.
Path 263 | total_timesteps 5194.
Path 264 | total_timesteps 5208.
Path 265 | total_timesteps 5227.
Path 266 | total_timesteps 5245.
Path 267 | total_timesteps 5257.
Path 268 | total_timesteps 5278.
Path 269 | total_timesteps 5308.
Path 270 | total_timesteps 5327.
Path 271 | total_timesteps 5338.
Path 272 | total_timesteps 5355.
Path 273 | total_timesteps 5372.
Path 274 | total_timesteps 5384.
Path 275 | total_timesteps 5415.
Path 276 | total_timesteps 5436.
Path 277 | total_timesteps 5467.
Path 278 | total_timesteps 5481.
Path 279 | total_timesteps 5513.
Path 280 | total_timesteps 5524.
Path 281 | total_timesteps 5544.
Path 282 | total_timesteps 5561.
Path 283 | total_timesteps 5573.
Path 284 | total_timesteps 5594.
Path 285 | total_timesteps 5610.
Path 286 | total_timesteps 5625.
Path 287 | total_timesteps 5642.
Path 288 | total_timesteps 5666.
Path 289 | total_timesteps 5683.
Path 290 | total_timesteps 5703.
Path 291 | total_timesteps 5720.
Path 292 | total_timesteps 5738.
Path 293 | total_timesteps 5766.
Path 294 | total_timesteps 5784.
Path 295 | total_timesteps 5793.
Path 296 | total_timesteps 5805.
Path 297 | total_timesteps 5816.
Path 298 | total_timesteps 5833.
Path 299 | total_timesteps 5843.
Path 300 | total_timesteps 5866.
Path 301 | total_timesteps 5878.
Path 302 | total_timesteps 5889.
Path 303 | total_timesteps 5903.
Path 304 | total_timesteps 5915.
Path 305 | total_timesteps 5952.
Path 306 | total_timesteps 5974.
Path 307 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.46    |
| Iteration     | 10       |
| MaximumReturn | 6.78     |
| MinimumReturn | -23.4    |
| TotalSamples  | 48084    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.008271201513707638
Validation loss = 0.008063036948442459
Validation loss = 0.0076774186454713345
Validation loss = 0.00796105619519949
Validation loss = 0.00797139573842287
Validation loss = 0.007228778675198555
Validation loss = 0.007757002953439951
Validation loss = 0.00834542978554964
Validation loss = 0.007484404370188713
Validation loss = 0.007837992161512375
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 63.
Path 5 | total_timesteps 78.
Path 6 | total_timesteps 100.
Path 7 | total_timesteps 121.
Path 8 | total_timesteps 141.
Path 9 | total_timesteps 154.
Path 10 | total_timesteps 162.
Path 11 | total_timesteps 178.
Path 12 | total_timesteps 194.
Path 13 | total_timesteps 207.
Path 14 | total_timesteps 226.
Path 15 | total_timesteps 249.
Path 16 | total_timesteps 266.
Path 17 | total_timesteps 278.
Path 18 | total_timesteps 293.
Path 19 | total_timesteps 317.
Path 20 | total_timesteps 337.
Path 21 | total_timesteps 354.
Path 22 | total_timesteps 367.
Path 23 | total_timesteps 385.
Path 24 | total_timesteps 398.
Path 25 | total_timesteps 414.
Path 26 | total_timesteps 441.
Path 27 | total_timesteps 456.
Path 28 | total_timesteps 476.
Path 29 | total_timesteps 500.
Path 30 | total_timesteps 518.
Path 31 | total_timesteps 534.
Path 32 | total_timesteps 557.
Path 33 | total_timesteps 575.
Path 34 | total_timesteps 595.
Path 35 | total_timesteps 607.
Path 36 | total_timesteps 630.
Path 37 | total_timesteps 645.
Path 38 | total_timesteps 664.
Path 39 | total_timesteps 676.
Path 40 | total_timesteps 685.
Path 41 | total_timesteps 705.
Path 42 | total_timesteps 738.
Path 43 | total_timesteps 756.
Path 44 | total_timesteps 769.
Path 45 | total_timesteps 787.
Path 46 | total_timesteps 811.
Path 47 | total_timesteps 833.
Path 48 | total_timesteps 849.
Path 49 | total_timesteps 859.
Path 50 | total_timesteps 879.
Path 51 | total_timesteps 889.
Path 52 | total_timesteps 903.
Path 53 | total_timesteps 919.
Path 54 | total_timesteps 931.
Path 55 | total_timesteps 948.
Path 56 | total_timesteps 961.
Path 57 | total_timesteps 981.
Path 58 | total_timesteps 993.
Path 59 | total_timesteps 1010.
Path 60 | total_timesteps 1023.
Path 61 | total_timesteps 1035.
Path 62 | total_timesteps 1054.
Path 63 | total_timesteps 1066.
Path 64 | total_timesteps 1081.
Path 65 | total_timesteps 1112.
Path 66 | total_timesteps 1125.
Path 67 | total_timesteps 1134.
Path 68 | total_timesteps 1156.
Path 69 | total_timesteps 1165.
Path 70 | total_timesteps 1184.
Path 71 | total_timesteps 1199.
Path 72 | total_timesteps 1215.
Path 73 | total_timesteps 1227.
Path 74 | total_timesteps 1243.
Path 75 | total_timesteps 1262.
Path 76 | total_timesteps 1288.
Path 77 | total_timesteps 1309.
Path 78 | total_timesteps 1323.
Path 79 | total_timesteps 1338.
Path 80 | total_timesteps 1361.
Path 81 | total_timesteps 1377.
Path 82 | total_timesteps 1390.
Path 83 | total_timesteps 1400.
Path 84 | total_timesteps 1418.
Path 85 | total_timesteps 1437.
Path 86 | total_timesteps 1458.
Path 87 | total_timesteps 1469.
Path 88 | total_timesteps 1488.
Path 89 | total_timesteps 1521.
Path 90 | total_timesteps 1538.
Path 91 | total_timesteps 1552.
Path 92 | total_timesteps 1568.
Path 93 | total_timesteps 1583.
Path 94 | total_timesteps 1593.
Path 95 | total_timesteps 1608.
Path 96 | total_timesteps 1620.
Path 97 | total_timesteps 1639.
Path 98 | total_timesteps 1656.
Path 99 | total_timesteps 1668.
Path 100 | total_timesteps 1688.
Path 101 | total_timesteps 1700.
Path 102 | total_timesteps 1721.
Path 103 | total_timesteps 1730.
Path 104 | total_timesteps 1742.
Path 105 | total_timesteps 1752.
Path 106 | total_timesteps 1765.
Path 107 | total_timesteps 1775.
Path 108 | total_timesteps 1785.
Path 109 | total_timesteps 1796.
Path 110 | total_timesteps 1809.
Path 111 | total_timesteps 1822.
Path 112 | total_timesteps 1837.
Path 113 | total_timesteps 1859.
Path 114 | total_timesteps 1872.
Path 115 | total_timesteps 1882.
Path 116 | total_timesteps 1903.
Path 117 | total_timesteps 1914.
Path 118 | total_timesteps 1928.
Path 119 | total_timesteps 1947.
Path 120 | total_timesteps 1967.
Path 121 | total_timesteps 1980.
Path 122 | total_timesteps 1998.
Path 123 | total_timesteps 2014.
Path 124 | total_timesteps 2026.
Path 125 | total_timesteps 2038.
Path 126 | total_timesteps 2049.
Path 127 | total_timesteps 2064.
Path 128 | total_timesteps 2081.
Path 129 | total_timesteps 2091.
Path 130 | total_timesteps 2116.
Path 131 | total_timesteps 2131.
Path 132 | total_timesteps 2146.
Path 133 | total_timesteps 2160.
Path 134 | total_timesteps 2188.
Path 135 | total_timesteps 2204.
Path 136 | total_timesteps 2216.
Path 137 | total_timesteps 2229.
Path 138 | total_timesteps 2244.
Path 139 | total_timesteps 2260.
Path 140 | total_timesteps 2279.
Path 141 | total_timesteps 2297.
Path 142 | total_timesteps 2313.
Path 143 | total_timesteps 2323.
Path 144 | total_timesteps 2338.
Path 145 | total_timesteps 2351.
Path 146 | total_timesteps 2369.
Path 147 | total_timesteps 2395.
Path 148 | total_timesteps 2409.
Path 149 | total_timesteps 2426.
Path 150 | total_timesteps 2449.
Path 151 | total_timesteps 2471.
Path 152 | total_timesteps 2489.
Path 153 | total_timesteps 2508.
Path 154 | total_timesteps 2518.
Path 155 | total_timesteps 2535.
Path 156 | total_timesteps 2550.
Path 157 | total_timesteps 2570.
Path 158 | total_timesteps 2586.
Path 159 | total_timesteps 2596.
Path 160 | total_timesteps 2607.
Path 161 | total_timesteps 2622.
Path 162 | total_timesteps 2637.
Path 163 | total_timesteps 2654.
Path 164 | total_timesteps 2671.
Path 165 | total_timesteps 2696.
Path 166 | total_timesteps 2709.
Path 167 | total_timesteps 2736.
Path 168 | total_timesteps 2753.
Path 169 | total_timesteps 2781.
Path 170 | total_timesteps 2812.
Path 171 | total_timesteps 2827.
Path 172 | total_timesteps 2838.
Path 173 | total_timesteps 2852.
Path 174 | total_timesteps 2861.
Path 175 | total_timesteps 2881.
Path 176 | total_timesteps 2901.
Path 177 | total_timesteps 2914.
Path 178 | total_timesteps 2925.
Path 179 | total_timesteps 2935.
Path 180 | total_timesteps 2948.
Path 181 | total_timesteps 2966.
Path 182 | total_timesteps 2984.
Path 183 | total_timesteps 2998.
Path 184 | total_timesteps 3017.
Path 185 | total_timesteps 3031.
Path 186 | total_timesteps 3055.
Path 187 | total_timesteps 3072.
Path 188 | total_timesteps 3091.
Path 189 | total_timesteps 3100.
Path 190 | total_timesteps 3116.
Path 191 | total_timesteps 3129.
Path 192 | total_timesteps 3141.
Path 193 | total_timesteps 3160.
Path 194 | total_timesteps 3179.
Path 195 | total_timesteps 3193.
Path 196 | total_timesteps 3210.
Path 197 | total_timesteps 3233.
Path 198 | total_timesteps 3249.
Path 199 | total_timesteps 3260.
Path 200 | total_timesteps 3278.
Path 201 | total_timesteps 3296.
Path 202 | total_timesteps 3309.
Path 203 | total_timesteps 3325.
Path 204 | total_timesteps 3346.
Path 205 | total_timesteps 3367.
Path 206 | total_timesteps 3377.
Path 207 | total_timesteps 3395.
Path 208 | total_timesteps 3413.
Path 209 | total_timesteps 3422.
Path 210 | total_timesteps 3443.
Path 211 | total_timesteps 3458.
Path 212 | total_timesteps 3477.
Path 213 | total_timesteps 3500.
Path 214 | total_timesteps 3518.
Path 215 | total_timesteps 3534.
Path 216 | total_timesteps 3552.
Path 217 | total_timesteps 3570.
Path 218 | total_timesteps 3580.
Path 219 | total_timesteps 3602.
Path 220 | total_timesteps 3615.
Path 221 | total_timesteps 3627.
Path 222 | total_timesteps 3649.
Path 223 | total_timesteps 3668.
Path 224 | total_timesteps 3677.
Path 225 | total_timesteps 3690.
Path 226 | total_timesteps 3707.
Path 227 | total_timesteps 3720.
Path 228 | total_timesteps 3732.
Path 229 | total_timesteps 3757.
Path 230 | total_timesteps 3780.
Path 231 | total_timesteps 3796.
Path 232 | total_timesteps 3817.
Path 233 | total_timesteps 3831.
Path 234 | total_timesteps 3842.
Path 235 | total_timesteps 3856.
Path 236 | total_timesteps 3872.
Path 237 | total_timesteps 3890.
Path 238 | total_timesteps 3903.
Path 239 | total_timesteps 3918.
Path 240 | total_timesteps 3935.
Path 241 | total_timesteps 3961.
Path 242 | total_timesteps 3969.
Path 243 | total_timesteps 3980.
Path 244 | total_timesteps 3994.
Path 245 | total_timesteps 4013.
Path 246 | total_timesteps 4025.
Path 247 | total_timesteps 4035.
Path 248 | total_timesteps 4056.
Path 249 | total_timesteps 4069.
Path 250 | total_timesteps 4084.
Path 251 | total_timesteps 4098.
Path 252 | total_timesteps 4113.
Path 253 | total_timesteps 4125.
Path 254 | total_timesteps 4134.
Path 255 | total_timesteps 4153.
Path 256 | total_timesteps 4165.
Path 257 | total_timesteps 4179.
Path 258 | total_timesteps 4201.
Path 259 | total_timesteps 4217.
Path 260 | total_timesteps 4235.
Path 261 | total_timesteps 4244.
Path 262 | total_timesteps 4256.
Path 263 | total_timesteps 4272.
Path 264 | total_timesteps 4289.
Path 265 | total_timesteps 4299.
Path 266 | total_timesteps 4313.
Path 267 | total_timesteps 4327.
Path 268 | total_timesteps 4354.
Path 269 | total_timesteps 4365.
Path 270 | total_timesteps 4391.
Path 271 | total_timesteps 4410.
Path 272 | total_timesteps 4419.
Path 273 | total_timesteps 4430.
Path 274 | total_timesteps 4444.
Path 275 | total_timesteps 4458.
Path 276 | total_timesteps 4474.
Path 277 | total_timesteps 4497.
Path 278 | total_timesteps 4513.
Path 279 | total_timesteps 4528.
Path 280 | total_timesteps 4547.
Path 281 | total_timesteps 4561.
Path 282 | total_timesteps 4572.
Path 283 | total_timesteps 4589.
Path 284 | total_timesteps 4602.
Path 285 | total_timesteps 4616.
Path 286 | total_timesteps 4631.
Path 287 | total_timesteps 4642.
Path 288 | total_timesteps 4653.
Path 289 | total_timesteps 4668.
Path 290 | total_timesteps 4684.
Path 291 | total_timesteps 4694.
Path 292 | total_timesteps 4709.
Path 293 | total_timesteps 4729.
Path 294 | total_timesteps 4744.
Path 295 | total_timesteps 4758.
Path 296 | total_timesteps 4771.
Path 297 | total_timesteps 4788.
Path 298 | total_timesteps 4796.
Path 299 | total_timesteps 4809.
Path 300 | total_timesteps 4824.
Path 301 | total_timesteps 4850.
Path 302 | total_timesteps 4863.
Path 303 | total_timesteps 4874.
Path 304 | total_timesteps 4892.
Path 305 | total_timesteps 4927.
Path 306 | total_timesteps 4945.
Path 307 | total_timesteps 4957.
Path 308 | total_timesteps 4979.
Path 309 | total_timesteps 5001.
Path 310 | total_timesteps 5014.
Path 311 | total_timesteps 5022.
Path 312 | total_timesteps 5049.
Path 313 | total_timesteps 5063.
Path 314 | total_timesteps 5078.
Path 315 | total_timesteps 5091.
Path 316 | total_timesteps 5127.
Path 317 | total_timesteps 5143.
Path 318 | total_timesteps 5163.
Path 319 | total_timesteps 5178.
Path 320 | total_timesteps 5192.
Path 321 | total_timesteps 5208.
Path 322 | total_timesteps 5218.
Path 323 | total_timesteps 5229.
Path 324 | total_timesteps 5244.
Path 325 | total_timesteps 5260.
Path 326 | total_timesteps 5281.
Path 327 | total_timesteps 5291.
Path 328 | total_timesteps 5313.
Path 329 | total_timesteps 5323.
Path 330 | total_timesteps 5339.
Path 331 | total_timesteps 5363.
Path 332 | total_timesteps 5387.
Path 333 | total_timesteps 5418.
Path 334 | total_timesteps 5441.
Path 335 | total_timesteps 5456.
Path 336 | total_timesteps 5471.
Path 337 | total_timesteps 5486.
Path 338 | total_timesteps 5497.
Path 339 | total_timesteps 5520.
Path 340 | total_timesteps 5536.
Path 341 | total_timesteps 5549.
Path 342 | total_timesteps 5566.
Path 343 | total_timesteps 5582.
Path 344 | total_timesteps 5593.
Path 345 | total_timesteps 5610.
Path 346 | total_timesteps 5630.
Path 347 | total_timesteps 5649.
Path 348 | total_timesteps 5662.
Path 349 | total_timesteps 5677.
Path 350 | total_timesteps 5696.
Path 351 | total_timesteps 5709.
Path 352 | total_timesteps 5722.
Path 353 | total_timesteps 5737.
Path 354 | total_timesteps 5753.
Path 355 | total_timesteps 5768.
Path 356 | total_timesteps 5781.
Path 357 | total_timesteps 5791.
Path 358 | total_timesteps 5810.
Path 359 | total_timesteps 5819.
Path 360 | total_timesteps 5841.
Path 361 | total_timesteps 5854.
Path 362 | total_timesteps 5884.
Path 363 | total_timesteps 5899.
Path 364 | total_timesteps 5914.
Path 365 | total_timesteps 5928.
Path 366 | total_timesteps 5938.
Path 367 | total_timesteps 5955.
Path 368 | total_timesteps 5969.
Path 369 | total_timesteps 5984.
Path 370 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.9    |
| Iteration     | 11       |
| MaximumReturn | 4.87     |
| MinimumReturn | -23.4    |
| TotalSamples  | 52088    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.007677543442696333
Validation loss = 0.007398996036499739
Validation loss = 0.007095633074641228
Validation loss = 0.006918631494045258
Validation loss = 0.0064948550425469875
Validation loss = 0.006632191129028797
Validation loss = 0.006932677701115608
Validation loss = 0.00786960031837225
Validation loss = 0.007136613130569458
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 48.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 72.
Path 5 | total_timesteps 94.
Path 6 | total_timesteps 110.
Path 7 | total_timesteps 128.
Path 8 | total_timesteps 139.
Path 9 | total_timesteps 153.
Path 10 | total_timesteps 165.
Path 11 | total_timesteps 184.
Path 12 | total_timesteps 199.
Path 13 | total_timesteps 210.
Path 14 | total_timesteps 224.
Path 15 | total_timesteps 236.
Path 16 | total_timesteps 259.
Path 17 | total_timesteps 274.
Path 18 | total_timesteps 287.
Path 19 | total_timesteps 303.
Path 20 | total_timesteps 325.
Path 21 | total_timesteps 334.
Path 22 | total_timesteps 343.
Path 23 | total_timesteps 354.
Path 24 | total_timesteps 363.
Path 25 | total_timesteps 375.
Path 26 | total_timesteps 385.
Path 27 | total_timesteps 410.
Path 28 | total_timesteps 424.
Path 29 | total_timesteps 447.
Path 30 | total_timesteps 460.
Path 31 | total_timesteps 472.
Path 32 | total_timesteps 483.
Path 33 | total_timesteps 495.
Path 34 | total_timesteps 502.
Path 35 | total_timesteps 521.
Path 36 | total_timesteps 543.
Path 37 | total_timesteps 554.
Path 38 | total_timesteps 565.
Path 39 | total_timesteps 582.
Path 40 | total_timesteps 598.
Path 41 | total_timesteps 622.
Path 42 | total_timesteps 634.
Path 43 | total_timesteps 648.
Path 44 | total_timesteps 665.
Path 45 | total_timesteps 683.
Path 46 | total_timesteps 695.
Path 47 | total_timesteps 707.
Path 48 | total_timesteps 719.
Path 49 | total_timesteps 734.
Path 50 | total_timesteps 747.
Path 51 | total_timesteps 759.
Path 52 | total_timesteps 766.
Path 53 | total_timesteps 778.
Path 54 | total_timesteps 797.
Path 55 | total_timesteps 810.
Path 56 | total_timesteps 822.
Path 57 | total_timesteps 837.
Path 58 | total_timesteps 849.
Path 59 | total_timesteps 861.
Path 60 | total_timesteps 874.
Path 61 | total_timesteps 884.
Path 62 | total_timesteps 898.
Path 63 | total_timesteps 910.
Path 64 | total_timesteps 930.
Path 65 | total_timesteps 941.
Path 66 | total_timesteps 977.
Path 67 | total_timesteps 999.
Path 68 | total_timesteps 1009.
Path 69 | total_timesteps 1024.
Path 70 | total_timesteps 1036.
Path 71 | total_timesteps 1050.
Path 72 | total_timesteps 1066.
Path 73 | total_timesteps 1079.
Path 74 | total_timesteps 1102.
Path 75 | total_timesteps 1117.
Path 76 | total_timesteps 1132.
Path 77 | total_timesteps 1146.
Path 78 | total_timesteps 1155.
Path 79 | total_timesteps 1171.
Path 80 | total_timesteps 1189.
Path 81 | total_timesteps 1216.
Path 82 | total_timesteps 1231.
Path 83 | total_timesteps 1251.
Path 84 | total_timesteps 1268.
Path 85 | total_timesteps 1291.
Path 86 | total_timesteps 1304.
Path 87 | total_timesteps 1338.
Path 88 | total_timesteps 1354.
Path 89 | total_timesteps 1364.
Path 90 | total_timesteps 1374.
Path 91 | total_timesteps 1387.
Path 92 | total_timesteps 1401.
Path 93 | total_timesteps 1413.
Path 94 | total_timesteps 1428.
Path 95 | total_timesteps 1439.
Path 96 | total_timesteps 1450.
Path 97 | total_timesteps 1461.
Path 98 | total_timesteps 1476.
Path 99 | total_timesteps 1485.
Path 100 | total_timesteps 1498.
Path 101 | total_timesteps 1510.
Path 102 | total_timesteps 1526.
Path 103 | total_timesteps 1541.
Path 104 | total_timesteps 1548.
Path 105 | total_timesteps 1557.
Path 106 | total_timesteps 1569.
Path 107 | total_timesteps 1598.
Path 108 | total_timesteps 1612.
Path 109 | total_timesteps 1634.
Path 110 | total_timesteps 1644.
Path 111 | total_timesteps 1657.
Path 112 | total_timesteps 1673.
Path 113 | total_timesteps 1683.
Path 114 | total_timesteps 1695.
Path 115 | total_timesteps 1707.
Path 116 | total_timesteps 1724.
Path 117 | total_timesteps 1740.
Path 118 | total_timesteps 1753.
Path 119 | total_timesteps 1767.
Path 120 | total_timesteps 1778.
Path 121 | total_timesteps 1788.
Path 122 | total_timesteps 1800.
Path 123 | total_timesteps 1812.
Path 124 | total_timesteps 1824.
Path 125 | total_timesteps 1834.
Path 126 | total_timesteps 1846.
Path 127 | total_timesteps 1865.
Path 128 | total_timesteps 1878.
Path 129 | total_timesteps 1891.
Path 130 | total_timesteps 1904.
Path 131 | total_timesteps 1916.
Path 132 | total_timesteps 1931.
Path 133 | total_timesteps 1941.
Path 134 | total_timesteps 1970.
Path 135 | total_timesteps 1980.
Path 136 | total_timesteps 1991.
Path 137 | total_timesteps 2008.
Path 138 | total_timesteps 2019.
Path 139 | total_timesteps 2032.
Path 140 | total_timesteps 2049.
Path 141 | total_timesteps 2066.
Path 142 | total_timesteps 2079.
Path 143 | total_timesteps 2100.
Path 144 | total_timesteps 2117.
Path 145 | total_timesteps 2132.
Path 146 | total_timesteps 2144.
Path 147 | total_timesteps 2165.
Path 148 | total_timesteps 2178.
Path 149 | total_timesteps 2194.
Path 150 | total_timesteps 2212.
Path 151 | total_timesteps 2221.
Path 152 | total_timesteps 2233.
Path 153 | total_timesteps 2254.
Path 154 | total_timesteps 2268.
Path 155 | total_timesteps 2280.
Path 156 | total_timesteps 2289.
Path 157 | total_timesteps 2304.
Path 158 | total_timesteps 2331.
Path 159 | total_timesteps 2342.
Path 160 | total_timesteps 2368.
Path 161 | total_timesteps 2380.
Path 162 | total_timesteps 2397.
Path 163 | total_timesteps 2405.
Path 164 | total_timesteps 2417.
Path 165 | total_timesteps 2435.
Path 166 | total_timesteps 2453.
Path 167 | total_timesteps 2478.
Path 168 | total_timesteps 2487.
Path 169 | total_timesteps 2498.
Path 170 | total_timesteps 2510.
Path 171 | total_timesteps 2518.
Path 172 | total_timesteps 2539.
Path 173 | total_timesteps 2553.
Path 174 | total_timesteps 2567.
Path 175 | total_timesteps 2583.
Path 176 | total_timesteps 2591.
Path 177 | total_timesteps 2601.
Path 178 | total_timesteps 2617.
Path 179 | total_timesteps 2626.
Path 180 | total_timesteps 2635.
Path 181 | total_timesteps 2655.
Path 182 | total_timesteps 2676.
Path 183 | total_timesteps 2690.
Path 184 | total_timesteps 2708.
Path 185 | total_timesteps 2723.
Path 186 | total_timesteps 2735.
Path 187 | total_timesteps 2755.
Path 188 | total_timesteps 2762.
Path 189 | total_timesteps 2772.
Path 190 | total_timesteps 2797.
Path 191 | total_timesteps 2811.
Path 192 | total_timesteps 2825.
Path 193 | total_timesteps 2843.
Path 194 | total_timesteps 2854.
Path 195 | total_timesteps 2866.
Path 196 | total_timesteps 2876.
Path 197 | total_timesteps 2889.
Path 198 | total_timesteps 2902.
Path 199 | total_timesteps 2915.
Path 200 | total_timesteps 2930.
Path 201 | total_timesteps 2945.
Path 202 | total_timesteps 2969.
Path 203 | total_timesteps 2981.
Path 204 | total_timesteps 2996.
Path 205 | total_timesteps 3012.
Path 206 | total_timesteps 3024.
Path 207 | total_timesteps 3037.
Path 208 | total_timesteps 3049.
Path 209 | total_timesteps 3063.
Path 210 | total_timesteps 3074.
Path 211 | total_timesteps 3087.
Path 212 | total_timesteps 3099.
Path 213 | total_timesteps 3107.
Path 214 | total_timesteps 3118.
Path 215 | total_timesteps 3128.
Path 216 | total_timesteps 3142.
Path 217 | total_timesteps 3152.
Path 218 | total_timesteps 3163.
Path 219 | total_timesteps 3172.
Path 220 | total_timesteps 3190.
Path 221 | total_timesteps 3201.
Path 222 | total_timesteps 3216.
Path 223 | total_timesteps 3227.
Path 224 | total_timesteps 3238.
Path 225 | total_timesteps 3252.
Path 226 | total_timesteps 3269.
Path 227 | total_timesteps 3283.
Path 228 | total_timesteps 3292.
Path 229 | total_timesteps 3303.
Path 230 | total_timesteps 3322.
Path 231 | total_timesteps 3330.
Path 232 | total_timesteps 3341.
Path 233 | total_timesteps 3359.
Path 234 | total_timesteps 3378.
Path 235 | total_timesteps 3388.
Path 236 | total_timesteps 3399.
Path 237 | total_timesteps 3412.
Path 238 | total_timesteps 3427.
Path 239 | total_timesteps 3463.
Path 240 | total_timesteps 3477.
Path 241 | total_timesteps 3487.
Path 242 | total_timesteps 3499.
Path 243 | total_timesteps 3512.
Path 244 | total_timesteps 3520.
Path 245 | total_timesteps 3538.
Path 246 | total_timesteps 3554.
Path 247 | total_timesteps 3572.
Path 248 | total_timesteps 3595.
Path 249 | total_timesteps 3606.
Path 250 | total_timesteps 3622.
Path 251 | total_timesteps 3638.
Path 252 | total_timesteps 3648.
Path 253 | total_timesteps 3659.
Path 254 | total_timesteps 3668.
Path 255 | total_timesteps 3678.
Path 256 | total_timesteps 3691.
Path 257 | total_timesteps 3703.
Path 258 | total_timesteps 3713.
Path 259 | total_timesteps 3725.
Path 260 | total_timesteps 3737.
Path 261 | total_timesteps 3754.
Path 262 | total_timesteps 3766.
Path 263 | total_timesteps 3781.
Path 264 | total_timesteps 3792.
Path 265 | total_timesteps 3804.
Path 266 | total_timesteps 3812.
Path 267 | total_timesteps 3823.
Path 268 | total_timesteps 3842.
Path 269 | total_timesteps 3858.
Path 270 | total_timesteps 3865.
Path 271 | total_timesteps 3889.
Path 272 | total_timesteps 3915.
Path 273 | total_timesteps 3927.
Path 274 | total_timesteps 3948.
Path 275 | total_timesteps 3962.
Path 276 | total_timesteps 3973.
Path 277 | total_timesteps 3986.
Path 278 | total_timesteps 3999.
Path 279 | total_timesteps 4013.
Path 280 | total_timesteps 4020.
Path 281 | total_timesteps 4028.
Path 282 | total_timesteps 4037.
Path 283 | total_timesteps 4051.
Path 284 | total_timesteps 4063.
Path 285 | total_timesteps 4078.
Path 286 | total_timesteps 4090.
Path 287 | total_timesteps 4106.
Path 288 | total_timesteps 4118.
Path 289 | total_timesteps 4133.
Path 290 | total_timesteps 4145.
Path 291 | total_timesteps 4155.
Path 292 | total_timesteps 4167.
Path 293 | total_timesteps 4189.
Path 294 | total_timesteps 4204.
Path 295 | total_timesteps 4218.
Path 296 | total_timesteps 4232.
Path 297 | total_timesteps 4248.
Path 298 | total_timesteps 4260.
Path 299 | total_timesteps 4273.
Path 300 | total_timesteps 4299.
Path 301 | total_timesteps 4311.
Path 302 | total_timesteps 4327.
Path 303 | total_timesteps 4335.
Path 304 | total_timesteps 4345.
Path 305 | total_timesteps 4357.
Path 306 | total_timesteps 4368.
Path 307 | total_timesteps 4382.
Path 308 | total_timesteps 4392.
Path 309 | total_timesteps 4407.
Path 310 | total_timesteps 4420.
Path 311 | total_timesteps 4435.
Path 312 | total_timesteps 4444.
Path 313 | total_timesteps 4456.
Path 314 | total_timesteps 4467.
Path 315 | total_timesteps 4479.
Path 316 | total_timesteps 4488.
Path 317 | total_timesteps 4503.
Path 318 | total_timesteps 4521.
Path 319 | total_timesteps 4532.
Path 320 | total_timesteps 4559.
Path 321 | total_timesteps 4567.
Path 322 | total_timesteps 4575.
Path 323 | total_timesteps 4595.
Path 324 | total_timesteps 4616.
Path 325 | total_timesteps 4631.
Path 326 | total_timesteps 4642.
Path 327 | total_timesteps 4652.
Path 328 | total_timesteps 4666.
Path 329 | total_timesteps 4680.
Path 330 | total_timesteps 4690.
Path 331 | total_timesteps 4705.
Path 332 | total_timesteps 4712.
Path 333 | total_timesteps 4723.
Path 334 | total_timesteps 4739.
Path 335 | total_timesteps 4750.
Path 336 | total_timesteps 4763.
Path 337 | total_timesteps 4779.
Path 338 | total_timesteps 4787.
Path 339 | total_timesteps 4803.
Path 340 | total_timesteps 4813.
Path 341 | total_timesteps 4827.
Path 342 | total_timesteps 4846.
Path 343 | total_timesteps 4860.
Path 344 | total_timesteps 4877.
Path 345 | total_timesteps 4889.
Path 346 | total_timesteps 4904.
Path 347 | total_timesteps 4915.
Path 348 | total_timesteps 4927.
Path 349 | total_timesteps 4938.
Path 350 | total_timesteps 4947.
Path 351 | total_timesteps 4963.
Path 352 | total_timesteps 4983.
Path 353 | total_timesteps 5019.
Path 354 | total_timesteps 5027.
Path 355 | total_timesteps 5041.
Path 356 | total_timesteps 5060.
Path 357 | total_timesteps 5075.
Path 358 | total_timesteps 5084.
Path 359 | total_timesteps 5098.
Path 360 | total_timesteps 5116.
Path 361 | total_timesteps 5127.
Path 362 | total_timesteps 5143.
Path 363 | total_timesteps 5155.
Path 364 | total_timesteps 5169.
Path 365 | total_timesteps 5179.
Path 366 | total_timesteps 5191.
Path 367 | total_timesteps 5206.
Path 368 | total_timesteps 5220.
Path 369 | total_timesteps 5229.
Path 370 | total_timesteps 5254.
Path 371 | total_timesteps 5262.
Path 372 | total_timesteps 5276.
Path 373 | total_timesteps 5295.
Path 374 | total_timesteps 5310.
Path 375 | total_timesteps 5332.
Path 376 | total_timesteps 5364.
Path 377 | total_timesteps 5374.
Path 378 | total_timesteps 5391.
Path 379 | total_timesteps 5404.
Path 380 | total_timesteps 5411.
Path 381 | total_timesteps 5424.
Path 382 | total_timesteps 5435.
Path 383 | total_timesteps 5450.
Path 384 | total_timesteps 5464.
Path 385 | total_timesteps 5474.
Path 386 | total_timesteps 5482.
Path 387 | total_timesteps 5497.
Path 388 | total_timesteps 5508.
Path 389 | total_timesteps 5520.
Path 390 | total_timesteps 5528.
Path 391 | total_timesteps 5542.
Path 392 | total_timesteps 5561.
Path 393 | total_timesteps 5572.
Path 394 | total_timesteps 5581.
Path 395 | total_timesteps 5591.
Path 396 | total_timesteps 5602.
Path 397 | total_timesteps 5617.
Path 398 | total_timesteps 5632.
Path 399 | total_timesteps 5643.
Path 400 | total_timesteps 5658.
Path 401 | total_timesteps 5674.
Path 402 | total_timesteps 5687.
Path 403 | total_timesteps 5702.
Path 404 | total_timesteps 5714.
Path 405 | total_timesteps 5726.
Path 406 | total_timesteps 5738.
Path 407 | total_timesteps 5750.
Path 408 | total_timesteps 5769.
Path 409 | total_timesteps 5789.
Path 410 | total_timesteps 5806.
Path 411 | total_timesteps 5818.
Path 412 | total_timesteps 5835.
Path 413 | total_timesteps 5851.
Path 414 | total_timesteps 5865.
Path 415 | total_timesteps 5876.
Path 416 | total_timesteps 5889.
Path 417 | total_timesteps 5926.
Path 418 | total_timesteps 5943.
Path 419 | total_timesteps 5962.
Path 420 | total_timesteps 5976.
Path 421 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.76    |
| Iteration     | 12       |
| MaximumReturn | 5.92     |
| MinimumReturn | -22.5    |
| TotalSamples  | 56093    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.006505037657916546
Validation loss = 0.006575504783540964
Validation loss = 0.006646319292485714
Validation loss = 0.006977472919970751
Validation loss = 0.006301498971879482
Validation loss = 0.006478612776845694
Validation loss = 0.0063192411325871944
Validation loss = 0.005890218075364828
Validation loss = 0.006235995329916477
Validation loss = 0.006616934202611446
Validation loss = 0.0055970698595047
Validation loss = 0.006457461975514889
Validation loss = 0.005316286813467741
Validation loss = 0.00791992712765932
Validation loss = 0.005867809988558292
Validation loss = 0.0057062432169914246
Validation loss = 0.00616026995703578
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 22.
Path 2 | total_timesteps 32.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 63.
Path 5 | total_timesteps 76.
Path 6 | total_timesteps 91.
Path 7 | total_timesteps 111.
Path 8 | total_timesteps 120.
Path 9 | total_timesteps 128.
Path 10 | total_timesteps 147.
Path 11 | total_timesteps 164.
Path 12 | total_timesteps 180.
Path 13 | total_timesteps 210.
Path 14 | total_timesteps 224.
Path 15 | total_timesteps 241.
Path 16 | total_timesteps 257.
Path 17 | total_timesteps 268.
Path 18 | total_timesteps 279.
Path 19 | total_timesteps 290.
Path 20 | total_timesteps 308.
Path 21 | total_timesteps 317.
Path 22 | total_timesteps 330.
Path 23 | total_timesteps 341.
Path 24 | total_timesteps 351.
Path 25 | total_timesteps 363.
Path 26 | total_timesteps 376.
Path 27 | total_timesteps 393.
Path 28 | total_timesteps 418.
Path 29 | total_timesteps 431.
Path 30 | total_timesteps 444.
Path 31 | total_timesteps 456.
Path 32 | total_timesteps 472.
Path 33 | total_timesteps 485.
Path 34 | total_timesteps 515.
Path 35 | total_timesteps 525.
Path 36 | total_timesteps 547.
Path 37 | total_timesteps 561.
Path 38 | total_timesteps 577.
Path 39 | total_timesteps 592.
Path 40 | total_timesteps 612.
Path 41 | total_timesteps 629.
Path 42 | total_timesteps 651.
Path 43 | total_timesteps 664.
Path 44 | total_timesteps 686.
Path 45 | total_timesteps 706.
Path 46 | total_timesteps 716.
Path 47 | total_timesteps 724.
Path 48 | total_timesteps 738.
Path 49 | total_timesteps 748.
Path 50 | total_timesteps 762.
Path 51 | total_timesteps 772.
Path 52 | total_timesteps 787.
Path 53 | total_timesteps 796.
Path 54 | total_timesteps 810.
Path 55 | total_timesteps 821.
Path 56 | total_timesteps 834.
Path 57 | total_timesteps 853.
Path 58 | total_timesteps 880.
Path 59 | total_timesteps 891.
Path 60 | total_timesteps 910.
Path 61 | total_timesteps 920.
Path 62 | total_timesteps 932.
Path 63 | total_timesteps 944.
Path 64 | total_timesteps 956.
Path 65 | total_timesteps 972.
Path 66 | total_timesteps 984.
Path 67 | total_timesteps 1010.
Path 68 | total_timesteps 1023.
Path 69 | total_timesteps 1037.
Path 70 | total_timesteps 1053.
Path 71 | total_timesteps 1066.
Path 72 | total_timesteps 1077.
Path 73 | total_timesteps 1087.
Path 74 | total_timesteps 1099.
Path 75 | total_timesteps 1109.
Path 76 | total_timesteps 1124.
Path 77 | total_timesteps 1136.
Path 78 | total_timesteps 1153.
Path 79 | total_timesteps 1168.
Path 80 | total_timesteps 1177.
Path 81 | total_timesteps 1195.
Path 82 | total_timesteps 1216.
Path 83 | total_timesteps 1228.
Path 84 | total_timesteps 1240.
Path 85 | total_timesteps 1256.
Path 86 | total_timesteps 1269.
Path 87 | total_timesteps 1281.
Path 88 | total_timesteps 1299.
Path 89 | total_timesteps 1320.
Path 90 | total_timesteps 1330.
Path 91 | total_timesteps 1346.
Path 92 | total_timesteps 1364.
Path 93 | total_timesteps 1379.
Path 94 | total_timesteps 1395.
Path 95 | total_timesteps 1419.
Path 96 | total_timesteps 1431.
Path 97 | total_timesteps 1442.
Path 98 | total_timesteps 1455.
Path 99 | total_timesteps 1468.
Path 100 | total_timesteps 1481.
Path 101 | total_timesteps 1492.
Path 102 | total_timesteps 1516.
Path 103 | total_timesteps 1532.
Path 104 | total_timesteps 1558.
Path 105 | total_timesteps 1569.
Path 106 | total_timesteps 1591.
Path 107 | total_timesteps 1605.
Path 108 | total_timesteps 1619.
Path 109 | total_timesteps 1639.
Path 110 | total_timesteps 1654.
Path 111 | total_timesteps 1687.
Path 112 | total_timesteps 1698.
Path 113 | total_timesteps 1712.
Path 114 | total_timesteps 1723.
Path 115 | total_timesteps 1739.
Path 116 | total_timesteps 1756.
Path 117 | total_timesteps 1771.
Path 118 | total_timesteps 1781.
Path 119 | total_timesteps 1797.
Path 120 | total_timesteps 1815.
Path 121 | total_timesteps 1827.
Path 122 | total_timesteps 1844.
Path 123 | total_timesteps 1856.
Path 124 | total_timesteps 1867.
Path 125 | total_timesteps 1879.
Path 126 | total_timesteps 1890.
Path 127 | total_timesteps 1905.
Path 128 | total_timesteps 1920.
Path 129 | total_timesteps 1935.
Path 130 | total_timesteps 1945.
Path 131 | total_timesteps 1955.
Path 132 | total_timesteps 1966.
Path 133 | total_timesteps 1985.
Path 134 | total_timesteps 1994.
Path 135 | total_timesteps 2006.
Path 136 | total_timesteps 2021.
Path 137 | total_timesteps 2035.
Path 138 | total_timesteps 2047.
Path 139 | total_timesteps 2062.
Path 140 | total_timesteps 2075.
Path 141 | total_timesteps 2092.
Path 142 | total_timesteps 2104.
Path 143 | total_timesteps 2113.
Path 144 | total_timesteps 2127.
Path 145 | total_timesteps 2137.
Path 146 | total_timesteps 2152.
Path 147 | total_timesteps 2165.
Path 148 | total_timesteps 2180.
Path 149 | total_timesteps 2193.
Path 150 | total_timesteps 2203.
Path 151 | total_timesteps 2222.
Path 152 | total_timesteps 2231.
Path 153 | total_timesteps 2242.
Path 154 | total_timesteps 2252.
Path 155 | total_timesteps 2262.
Path 156 | total_timesteps 2278.
Path 157 | total_timesteps 2291.
Path 158 | total_timesteps 2324.
Path 159 | total_timesteps 2336.
Path 160 | total_timesteps 2346.
Path 161 | total_timesteps 2360.
Path 162 | total_timesteps 2372.
Path 163 | total_timesteps 2384.
Path 164 | total_timesteps 2398.
Path 165 | total_timesteps 2413.
Path 166 | total_timesteps 2441.
Path 167 | total_timesteps 2456.
Path 168 | total_timesteps 2469.
Path 169 | total_timesteps 2481.
Path 170 | total_timesteps 2491.
Path 171 | total_timesteps 2511.
Path 172 | total_timesteps 2519.
Path 173 | total_timesteps 2530.
Path 174 | total_timesteps 2539.
Path 175 | total_timesteps 2550.
Path 176 | total_timesteps 2563.
Path 177 | total_timesteps 2579.
Path 178 | total_timesteps 2596.
Path 179 | total_timesteps 2616.
Path 180 | total_timesteps 2632.
Path 181 | total_timesteps 2652.
Path 182 | total_timesteps 2658.
Path 183 | total_timesteps 2668.
Path 184 | total_timesteps 2684.
Path 185 | total_timesteps 2698.
Path 186 | total_timesteps 2719.
Path 187 | total_timesteps 2730.
Path 188 | total_timesteps 2738.
Path 189 | total_timesteps 2749.
Path 190 | total_timesteps 2762.
Path 191 | total_timesteps 2779.
Path 192 | total_timesteps 2787.
Path 193 | total_timesteps 2805.
Path 194 | total_timesteps 2815.
Path 195 | total_timesteps 2824.
Path 196 | total_timesteps 2845.
Path 197 | total_timesteps 2857.
Path 198 | total_timesteps 2865.
Path 199 | total_timesteps 2878.
Path 200 | total_timesteps 2893.
Path 201 | total_timesteps 2912.
Path 202 | total_timesteps 2933.
Path 203 | total_timesteps 2949.
Path 204 | total_timesteps 2963.
Path 205 | total_timesteps 2978.
Path 206 | total_timesteps 2991.
Path 207 | total_timesteps 3009.
Path 208 | total_timesteps 3027.
Path 209 | total_timesteps 3036.
Path 210 | total_timesteps 3052.
Path 211 | total_timesteps 3063.
Path 212 | total_timesteps 3076.
Path 213 | total_timesteps 3085.
Path 214 | total_timesteps 3099.
Path 215 | total_timesteps 3112.
Path 216 | total_timesteps 3125.
Path 217 | total_timesteps 3150.
Path 218 | total_timesteps 3170.
Path 219 | total_timesteps 3180.
Path 220 | total_timesteps 3188.
Path 221 | total_timesteps 3207.
Path 222 | total_timesteps 3222.
Path 223 | total_timesteps 3239.
Path 224 | total_timesteps 3254.
Path 225 | total_timesteps 3265.
Path 226 | total_timesteps 3294.
Path 227 | total_timesteps 3308.
Path 228 | total_timesteps 3326.
Path 229 | total_timesteps 3341.
Path 230 | total_timesteps 3352.
Path 231 | total_timesteps 3370.
Path 232 | total_timesteps 3380.
Path 233 | total_timesteps 3395.
Path 234 | total_timesteps 3409.
Path 235 | total_timesteps 3426.
Path 236 | total_timesteps 3435.
Path 237 | total_timesteps 3452.
Path 238 | total_timesteps 3470.
Path 239 | total_timesteps 3480.
Path 240 | total_timesteps 3488.
Path 241 | total_timesteps 3510.
Path 242 | total_timesteps 3522.
Path 243 | total_timesteps 3565.
Path 244 | total_timesteps 3573.
Path 245 | total_timesteps 3586.
Path 246 | total_timesteps 3597.
Path 247 | total_timesteps 3614.
Path 248 | total_timesteps 3624.
Path 249 | total_timesteps 3646.
Path 250 | total_timesteps 3663.
Path 251 | total_timesteps 3675.
Path 252 | total_timesteps 3689.
Path 253 | total_timesteps 3701.
Path 254 | total_timesteps 3712.
Path 255 | total_timesteps 3727.
Path 256 | total_timesteps 3737.
Path 257 | total_timesteps 3750.
Path 258 | total_timesteps 3765.
Path 259 | total_timesteps 3775.
Path 260 | total_timesteps 3784.
Path 261 | total_timesteps 3812.
Path 262 | total_timesteps 3827.
Path 263 | total_timesteps 3841.
Path 264 | total_timesteps 3851.
Path 265 | total_timesteps 3864.
Path 266 | total_timesteps 3882.
Path 267 | total_timesteps 3893.
Path 268 | total_timesteps 3917.
Path 269 | total_timesteps 3932.
Path 270 | total_timesteps 3942.
Path 271 | total_timesteps 3960.
Path 272 | total_timesteps 3974.
Path 273 | total_timesteps 3990.
Path 274 | total_timesteps 4000.
Path 275 | total_timesteps 4019.
Path 276 | total_timesteps 4036.
Path 277 | total_timesteps 4048.
Path 278 | total_timesteps 4065.
Path 279 | total_timesteps 4082.
Path 280 | total_timesteps 4100.
Path 281 | total_timesteps 4112.
Path 282 | total_timesteps 4120.
Path 283 | total_timesteps 4130.
Path 284 | total_timesteps 4142.
Path 285 | total_timesteps 4156.
Path 286 | total_timesteps 4169.
Path 287 | total_timesteps 4191.
Path 288 | total_timesteps 4202.
Path 289 | total_timesteps 4215.
Path 290 | total_timesteps 4224.
Path 291 | total_timesteps 4236.
Path 292 | total_timesteps 4271.
Path 293 | total_timesteps 4282.
Path 294 | total_timesteps 4298.
Path 295 | total_timesteps 4307.
Path 296 | total_timesteps 4317.
Path 297 | total_timesteps 4325.
Path 298 | total_timesteps 4340.
Path 299 | total_timesteps 4349.
Path 300 | total_timesteps 4362.
Path 301 | total_timesteps 4380.
Path 302 | total_timesteps 4392.
Path 303 | total_timesteps 4403.
Path 304 | total_timesteps 4417.
Path 305 | total_timesteps 4427.
Path 306 | total_timesteps 4438.
Path 307 | total_timesteps 4448.
Path 308 | total_timesteps 4461.
Path 309 | total_timesteps 4472.
Path 310 | total_timesteps 4483.
Path 311 | total_timesteps 4495.
Path 312 | total_timesteps 4506.
Path 313 | total_timesteps 4520.
Path 314 | total_timesteps 4539.
Path 315 | total_timesteps 4556.
Path 316 | total_timesteps 4593.
Path 317 | total_timesteps 4604.
Path 318 | total_timesteps 4615.
Path 319 | total_timesteps 4636.
Path 320 | total_timesteps 4647.
Path 321 | total_timesteps 4656.
Path 322 | total_timesteps 4669.
Path 323 | total_timesteps 4682.
Path 324 | total_timesteps 4708.
Path 325 | total_timesteps 4727.
Path 326 | total_timesteps 4740.
Path 327 | total_timesteps 4752.
Path 328 | total_timesteps 4766.
Path 329 | total_timesteps 4776.
Path 330 | total_timesteps 4804.
Path 331 | total_timesteps 4819.
Path 332 | total_timesteps 4834.
Path 333 | total_timesteps 4853.
Path 334 | total_timesteps 4868.
Path 335 | total_timesteps 4886.
Path 336 | total_timesteps 4897.
Path 337 | total_timesteps 4916.
Path 338 | total_timesteps 4931.
Path 339 | total_timesteps 4943.
Path 340 | total_timesteps 4953.
Path 341 | total_timesteps 4966.
Path 342 | total_timesteps 4978.
Path 343 | total_timesteps 4991.
Path 344 | total_timesteps 5018.
Path 345 | total_timesteps 5032.
Path 346 | total_timesteps 5041.
Path 347 | total_timesteps 5052.
Path 348 | total_timesteps 5062.
Path 349 | total_timesteps 5075.
Path 350 | total_timesteps 5091.
Path 351 | total_timesteps 5104.
Path 352 | total_timesteps 5123.
Path 353 | total_timesteps 5138.
Path 354 | total_timesteps 5151.
Path 355 | total_timesteps 5159.
Path 356 | total_timesteps 5176.
Path 357 | total_timesteps 5208.
Path 358 | total_timesteps 5218.
Path 359 | total_timesteps 5234.
Path 360 | total_timesteps 5247.
Path 361 | total_timesteps 5262.
Path 362 | total_timesteps 5273.
Path 363 | total_timesteps 5288.
Path 364 | total_timesteps 5303.
Path 365 | total_timesteps 5314.
Path 366 | total_timesteps 5326.
Path 367 | total_timesteps 5348.
Path 368 | total_timesteps 5363.
Path 369 | total_timesteps 5373.
Path 370 | total_timesteps 5393.
Path 371 | total_timesteps 5406.
Path 372 | total_timesteps 5418.
Path 373 | total_timesteps 5428.
Path 374 | total_timesteps 5442.
Path 375 | total_timesteps 5456.
Path 376 | total_timesteps 5464.
Path 377 | total_timesteps 5472.
Path 378 | total_timesteps 5496.
Path 379 | total_timesteps 5506.
Path 380 | total_timesteps 5528.
Path 381 | total_timesteps 5539.
Path 382 | total_timesteps 5551.
Path 383 | total_timesteps 5565.
Path 384 | total_timesteps 5576.
Path 385 | total_timesteps 5590.
Path 386 | total_timesteps 5614.
Path 387 | total_timesteps 5629.
Path 388 | total_timesteps 5658.
Path 389 | total_timesteps 5670.
Path 390 | total_timesteps 5677.
Path 391 | total_timesteps 5691.
Path 392 | total_timesteps 5706.
Path 393 | total_timesteps 5719.
Path 394 | total_timesteps 5734.
Path 395 | total_timesteps 5751.
Path 396 | total_timesteps 5760.
Path 397 | total_timesteps 5772.
Path 398 | total_timesteps 5785.
Path 399 | total_timesteps 5797.
Path 400 | total_timesteps 5812.
Path 401 | total_timesteps 5827.
Path 402 | total_timesteps 5845.
Path 403 | total_timesteps 5861.
Path 404 | total_timesteps 5874.
Path 405 | total_timesteps 5890.
Path 406 | total_timesteps 5907.
Path 407 | total_timesteps 5921.
Path 408 | total_timesteps 5929.
Path 409 | total_timesteps 5943.
Path 410 | total_timesteps 5956.
Path 411 | total_timesteps 5964.
Path 412 | total_timesteps 5978.
Path 413 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.37    |
| Iteration     | 13       |
| MaximumReturn | 1.58     |
| MinimumReturn | -20.7    |
| TotalSamples  | 60093    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.005486440844833851
Validation loss = 0.005498932208865881
Validation loss = 0.005196590442210436
Validation loss = 0.005656161345541477
Validation loss = 0.0053756651468575
Validation loss = 0.00562941562384367
Validation loss = 0.004815451800823212
Validation loss = 0.004941262304782867
Validation loss = 0.004919265862554312
Validation loss = 0.0048859259113669395
Validation loss = 0.004778384231030941
Validation loss = 0.005101235583424568
Validation loss = 0.005040663760155439
Validation loss = 0.005108701530843973
Validation loss = 0.005162374582141638
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 44.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 62.
Path 6 | total_timesteps 74.
Path 7 | total_timesteps 87.
Path 8 | total_timesteps 106.
Path 9 | total_timesteps 118.
Path 10 | total_timesteps 130.
Path 11 | total_timesteps 146.
Path 12 | total_timesteps 169.
Path 13 | total_timesteps 194.
Path 14 | total_timesteps 211.
Path 15 | total_timesteps 228.
Path 16 | total_timesteps 242.
Path 17 | total_timesteps 266.
Path 18 | total_timesteps 283.
Path 19 | total_timesteps 294.
Path 20 | total_timesteps 305.
Path 21 | total_timesteps 323.
Path 22 | total_timesteps 341.
Path 23 | total_timesteps 356.
Path 24 | total_timesteps 376.
Path 25 | total_timesteps 389.
Path 26 | total_timesteps 400.
Path 27 | total_timesteps 415.
Path 28 | total_timesteps 432.
Path 29 | total_timesteps 440.
Path 30 | total_timesteps 473.
Path 31 | total_timesteps 495.
Path 32 | total_timesteps 518.
Path 33 | total_timesteps 525.
Path 34 | total_timesteps 547.
Path 35 | total_timesteps 558.
Path 36 | total_timesteps 568.
Path 37 | total_timesteps 581.
Path 38 | total_timesteps 590.
Path 39 | total_timesteps 615.
Path 40 | total_timesteps 626.
Path 41 | total_timesteps 637.
Path 42 | total_timesteps 652.
Path 43 | total_timesteps 663.
Path 44 | total_timesteps 679.
Path 45 | total_timesteps 697.
Path 46 | total_timesteps 715.
Path 47 | total_timesteps 724.
Path 48 | total_timesteps 744.
Path 49 | total_timesteps 760.
Path 50 | total_timesteps 777.
Path 51 | total_timesteps 786.
Path 52 | total_timesteps 799.
Path 53 | total_timesteps 809.
Path 54 | total_timesteps 818.
Path 55 | total_timesteps 829.
Path 56 | total_timesteps 842.
Path 57 | total_timesteps 864.
Path 58 | total_timesteps 885.
Path 59 | total_timesteps 901.
Path 60 | total_timesteps 916.
Path 61 | total_timesteps 927.
Path 62 | total_timesteps 944.
Path 63 | total_timesteps 956.
Path 64 | total_timesteps 968.
Path 65 | total_timesteps 988.
Path 66 | total_timesteps 1004.
Path 67 | total_timesteps 1019.
Path 68 | total_timesteps 1030.
Path 69 | total_timesteps 1041.
Path 70 | total_timesteps 1061.
Path 71 | total_timesteps 1077.
Path 72 | total_timesteps 1093.
Path 73 | total_timesteps 1108.
Path 74 | total_timesteps 1126.
Path 75 | total_timesteps 1137.
Path 76 | total_timesteps 1147.
Path 77 | total_timesteps 1158.
Path 78 | total_timesteps 1178.
Path 79 | total_timesteps 1197.
Path 80 | total_timesteps 1219.
Path 81 | total_timesteps 1235.
Path 82 | total_timesteps 1257.
Path 83 | total_timesteps 1268.
Path 84 | total_timesteps 1277.
Path 85 | total_timesteps 1291.
Path 86 | total_timesteps 1299.
Path 87 | total_timesteps 1318.
Path 88 | total_timesteps 1327.
Path 89 | total_timesteps 1336.
Path 90 | total_timesteps 1348.
Path 91 | total_timesteps 1361.
Path 92 | total_timesteps 1378.
Path 93 | total_timesteps 1394.
Path 94 | total_timesteps 1411.
Path 95 | total_timesteps 1427.
Path 96 | total_timesteps 1435.
Path 97 | total_timesteps 1452.
Path 98 | total_timesteps 1464.
Path 99 | total_timesteps 1477.
Path 100 | total_timesteps 1489.
Path 101 | total_timesteps 1502.
Path 102 | total_timesteps 1520.
Path 103 | total_timesteps 1540.
Path 104 | total_timesteps 1549.
Path 105 | total_timesteps 1575.
Path 106 | total_timesteps 1593.
Path 107 | total_timesteps 1601.
Path 108 | total_timesteps 1612.
Path 109 | total_timesteps 1624.
Path 110 | total_timesteps 1633.
Path 111 | total_timesteps 1644.
Path 112 | total_timesteps 1658.
Path 113 | total_timesteps 1668.
Path 114 | total_timesteps 1680.
Path 115 | total_timesteps 1699.
Path 116 | total_timesteps 1709.
Path 117 | total_timesteps 1723.
Path 118 | total_timesteps 1734.
Path 119 | total_timesteps 1750.
Path 120 | total_timesteps 1757.
Path 121 | total_timesteps 1768.
Path 122 | total_timesteps 1778.
Path 123 | total_timesteps 1798.
Path 124 | total_timesteps 1807.
Path 125 | total_timesteps 1825.
Path 126 | total_timesteps 1847.
Path 127 | total_timesteps 1854.
Path 128 | total_timesteps 1874.
Path 129 | total_timesteps 1888.
Path 130 | total_timesteps 1911.
Path 131 | total_timesteps 1927.
Path 132 | total_timesteps 1938.
Path 133 | total_timesteps 1952.
Path 134 | total_timesteps 1964.
Path 135 | total_timesteps 1974.
Path 136 | total_timesteps 1987.
Path 137 | total_timesteps 1997.
Path 138 | total_timesteps 2015.
Path 139 | total_timesteps 2030.
Path 140 | total_timesteps 2039.
Path 141 | total_timesteps 2053.
Path 142 | total_timesteps 2068.
Path 143 | total_timesteps 2079.
Path 144 | total_timesteps 2094.
Path 145 | total_timesteps 2104.
Path 146 | total_timesteps 2114.
Path 147 | total_timesteps 2125.
Path 148 | total_timesteps 2143.
Path 149 | total_timesteps 2162.
Path 150 | total_timesteps 2173.
Path 151 | total_timesteps 2188.
Path 152 | total_timesteps 2199.
Path 153 | total_timesteps 2218.
Path 154 | total_timesteps 2226.
Path 155 | total_timesteps 2237.
Path 156 | total_timesteps 2250.
Path 157 | total_timesteps 2276.
Path 158 | total_timesteps 2289.
Path 159 | total_timesteps 2304.
Path 160 | total_timesteps 2320.
Path 161 | total_timesteps 2336.
Path 162 | total_timesteps 2347.
Path 163 | total_timesteps 2357.
Path 164 | total_timesteps 2371.
Path 165 | total_timesteps 2396.
Path 166 | total_timesteps 2409.
Path 167 | total_timesteps 2426.
Path 168 | total_timesteps 2437.
Path 169 | total_timesteps 2456.
Path 170 | total_timesteps 2470.
Path 171 | total_timesteps 2484.
Path 172 | total_timesteps 2496.
Path 173 | total_timesteps 2507.
Path 174 | total_timesteps 2517.
Path 175 | total_timesteps 2537.
Path 176 | total_timesteps 2549.
Path 177 | total_timesteps 2559.
Path 178 | total_timesteps 2570.
Path 179 | total_timesteps 2583.
Path 180 | total_timesteps 2596.
Path 181 | total_timesteps 2608.
Path 182 | total_timesteps 2621.
Path 183 | total_timesteps 2632.
Path 184 | total_timesteps 2641.
Path 185 | total_timesteps 2658.
Path 186 | total_timesteps 2671.
Path 187 | total_timesteps 2692.
Path 188 | total_timesteps 2712.
Path 189 | total_timesteps 2725.
Path 190 | total_timesteps 2743.
Path 191 | total_timesteps 2772.
Path 192 | total_timesteps 2783.
Path 193 | total_timesteps 2791.
Path 194 | total_timesteps 2821.
Path 195 | total_timesteps 2836.
Path 196 | total_timesteps 2855.
Path 197 | total_timesteps 2868.
Path 198 | total_timesteps 2881.
Path 199 | total_timesteps 2899.
Path 200 | total_timesteps 2922.
Path 201 | total_timesteps 2940.
Path 202 | total_timesteps 2964.
Path 203 | total_timesteps 3001.
Path 204 | total_timesteps 3021.
Path 205 | total_timesteps 3033.
Path 206 | total_timesteps 3046.
Path 207 | total_timesteps 3066.
Path 208 | total_timesteps 3078.
Path 209 | total_timesteps 3088.
Path 210 | total_timesteps 3104.
Path 211 | total_timesteps 3117.
Path 212 | total_timesteps 3127.
Path 213 | total_timesteps 3138.
Path 214 | total_timesteps 3156.
Path 215 | total_timesteps 3170.
Path 216 | total_timesteps 3181.
Path 217 | total_timesteps 3198.
Path 218 | total_timesteps 3212.
Path 219 | total_timesteps 3223.
Path 220 | total_timesteps 3235.
Path 221 | total_timesteps 3254.
Path 222 | total_timesteps 3268.
Path 223 | total_timesteps 3283.
Path 224 | total_timesteps 3299.
Path 225 | total_timesteps 3319.
Path 226 | total_timesteps 3334.
Path 227 | total_timesteps 3354.
Path 228 | total_timesteps 3368.
Path 229 | total_timesteps 3377.
Path 230 | total_timesteps 3392.
Path 231 | total_timesteps 3406.
Path 232 | total_timesteps 3415.
Path 233 | total_timesteps 3429.
Path 234 | total_timesteps 3443.
Path 235 | total_timesteps 3466.
Path 236 | total_timesteps 3473.
Path 237 | total_timesteps 3484.
Path 238 | total_timesteps 3498.
Path 239 | total_timesteps 3508.
Path 240 | total_timesteps 3527.
Path 241 | total_timesteps 3543.
Path 242 | total_timesteps 3563.
Path 243 | total_timesteps 3574.
Path 244 | total_timesteps 3584.
Path 245 | total_timesteps 3599.
Path 246 | total_timesteps 3610.
Path 247 | total_timesteps 3621.
Path 248 | total_timesteps 3634.
Path 249 | total_timesteps 3643.
Path 250 | total_timesteps 3657.
Path 251 | total_timesteps 3680.
Path 252 | total_timesteps 3689.
Path 253 | total_timesteps 3700.
Path 254 | total_timesteps 3712.
Path 255 | total_timesteps 3724.
Path 256 | total_timesteps 3735.
Path 257 | total_timesteps 3747.
Path 258 | total_timesteps 3761.
Path 259 | total_timesteps 3780.
Path 260 | total_timesteps 3802.
Path 261 | total_timesteps 3817.
Path 262 | total_timesteps 3831.
Path 263 | total_timesteps 3838.
Path 264 | total_timesteps 3847.
Path 265 | total_timesteps 3857.
Path 266 | total_timesteps 3872.
Path 267 | total_timesteps 3887.
Path 268 | total_timesteps 3895.
Path 269 | total_timesteps 3906.
Path 270 | total_timesteps 3916.
Path 271 | total_timesteps 3936.
Path 272 | total_timesteps 3947.
Path 273 | total_timesteps 3962.
Path 274 | total_timesteps 3982.
Path 275 | total_timesteps 4002.
Path 276 | total_timesteps 4017.
Path 277 | total_timesteps 4036.
Path 278 | total_timesteps 4048.
Path 279 | total_timesteps 4059.
Path 280 | total_timesteps 4069.
Path 281 | total_timesteps 4086.
Path 282 | total_timesteps 4103.
Path 283 | total_timesteps 4113.
Path 284 | total_timesteps 4124.
Path 285 | total_timesteps 4143.
Path 286 | total_timesteps 4159.
Path 287 | total_timesteps 4171.
Path 288 | total_timesteps 4182.
Path 289 | total_timesteps 4202.
Path 290 | total_timesteps 4216.
Path 291 | total_timesteps 4236.
Path 292 | total_timesteps 4246.
Path 293 | total_timesteps 4270.
Path 294 | total_timesteps 4285.
Path 295 | total_timesteps 4302.
Path 296 | total_timesteps 4316.
Path 297 | total_timesteps 4326.
Path 298 | total_timesteps 4353.
Path 299 | total_timesteps 4363.
Path 300 | total_timesteps 4375.
Path 301 | total_timesteps 4397.
Path 302 | total_timesteps 4413.
Path 303 | total_timesteps 4429.
Path 304 | total_timesteps 4441.
Path 305 | total_timesteps 4462.
Path 306 | total_timesteps 4480.
Path 307 | total_timesteps 4494.
Path 308 | total_timesteps 4505.
Path 309 | total_timesteps 4526.
Path 310 | total_timesteps 4540.
Path 311 | total_timesteps 4553.
Path 312 | total_timesteps 4579.
Path 313 | total_timesteps 4588.
Path 314 | total_timesteps 4605.
Path 315 | total_timesteps 4616.
Path 316 | total_timesteps 4633.
Path 317 | total_timesteps 4658.
Path 318 | total_timesteps 4668.
Path 319 | total_timesteps 4675.
Path 320 | total_timesteps 4691.
Path 321 | total_timesteps 4714.
Path 322 | total_timesteps 4738.
Path 323 | total_timesteps 4747.
Path 324 | total_timesteps 4762.
Path 325 | total_timesteps 4787.
Path 326 | total_timesteps 4798.
Path 327 | total_timesteps 4817.
Path 328 | total_timesteps 4836.
Path 329 | total_timesteps 4848.
Path 330 | total_timesteps 4866.
Path 331 | total_timesteps 4876.
Path 332 | total_timesteps 4899.
Path 333 | total_timesteps 4911.
Path 334 | total_timesteps 4926.
Path 335 | total_timesteps 4943.
Path 336 | total_timesteps 4957.
Path 337 | total_timesteps 4972.
Path 338 | total_timesteps 4992.
Path 339 | total_timesteps 5007.
Path 340 | total_timesteps 5022.
Path 341 | total_timesteps 5038.
Path 342 | total_timesteps 5052.
Path 343 | total_timesteps 5065.
Path 344 | total_timesteps 5080.
Path 345 | total_timesteps 5092.
Path 346 | total_timesteps 5109.
Path 347 | total_timesteps 5122.
Path 348 | total_timesteps 5137.
Path 349 | total_timesteps 5153.
Path 350 | total_timesteps 5168.
Path 351 | total_timesteps 5183.
Path 352 | total_timesteps 5192.
Path 353 | total_timesteps 5202.
Path 354 | total_timesteps 5220.
Path 355 | total_timesteps 5236.
Path 356 | total_timesteps 5253.
Path 357 | total_timesteps 5264.
Path 358 | total_timesteps 5277.
Path 359 | total_timesteps 5290.
Path 360 | total_timesteps 5305.
Path 361 | total_timesteps 5321.
Path 362 | total_timesteps 5337.
Path 363 | total_timesteps 5372.
Path 364 | total_timesteps 5389.
Path 365 | total_timesteps 5400.
Path 366 | total_timesteps 5413.
Path 367 | total_timesteps 5441.
Path 368 | total_timesteps 5455.
Path 369 | total_timesteps 5466.
Path 370 | total_timesteps 5479.
Path 371 | total_timesteps 5504.
Path 372 | total_timesteps 5514.
Path 373 | total_timesteps 5526.
Path 374 | total_timesteps 5536.
Path 375 | total_timesteps 5544.
Path 376 | total_timesteps 5553.
Path 377 | total_timesteps 5573.
Path 378 | total_timesteps 5588.
Path 379 | total_timesteps 5614.
Path 380 | total_timesteps 5629.
Path 381 | total_timesteps 5640.
Path 382 | total_timesteps 5649.
Path 383 | total_timesteps 5660.
Path 384 | total_timesteps 5675.
Path 385 | total_timesteps 5687.
Path 386 | total_timesteps 5698.
Path 387 | total_timesteps 5719.
Path 388 | total_timesteps 5733.
Path 389 | total_timesteps 5744.
Path 390 | total_timesteps 5763.
Path 391 | total_timesteps 5779.
Path 392 | total_timesteps 5798.
Path 393 | total_timesteps 5814.
Path 394 | total_timesteps 5823.
Path 395 | total_timesteps 5836.
Path 396 | total_timesteps 5852.
Path 397 | total_timesteps 5865.
Path 398 | total_timesteps 5879.
Path 399 | total_timesteps 5888.
Path 400 | total_timesteps 5898.
Path 401 | total_timesteps 5917.
Path 402 | total_timesteps 5934.
Path 403 | total_timesteps 5943.
Path 404 | total_timesteps 5953.
Path 405 | total_timesteps 5977.
Path 406 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.37    |
| Iteration     | 14       |
| MaximumReturn | 0.714    |
| MinimumReturn | -23.2    |
| TotalSamples  | 64101    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.0050249965861439705
Validation loss = 0.0045824237167835236
Validation loss = 0.004338850267231464
Validation loss = 0.004402729216963053
Validation loss = 0.004493733402341604
Validation loss = 0.004603562876582146
Validation loss = 0.004672992043197155
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 76.
Path 7 | total_timesteps 88.
Path 8 | total_timesteps 105.
Path 9 | total_timesteps 119.
Path 10 | total_timesteps 132.
Path 11 | total_timesteps 144.
Path 12 | total_timesteps 157.
Path 13 | total_timesteps 166.
Path 14 | total_timesteps 184.
Path 15 | total_timesteps 214.
Path 16 | total_timesteps 228.
Path 17 | total_timesteps 239.
Path 18 | total_timesteps 260.
Path 19 | total_timesteps 271.
Path 20 | total_timesteps 287.
Path 21 | total_timesteps 299.
Path 22 | total_timesteps 308.
Path 23 | total_timesteps 324.
Path 24 | total_timesteps 335.
Path 25 | total_timesteps 343.
Path 26 | total_timesteps 361.
Path 27 | total_timesteps 372.
Path 28 | total_timesteps 383.
Path 29 | total_timesteps 403.
Path 30 | total_timesteps 415.
Path 31 | total_timesteps 430.
Path 32 | total_timesteps 443.
Path 33 | total_timesteps 461.
Path 34 | total_timesteps 476.
Path 35 | total_timesteps 504.
Path 36 | total_timesteps 519.
Path 37 | total_timesteps 533.
Path 38 | total_timesteps 545.
Path 39 | total_timesteps 552.
Path 40 | total_timesteps 566.
Path 41 | total_timesteps 576.
Path 42 | total_timesteps 588.
Path 43 | total_timesteps 598.
Path 44 | total_timesteps 610.
Path 45 | total_timesteps 621.
Path 46 | total_timesteps 632.
Path 47 | total_timesteps 644.
Path 48 | total_timesteps 663.
Path 49 | total_timesteps 673.
Path 50 | total_timesteps 690.
Path 51 | total_timesteps 704.
Path 52 | total_timesteps 715.
Path 53 | total_timesteps 726.
Path 54 | total_timesteps 749.
Path 55 | total_timesteps 759.
Path 56 | total_timesteps 781.
Path 57 | total_timesteps 794.
Path 58 | total_timesteps 816.
Path 59 | total_timesteps 832.
Path 60 | total_timesteps 852.
Path 61 | total_timesteps 871.
Path 62 | total_timesteps 883.
Path 63 | total_timesteps 903.
Path 64 | total_timesteps 918.
Path 65 | total_timesteps 929.
Path 66 | total_timesteps 945.
Path 67 | total_timesteps 961.
Path 68 | total_timesteps 977.
Path 69 | total_timesteps 988.
Path 70 | total_timesteps 1007.
Path 71 | total_timesteps 1027.
Path 72 | total_timesteps 1042.
Path 73 | total_timesteps 1061.
Path 74 | total_timesteps 1075.
Path 75 | total_timesteps 1086.
Path 76 | total_timesteps 1103.
Path 77 | total_timesteps 1110.
Path 78 | total_timesteps 1124.
Path 79 | total_timesteps 1142.
Path 80 | total_timesteps 1152.
Path 81 | total_timesteps 1163.
Path 82 | total_timesteps 1183.
Path 83 | total_timesteps 1195.
Path 84 | total_timesteps 1207.
Path 85 | total_timesteps 1224.
Path 86 | total_timesteps 1236.
Path 87 | total_timesteps 1247.
Path 88 | total_timesteps 1269.
Path 89 | total_timesteps 1291.
Path 90 | total_timesteps 1316.
Path 91 | total_timesteps 1336.
Path 92 | total_timesteps 1353.
Path 93 | total_timesteps 1366.
Path 94 | total_timesteps 1378.
Path 95 | total_timesteps 1387.
Path 96 | total_timesteps 1399.
Path 97 | total_timesteps 1407.
Path 98 | total_timesteps 1419.
Path 99 | total_timesteps 1433.
Path 100 | total_timesteps 1451.
Path 101 | total_timesteps 1473.
Path 102 | total_timesteps 1493.
Path 103 | total_timesteps 1503.
Path 104 | total_timesteps 1519.
Path 105 | total_timesteps 1532.
Path 106 | total_timesteps 1543.
Path 107 | total_timesteps 1563.
Path 108 | total_timesteps 1569.
Path 109 | total_timesteps 1588.
Path 110 | total_timesteps 1599.
Path 111 | total_timesteps 1618.
Path 112 | total_timesteps 1635.
Path 113 | total_timesteps 1645.
Path 114 | total_timesteps 1656.
Path 115 | total_timesteps 1667.
Path 116 | total_timesteps 1681.
Path 117 | total_timesteps 1692.
Path 118 | total_timesteps 1706.
Path 119 | total_timesteps 1727.
Path 120 | total_timesteps 1736.
Path 121 | total_timesteps 1748.
Path 122 | total_timesteps 1763.
Path 123 | total_timesteps 1776.
Path 124 | total_timesteps 1785.
Path 125 | total_timesteps 1800.
Path 126 | total_timesteps 1820.
Path 127 | total_timesteps 1832.
Path 128 | total_timesteps 1850.
Path 129 | total_timesteps 1872.
Path 130 | total_timesteps 1896.
Path 131 | total_timesteps 1904.
Path 132 | total_timesteps 1914.
Path 133 | total_timesteps 1932.
Path 134 | total_timesteps 1966.
Path 135 | total_timesteps 1976.
Path 136 | total_timesteps 1986.
Path 137 | total_timesteps 2003.
Path 138 | total_timesteps 2014.
Path 139 | total_timesteps 2024.
Path 140 | total_timesteps 2045.
Path 141 | total_timesteps 2067.
Path 142 | total_timesteps 2082.
Path 143 | total_timesteps 2095.
Path 144 | total_timesteps 2111.
Path 145 | total_timesteps 2127.
Path 146 | total_timesteps 2142.
Path 147 | total_timesteps 2158.
Path 148 | total_timesteps 2176.
Path 149 | total_timesteps 2187.
Path 150 | total_timesteps 2199.
Path 151 | total_timesteps 2209.
Path 152 | total_timesteps 2228.
Path 153 | total_timesteps 2238.
Path 154 | total_timesteps 2252.
Path 155 | total_timesteps 2266.
Path 156 | total_timesteps 2283.
Path 157 | total_timesteps 2303.
Path 158 | total_timesteps 2317.
Path 159 | total_timesteps 2326.
Path 160 | total_timesteps 2335.
Path 161 | total_timesteps 2347.
Path 162 | total_timesteps 2360.
Path 163 | total_timesteps 2369.
Path 164 | total_timesteps 2380.
Path 165 | total_timesteps 2394.
Path 166 | total_timesteps 2406.
Path 167 | total_timesteps 2421.
Path 168 | total_timesteps 2441.
Path 169 | total_timesteps 2450.
Path 170 | total_timesteps 2463.
Path 171 | total_timesteps 2483.
Path 172 | total_timesteps 2499.
Path 173 | total_timesteps 2517.
Path 174 | total_timesteps 2530.
Path 175 | total_timesteps 2558.
Path 176 | total_timesteps 2570.
Path 177 | total_timesteps 2589.
Path 178 | total_timesteps 2619.
Path 179 | total_timesteps 2632.
Path 180 | total_timesteps 2651.
Path 181 | total_timesteps 2660.
Path 182 | total_timesteps 2673.
Path 183 | total_timesteps 2681.
Path 184 | total_timesteps 2692.
Path 185 | total_timesteps 2713.
Path 186 | total_timesteps 2727.
Path 187 | total_timesteps 2740.
Path 188 | total_timesteps 2755.
Path 189 | total_timesteps 2767.
Path 190 | total_timesteps 2784.
Path 191 | total_timesteps 2802.
Path 192 | total_timesteps 2816.
Path 193 | total_timesteps 2828.
Path 194 | total_timesteps 2839.
Path 195 | total_timesteps 2853.
Path 196 | total_timesteps 2866.
Path 197 | total_timesteps 2883.
Path 198 | total_timesteps 2898.
Path 199 | total_timesteps 2915.
Path 200 | total_timesteps 2926.
Path 201 | total_timesteps 2947.
Path 202 | total_timesteps 2965.
Path 203 | total_timesteps 2985.
Path 204 | total_timesteps 2997.
Path 205 | total_timesteps 3006.
Path 206 | total_timesteps 3017.
Path 207 | total_timesteps 3026.
Path 208 | total_timesteps 3042.
Path 209 | total_timesteps 3048.
Path 210 | total_timesteps 3068.
Path 211 | total_timesteps 3087.
Path 212 | total_timesteps 3100.
Path 213 | total_timesteps 3113.
Path 214 | total_timesteps 3136.
Path 215 | total_timesteps 3147.
Path 216 | total_timesteps 3155.
Path 217 | total_timesteps 3173.
Path 218 | total_timesteps 3184.
Path 219 | total_timesteps 3194.
Path 220 | total_timesteps 3211.
Path 221 | total_timesteps 3224.
Path 222 | total_timesteps 3236.
Path 223 | total_timesteps 3249.
Path 224 | total_timesteps 3259.
Path 225 | total_timesteps 3273.
Path 226 | total_timesteps 3292.
Path 227 | total_timesteps 3309.
Path 228 | total_timesteps 3333.
Path 229 | total_timesteps 3348.
Path 230 | total_timesteps 3362.
Path 231 | total_timesteps 3377.
Path 232 | total_timesteps 3386.
Path 233 | total_timesteps 3405.
Path 234 | total_timesteps 3435.
Path 235 | total_timesteps 3454.
Path 236 | total_timesteps 3473.
Path 237 | total_timesteps 3494.
Path 238 | total_timesteps 3503.
Path 239 | total_timesteps 3519.
Path 240 | total_timesteps 3535.
Path 241 | total_timesteps 3549.
Path 242 | total_timesteps 3564.
Path 243 | total_timesteps 3576.
Path 244 | total_timesteps 3587.
Path 245 | total_timesteps 3609.
Path 246 | total_timesteps 3623.
Path 247 | total_timesteps 3634.
Path 248 | total_timesteps 3644.
Path 249 | total_timesteps 3654.
Path 250 | total_timesteps 3665.
Path 251 | total_timesteps 3691.
Path 252 | total_timesteps 3707.
Path 253 | total_timesteps 3740.
Path 254 | total_timesteps 3752.
Path 255 | total_timesteps 3772.
Path 256 | total_timesteps 3782.
Path 257 | total_timesteps 3794.
Path 258 | total_timesteps 3805.
Path 259 | total_timesteps 3821.
Path 260 | total_timesteps 3839.
Path 261 | total_timesteps 3856.
Path 262 | total_timesteps 3867.
Path 263 | total_timesteps 3878.
Path 264 | total_timesteps 3893.
Path 265 | total_timesteps 3906.
Path 266 | total_timesteps 3922.
Path 267 | total_timesteps 3943.
Path 268 | total_timesteps 3959.
Path 269 | total_timesteps 3970.
Path 270 | total_timesteps 3989.
Path 271 | total_timesteps 4010.
Path 272 | total_timesteps 4031.
Path 273 | total_timesteps 4048.
Path 274 | total_timesteps 4064.
Path 275 | total_timesteps 4080.
Path 276 | total_timesteps 4094.
Path 277 | total_timesteps 4107.
Path 278 | total_timesteps 4121.
Path 279 | total_timesteps 4132.
Path 280 | total_timesteps 4147.
Path 281 | total_timesteps 4157.
Path 282 | total_timesteps 4169.
Path 283 | total_timesteps 4178.
Path 284 | total_timesteps 4232.
Path 285 | total_timesteps 4241.
Path 286 | total_timesteps 4255.
Path 287 | total_timesteps 4262.
Path 288 | total_timesteps 4271.
Path 289 | total_timesteps 4286.
Path 290 | total_timesteps 4302.
Path 291 | total_timesteps 4328.
Path 292 | total_timesteps 4339.
Path 293 | total_timesteps 4355.
Path 294 | total_timesteps 4364.
Path 295 | total_timesteps 4381.
Path 296 | total_timesteps 4432.
Path 297 | total_timesteps 4448.
Path 298 | total_timesteps 4459.
Path 299 | total_timesteps 4477.
Path 300 | total_timesteps 4494.
Path 301 | total_timesteps 4509.
Path 302 | total_timesteps 4519.
Path 303 | total_timesteps 4541.
Path 304 | total_timesteps 4583.
Path 305 | total_timesteps 4598.
Path 306 | total_timesteps 4610.
Path 307 | total_timesteps 4624.
Path 308 | total_timesteps 4642.
Path 309 | total_timesteps 4651.
Path 310 | total_timesteps 4665.
Path 311 | total_timesteps 4677.
Path 312 | total_timesteps 4687.
Path 313 | total_timesteps 4697.
Path 314 | total_timesteps 4706.
Path 315 | total_timesteps 4718.
Path 316 | total_timesteps 4728.
Path 317 | total_timesteps 4741.
Path 318 | total_timesteps 4755.
Path 319 | total_timesteps 4767.
Path 320 | total_timesteps 4775.
Path 321 | total_timesteps 4787.
Path 322 | total_timesteps 4799.
Path 323 | total_timesteps 4810.
Path 324 | total_timesteps 4824.
Path 325 | total_timesteps 4835.
Path 326 | total_timesteps 4846.
Path 327 | total_timesteps 4861.
Path 328 | total_timesteps 4881.
Path 329 | total_timesteps 4893.
Path 330 | total_timesteps 4907.
Path 331 | total_timesteps 4931.
Path 332 | total_timesteps 4948.
Path 333 | total_timesteps 4960.
Path 334 | total_timesteps 4972.
Path 335 | total_timesteps 4983.
Path 336 | total_timesteps 4994.
Path 337 | total_timesteps 5010.
Path 338 | total_timesteps 5024.
Path 339 | total_timesteps 5035.
Path 340 | total_timesteps 5051.
Path 341 | total_timesteps 5068.
Path 342 | total_timesteps 5078.
Path 343 | total_timesteps 5091.
Path 344 | total_timesteps 5107.
Path 345 | total_timesteps 5130.
Path 346 | total_timesteps 5148.
Path 347 | total_timesteps 5161.
Path 348 | total_timesteps 5172.
Path 349 | total_timesteps 5189.
Path 350 | total_timesteps 5215.
Path 351 | total_timesteps 5229.
Path 352 | total_timesteps 5256.
Path 353 | total_timesteps 5268.
Path 354 | total_timesteps 5282.
Path 355 | total_timesteps 5300.
Path 356 | total_timesteps 5318.
Path 357 | total_timesteps 5329.
Path 358 | total_timesteps 5348.
Path 359 | total_timesteps 5363.
Path 360 | total_timesteps 5380.
Path 361 | total_timesteps 5391.
Path 362 | total_timesteps 5407.
Path 363 | total_timesteps 5429.
Path 364 | total_timesteps 5452.
Path 365 | total_timesteps 5460.
Path 366 | total_timesteps 5470.
Path 367 | total_timesteps 5481.
Path 368 | total_timesteps 5495.
Path 369 | total_timesteps 5509.
Path 370 | total_timesteps 5526.
Path 371 | total_timesteps 5547.
Path 372 | total_timesteps 5559.
Path 373 | total_timesteps 5572.
Path 374 | total_timesteps 5584.
Path 375 | total_timesteps 5598.
Path 376 | total_timesteps 5611.
Path 377 | total_timesteps 5628.
Path 378 | total_timesteps 5645.
Path 379 | total_timesteps 5656.
Path 380 | total_timesteps 5664.
Path 381 | total_timesteps 5675.
Path 382 | total_timesteps 5697.
Path 383 | total_timesteps 5706.
Path 384 | total_timesteps 5717.
Path 385 | total_timesteps 5727.
Path 386 | total_timesteps 5743.
Path 387 | total_timesteps 5759.
Path 388 | total_timesteps 5779.
Path 389 | total_timesteps 5787.
Path 390 | total_timesteps 5814.
Path 391 | total_timesteps 5825.
Path 392 | total_timesteps 5851.
Path 393 | total_timesteps 5858.
Path 394 | total_timesteps 5872.
Path 395 | total_timesteps 5889.
Path 396 | total_timesteps 5902.
Path 397 | total_timesteps 5912.
Path 398 | total_timesteps 5925.
Path 399 | total_timesteps 5937.
Path 400 | total_timesteps 5946.
Path 401 | total_timesteps 5957.
Path 402 | total_timesteps 5978.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.43    |
| Iteration     | 15       |
| MaximumReturn | 5.77     |
| MinimumReturn | -21.3    |
| TotalSamples  | 68103    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.004874187521636486
Validation loss = 0.004649514798074961
Validation loss = 0.004266603384166956
Validation loss = 0.0042829508893191814
Validation loss = 0.004145246930420399
Validation loss = 0.004156741313636303
Validation loss = 0.004281928297132254
Validation loss = 0.004207149613648653
Validation loss = 0.004527465905994177
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 40.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 78.
Path 8 | total_timesteps 89.
Path 9 | total_timesteps 99.
Path 10 | total_timesteps 111.
Path 11 | total_timesteps 123.
Path 12 | total_timesteps 131.
Path 13 | total_timesteps 140.
Path 14 | total_timesteps 149.
Path 15 | total_timesteps 159.
Path 16 | total_timesteps 174.
Path 17 | total_timesteps 183.
Path 18 | total_timesteps 195.
Path 19 | total_timesteps 206.
Path 20 | total_timesteps 220.
Path 21 | total_timesteps 234.
Path 22 | total_timesteps 242.
Path 23 | total_timesteps 252.
Path 24 | total_timesteps 266.
Path 25 | total_timesteps 285.
Path 26 | total_timesteps 295.
Path 27 | total_timesteps 306.
Path 28 | total_timesteps 319.
Path 29 | total_timesteps 337.
Path 30 | total_timesteps 348.
Path 31 | total_timesteps 363.
Path 32 | total_timesteps 372.
Path 33 | total_timesteps 381.
Path 34 | total_timesteps 406.
Path 35 | total_timesteps 421.
Path 36 | total_timesteps 443.
Path 37 | total_timesteps 451.
Path 38 | total_timesteps 459.
Path 39 | total_timesteps 468.
Path 40 | total_timesteps 480.
Path 41 | total_timesteps 492.
Path 42 | total_timesteps 501.
Path 43 | total_timesteps 512.
Path 44 | total_timesteps 522.
Path 45 | total_timesteps 549.
Path 46 | total_timesteps 560.
Path 47 | total_timesteps 569.
Path 48 | total_timesteps 578.
Path 49 | total_timesteps 592.
Path 50 | total_timesteps 604.
Path 51 | total_timesteps 614.
Path 52 | total_timesteps 625.
Path 53 | total_timesteps 637.
Path 54 | total_timesteps 645.
Path 55 | total_timesteps 653.
Path 56 | total_timesteps 668.
Path 57 | total_timesteps 684.
Path 58 | total_timesteps 693.
Path 59 | total_timesteps 703.
Path 60 | total_timesteps 713.
Path 61 | total_timesteps 725.
Path 62 | total_timesteps 737.
Path 63 | total_timesteps 763.
Path 64 | total_timesteps 778.
Path 65 | total_timesteps 787.
Path 66 | total_timesteps 797.
Path 67 | total_timesteps 805.
Path 68 | total_timesteps 816.
Path 69 | total_timesteps 828.
Path 70 | total_timesteps 839.
Path 71 | total_timesteps 866.
Path 72 | total_timesteps 883.
Path 73 | total_timesteps 897.
Path 74 | total_timesteps 906.
Path 75 | total_timesteps 914.
Path 76 | total_timesteps 927.
Path 77 | total_timesteps 940.
Path 78 | total_timesteps 950.
Path 79 | total_timesteps 958.
Path 80 | total_timesteps 970.
Path 81 | total_timesteps 984.
Path 82 | total_timesteps 1001.
Path 83 | total_timesteps 1009.
Path 84 | total_timesteps 1020.
Path 85 | total_timesteps 1033.
Path 86 | total_timesteps 1040.
Path 87 | total_timesteps 1052.
Path 88 | total_timesteps 1065.
Path 89 | total_timesteps 1075.
Path 90 | total_timesteps 1086.
Path 91 | total_timesteps 1095.
Path 92 | total_timesteps 1106.
Path 93 | total_timesteps 1116.
Path 94 | total_timesteps 1129.
Path 95 | total_timesteps 1136.
Path 96 | total_timesteps 1147.
Path 97 | total_timesteps 1160.
Path 98 | total_timesteps 1174.
Path 99 | total_timesteps 1184.
Path 100 | total_timesteps 1196.
Path 101 | total_timesteps 1206.
Path 102 | total_timesteps 1218.
Path 103 | total_timesteps 1228.
Path 104 | total_timesteps 1240.
Path 105 | total_timesteps 1255.
Path 106 | total_timesteps 1265.
Path 107 | total_timesteps 1275.
Path 108 | total_timesteps 1285.
Path 109 | total_timesteps 1298.
Path 110 | total_timesteps 1310.
Path 111 | total_timesteps 1319.
Path 112 | total_timesteps 1334.
Path 113 | total_timesteps 1351.
Path 114 | total_timesteps 1361.
Path 115 | total_timesteps 1371.
Path 116 | total_timesteps 1389.
Path 117 | total_timesteps 1400.
Path 118 | total_timesteps 1410.
Path 119 | total_timesteps 1422.
Path 120 | total_timesteps 1433.
Path 121 | total_timesteps 1448.
Path 122 | total_timesteps 1459.
Path 123 | total_timesteps 1468.
Path 124 | total_timesteps 1488.
Path 125 | total_timesteps 1498.
Path 126 | total_timesteps 1507.
Path 127 | total_timesteps 1525.
Path 128 | total_timesteps 1540.
Path 129 | total_timesteps 1548.
Path 130 | total_timesteps 1561.
Path 131 | total_timesteps 1584.
Path 132 | total_timesteps 1593.
Path 133 | total_timesteps 1606.
Path 134 | total_timesteps 1622.
Path 135 | total_timesteps 1635.
Path 136 | total_timesteps 1649.
Path 137 | total_timesteps 1661.
Path 138 | total_timesteps 1674.
Path 139 | total_timesteps 1693.
Path 140 | total_timesteps 1704.
Path 141 | total_timesteps 1714.
Path 142 | total_timesteps 1724.
Path 143 | total_timesteps 1742.
Path 144 | total_timesteps 1750.
Path 145 | total_timesteps 1761.
Path 146 | total_timesteps 1772.
Path 147 | total_timesteps 1791.
Path 148 | total_timesteps 1804.
Path 149 | total_timesteps 1815.
Path 150 | total_timesteps 1823.
Path 151 | total_timesteps 1832.
Path 152 | total_timesteps 1843.
Path 153 | total_timesteps 1850.
Path 154 | total_timesteps 1863.
Path 155 | total_timesteps 1873.
Path 156 | total_timesteps 1887.
Path 157 | total_timesteps 1898.
Path 158 | total_timesteps 1922.
Path 159 | total_timesteps 1931.
Path 160 | total_timesteps 1939.
Path 161 | total_timesteps 1960.
Path 162 | total_timesteps 1969.
Path 163 | total_timesteps 1979.
Path 164 | total_timesteps 1988.
Path 165 | total_timesteps 1998.
Path 166 | total_timesteps 2009.
Path 167 | total_timesteps 2026.
Path 168 | total_timesteps 2040.
Path 169 | total_timesteps 2050.
Path 170 | total_timesteps 2059.
Path 171 | total_timesteps 2077.
Path 172 | total_timesteps 2104.
Path 173 | total_timesteps 2118.
Path 174 | total_timesteps 2130.
Path 175 | total_timesteps 2143.
Path 176 | total_timesteps 2151.
Path 177 | total_timesteps 2163.
Path 178 | total_timesteps 2175.
Path 179 | total_timesteps 2185.
Path 180 | total_timesteps 2197.
Path 181 | total_timesteps 2213.
Path 182 | total_timesteps 2233.
Path 183 | total_timesteps 2243.
Path 184 | total_timesteps 2260.
Path 185 | total_timesteps 2273.
Path 186 | total_timesteps 2280.
Path 187 | total_timesteps 2295.
Path 188 | total_timesteps 2305.
Path 189 | total_timesteps 2320.
Path 190 | total_timesteps 2333.
Path 191 | total_timesteps 2342.
Path 192 | total_timesteps 2353.
Path 193 | total_timesteps 2362.
Path 194 | total_timesteps 2375.
Path 195 | total_timesteps 2397.
Path 196 | total_timesteps 2414.
Path 197 | total_timesteps 2422.
Path 198 | total_timesteps 2433.
Path 199 | total_timesteps 2452.
Path 200 | total_timesteps 2473.
Path 201 | total_timesteps 2483.
Path 202 | total_timesteps 2500.
Path 203 | total_timesteps 2515.
Path 204 | total_timesteps 2523.
Path 205 | total_timesteps 2541.
Path 206 | total_timesteps 2551.
Path 207 | total_timesteps 2562.
Path 208 | total_timesteps 2580.
Path 209 | total_timesteps 2588.
Path 210 | total_timesteps 2600.
Path 211 | total_timesteps 2610.
Path 212 | total_timesteps 2619.
Path 213 | total_timesteps 2633.
Path 214 | total_timesteps 2643.
Path 215 | total_timesteps 2654.
Path 216 | total_timesteps 2666.
Path 217 | total_timesteps 2678.
Path 218 | total_timesteps 2691.
Path 219 | total_timesteps 2706.
Path 220 | total_timesteps 2722.
Path 221 | total_timesteps 2731.
Path 222 | total_timesteps 2742.
Path 223 | total_timesteps 2754.
Path 224 | total_timesteps 2771.
Path 225 | total_timesteps 2781.
Path 226 | total_timesteps 2790.
Path 227 | total_timesteps 2802.
Path 228 | total_timesteps 2812.
Path 229 | total_timesteps 2829.
Path 230 | total_timesteps 2838.
Path 231 | total_timesteps 2857.
Path 232 | total_timesteps 2865.
Path 233 | total_timesteps 2874.
Path 234 | total_timesteps 2884.
Path 235 | total_timesteps 2900.
Path 236 | total_timesteps 2916.
Path 237 | total_timesteps 2931.
Path 238 | total_timesteps 2942.
Path 239 | total_timesteps 2952.
Path 240 | total_timesteps 2960.
Path 241 | total_timesteps 2978.
Path 242 | total_timesteps 2993.
Path 243 | total_timesteps 3010.
Path 244 | total_timesteps 3021.
Path 245 | total_timesteps 3029.
Path 246 | total_timesteps 3047.
Path 247 | total_timesteps 3057.
Path 248 | total_timesteps 3070.
Path 249 | total_timesteps 3084.
Path 250 | total_timesteps 3094.
Path 251 | total_timesteps 3106.
Path 252 | total_timesteps 3116.
Path 253 | total_timesteps 3130.
Path 254 | total_timesteps 3148.
Path 255 | total_timesteps 3163.
Path 256 | total_timesteps 3172.
Path 257 | total_timesteps 3179.
Path 258 | total_timesteps 3195.
Path 259 | total_timesteps 3204.
Path 260 | total_timesteps 3214.
Path 261 | total_timesteps 3223.
Path 262 | total_timesteps 3234.
Path 263 | total_timesteps 3245.
Path 264 | total_timesteps 3254.
Path 265 | total_timesteps 3263.
Path 266 | total_timesteps 3272.
Path 267 | total_timesteps 3282.
Path 268 | total_timesteps 3292.
Path 269 | total_timesteps 3303.
Path 270 | total_timesteps 3315.
Path 271 | total_timesteps 3330.
Path 272 | total_timesteps 3348.
Path 273 | total_timesteps 3368.
Path 274 | total_timesteps 3380.
Path 275 | total_timesteps 3395.
Path 276 | total_timesteps 3408.
Path 277 | total_timesteps 3421.
Path 278 | total_timesteps 3436.
Path 279 | total_timesteps 3448.
Path 280 | total_timesteps 3459.
Path 281 | total_timesteps 3470.
Path 282 | total_timesteps 3481.
Path 283 | total_timesteps 3493.
Path 284 | total_timesteps 3505.
Path 285 | total_timesteps 3519.
Path 286 | total_timesteps 3527.
Path 287 | total_timesteps 3537.
Path 288 | total_timesteps 3550.
Path 289 | total_timesteps 3563.
Path 290 | total_timesteps 3573.
Path 291 | total_timesteps 3584.
Path 292 | total_timesteps 3595.
Path 293 | total_timesteps 3616.
Path 294 | total_timesteps 3627.
Path 295 | total_timesteps 3636.
Path 296 | total_timesteps 3647.
Path 297 | total_timesteps 3658.
Path 298 | total_timesteps 3672.
Path 299 | total_timesteps 3685.
Path 300 | total_timesteps 3694.
Path 301 | total_timesteps 3702.
Path 302 | total_timesteps 3712.
Path 303 | total_timesteps 3725.
Path 304 | total_timesteps 3738.
Path 305 | total_timesteps 3748.
Path 306 | total_timesteps 3761.
Path 307 | total_timesteps 3773.
Path 308 | total_timesteps 3785.
Path 309 | total_timesteps 3803.
Path 310 | total_timesteps 3814.
Path 311 | total_timesteps 3829.
Path 312 | total_timesteps 3841.
Path 313 | total_timesteps 3848.
Path 314 | total_timesteps 3861.
Path 315 | total_timesteps 3872.
Path 316 | total_timesteps 3889.
Path 317 | total_timesteps 3900.
Path 318 | total_timesteps 3913.
Path 319 | total_timesteps 3922.
Path 320 | total_timesteps 3933.
Path 321 | total_timesteps 3945.
Path 322 | total_timesteps 3954.
Path 323 | total_timesteps 3969.
Path 324 | total_timesteps 3980.
Path 325 | total_timesteps 3993.
Path 326 | total_timesteps 4002.
Path 327 | total_timesteps 4013.
Path 328 | total_timesteps 4026.
Path 329 | total_timesteps 4036.
Path 330 | total_timesteps 4045.
Path 331 | total_timesteps 4054.
Path 332 | total_timesteps 4062.
Path 333 | total_timesteps 4075.
Path 334 | total_timesteps 4086.
Path 335 | total_timesteps 4097.
Path 336 | total_timesteps 4116.
Path 337 | total_timesteps 4132.
Path 338 | total_timesteps 4146.
Path 339 | total_timesteps 4157.
Path 340 | total_timesteps 4173.
Path 341 | total_timesteps 4183.
Path 342 | total_timesteps 4194.
Path 343 | total_timesteps 4205.
Path 344 | total_timesteps 4215.
Path 345 | total_timesteps 4225.
Path 346 | total_timesteps 4233.
Path 347 | total_timesteps 4247.
Path 348 | total_timesteps 4255.
Path 349 | total_timesteps 4273.
Path 350 | total_timesteps 4286.
Path 351 | total_timesteps 4294.
Path 352 | total_timesteps 4303.
Path 353 | total_timesteps 4316.
Path 354 | total_timesteps 4325.
Path 355 | total_timesteps 4337.
Path 356 | total_timesteps 4351.
Path 357 | total_timesteps 4371.
Path 358 | total_timesteps 4388.
Path 359 | total_timesteps 4399.
Path 360 | total_timesteps 4409.
Path 361 | total_timesteps 4418.
Path 362 | total_timesteps 4429.
Path 363 | total_timesteps 4446.
Path 364 | total_timesteps 4454.
Path 365 | total_timesteps 4465.
Path 366 | total_timesteps 4482.
Path 367 | total_timesteps 4494.
Path 368 | total_timesteps 4505.
Path 369 | total_timesteps 4518.
Path 370 | total_timesteps 4530.
Path 371 | total_timesteps 4545.
Path 372 | total_timesteps 4562.
Path 373 | total_timesteps 4572.
Path 374 | total_timesteps 4586.
Path 375 | total_timesteps 4594.
Path 376 | total_timesteps 4601.
Path 377 | total_timesteps 4613.
Path 378 | total_timesteps 4632.
Path 379 | total_timesteps 4646.
Path 380 | total_timesteps 4657.
Path 381 | total_timesteps 4667.
Path 382 | total_timesteps 4683.
Path 383 | total_timesteps 4695.
Path 384 | total_timesteps 4703.
Path 385 | total_timesteps 4717.
Path 386 | total_timesteps 4728.
Path 387 | total_timesteps 4742.
Path 388 | total_timesteps 4754.
Path 389 | total_timesteps 4764.
Path 390 | total_timesteps 4781.
Path 391 | total_timesteps 4797.
Path 392 | total_timesteps 4809.
Path 393 | total_timesteps 4824.
Path 394 | total_timesteps 4838.
Path 395 | total_timesteps 4847.
Path 396 | total_timesteps 4857.
Path 397 | total_timesteps 4864.
Path 398 | total_timesteps 4874.
Path 399 | total_timesteps 4887.
Path 400 | total_timesteps 4900.
Path 401 | total_timesteps 4916.
Path 402 | total_timesteps 4928.
Path 403 | total_timesteps 4940.
Path 404 | total_timesteps 4953.
Path 405 | total_timesteps 4967.
Path 406 | total_timesteps 4975.
Path 407 | total_timesteps 4988.
Path 408 | total_timesteps 4997.
Path 409 | total_timesteps 5007.
Path 410 | total_timesteps 5021.
Path 411 | total_timesteps 5034.
Path 412 | total_timesteps 5054.
Path 413 | total_timesteps 5073.
Path 414 | total_timesteps 5083.
Path 415 | total_timesteps 5091.
Path 416 | total_timesteps 5103.
Path 417 | total_timesteps 5113.
Path 418 | total_timesteps 5124.
Path 419 | total_timesteps 5139.
Path 420 | total_timesteps 5153.
Path 421 | total_timesteps 5175.
Path 422 | total_timesteps 5187.
Path 423 | total_timesteps 5200.
Path 424 | total_timesteps 5221.
Path 425 | total_timesteps 5229.
Path 426 | total_timesteps 5243.
Path 427 | total_timesteps 5256.
Path 428 | total_timesteps 5267.
Path 429 | total_timesteps 5275.
Path 430 | total_timesteps 5284.
Path 431 | total_timesteps 5297.
Path 432 | total_timesteps 5317.
Path 433 | total_timesteps 5328.
Path 434 | total_timesteps 5351.
Path 435 | total_timesteps 5366.
Path 436 | total_timesteps 5376.
Path 437 | total_timesteps 5383.
Path 438 | total_timesteps 5396.
Path 439 | total_timesteps 5405.
Path 440 | total_timesteps 5418.
Path 441 | total_timesteps 5429.
Path 442 | total_timesteps 5439.
Path 443 | total_timesteps 5449.
Path 444 | total_timesteps 5460.
Path 445 | total_timesteps 5469.
Path 446 | total_timesteps 5481.
Path 447 | total_timesteps 5494.
Path 448 | total_timesteps 5506.
Path 449 | total_timesteps 5515.
Path 450 | total_timesteps 5529.
Path 451 | total_timesteps 5536.
Path 452 | total_timesteps 5547.
Path 453 | total_timesteps 5555.
Path 454 | total_timesteps 5564.
Path 455 | total_timesteps 5574.
Path 456 | total_timesteps 5581.
Path 457 | total_timesteps 5590.
Path 458 | total_timesteps 5602.
Path 459 | total_timesteps 5618.
Path 460 | total_timesteps 5633.
Path 461 | total_timesteps 5649.
Path 462 | total_timesteps 5667.
Path 463 | total_timesteps 5682.
Path 464 | total_timesteps 5694.
Path 465 | total_timesteps 5708.
Path 466 | total_timesteps 5728.
Path 467 | total_timesteps 5740.
Path 468 | total_timesteps 5753.
Path 469 | total_timesteps 5771.
Path 470 | total_timesteps 5784.
Path 471 | total_timesteps 5803.
Path 472 | total_timesteps 5814.
Path 473 | total_timesteps 5827.
Path 474 | total_timesteps 5838.
Path 475 | total_timesteps 5854.
Path 476 | total_timesteps 5869.
Path 477 | total_timesteps 5884.
Path 478 | total_timesteps 5896.
Path 479 | total_timesteps 5909.
Path 480 | total_timesteps 5923.
Path 481 | total_timesteps 5932.
Path 482 | total_timesteps 5944.
Path 483 | total_timesteps 5958.
Path 484 | total_timesteps 5971.
Path 485 | total_timesteps 5978.
Path 486 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.86    |
| Iteration     | 16       |
| MaximumReturn | 2.59     |
| MinimumReturn | -21.1    |
| TotalSamples  | 72108    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.00404231995344162
Validation loss = 0.004101424012333155
Validation loss = 0.004201109521090984
Validation loss = 0.005183294415473938
Validation loss = 0.0038098336663097143
Validation loss = 0.004806470591574907
Validation loss = 0.004075291100889444
Validation loss = 0.004991041496396065
Validation loss = 0.00375778554007411
Validation loss = 0.004167910199612379
Validation loss = 0.004175874870270491
Validation loss = 0.004648379981517792
Validation loss = 0.003655222477391362
Validation loss = 0.0036460543051362038
Validation loss = 0.0038203198928385973
Validation loss = 0.003985919989645481
Validation loss = 0.003566744038835168
Validation loss = 0.003907972946763039
Validation loss = 0.0035698674619197845
Validation loss = 0.003976957406848669
Validation loss = 0.003505212953314185
Validation loss = 0.003476209007203579
Validation loss = 0.003609616309404373
Validation loss = 0.003614255925640464
Validation loss = 0.0033292232546955347
Validation loss = 0.003297969466075301
Validation loss = 0.003756034653633833
Validation loss = 0.0037445356138050556
Validation loss = 0.0035057952627539635
Validation loss = 0.0034698424860835075
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 84.
Path 9 | total_timesteps 96.
Path 10 | total_timesteps 113.
Path 11 | total_timesteps 124.
Path 12 | total_timesteps 135.
Path 13 | total_timesteps 148.
Path 14 | total_timesteps 163.
Path 15 | total_timesteps 175.
Path 16 | total_timesteps 188.
Path 17 | total_timesteps 198.
Path 18 | total_timesteps 210.
Path 19 | total_timesteps 221.
Path 20 | total_timesteps 237.
Path 21 | total_timesteps 247.
Path 22 | total_timesteps 257.
Path 23 | total_timesteps 267.
Path 24 | total_timesteps 278.
Path 25 | total_timesteps 289.
Path 26 | total_timesteps 297.
Path 27 | total_timesteps 310.
Path 28 | total_timesteps 322.
Path 29 | total_timesteps 335.
Path 30 | total_timesteps 347.
Path 31 | total_timesteps 364.
Path 32 | total_timesteps 386.
Path 33 | total_timesteps 398.
Path 34 | total_timesteps 414.
Path 35 | total_timesteps 428.
Path 36 | total_timesteps 437.
Path 37 | total_timesteps 448.
Path 38 | total_timesteps 458.
Path 39 | total_timesteps 466.
Path 40 | total_timesteps 481.
Path 41 | total_timesteps 491.
Path 42 | total_timesteps 501.
Path 43 | total_timesteps 511.
Path 44 | total_timesteps 521.
Path 45 | total_timesteps 529.
Path 46 | total_timesteps 538.
Path 47 | total_timesteps 549.
Path 48 | total_timesteps 561.
Path 49 | total_timesteps 572.
Path 50 | total_timesteps 587.
Path 51 | total_timesteps 595.
Path 52 | total_timesteps 604.
Path 53 | total_timesteps 618.
Path 54 | total_timesteps 630.
Path 55 | total_timesteps 643.
Path 56 | total_timesteps 656.
Path 57 | total_timesteps 666.
Path 58 | total_timesteps 678.
Path 59 | total_timesteps 687.
Path 60 | total_timesteps 695.
Path 61 | total_timesteps 704.
Path 62 | total_timesteps 720.
Path 63 | total_timesteps 733.
Path 64 | total_timesteps 742.
Path 65 | total_timesteps 750.
Path 66 | total_timesteps 758.
Path 67 | total_timesteps 768.
Path 68 | total_timesteps 780.
Path 69 | total_timesteps 792.
Path 70 | total_timesteps 802.
Path 71 | total_timesteps 812.
Path 72 | total_timesteps 823.
Path 73 | total_timesteps 837.
Path 74 | total_timesteps 848.
Path 75 | total_timesteps 856.
Path 76 | total_timesteps 867.
Path 77 | total_timesteps 878.
Path 78 | total_timesteps 893.
Path 79 | total_timesteps 905.
Path 80 | total_timesteps 924.
Path 81 | total_timesteps 934.
Path 82 | total_timesteps 948.
Path 83 | total_timesteps 961.
Path 84 | total_timesteps 974.
Path 85 | total_timesteps 989.
Path 86 | total_timesteps 1003.
Path 87 | total_timesteps 1016.
Path 88 | total_timesteps 1028.
Path 89 | total_timesteps 1038.
Path 90 | total_timesteps 1050.
Path 91 | total_timesteps 1065.
Path 92 | total_timesteps 1080.
Path 93 | total_timesteps 1092.
Path 94 | total_timesteps 1117.
Path 95 | total_timesteps 1126.
Path 96 | total_timesteps 1137.
Path 97 | total_timesteps 1151.
Path 98 | total_timesteps 1164.
Path 99 | total_timesteps 1174.
Path 100 | total_timesteps 1189.
Path 101 | total_timesteps 1198.
Path 102 | total_timesteps 1227.
Path 103 | total_timesteps 1234.
Path 104 | total_timesteps 1247.
Path 105 | total_timesteps 1256.
Path 106 | total_timesteps 1266.
Path 107 | total_timesteps 1275.
Path 108 | total_timesteps 1285.
Path 109 | total_timesteps 1298.
Path 110 | total_timesteps 1306.
Path 111 | total_timesteps 1316.
Path 112 | total_timesteps 1329.
Path 113 | total_timesteps 1338.
Path 114 | total_timesteps 1348.
Path 115 | total_timesteps 1357.
Path 116 | total_timesteps 1368.
Path 117 | total_timesteps 1377.
Path 118 | total_timesteps 1389.
Path 119 | total_timesteps 1397.
Path 120 | total_timesteps 1409.
Path 121 | total_timesteps 1425.
Path 122 | total_timesteps 1439.
Path 123 | total_timesteps 1448.
Path 124 | total_timesteps 1459.
Path 125 | total_timesteps 1474.
Path 126 | total_timesteps 1484.
Path 127 | total_timesteps 1495.
Path 128 | total_timesteps 1505.
Path 129 | total_timesteps 1517.
Path 130 | total_timesteps 1527.
Path 131 | total_timesteps 1544.
Path 132 | total_timesteps 1555.
Path 133 | total_timesteps 1565.
Path 134 | total_timesteps 1574.
Path 135 | total_timesteps 1584.
Path 136 | total_timesteps 1598.
Path 137 | total_timesteps 1609.
Path 138 | total_timesteps 1619.
Path 139 | total_timesteps 1628.
Path 140 | total_timesteps 1639.
Path 141 | total_timesteps 1655.
Path 142 | total_timesteps 1668.
Path 143 | total_timesteps 1679.
Path 144 | total_timesteps 1692.
Path 145 | total_timesteps 1699.
Path 146 | total_timesteps 1709.
Path 147 | total_timesteps 1721.
Path 148 | total_timesteps 1735.
Path 149 | total_timesteps 1747.
Path 150 | total_timesteps 1756.
Path 151 | total_timesteps 1766.
Path 152 | total_timesteps 1778.
Path 153 | total_timesteps 1789.
Path 154 | total_timesteps 1799.
Path 155 | total_timesteps 1809.
Path 156 | total_timesteps 1823.
Path 157 | total_timesteps 1832.
Path 158 | total_timesteps 1842.
Path 159 | total_timesteps 1850.
Path 160 | total_timesteps 1859.
Path 161 | total_timesteps 1871.
Path 162 | total_timesteps 1883.
Path 163 | total_timesteps 1919.
Path 164 | total_timesteps 1928.
Path 165 | total_timesteps 1938.
Path 166 | total_timesteps 1947.
Path 167 | total_timesteps 1965.
Path 168 | total_timesteps 1976.
Path 169 | total_timesteps 1986.
Path 170 | total_timesteps 1995.
Path 171 | total_timesteps 2007.
Path 172 | total_timesteps 2017.
Path 173 | total_timesteps 2027.
Path 174 | total_timesteps 2040.
Path 175 | total_timesteps 2059.
Path 176 | total_timesteps 2075.
Path 177 | total_timesteps 2083.
Path 178 | total_timesteps 2098.
Path 179 | total_timesteps 2107.
Path 180 | total_timesteps 2118.
Path 181 | total_timesteps 2127.
Path 182 | total_timesteps 2141.
Path 183 | total_timesteps 2165.
Path 184 | total_timesteps 2176.
Path 185 | total_timesteps 2188.
Path 186 | total_timesteps 2198.
Path 187 | total_timesteps 2210.
Path 188 | total_timesteps 2225.
Path 189 | total_timesteps 2237.
Path 190 | total_timesteps 2249.
Path 191 | total_timesteps 2284.
Path 192 | total_timesteps 2294.
Path 193 | total_timesteps 2308.
Path 194 | total_timesteps 2320.
Path 195 | total_timesteps 2333.
Path 196 | total_timesteps 2344.
Path 197 | total_timesteps 2354.
Path 198 | total_timesteps 2367.
Path 199 | total_timesteps 2378.
Path 200 | total_timesteps 2388.
Path 201 | total_timesteps 2398.
Path 202 | total_timesteps 2415.
Path 203 | total_timesteps 2426.
Path 204 | total_timesteps 2435.
Path 205 | total_timesteps 2445.
Path 206 | total_timesteps 2457.
Path 207 | total_timesteps 2465.
Path 208 | total_timesteps 2478.
Path 209 | total_timesteps 2491.
Path 210 | total_timesteps 2505.
Path 211 | total_timesteps 2513.
Path 212 | total_timesteps 2524.
Path 213 | total_timesteps 2535.
Path 214 | total_timesteps 2543.
Path 215 | total_timesteps 2552.
Path 216 | total_timesteps 2564.
Path 217 | total_timesteps 2576.
Path 218 | total_timesteps 2586.
Path 219 | total_timesteps 2597.
Path 220 | total_timesteps 2622.
Path 221 | total_timesteps 2633.
Path 222 | total_timesteps 2643.
Path 223 | total_timesteps 2652.
Path 224 | total_timesteps 2676.
Path 225 | total_timesteps 2686.
Path 226 | total_timesteps 2700.
Path 227 | total_timesteps 2713.
Path 228 | total_timesteps 2725.
Path 229 | total_timesteps 2734.
Path 230 | total_timesteps 2750.
Path 231 | total_timesteps 2761.
Path 232 | total_timesteps 2769.
Path 233 | total_timesteps 2784.
Path 234 | total_timesteps 2802.
Path 235 | total_timesteps 2814.
Path 236 | total_timesteps 2823.
Path 237 | total_timesteps 2835.
Path 238 | total_timesteps 2844.
Path 239 | total_timesteps 2856.
Path 240 | total_timesteps 2866.
Path 241 | total_timesteps 2873.
Path 242 | total_timesteps 2886.
Path 243 | total_timesteps 2899.
Path 244 | total_timesteps 2911.
Path 245 | total_timesteps 2919.
Path 246 | total_timesteps 2932.
Path 247 | total_timesteps 2947.
Path 248 | total_timesteps 2959.
Path 249 | total_timesteps 2968.
Path 250 | total_timesteps 2979.
Path 251 | total_timesteps 2991.
Path 252 | total_timesteps 3002.
Path 253 | total_timesteps 3022.
Path 254 | total_timesteps 3034.
Path 255 | total_timesteps 3044.
Path 256 | total_timesteps 3062.
Path 257 | total_timesteps 3071.
Path 258 | total_timesteps 3083.
Path 259 | total_timesteps 3094.
Path 260 | total_timesteps 3106.
Path 261 | total_timesteps 3119.
Path 262 | total_timesteps 3131.
Path 263 | total_timesteps 3140.
Path 264 | total_timesteps 3154.
Path 265 | total_timesteps 3166.
Path 266 | total_timesteps 3178.
Path 267 | total_timesteps 3187.
Path 268 | total_timesteps 3200.
Path 269 | total_timesteps 3209.
Path 270 | total_timesteps 3225.
Path 271 | total_timesteps 3233.
Path 272 | total_timesteps 3244.
Path 273 | total_timesteps 3252.
Path 274 | total_timesteps 3265.
Path 275 | total_timesteps 3277.
Path 276 | total_timesteps 3289.
Path 277 | total_timesteps 3303.
Path 278 | total_timesteps 3309.
Path 279 | total_timesteps 3319.
Path 280 | total_timesteps 3334.
Path 281 | total_timesteps 3342.
Path 282 | total_timesteps 3355.
Path 283 | total_timesteps 3366.
Path 284 | total_timesteps 3379.
Path 285 | total_timesteps 3391.
Path 286 | total_timesteps 3402.
Path 287 | total_timesteps 3414.
Path 288 | total_timesteps 3430.
Path 289 | total_timesteps 3441.
Path 290 | total_timesteps 3451.
Path 291 | total_timesteps 3473.
Path 292 | total_timesteps 3486.
Path 293 | total_timesteps 3496.
Path 294 | total_timesteps 3507.
Path 295 | total_timesteps 3519.
Path 296 | total_timesteps 3529.
Path 297 | total_timesteps 3542.
Path 298 | total_timesteps 3552.
Path 299 | total_timesteps 3565.
Path 300 | total_timesteps 3578.
Path 301 | total_timesteps 3592.
Path 302 | total_timesteps 3610.
Path 303 | total_timesteps 3619.
Path 304 | total_timesteps 3630.
Path 305 | total_timesteps 3640.
Path 306 | total_timesteps 3656.
Path 307 | total_timesteps 3671.
Path 308 | total_timesteps 3687.
Path 309 | total_timesteps 3697.
Path 310 | total_timesteps 3713.
Path 311 | total_timesteps 3729.
Path 312 | total_timesteps 3738.
Path 313 | total_timesteps 3746.
Path 314 | total_timesteps 3757.
Path 315 | total_timesteps 3769.
Path 316 | total_timesteps 3779.
Path 317 | total_timesteps 3790.
Path 318 | total_timesteps 3799.
Path 319 | total_timesteps 3809.
Path 320 | total_timesteps 3822.
Path 321 | total_timesteps 3840.
Path 322 | total_timesteps 3849.
Path 323 | total_timesteps 3860.
Path 324 | total_timesteps 3872.
Path 325 | total_timesteps 3889.
Path 326 | total_timesteps 3897.
Path 327 | total_timesteps 3914.
Path 328 | total_timesteps 3931.
Path 329 | total_timesteps 3938.
Path 330 | total_timesteps 3947.
Path 331 | total_timesteps 3957.
Path 332 | total_timesteps 3971.
Path 333 | total_timesteps 3983.
Path 334 | total_timesteps 3999.
Path 335 | total_timesteps 4007.
Path 336 | total_timesteps 4021.
Path 337 | total_timesteps 4032.
Path 338 | total_timesteps 4045.
Path 339 | total_timesteps 4071.
Path 340 | total_timesteps 4086.
Path 341 | total_timesteps 4096.
Path 342 | total_timesteps 4103.
Path 343 | total_timesteps 4115.
Path 344 | total_timesteps 4125.
Path 345 | total_timesteps 4141.
Path 346 | total_timesteps 4151.
Path 347 | total_timesteps 4159.
Path 348 | total_timesteps 4168.
Path 349 | total_timesteps 4177.
Path 350 | total_timesteps 4186.
Path 351 | total_timesteps 4200.
Path 352 | total_timesteps 4211.
Path 353 | total_timesteps 4224.
Path 354 | total_timesteps 4234.
Path 355 | total_timesteps 4243.
Path 356 | total_timesteps 4259.
Path 357 | total_timesteps 4268.
Path 358 | total_timesteps 4281.
Path 359 | total_timesteps 4293.
Path 360 | total_timesteps 4313.
Path 361 | total_timesteps 4323.
Path 362 | total_timesteps 4337.
Path 363 | total_timesteps 4349.
Path 364 | total_timesteps 4358.
Path 365 | total_timesteps 4369.
Path 366 | total_timesteps 4379.
Path 367 | total_timesteps 4389.
Path 368 | total_timesteps 4399.
Path 369 | total_timesteps 4410.
Path 370 | total_timesteps 4421.
Path 371 | total_timesteps 4436.
Path 372 | total_timesteps 4448.
Path 373 | total_timesteps 4458.
Path 374 | total_timesteps 4468.
Path 375 | total_timesteps 4479.
Path 376 | total_timesteps 4497.
Path 377 | total_timesteps 4507.
Path 378 | total_timesteps 4517.
Path 379 | total_timesteps 4530.
Path 380 | total_timesteps 4552.
Path 381 | total_timesteps 4567.
Path 382 | total_timesteps 4581.
Path 383 | total_timesteps 4589.
Path 384 | total_timesteps 4607.
Path 385 | total_timesteps 4622.
Path 386 | total_timesteps 4639.
Path 387 | total_timesteps 4648.
Path 388 | total_timesteps 4659.
Path 389 | total_timesteps 4673.
Path 390 | total_timesteps 4683.
Path 391 | total_timesteps 4692.
Path 392 | total_timesteps 4708.
Path 393 | total_timesteps 4721.
Path 394 | total_timesteps 4730.
Path 395 | total_timesteps 4739.
Path 396 | total_timesteps 4752.
Path 397 | total_timesteps 4765.
Path 398 | total_timesteps 4775.
Path 399 | total_timesteps 4784.
Path 400 | total_timesteps 4797.
Path 401 | total_timesteps 4808.
Path 402 | total_timesteps 4822.
Path 403 | total_timesteps 4832.
Path 404 | total_timesteps 4842.
Path 405 | total_timesteps 4852.
Path 406 | total_timesteps 4864.
Path 407 | total_timesteps 4873.
Path 408 | total_timesteps 4884.
Path 409 | total_timesteps 4892.
Path 410 | total_timesteps 4902.
Path 411 | total_timesteps 4914.
Path 412 | total_timesteps 4925.
Path 413 | total_timesteps 4936.
Path 414 | total_timesteps 4943.
Path 415 | total_timesteps 4955.
Path 416 | total_timesteps 4963.
Path 417 | total_timesteps 4974.
Path 418 | total_timesteps 4986.
Path 419 | total_timesteps 4998.
Path 420 | total_timesteps 5013.
Path 421 | total_timesteps 5023.
Path 422 | total_timesteps 5034.
Path 423 | total_timesteps 5044.
Path 424 | total_timesteps 5056.
Path 425 | total_timesteps 5067.
Path 426 | total_timesteps 5081.
Path 427 | total_timesteps 5091.
Path 428 | total_timesteps 5100.
Path 429 | total_timesteps 5111.
Path 430 | total_timesteps 5121.
Path 431 | total_timesteps 5140.
Path 432 | total_timesteps 5162.
Path 433 | total_timesteps 5180.
Path 434 | total_timesteps 5190.
Path 435 | total_timesteps 5201.
Path 436 | total_timesteps 5215.
Path 437 | total_timesteps 5224.
Path 438 | total_timesteps 5243.
Path 439 | total_timesteps 5252.
Path 440 | total_timesteps 5268.
Path 441 | total_timesteps 5282.
Path 442 | total_timesteps 5293.
Path 443 | total_timesteps 5307.
Path 444 | total_timesteps 5318.
Path 445 | total_timesteps 5332.
Path 446 | total_timesteps 5345.
Path 447 | total_timesteps 5359.
Path 448 | total_timesteps 5372.
Path 449 | total_timesteps 5380.
Path 450 | total_timesteps 5389.
Path 451 | total_timesteps 5402.
Path 452 | total_timesteps 5417.
Path 453 | total_timesteps 5429.
Path 454 | total_timesteps 5441.
Path 455 | total_timesteps 5451.
Path 456 | total_timesteps 5465.
Path 457 | total_timesteps 5475.
Path 458 | total_timesteps 5484.
Path 459 | total_timesteps 5494.
Path 460 | total_timesteps 5504.
Path 461 | total_timesteps 5515.
Path 462 | total_timesteps 5530.
Path 463 | total_timesteps 5539.
Path 464 | total_timesteps 5550.
Path 465 | total_timesteps 5559.
Path 466 | total_timesteps 5570.
Path 467 | total_timesteps 5583.
Path 468 | total_timesteps 5592.
Path 469 | total_timesteps 5602.
Path 470 | total_timesteps 5613.
Path 471 | total_timesteps 5628.
Path 472 | total_timesteps 5643.
Path 473 | total_timesteps 5659.
Path 474 | total_timesteps 5669.
Path 475 | total_timesteps 5679.
Path 476 | total_timesteps 5688.
Path 477 | total_timesteps 5698.
Path 478 | total_timesteps 5711.
Path 479 | total_timesteps 5719.
Path 480 | total_timesteps 5733.
Path 481 | total_timesteps 5746.
Path 482 | total_timesteps 5762.
Path 483 | total_timesteps 5772.
Path 484 | total_timesteps 5784.
Path 485 | total_timesteps 5797.
Path 486 | total_timesteps 5811.
Path 487 | total_timesteps 5825.
Path 488 | total_timesteps 5845.
Path 489 | total_timesteps 5861.
Path 490 | total_timesteps 5873.
Path 491 | total_timesteps 5885.
Path 492 | total_timesteps 5893.
Path 493 | total_timesteps 5901.
Path 494 | total_timesteps 5917.
Path 495 | total_timesteps 5931.
Path 496 | total_timesteps 5952.
Path 497 | total_timesteps 5965.
Path 498 | total_timesteps 5976.
Path 499 | total_timesteps 5989.
Path 500 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.81    |
| Iteration     | 17       |
| MaximumReturn | 4.92     |
| MinimumReturn | -17.9    |
| TotalSamples  | 76118    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.003650777041912079
Validation loss = 0.0036663361825048923
Validation loss = 0.0033241945784538984
Validation loss = 0.0034259106032550335
Validation loss = 0.0031855255365371704
Validation loss = 0.0032963419798761606
Validation loss = 0.003202640451490879
Validation loss = 0.0031668548472225666
Validation loss = 0.00309577165171504
Validation loss = 0.003841562196612358
Validation loss = 0.0031980497296899557
Validation loss = 0.003172818571329117
Validation loss = 0.003234574804082513
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 87.
Path 9 | total_timesteps 95.
Path 10 | total_timesteps 107.
Path 11 | total_timesteps 116.
Path 12 | total_timesteps 126.
Path 13 | total_timesteps 139.
Path 14 | total_timesteps 150.
Path 15 | total_timesteps 164.
Path 16 | total_timesteps 177.
Path 17 | total_timesteps 188.
Path 18 | total_timesteps 203.
Path 19 | total_timesteps 213.
Path 20 | total_timesteps 231.
Path 21 | total_timesteps 243.
Path 22 | total_timesteps 255.
Path 23 | total_timesteps 269.
Path 24 | total_timesteps 278.
Path 25 | total_timesteps 286.
Path 26 | total_timesteps 295.
Path 27 | total_timesteps 305.
Path 28 | total_timesteps 316.
Path 29 | total_timesteps 326.
Path 30 | total_timesteps 342.
Path 31 | total_timesteps 349.
Path 32 | total_timesteps 361.
Path 33 | total_timesteps 378.
Path 34 | total_timesteps 391.
Path 35 | total_timesteps 402.
Path 36 | total_timesteps 413.
Path 37 | total_timesteps 430.
Path 38 | total_timesteps 440.
Path 39 | total_timesteps 453.
Path 40 | total_timesteps 463.
Path 41 | total_timesteps 472.
Path 42 | total_timesteps 484.
Path 43 | total_timesteps 494.
Path 44 | total_timesteps 509.
Path 45 | total_timesteps 517.
Path 46 | total_timesteps 538.
Path 47 | total_timesteps 551.
Path 48 | total_timesteps 560.
Path 49 | total_timesteps 573.
Path 50 | total_timesteps 583.
Path 51 | total_timesteps 599.
Path 52 | total_timesteps 614.
Path 53 | total_timesteps 623.
Path 54 | total_timesteps 639.
Path 55 | total_timesteps 649.
Path 56 | total_timesteps 659.
Path 57 | total_timesteps 673.
Path 58 | total_timesteps 686.
Path 59 | total_timesteps 698.
Path 60 | total_timesteps 708.
Path 61 | total_timesteps 716.
Path 62 | total_timesteps 728.
Path 63 | total_timesteps 735.
Path 64 | total_timesteps 751.
Path 65 | total_timesteps 759.
Path 66 | total_timesteps 775.
Path 67 | total_timesteps 783.
Path 68 | total_timesteps 794.
Path 69 | total_timesteps 808.
Path 70 | total_timesteps 822.
Path 71 | total_timesteps 833.
Path 72 | total_timesteps 843.
Path 73 | total_timesteps 858.
Path 74 | total_timesteps 869.
Path 75 | total_timesteps 878.
Path 76 | total_timesteps 898.
Path 77 | total_timesteps 911.
Path 78 | total_timesteps 922.
Path 79 | total_timesteps 935.
Path 80 | total_timesteps 945.
Path 81 | total_timesteps 954.
Path 82 | total_timesteps 967.
Path 83 | total_timesteps 975.
Path 84 | total_timesteps 984.
Path 85 | total_timesteps 992.
Path 86 | total_timesteps 1008.
Path 87 | total_timesteps 1020.
Path 88 | total_timesteps 1031.
Path 89 | total_timesteps 1044.
Path 90 | total_timesteps 1053.
Path 91 | total_timesteps 1066.
Path 92 | total_timesteps 1082.
Path 93 | total_timesteps 1095.
Path 94 | total_timesteps 1108.
Path 95 | total_timesteps 1120.
Path 96 | total_timesteps 1130.
Path 97 | total_timesteps 1143.
Path 98 | total_timesteps 1152.
Path 99 | total_timesteps 1167.
Path 100 | total_timesteps 1178.
Path 101 | total_timesteps 1197.
Path 102 | total_timesteps 1207.
Path 103 | total_timesteps 1218.
Path 104 | total_timesteps 1227.
Path 105 | total_timesteps 1235.
Path 106 | total_timesteps 1244.
Path 107 | total_timesteps 1257.
Path 108 | total_timesteps 1269.
Path 109 | total_timesteps 1278.
Path 110 | total_timesteps 1292.
Path 111 | total_timesteps 1305.
Path 112 | total_timesteps 1313.
Path 113 | total_timesteps 1323.
Path 114 | total_timesteps 1336.
Path 115 | total_timesteps 1346.
Path 116 | total_timesteps 1353.
Path 117 | total_timesteps 1366.
Path 118 | total_timesteps 1381.
Path 119 | total_timesteps 1395.
Path 120 | total_timesteps 1402.
Path 121 | total_timesteps 1422.
Path 122 | total_timesteps 1430.
Path 123 | total_timesteps 1442.
Path 124 | total_timesteps 1456.
Path 125 | total_timesteps 1467.
Path 126 | total_timesteps 1479.
Path 127 | total_timesteps 1490.
Path 128 | total_timesteps 1499.
Path 129 | total_timesteps 1506.
Path 130 | total_timesteps 1523.
Path 131 | total_timesteps 1532.
Path 132 | total_timesteps 1544.
Path 133 | total_timesteps 1556.
Path 134 | total_timesteps 1564.
Path 135 | total_timesteps 1585.
Path 136 | total_timesteps 1594.
Path 137 | total_timesteps 1605.
Path 138 | total_timesteps 1619.
Path 139 | total_timesteps 1629.
Path 140 | total_timesteps 1639.
Path 141 | total_timesteps 1646.
Path 142 | total_timesteps 1656.
Path 143 | total_timesteps 1670.
Path 144 | total_timesteps 1681.
Path 145 | total_timesteps 1696.
Path 146 | total_timesteps 1708.
Path 147 | total_timesteps 1718.
Path 148 | total_timesteps 1728.
Path 149 | total_timesteps 1743.
Path 150 | total_timesteps 1759.
Path 151 | total_timesteps 1771.
Path 152 | total_timesteps 1781.
Path 153 | total_timesteps 1793.
Path 154 | total_timesteps 1801.
Path 155 | total_timesteps 1812.
Path 156 | total_timesteps 1824.
Path 157 | total_timesteps 1841.
Path 158 | total_timesteps 1851.
Path 159 | total_timesteps 1860.
Path 160 | total_timesteps 1869.
Path 161 | total_timesteps 1877.
Path 162 | total_timesteps 1892.
Path 163 | total_timesteps 1900.
Path 164 | total_timesteps 1907.
Path 165 | total_timesteps 1925.
Path 166 | total_timesteps 1937.
Path 167 | total_timesteps 1951.
Path 168 | total_timesteps 1959.
Path 169 | total_timesteps 1976.
Path 170 | total_timesteps 1988.
Path 171 | total_timesteps 1998.
Path 172 | total_timesteps 2012.
Path 173 | total_timesteps 2027.
Path 174 | total_timesteps 2040.
Path 175 | total_timesteps 2050.
Path 176 | total_timesteps 2058.
Path 177 | total_timesteps 2067.
Path 178 | total_timesteps 2080.
Path 179 | total_timesteps 2089.
Path 180 | total_timesteps 2099.
Path 181 | total_timesteps 2114.
Path 182 | total_timesteps 2123.
Path 183 | total_timesteps 2133.
Path 184 | total_timesteps 2143.
Path 185 | total_timesteps 2151.
Path 186 | total_timesteps 2160.
Path 187 | total_timesteps 2172.
Path 188 | total_timesteps 2185.
Path 189 | total_timesteps 2194.
Path 190 | total_timesteps 2213.
Path 191 | total_timesteps 2224.
Path 192 | total_timesteps 2234.
Path 193 | total_timesteps 2248.
Path 194 | total_timesteps 2257.
Path 195 | total_timesteps 2271.
Path 196 | total_timesteps 2284.
Path 197 | total_timesteps 2293.
Path 198 | total_timesteps 2302.
Path 199 | total_timesteps 2312.
Path 200 | total_timesteps 2324.
Path 201 | total_timesteps 2333.
Path 202 | total_timesteps 2343.
Path 203 | total_timesteps 2352.
Path 204 | total_timesteps 2361.
Path 205 | total_timesteps 2371.
Path 206 | total_timesteps 2384.
Path 207 | total_timesteps 2395.
Path 208 | total_timesteps 2410.
Path 209 | total_timesteps 2430.
Path 210 | total_timesteps 2439.
Path 211 | total_timesteps 2451.
Path 212 | total_timesteps 2460.
Path 213 | total_timesteps 2469.
Path 214 | total_timesteps 2477.
Path 215 | total_timesteps 2489.
Path 216 | total_timesteps 2500.
Path 217 | total_timesteps 2511.
Path 218 | total_timesteps 2522.
Path 219 | total_timesteps 2534.
Path 220 | total_timesteps 2542.
Path 221 | total_timesteps 2551.
Path 222 | total_timesteps 2561.
Path 223 | total_timesteps 2571.
Path 224 | total_timesteps 2583.
Path 225 | total_timesteps 2592.
Path 226 | total_timesteps 2602.
Path 227 | total_timesteps 2614.
Path 228 | total_timesteps 2631.
Path 229 | total_timesteps 2645.
Path 230 | total_timesteps 2656.
Path 231 | total_timesteps 2668.
Path 232 | total_timesteps 2678.
Path 233 | total_timesteps 2687.
Path 234 | total_timesteps 2698.
Path 235 | total_timesteps 2710.
Path 236 | total_timesteps 2724.
Path 237 | total_timesteps 2736.
Path 238 | total_timesteps 2756.
Path 239 | total_timesteps 2767.
Path 240 | total_timesteps 2783.
Path 241 | total_timesteps 2791.
Path 242 | total_timesteps 2811.
Path 243 | total_timesteps 2823.
Path 244 | total_timesteps 2835.
Path 245 | total_timesteps 2846.
Path 246 | total_timesteps 2856.
Path 247 | total_timesteps 2866.
Path 248 | total_timesteps 2878.
Path 249 | total_timesteps 2887.
Path 250 | total_timesteps 2895.
Path 251 | total_timesteps 2907.
Path 252 | total_timesteps 2918.
Path 253 | total_timesteps 2926.
Path 254 | total_timesteps 2938.
Path 255 | total_timesteps 2948.
Path 256 | total_timesteps 2956.
Path 257 | total_timesteps 2971.
Path 258 | total_timesteps 2982.
Path 259 | total_timesteps 2999.
Path 260 | total_timesteps 3011.
Path 261 | total_timesteps 3023.
Path 262 | total_timesteps 3035.
Path 263 | total_timesteps 3047.
Path 264 | total_timesteps 3058.
Path 265 | total_timesteps 3071.
Path 266 | total_timesteps 3093.
Path 267 | total_timesteps 3100.
Path 268 | total_timesteps 3107.
Path 269 | total_timesteps 3121.
Path 270 | total_timesteps 3132.
Path 271 | total_timesteps 3145.
Path 272 | total_timesteps 3156.
Path 273 | total_timesteps 3165.
Path 274 | total_timesteps 3178.
Path 275 | total_timesteps 3187.
Path 276 | total_timesteps 3197.
Path 277 | total_timesteps 3211.
Path 278 | total_timesteps 3222.
Path 279 | total_timesteps 3231.
Path 280 | total_timesteps 3239.
Path 281 | total_timesteps 3248.
Path 282 | total_timesteps 3258.
Path 283 | total_timesteps 3276.
Path 284 | total_timesteps 3288.
Path 285 | total_timesteps 3303.
Path 286 | total_timesteps 3317.
Path 287 | total_timesteps 3330.
Path 288 | total_timesteps 3344.
Path 289 | total_timesteps 3352.
Path 290 | total_timesteps 3365.
Path 291 | total_timesteps 3373.
Path 292 | total_timesteps 3385.
Path 293 | total_timesteps 3397.
Path 294 | total_timesteps 3412.
Path 295 | total_timesteps 3422.
Path 296 | total_timesteps 3430.
Path 297 | total_timesteps 3440.
Path 298 | total_timesteps 3447.
Path 299 | total_timesteps 3462.
Path 300 | total_timesteps 3470.
Path 301 | total_timesteps 3479.
Path 302 | total_timesteps 3502.
Path 303 | total_timesteps 3511.
Path 304 | total_timesteps 3521.
Path 305 | total_timesteps 3535.
Path 306 | total_timesteps 3549.
Path 307 | total_timesteps 3560.
Path 308 | total_timesteps 3573.
Path 309 | total_timesteps 3585.
Path 310 | total_timesteps 3594.
Path 311 | total_timesteps 3604.
Path 312 | total_timesteps 3621.
Path 313 | total_timesteps 3631.
Path 314 | total_timesteps 3644.
Path 315 | total_timesteps 3654.
Path 316 | total_timesteps 3664.
Path 317 | total_timesteps 3673.
Path 318 | total_timesteps 3688.
Path 319 | total_timesteps 3698.
Path 320 | total_timesteps 3708.
Path 321 | total_timesteps 3723.
Path 322 | total_timesteps 3733.
Path 323 | total_timesteps 3744.
Path 324 | total_timesteps 3756.
Path 325 | total_timesteps 3766.
Path 326 | total_timesteps 3781.
Path 327 | total_timesteps 3794.
Path 328 | total_timesteps 3804.
Path 329 | total_timesteps 3813.
Path 330 | total_timesteps 3829.
Path 331 | total_timesteps 3837.
Path 332 | total_timesteps 3847.
Path 333 | total_timesteps 3858.
Path 334 | total_timesteps 3871.
Path 335 | total_timesteps 3884.
Path 336 | total_timesteps 3899.
Path 337 | total_timesteps 3911.
Path 338 | total_timesteps 3921.
Path 339 | total_timesteps 3938.
Path 340 | total_timesteps 3949.
Path 341 | total_timesteps 3958.
Path 342 | total_timesteps 3970.
Path 343 | total_timesteps 3978.
Path 344 | total_timesteps 3992.
Path 345 | total_timesteps 4005.
Path 346 | total_timesteps 4015.
Path 347 | total_timesteps 4021.
Path 348 | total_timesteps 4036.
Path 349 | total_timesteps 4045.
Path 350 | total_timesteps 4059.
Path 351 | total_timesteps 4071.
Path 352 | total_timesteps 4078.
Path 353 | total_timesteps 4090.
Path 354 | total_timesteps 4102.
Path 355 | total_timesteps 4113.
Path 356 | total_timesteps 4125.
Path 357 | total_timesteps 4137.
Path 358 | total_timesteps 4145.
Path 359 | total_timesteps 4154.
Path 360 | total_timesteps 4162.
Path 361 | total_timesteps 4173.
Path 362 | total_timesteps 4182.
Path 363 | total_timesteps 4198.
Path 364 | total_timesteps 4212.
Path 365 | total_timesteps 4222.
Path 366 | total_timesteps 4232.
Path 367 | total_timesteps 4243.
Path 368 | total_timesteps 4255.
Path 369 | total_timesteps 4264.
Path 370 | total_timesteps 4275.
Path 371 | total_timesteps 4288.
Path 372 | total_timesteps 4308.
Path 373 | total_timesteps 4320.
Path 374 | total_timesteps 4334.
Path 375 | total_timesteps 4343.
Path 376 | total_timesteps 4353.
Path 377 | total_timesteps 4363.
Path 378 | total_timesteps 4371.
Path 379 | total_timesteps 4384.
Path 380 | total_timesteps 4394.
Path 381 | total_timesteps 4406.
Path 382 | total_timesteps 4414.
Path 383 | total_timesteps 4426.
Path 384 | total_timesteps 4435.
Path 385 | total_timesteps 4448.
Path 386 | total_timesteps 4463.
Path 387 | total_timesteps 4474.
Path 388 | total_timesteps 4485.
Path 389 | total_timesteps 4495.
Path 390 | total_timesteps 4508.
Path 391 | total_timesteps 4519.
Path 392 | total_timesteps 4530.
Path 393 | total_timesteps 4540.
Path 394 | total_timesteps 4553.
Path 395 | total_timesteps 4567.
Path 396 | total_timesteps 4575.
Path 397 | total_timesteps 4586.
Path 398 | total_timesteps 4595.
Path 399 | total_timesteps 4610.
Path 400 | total_timesteps 4620.
Path 401 | total_timesteps 4631.
Path 402 | total_timesteps 4641.
Path 403 | total_timesteps 4650.
Path 404 | total_timesteps 4662.
Path 405 | total_timesteps 4679.
Path 406 | total_timesteps 4690.
Path 407 | total_timesteps 4699.
Path 408 | total_timesteps 4711.
Path 409 | total_timesteps 4720.
Path 410 | total_timesteps 4735.
Path 411 | total_timesteps 4752.
Path 412 | total_timesteps 4767.
Path 413 | total_timesteps 4778.
Path 414 | total_timesteps 4787.
Path 415 | total_timesteps 4799.
Path 416 | total_timesteps 4810.
Path 417 | total_timesteps 4830.
Path 418 | total_timesteps 4843.
Path 419 | total_timesteps 4856.
Path 420 | total_timesteps 4865.
Path 421 | total_timesteps 4885.
Path 422 | total_timesteps 4899.
Path 423 | total_timesteps 4906.
Path 424 | total_timesteps 4914.
Path 425 | total_timesteps 4923.
Path 426 | total_timesteps 4940.
Path 427 | total_timesteps 4948.
Path 428 | total_timesteps 4960.
Path 429 | total_timesteps 4973.
Path 430 | total_timesteps 4984.
Path 431 | total_timesteps 4993.
Path 432 | total_timesteps 5002.
Path 433 | total_timesteps 5012.
Path 434 | total_timesteps 5026.
Path 435 | total_timesteps 5036.
Path 436 | total_timesteps 5051.
Path 437 | total_timesteps 5066.
Path 438 | total_timesteps 5075.
Path 439 | total_timesteps 5086.
Path 440 | total_timesteps 5097.
Path 441 | total_timesteps 5110.
Path 442 | total_timesteps 5123.
Path 443 | total_timesteps 5131.
Path 444 | total_timesteps 5142.
Path 445 | total_timesteps 5156.
Path 446 | total_timesteps 5167.
Path 447 | total_timesteps 5185.
Path 448 | total_timesteps 5197.
Path 449 | total_timesteps 5208.
Path 450 | total_timesteps 5221.
Path 451 | total_timesteps 5231.
Path 452 | total_timesteps 5242.
Path 453 | total_timesteps 5250.
Path 454 | total_timesteps 5258.
Path 455 | total_timesteps 5274.
Path 456 | total_timesteps 5284.
Path 457 | total_timesteps 5295.
Path 458 | total_timesteps 5306.
Path 459 | total_timesteps 5313.
Path 460 | total_timesteps 5327.
Path 461 | total_timesteps 5342.
Path 462 | total_timesteps 5355.
Path 463 | total_timesteps 5377.
Path 464 | total_timesteps 5388.
Path 465 | total_timesteps 5400.
Path 466 | total_timesteps 5420.
Path 467 | total_timesteps 5435.
Path 468 | total_timesteps 5442.
Path 469 | total_timesteps 5449.
Path 470 | total_timesteps 5459.
Path 471 | total_timesteps 5473.
Path 472 | total_timesteps 5484.
Path 473 | total_timesteps 5496.
Path 474 | total_timesteps 5514.
Path 475 | total_timesteps 5530.
Path 476 | total_timesteps 5538.
Path 477 | total_timesteps 5547.
Path 478 | total_timesteps 5565.
Path 479 | total_timesteps 5575.
Path 480 | total_timesteps 5584.
Path 481 | total_timesteps 5596.
Path 482 | total_timesteps 5606.
Path 483 | total_timesteps 5618.
Path 484 | total_timesteps 5627.
Path 485 | total_timesteps 5635.
Path 486 | total_timesteps 5650.
Path 487 | total_timesteps 5662.
Path 488 | total_timesteps 5674.
Path 489 | total_timesteps 5686.
Path 490 | total_timesteps 5695.
Path 491 | total_timesteps 5704.
Path 492 | total_timesteps 5726.
Path 493 | total_timesteps 5739.
Path 494 | total_timesteps 5750.
Path 495 | total_timesteps 5762.
Path 496 | total_timesteps 5774.
Path 497 | total_timesteps 5782.
Path 498 | total_timesteps 5792.
Path 499 | total_timesteps 5803.
Path 500 | total_timesteps 5813.
Path 501 | total_timesteps 5823.
Path 502 | total_timesteps 5834.
Path 503 | total_timesteps 5850.
Path 504 | total_timesteps 5866.
Path 505 | total_timesteps 5882.
Path 506 | total_timesteps 5895.
Path 507 | total_timesteps 5905.
Path 508 | total_timesteps 5916.
Path 509 | total_timesteps 5925.
Path 510 | total_timesteps 5935.
Path 511 | total_timesteps 5948.
Path 512 | total_timesteps 5956.
Path 513 | total_timesteps 5970.
Path 514 | total_timesteps 5981.
Path 515 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.61    |
| Iteration     | 18       |
| MaximumReturn | 0.667    |
| MinimumReturn | -19.6    |
| TotalSamples  | 80118    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.0031510679982602596
Validation loss = 0.0031767680775374174
Validation loss = 0.00353009020909667
Validation loss = 0.002861813874915242
Validation loss = 0.0030744862742722034
Validation loss = 0.0028757948894053698
Validation loss = 0.003526359796524048
Validation loss = 0.0029454368632286787
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 61.
Path 7 | total_timesteps 73.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 98.
Path 10 | total_timesteps 112.
Path 11 | total_timesteps 130.
Path 12 | total_timesteps 141.
Path 13 | total_timesteps 152.
Path 14 | total_timesteps 159.
Path 15 | total_timesteps 169.
Path 16 | total_timesteps 178.
Path 17 | total_timesteps 186.
Path 18 | total_timesteps 196.
Path 19 | total_timesteps 206.
Path 20 | total_timesteps 216.
Path 21 | total_timesteps 227.
Path 22 | total_timesteps 242.
Path 23 | total_timesteps 250.
Path 24 | total_timesteps 260.
Path 25 | total_timesteps 274.
Path 26 | total_timesteps 282.
Path 27 | total_timesteps 292.
Path 28 | total_timesteps 301.
Path 29 | total_timesteps 310.
Path 30 | total_timesteps 324.
Path 31 | total_timesteps 336.
Path 32 | total_timesteps 350.
Path 33 | total_timesteps 360.
Path 34 | total_timesteps 373.
Path 35 | total_timesteps 383.
Path 36 | total_timesteps 391.
Path 37 | total_timesteps 400.
Path 38 | total_timesteps 410.
Path 39 | total_timesteps 417.
Path 40 | total_timesteps 425.
Path 41 | total_timesteps 436.
Path 42 | total_timesteps 445.
Path 43 | total_timesteps 455.
Path 44 | total_timesteps 466.
Path 45 | total_timesteps 475.
Path 46 | total_timesteps 486.
Path 47 | total_timesteps 493.
Path 48 | total_timesteps 505.
Path 49 | total_timesteps 514.
Path 50 | total_timesteps 520.
Path 51 | total_timesteps 529.
Path 52 | total_timesteps 540.
Path 53 | total_timesteps 551.
Path 54 | total_timesteps 561.
Path 55 | total_timesteps 573.
Path 56 | total_timesteps 584.
Path 57 | total_timesteps 591.
Path 58 | total_timesteps 602.
Path 59 | total_timesteps 612.
Path 60 | total_timesteps 621.
Path 61 | total_timesteps 634.
Path 62 | total_timesteps 643.
Path 63 | total_timesteps 658.
Path 64 | total_timesteps 670.
Path 65 | total_timesteps 682.
Path 66 | total_timesteps 689.
Path 67 | total_timesteps 696.
Path 68 | total_timesteps 708.
Path 69 | total_timesteps 715.
Path 70 | total_timesteps 723.
Path 71 | total_timesteps 732.
Path 72 | total_timesteps 741.
Path 73 | total_timesteps 754.
Path 74 | total_timesteps 770.
Path 75 | total_timesteps 784.
Path 76 | total_timesteps 791.
Path 77 | total_timesteps 799.
Path 78 | total_timesteps 811.
Path 79 | total_timesteps 820.
Path 80 | total_timesteps 831.
Path 81 | total_timesteps 843.
Path 82 | total_timesteps 852.
Path 83 | total_timesteps 866.
Path 84 | total_timesteps 875.
Path 85 | total_timesteps 886.
Path 86 | total_timesteps 899.
Path 87 | total_timesteps 908.
Path 88 | total_timesteps 917.
Path 89 | total_timesteps 926.
Path 90 | total_timesteps 936.
Path 91 | total_timesteps 945.
Path 92 | total_timesteps 957.
Path 93 | total_timesteps 968.
Path 94 | total_timesteps 978.
Path 95 | total_timesteps 986.
Path 96 | total_timesteps 993.
Path 97 | total_timesteps 1004.
Path 98 | total_timesteps 1012.
Path 99 | total_timesteps 1020.
Path 100 | total_timesteps 1031.
Path 101 | total_timesteps 1044.
Path 102 | total_timesteps 1057.
Path 103 | total_timesteps 1067.
Path 104 | total_timesteps 1077.
Path 105 | total_timesteps 1092.
Path 106 | total_timesteps 1100.
Path 107 | total_timesteps 1112.
Path 108 | total_timesteps 1123.
Path 109 | total_timesteps 1132.
Path 110 | total_timesteps 1140.
Path 111 | total_timesteps 1149.
Path 112 | total_timesteps 1164.
Path 113 | total_timesteps 1174.
Path 114 | total_timesteps 1185.
Path 115 | total_timesteps 1197.
Path 116 | total_timesteps 1205.
Path 117 | total_timesteps 1214.
Path 118 | total_timesteps 1225.
Path 119 | total_timesteps 1244.
Path 120 | total_timesteps 1254.
Path 121 | total_timesteps 1262.
Path 122 | total_timesteps 1271.
Path 123 | total_timesteps 1282.
Path 124 | total_timesteps 1290.
Path 125 | total_timesteps 1299.
Path 126 | total_timesteps 1314.
Path 127 | total_timesteps 1321.
Path 128 | total_timesteps 1330.
Path 129 | total_timesteps 1342.
Path 130 | total_timesteps 1351.
Path 131 | total_timesteps 1360.
Path 132 | total_timesteps 1370.
Path 133 | total_timesteps 1380.
Path 134 | total_timesteps 1392.
Path 135 | total_timesteps 1399.
Path 136 | total_timesteps 1411.
Path 137 | total_timesteps 1422.
Path 138 | total_timesteps 1433.
Path 139 | total_timesteps 1441.
Path 140 | total_timesteps 1452.
Path 141 | total_timesteps 1462.
Path 142 | total_timesteps 1477.
Path 143 | total_timesteps 1485.
Path 144 | total_timesteps 1497.
Path 145 | total_timesteps 1508.
Path 146 | total_timesteps 1520.
Path 147 | total_timesteps 1538.
Path 148 | total_timesteps 1546.
Path 149 | total_timesteps 1553.
Path 150 | total_timesteps 1561.
Path 151 | total_timesteps 1570.
Path 152 | total_timesteps 1579.
Path 153 | total_timesteps 1589.
Path 154 | total_timesteps 1605.
Path 155 | total_timesteps 1612.
Path 156 | total_timesteps 1623.
Path 157 | total_timesteps 1636.
Path 158 | total_timesteps 1643.
Path 159 | total_timesteps 1654.
Path 160 | total_timesteps 1666.
Path 161 | total_timesteps 1680.
Path 162 | total_timesteps 1693.
Path 163 | total_timesteps 1705.
Path 164 | total_timesteps 1715.
Path 165 | total_timesteps 1724.
Path 166 | total_timesteps 1732.
Path 167 | total_timesteps 1740.
Path 168 | total_timesteps 1755.
Path 169 | total_timesteps 1766.
Path 170 | total_timesteps 1777.
Path 171 | total_timesteps 1789.
Path 172 | total_timesteps 1799.
Path 173 | total_timesteps 1807.
Path 174 | total_timesteps 1814.
Path 175 | total_timesteps 1835.
Path 176 | total_timesteps 1843.
Path 177 | total_timesteps 1850.
Path 178 | total_timesteps 1880.
Path 179 | total_timesteps 1898.
Path 180 | total_timesteps 1908.
Path 181 | total_timesteps 1922.
Path 182 | total_timesteps 1929.
Path 183 | total_timesteps 1936.
Path 184 | total_timesteps 1943.
Path 185 | total_timesteps 1951.
Path 186 | total_timesteps 1962.
Path 187 | total_timesteps 1970.
Path 188 | total_timesteps 1979.
Path 189 | total_timesteps 1986.
Path 190 | total_timesteps 1994.
Path 191 | total_timesteps 2002.
Path 192 | total_timesteps 2015.
Path 193 | total_timesteps 2029.
Path 194 | total_timesteps 2038.
Path 195 | total_timesteps 2047.
Path 196 | total_timesteps 2056.
Path 197 | total_timesteps 2068.
Path 198 | total_timesteps 2079.
Path 199 | total_timesteps 2092.
Path 200 | total_timesteps 2102.
Path 201 | total_timesteps 2111.
Path 202 | total_timesteps 2125.
Path 203 | total_timesteps 2139.
Path 204 | total_timesteps 2149.
Path 205 | total_timesteps 2156.
Path 206 | total_timesteps 2164.
Path 207 | total_timesteps 2174.
Path 208 | total_timesteps 2189.
Path 209 | total_timesteps 2200.
Path 210 | total_timesteps 2208.
Path 211 | total_timesteps 2229.
Path 212 | total_timesteps 2239.
Path 213 | total_timesteps 2249.
Path 214 | total_timesteps 2256.
Path 215 | total_timesteps 2264.
Path 216 | total_timesteps 2274.
Path 217 | total_timesteps 2287.
Path 218 | total_timesteps 2295.
Path 219 | total_timesteps 2305.
Path 220 | total_timesteps 2313.
Path 221 | total_timesteps 2323.
Path 222 | total_timesteps 2330.
Path 223 | total_timesteps 2340.
Path 224 | total_timesteps 2352.
Path 225 | total_timesteps 2358.
Path 226 | total_timesteps 2369.
Path 227 | total_timesteps 2377.
Path 228 | total_timesteps 2395.
Path 229 | total_timesteps 2405.
Path 230 | total_timesteps 2415.
Path 231 | total_timesteps 2425.
Path 232 | total_timesteps 2436.
Path 233 | total_timesteps 2449.
Path 234 | total_timesteps 2458.
Path 235 | total_timesteps 2468.
Path 236 | total_timesteps 2478.
Path 237 | total_timesteps 2493.
Path 238 | total_timesteps 2503.
Path 239 | total_timesteps 2511.
Path 240 | total_timesteps 2520.
Path 241 | total_timesteps 2527.
Path 242 | total_timesteps 2534.
Path 243 | total_timesteps 2543.
Path 244 | total_timesteps 2560.
Path 245 | total_timesteps 2573.
Path 246 | total_timesteps 2583.
Path 247 | total_timesteps 2593.
Path 248 | total_timesteps 2600.
Path 249 | total_timesteps 2611.
Path 250 | total_timesteps 2620.
Path 251 | total_timesteps 2631.
Path 252 | total_timesteps 2640.
Path 253 | total_timesteps 2657.
Path 254 | total_timesteps 2667.
Path 255 | total_timesteps 2675.
Path 256 | total_timesteps 2690.
Path 257 | total_timesteps 2699.
Path 258 | total_timesteps 2707.
Path 259 | total_timesteps 2720.
Path 260 | total_timesteps 2733.
Path 261 | total_timesteps 2740.
Path 262 | total_timesteps 2749.
Path 263 | total_timesteps 2758.
Path 264 | total_timesteps 2767.
Path 265 | total_timesteps 2781.
Path 266 | total_timesteps 2791.
Path 267 | total_timesteps 2802.
Path 268 | total_timesteps 2814.
Path 269 | total_timesteps 2822.
Path 270 | total_timesteps 2834.
Path 271 | total_timesteps 2841.
Path 272 | total_timesteps 2850.
Path 273 | total_timesteps 2864.
Path 274 | total_timesteps 2879.
Path 275 | total_timesteps 2889.
Path 276 | total_timesteps 2899.
Path 277 | total_timesteps 2906.
Path 278 | total_timesteps 2917.
Path 279 | total_timesteps 2924.
Path 280 | total_timesteps 2932.
Path 281 | total_timesteps 2942.
Path 282 | total_timesteps 2951.
Path 283 | total_timesteps 2958.
Path 284 | total_timesteps 2972.
Path 285 | total_timesteps 2987.
Path 286 | total_timesteps 2996.
Path 287 | total_timesteps 3019.
Path 288 | total_timesteps 3027.
Path 289 | total_timesteps 3035.
Path 290 | total_timesteps 3043.
Path 291 | total_timesteps 3053.
Path 292 | total_timesteps 3064.
Path 293 | total_timesteps 3075.
Path 294 | total_timesteps 3087.
Path 295 | total_timesteps 3095.
Path 296 | total_timesteps 3106.
Path 297 | total_timesteps 3121.
Path 298 | total_timesteps 3129.
Path 299 | total_timesteps 3151.
Path 300 | total_timesteps 3160.
Path 301 | total_timesteps 3171.
Path 302 | total_timesteps 3181.
Path 303 | total_timesteps 3204.
Path 304 | total_timesteps 3215.
Path 305 | total_timesteps 3224.
Path 306 | total_timesteps 3235.
Path 307 | total_timesteps 3249.
Path 308 | total_timesteps 3262.
Path 309 | total_timesteps 3270.
Path 310 | total_timesteps 3279.
Path 311 | total_timesteps 3290.
Path 312 | total_timesteps 3304.
Path 313 | total_timesteps 3314.
Path 314 | total_timesteps 3327.
Path 315 | total_timesteps 3336.
Path 316 | total_timesteps 3350.
Path 317 | total_timesteps 3360.
Path 318 | total_timesteps 3371.
Path 319 | total_timesteps 3382.
Path 320 | total_timesteps 3393.
Path 321 | total_timesteps 3401.
Path 322 | total_timesteps 3410.
Path 323 | total_timesteps 3424.
Path 324 | total_timesteps 3435.
Path 325 | total_timesteps 3445.
Path 326 | total_timesteps 3453.
Path 327 | total_timesteps 3462.
Path 328 | total_timesteps 3470.
Path 329 | total_timesteps 3479.
Path 330 | total_timesteps 3490.
Path 331 | total_timesteps 3498.
Path 332 | total_timesteps 3507.
Path 333 | total_timesteps 3515.
Path 334 | total_timesteps 3524.
Path 335 | total_timesteps 3535.
Path 336 | total_timesteps 3544.
Path 337 | total_timesteps 3555.
Path 338 | total_timesteps 3564.
Path 339 | total_timesteps 3574.
Path 340 | total_timesteps 3584.
Path 341 | total_timesteps 3595.
Path 342 | total_timesteps 3611.
Path 343 | total_timesteps 3626.
Path 344 | total_timesteps 3637.
Path 345 | total_timesteps 3648.
Path 346 | total_timesteps 3662.
Path 347 | total_timesteps 3675.
Path 348 | total_timesteps 3685.
Path 349 | total_timesteps 3692.
Path 350 | total_timesteps 3714.
Path 351 | total_timesteps 3724.
Path 352 | total_timesteps 3735.
Path 353 | total_timesteps 3745.
Path 354 | total_timesteps 3758.
Path 355 | total_timesteps 3767.
Path 356 | total_timesteps 3778.
Path 357 | total_timesteps 3787.
Path 358 | total_timesteps 3806.
Path 359 | total_timesteps 3816.
Path 360 | total_timesteps 3825.
Path 361 | total_timesteps 3834.
Path 362 | total_timesteps 3844.
Path 363 | total_timesteps 3852.
Path 364 | total_timesteps 3862.
Path 365 | total_timesteps 3873.
Path 366 | total_timesteps 3879.
Path 367 | total_timesteps 3892.
Path 368 | total_timesteps 3900.
Path 369 | total_timesteps 3912.
Path 370 | total_timesteps 3928.
Path 371 | total_timesteps 3936.
Path 372 | total_timesteps 3945.
Path 373 | total_timesteps 3962.
Path 374 | total_timesteps 3976.
Path 375 | total_timesteps 3988.
Path 376 | total_timesteps 3995.
Path 377 | total_timesteps 4006.
Path 378 | total_timesteps 4015.
Path 379 | total_timesteps 4022.
Path 380 | total_timesteps 4029.
Path 381 | total_timesteps 4036.
Path 382 | total_timesteps 4046.
Path 383 | total_timesteps 4058.
Path 384 | total_timesteps 4066.
Path 385 | total_timesteps 4082.
Path 386 | total_timesteps 4093.
Path 387 | total_timesteps 4104.
Path 388 | total_timesteps 4115.
Path 389 | total_timesteps 4123.
Path 390 | total_timesteps 4134.
Path 391 | total_timesteps 4142.
Path 392 | total_timesteps 4150.
Path 393 | total_timesteps 4160.
Path 394 | total_timesteps 4173.
Path 395 | total_timesteps 4181.
Path 396 | total_timesteps 4190.
Path 397 | total_timesteps 4199.
Path 398 | total_timesteps 4208.
Path 399 | total_timesteps 4221.
Path 400 | total_timesteps 4235.
Path 401 | total_timesteps 4243.
Path 402 | total_timesteps 4256.
Path 403 | total_timesteps 4262.
Path 404 | total_timesteps 4271.
Path 405 | total_timesteps 4278.
Path 406 | total_timesteps 4291.
Path 407 | total_timesteps 4301.
Path 408 | total_timesteps 4312.
Path 409 | total_timesteps 4321.
Path 410 | total_timesteps 4332.
Path 411 | total_timesteps 4342.
Path 412 | total_timesteps 4354.
Path 413 | total_timesteps 4365.
Path 414 | total_timesteps 4377.
Path 415 | total_timesteps 4389.
Path 416 | total_timesteps 4397.
Path 417 | total_timesteps 4406.
Path 418 | total_timesteps 4415.
Path 419 | total_timesteps 4432.
Path 420 | total_timesteps 4444.
Path 421 | total_timesteps 4452.
Path 422 | total_timesteps 4463.
Path 423 | total_timesteps 4478.
Path 424 | total_timesteps 4485.
Path 425 | total_timesteps 4492.
Path 426 | total_timesteps 4502.
Path 427 | total_timesteps 4516.
Path 428 | total_timesteps 4526.
Path 429 | total_timesteps 4537.
Path 430 | total_timesteps 4544.
Path 431 | total_timesteps 4556.
Path 432 | total_timesteps 4565.
Path 433 | total_timesteps 4572.
Path 434 | total_timesteps 4583.
Path 435 | total_timesteps 4591.
Path 436 | total_timesteps 4600.
Path 437 | total_timesteps 4609.
Path 438 | total_timesteps 4618.
Path 439 | total_timesteps 4626.
Path 440 | total_timesteps 4634.
Path 441 | total_timesteps 4643.
Path 442 | total_timesteps 4652.
Path 443 | total_timesteps 4662.
Path 444 | total_timesteps 4675.
Path 445 | total_timesteps 4684.
Path 446 | total_timesteps 4694.
Path 447 | total_timesteps 4706.
Path 448 | total_timesteps 4714.
Path 449 | total_timesteps 4732.
Path 450 | total_timesteps 4743.
Path 451 | total_timesteps 4750.
Path 452 | total_timesteps 4760.
Path 453 | total_timesteps 4771.
Path 454 | total_timesteps 4786.
Path 455 | total_timesteps 4797.
Path 456 | total_timesteps 4806.
Path 457 | total_timesteps 4818.
Path 458 | total_timesteps 4826.
Path 459 | total_timesteps 4836.
Path 460 | total_timesteps 4847.
Path 461 | total_timesteps 4854.
Path 462 | total_timesteps 4862.
Path 463 | total_timesteps 4874.
Path 464 | total_timesteps 4883.
Path 465 | total_timesteps 4890.
Path 466 | total_timesteps 4899.
Path 467 | total_timesteps 4912.
Path 468 | total_timesteps 4925.
Path 469 | total_timesteps 4934.
Path 470 | total_timesteps 4944.
Path 471 | total_timesteps 4953.
Path 472 | total_timesteps 4963.
Path 473 | total_timesteps 4975.
Path 474 | total_timesteps 4985.
Path 475 | total_timesteps 4995.
Path 476 | total_timesteps 5006.
Path 477 | total_timesteps 5020.
Path 478 | total_timesteps 5027.
Path 479 | total_timesteps 5034.
Path 480 | total_timesteps 5043.
Path 481 | total_timesteps 5054.
Path 482 | total_timesteps 5064.
Path 483 | total_timesteps 5077.
Path 484 | total_timesteps 5088.
Path 485 | total_timesteps 5107.
Path 486 | total_timesteps 5117.
Path 487 | total_timesteps 5129.
Path 488 | total_timesteps 5141.
Path 489 | total_timesteps 5154.
Path 490 | total_timesteps 5169.
Path 491 | total_timesteps 5177.
Path 492 | total_timesteps 5191.
Path 493 | total_timesteps 5200.
Path 494 | total_timesteps 5208.
Path 495 | total_timesteps 5219.
Path 496 | total_timesteps 5226.
Path 497 | total_timesteps 5239.
Path 498 | total_timesteps 5247.
Path 499 | total_timesteps 5261.
Path 500 | total_timesteps 5273.
Path 501 | total_timesteps 5280.
Path 502 | total_timesteps 5292.
Path 503 | total_timesteps 5307.
Path 504 | total_timesteps 5318.
Path 505 | total_timesteps 5329.
Path 506 | total_timesteps 5340.
Path 507 | total_timesteps 5348.
Path 508 | total_timesteps 5357.
Path 509 | total_timesteps 5370.
Path 510 | total_timesteps 5387.
Path 511 | total_timesteps 5398.
Path 512 | total_timesteps 5406.
Path 513 | total_timesteps 5416.
Path 514 | total_timesteps 5426.
Path 515 | total_timesteps 5437.
Path 516 | total_timesteps 5450.
Path 517 | total_timesteps 5461.
Path 518 | total_timesteps 5471.
Path 519 | total_timesteps 5478.
Path 520 | total_timesteps 5491.
Path 521 | total_timesteps 5500.
Path 522 | total_timesteps 5511.
Path 523 | total_timesteps 5521.
Path 524 | total_timesteps 5530.
Path 525 | total_timesteps 5545.
Path 526 | total_timesteps 5554.
Path 527 | total_timesteps 5563.
Path 528 | total_timesteps 5571.
Path 529 | total_timesteps 5579.
Path 530 | total_timesteps 5586.
Path 531 | total_timesteps 5595.
Path 532 | total_timesteps 5611.
Path 533 | total_timesteps 5626.
Path 534 | total_timesteps 5640.
Path 535 | total_timesteps 5650.
Path 536 | total_timesteps 5673.
Path 537 | total_timesteps 5683.
Path 538 | total_timesteps 5699.
Path 539 | total_timesteps 5713.
Path 540 | total_timesteps 5725.
Path 541 | total_timesteps 5736.
Path 542 | total_timesteps 5746.
Path 543 | total_timesteps 5754.
Path 544 | total_timesteps 5770.
Path 545 | total_timesteps 5784.
Path 546 | total_timesteps 5795.
Path 547 | total_timesteps 5804.
Path 548 | total_timesteps 5815.
Path 549 | total_timesteps 5823.
Path 550 | total_timesteps 5834.
Path 551 | total_timesteps 5843.
Path 552 | total_timesteps 5856.
Path 553 | total_timesteps 5864.
Path 554 | total_timesteps 5874.
Path 555 | total_timesteps 5882.
Path 556 | total_timesteps 5888.
Path 557 | total_timesteps 5896.
Path 558 | total_timesteps 5904.
Path 559 | total_timesteps 5915.
Path 560 | total_timesteps 5925.
Path 561 | total_timesteps 5939.
Path 562 | total_timesteps 5947.
Path 563 | total_timesteps 5953.
Path 564 | total_timesteps 5961.
Path 565 | total_timesteps 5974.
Path 566 | total_timesteps 5981.
Path 567 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.21    |
| Iteration     | 19       |
| MaximumReturn | -0.619   |
| MinimumReturn | -16.4    |
| TotalSamples  | 84120    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.0029861079528927803
Validation loss = 0.0028937426395714283
Validation loss = 0.0028569737914949656
Validation loss = 0.0029834015294909477
Validation loss = 0.0026655199471861124
Validation loss = 0.0028997205663472414
Validation loss = 0.0026340221520513296
Validation loss = 0.0030436930246651173
Validation loss = 0.0030419875402003527
Validation loss = 0.0029108726885169744
Validation loss = 0.0026901732198894024
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 77.
Path 8 | total_timesteps 95.
Path 9 | total_timesteps 103.
Path 10 | total_timesteps 112.
Path 11 | total_timesteps 121.
Path 12 | total_timesteps 130.
Path 13 | total_timesteps 140.
Path 14 | total_timesteps 155.
Path 15 | total_timesteps 167.
Path 16 | total_timesteps 175.
Path 17 | total_timesteps 184.
Path 18 | total_timesteps 191.
Path 19 | total_timesteps 201.
Path 20 | total_timesteps 210.
Path 21 | total_timesteps 221.
Path 22 | total_timesteps 231.
Path 23 | total_timesteps 240.
Path 24 | total_timesteps 250.
Path 25 | total_timesteps 259.
Path 26 | total_timesteps 270.
Path 27 | total_timesteps 278.
Path 28 | total_timesteps 287.
Path 29 | total_timesteps 296.
Path 30 | total_timesteps 306.
Path 31 | total_timesteps 315.
Path 32 | total_timesteps 325.
Path 33 | total_timesteps 342.
Path 34 | total_timesteps 352.
Path 35 | total_timesteps 361.
Path 36 | total_timesteps 370.
Path 37 | total_timesteps 380.
Path 38 | total_timesteps 392.
Path 39 | total_timesteps 399.
Path 40 | total_timesteps 409.
Path 41 | total_timesteps 418.
Path 42 | total_timesteps 425.
Path 43 | total_timesteps 437.
Path 44 | total_timesteps 448.
Path 45 | total_timesteps 455.
Path 46 | total_timesteps 463.
Path 47 | total_timesteps 471.
Path 48 | total_timesteps 482.
Path 49 | total_timesteps 493.
Path 50 | total_timesteps 502.
Path 51 | total_timesteps 511.
Path 52 | total_timesteps 524.
Path 53 | total_timesteps 533.
Path 54 | total_timesteps 542.
Path 55 | total_timesteps 550.
Path 56 | total_timesteps 559.
Path 57 | total_timesteps 567.
Path 58 | total_timesteps 574.
Path 59 | total_timesteps 586.
Path 60 | total_timesteps 597.
Path 61 | total_timesteps 605.
Path 62 | total_timesteps 614.
Path 63 | total_timesteps 627.
Path 64 | total_timesteps 639.
Path 65 | total_timesteps 649.
Path 66 | total_timesteps 661.
Path 67 | total_timesteps 673.
Path 68 | total_timesteps 684.
Path 69 | total_timesteps 693.
Path 70 | total_timesteps 704.
Path 71 | total_timesteps 712.
Path 72 | total_timesteps 723.
Path 73 | total_timesteps 731.
Path 74 | total_timesteps 742.
Path 75 | total_timesteps 752.
Path 76 | total_timesteps 761.
Path 77 | total_timesteps 769.
Path 78 | total_timesteps 784.
Path 79 | total_timesteps 794.
Path 80 | total_timesteps 805.
Path 81 | total_timesteps 820.
Path 82 | total_timesteps 834.
Path 83 | total_timesteps 844.
Path 84 | total_timesteps 857.
Path 85 | total_timesteps 868.
Path 86 | total_timesteps 879.
Path 87 | total_timesteps 887.
Path 88 | total_timesteps 899.
Path 89 | total_timesteps 913.
Path 90 | total_timesteps 936.
Path 91 | total_timesteps 947.
Path 92 | total_timesteps 958.
Path 93 | total_timesteps 969.
Path 94 | total_timesteps 977.
Path 95 | total_timesteps 988.
Path 96 | total_timesteps 996.
Path 97 | total_timesteps 1005.
Path 98 | total_timesteps 1012.
Path 99 | total_timesteps 1024.
Path 100 | total_timesteps 1042.
Path 101 | total_timesteps 1053.
Path 102 | total_timesteps 1059.
Path 103 | total_timesteps 1071.
Path 104 | total_timesteps 1087.
Path 105 | total_timesteps 1094.
Path 106 | total_timesteps 1103.
Path 107 | total_timesteps 1113.
Path 108 | total_timesteps 1124.
Path 109 | total_timesteps 1135.
Path 110 | total_timesteps 1148.
Path 111 | total_timesteps 1157.
Path 112 | total_timesteps 1164.
Path 113 | total_timesteps 1174.
Path 114 | total_timesteps 1185.
Path 115 | total_timesteps 1195.
Path 116 | total_timesteps 1203.
Path 117 | total_timesteps 1211.
Path 118 | total_timesteps 1221.
Path 119 | total_timesteps 1232.
Path 120 | total_timesteps 1240.
Path 121 | total_timesteps 1250.
Path 122 | total_timesteps 1260.
Path 123 | total_timesteps 1268.
Path 124 | total_timesteps 1277.
Path 125 | total_timesteps 1289.
Path 126 | total_timesteps 1300.
Path 127 | total_timesteps 1310.
Path 128 | total_timesteps 1322.
Path 129 | total_timesteps 1329.
Path 130 | total_timesteps 1337.
Path 131 | total_timesteps 1346.
Path 132 | total_timesteps 1358.
Path 133 | total_timesteps 1367.
Path 134 | total_timesteps 1376.
Path 135 | total_timesteps 1386.
Path 136 | total_timesteps 1396.
Path 137 | total_timesteps 1410.
Path 138 | total_timesteps 1418.
Path 139 | total_timesteps 1432.
Path 140 | total_timesteps 1443.
Path 141 | total_timesteps 1455.
Path 142 | total_timesteps 1464.
Path 143 | total_timesteps 1472.
Path 144 | total_timesteps 1480.
Path 145 | total_timesteps 1490.
Path 146 | total_timesteps 1503.
Path 147 | total_timesteps 1513.
Path 148 | total_timesteps 1527.
Path 149 | total_timesteps 1538.
Path 150 | total_timesteps 1547.
Path 151 | total_timesteps 1560.
Path 152 | total_timesteps 1570.
Path 153 | total_timesteps 1582.
Path 154 | total_timesteps 1591.
Path 155 | total_timesteps 1600.
Path 156 | total_timesteps 1609.
Path 157 | total_timesteps 1617.
Path 158 | total_timesteps 1632.
Path 159 | total_timesteps 1640.
Path 160 | total_timesteps 1654.
Path 161 | total_timesteps 1667.
Path 162 | total_timesteps 1675.
Path 163 | total_timesteps 1682.
Path 164 | total_timesteps 1695.
Path 165 | total_timesteps 1706.
Path 166 | total_timesteps 1716.
Path 167 | total_timesteps 1736.
Path 168 | total_timesteps 1746.
Path 169 | total_timesteps 1753.
Path 170 | total_timesteps 1760.
Path 171 | total_timesteps 1768.
Path 172 | total_timesteps 1779.
Path 173 | total_timesteps 1789.
Path 174 | total_timesteps 1800.
Path 175 | total_timesteps 1807.
Path 176 | total_timesteps 1815.
Path 177 | total_timesteps 1822.
Path 178 | total_timesteps 1831.
Path 179 | total_timesteps 1842.
Path 180 | total_timesteps 1852.
Path 181 | total_timesteps 1863.
Path 182 | total_timesteps 1872.
Path 183 | total_timesteps 1879.
Path 184 | total_timesteps 1890.
Path 185 | total_timesteps 1900.
Path 186 | total_timesteps 1910.
Path 187 | total_timesteps 1922.
Path 188 | total_timesteps 1931.
Path 189 | total_timesteps 1939.
Path 190 | total_timesteps 1947.
Path 191 | total_timesteps 1957.
Path 192 | total_timesteps 1970.
Path 193 | total_timesteps 1980.
Path 194 | total_timesteps 1991.
Path 195 | total_timesteps 2003.
Path 196 | total_timesteps 2014.
Path 197 | total_timesteps 2025.
Path 198 | total_timesteps 2035.
Path 199 | total_timesteps 2043.
Path 200 | total_timesteps 2052.
Path 201 | total_timesteps 2059.
Path 202 | total_timesteps 2066.
Path 203 | total_timesteps 2074.
Path 204 | total_timesteps 2085.
Path 205 | total_timesteps 2097.
Path 206 | total_timesteps 2105.
Path 207 | total_timesteps 2113.
Path 208 | total_timesteps 2121.
Path 209 | total_timesteps 2128.
Path 210 | total_timesteps 2135.
Path 211 | total_timesteps 2144.
Path 212 | total_timesteps 2154.
Path 213 | total_timesteps 2164.
Path 214 | total_timesteps 2172.
Path 215 | total_timesteps 2187.
Path 216 | total_timesteps 2199.
Path 217 | total_timesteps 2208.
Path 218 | total_timesteps 2216.
Path 219 | total_timesteps 2228.
Path 220 | total_timesteps 2240.
Path 221 | total_timesteps 2249.
Path 222 | total_timesteps 2259.
Path 223 | total_timesteps 2268.
Path 224 | total_timesteps 2278.
Path 225 | total_timesteps 2289.
Path 226 | total_timesteps 2301.
Path 227 | total_timesteps 2310.
Path 228 | total_timesteps 2321.
Path 229 | total_timesteps 2334.
Path 230 | total_timesteps 2341.
Path 231 | total_timesteps 2350.
Path 232 | total_timesteps 2360.
Path 233 | total_timesteps 2373.
Path 234 | total_timesteps 2382.
Path 235 | total_timesteps 2391.
Path 236 | total_timesteps 2402.
Path 237 | total_timesteps 2409.
Path 238 | total_timesteps 2419.
Path 239 | total_timesteps 2429.
Path 240 | total_timesteps 2442.
Path 241 | total_timesteps 2451.
Path 242 | total_timesteps 2462.
Path 243 | total_timesteps 2475.
Path 244 | total_timesteps 2485.
Path 245 | total_timesteps 2496.
Path 246 | total_timesteps 2504.
Path 247 | total_timesteps 2514.
Path 248 | total_timesteps 2524.
Path 249 | total_timesteps 2536.
Path 250 | total_timesteps 2546.
Path 251 | total_timesteps 2554.
Path 252 | total_timesteps 2564.
Path 253 | total_timesteps 2573.
Path 254 | total_timesteps 2582.
Path 255 | total_timesteps 2590.
Path 256 | total_timesteps 2599.
Path 257 | total_timesteps 2613.
Path 258 | total_timesteps 2620.
Path 259 | total_timesteps 2629.
Path 260 | total_timesteps 2637.
Path 261 | total_timesteps 2646.
Path 262 | total_timesteps 2657.
Path 263 | total_timesteps 2670.
Path 264 | total_timesteps 2677.
Path 265 | total_timesteps 2684.
Path 266 | total_timesteps 2696.
Path 267 | total_timesteps 2705.
Path 268 | total_timesteps 2719.
Path 269 | total_timesteps 2727.
Path 270 | total_timesteps 2739.
Path 271 | total_timesteps 2747.
Path 272 | total_timesteps 2753.
Path 273 | total_timesteps 2762.
Path 274 | total_timesteps 2773.
Path 275 | total_timesteps 2782.
Path 276 | total_timesteps 2791.
Path 277 | total_timesteps 2798.
Path 278 | total_timesteps 2807.
Path 279 | total_timesteps 2818.
Path 280 | total_timesteps 2827.
Path 281 | total_timesteps 2838.
Path 282 | total_timesteps 2847.
Path 283 | total_timesteps 2859.
Path 284 | total_timesteps 2873.
Path 285 | total_timesteps 2885.
Path 286 | total_timesteps 2897.
Path 287 | total_timesteps 2906.
Path 288 | total_timesteps 2917.
Path 289 | total_timesteps 2926.
Path 290 | total_timesteps 2933.
Path 291 | total_timesteps 2943.
Path 292 | total_timesteps 2953.
Path 293 | total_timesteps 2968.
Path 294 | total_timesteps 2976.
Path 295 | total_timesteps 2989.
Path 296 | total_timesteps 3003.
Path 297 | total_timesteps 3012.
Path 298 | total_timesteps 3020.
Path 299 | total_timesteps 3031.
Path 300 | total_timesteps 3042.
Path 301 | total_timesteps 3055.
Path 302 | total_timesteps 3064.
Path 303 | total_timesteps 3076.
Path 304 | total_timesteps 3087.
Path 305 | total_timesteps 3098.
Path 306 | total_timesteps 3111.
Path 307 | total_timesteps 3121.
Path 308 | total_timesteps 3132.
Path 309 | total_timesteps 3142.
Path 310 | total_timesteps 3151.
Path 311 | total_timesteps 3163.
Path 312 | total_timesteps 3172.
Path 313 | total_timesteps 3180.
Path 314 | total_timesteps 3191.
Path 315 | total_timesteps 3201.
Path 316 | total_timesteps 3210.
Path 317 | total_timesteps 3221.
Path 318 | total_timesteps 3233.
Path 319 | total_timesteps 3245.
Path 320 | total_timesteps 3256.
Path 321 | total_timesteps 3281.
Path 322 | total_timesteps 3291.
Path 323 | total_timesteps 3299.
Path 324 | total_timesteps 3309.
Path 325 | total_timesteps 3318.
Path 326 | total_timesteps 3328.
Path 327 | total_timesteps 3336.
Path 328 | total_timesteps 3346.
Path 329 | total_timesteps 3360.
Path 330 | total_timesteps 3374.
Path 331 | total_timesteps 3382.
Path 332 | total_timesteps 3391.
Path 333 | total_timesteps 3400.
Path 334 | total_timesteps 3410.
Path 335 | total_timesteps 3419.
Path 336 | total_timesteps 3426.
Path 337 | total_timesteps 3434.
Path 338 | total_timesteps 3443.
Path 339 | total_timesteps 3452.
Path 340 | total_timesteps 3461.
Path 341 | total_timesteps 3472.
Path 342 | total_timesteps 3480.
Path 343 | total_timesteps 3490.
Path 344 | total_timesteps 3502.
Path 345 | total_timesteps 3513.
Path 346 | total_timesteps 3520.
Path 347 | total_timesteps 3530.
Path 348 | total_timesteps 3540.
Path 349 | total_timesteps 3548.
Path 350 | total_timesteps 3559.
Path 351 | total_timesteps 3570.
Path 352 | total_timesteps 3577.
Path 353 | total_timesteps 3587.
Path 354 | total_timesteps 3596.
Path 355 | total_timesteps 3605.
Path 356 | total_timesteps 3613.
Path 357 | total_timesteps 3624.
Path 358 | total_timesteps 3644.
Path 359 | total_timesteps 3658.
Path 360 | total_timesteps 3667.
Path 361 | total_timesteps 3674.
Path 362 | total_timesteps 3689.
Path 363 | total_timesteps 3717.
Path 364 | total_timesteps 3728.
Path 365 | total_timesteps 3735.
Path 366 | total_timesteps 3746.
Path 367 | total_timesteps 3762.
Path 368 | total_timesteps 3773.
Path 369 | total_timesteps 3785.
Path 370 | total_timesteps 3796.
Path 371 | total_timesteps 3804.
Path 372 | total_timesteps 3811.
Path 373 | total_timesteps 3821.
Path 374 | total_timesteps 3832.
Path 375 | total_timesteps 3849.
Path 376 | total_timesteps 3864.
Path 377 | total_timesteps 3871.
Path 378 | total_timesteps 3883.
Path 379 | total_timesteps 3893.
Path 380 | total_timesteps 3907.
Path 381 | total_timesteps 3915.
Path 382 | total_timesteps 3926.
Path 383 | total_timesteps 3939.
Path 384 | total_timesteps 3948.
Path 385 | total_timesteps 3956.
Path 386 | total_timesteps 3964.
Path 387 | total_timesteps 3974.
Path 388 | total_timesteps 3986.
Path 389 | total_timesteps 3995.
Path 390 | total_timesteps 4006.
Path 391 | total_timesteps 4015.
Path 392 | total_timesteps 4023.
Path 393 | total_timesteps 4039.
Path 394 | total_timesteps 4051.
Path 395 | total_timesteps 4061.
Path 396 | total_timesteps 4069.
Path 397 | total_timesteps 4076.
Path 398 | total_timesteps 4086.
Path 399 | total_timesteps 4099.
Path 400 | total_timesteps 4111.
Path 401 | total_timesteps 4127.
Path 402 | total_timesteps 4135.
Path 403 | total_timesteps 4143.
Path 404 | total_timesteps 4153.
Path 405 | total_timesteps 4161.
Path 406 | total_timesteps 4172.
Path 407 | total_timesteps 4181.
Path 408 | total_timesteps 4201.
Path 409 | total_timesteps 4208.
Path 410 | total_timesteps 4217.
Path 411 | total_timesteps 4226.
Path 412 | total_timesteps 4237.
Path 413 | total_timesteps 4247.
Path 414 | total_timesteps 4255.
Path 415 | total_timesteps 4263.
Path 416 | total_timesteps 4271.
Path 417 | total_timesteps 4282.
Path 418 | total_timesteps 4291.
Path 419 | total_timesteps 4301.
Path 420 | total_timesteps 4312.
Path 421 | total_timesteps 4323.
Path 422 | total_timesteps 4333.
Path 423 | total_timesteps 4341.
Path 424 | total_timesteps 4349.
Path 425 | total_timesteps 4361.
Path 426 | total_timesteps 4375.
Path 427 | total_timesteps 4384.
Path 428 | total_timesteps 4393.
Path 429 | total_timesteps 4404.
Path 430 | total_timesteps 4412.
Path 431 | total_timesteps 4426.
Path 432 | total_timesteps 4434.
Path 433 | total_timesteps 4444.
Path 434 | total_timesteps 4460.
Path 435 | total_timesteps 4471.
Path 436 | total_timesteps 4480.
Path 437 | total_timesteps 4490.
Path 438 | total_timesteps 4497.
Path 439 | total_timesteps 4509.
Path 440 | total_timesteps 4519.
Path 441 | total_timesteps 4533.
Path 442 | total_timesteps 4546.
Path 443 | total_timesteps 4552.
Path 444 | total_timesteps 4562.
Path 445 | total_timesteps 4573.
Path 446 | total_timesteps 4586.
Path 447 | total_timesteps 4593.
Path 448 | total_timesteps 4601.
Path 449 | total_timesteps 4611.
Path 450 | total_timesteps 4619.
Path 451 | total_timesteps 4627.
Path 452 | total_timesteps 4633.
Path 453 | total_timesteps 4640.
Path 454 | total_timesteps 4651.
Path 455 | total_timesteps 4659.
Path 456 | total_timesteps 4668.
Path 457 | total_timesteps 4676.
Path 458 | total_timesteps 4689.
Path 459 | total_timesteps 4697.
Path 460 | total_timesteps 4705.
Path 461 | total_timesteps 4712.
Path 462 | total_timesteps 4722.
Path 463 | total_timesteps 4733.
Path 464 | total_timesteps 4742.
Path 465 | total_timesteps 4754.
Path 466 | total_timesteps 4765.
Path 467 | total_timesteps 4775.
Path 468 | total_timesteps 4784.
Path 469 | total_timesteps 4792.
Path 470 | total_timesteps 4805.
Path 471 | total_timesteps 4813.
Path 472 | total_timesteps 4819.
Path 473 | total_timesteps 4827.
Path 474 | total_timesteps 4849.
Path 475 | total_timesteps 4859.
Path 476 | total_timesteps 4870.
Path 477 | total_timesteps 4881.
Path 478 | total_timesteps 4890.
Path 479 | total_timesteps 4901.
Path 480 | total_timesteps 4910.
Path 481 | total_timesteps 4918.
Path 482 | total_timesteps 4929.
Path 483 | total_timesteps 4937.
Path 484 | total_timesteps 4944.
Path 485 | total_timesteps 4955.
Path 486 | total_timesteps 4963.
Path 487 | total_timesteps 4974.
Path 488 | total_timesteps 4985.
Path 489 | total_timesteps 4999.
Path 490 | total_timesteps 5007.
Path 491 | total_timesteps 5016.
Path 492 | total_timesteps 5027.
Path 493 | total_timesteps 5039.
Path 494 | total_timesteps 5049.
Path 495 | total_timesteps 5057.
Path 496 | total_timesteps 5076.
Path 497 | total_timesteps 5083.
Path 498 | total_timesteps 5091.
Path 499 | total_timesteps 5103.
Path 500 | total_timesteps 5113.
Path 501 | total_timesteps 5123.
Path 502 | total_timesteps 5132.
Path 503 | total_timesteps 5140.
Path 504 | total_timesteps 5149.
Path 505 | total_timesteps 5158.
Path 506 | total_timesteps 5167.
Path 507 | total_timesteps 5180.
Path 508 | total_timesteps 5193.
Path 509 | total_timesteps 5202.
Path 510 | total_timesteps 5215.
Path 511 | total_timesteps 5224.
Path 512 | total_timesteps 5235.
Path 513 | total_timesteps 5244.
Path 514 | total_timesteps 5255.
Path 515 | total_timesteps 5267.
Path 516 | total_timesteps 5276.
Path 517 | total_timesteps 5289.
Path 518 | total_timesteps 5297.
Path 519 | total_timesteps 5304.
Path 520 | total_timesteps 5315.
Path 521 | total_timesteps 5323.
Path 522 | total_timesteps 5335.
Path 523 | total_timesteps 5344.
Path 524 | total_timesteps 5355.
Path 525 | total_timesteps 5363.
Path 526 | total_timesteps 5373.
Path 527 | total_timesteps 5382.
Path 528 | total_timesteps 5392.
Path 529 | total_timesteps 5399.
Path 530 | total_timesteps 5408.
Path 531 | total_timesteps 5419.
Path 532 | total_timesteps 5432.
Path 533 | total_timesteps 5442.
Path 534 | total_timesteps 5450.
Path 535 | total_timesteps 5459.
Path 536 | total_timesteps 5473.
Path 537 | total_timesteps 5482.
Path 538 | total_timesteps 5495.
Path 539 | total_timesteps 5509.
Path 540 | total_timesteps 5519.
Path 541 | total_timesteps 5528.
Path 542 | total_timesteps 5538.
Path 543 | total_timesteps 5548.
Path 544 | total_timesteps 5557.
Path 545 | total_timesteps 5567.
Path 546 | total_timesteps 5578.
Path 547 | total_timesteps 5588.
Path 548 | total_timesteps 5599.
Path 549 | total_timesteps 5607.
Path 550 | total_timesteps 5619.
Path 551 | total_timesteps 5631.
Path 552 | total_timesteps 5638.
Path 553 | total_timesteps 5647.
Path 554 | total_timesteps 5655.
Path 555 | total_timesteps 5673.
Path 556 | total_timesteps 5682.
Path 557 | total_timesteps 5693.
Path 558 | total_timesteps 5702.
Path 559 | total_timesteps 5713.
Path 560 | total_timesteps 5722.
Path 561 | total_timesteps 5732.
Path 562 | total_timesteps 5740.
Path 563 | total_timesteps 5747.
Path 564 | total_timesteps 5755.
Path 565 | total_timesteps 5764.
Path 566 | total_timesteps 5774.
Path 567 | total_timesteps 5782.
Path 568 | total_timesteps 5792.
Path 569 | total_timesteps 5803.
Path 570 | total_timesteps 5812.
Path 571 | total_timesteps 5820.
Path 572 | total_timesteps 5828.
Path 573 | total_timesteps 5839.
Path 574 | total_timesteps 5851.
Path 575 | total_timesteps 5861.
Path 576 | total_timesteps 5870.
Path 577 | total_timesteps 5878.
Path 578 | total_timesteps 5894.
Path 579 | total_timesteps 5903.
Path 580 | total_timesteps 5910.
Path 581 | total_timesteps 5925.
Path 582 | total_timesteps 5934.
Path 583 | total_timesteps 5942.
Path 584 | total_timesteps 5950.
Path 585 | total_timesteps 5961.
Path 586 | total_timesteps 5973.
Path 587 | total_timesteps 5981.
Path 588 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.04    |
| Iteration     | 20       |
| MaximumReturn | 2.16     |
| MinimumReturn | -15.8    |
| TotalSamples  | 88121    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.0027346396818757057
Validation loss = 0.0027719067875295877
Validation loss = 0.002813831204548478
Validation loss = 0.002679041586816311
Validation loss = 0.0025725490413606167
Validation loss = 0.0027525199111551046
Validation loss = 0.0027841031551361084
Validation loss = 0.00283664558082819
Validation loss = 0.002453703200444579
Validation loss = 0.002540872199460864
Validation loss = 0.0025620644446462393
Validation loss = 0.0026000940706580877
Validation loss = 0.0027194309514015913
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 86.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 130.
Path 14 | total_timesteps 137.
Path 15 | total_timesteps 149.
Path 16 | total_timesteps 166.
Path 17 | total_timesteps 174.
Path 18 | total_timesteps 183.
Path 19 | total_timesteps 192.
Path 20 | total_timesteps 200.
Path 21 | total_timesteps 210.
Path 22 | total_timesteps 226.
Path 23 | total_timesteps 234.
Path 24 | total_timesteps 244.
Path 25 | total_timesteps 251.
Path 26 | total_timesteps 258.
Path 27 | total_timesteps 266.
Path 28 | total_timesteps 274.
Path 29 | total_timesteps 285.
Path 30 | total_timesteps 298.
Path 31 | total_timesteps 311.
Path 32 | total_timesteps 322.
Path 33 | total_timesteps 330.
Path 34 | total_timesteps 342.
Path 35 | total_timesteps 351.
Path 36 | total_timesteps 360.
Path 37 | total_timesteps 373.
Path 38 | total_timesteps 382.
Path 39 | total_timesteps 391.
Path 40 | total_timesteps 398.
Path 41 | total_timesteps 409.
Path 42 | total_timesteps 419.
Path 43 | total_timesteps 427.
Path 44 | total_timesteps 434.
Path 45 | total_timesteps 443.
Path 46 | total_timesteps 456.
Path 47 | total_timesteps 464.
Path 48 | total_timesteps 472.
Path 49 | total_timesteps 482.
Path 50 | total_timesteps 491.
Path 51 | total_timesteps 501.
Path 52 | total_timesteps 511.
Path 53 | total_timesteps 522.
Path 54 | total_timesteps 532.
Path 55 | total_timesteps 543.
Path 56 | total_timesteps 553.
Path 57 | total_timesteps 563.
Path 58 | total_timesteps 573.
Path 59 | total_timesteps 580.
Path 60 | total_timesteps 588.
Path 61 | total_timesteps 596.
Path 62 | total_timesteps 605.
Path 63 | total_timesteps 614.
Path 64 | total_timesteps 624.
Path 65 | total_timesteps 632.
Path 66 | total_timesteps 642.
Path 67 | total_timesteps 654.
Path 68 | total_timesteps 664.
Path 69 | total_timesteps 682.
Path 70 | total_timesteps 692.
Path 71 | total_timesteps 703.
Path 72 | total_timesteps 711.
Path 73 | total_timesteps 720.
Path 74 | total_timesteps 733.
Path 75 | total_timesteps 743.
Path 76 | total_timesteps 760.
Path 77 | total_timesteps 768.
Path 78 | total_timesteps 779.
Path 79 | total_timesteps 798.
Path 80 | total_timesteps 804.
Path 81 | total_timesteps 813.
Path 82 | total_timesteps 825.
Path 83 | total_timesteps 834.
Path 84 | total_timesteps 843.
Path 85 | total_timesteps 852.
Path 86 | total_timesteps 868.
Path 87 | total_timesteps 878.
Path 88 | total_timesteps 886.
Path 89 | total_timesteps 895.
Path 90 | total_timesteps 906.
Path 91 | total_timesteps 918.
Path 92 | total_timesteps 929.
Path 93 | total_timesteps 940.
Path 94 | total_timesteps 948.
Path 95 | total_timesteps 962.
Path 96 | total_timesteps 972.
Path 97 | total_timesteps 980.
Path 98 | total_timesteps 990.
Path 99 | total_timesteps 998.
Path 100 | total_timesteps 1008.
Path 101 | total_timesteps 1019.
Path 102 | total_timesteps 1026.
Path 103 | total_timesteps 1034.
Path 104 | total_timesteps 1046.
Path 105 | total_timesteps 1056.
Path 106 | total_timesteps 1066.
Path 107 | total_timesteps 1073.
Path 108 | total_timesteps 1081.
Path 109 | total_timesteps 1091.
Path 110 | total_timesteps 1099.
Path 111 | total_timesteps 1105.
Path 112 | total_timesteps 1116.
Path 113 | total_timesteps 1123.
Path 114 | total_timesteps 1133.
Path 115 | total_timesteps 1143.
Path 116 | total_timesteps 1151.
Path 117 | total_timesteps 1158.
Path 118 | total_timesteps 1167.
Path 119 | total_timesteps 1177.
Path 120 | total_timesteps 1189.
Path 121 | total_timesteps 1197.
Path 122 | total_timesteps 1210.
Path 123 | total_timesteps 1221.
Path 124 | total_timesteps 1227.
Path 125 | total_timesteps 1242.
Path 126 | total_timesteps 1251.
Path 127 | total_timesteps 1265.
Path 128 | total_timesteps 1272.
Path 129 | total_timesteps 1284.
Path 130 | total_timesteps 1291.
Path 131 | total_timesteps 1299.
Path 132 | total_timesteps 1315.
Path 133 | total_timesteps 1323.
Path 134 | total_timesteps 1334.
Path 135 | total_timesteps 1345.
Path 136 | total_timesteps 1355.
Path 137 | total_timesteps 1362.
Path 138 | total_timesteps 1373.
Path 139 | total_timesteps 1383.
Path 140 | total_timesteps 1390.
Path 141 | total_timesteps 1397.
Path 142 | total_timesteps 1410.
Path 143 | total_timesteps 1424.
Path 144 | total_timesteps 1433.
Path 145 | total_timesteps 1440.
Path 146 | total_timesteps 1453.
Path 147 | total_timesteps 1466.
Path 148 | total_timesteps 1477.
Path 149 | total_timesteps 1486.
Path 150 | total_timesteps 1497.
Path 151 | total_timesteps 1505.
Path 152 | total_timesteps 1513.
Path 153 | total_timesteps 1528.
Path 154 | total_timesteps 1540.
Path 155 | total_timesteps 1552.
Path 156 | total_timesteps 1564.
Path 157 | total_timesteps 1573.
Path 158 | total_timesteps 1580.
Path 159 | total_timesteps 1588.
Path 160 | total_timesteps 1597.
Path 161 | total_timesteps 1606.
Path 162 | total_timesteps 1615.
Path 163 | total_timesteps 1624.
Path 164 | total_timesteps 1634.
Path 165 | total_timesteps 1648.
Path 166 | total_timesteps 1656.
Path 167 | total_timesteps 1672.
Path 168 | total_timesteps 1678.
Path 169 | total_timesteps 1688.
Path 170 | total_timesteps 1708.
Path 171 | total_timesteps 1717.
Path 172 | total_timesteps 1725.
Path 173 | total_timesteps 1736.
Path 174 | total_timesteps 1745.
Path 175 | total_timesteps 1758.
Path 176 | total_timesteps 1766.
Path 177 | total_timesteps 1779.
Path 178 | total_timesteps 1790.
Path 179 | total_timesteps 1800.
Path 180 | total_timesteps 1808.
Path 181 | total_timesteps 1818.
Path 182 | total_timesteps 1826.
Path 183 | total_timesteps 1834.
Path 184 | total_timesteps 1853.
Path 185 | total_timesteps 1863.
Path 186 | total_timesteps 1873.
Path 187 | total_timesteps 1883.
Path 188 | total_timesteps 1892.
Path 189 | total_timesteps 1903.
Path 190 | total_timesteps 1912.
Path 191 | total_timesteps 1921.
Path 192 | total_timesteps 1930.
Path 193 | total_timesteps 1939.
Path 194 | total_timesteps 1948.
Path 195 | total_timesteps 1957.
Path 196 | total_timesteps 1964.
Path 197 | total_timesteps 1976.
Path 198 | total_timesteps 1987.
Path 199 | total_timesteps 1993.
Path 200 | total_timesteps 2002.
Path 201 | total_timesteps 2015.
Path 202 | total_timesteps 2030.
Path 203 | total_timesteps 2039.
Path 204 | total_timesteps 2048.
Path 205 | total_timesteps 2055.
Path 206 | total_timesteps 2062.
Path 207 | total_timesteps 2071.
Path 208 | total_timesteps 2078.
Path 209 | total_timesteps 2086.
Path 210 | total_timesteps 2094.
Path 211 | total_timesteps 2103.
Path 212 | total_timesteps 2111.
Path 213 | total_timesteps 2123.
Path 214 | total_timesteps 2132.
Path 215 | total_timesteps 2139.
Path 216 | total_timesteps 2149.
Path 217 | total_timesteps 2158.
Path 218 | total_timesteps 2165.
Path 219 | total_timesteps 2174.
Path 220 | total_timesteps 2181.
Path 221 | total_timesteps 2191.
Path 222 | total_timesteps 2203.
Path 223 | total_timesteps 2212.
Path 224 | total_timesteps 2226.
Path 225 | total_timesteps 2240.
Path 226 | total_timesteps 2256.
Path 227 | total_timesteps 2266.
Path 228 | total_timesteps 2275.
Path 229 | total_timesteps 2286.
Path 230 | total_timesteps 2292.
Path 231 | total_timesteps 2300.
Path 232 | total_timesteps 2310.
Path 233 | total_timesteps 2317.
Path 234 | total_timesteps 2326.
Path 235 | total_timesteps 2337.
Path 236 | total_timesteps 2347.
Path 237 | total_timesteps 2356.
Path 238 | total_timesteps 2364.
Path 239 | total_timesteps 2373.
Path 240 | total_timesteps 2382.
Path 241 | total_timesteps 2389.
Path 242 | total_timesteps 2397.
Path 243 | total_timesteps 2406.
Path 244 | total_timesteps 2413.
Path 245 | total_timesteps 2419.
Path 246 | total_timesteps 2426.
Path 247 | total_timesteps 2435.
Path 248 | total_timesteps 2447.
Path 249 | total_timesteps 2454.
Path 250 | total_timesteps 2465.
Path 251 | total_timesteps 2472.
Path 252 | total_timesteps 2481.
Path 253 | total_timesteps 2488.
Path 254 | total_timesteps 2497.
Path 255 | total_timesteps 2509.
Path 256 | total_timesteps 2518.
Path 257 | total_timesteps 2528.
Path 258 | total_timesteps 2539.
Path 259 | total_timesteps 2550.
Path 260 | total_timesteps 2559.
Path 261 | total_timesteps 2574.
Path 262 | total_timesteps 2581.
Path 263 | total_timesteps 2590.
Path 264 | total_timesteps 2603.
Path 265 | total_timesteps 2611.
Path 266 | total_timesteps 2619.
Path 267 | total_timesteps 2630.
Path 268 | total_timesteps 2642.
Path 269 | total_timesteps 2650.
Path 270 | total_timesteps 2662.
Path 271 | total_timesteps 2675.
Path 272 | total_timesteps 2682.
Path 273 | total_timesteps 2693.
Path 274 | total_timesteps 2702.
Path 275 | total_timesteps 2709.
Path 276 | total_timesteps 2719.
Path 277 | total_timesteps 2726.
Path 278 | total_timesteps 2734.
Path 279 | total_timesteps 2744.
Path 280 | total_timesteps 2757.
Path 281 | total_timesteps 2772.
Path 282 | total_timesteps 2783.
Path 283 | total_timesteps 2794.
Path 284 | total_timesteps 2810.
Path 285 | total_timesteps 2820.
Path 286 | total_timesteps 2828.
Path 287 | total_timesteps 2837.
Path 288 | total_timesteps 2850.
Path 289 | total_timesteps 2863.
Path 290 | total_timesteps 2871.
Path 291 | total_timesteps 2880.
Path 292 | total_timesteps 2887.
Path 293 | total_timesteps 2894.
Path 294 | total_timesteps 2904.
Path 295 | total_timesteps 2913.
Path 296 | total_timesteps 2922.
Path 297 | total_timesteps 2929.
Path 298 | total_timesteps 2939.
Path 299 | total_timesteps 2947.
Path 300 | total_timesteps 2954.
Path 301 | total_timesteps 2961.
Path 302 | total_timesteps 2972.
Path 303 | total_timesteps 2983.
Path 304 | total_timesteps 2993.
Path 305 | total_timesteps 3001.
Path 306 | total_timesteps 3008.
Path 307 | total_timesteps 3016.
Path 308 | total_timesteps 3026.
Path 309 | total_timesteps 3036.
Path 310 | total_timesteps 3049.
Path 311 | total_timesteps 3060.
Path 312 | total_timesteps 3067.
Path 313 | total_timesteps 3076.
Path 314 | total_timesteps 3087.
Path 315 | total_timesteps 3096.
Path 316 | total_timesteps 3104.
Path 317 | total_timesteps 3115.
Path 318 | total_timesteps 3127.
Path 319 | total_timesteps 3134.
Path 320 | total_timesteps 3152.
Path 321 | total_timesteps 3163.
Path 322 | total_timesteps 3172.
Path 323 | total_timesteps 3181.
Path 324 | total_timesteps 3195.
Path 325 | total_timesteps 3207.
Path 326 | total_timesteps 3214.
Path 327 | total_timesteps 3222.
Path 328 | total_timesteps 3233.
Path 329 | total_timesteps 3250.
Path 330 | total_timesteps 3260.
Path 331 | total_timesteps 3270.
Path 332 | total_timesteps 3282.
Path 333 | total_timesteps 3292.
Path 334 | total_timesteps 3299.
Path 335 | total_timesteps 3310.
Path 336 | total_timesteps 3317.
Path 337 | total_timesteps 3327.
Path 338 | total_timesteps 3336.
Path 339 | total_timesteps 3345.
Path 340 | total_timesteps 3355.
Path 341 | total_timesteps 3369.
Path 342 | total_timesteps 3377.
Path 343 | total_timesteps 3387.
Path 344 | total_timesteps 3397.
Path 345 | total_timesteps 3409.
Path 346 | total_timesteps 3419.
Path 347 | total_timesteps 3428.
Path 348 | total_timesteps 3439.
Path 349 | total_timesteps 3449.
Path 350 | total_timesteps 3464.
Path 351 | total_timesteps 3477.
Path 352 | total_timesteps 3486.
Path 353 | total_timesteps 3496.
Path 354 | total_timesteps 3504.
Path 355 | total_timesteps 3514.
Path 356 | total_timesteps 3525.
Path 357 | total_timesteps 3539.
Path 358 | total_timesteps 3548.
Path 359 | total_timesteps 3558.
Path 360 | total_timesteps 3570.
Path 361 | total_timesteps 3577.
Path 362 | total_timesteps 3585.
Path 363 | total_timesteps 3600.
Path 364 | total_timesteps 3610.
Path 365 | total_timesteps 3619.
Path 366 | total_timesteps 3627.
Path 367 | total_timesteps 3638.
Path 368 | total_timesteps 3647.
Path 369 | total_timesteps 3656.
Path 370 | total_timesteps 3665.
Path 371 | total_timesteps 3678.
Path 372 | total_timesteps 3687.
Path 373 | total_timesteps 3697.
Path 374 | total_timesteps 3715.
Path 375 | total_timesteps 3725.
Path 376 | total_timesteps 3733.
Path 377 | total_timesteps 3739.
Path 378 | total_timesteps 3757.
Path 379 | total_timesteps 3765.
Path 380 | total_timesteps 3775.
Path 381 | total_timesteps 3782.
Path 382 | total_timesteps 3794.
Path 383 | total_timesteps 3803.
Path 384 | total_timesteps 3816.
Path 385 | total_timesteps 3828.
Path 386 | total_timesteps 3834.
Path 387 | total_timesteps 3846.
Path 388 | total_timesteps 3857.
Path 389 | total_timesteps 3869.
Path 390 | total_timesteps 3877.
Path 391 | total_timesteps 3886.
Path 392 | total_timesteps 3897.
Path 393 | total_timesteps 3908.
Path 394 | total_timesteps 3917.
Path 395 | total_timesteps 3931.
Path 396 | total_timesteps 3940.
Path 397 | total_timesteps 3950.
Path 398 | total_timesteps 3961.
Path 399 | total_timesteps 3969.
Path 400 | total_timesteps 3977.
Path 401 | total_timesteps 3992.
Path 402 | total_timesteps 4001.
Path 403 | total_timesteps 4014.
Path 404 | total_timesteps 4022.
Path 405 | total_timesteps 4030.
Path 406 | total_timesteps 4047.
Path 407 | total_timesteps 4055.
Path 408 | total_timesteps 4064.
Path 409 | total_timesteps 4073.
Path 410 | total_timesteps 4082.
Path 411 | total_timesteps 4089.
Path 412 | total_timesteps 4101.
Path 413 | total_timesteps 4110.
Path 414 | total_timesteps 4131.
Path 415 | total_timesteps 4142.
Path 416 | total_timesteps 4150.
Path 417 | total_timesteps 4159.
Path 418 | total_timesteps 4170.
Path 419 | total_timesteps 4180.
Path 420 | total_timesteps 4191.
Path 421 | total_timesteps 4202.
Path 422 | total_timesteps 4223.
Path 423 | total_timesteps 4230.
Path 424 | total_timesteps 4241.
Path 425 | total_timesteps 4252.
Path 426 | total_timesteps 4268.
Path 427 | total_timesteps 4280.
Path 428 | total_timesteps 4291.
Path 429 | total_timesteps 4304.
Path 430 | total_timesteps 4316.
Path 431 | total_timesteps 4324.
Path 432 | total_timesteps 4332.
Path 433 | total_timesteps 4341.
Path 434 | total_timesteps 4350.
Path 435 | total_timesteps 4360.
Path 436 | total_timesteps 4372.
Path 437 | total_timesteps 4382.
Path 438 | total_timesteps 4392.
Path 439 | total_timesteps 4401.
Path 440 | total_timesteps 4410.
Path 441 | total_timesteps 4425.
Path 442 | total_timesteps 4433.
Path 443 | total_timesteps 4440.
Path 444 | total_timesteps 4448.
Path 445 | total_timesteps 4458.
Path 446 | total_timesteps 4466.
Path 447 | total_timesteps 4476.
Path 448 | total_timesteps 4487.
Path 449 | total_timesteps 4498.
Path 450 | total_timesteps 4510.
Path 451 | total_timesteps 4519.
Path 452 | total_timesteps 4527.
Path 453 | total_timesteps 4538.
Path 454 | total_timesteps 4547.
Path 455 | total_timesteps 4559.
Path 456 | total_timesteps 4566.
Path 457 | total_timesteps 4579.
Path 458 | total_timesteps 4587.
Path 459 | total_timesteps 4596.
Path 460 | total_timesteps 4605.
Path 461 | total_timesteps 4622.
Path 462 | total_timesteps 4635.
Path 463 | total_timesteps 4644.
Path 464 | total_timesteps 4652.
Path 465 | total_timesteps 4659.
Path 466 | total_timesteps 4666.
Path 467 | total_timesteps 4673.
Path 468 | total_timesteps 4681.
Path 469 | total_timesteps 4688.
Path 470 | total_timesteps 4697.
Path 471 | total_timesteps 4709.
Path 472 | total_timesteps 4717.
Path 473 | total_timesteps 4724.
Path 474 | total_timesteps 4731.
Path 475 | total_timesteps 4742.
Path 476 | total_timesteps 4751.
Path 477 | total_timesteps 4759.
Path 478 | total_timesteps 4770.
Path 479 | total_timesteps 4778.
Path 480 | total_timesteps 4791.
Path 481 | total_timesteps 4800.
Path 482 | total_timesteps 4811.
Path 483 | total_timesteps 4821.
Path 484 | total_timesteps 4830.
Path 485 | total_timesteps 4838.
Path 486 | total_timesteps 4847.
Path 487 | total_timesteps 4855.
Path 488 | total_timesteps 4865.
Path 489 | total_timesteps 4874.
Path 490 | total_timesteps 4882.
Path 491 | total_timesteps 4891.
Path 492 | total_timesteps 4899.
Path 493 | total_timesteps 4909.
Path 494 | total_timesteps 4918.
Path 495 | total_timesteps 4929.
Path 496 | total_timesteps 4939.
Path 497 | total_timesteps 4950.
Path 498 | total_timesteps 4959.
Path 499 | total_timesteps 4969.
Path 500 | total_timesteps 4979.
Path 501 | total_timesteps 4990.
Path 502 | total_timesteps 5000.
Path 503 | total_timesteps 5012.
Path 504 | total_timesteps 5021.
Path 505 | total_timesteps 5028.
Path 506 | total_timesteps 5037.
Path 507 | total_timesteps 5050.
Path 508 | total_timesteps 5059.
Path 509 | total_timesteps 5066.
Path 510 | total_timesteps 5075.
Path 511 | total_timesteps 5085.
Path 512 | total_timesteps 5093.
Path 513 | total_timesteps 5102.
Path 514 | total_timesteps 5110.
Path 515 | total_timesteps 5121.
Path 516 | total_timesteps 5129.
Path 517 | total_timesteps 5140.
Path 518 | total_timesteps 5148.
Path 519 | total_timesteps 5159.
Path 520 | total_timesteps 5170.
Path 521 | total_timesteps 5180.
Path 522 | total_timesteps 5188.
Path 523 | total_timesteps 5199.
Path 524 | total_timesteps 5209.
Path 525 | total_timesteps 5218.
Path 526 | total_timesteps 5226.
Path 527 | total_timesteps 5233.
Path 528 | total_timesteps 5243.
Path 529 | total_timesteps 5253.
Path 530 | total_timesteps 5274.
Path 531 | total_timesteps 5282.
Path 532 | total_timesteps 5289.
Path 533 | total_timesteps 5298.
Path 534 | total_timesteps 5311.
Path 535 | total_timesteps 5322.
Path 536 | total_timesteps 5334.
Path 537 | total_timesteps 5353.
Path 538 | total_timesteps 5364.
Path 539 | total_timesteps 5374.
Path 540 | total_timesteps 5382.
Path 541 | total_timesteps 5391.
Path 542 | total_timesteps 5402.
Path 543 | total_timesteps 5411.
Path 544 | total_timesteps 5419.
Path 545 | total_timesteps 5435.
Path 546 | total_timesteps 5444.
Path 547 | total_timesteps 5455.
Path 548 | total_timesteps 5466.
Path 549 | total_timesteps 5476.
Path 550 | total_timesteps 5485.
Path 551 | total_timesteps 5497.
Path 552 | total_timesteps 5505.
Path 553 | total_timesteps 5514.
Path 554 | total_timesteps 5525.
Path 555 | total_timesteps 5534.
Path 556 | total_timesteps 5543.
Path 557 | total_timesteps 5553.
Path 558 | total_timesteps 5559.
Path 559 | total_timesteps 5567.
Path 560 | total_timesteps 5578.
Path 561 | total_timesteps 5585.
Path 562 | total_timesteps 5594.
Path 563 | total_timesteps 5605.
Path 564 | total_timesteps 5614.
Path 565 | total_timesteps 5622.
Path 566 | total_timesteps 5632.
Path 567 | total_timesteps 5640.
Path 568 | total_timesteps 5648.
Path 569 | total_timesteps 5657.
Path 570 | total_timesteps 5665.
Path 571 | total_timesteps 5675.
Path 572 | total_timesteps 5685.
Path 573 | total_timesteps 5699.
Path 574 | total_timesteps 5707.
Path 575 | total_timesteps 5717.
Path 576 | total_timesteps 5725.
Path 577 | total_timesteps 5738.
Path 578 | total_timesteps 5749.
Path 579 | total_timesteps 5762.
Path 580 | total_timesteps 5774.
Path 581 | total_timesteps 5785.
Path 582 | total_timesteps 5799.
Path 583 | total_timesteps 5808.
Path 584 | total_timesteps 5818.
Path 585 | total_timesteps 5827.
Path 586 | total_timesteps 5840.
Path 587 | total_timesteps 5853.
Path 588 | total_timesteps 5861.
Path 589 | total_timesteps 5869.
Path 590 | total_timesteps 5879.
Path 591 | total_timesteps 5896.
Path 592 | total_timesteps 5906.
Path 593 | total_timesteps 5925.
Path 594 | total_timesteps 5934.
Path 595 | total_timesteps 5942.
Path 596 | total_timesteps 5950.
Path 597 | total_timesteps 5958.
Path 598 | total_timesteps 5966.
Path 599 | total_timesteps 5976.
Path 600 | total_timesteps 5986.
Path 601 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.1     |
| Iteration     | 21       |
| MaximumReturn | 3.51     |
| MinimumReturn | -16.4    |
| TotalSamples  | 92125    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0026064158882945776
Validation loss = 0.0027196628507226706
Validation loss = 0.0028203760739415884
Validation loss = 0.0026787070091813803
Validation loss = 0.0025739085394889116
Validation loss = 0.0026027183048427105
Validation loss = 0.002575477585196495
Validation loss = 0.002335209399461746
Validation loss = 0.0024591200053691864
Validation loss = 0.002460597548633814
Validation loss = 0.0022930472623556852
Validation loss = 0.002605305751785636
Validation loss = 0.002290191128849983
Validation loss = 0.0024207362439483404
Validation loss = 0.002421643817797303
Validation loss = 0.0026170522905886173
Validation loss = 0.002869040472432971
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 69.
Path 8 | total_timesteps 78.
Path 9 | total_timesteps 88.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 108.
Path 12 | total_timesteps 118.
Path 13 | total_timesteps 131.
Path 14 | total_timesteps 139.
Path 15 | total_timesteps 149.
Path 16 | total_timesteps 158.
Path 17 | total_timesteps 168.
Path 18 | total_timesteps 174.
Path 19 | total_timesteps 185.
Path 20 | total_timesteps 193.
Path 21 | total_timesteps 206.
Path 22 | total_timesteps 223.
Path 23 | total_timesteps 237.
Path 24 | total_timesteps 244.
Path 25 | total_timesteps 253.
Path 26 | total_timesteps 265.
Path 27 | total_timesteps 276.
Path 28 | total_timesteps 285.
Path 29 | total_timesteps 293.
Path 30 | total_timesteps 302.
Path 31 | total_timesteps 310.
Path 32 | total_timesteps 330.
Path 33 | total_timesteps 340.
Path 34 | total_timesteps 350.
Path 35 | total_timesteps 357.
Path 36 | total_timesteps 365.
Path 37 | total_timesteps 375.
Path 38 | total_timesteps 388.
Path 39 | total_timesteps 401.
Path 40 | total_timesteps 410.
Path 41 | total_timesteps 419.
Path 42 | total_timesteps 428.
Path 43 | total_timesteps 443.
Path 44 | total_timesteps 454.
Path 45 | total_timesteps 469.
Path 46 | total_timesteps 476.
Path 47 | total_timesteps 484.
Path 48 | total_timesteps 492.
Path 49 | total_timesteps 503.
Path 50 | total_timesteps 513.
Path 51 | total_timesteps 533.
Path 52 | total_timesteps 543.
Path 53 | total_timesteps 550.
Path 54 | total_timesteps 558.
Path 55 | total_timesteps 570.
Path 56 | total_timesteps 578.
Path 57 | total_timesteps 586.
Path 58 | total_timesteps 596.
Path 59 | total_timesteps 606.
Path 60 | total_timesteps 619.
Path 61 | total_timesteps 625.
Path 62 | total_timesteps 635.
Path 63 | total_timesteps 644.
Path 64 | total_timesteps 650.
Path 65 | total_timesteps 659.
Path 66 | total_timesteps 669.
Path 67 | total_timesteps 680.
Path 68 | total_timesteps 691.
Path 69 | total_timesteps 698.
Path 70 | total_timesteps 706.
Path 71 | total_timesteps 714.
Path 72 | total_timesteps 726.
Path 73 | total_timesteps 735.
Path 74 | total_timesteps 745.
Path 75 | total_timesteps 757.
Path 76 | total_timesteps 765.
Path 77 | total_timesteps 773.
Path 78 | total_timesteps 786.
Path 79 | total_timesteps 795.
Path 80 | total_timesteps 806.
Path 81 | total_timesteps 816.
Path 82 | total_timesteps 826.
Path 83 | total_timesteps 836.
Path 84 | total_timesteps 843.
Path 85 | total_timesteps 854.
Path 86 | total_timesteps 870.
Path 87 | total_timesteps 880.
Path 88 | total_timesteps 887.
Path 89 | total_timesteps 897.
Path 90 | total_timesteps 904.
Path 91 | total_timesteps 913.
Path 92 | total_timesteps 921.
Path 93 | total_timesteps 929.
Path 94 | total_timesteps 938.
Path 95 | total_timesteps 947.
Path 96 | total_timesteps 958.
Path 97 | total_timesteps 970.
Path 98 | total_timesteps 987.
Path 99 | total_timesteps 995.
Path 100 | total_timesteps 1003.
Path 101 | total_timesteps 1014.
Path 102 | total_timesteps 1027.
Path 103 | total_timesteps 1043.
Path 104 | total_timesteps 1054.
Path 105 | total_timesteps 1064.
Path 106 | total_timesteps 1078.
Path 107 | total_timesteps 1086.
Path 108 | total_timesteps 1095.
Path 109 | total_timesteps 1108.
Path 110 | total_timesteps 1120.
Path 111 | total_timesteps 1133.
Path 112 | total_timesteps 1145.
Path 113 | total_timesteps 1159.
Path 114 | total_timesteps 1172.
Path 115 | total_timesteps 1187.
Path 116 | total_timesteps 1198.
Path 117 | total_timesteps 1210.
Path 118 | total_timesteps 1220.
Path 119 | total_timesteps 1228.
Path 120 | total_timesteps 1236.
Path 121 | total_timesteps 1245.
Path 122 | total_timesteps 1258.
Path 123 | total_timesteps 1267.
Path 124 | total_timesteps 1279.
Path 125 | total_timesteps 1288.
Path 126 | total_timesteps 1299.
Path 127 | total_timesteps 1310.
Path 128 | total_timesteps 1319.
Path 129 | total_timesteps 1332.
Path 130 | total_timesteps 1339.
Path 131 | total_timesteps 1346.
Path 132 | total_timesteps 1362.
Path 133 | total_timesteps 1374.
Path 134 | total_timesteps 1381.
Path 135 | total_timesteps 1391.
Path 136 | total_timesteps 1401.
Path 137 | total_timesteps 1411.
Path 138 | total_timesteps 1421.
Path 139 | total_timesteps 1432.
Path 140 | total_timesteps 1443.
Path 141 | total_timesteps 1455.
Path 142 | total_timesteps 1466.
Path 143 | total_timesteps 1481.
Path 144 | total_timesteps 1490.
Path 145 | total_timesteps 1500.
Path 146 | total_timesteps 1512.
Path 147 | total_timesteps 1522.
Path 148 | total_timesteps 1531.
Path 149 | total_timesteps 1542.
Path 150 | total_timesteps 1553.
Path 151 | total_timesteps 1563.
Path 152 | total_timesteps 1572.
Path 153 | total_timesteps 1584.
Path 154 | total_timesteps 1591.
Path 155 | total_timesteps 1601.
Path 156 | total_timesteps 1614.
Path 157 | total_timesteps 1621.
Path 158 | total_timesteps 1629.
Path 159 | total_timesteps 1639.
Path 160 | total_timesteps 1648.
Path 161 | total_timesteps 1663.
Path 162 | total_timesteps 1674.
Path 163 | total_timesteps 1686.
Path 164 | total_timesteps 1694.
Path 165 | total_timesteps 1705.
Path 166 | total_timesteps 1713.
Path 167 | total_timesteps 1723.
Path 168 | total_timesteps 1742.
Path 169 | total_timesteps 1755.
Path 170 | total_timesteps 1765.
Path 171 | total_timesteps 1775.
Path 172 | total_timesteps 1786.
Path 173 | total_timesteps 1796.
Path 174 | total_timesteps 1808.
Path 175 | total_timesteps 1816.
Path 176 | total_timesteps 1823.
Path 177 | total_timesteps 1836.
Path 178 | total_timesteps 1847.
Path 179 | total_timesteps 1860.
Path 180 | total_timesteps 1872.
Path 181 | total_timesteps 1880.
Path 182 | total_timesteps 1888.
Path 183 | total_timesteps 1907.
Path 184 | total_timesteps 1917.
Path 185 | total_timesteps 1930.
Path 186 | total_timesteps 1940.
Path 187 | total_timesteps 1950.
Path 188 | total_timesteps 1961.
Path 189 | total_timesteps 1972.
Path 190 | total_timesteps 1982.
Path 191 | total_timesteps 1993.
Path 192 | total_timesteps 2004.
Path 193 | total_timesteps 2013.
Path 194 | total_timesteps 2029.
Path 195 | total_timesteps 2039.
Path 196 | total_timesteps 2050.
Path 197 | total_timesteps 2065.
Path 198 | total_timesteps 2076.
Path 199 | total_timesteps 2085.
Path 200 | total_timesteps 2092.
Path 201 | total_timesteps 2104.
Path 202 | total_timesteps 2112.
Path 203 | total_timesteps 2121.
Path 204 | total_timesteps 2130.
Path 205 | total_timesteps 2139.
Path 206 | total_timesteps 2149.
Path 207 | total_timesteps 2161.
Path 208 | total_timesteps 2169.
Path 209 | total_timesteps 2177.
Path 210 | total_timesteps 2184.
Path 211 | total_timesteps 2193.
Path 212 | total_timesteps 2204.
Path 213 | total_timesteps 2213.
Path 214 | total_timesteps 2221.
Path 215 | total_timesteps 2231.
Path 216 | total_timesteps 2244.
Path 217 | total_timesteps 2252.
Path 218 | total_timesteps 2261.
Path 219 | total_timesteps 2270.
Path 220 | total_timesteps 2279.
Path 221 | total_timesteps 2287.
Path 222 | total_timesteps 2295.
Path 223 | total_timesteps 2304.
Path 224 | total_timesteps 2313.
Path 225 | total_timesteps 2321.
Path 226 | total_timesteps 2329.
Path 227 | total_timesteps 2337.
Path 228 | total_timesteps 2345.
Path 229 | total_timesteps 2354.
Path 230 | total_timesteps 2364.
Path 231 | total_timesteps 2375.
Path 232 | total_timesteps 2386.
Path 233 | total_timesteps 2393.
Path 234 | total_timesteps 2400.
Path 235 | total_timesteps 2409.
Path 236 | total_timesteps 2421.
Path 237 | total_timesteps 2430.
Path 238 | total_timesteps 2438.
Path 239 | total_timesteps 2448.
Path 240 | total_timesteps 2459.
Path 241 | total_timesteps 2475.
Path 242 | total_timesteps 2488.
Path 243 | total_timesteps 2501.
Path 244 | total_timesteps 2515.
Path 245 | total_timesteps 2523.
Path 246 | total_timesteps 2531.
Path 247 | total_timesteps 2547.
Path 248 | total_timesteps 2555.
Path 249 | total_timesteps 2564.
Path 250 | total_timesteps 2572.
Path 251 | total_timesteps 2580.
Path 252 | total_timesteps 2592.
Path 253 | total_timesteps 2603.
Path 254 | total_timesteps 2610.
Path 255 | total_timesteps 2627.
Path 256 | total_timesteps 2634.
Path 257 | total_timesteps 2642.
Path 258 | total_timesteps 2650.
Path 259 | total_timesteps 2662.
Path 260 | total_timesteps 2669.
Path 261 | total_timesteps 2680.
Path 262 | total_timesteps 2690.
Path 263 | total_timesteps 2699.
Path 264 | total_timesteps 2709.
Path 265 | total_timesteps 2718.
Path 266 | total_timesteps 2727.
Path 267 | total_timesteps 2735.
Path 268 | total_timesteps 2743.
Path 269 | total_timesteps 2757.
Path 270 | total_timesteps 2764.
Path 271 | total_timesteps 2775.
Path 272 | total_timesteps 2783.
Path 273 | total_timesteps 2795.
Path 274 | total_timesteps 2803.
Path 275 | total_timesteps 2822.
Path 276 | total_timesteps 2832.
Path 277 | total_timesteps 2842.
Path 278 | total_timesteps 2853.
Path 279 | total_timesteps 2868.
Path 280 | total_timesteps 2877.
Path 281 | total_timesteps 2893.
Path 282 | total_timesteps 2906.
Path 283 | total_timesteps 2913.
Path 284 | total_timesteps 2921.
Path 285 | total_timesteps 2930.
Path 286 | total_timesteps 2938.
Path 287 | total_timesteps 2945.
Path 288 | total_timesteps 2953.
Path 289 | total_timesteps 2964.
Path 290 | total_timesteps 2973.
Path 291 | total_timesteps 2983.
Path 292 | total_timesteps 2992.
Path 293 | total_timesteps 3001.
Path 294 | total_timesteps 3009.
Path 295 | total_timesteps 3019.
Path 296 | total_timesteps 3028.
Path 297 | total_timesteps 3037.
Path 298 | total_timesteps 3047.
Path 299 | total_timesteps 3057.
Path 300 | total_timesteps 3068.
Path 301 | total_timesteps 3078.
Path 302 | total_timesteps 3089.
Path 303 | total_timesteps 3101.
Path 304 | total_timesteps 3116.
Path 305 | total_timesteps 3124.
Path 306 | total_timesteps 3132.
Path 307 | total_timesteps 3143.
Path 308 | total_timesteps 3152.
Path 309 | total_timesteps 3160.
Path 310 | total_timesteps 3174.
Path 311 | total_timesteps 3191.
Path 312 | total_timesteps 3198.
Path 313 | total_timesteps 3206.
Path 314 | total_timesteps 3214.
Path 315 | total_timesteps 3222.
Path 316 | total_timesteps 3236.
Path 317 | total_timesteps 3243.
Path 318 | total_timesteps 3251.
Path 319 | total_timesteps 3263.
Path 320 | total_timesteps 3270.
Path 321 | total_timesteps 3278.
Path 322 | total_timesteps 3289.
Path 323 | total_timesteps 3297.
Path 324 | total_timesteps 3304.
Path 325 | total_timesteps 3318.
Path 326 | total_timesteps 3326.
Path 327 | total_timesteps 3337.
Path 328 | total_timesteps 3349.
Path 329 | total_timesteps 3358.
Path 330 | total_timesteps 3366.
Path 331 | total_timesteps 3376.
Path 332 | total_timesteps 3385.
Path 333 | total_timesteps 3395.
Path 334 | total_timesteps 3404.
Path 335 | total_timesteps 3413.
Path 336 | total_timesteps 3425.
Path 337 | total_timesteps 3434.
Path 338 | total_timesteps 3444.
Path 339 | total_timesteps 3460.
Path 340 | total_timesteps 3470.
Path 341 | total_timesteps 3479.
Path 342 | total_timesteps 3491.
Path 343 | total_timesteps 3503.
Path 344 | total_timesteps 3511.
Path 345 | total_timesteps 3521.
Path 346 | total_timesteps 3529.
Path 347 | total_timesteps 3539.
Path 348 | total_timesteps 3546.
Path 349 | total_timesteps 3559.
Path 350 | total_timesteps 3570.
Path 351 | total_timesteps 3579.
Path 352 | total_timesteps 3588.
Path 353 | total_timesteps 3594.
Path 354 | total_timesteps 3607.
Path 355 | total_timesteps 3615.
Path 356 | total_timesteps 3623.
Path 357 | total_timesteps 3634.
Path 358 | total_timesteps 3643.
Path 359 | total_timesteps 3651.
Path 360 | total_timesteps 3659.
Path 361 | total_timesteps 3666.
Path 362 | total_timesteps 3680.
Path 363 | total_timesteps 3688.
Path 364 | total_timesteps 3696.
Path 365 | total_timesteps 3704.
Path 366 | total_timesteps 3717.
Path 367 | total_timesteps 3726.
Path 368 | total_timesteps 3734.
Path 369 | total_timesteps 3742.
Path 370 | total_timesteps 3756.
Path 371 | total_timesteps 3769.
Path 372 | total_timesteps 3781.
Path 373 | total_timesteps 3787.
Path 374 | total_timesteps 3796.
Path 375 | total_timesteps 3805.
Path 376 | total_timesteps 3816.
Path 377 | total_timesteps 3828.
Path 378 | total_timesteps 3838.
Path 379 | total_timesteps 3848.
Path 380 | total_timesteps 3858.
Path 381 | total_timesteps 3868.
Path 382 | total_timesteps 3879.
Path 383 | total_timesteps 3885.
Path 384 | total_timesteps 3895.
Path 385 | total_timesteps 3904.
Path 386 | total_timesteps 3912.
Path 387 | total_timesteps 3921.
Path 388 | total_timesteps 3930.
Path 389 | total_timesteps 3943.
Path 390 | total_timesteps 3952.
Path 391 | total_timesteps 3963.
Path 392 | total_timesteps 3973.
Path 393 | total_timesteps 3983.
Path 394 | total_timesteps 3992.
Path 395 | total_timesteps 4004.
Path 396 | total_timesteps 4014.
Path 397 | total_timesteps 4021.
Path 398 | total_timesteps 4029.
Path 399 | total_timesteps 4039.
Path 400 | total_timesteps 4048.
Path 401 | total_timesteps 4056.
Path 402 | total_timesteps 4066.
Path 403 | total_timesteps 4074.
Path 404 | total_timesteps 4084.
Path 405 | total_timesteps 4091.
Path 406 | total_timesteps 4099.
Path 407 | total_timesteps 4107.
Path 408 | total_timesteps 4116.
Path 409 | total_timesteps 4125.
Path 410 | total_timesteps 4132.
Path 411 | total_timesteps 4139.
Path 412 | total_timesteps 4147.
Path 413 | total_timesteps 4156.
Path 414 | total_timesteps 4165.
Path 415 | total_timesteps 4177.
Path 416 | total_timesteps 4187.
Path 417 | total_timesteps 4197.
Path 418 | total_timesteps 4206.
Path 419 | total_timesteps 4226.
Path 420 | total_timesteps 4235.
Path 421 | total_timesteps 4245.
Path 422 | total_timesteps 4254.
Path 423 | total_timesteps 4264.
Path 424 | total_timesteps 4272.
Path 425 | total_timesteps 4283.
Path 426 | total_timesteps 4291.
Path 427 | total_timesteps 4300.
Path 428 | total_timesteps 4309.
Path 429 | total_timesteps 4316.
Path 430 | total_timesteps 4329.
Path 431 | total_timesteps 4343.
Path 432 | total_timesteps 4354.
Path 433 | total_timesteps 4364.
Path 434 | total_timesteps 4374.
Path 435 | total_timesteps 4384.
Path 436 | total_timesteps 4393.
Path 437 | total_timesteps 4403.
Path 438 | total_timesteps 4411.
Path 439 | total_timesteps 4423.
Path 440 | total_timesteps 4432.
Path 441 | total_timesteps 4442.
Path 442 | total_timesteps 4451.
Path 443 | total_timesteps 4458.
Path 444 | total_timesteps 4466.
Path 445 | total_timesteps 4475.
Path 446 | total_timesteps 4484.
Path 447 | total_timesteps 4497.
Path 448 | total_timesteps 4507.
Path 449 | total_timesteps 4515.
Path 450 | total_timesteps 4524.
Path 451 | total_timesteps 4535.
Path 452 | total_timesteps 4544.
Path 453 | total_timesteps 4555.
Path 454 | total_timesteps 4568.
Path 455 | total_timesteps 4582.
Path 456 | total_timesteps 4593.
Path 457 | total_timesteps 4602.
Path 458 | total_timesteps 4611.
Path 459 | total_timesteps 4622.
Path 460 | total_timesteps 4630.
Path 461 | total_timesteps 4640.
Path 462 | total_timesteps 4649.
Path 463 | total_timesteps 4662.
Path 464 | total_timesteps 4669.
Path 465 | total_timesteps 4677.
Path 466 | total_timesteps 4696.
Path 467 | total_timesteps 4708.
Path 468 | total_timesteps 4720.
Path 469 | total_timesteps 4733.
Path 470 | total_timesteps 4748.
Path 471 | total_timesteps 4759.
Path 472 | total_timesteps 4767.
Path 473 | total_timesteps 4776.
Path 474 | total_timesteps 4783.
Path 475 | total_timesteps 4791.
Path 476 | total_timesteps 4800.
Path 477 | total_timesteps 4809.
Path 478 | total_timesteps 4818.
Path 479 | total_timesteps 4826.
Path 480 | total_timesteps 4842.
Path 481 | total_timesteps 4852.
Path 482 | total_timesteps 4863.
Path 483 | total_timesteps 4874.
Path 484 | total_timesteps 4886.
Path 485 | total_timesteps 4896.
Path 486 | total_timesteps 4905.
Path 487 | total_timesteps 4915.
Path 488 | total_timesteps 4923.
Path 489 | total_timesteps 4931.
Path 490 | total_timesteps 4943.
Path 491 | total_timesteps 4955.
Path 492 | total_timesteps 4962.
Path 493 | total_timesteps 4974.
Path 494 | total_timesteps 4984.
Path 495 | total_timesteps 4997.
Path 496 | total_timesteps 5011.
Path 497 | total_timesteps 5020.
Path 498 | total_timesteps 5029.
Path 499 | total_timesteps 5038.
Path 500 | total_timesteps 5047.
Path 501 | total_timesteps 5055.
Path 502 | total_timesteps 5067.
Path 503 | total_timesteps 5076.
Path 504 | total_timesteps 5086.
Path 505 | total_timesteps 5101.
Path 506 | total_timesteps 5115.
Path 507 | total_timesteps 5124.
Path 508 | total_timesteps 5134.
Path 509 | total_timesteps 5147.
Path 510 | total_timesteps 5158.
Path 511 | total_timesteps 5174.
Path 512 | total_timesteps 5186.
Path 513 | total_timesteps 5194.
Path 514 | total_timesteps 5204.
Path 515 | total_timesteps 5216.
Path 516 | total_timesteps 5226.
Path 517 | total_timesteps 5236.
Path 518 | total_timesteps 5244.
Path 519 | total_timesteps 5254.
Path 520 | total_timesteps 5262.
Path 521 | total_timesteps 5269.
Path 522 | total_timesteps 5277.
Path 523 | total_timesteps 5288.
Path 524 | total_timesteps 5299.
Path 525 | total_timesteps 5308.
Path 526 | total_timesteps 5317.
Path 527 | total_timesteps 5326.
Path 528 | total_timesteps 5335.
Path 529 | total_timesteps 5345.
Path 530 | total_timesteps 5353.
Path 531 | total_timesteps 5361.
Path 532 | total_timesteps 5371.
Path 533 | total_timesteps 5379.
Path 534 | total_timesteps 5388.
Path 535 | total_timesteps 5395.
Path 536 | total_timesteps 5404.
Path 537 | total_timesteps 5415.
Path 538 | total_timesteps 5424.
Path 539 | total_timesteps 5434.
Path 540 | total_timesteps 5442.
Path 541 | total_timesteps 5451.
Path 542 | total_timesteps 5462.
Path 543 | total_timesteps 5471.
Path 544 | total_timesteps 5479.
Path 545 | total_timesteps 5489.
Path 546 | total_timesteps 5501.
Path 547 | total_timesteps 5511.
Path 548 | total_timesteps 5520.
Path 549 | total_timesteps 5528.
Path 550 | total_timesteps 5539.
Path 551 | total_timesteps 5548.
Path 552 | total_timesteps 5556.
Path 553 | total_timesteps 5564.
Path 554 | total_timesteps 5573.
Path 555 | total_timesteps 5582.
Path 556 | total_timesteps 5592.
Path 557 | total_timesteps 5602.
Path 558 | total_timesteps 5615.
Path 559 | total_timesteps 5627.
Path 560 | total_timesteps 5643.
Path 561 | total_timesteps 5652.
Path 562 | total_timesteps 5663.
Path 563 | total_timesteps 5673.
Path 564 | total_timesteps 5684.
Path 565 | total_timesteps 5694.
Path 566 | total_timesteps 5706.
Path 567 | total_timesteps 5713.
Path 568 | total_timesteps 5724.
Path 569 | total_timesteps 5734.
Path 570 | total_timesteps 5741.
Path 571 | total_timesteps 5750.
Path 572 | total_timesteps 5759.
Path 573 | total_timesteps 5773.
Path 574 | total_timesteps 5783.
Path 575 | total_timesteps 5789.
Path 576 | total_timesteps 5798.
Path 577 | total_timesteps 5809.
Path 578 | total_timesteps 5829.
Path 579 | total_timesteps 5839.
Path 580 | total_timesteps 5854.
Path 581 | total_timesteps 5861.
Path 582 | total_timesteps 5876.
Path 583 | total_timesteps 5887.
Path 584 | total_timesteps 5902.
Path 585 | total_timesteps 5912.
Path 586 | total_timesteps 5920.
Path 587 | total_timesteps 5929.
Path 588 | total_timesteps 5937.
Path 589 | total_timesteps 5949.
Path 590 | total_timesteps 5958.
Path 591 | total_timesteps 5971.
Path 592 | total_timesteps 5978.
Path 593 | total_timesteps 5986.
Path 594 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.37    |
| Iteration     | 22       |
| MaximumReturn | -0.422   |
| MinimumReturn | -18.4    |
| TotalSamples  | 96129    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.002809586701914668
Validation loss = 0.002300876658409834
Validation loss = 0.0022391658276319504
Validation loss = 0.00283813220448792
Validation loss = 0.0022058335598558187
Validation loss = 0.002290532225742936
Validation loss = 0.0022769637871533632
Validation loss = 0.0023748218081891537
Validation loss = 0.0022603871766477823
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 31.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 52.
Path 7 | total_timesteps 60.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 80.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 100.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 120.
Path 14 | total_timesteps 131.
Path 15 | total_timesteps 140.
Path 16 | total_timesteps 150.
Path 17 | total_timesteps 157.
Path 18 | total_timesteps 168.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 200.
Path 21 | total_timesteps 209.
Path 22 | total_timesteps 220.
Path 23 | total_timesteps 229.
Path 24 | total_timesteps 241.
Path 25 | total_timesteps 252.
Path 26 | total_timesteps 260.
Path 27 | total_timesteps 272.
Path 28 | total_timesteps 279.
Path 29 | total_timesteps 289.
Path 30 | total_timesteps 300.
Path 31 | total_timesteps 308.
Path 32 | total_timesteps 317.
Path 33 | total_timesteps 327.
Path 34 | total_timesteps 334.
Path 35 | total_timesteps 342.
Path 36 | total_timesteps 351.
Path 37 | total_timesteps 368.
Path 38 | total_timesteps 385.
Path 39 | total_timesteps 394.
Path 40 | total_timesteps 403.
Path 41 | total_timesteps 412.
Path 42 | total_timesteps 425.
Path 43 | total_timesteps 433.
Path 44 | total_timesteps 442.
Path 45 | total_timesteps 453.
Path 46 | total_timesteps 461.
Path 47 | total_timesteps 470.
Path 48 | total_timesteps 480.
Path 49 | total_timesteps 489.
Path 50 | total_timesteps 497.
Path 51 | total_timesteps 507.
Path 52 | total_timesteps 520.
Path 53 | total_timesteps 528.
Path 54 | total_timesteps 539.
Path 55 | total_timesteps 546.
Path 56 | total_timesteps 563.
Path 57 | total_timesteps 574.
Path 58 | total_timesteps 583.
Path 59 | total_timesteps 593.
Path 60 | total_timesteps 606.
Path 61 | total_timesteps 618.
Path 62 | total_timesteps 630.
Path 63 | total_timesteps 642.
Path 64 | total_timesteps 650.
Path 65 | total_timesteps 664.
Path 66 | total_timesteps 673.
Path 67 | total_timesteps 683.
Path 68 | total_timesteps 690.
Path 69 | total_timesteps 704.
Path 70 | total_timesteps 722.
Path 71 | total_timesteps 731.
Path 72 | total_timesteps 739.
Path 73 | total_timesteps 752.
Path 74 | total_timesteps 761.
Path 75 | total_timesteps 773.
Path 76 | total_timesteps 782.
Path 77 | total_timesteps 792.
Path 78 | total_timesteps 803.
Path 79 | total_timesteps 813.
Path 80 | total_timesteps 827.
Path 81 | total_timesteps 836.
Path 82 | total_timesteps 845.
Path 83 | total_timesteps 852.
Path 84 | total_timesteps 859.
Path 85 | total_timesteps 869.
Path 86 | total_timesteps 880.
Path 87 | total_timesteps 888.
Path 88 | total_timesteps 898.
Path 89 | total_timesteps 912.
Path 90 | total_timesteps 923.
Path 91 | total_timesteps 934.
Path 92 | total_timesteps 946.
Path 93 | total_timesteps 955.
Path 94 | total_timesteps 964.
Path 95 | total_timesteps 971.
Path 96 | total_timesteps 984.
Path 97 | total_timesteps 990.
Path 98 | total_timesteps 1001.
Path 99 | total_timesteps 1011.
Path 100 | total_timesteps 1021.
Path 101 | total_timesteps 1031.
Path 102 | total_timesteps 1042.
Path 103 | total_timesteps 1050.
Path 104 | total_timesteps 1062.
Path 105 | total_timesteps 1073.
Path 106 | total_timesteps 1085.
Path 107 | total_timesteps 1094.
Path 108 | total_timesteps 1100.
Path 109 | total_timesteps 1112.
Path 110 | total_timesteps 1121.
Path 111 | total_timesteps 1133.
Path 112 | total_timesteps 1141.
Path 113 | total_timesteps 1153.
Path 114 | total_timesteps 1167.
Path 115 | total_timesteps 1177.
Path 116 | total_timesteps 1186.
Path 117 | total_timesteps 1193.
Path 118 | total_timesteps 1207.
Path 119 | total_timesteps 1218.
Path 120 | total_timesteps 1227.
Path 121 | total_timesteps 1238.
Path 122 | total_timesteps 1249.
Path 123 | total_timesteps 1257.
Path 124 | total_timesteps 1267.
Path 125 | total_timesteps 1278.
Path 126 | total_timesteps 1293.
Path 127 | total_timesteps 1304.
Path 128 | total_timesteps 1312.
Path 129 | total_timesteps 1321.
Path 130 | total_timesteps 1330.
Path 131 | total_timesteps 1342.
Path 132 | total_timesteps 1350.
Path 133 | total_timesteps 1359.
Path 134 | total_timesteps 1366.
Path 135 | total_timesteps 1378.
Path 136 | total_timesteps 1393.
Path 137 | total_timesteps 1400.
Path 138 | total_timesteps 1410.
Path 139 | total_timesteps 1423.
Path 140 | total_timesteps 1431.
Path 141 | total_timesteps 1444.
Path 142 | total_timesteps 1456.
Path 143 | total_timesteps 1468.
Path 144 | total_timesteps 1477.
Path 145 | total_timesteps 1488.
Path 146 | total_timesteps 1498.
Path 147 | total_timesteps 1506.
Path 148 | total_timesteps 1515.
Path 149 | total_timesteps 1526.
Path 150 | total_timesteps 1537.
Path 151 | total_timesteps 1547.
Path 152 | total_timesteps 1558.
Path 153 | total_timesteps 1570.
Path 154 | total_timesteps 1581.
Path 155 | total_timesteps 1589.
Path 156 | total_timesteps 1600.
Path 157 | total_timesteps 1613.
Path 158 | total_timesteps 1626.
Path 159 | total_timesteps 1636.
Path 160 | total_timesteps 1647.
Path 161 | total_timesteps 1661.
Path 162 | total_timesteps 1675.
Path 163 | total_timesteps 1685.
Path 164 | total_timesteps 1697.
Path 165 | total_timesteps 1706.
Path 166 | total_timesteps 1715.
Path 167 | total_timesteps 1724.
Path 168 | total_timesteps 1733.
Path 169 | total_timesteps 1742.
Path 170 | total_timesteps 1760.
Path 171 | total_timesteps 1771.
Path 172 | total_timesteps 1781.
Path 173 | total_timesteps 1790.
Path 174 | total_timesteps 1799.
Path 175 | total_timesteps 1809.
Path 176 | total_timesteps 1826.
Path 177 | total_timesteps 1838.
Path 178 | total_timesteps 1845.
Path 179 | total_timesteps 1853.
Path 180 | total_timesteps 1862.
Path 181 | total_timesteps 1873.
Path 182 | total_timesteps 1884.
Path 183 | total_timesteps 1893.
Path 184 | total_timesteps 1905.
Path 185 | total_timesteps 1919.
Path 186 | total_timesteps 1929.
Path 187 | total_timesteps 1941.
Path 188 | total_timesteps 1952.
Path 189 | total_timesteps 1962.
Path 190 | total_timesteps 1971.
Path 191 | total_timesteps 1977.
Path 192 | total_timesteps 1991.
Path 193 | total_timesteps 2000.
Path 194 | total_timesteps 2006.
Path 195 | total_timesteps 2016.
Path 196 | total_timesteps 2022.
Path 197 | total_timesteps 2033.
Path 198 | total_timesteps 2044.
Path 199 | total_timesteps 2056.
Path 200 | total_timesteps 2065.
Path 201 | total_timesteps 2072.
Path 202 | total_timesteps 2081.
Path 203 | total_timesteps 2090.
Path 204 | total_timesteps 2105.
Path 205 | total_timesteps 2114.
Path 206 | total_timesteps 2132.
Path 207 | total_timesteps 2140.
Path 208 | total_timesteps 2150.
Path 209 | total_timesteps 2158.
Path 210 | total_timesteps 2168.
Path 211 | total_timesteps 2187.
Path 212 | total_timesteps 2198.
Path 213 | total_timesteps 2207.
Path 214 | total_timesteps 2214.
Path 215 | total_timesteps 2225.
Path 216 | total_timesteps 2234.
Path 217 | total_timesteps 2243.
Path 218 | total_timesteps 2256.
Path 219 | total_timesteps 2272.
Path 220 | total_timesteps 2280.
Path 221 | total_timesteps 2288.
Path 222 | total_timesteps 2297.
Path 223 | total_timesteps 2304.
Path 224 | total_timesteps 2316.
Path 225 | total_timesteps 2325.
Path 226 | total_timesteps 2334.
Path 227 | total_timesteps 2345.
Path 228 | total_timesteps 2353.
Path 229 | total_timesteps 2360.
Path 230 | total_timesteps 2371.
Path 231 | total_timesteps 2383.
Path 232 | total_timesteps 2393.
Path 233 | total_timesteps 2403.
Path 234 | total_timesteps 2415.
Path 235 | total_timesteps 2429.
Path 236 | total_timesteps 2439.
Path 237 | total_timesteps 2449.
Path 238 | total_timesteps 2456.
Path 239 | total_timesteps 2465.
Path 240 | total_timesteps 2474.
Path 241 | total_timesteps 2486.
Path 242 | total_timesteps 2494.
Path 243 | total_timesteps 2501.
Path 244 | total_timesteps 2511.
Path 245 | total_timesteps 2520.
Path 246 | total_timesteps 2530.
Path 247 | total_timesteps 2540.
Path 248 | total_timesteps 2549.
Path 249 | total_timesteps 2558.
Path 250 | total_timesteps 2566.
Path 251 | total_timesteps 2574.
Path 252 | total_timesteps 2582.
Path 253 | total_timesteps 2591.
Path 254 | total_timesteps 2607.
Path 255 | total_timesteps 2621.
Path 256 | total_timesteps 2631.
Path 257 | total_timesteps 2641.
Path 258 | total_timesteps 2653.
Path 259 | total_timesteps 2663.
Path 260 | total_timesteps 2676.
Path 261 | total_timesteps 2688.
Path 262 | total_timesteps 2696.
Path 263 | total_timesteps 2714.
Path 264 | total_timesteps 2725.
Path 265 | total_timesteps 2738.
Path 266 | total_timesteps 2748.
Path 267 | total_timesteps 2757.
Path 268 | total_timesteps 2767.
Path 269 | total_timesteps 2778.
Path 270 | total_timesteps 2790.
Path 271 | total_timesteps 2801.
Path 272 | total_timesteps 2814.
Path 273 | total_timesteps 2829.
Path 274 | total_timesteps 2845.
Path 275 | total_timesteps 2854.
Path 276 | total_timesteps 2864.
Path 277 | total_timesteps 2872.
Path 278 | total_timesteps 2881.
Path 279 | total_timesteps 2890.
Path 280 | total_timesteps 2900.
Path 281 | total_timesteps 2908.
Path 282 | total_timesteps 2915.
Path 283 | total_timesteps 2925.
Path 284 | total_timesteps 2935.
Path 285 | total_timesteps 2945.
Path 286 | total_timesteps 2953.
Path 287 | total_timesteps 2963.
Path 288 | total_timesteps 2973.
Path 289 | total_timesteps 2984.
Path 290 | total_timesteps 2995.
Path 291 | total_timesteps 3005.
Path 292 | total_timesteps 3014.
Path 293 | total_timesteps 3025.
Path 294 | total_timesteps 3035.
Path 295 | total_timesteps 3044.
Path 296 | total_timesteps 3051.
Path 297 | total_timesteps 3062.
Path 298 | total_timesteps 3080.
Path 299 | total_timesteps 3090.
Path 300 | total_timesteps 3100.
Path 301 | total_timesteps 3107.
Path 302 | total_timesteps 3113.
Path 303 | total_timesteps 3121.
Path 304 | total_timesteps 3129.
Path 305 | total_timesteps 3138.
Path 306 | total_timesteps 3149.
Path 307 | total_timesteps 3160.
Path 308 | total_timesteps 3168.
Path 309 | total_timesteps 3179.
Path 310 | total_timesteps 3188.
Path 311 | total_timesteps 3200.
Path 312 | total_timesteps 3208.
Path 313 | total_timesteps 3217.
Path 314 | total_timesteps 3225.
Path 315 | total_timesteps 3236.
Path 316 | total_timesteps 3245.
Path 317 | total_timesteps 3257.
Path 318 | total_timesteps 3269.
Path 319 | total_timesteps 3282.
Path 320 | total_timesteps 3290.
Path 321 | total_timesteps 3297.
Path 322 | total_timesteps 3306.
Path 323 | total_timesteps 3316.
Path 324 | total_timesteps 3326.
Path 325 | total_timesteps 3334.
Path 326 | total_timesteps 3351.
Path 327 | total_timesteps 3361.
Path 328 | total_timesteps 3371.
Path 329 | total_timesteps 3379.
Path 330 | total_timesteps 3389.
Path 331 | total_timesteps 3397.
Path 332 | total_timesteps 3406.
Path 333 | total_timesteps 3418.
Path 334 | total_timesteps 3432.
Path 335 | total_timesteps 3441.
Path 336 | total_timesteps 3449.
Path 337 | total_timesteps 3459.
Path 338 | total_timesteps 3471.
Path 339 | total_timesteps 3482.
Path 340 | total_timesteps 3491.
Path 341 | total_timesteps 3501.
Path 342 | total_timesteps 3508.
Path 343 | total_timesteps 3520.
Path 344 | total_timesteps 3533.
Path 345 | total_timesteps 3542.
Path 346 | total_timesteps 3550.
Path 347 | total_timesteps 3558.
Path 348 | total_timesteps 3566.
Path 349 | total_timesteps 3575.
Path 350 | total_timesteps 3583.
Path 351 | total_timesteps 3592.
Path 352 | total_timesteps 3600.
Path 353 | total_timesteps 3611.
Path 354 | total_timesteps 3619.
Path 355 | total_timesteps 3634.
Path 356 | total_timesteps 3648.
Path 357 | total_timesteps 3657.
Path 358 | total_timesteps 3667.
Path 359 | total_timesteps 3675.
Path 360 | total_timesteps 3690.
Path 361 | total_timesteps 3701.
Path 362 | total_timesteps 3710.
Path 363 | total_timesteps 3719.
Path 364 | total_timesteps 3729.
Path 365 | total_timesteps 3739.
Path 366 | total_timesteps 3756.
Path 367 | total_timesteps 3764.
Path 368 | total_timesteps 3775.
Path 369 | total_timesteps 3785.
Path 370 | total_timesteps 3796.
Path 371 | total_timesteps 3805.
Path 372 | total_timesteps 3818.
Path 373 | total_timesteps 3829.
Path 374 | total_timesteps 3835.
Path 375 | total_timesteps 3846.
Path 376 | total_timesteps 3855.
Path 377 | total_timesteps 3863.
Path 378 | total_timesteps 3873.
Path 379 | total_timesteps 3883.
Path 380 | total_timesteps 3891.
Path 381 | total_timesteps 3903.
Path 382 | total_timesteps 3912.
Path 383 | total_timesteps 3921.
Path 384 | total_timesteps 3930.
Path 385 | total_timesteps 3941.
Path 386 | total_timesteps 3952.
Path 387 | total_timesteps 3962.
Path 388 | total_timesteps 3976.
Path 389 | total_timesteps 3982.
Path 390 | total_timesteps 3992.
Path 391 | total_timesteps 4004.
Path 392 | total_timesteps 4013.
Path 393 | total_timesteps 4021.
Path 394 | total_timesteps 4028.
Path 395 | total_timesteps 4043.
Path 396 | total_timesteps 4053.
Path 397 | total_timesteps 4066.
Path 398 | total_timesteps 4074.
Path 399 | total_timesteps 4085.
Path 400 | total_timesteps 4094.
Path 401 | total_timesteps 4103.
Path 402 | total_timesteps 4116.
Path 403 | total_timesteps 4125.
Path 404 | total_timesteps 4135.
Path 405 | total_timesteps 4144.
Path 406 | total_timesteps 4152.
Path 407 | total_timesteps 4160.
Path 408 | total_timesteps 4170.
Path 409 | total_timesteps 4182.
Path 410 | total_timesteps 4190.
Path 411 | total_timesteps 4198.
Path 412 | total_timesteps 4208.
Path 413 | total_timesteps 4216.
Path 414 | total_timesteps 4224.
Path 415 | total_timesteps 4232.
Path 416 | total_timesteps 4243.
Path 417 | total_timesteps 4256.
Path 418 | total_timesteps 4265.
Path 419 | total_timesteps 4277.
Path 420 | total_timesteps 4287.
Path 421 | total_timesteps 4297.
Path 422 | total_timesteps 4314.
Path 423 | total_timesteps 4327.
Path 424 | total_timesteps 4336.
Path 425 | total_timesteps 4348.
Path 426 | total_timesteps 4357.
Path 427 | total_timesteps 4364.
Path 428 | total_timesteps 4374.
Path 429 | total_timesteps 4382.
Path 430 | total_timesteps 4391.
Path 431 | total_timesteps 4403.
Path 432 | total_timesteps 4411.
Path 433 | total_timesteps 4424.
Path 434 | total_timesteps 4432.
Path 435 | total_timesteps 4448.
Path 436 | total_timesteps 4458.
Path 437 | total_timesteps 4465.
Path 438 | total_timesteps 4474.
Path 439 | total_timesteps 4484.
Path 440 | total_timesteps 4492.
Path 441 | total_timesteps 4504.
Path 442 | total_timesteps 4513.
Path 443 | total_timesteps 4523.
Path 444 | total_timesteps 4533.
Path 445 | total_timesteps 4543.
Path 446 | total_timesteps 4551.
Path 447 | total_timesteps 4562.
Path 448 | total_timesteps 4571.
Path 449 | total_timesteps 4578.
Path 450 | total_timesteps 4585.
Path 451 | total_timesteps 4592.
Path 452 | total_timesteps 4601.
Path 453 | total_timesteps 4611.
Path 454 | total_timesteps 4631.
Path 455 | total_timesteps 4646.
Path 456 | total_timesteps 4657.
Path 457 | total_timesteps 4665.
Path 458 | total_timesteps 4676.
Path 459 | total_timesteps 4688.
Path 460 | total_timesteps 4698.
Path 461 | total_timesteps 4706.
Path 462 | total_timesteps 4715.
Path 463 | total_timesteps 4727.
Path 464 | total_timesteps 4739.
Path 465 | total_timesteps 4748.
Path 466 | total_timesteps 4762.
Path 467 | total_timesteps 4773.
Path 468 | total_timesteps 4786.
Path 469 | total_timesteps 4794.
Path 470 | total_timesteps 4803.
Path 471 | total_timesteps 4812.
Path 472 | total_timesteps 4823.
Path 473 | total_timesteps 4832.
Path 474 | total_timesteps 4842.
Path 475 | total_timesteps 4850.
Path 476 | total_timesteps 4859.
Path 477 | total_timesteps 4868.
Path 478 | total_timesteps 4879.
Path 479 | total_timesteps 4889.
Path 480 | total_timesteps 4898.
Path 481 | total_timesteps 4908.
Path 482 | total_timesteps 4915.
Path 483 | total_timesteps 4925.
Path 484 | total_timesteps 4934.
Path 485 | total_timesteps 4943.
Path 486 | total_timesteps 4953.
Path 487 | total_timesteps 4960.
Path 488 | total_timesteps 4968.
Path 489 | total_timesteps 4977.
Path 490 | total_timesteps 4989.
Path 491 | total_timesteps 4997.
Path 492 | total_timesteps 5007.
Path 493 | total_timesteps 5017.
Path 494 | total_timesteps 5028.
Path 495 | total_timesteps 5040.
Path 496 | total_timesteps 5049.
Path 497 | total_timesteps 5059.
Path 498 | total_timesteps 5069.
Path 499 | total_timesteps 5077.
Path 500 | total_timesteps 5090.
Path 501 | total_timesteps 5100.
Path 502 | total_timesteps 5108.
Path 503 | total_timesteps 5117.
Path 504 | total_timesteps 5125.
Path 505 | total_timesteps 5136.
Path 506 | total_timesteps 5144.
Path 507 | total_timesteps 5151.
Path 508 | total_timesteps 5160.
Path 509 | total_timesteps 5170.
Path 510 | total_timesteps 5178.
Path 511 | total_timesteps 5188.
Path 512 | total_timesteps 5197.
Path 513 | total_timesteps 5205.
Path 514 | total_timesteps 5215.
Path 515 | total_timesteps 5223.
Path 516 | total_timesteps 5234.
Path 517 | total_timesteps 5242.
Path 518 | total_timesteps 5251.
Path 519 | total_timesteps 5260.
Path 520 | total_timesteps 5270.
Path 521 | total_timesteps 5280.
Path 522 | total_timesteps 5289.
Path 523 | total_timesteps 5299.
Path 524 | total_timesteps 5308.
Path 525 | total_timesteps 5322.
Path 526 | total_timesteps 5332.
Path 527 | total_timesteps 5343.
Path 528 | total_timesteps 5354.
Path 529 | total_timesteps 5363.
Path 530 | total_timesteps 5372.
Path 531 | total_timesteps 5382.
Path 532 | total_timesteps 5389.
Path 533 | total_timesteps 5399.
Path 534 | total_timesteps 5408.
Path 535 | total_timesteps 5416.
Path 536 | total_timesteps 5424.
Path 537 | total_timesteps 5433.
Path 538 | total_timesteps 5445.
Path 539 | total_timesteps 5459.
Path 540 | total_timesteps 5468.
Path 541 | total_timesteps 5479.
Path 542 | total_timesteps 5494.
Path 543 | total_timesteps 5501.
Path 544 | total_timesteps 5513.
Path 545 | total_timesteps 5523.
Path 546 | total_timesteps 5532.
Path 547 | total_timesteps 5543.
Path 548 | total_timesteps 5553.
Path 549 | total_timesteps 5562.
Path 550 | total_timesteps 5576.
Path 551 | total_timesteps 5586.
Path 552 | total_timesteps 5597.
Path 553 | total_timesteps 5606.
Path 554 | total_timesteps 5614.
Path 555 | total_timesteps 5623.
Path 556 | total_timesteps 5631.
Path 557 | total_timesteps 5639.
Path 558 | total_timesteps 5650.
Path 559 | total_timesteps 5658.
Path 560 | total_timesteps 5669.
Path 561 | total_timesteps 5678.
Path 562 | total_timesteps 5686.
Path 563 | total_timesteps 5693.
Path 564 | total_timesteps 5703.
Path 565 | total_timesteps 5712.
Path 566 | total_timesteps 5720.
Path 567 | total_timesteps 5731.
Path 568 | total_timesteps 5741.
Path 569 | total_timesteps 5758.
Path 570 | total_timesteps 5768.
Path 571 | total_timesteps 5778.
Path 572 | total_timesteps 5786.
Path 573 | total_timesteps 5799.
Path 574 | total_timesteps 5810.
Path 575 | total_timesteps 5818.
Path 576 | total_timesteps 5826.
Path 577 | total_timesteps 5836.
Path 578 | total_timesteps 5845.
Path 579 | total_timesteps 5854.
Path 580 | total_timesteps 5864.
Path 581 | total_timesteps 5873.
Path 582 | total_timesteps 5881.
Path 583 | total_timesteps 5891.
Path 584 | total_timesteps 5899.
Path 585 | total_timesteps 5911.
Path 586 | total_timesteps 5924.
Path 587 | total_timesteps 5936.
Path 588 | total_timesteps 5949.
Path 589 | total_timesteps 5960.
Path 590 | total_timesteps 5970.
Path 591 | total_timesteps 5978.
Path 592 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.12    |
| Iteration     | 23       |
| MaximumReturn | -0.984   |
| MinimumReturn | -18.5    |
| TotalSamples  | 100129   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.0024169471580535173
Validation loss = 0.002362089231610298
Validation loss = 0.0022561547812074423
Validation loss = 0.0023622794542461634
Validation loss = 0.00241798534989357
Validation loss = 0.0022847021464258432
Validation loss = 0.002397460862994194
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 40.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 82.
Path 9 | total_timesteps 95.
Path 10 | total_timesteps 103.
Path 11 | total_timesteps 112.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 132.
Path 14 | total_timesteps 139.
Path 15 | total_timesteps 152.
Path 16 | total_timesteps 163.
Path 17 | total_timesteps 172.
Path 18 | total_timesteps 180.
Path 19 | total_timesteps 195.
Path 20 | total_timesteps 203.
Path 21 | total_timesteps 212.
Path 22 | total_timesteps 220.
Path 23 | total_timesteps 228.
Path 24 | total_timesteps 235.
Path 25 | total_timesteps 246.
Path 26 | total_timesteps 254.
Path 27 | total_timesteps 264.
Path 28 | total_timesteps 274.
Path 29 | total_timesteps 282.
Path 30 | total_timesteps 291.
Path 31 | total_timesteps 299.
Path 32 | total_timesteps 308.
Path 33 | total_timesteps 318.
Path 34 | total_timesteps 327.
Path 35 | total_timesteps 338.
Path 36 | total_timesteps 345.
Path 37 | total_timesteps 356.
Path 38 | total_timesteps 366.
Path 39 | total_timesteps 376.
Path 40 | total_timesteps 386.
Path 41 | total_timesteps 393.
Path 42 | total_timesteps 404.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 427.
Path 45 | total_timesteps 440.
Path 46 | total_timesteps 452.
Path 47 | total_timesteps 459.
Path 48 | total_timesteps 469.
Path 49 | total_timesteps 483.
Path 50 | total_timesteps 494.
Path 51 | total_timesteps 502.
Path 52 | total_timesteps 515.
Path 53 | total_timesteps 528.
Path 54 | total_timesteps 536.
Path 55 | total_timesteps 544.
Path 56 | total_timesteps 558.
Path 57 | total_timesteps 572.
Path 58 | total_timesteps 584.
Path 59 | total_timesteps 593.
Path 60 | total_timesteps 600.
Path 61 | total_timesteps 608.
Path 62 | total_timesteps 619.
Path 63 | total_timesteps 630.
Path 64 | total_timesteps 643.
Path 65 | total_timesteps 655.
Path 66 | total_timesteps 665.
Path 67 | total_timesteps 676.
Path 68 | total_timesteps 684.
Path 69 | total_timesteps 697.
Path 70 | total_timesteps 709.
Path 71 | total_timesteps 720.
Path 72 | total_timesteps 733.
Path 73 | total_timesteps 747.
Path 74 | total_timesteps 760.
Path 75 | total_timesteps 771.
Path 76 | total_timesteps 779.
Path 77 | total_timesteps 787.
Path 78 | total_timesteps 795.
Path 79 | total_timesteps 804.
Path 80 | total_timesteps 817.
Path 81 | total_timesteps 825.
Path 82 | total_timesteps 836.
Path 83 | total_timesteps 845.
Path 84 | total_timesteps 860.
Path 85 | total_timesteps 867.
Path 86 | total_timesteps 876.
Path 87 | total_timesteps 886.
Path 88 | total_timesteps 893.
Path 89 | total_timesteps 902.
Path 90 | total_timesteps 910.
Path 91 | total_timesteps 919.
Path 92 | total_timesteps 929.
Path 93 | total_timesteps 938.
Path 94 | total_timesteps 948.
Path 95 | total_timesteps 957.
Path 96 | total_timesteps 968.
Path 97 | total_timesteps 976.
Path 98 | total_timesteps 983.
Path 99 | total_timesteps 991.
Path 100 | total_timesteps 1001.
Path 101 | total_timesteps 1011.
Path 102 | total_timesteps 1021.
Path 103 | total_timesteps 1030.
Path 104 | total_timesteps 1039.
Path 105 | total_timesteps 1048.
Path 106 | total_timesteps 1057.
Path 107 | total_timesteps 1068.
Path 108 | total_timesteps 1077.
Path 109 | total_timesteps 1091.
Path 110 | total_timesteps 1098.
Path 111 | total_timesteps 1109.
Path 112 | total_timesteps 1120.
Path 113 | total_timesteps 1128.
Path 114 | total_timesteps 1138.
Path 115 | total_timesteps 1148.
Path 116 | total_timesteps 1158.
Path 117 | total_timesteps 1168.
Path 118 | total_timesteps 1176.
Path 119 | total_timesteps 1186.
Path 120 | total_timesteps 1196.
Path 121 | total_timesteps 1204.
Path 122 | total_timesteps 1221.
Path 123 | total_timesteps 1230.
Path 124 | total_timesteps 1241.
Path 125 | total_timesteps 1249.
Path 126 | total_timesteps 1258.
Path 127 | total_timesteps 1266.
Path 128 | total_timesteps 1274.
Path 129 | total_timesteps 1284.
Path 130 | total_timesteps 1292.
Path 131 | total_timesteps 1302.
Path 132 | total_timesteps 1311.
Path 133 | total_timesteps 1321.
Path 134 | total_timesteps 1331.
Path 135 | total_timesteps 1340.
Path 136 | total_timesteps 1350.
Path 137 | total_timesteps 1363.
Path 138 | total_timesteps 1372.
Path 139 | total_timesteps 1382.
Path 140 | total_timesteps 1394.
Path 141 | total_timesteps 1402.
Path 142 | total_timesteps 1410.
Path 143 | total_timesteps 1419.
Path 144 | total_timesteps 1429.
Path 145 | total_timesteps 1442.
Path 146 | total_timesteps 1451.
Path 147 | total_timesteps 1466.
Path 148 | total_timesteps 1475.
Path 149 | total_timesteps 1485.
Path 150 | total_timesteps 1496.
Path 151 | total_timesteps 1506.
Path 152 | total_timesteps 1515.
Path 153 | total_timesteps 1526.
Path 154 | total_timesteps 1533.
Path 155 | total_timesteps 1542.
Path 156 | total_timesteps 1550.
Path 157 | total_timesteps 1561.
Path 158 | total_timesteps 1572.
Path 159 | total_timesteps 1582.
Path 160 | total_timesteps 1592.
Path 161 | total_timesteps 1602.
Path 162 | total_timesteps 1612.
Path 163 | total_timesteps 1624.
Path 164 | total_timesteps 1633.
Path 165 | total_timesteps 1641.
Path 166 | total_timesteps 1652.
Path 167 | total_timesteps 1672.
Path 168 | total_timesteps 1681.
Path 169 | total_timesteps 1688.
Path 170 | total_timesteps 1700.
Path 171 | total_timesteps 1709.
Path 172 | total_timesteps 1719.
Path 173 | total_timesteps 1727.
Path 174 | total_timesteps 1743.
Path 175 | total_timesteps 1752.
Path 176 | total_timesteps 1765.
Path 177 | total_timesteps 1773.
Path 178 | total_timesteps 1784.
Path 179 | total_timesteps 1794.
Path 180 | total_timesteps 1806.
Path 181 | total_timesteps 1818.
Path 182 | total_timesteps 1830.
Path 183 | total_timesteps 1841.
Path 184 | total_timesteps 1850.
Path 185 | total_timesteps 1858.
Path 186 | total_timesteps 1867.
Path 187 | total_timesteps 1877.
Path 188 | total_timesteps 1886.
Path 189 | total_timesteps 1897.
Path 190 | total_timesteps 1909.
Path 191 | total_timesteps 1917.
Path 192 | total_timesteps 1926.
Path 193 | total_timesteps 1938.
Path 194 | total_timesteps 1948.
Path 195 | total_timesteps 1960.
Path 196 | total_timesteps 1969.
Path 197 | total_timesteps 1982.
Path 198 | total_timesteps 1991.
Path 199 | total_timesteps 1997.
Path 200 | total_timesteps 2008.
Path 201 | total_timesteps 2022.
Path 202 | total_timesteps 2029.
Path 203 | total_timesteps 2039.
Path 204 | total_timesteps 2050.
Path 205 | total_timesteps 2058.
Path 206 | total_timesteps 2073.
Path 207 | total_timesteps 2081.
Path 208 | total_timesteps 2091.
Path 209 | total_timesteps 2104.
Path 210 | total_timesteps 2113.
Path 211 | total_timesteps 2124.
Path 212 | total_timesteps 2130.
Path 213 | total_timesteps 2138.
Path 214 | total_timesteps 2155.
Path 215 | total_timesteps 2164.
Path 216 | total_timesteps 2176.
Path 217 | total_timesteps 2183.
Path 218 | total_timesteps 2191.
Path 219 | total_timesteps 2203.
Path 220 | total_timesteps 2211.
Path 221 | total_timesteps 2220.
Path 222 | total_timesteps 2232.
Path 223 | total_timesteps 2246.
Path 224 | total_timesteps 2253.
Path 225 | total_timesteps 2262.
Path 226 | total_timesteps 2275.
Path 227 | total_timesteps 2287.
Path 228 | total_timesteps 2295.
Path 229 | total_timesteps 2306.
Path 230 | total_timesteps 2314.
Path 231 | total_timesteps 2325.
Path 232 | total_timesteps 2333.
Path 233 | total_timesteps 2342.
Path 234 | total_timesteps 2359.
Path 235 | total_timesteps 2368.
Path 236 | total_timesteps 2379.
Path 237 | total_timesteps 2391.
Path 238 | total_timesteps 2399.
Path 239 | total_timesteps 2410.
Path 240 | total_timesteps 2422.
Path 241 | total_timesteps 2433.
Path 242 | total_timesteps 2441.
Path 243 | total_timesteps 2453.
Path 244 | total_timesteps 2461.
Path 245 | total_timesteps 2467.
Path 246 | total_timesteps 2477.
Path 247 | total_timesteps 2489.
Path 248 | total_timesteps 2499.
Path 249 | total_timesteps 2509.
Path 250 | total_timesteps 2524.
Path 251 | total_timesteps 2532.
Path 252 | total_timesteps 2544.
Path 253 | total_timesteps 2552.
Path 254 | total_timesteps 2562.
Path 255 | total_timesteps 2570.
Path 256 | total_timesteps 2578.
Path 257 | total_timesteps 2588.
Path 258 | total_timesteps 2598.
Path 259 | total_timesteps 2608.
Path 260 | total_timesteps 2614.
Path 261 | total_timesteps 2624.
Path 262 | total_timesteps 2634.
Path 263 | total_timesteps 2648.
Path 264 | total_timesteps 2656.
Path 265 | total_timesteps 2666.
Path 266 | total_timesteps 2676.
Path 267 | total_timesteps 2689.
Path 268 | total_timesteps 2705.
Path 269 | total_timesteps 2714.
Path 270 | total_timesteps 2724.
Path 271 | total_timesteps 2736.
Path 272 | total_timesteps 2745.
Path 273 | total_timesteps 2755.
Path 274 | total_timesteps 2762.
Path 275 | total_timesteps 2770.
Path 276 | total_timesteps 2779.
Path 277 | total_timesteps 2791.
Path 278 | total_timesteps 2802.
Path 279 | total_timesteps 2814.
Path 280 | total_timesteps 2821.
Path 281 | total_timesteps 2828.
Path 282 | total_timesteps 2840.
Path 283 | total_timesteps 2851.
Path 284 | total_timesteps 2866.
Path 285 | total_timesteps 2883.
Path 286 | total_timesteps 2893.
Path 287 | total_timesteps 2908.
Path 288 | total_timesteps 2919.
Path 289 | total_timesteps 2930.
Path 290 | total_timesteps 2940.
Path 291 | total_timesteps 2948.
Path 292 | total_timesteps 2957.
Path 293 | total_timesteps 2967.
Path 294 | total_timesteps 2980.
Path 295 | total_timesteps 2992.
Path 296 | total_timesteps 3003.
Path 297 | total_timesteps 3016.
Path 298 | total_timesteps 3024.
Path 299 | total_timesteps 3031.
Path 300 | total_timesteps 3041.
Path 301 | total_timesteps 3048.
Path 302 | total_timesteps 3059.
Path 303 | total_timesteps 3068.
Path 304 | total_timesteps 3083.
Path 305 | total_timesteps 3092.
Path 306 | total_timesteps 3111.
Path 307 | total_timesteps 3121.
Path 308 | total_timesteps 3129.
Path 309 | total_timesteps 3139.
Path 310 | total_timesteps 3148.
Path 311 | total_timesteps 3159.
Path 312 | total_timesteps 3172.
Path 313 | total_timesteps 3183.
Path 314 | total_timesteps 3192.
Path 315 | total_timesteps 3201.
Path 316 | total_timesteps 3213.
Path 317 | total_timesteps 3225.
Path 318 | total_timesteps 3234.
Path 319 | total_timesteps 3246.
Path 320 | total_timesteps 3254.
Path 321 | total_timesteps 3263.
Path 322 | total_timesteps 3273.
Path 323 | total_timesteps 3287.
Path 324 | total_timesteps 3296.
Path 325 | total_timesteps 3303.
Path 326 | total_timesteps 3313.
Path 327 | total_timesteps 3322.
Path 328 | total_timesteps 3336.
Path 329 | total_timesteps 3343.
Path 330 | total_timesteps 3350.
Path 331 | total_timesteps 3358.
Path 332 | total_timesteps 3366.
Path 333 | total_timesteps 3377.
Path 334 | total_timesteps 3391.
Path 335 | total_timesteps 3405.
Path 336 | total_timesteps 3413.
Path 337 | total_timesteps 3422.
Path 338 | total_timesteps 3434.
Path 339 | total_timesteps 3442.
Path 340 | total_timesteps 3455.
Path 341 | total_timesteps 3468.
Path 342 | total_timesteps 3479.
Path 343 | total_timesteps 3491.
Path 344 | total_timesteps 3501.
Path 345 | total_timesteps 3507.
Path 346 | total_timesteps 3515.
Path 347 | total_timesteps 3525.
Path 348 | total_timesteps 3533.
Path 349 | total_timesteps 3546.
Path 350 | total_timesteps 3555.
Path 351 | total_timesteps 3568.
Path 352 | total_timesteps 3577.
Path 353 | total_timesteps 3590.
Path 354 | total_timesteps 3599.
Path 355 | total_timesteps 3607.
Path 356 | total_timesteps 3624.
Path 357 | total_timesteps 3631.
Path 358 | total_timesteps 3640.
Path 359 | total_timesteps 3647.
Path 360 | total_timesteps 3660.
Path 361 | total_timesteps 3671.
Path 362 | total_timesteps 3681.
Path 363 | total_timesteps 3692.
Path 364 | total_timesteps 3705.
Path 365 | total_timesteps 3718.
Path 366 | total_timesteps 3729.
Path 367 | total_timesteps 3741.
Path 368 | total_timesteps 3754.
Path 369 | total_timesteps 3764.
Path 370 | total_timesteps 3776.
Path 371 | total_timesteps 3789.
Path 372 | total_timesteps 3801.
Path 373 | total_timesteps 3809.
Path 374 | total_timesteps 3817.
Path 375 | total_timesteps 3826.
Path 376 | total_timesteps 3836.
Path 377 | total_timesteps 3846.
Path 378 | total_timesteps 3856.
Path 379 | total_timesteps 3866.
Path 380 | total_timesteps 3876.
Path 381 | total_timesteps 3889.
Path 382 | total_timesteps 3898.
Path 383 | total_timesteps 3907.
Path 384 | total_timesteps 3915.
Path 385 | total_timesteps 3925.
Path 386 | total_timesteps 3932.
Path 387 | total_timesteps 3942.
Path 388 | total_timesteps 3950.
Path 389 | total_timesteps 3960.
Path 390 | total_timesteps 3972.
Path 391 | total_timesteps 3982.
Path 392 | total_timesteps 3991.
Path 393 | total_timesteps 3998.
Path 394 | total_timesteps 4010.
Path 395 | total_timesteps 4022.
Path 396 | total_timesteps 4033.
Path 397 | total_timesteps 4042.
Path 398 | total_timesteps 4054.
Path 399 | total_timesteps 4062.
Path 400 | total_timesteps 4071.
Path 401 | total_timesteps 4078.
Path 402 | total_timesteps 4088.
Path 403 | total_timesteps 4095.
Path 404 | total_timesteps 4102.
Path 405 | total_timesteps 4109.
Path 406 | total_timesteps 4120.
Path 407 | total_timesteps 4130.
Path 408 | total_timesteps 4142.
Path 409 | total_timesteps 4149.
Path 410 | total_timesteps 4165.
Path 411 | total_timesteps 4172.
Path 412 | total_timesteps 4180.
Path 413 | total_timesteps 4192.
Path 414 | total_timesteps 4202.
Path 415 | total_timesteps 4221.
Path 416 | total_timesteps 4229.
Path 417 | total_timesteps 4243.
Path 418 | total_timesteps 4254.
Path 419 | total_timesteps 4263.
Path 420 | total_timesteps 4271.
Path 421 | total_timesteps 4281.
Path 422 | total_timesteps 4288.
Path 423 | total_timesteps 4297.
Path 424 | total_timesteps 4322.
Path 425 | total_timesteps 4333.
Path 426 | total_timesteps 4344.
Path 427 | total_timesteps 4353.
Path 428 | total_timesteps 4364.
Path 429 | total_timesteps 4375.
Path 430 | total_timesteps 4382.
Path 431 | total_timesteps 4390.
Path 432 | total_timesteps 4402.
Path 433 | total_timesteps 4411.
Path 434 | total_timesteps 4421.
Path 435 | total_timesteps 4429.
Path 436 | total_timesteps 4443.
Path 437 | total_timesteps 4452.
Path 438 | total_timesteps 4467.
Path 439 | total_timesteps 4477.
Path 440 | total_timesteps 4486.
Path 441 | total_timesteps 4501.
Path 442 | total_timesteps 4512.
Path 443 | total_timesteps 4522.
Path 444 | total_timesteps 4532.
Path 445 | total_timesteps 4540.
Path 446 | total_timesteps 4552.
Path 447 | total_timesteps 4559.
Path 448 | total_timesteps 4566.
Path 449 | total_timesteps 4581.
Path 450 | total_timesteps 4594.
Path 451 | total_timesteps 4606.
Path 452 | total_timesteps 4614.
Path 453 | total_timesteps 4623.
Path 454 | total_timesteps 4632.
Path 455 | total_timesteps 4639.
Path 456 | total_timesteps 4649.
Path 457 | total_timesteps 4661.
Path 458 | total_timesteps 4672.
Path 459 | total_timesteps 4686.
Path 460 | total_timesteps 4697.
Path 461 | total_timesteps 4707.
Path 462 | total_timesteps 4715.
Path 463 | total_timesteps 4723.
Path 464 | total_timesteps 4735.
Path 465 | total_timesteps 4743.
Path 466 | total_timesteps 4752.
Path 467 | total_timesteps 4769.
Path 468 | total_timesteps 4785.
Path 469 | total_timesteps 4794.
Path 470 | total_timesteps 4802.
Path 471 | total_timesteps 4809.
Path 472 | total_timesteps 4817.
Path 473 | total_timesteps 4834.
Path 474 | total_timesteps 4847.
Path 475 | total_timesteps 4856.
Path 476 | total_timesteps 4864.
Path 477 | total_timesteps 4882.
Path 478 | total_timesteps 4891.
Path 479 | total_timesteps 4902.
Path 480 | total_timesteps 4912.
Path 481 | total_timesteps 4926.
Path 482 | total_timesteps 4933.
Path 483 | total_timesteps 4942.
Path 484 | total_timesteps 4952.
Path 485 | total_timesteps 4963.
Path 486 | total_timesteps 4975.
Path 487 | total_timesteps 4982.
Path 488 | total_timesteps 4994.
Path 489 | total_timesteps 5003.
Path 490 | total_timesteps 5014.
Path 491 | total_timesteps 5021.
Path 492 | total_timesteps 5029.
Path 493 | total_timesteps 5042.
Path 494 | total_timesteps 5052.
Path 495 | total_timesteps 5063.
Path 496 | total_timesteps 5070.
Path 497 | total_timesteps 5078.
Path 498 | total_timesteps 5087.
Path 499 | total_timesteps 5100.
Path 500 | total_timesteps 5107.
Path 501 | total_timesteps 5120.
Path 502 | total_timesteps 5129.
Path 503 | total_timesteps 5138.
Path 504 | total_timesteps 5146.
Path 505 | total_timesteps 5157.
Path 506 | total_timesteps 5163.
Path 507 | total_timesteps 5172.
Path 508 | total_timesteps 5182.
Path 509 | total_timesteps 5205.
Path 510 | total_timesteps 5217.
Path 511 | total_timesteps 5228.
Path 512 | total_timesteps 5239.
Path 513 | total_timesteps 5248.
Path 514 | total_timesteps 5259.
Path 515 | total_timesteps 5271.
Path 516 | total_timesteps 5281.
Path 517 | total_timesteps 5289.
Path 518 | total_timesteps 5304.
Path 519 | total_timesteps 5317.
Path 520 | total_timesteps 5325.
Path 521 | total_timesteps 5334.
Path 522 | total_timesteps 5343.
Path 523 | total_timesteps 5354.
Path 524 | total_timesteps 5364.
Path 525 | total_timesteps 5373.
Path 526 | total_timesteps 5380.
Path 527 | total_timesteps 5394.
Path 528 | total_timesteps 5403.
Path 529 | total_timesteps 5414.
Path 530 | total_timesteps 5424.
Path 531 | total_timesteps 5431.
Path 532 | total_timesteps 5443.
Path 533 | total_timesteps 5454.
Path 534 | total_timesteps 5465.
Path 535 | total_timesteps 5477.
Path 536 | total_timesteps 5487.
Path 537 | total_timesteps 5497.
Path 538 | total_timesteps 5508.
Path 539 | total_timesteps 5519.
Path 540 | total_timesteps 5530.
Path 541 | total_timesteps 5538.
Path 542 | total_timesteps 5548.
Path 543 | total_timesteps 5556.
Path 544 | total_timesteps 5566.
Path 545 | total_timesteps 5576.
Path 546 | total_timesteps 5585.
Path 547 | total_timesteps 5597.
Path 548 | total_timesteps 5606.
Path 549 | total_timesteps 5620.
Path 550 | total_timesteps 5630.
Path 551 | total_timesteps 5640.
Path 552 | total_timesteps 5650.
Path 553 | total_timesteps 5662.
Path 554 | total_timesteps 5676.
Path 555 | total_timesteps 5688.
Path 556 | total_timesteps 5696.
Path 557 | total_timesteps 5712.
Path 558 | total_timesteps 5721.
Path 559 | total_timesteps 5730.
Path 560 | total_timesteps 5740.
Path 561 | total_timesteps 5749.
Path 562 | total_timesteps 5757.
Path 563 | total_timesteps 5767.
Path 564 | total_timesteps 5775.
Path 565 | total_timesteps 5783.
Path 566 | total_timesteps 5790.
Path 567 | total_timesteps 5803.
Path 568 | total_timesteps 5809.
Path 569 | total_timesteps 5821.
Path 570 | total_timesteps 5827.
Path 571 | total_timesteps 5834.
Path 572 | total_timesteps 5842.
Path 573 | total_timesteps 5851.
Path 574 | total_timesteps 5860.
Path 575 | total_timesteps 5870.
Path 576 | total_timesteps 5883.
Path 577 | total_timesteps 5894.
Path 578 | total_timesteps 5904.
Path 579 | total_timesteps 5914.
Path 580 | total_timesteps 5924.
Path 581 | total_timesteps 5933.
Path 582 | total_timesteps 5946.
Path 583 | total_timesteps 5954.
Path 584 | total_timesteps 5960.
Path 585 | total_timesteps 5971.
Path 586 | total_timesteps 5979.
Path 587 | total_timesteps 5989.
Path 588 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.88    |
| Iteration     | 24       |
| MaximumReturn | -0.799   |
| MinimumReturn | -14.1    |
| TotalSamples  | 104133   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.0021639575716108084
Validation loss = 0.002374716103076935
Validation loss = 0.0021911661606281996
Validation loss = 0.0019939416088163853
Validation loss = 0.0022092026192694902
Validation loss = 0.0019425838254392147
Validation loss = 0.0021983326878398657
Validation loss = 0.002280634129419923
Validation loss = 0.0021196287125349045
Validation loss = 0.0028380309231579304
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 54.
Path 7 | total_timesteps 64.
Path 8 | total_timesteps 77.
Path 9 | total_timesteps 85.
Path 10 | total_timesteps 93.
Path 11 | total_timesteps 103.
Path 12 | total_timesteps 115.
Path 13 | total_timesteps 126.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 147.
Path 16 | total_timesteps 158.
Path 17 | total_timesteps 166.
Path 18 | total_timesteps 180.
Path 19 | total_timesteps 192.
Path 20 | total_timesteps 203.
Path 21 | total_timesteps 210.
Path 22 | total_timesteps 222.
Path 23 | total_timesteps 231.
Path 24 | total_timesteps 240.
Path 25 | total_timesteps 248.
Path 26 | total_timesteps 259.
Path 27 | total_timesteps 269.
Path 28 | total_timesteps 279.
Path 29 | total_timesteps 291.
Path 30 | total_timesteps 305.
Path 31 | total_timesteps 315.
Path 32 | total_timesteps 323.
Path 33 | total_timesteps 334.
Path 34 | total_timesteps 345.
Path 35 | total_timesteps 355.
Path 36 | total_timesteps 361.
Path 37 | total_timesteps 370.
Path 38 | total_timesteps 380.
Path 39 | total_timesteps 390.
Path 40 | total_timesteps 399.
Path 41 | total_timesteps 409.
Path 42 | total_timesteps 418.
Path 43 | total_timesteps 424.
Path 44 | total_timesteps 431.
Path 45 | total_timesteps 440.
Path 46 | total_timesteps 449.
Path 47 | total_timesteps 459.
Path 48 | total_timesteps 470.
Path 49 | total_timesteps 479.
Path 50 | total_timesteps 489.
Path 51 | total_timesteps 497.
Path 52 | total_timesteps 505.
Path 53 | total_timesteps 514.
Path 54 | total_timesteps 523.
Path 55 | total_timesteps 531.
Path 56 | total_timesteps 538.
Path 57 | total_timesteps 547.
Path 58 | total_timesteps 556.
Path 59 | total_timesteps 568.
Path 60 | total_timesteps 583.
Path 61 | total_timesteps 592.
Path 62 | total_timesteps 602.
Path 63 | total_timesteps 609.
Path 64 | total_timesteps 617.
Path 65 | total_timesteps 625.
Path 66 | total_timesteps 634.
Path 67 | total_timesteps 646.
Path 68 | total_timesteps 654.
Path 69 | total_timesteps 663.
Path 70 | total_timesteps 671.
Path 71 | total_timesteps 679.
Path 72 | total_timesteps 688.
Path 73 | total_timesteps 695.
Path 74 | total_timesteps 703.
Path 75 | total_timesteps 712.
Path 76 | total_timesteps 726.
Path 77 | total_timesteps 735.
Path 78 | total_timesteps 744.
Path 79 | total_timesteps 755.
Path 80 | total_timesteps 768.
Path 81 | total_timesteps 775.
Path 82 | total_timesteps 790.
Path 83 | total_timesteps 800.
Path 84 | total_timesteps 806.
Path 85 | total_timesteps 814.
Path 86 | total_timesteps 831.
Path 87 | total_timesteps 844.
Path 88 | total_timesteps 855.
Path 89 | total_timesteps 864.
Path 90 | total_timesteps 874.
Path 91 | total_timesteps 881.
Path 92 | total_timesteps 893.
Path 93 | total_timesteps 901.
Path 94 | total_timesteps 914.
Path 95 | total_timesteps 922.
Path 96 | total_timesteps 931.
Path 97 | total_timesteps 941.
Path 98 | total_timesteps 949.
Path 99 | total_timesteps 956.
Path 100 | total_timesteps 964.
Path 101 | total_timesteps 978.
Path 102 | total_timesteps 987.
Path 103 | total_timesteps 994.
Path 104 | total_timesteps 1012.
Path 105 | total_timesteps 1019.
Path 106 | total_timesteps 1027.
Path 107 | total_timesteps 1036.
Path 108 | total_timesteps 1044.
Path 109 | total_timesteps 1052.
Path 110 | total_timesteps 1060.
Path 111 | total_timesteps 1071.
Path 112 | total_timesteps 1080.
Path 113 | total_timesteps 1087.
Path 114 | total_timesteps 1097.
Path 115 | total_timesteps 1104.
Path 116 | total_timesteps 1113.
Path 117 | total_timesteps 1124.
Path 118 | total_timesteps 1132.
Path 119 | total_timesteps 1142.
Path 120 | total_timesteps 1152.
Path 121 | total_timesteps 1162.
Path 122 | total_timesteps 1172.
Path 123 | total_timesteps 1186.
Path 124 | total_timesteps 1199.
Path 125 | total_timesteps 1205.
Path 126 | total_timesteps 1215.
Path 127 | total_timesteps 1223.
Path 128 | total_timesteps 1232.
Path 129 | total_timesteps 1241.
Path 130 | total_timesteps 1252.
Path 131 | total_timesteps 1260.
Path 132 | total_timesteps 1274.
Path 133 | total_timesteps 1290.
Path 134 | total_timesteps 1302.
Path 135 | total_timesteps 1311.
Path 136 | total_timesteps 1321.
Path 137 | total_timesteps 1334.
Path 138 | total_timesteps 1343.
Path 139 | total_timesteps 1353.
Path 140 | total_timesteps 1376.
Path 141 | total_timesteps 1383.
Path 142 | total_timesteps 1391.
Path 143 | total_timesteps 1400.
Path 144 | total_timesteps 1408.
Path 145 | total_timesteps 1415.
Path 146 | total_timesteps 1422.
Path 147 | total_timesteps 1430.
Path 148 | total_timesteps 1441.
Path 149 | total_timesteps 1449.
Path 150 | total_timesteps 1464.
Path 151 | total_timesteps 1475.
Path 152 | total_timesteps 1485.
Path 153 | total_timesteps 1493.
Path 154 | total_timesteps 1500.
Path 155 | total_timesteps 1510.
Path 156 | total_timesteps 1517.
Path 157 | total_timesteps 1526.
Path 158 | total_timesteps 1535.
Path 159 | total_timesteps 1541.
Path 160 | total_timesteps 1551.
Path 161 | total_timesteps 1559.
Path 162 | total_timesteps 1567.
Path 163 | total_timesteps 1576.
Path 164 | total_timesteps 1584.
Path 165 | total_timesteps 1596.
Path 166 | total_timesteps 1610.
Path 167 | total_timesteps 1621.
Path 168 | total_timesteps 1630.
Path 169 | total_timesteps 1639.
Path 170 | total_timesteps 1647.
Path 171 | total_timesteps 1653.
Path 172 | total_timesteps 1663.
Path 173 | total_timesteps 1669.
Path 174 | total_timesteps 1681.
Path 175 | total_timesteps 1694.
Path 176 | total_timesteps 1703.
Path 177 | total_timesteps 1712.
Path 178 | total_timesteps 1720.
Path 179 | total_timesteps 1728.
Path 180 | total_timesteps 1737.
Path 181 | total_timesteps 1745.
Path 182 | total_timesteps 1752.
Path 183 | total_timesteps 1761.
Path 184 | total_timesteps 1771.
Path 185 | total_timesteps 1778.
Path 186 | total_timesteps 1788.
Path 187 | total_timesteps 1796.
Path 188 | total_timesteps 1806.
Path 189 | total_timesteps 1815.
Path 190 | total_timesteps 1822.
Path 191 | total_timesteps 1830.
Path 192 | total_timesteps 1838.
Path 193 | total_timesteps 1845.
Path 194 | total_timesteps 1854.
Path 195 | total_timesteps 1861.
Path 196 | total_timesteps 1870.
Path 197 | total_timesteps 1881.
Path 198 | total_timesteps 1890.
Path 199 | total_timesteps 1897.
Path 200 | total_timesteps 1907.
Path 201 | total_timesteps 1915.
Path 202 | total_timesteps 1923.
Path 203 | total_timesteps 1931.
Path 204 | total_timesteps 1939.
Path 205 | total_timesteps 1947.
Path 206 | total_timesteps 1956.
Path 207 | total_timesteps 1965.
Path 208 | total_timesteps 1975.
Path 209 | total_timesteps 1987.
Path 210 | total_timesteps 1994.
Path 211 | total_timesteps 2001.
Path 212 | total_timesteps 2017.
Path 213 | total_timesteps 2027.
Path 214 | total_timesteps 2035.
Path 215 | total_timesteps 2048.
Path 216 | total_timesteps 2060.
Path 217 | total_timesteps 2067.
Path 218 | total_timesteps 2075.
Path 219 | total_timesteps 2085.
Path 220 | total_timesteps 2093.
Path 221 | total_timesteps 2103.
Path 222 | total_timesteps 2112.
Path 223 | total_timesteps 2120.
Path 224 | total_timesteps 2129.
Path 225 | total_timesteps 2137.
Path 226 | total_timesteps 2148.
Path 227 | total_timesteps 2157.
Path 228 | total_timesteps 2166.
Path 229 | total_timesteps 2173.
Path 230 | total_timesteps 2181.
Path 231 | total_timesteps 2195.
Path 232 | total_timesteps 2206.
Path 233 | total_timesteps 2213.
Path 234 | total_timesteps 2222.
Path 235 | total_timesteps 2231.
Path 236 | total_timesteps 2242.
Path 237 | total_timesteps 2250.
Path 238 | total_timesteps 2264.
Path 239 | total_timesteps 2274.
Path 240 | total_timesteps 2285.
Path 241 | total_timesteps 2298.
Path 242 | total_timesteps 2307.
Path 243 | total_timesteps 2318.
Path 244 | total_timesteps 2330.
Path 245 | total_timesteps 2340.
Path 246 | total_timesteps 2350.
Path 247 | total_timesteps 2360.
Path 248 | total_timesteps 2367.
Path 249 | total_timesteps 2376.
Path 250 | total_timesteps 2384.
Path 251 | total_timesteps 2394.
Path 252 | total_timesteps 2409.
Path 253 | total_timesteps 2417.
Path 254 | total_timesteps 2428.
Path 255 | total_timesteps 2436.
Path 256 | total_timesteps 2445.
Path 257 | total_timesteps 2453.
Path 258 | total_timesteps 2462.
Path 259 | total_timesteps 2471.
Path 260 | total_timesteps 2479.
Path 261 | total_timesteps 2491.
Path 262 | total_timesteps 2500.
Path 263 | total_timesteps 2508.
Path 264 | total_timesteps 2516.
Path 265 | total_timesteps 2529.
Path 266 | total_timesteps 2539.
Path 267 | total_timesteps 2548.
Path 268 | total_timesteps 2557.
Path 269 | total_timesteps 2566.
Path 270 | total_timesteps 2572.
Path 271 | total_timesteps 2582.
Path 272 | total_timesteps 2594.
Path 273 | total_timesteps 2601.
Path 274 | total_timesteps 2614.
Path 275 | total_timesteps 2623.
Path 276 | total_timesteps 2634.
Path 277 | total_timesteps 2642.
Path 278 | total_timesteps 2651.
Path 279 | total_timesteps 2660.
Path 280 | total_timesteps 2669.
Path 281 | total_timesteps 2679.
Path 282 | total_timesteps 2689.
Path 283 | total_timesteps 2699.
Path 284 | total_timesteps 2707.
Path 285 | total_timesteps 2719.
Path 286 | total_timesteps 2729.
Path 287 | total_timesteps 2738.
Path 288 | total_timesteps 2747.
Path 289 | total_timesteps 2755.
Path 290 | total_timesteps 2763.
Path 291 | total_timesteps 2773.
Path 292 | total_timesteps 2780.
Path 293 | total_timesteps 2788.
Path 294 | total_timesteps 2800.
Path 295 | total_timesteps 2808.
Path 296 | total_timesteps 2820.
Path 297 | total_timesteps 2828.
Path 298 | total_timesteps 2835.
Path 299 | total_timesteps 2846.
Path 300 | total_timesteps 2856.
Path 301 | total_timesteps 2865.
Path 302 | total_timesteps 2879.
Path 303 | total_timesteps 2886.
Path 304 | total_timesteps 2894.
Path 305 | total_timesteps 2902.
Path 306 | total_timesteps 2910.
Path 307 | total_timesteps 2925.
Path 308 | total_timesteps 2937.
Path 309 | total_timesteps 2947.
Path 310 | total_timesteps 2954.
Path 311 | total_timesteps 2965.
Path 312 | total_timesteps 2976.
Path 313 | total_timesteps 2985.
Path 314 | total_timesteps 2994.
Path 315 | total_timesteps 3001.
Path 316 | total_timesteps 3015.
Path 317 | total_timesteps 3026.
Path 318 | total_timesteps 3036.
Path 319 | total_timesteps 3049.
Path 320 | total_timesteps 3059.
Path 321 | total_timesteps 3067.
Path 322 | total_timesteps 3075.
Path 323 | total_timesteps 3083.
Path 324 | total_timesteps 3093.
Path 325 | total_timesteps 3101.
Path 326 | total_timesteps 3108.
Path 327 | total_timesteps 3117.
Path 328 | total_timesteps 3127.
Path 329 | total_timesteps 3135.
Path 330 | total_timesteps 3144.
Path 331 | total_timesteps 3153.
Path 332 | total_timesteps 3163.
Path 333 | total_timesteps 3175.
Path 334 | total_timesteps 3184.
Path 335 | total_timesteps 3193.
Path 336 | total_timesteps 3203.
Path 337 | total_timesteps 3212.
Path 338 | total_timesteps 3221.
Path 339 | total_timesteps 3230.
Path 340 | total_timesteps 3239.
Path 341 | total_timesteps 3246.
Path 342 | total_timesteps 3258.
Path 343 | total_timesteps 3271.
Path 344 | total_timesteps 3279.
Path 345 | total_timesteps 3287.
Path 346 | total_timesteps 3297.
Path 347 | total_timesteps 3312.
Path 348 | total_timesteps 3319.
Path 349 | total_timesteps 3326.
Path 350 | total_timesteps 3335.
Path 351 | total_timesteps 3346.
Path 352 | total_timesteps 3356.
Path 353 | total_timesteps 3365.
Path 354 | total_timesteps 3374.
Path 355 | total_timesteps 3382.
Path 356 | total_timesteps 3390.
Path 357 | total_timesteps 3396.
Path 358 | total_timesteps 3405.
Path 359 | total_timesteps 3416.
Path 360 | total_timesteps 3424.
Path 361 | total_timesteps 3434.
Path 362 | total_timesteps 3442.
Path 363 | total_timesteps 3450.
Path 364 | total_timesteps 3461.
Path 365 | total_timesteps 3471.
Path 366 | total_timesteps 3482.
Path 367 | total_timesteps 3491.
Path 368 | total_timesteps 3504.
Path 369 | total_timesteps 3519.
Path 370 | total_timesteps 3528.
Path 371 | total_timesteps 3537.
Path 372 | total_timesteps 3545.
Path 373 | total_timesteps 3554.
Path 374 | total_timesteps 3563.
Path 375 | total_timesteps 3576.
Path 376 | total_timesteps 3586.
Path 377 | total_timesteps 3594.
Path 378 | total_timesteps 3603.
Path 379 | total_timesteps 3618.
Path 380 | total_timesteps 3626.
Path 381 | total_timesteps 3635.
Path 382 | total_timesteps 3644.
Path 383 | total_timesteps 3653.
Path 384 | total_timesteps 3659.
Path 385 | total_timesteps 3668.
Path 386 | total_timesteps 3679.
Path 387 | total_timesteps 3691.
Path 388 | total_timesteps 3700.
Path 389 | total_timesteps 3707.
Path 390 | total_timesteps 3720.
Path 391 | total_timesteps 3727.
Path 392 | total_timesteps 3736.
Path 393 | total_timesteps 3746.
Path 394 | total_timesteps 3756.
Path 395 | total_timesteps 3764.
Path 396 | total_timesteps 3772.
Path 397 | total_timesteps 3783.
Path 398 | total_timesteps 3791.
Path 399 | total_timesteps 3804.
Path 400 | total_timesteps 3814.
Path 401 | total_timesteps 3826.
Path 402 | total_timesteps 3835.
Path 403 | total_timesteps 3849.
Path 404 | total_timesteps 3858.
Path 405 | total_timesteps 3867.
Path 406 | total_timesteps 3879.
Path 407 | total_timesteps 3893.
Path 408 | total_timesteps 3900.
Path 409 | total_timesteps 3913.
Path 410 | total_timesteps 3923.
Path 411 | total_timesteps 3930.
Path 412 | total_timesteps 3939.
Path 413 | total_timesteps 3952.
Path 414 | total_timesteps 3973.
Path 415 | total_timesteps 3986.
Path 416 | total_timesteps 3997.
Path 417 | total_timesteps 4006.
Path 418 | total_timesteps 4016.
Path 419 | total_timesteps 4026.
Path 420 | total_timesteps 4036.
Path 421 | total_timesteps 4046.
Path 422 | total_timesteps 4060.
Path 423 | total_timesteps 4068.
Path 424 | total_timesteps 4076.
Path 425 | total_timesteps 4086.
Path 426 | total_timesteps 4097.
Path 427 | total_timesteps 4112.
Path 428 | total_timesteps 4121.
Path 429 | total_timesteps 4131.
Path 430 | total_timesteps 4141.
Path 431 | total_timesteps 4150.
Path 432 | total_timesteps 4157.
Path 433 | total_timesteps 4167.
Path 434 | total_timesteps 4177.
Path 435 | total_timesteps 4185.
Path 436 | total_timesteps 4194.
Path 437 | total_timesteps 4202.
Path 438 | total_timesteps 4215.
Path 439 | total_timesteps 4224.
Path 440 | total_timesteps 4231.
Path 441 | total_timesteps 4238.
Path 442 | total_timesteps 4246.
Path 443 | total_timesteps 4253.
Path 444 | total_timesteps 4265.
Path 445 | total_timesteps 4278.
Path 446 | total_timesteps 4287.
Path 447 | total_timesteps 4295.
Path 448 | total_timesteps 4303.
Path 449 | total_timesteps 4310.
Path 450 | total_timesteps 4318.
Path 451 | total_timesteps 4331.
Path 452 | total_timesteps 4338.
Path 453 | total_timesteps 4351.
Path 454 | total_timesteps 4360.
Path 455 | total_timesteps 4371.
Path 456 | total_timesteps 4378.
Path 457 | total_timesteps 4384.
Path 458 | total_timesteps 4391.
Path 459 | total_timesteps 4398.
Path 460 | total_timesteps 4409.
Path 461 | total_timesteps 4417.
Path 462 | total_timesteps 4428.
Path 463 | total_timesteps 4438.
Path 464 | total_timesteps 4446.
Path 465 | total_timesteps 4453.
Path 466 | total_timesteps 4462.
Path 467 | total_timesteps 4473.
Path 468 | total_timesteps 4484.
Path 469 | total_timesteps 4492.
Path 470 | total_timesteps 4499.
Path 471 | total_timesteps 4509.
Path 472 | total_timesteps 4519.
Path 473 | total_timesteps 4532.
Path 474 | total_timesteps 4542.
Path 475 | total_timesteps 4553.
Path 476 | total_timesteps 4566.
Path 477 | total_timesteps 4575.
Path 478 | total_timesteps 4582.
Path 479 | total_timesteps 4592.
Path 480 | total_timesteps 4599.
Path 481 | total_timesteps 4608.
Path 482 | total_timesteps 4616.
Path 483 | total_timesteps 4623.
Path 484 | total_timesteps 4634.
Path 485 | total_timesteps 4642.
Path 486 | total_timesteps 4654.
Path 487 | total_timesteps 4665.
Path 488 | total_timesteps 4674.
Path 489 | total_timesteps 4690.
Path 490 | total_timesteps 4698.
Path 491 | total_timesteps 4713.
Path 492 | total_timesteps 4723.
Path 493 | total_timesteps 4730.
Path 494 | total_timesteps 4738.
Path 495 | total_timesteps 4748.
Path 496 | total_timesteps 4757.
Path 497 | total_timesteps 4770.
Path 498 | total_timesteps 4778.
Path 499 | total_timesteps 4788.
Path 500 | total_timesteps 4797.
Path 501 | total_timesteps 4806.
Path 502 | total_timesteps 4814.
Path 503 | total_timesteps 4821.
Path 504 | total_timesteps 4833.
Path 505 | total_timesteps 4841.
Path 506 | total_timesteps 4850.
Path 507 | total_timesteps 4860.
Path 508 | total_timesteps 4868.
Path 509 | total_timesteps 4876.
Path 510 | total_timesteps 4883.
Path 511 | total_timesteps 4893.
Path 512 | total_timesteps 4903.
Path 513 | total_timesteps 4914.
Path 514 | total_timesteps 4920.
Path 515 | total_timesteps 4929.
Path 516 | total_timesteps 4937.
Path 517 | total_timesteps 4945.
Path 518 | total_timesteps 4955.
Path 519 | total_timesteps 4967.
Path 520 | total_timesteps 4979.
Path 521 | total_timesteps 4987.
Path 522 | total_timesteps 4997.
Path 523 | total_timesteps 5006.
Path 524 | total_timesteps 5012.
Path 525 | total_timesteps 5022.
Path 526 | total_timesteps 5031.
Path 527 | total_timesteps 5052.
Path 528 | total_timesteps 5064.
Path 529 | total_timesteps 5074.
Path 530 | total_timesteps 5084.
Path 531 | total_timesteps 5093.
Path 532 | total_timesteps 5104.
Path 533 | total_timesteps 5119.
Path 534 | total_timesteps 5128.
Path 535 | total_timesteps 5139.
Path 536 | total_timesteps 5148.
Path 537 | total_timesteps 5157.
Path 538 | total_timesteps 5166.
Path 539 | total_timesteps 5175.
Path 540 | total_timesteps 5182.
Path 541 | total_timesteps 5188.
Path 542 | total_timesteps 5197.
Path 543 | total_timesteps 5205.
Path 544 | total_timesteps 5217.
Path 545 | total_timesteps 5225.
Path 546 | total_timesteps 5234.
Path 547 | total_timesteps 5244.
Path 548 | total_timesteps 5256.
Path 549 | total_timesteps 5270.
Path 550 | total_timesteps 5277.
Path 551 | total_timesteps 5289.
Path 552 | total_timesteps 5298.
Path 553 | total_timesteps 5307.
Path 554 | total_timesteps 5318.
Path 555 | total_timesteps 5332.
Path 556 | total_timesteps 5340.
Path 557 | total_timesteps 5349.
Path 558 | total_timesteps 5360.
Path 559 | total_timesteps 5372.
Path 560 | total_timesteps 5381.
Path 561 | total_timesteps 5393.
Path 562 | total_timesteps 5402.
Path 563 | total_timesteps 5411.
Path 564 | total_timesteps 5422.
Path 565 | total_timesteps 5433.
Path 566 | total_timesteps 5440.
Path 567 | total_timesteps 5448.
Path 568 | total_timesteps 5456.
Path 569 | total_timesteps 5466.
Path 570 | total_timesteps 5474.
Path 571 | total_timesteps 5481.
Path 572 | total_timesteps 5494.
Path 573 | total_timesteps 5503.
Path 574 | total_timesteps 5512.
Path 575 | total_timesteps 5524.
Path 576 | total_timesteps 5533.
Path 577 | total_timesteps 5543.
Path 578 | total_timesteps 5551.
Path 579 | total_timesteps 5564.
Path 580 | total_timesteps 5573.
Path 581 | total_timesteps 5580.
Path 582 | total_timesteps 5590.
Path 583 | total_timesteps 5597.
Path 584 | total_timesteps 5605.
Path 585 | total_timesteps 5615.
Path 586 | total_timesteps 5623.
Path 587 | total_timesteps 5635.
Path 588 | total_timesteps 5647.
Path 589 | total_timesteps 5657.
Path 590 | total_timesteps 5666.
Path 591 | total_timesteps 5676.
Path 592 | total_timesteps 5685.
Path 593 | total_timesteps 5694.
Path 594 | total_timesteps 5702.
Path 595 | total_timesteps 5715.
Path 596 | total_timesteps 5721.
Path 597 | total_timesteps 5729.
Path 598 | total_timesteps 5739.
Path 599 | total_timesteps 5749.
Path 600 | total_timesteps 5759.
Path 601 | total_timesteps 5768.
Path 602 | total_timesteps 5778.
Path 603 | total_timesteps 5790.
Path 604 | total_timesteps 5803.
Path 605 | total_timesteps 5810.
Path 606 | total_timesteps 5822.
Path 607 | total_timesteps 5831.
Path 608 | total_timesteps 5839.
Path 609 | total_timesteps 5848.
Path 610 | total_timesteps 5857.
Path 611 | total_timesteps 5871.
Path 612 | total_timesteps 5879.
Path 613 | total_timesteps 5887.
Path 614 | total_timesteps 5899.
Path 615 | total_timesteps 5906.
Path 616 | total_timesteps 5918.
Path 617 | total_timesteps 5927.
Path 618 | total_timesteps 5937.
Path 619 | total_timesteps 5944.
Path 620 | total_timesteps 5953.
Path 621 | total_timesteps 5973.
Path 622 | total_timesteps 5980.
Path 623 | total_timesteps 5991.
Path 624 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.15    |
| Iteration     | 25       |
| MaximumReturn | -1.3     |
| MinimumReturn | -20.1    |
| TotalSamples  | 108138   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.0021352535113692284
Validation loss = 0.002055700868368149
Validation loss = 0.002067377557978034
Validation loss = 0.002009022980928421
Validation loss = 0.0022342519368976355
Validation loss = 0.002015542471781373
Validation loss = 0.002384958788752556
Validation loss = 0.0019216708606109023
Validation loss = 0.0019774753600358963
Validation loss = 0.0020465035922825336
Validation loss = 0.0019376276759430766
Validation loss = 0.0022693644277751446
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 52.
Path 7 | total_timesteps 60.
Path 8 | total_timesteps 71.
Path 9 | total_timesteps 82.
Path 10 | total_timesteps 90.
Path 11 | total_timesteps 99.
Path 12 | total_timesteps 112.
Path 13 | total_timesteps 121.
Path 14 | total_timesteps 129.
Path 15 | total_timesteps 140.
Path 16 | total_timesteps 148.
Path 17 | total_timesteps 158.
Path 18 | total_timesteps 165.
Path 19 | total_timesteps 172.
Path 20 | total_timesteps 181.
Path 21 | total_timesteps 191.
Path 22 | total_timesteps 207.
Path 23 | total_timesteps 216.
Path 24 | total_timesteps 224.
Path 25 | total_timesteps 233.
Path 26 | total_timesteps 242.
Path 27 | total_timesteps 251.
Path 28 | total_timesteps 264.
Path 29 | total_timesteps 272.
Path 30 | total_timesteps 281.
Path 31 | total_timesteps 288.
Path 32 | total_timesteps 298.
Path 33 | total_timesteps 307.
Path 34 | total_timesteps 321.
Path 35 | total_timesteps 331.
Path 36 | total_timesteps 337.
Path 37 | total_timesteps 345.
Path 38 | total_timesteps 354.
Path 39 | total_timesteps 363.
Path 40 | total_timesteps 373.
Path 41 | total_timesteps 381.
Path 42 | total_timesteps 392.
Path 43 | total_timesteps 398.
Path 44 | total_timesteps 412.
Path 45 | total_timesteps 426.
Path 46 | total_timesteps 434.
Path 47 | total_timesteps 442.
Path 48 | total_timesteps 453.
Path 49 | total_timesteps 466.
Path 50 | total_timesteps 478.
Path 51 | total_timesteps 491.
Path 52 | total_timesteps 500.
Path 53 | total_timesteps 507.
Path 54 | total_timesteps 515.
Path 55 | total_timesteps 523.
Path 56 | total_timesteps 532.
Path 57 | total_timesteps 545.
Path 58 | total_timesteps 555.
Path 59 | total_timesteps 564.
Path 60 | total_timesteps 575.
Path 61 | total_timesteps 584.
Path 62 | total_timesteps 592.
Path 63 | total_timesteps 600.
Path 64 | total_timesteps 609.
Path 65 | total_timesteps 622.
Path 66 | total_timesteps 631.
Path 67 | total_timesteps 638.
Path 68 | total_timesteps 646.
Path 69 | total_timesteps 657.
Path 70 | total_timesteps 664.
Path 71 | total_timesteps 670.
Path 72 | total_timesteps 680.
Path 73 | total_timesteps 691.
Path 74 | total_timesteps 704.
Path 75 | total_timesteps 713.
Path 76 | total_timesteps 722.
Path 77 | total_timesteps 732.
Path 78 | total_timesteps 743.
Path 79 | total_timesteps 754.
Path 80 | total_timesteps 763.
Path 81 | total_timesteps 771.
Path 82 | total_timesteps 780.
Path 83 | total_timesteps 789.
Path 84 | total_timesteps 800.
Path 85 | total_timesteps 807.
Path 86 | total_timesteps 820.
Path 87 | total_timesteps 830.
Path 88 | total_timesteps 840.
Path 89 | total_timesteps 851.
Path 90 | total_timesteps 862.
Path 91 | total_timesteps 874.
Path 92 | total_timesteps 883.
Path 93 | total_timesteps 896.
Path 94 | total_timesteps 905.
Path 95 | total_timesteps 918.
Path 96 | total_timesteps 928.
Path 97 | total_timesteps 939.
Path 98 | total_timesteps 946.
Path 99 | total_timesteps 955.
Path 100 | total_timesteps 963.
Path 101 | total_timesteps 972.
Path 102 | total_timesteps 981.
Path 103 | total_timesteps 989.
Path 104 | total_timesteps 998.
Path 105 | total_timesteps 1009.
Path 106 | total_timesteps 1017.
Path 107 | total_timesteps 1025.
Path 108 | total_timesteps 1036.
Path 109 | total_timesteps 1047.
Path 110 | total_timesteps 1056.
Path 111 | total_timesteps 1064.
Path 112 | total_timesteps 1071.
Path 113 | total_timesteps 1083.
Path 114 | total_timesteps 1092.
Path 115 | total_timesteps 1100.
Path 116 | total_timesteps 1107.
Path 117 | total_timesteps 1113.
Path 118 | total_timesteps 1121.
Path 119 | total_timesteps 1129.
Path 120 | total_timesteps 1139.
Path 121 | total_timesteps 1155.
Path 122 | total_timesteps 1162.
Path 123 | total_timesteps 1172.
Path 124 | total_timesteps 1185.
Path 125 | total_timesteps 1192.
Path 126 | total_timesteps 1199.
Path 127 | total_timesteps 1207.
Path 128 | total_timesteps 1219.
Path 129 | total_timesteps 1227.
Path 130 | total_timesteps 1234.
Path 131 | total_timesteps 1247.
Path 132 | total_timesteps 1258.
Path 133 | total_timesteps 1272.
Path 134 | total_timesteps 1281.
Path 135 | total_timesteps 1288.
Path 136 | total_timesteps 1297.
Path 137 | total_timesteps 1310.
Path 138 | total_timesteps 1319.
Path 139 | total_timesteps 1331.
Path 140 | total_timesteps 1347.
Path 141 | total_timesteps 1354.
Path 142 | total_timesteps 1362.
Path 143 | total_timesteps 1371.
Path 144 | total_timesteps 1381.
Path 145 | total_timesteps 1393.
Path 146 | total_timesteps 1404.
Path 147 | total_timesteps 1413.
Path 148 | total_timesteps 1421.
Path 149 | total_timesteps 1429.
Path 150 | total_timesteps 1436.
Path 151 | total_timesteps 1453.
Path 152 | total_timesteps 1459.
Path 153 | total_timesteps 1467.
Path 154 | total_timesteps 1475.
Path 155 | total_timesteps 1484.
Path 156 | total_timesteps 1493.
Path 157 | total_timesteps 1501.
Path 158 | total_timesteps 1511.
Path 159 | total_timesteps 1523.
Path 160 | total_timesteps 1533.
Path 161 | total_timesteps 1541.
Path 162 | total_timesteps 1548.
Path 163 | total_timesteps 1557.
Path 164 | total_timesteps 1563.
Path 165 | total_timesteps 1571.
Path 166 | total_timesteps 1585.
Path 167 | total_timesteps 1594.
Path 168 | total_timesteps 1614.
Path 169 | total_timesteps 1621.
Path 170 | total_timesteps 1630.
Path 171 | total_timesteps 1638.
Path 172 | total_timesteps 1650.
Path 173 | total_timesteps 1658.
Path 174 | total_timesteps 1669.
Path 175 | total_timesteps 1680.
Path 176 | total_timesteps 1690.
Path 177 | total_timesteps 1699.
Path 178 | total_timesteps 1709.
Path 179 | total_timesteps 1718.
Path 180 | total_timesteps 1727.
Path 181 | total_timesteps 1737.
Path 182 | total_timesteps 1745.
Path 183 | total_timesteps 1754.
Path 184 | total_timesteps 1761.
Path 185 | total_timesteps 1775.
Path 186 | total_timesteps 1785.
Path 187 | total_timesteps 1800.
Path 188 | total_timesteps 1807.
Path 189 | total_timesteps 1820.
Path 190 | total_timesteps 1829.
Path 191 | total_timesteps 1837.
Path 192 | total_timesteps 1847.
Path 193 | total_timesteps 1857.
Path 194 | total_timesteps 1865.
Path 195 | total_timesteps 1873.
Path 196 | total_timesteps 1881.
Path 197 | total_timesteps 1889.
Path 198 | total_timesteps 1898.
Path 199 | total_timesteps 1910.
Path 200 | total_timesteps 1923.
Path 201 | total_timesteps 1932.
Path 202 | total_timesteps 1939.
Path 203 | total_timesteps 1951.
Path 204 | total_timesteps 1966.
Path 205 | total_timesteps 1974.
Path 206 | total_timesteps 1983.
Path 207 | total_timesteps 1989.
Path 208 | total_timesteps 1997.
Path 209 | total_timesteps 2006.
Path 210 | total_timesteps 2012.
Path 211 | total_timesteps 2024.
Path 212 | total_timesteps 2033.
Path 213 | total_timesteps 2042.
Path 214 | total_timesteps 2050.
Path 215 | total_timesteps 2057.
Path 216 | total_timesteps 2064.
Path 217 | total_timesteps 2075.
Path 218 | total_timesteps 2088.
Path 219 | total_timesteps 2095.
Path 220 | total_timesteps 2103.
Path 221 | total_timesteps 2120.
Path 222 | total_timesteps 2128.
Path 223 | total_timesteps 2143.
Path 224 | total_timesteps 2155.
Path 225 | total_timesteps 2165.
Path 226 | total_timesteps 2174.
Path 227 | total_timesteps 2182.
Path 228 | total_timesteps 2198.
Path 229 | total_timesteps 2206.
Path 230 | total_timesteps 2214.
Path 231 | total_timesteps 2225.
Path 232 | total_timesteps 2233.
Path 233 | total_timesteps 2241.
Path 234 | total_timesteps 2253.
Path 235 | total_timesteps 2261.
Path 236 | total_timesteps 2271.
Path 237 | total_timesteps 2283.
Path 238 | total_timesteps 2291.
Path 239 | total_timesteps 2299.
Path 240 | total_timesteps 2306.
Path 241 | total_timesteps 2314.
Path 242 | total_timesteps 2325.
Path 243 | total_timesteps 2335.
Path 244 | total_timesteps 2347.
Path 245 | total_timesteps 2359.
Path 246 | total_timesteps 2367.
Path 247 | total_timesteps 2374.
Path 248 | total_timesteps 2384.
Path 249 | total_timesteps 2393.
Path 250 | total_timesteps 2401.
Path 251 | total_timesteps 2408.
Path 252 | total_timesteps 2419.
Path 253 | total_timesteps 2435.
Path 254 | total_timesteps 2444.
Path 255 | total_timesteps 2456.
Path 256 | total_timesteps 2463.
Path 257 | total_timesteps 2473.
Path 258 | total_timesteps 2480.
Path 259 | total_timesteps 2490.
Path 260 | total_timesteps 2502.
Path 261 | total_timesteps 2509.
Path 262 | total_timesteps 2517.
Path 263 | total_timesteps 2526.
Path 264 | total_timesteps 2536.
Path 265 | total_timesteps 2542.
Path 266 | total_timesteps 2549.
Path 267 | total_timesteps 2559.
Path 268 | total_timesteps 2568.
Path 269 | total_timesteps 2575.
Path 270 | total_timesteps 2583.
Path 271 | total_timesteps 2592.
Path 272 | total_timesteps 2602.
Path 273 | total_timesteps 2611.
Path 274 | total_timesteps 2621.
Path 275 | total_timesteps 2629.
Path 276 | total_timesteps 2637.
Path 277 | total_timesteps 2649.
Path 278 | total_timesteps 2658.
Path 279 | total_timesteps 2671.
Path 280 | total_timesteps 2680.
Path 281 | total_timesteps 2694.
Path 282 | total_timesteps 2703.
Path 283 | total_timesteps 2711.
Path 284 | total_timesteps 2719.
Path 285 | total_timesteps 2727.
Path 286 | total_timesteps 2734.
Path 287 | total_timesteps 2744.
Path 288 | total_timesteps 2752.
Path 289 | total_timesteps 2760.
Path 290 | total_timesteps 2768.
Path 291 | total_timesteps 2779.
Path 292 | total_timesteps 2790.
Path 293 | total_timesteps 2797.
Path 294 | total_timesteps 2809.
Path 295 | total_timesteps 2820.
Path 296 | total_timesteps 2834.
Path 297 | total_timesteps 2845.
Path 298 | total_timesteps 2855.
Path 299 | total_timesteps 2863.
Path 300 | total_timesteps 2873.
Path 301 | total_timesteps 2883.
Path 302 | total_timesteps 2892.
Path 303 | total_timesteps 2903.
Path 304 | total_timesteps 2911.
Path 305 | total_timesteps 2923.
Path 306 | total_timesteps 2930.
Path 307 | total_timesteps 2942.
Path 308 | total_timesteps 2950.
Path 309 | total_timesteps 2961.
Path 310 | total_timesteps 2970.
Path 311 | total_timesteps 2980.
Path 312 | total_timesteps 2987.
Path 313 | total_timesteps 2994.
Path 314 | total_timesteps 3006.
Path 315 | total_timesteps 3013.
Path 316 | total_timesteps 3020.
Path 317 | total_timesteps 3028.
Path 318 | total_timesteps 3037.
Path 319 | total_timesteps 3045.
Path 320 | total_timesteps 3058.
Path 321 | total_timesteps 3065.
Path 322 | total_timesteps 3072.
Path 323 | total_timesteps 3081.
Path 324 | total_timesteps 3088.
Path 325 | total_timesteps 3099.
Path 326 | total_timesteps 3107.
Path 327 | total_timesteps 3114.
Path 328 | total_timesteps 3122.
Path 329 | total_timesteps 3132.
Path 330 | total_timesteps 3139.
Path 331 | total_timesteps 3147.
Path 332 | total_timesteps 3154.
Path 333 | total_timesteps 3169.
Path 334 | total_timesteps 3179.
Path 335 | total_timesteps 3188.
Path 336 | total_timesteps 3198.
Path 337 | total_timesteps 3207.
Path 338 | total_timesteps 3218.
Path 339 | total_timesteps 3226.
Path 340 | total_timesteps 3242.
Path 341 | total_timesteps 3250.
Path 342 | total_timesteps 3260.
Path 343 | total_timesteps 3268.
Path 344 | total_timesteps 3275.
Path 345 | total_timesteps 3284.
Path 346 | total_timesteps 3292.
Path 347 | total_timesteps 3300.
Path 348 | total_timesteps 3308.
Path 349 | total_timesteps 3321.
Path 350 | total_timesteps 3330.
Path 351 | total_timesteps 3336.
Path 352 | total_timesteps 3347.
Path 353 | total_timesteps 3355.
Path 354 | total_timesteps 3363.
Path 355 | total_timesteps 3371.
Path 356 | total_timesteps 3381.
Path 357 | total_timesteps 3388.
Path 358 | total_timesteps 3402.
Path 359 | total_timesteps 3409.
Path 360 | total_timesteps 3421.
Path 361 | total_timesteps 3430.
Path 362 | total_timesteps 3438.
Path 363 | total_timesteps 3446.
Path 364 | total_timesteps 3456.
Path 365 | total_timesteps 3465.
Path 366 | total_timesteps 3473.
Path 367 | total_timesteps 3482.
Path 368 | total_timesteps 3490.
Path 369 | total_timesteps 3497.
Path 370 | total_timesteps 3508.
Path 371 | total_timesteps 3521.
Path 372 | total_timesteps 3533.
Path 373 | total_timesteps 3543.
Path 374 | total_timesteps 3555.
Path 375 | total_timesteps 3563.
Path 376 | total_timesteps 3572.
Path 377 | total_timesteps 3579.
Path 378 | total_timesteps 3593.
Path 379 | total_timesteps 3600.
Path 380 | total_timesteps 3607.
Path 381 | total_timesteps 3617.
Path 382 | total_timesteps 3628.
Path 383 | total_timesteps 3639.
Path 384 | total_timesteps 3649.
Path 385 | total_timesteps 3663.
Path 386 | total_timesteps 3671.
Path 387 | total_timesteps 3680.
Path 388 | total_timesteps 3687.
Path 389 | total_timesteps 3696.
Path 390 | total_timesteps 3705.
Path 391 | total_timesteps 3717.
Path 392 | total_timesteps 3726.
Path 393 | total_timesteps 3735.
Path 394 | total_timesteps 3742.
Path 395 | total_timesteps 3749.
Path 396 | total_timesteps 3761.
Path 397 | total_timesteps 3768.
Path 398 | total_timesteps 3778.
Path 399 | total_timesteps 3788.
Path 400 | total_timesteps 3796.
Path 401 | total_timesteps 3806.
Path 402 | total_timesteps 3814.
Path 403 | total_timesteps 3824.
Path 404 | total_timesteps 3834.
Path 405 | total_timesteps 3842.
Path 406 | total_timesteps 3849.
Path 407 | total_timesteps 3856.
Path 408 | total_timesteps 3864.
Path 409 | total_timesteps 3873.
Path 410 | total_timesteps 3882.
Path 411 | total_timesteps 3892.
Path 412 | total_timesteps 3905.
Path 413 | total_timesteps 3912.
Path 414 | total_timesteps 3924.
Path 415 | total_timesteps 3934.
Path 416 | total_timesteps 3945.
Path 417 | total_timesteps 3952.
Path 418 | total_timesteps 3961.
Path 419 | total_timesteps 3969.
Path 420 | total_timesteps 3975.
Path 421 | total_timesteps 3991.
Path 422 | total_timesteps 4001.
Path 423 | total_timesteps 4010.
Path 424 | total_timesteps 4018.
Path 425 | total_timesteps 4030.
Path 426 | total_timesteps 4038.
Path 427 | total_timesteps 4049.
Path 428 | total_timesteps 4056.
Path 429 | total_timesteps 4067.
Path 430 | total_timesteps 4076.
Path 431 | total_timesteps 4085.
Path 432 | total_timesteps 4095.
Path 433 | total_timesteps 4107.
Path 434 | total_timesteps 4117.
Path 435 | total_timesteps 4128.
Path 436 | total_timesteps 4137.
Path 437 | total_timesteps 4144.
Path 438 | total_timesteps 4160.
Path 439 | total_timesteps 4168.
Path 440 | total_timesteps 4181.
Path 441 | total_timesteps 4188.
Path 442 | total_timesteps 4197.
Path 443 | total_timesteps 4205.
Path 444 | total_timesteps 4214.
Path 445 | total_timesteps 4222.
Path 446 | total_timesteps 4232.
Path 447 | total_timesteps 4241.
Path 448 | total_timesteps 4252.
Path 449 | total_timesteps 4261.
Path 450 | total_timesteps 4271.
Path 451 | total_timesteps 4280.
Path 452 | total_timesteps 4291.
Path 453 | total_timesteps 4304.
Path 454 | total_timesteps 4315.
Path 455 | total_timesteps 4325.
Path 456 | total_timesteps 4334.
Path 457 | total_timesteps 4344.
Path 458 | total_timesteps 4355.
Path 459 | total_timesteps 4365.
Path 460 | total_timesteps 4379.
Path 461 | total_timesteps 4391.
Path 462 | total_timesteps 4399.
Path 463 | total_timesteps 4411.
Path 464 | total_timesteps 4421.
Path 465 | total_timesteps 4429.
Path 466 | total_timesteps 4442.
Path 467 | total_timesteps 4450.
Path 468 | total_timesteps 4461.
Path 469 | total_timesteps 4470.
Path 470 | total_timesteps 4479.
Path 471 | total_timesteps 4486.
Path 472 | total_timesteps 4494.
Path 473 | total_timesteps 4502.
Path 474 | total_timesteps 4509.
Path 475 | total_timesteps 4518.
Path 476 | total_timesteps 4528.
Path 477 | total_timesteps 4539.
Path 478 | total_timesteps 4548.
Path 479 | total_timesteps 4557.
Path 480 | total_timesteps 4567.
Path 481 | total_timesteps 4577.
Path 482 | total_timesteps 4585.
Path 483 | total_timesteps 4594.
Path 484 | total_timesteps 4602.
Path 485 | total_timesteps 4610.
Path 486 | total_timesteps 4618.
Path 487 | total_timesteps 4627.
Path 488 | total_timesteps 4635.
Path 489 | total_timesteps 4645.
Path 490 | total_timesteps 4658.
Path 491 | total_timesteps 4667.
Path 492 | total_timesteps 4678.
Path 493 | total_timesteps 4687.
Path 494 | total_timesteps 4695.
Path 495 | total_timesteps 4706.
Path 496 | total_timesteps 4714.
Path 497 | total_timesteps 4722.
Path 498 | total_timesteps 4729.
Path 499 | total_timesteps 4739.
Path 500 | total_timesteps 4748.
Path 501 | total_timesteps 4757.
Path 502 | total_timesteps 4764.
Path 503 | total_timesteps 4772.
Path 504 | total_timesteps 4781.
Path 505 | total_timesteps 4789.
Path 506 | total_timesteps 4803.
Path 507 | total_timesteps 4813.
Path 508 | total_timesteps 4820.
Path 509 | total_timesteps 4829.
Path 510 | total_timesteps 4839.
Path 511 | total_timesteps 4846.
Path 512 | total_timesteps 4855.
Path 513 | total_timesteps 4862.
Path 514 | total_timesteps 4870.
Path 515 | total_timesteps 4878.
Path 516 | total_timesteps 4889.
Path 517 | total_timesteps 4897.
Path 518 | total_timesteps 4906.
Path 519 | total_timesteps 4917.
Path 520 | total_timesteps 4927.
Path 521 | total_timesteps 4937.
Path 522 | total_timesteps 4951.
Path 523 | total_timesteps 4959.
Path 524 | total_timesteps 4971.
Path 525 | total_timesteps 4980.
Path 526 | total_timesteps 4990.
Path 527 | total_timesteps 5002.
Path 528 | total_timesteps 5019.
Path 529 | total_timesteps 5028.
Path 530 | total_timesteps 5039.
Path 531 | total_timesteps 5049.
Path 532 | total_timesteps 5060.
Path 533 | total_timesteps 5069.
Path 534 | total_timesteps 5078.
Path 535 | total_timesteps 5086.
Path 536 | total_timesteps 5098.
Path 537 | total_timesteps 5106.
Path 538 | total_timesteps 5117.
Path 539 | total_timesteps 5126.
Path 540 | total_timesteps 5135.
Path 541 | total_timesteps 5144.
Path 542 | total_timesteps 5153.
Path 543 | total_timesteps 5166.
Path 544 | total_timesteps 5179.
Path 545 | total_timesteps 5189.
Path 546 | total_timesteps 5198.
Path 547 | total_timesteps 5206.
Path 548 | total_timesteps 5214.
Path 549 | total_timesteps 5231.
Path 550 | total_timesteps 5238.
Path 551 | total_timesteps 5245.
Path 552 | total_timesteps 5253.
Path 553 | total_timesteps 5263.
Path 554 | total_timesteps 5277.
Path 555 | total_timesteps 5285.
Path 556 | total_timesteps 5294.
Path 557 | total_timesteps 5303.
Path 558 | total_timesteps 5312.
Path 559 | total_timesteps 5321.
Path 560 | total_timesteps 5331.
Path 561 | total_timesteps 5339.
Path 562 | total_timesteps 5349.
Path 563 | total_timesteps 5358.
Path 564 | total_timesteps 5369.
Path 565 | total_timesteps 5378.
Path 566 | total_timesteps 5390.
Path 567 | total_timesteps 5400.
Path 568 | total_timesteps 5410.
Path 569 | total_timesteps 5420.
Path 570 | total_timesteps 5431.
Path 571 | total_timesteps 5439.
Path 572 | total_timesteps 5448.
Path 573 | total_timesteps 5458.
Path 574 | total_timesteps 5466.
Path 575 | total_timesteps 5475.
Path 576 | total_timesteps 5483.
Path 577 | total_timesteps 5498.
Path 578 | total_timesteps 5506.
Path 579 | total_timesteps 5516.
Path 580 | total_timesteps 5526.
Path 581 | total_timesteps 5534.
Path 582 | total_timesteps 5549.
Path 583 | total_timesteps 5558.
Path 584 | total_timesteps 5564.
Path 585 | total_timesteps 5574.
Path 586 | total_timesteps 5586.
Path 587 | total_timesteps 5597.
Path 588 | total_timesteps 5606.
Path 589 | total_timesteps 5614.
Path 590 | total_timesteps 5626.
Path 591 | total_timesteps 5635.
Path 592 | total_timesteps 5642.
Path 593 | total_timesteps 5654.
Path 594 | total_timesteps 5660.
Path 595 | total_timesteps 5668.
Path 596 | total_timesteps 5677.
Path 597 | total_timesteps 5689.
Path 598 | total_timesteps 5699.
Path 599 | total_timesteps 5707.
Path 600 | total_timesteps 5716.
Path 601 | total_timesteps 5724.
Path 602 | total_timesteps 5733.
Path 603 | total_timesteps 5745.
Path 604 | total_timesteps 5754.
Path 605 | total_timesteps 5762.
Path 606 | total_timesteps 5769.
Path 607 | total_timesteps 5782.
Path 608 | total_timesteps 5791.
Path 609 | total_timesteps 5798.
Path 610 | total_timesteps 5808.
Path 611 | total_timesteps 5816.
Path 612 | total_timesteps 5825.
Path 613 | total_timesteps 5834.
Path 614 | total_timesteps 5846.
Path 615 | total_timesteps 5855.
Path 616 | total_timesteps 5866.
Path 617 | total_timesteps 5874.
Path 618 | total_timesteps 5882.
Path 619 | total_timesteps 5890.
Path 620 | total_timesteps 5900.
Path 621 | total_timesteps 5909.
Path 622 | total_timesteps 5918.
Path 623 | total_timesteps 5927.
Path 624 | total_timesteps 5935.
Path 625 | total_timesteps 5945.
Path 626 | total_timesteps 5952.
Path 627 | total_timesteps 5961.
Path 628 | total_timesteps 5970.
Path 629 | total_timesteps 5985.
Path 630 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.9     |
| Iteration     | 26       |
| MaximumReturn | 2.08     |
| MinimumReturn | -15.5    |
| TotalSamples  | 112138   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.0020032699685543776
Validation loss = 0.002220733556896448
Validation loss = 0.0019662007689476013
Validation loss = 0.0019852272234857082
Validation loss = 0.002040126593783498
Validation loss = 0.002195813227444887
Validation loss = 0.002061349106952548
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 69.
Path 8 | total_timesteps 78.
Path 9 | total_timesteps 93.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 116.
Path 12 | total_timesteps 124.
Path 13 | total_timesteps 133.
Path 14 | total_timesteps 141.
Path 15 | total_timesteps 149.
Path 16 | total_timesteps 158.
Path 17 | total_timesteps 172.
Path 18 | total_timesteps 182.
Path 19 | total_timesteps 192.
Path 20 | total_timesteps 200.
Path 21 | total_timesteps 209.
Path 22 | total_timesteps 215.
Path 23 | total_timesteps 224.
Path 24 | total_timesteps 235.
Path 25 | total_timesteps 249.
Path 26 | total_timesteps 258.
Path 27 | total_timesteps 267.
Path 28 | total_timesteps 276.
Path 29 | total_timesteps 289.
Path 30 | total_timesteps 301.
Path 31 | total_timesteps 308.
Path 32 | total_timesteps 314.
Path 33 | total_timesteps 324.
Path 34 | total_timesteps 331.
Path 35 | total_timesteps 340.
Path 36 | total_timesteps 353.
Path 37 | total_timesteps 362.
Path 38 | total_timesteps 369.
Path 39 | total_timesteps 375.
Path 40 | total_timesteps 383.
Path 41 | total_timesteps 391.
Path 42 | total_timesteps 399.
Path 43 | total_timesteps 407.
Path 44 | total_timesteps 414.
Path 45 | total_timesteps 424.
Path 46 | total_timesteps 433.
Path 47 | total_timesteps 443.
Path 48 | total_timesteps 454.
Path 49 | total_timesteps 465.
Path 50 | total_timesteps 476.
Path 51 | total_timesteps 488.
Path 52 | total_timesteps 497.
Path 53 | total_timesteps 504.
Path 54 | total_timesteps 516.
Path 55 | total_timesteps 525.
Path 56 | total_timesteps 531.
Path 57 | total_timesteps 541.
Path 58 | total_timesteps 551.
Path 59 | total_timesteps 559.
Path 60 | total_timesteps 567.
Path 61 | total_timesteps 576.
Path 62 | total_timesteps 584.
Path 63 | total_timesteps 592.
Path 64 | total_timesteps 600.
Path 65 | total_timesteps 610.
Path 66 | total_timesteps 620.
Path 67 | total_timesteps 633.
Path 68 | total_timesteps 642.
Path 69 | total_timesteps 651.
Path 70 | total_timesteps 659.
Path 71 | total_timesteps 668.
Path 72 | total_timesteps 678.
Path 73 | total_timesteps 686.
Path 74 | total_timesteps 693.
Path 75 | total_timesteps 701.
Path 76 | total_timesteps 711.
Path 77 | total_timesteps 718.
Path 78 | total_timesteps 728.
Path 79 | total_timesteps 737.
Path 80 | total_timesteps 744.
Path 81 | total_timesteps 755.
Path 82 | total_timesteps 762.
Path 83 | total_timesteps 770.
Path 84 | total_timesteps 778.
Path 85 | total_timesteps 785.
Path 86 | total_timesteps 792.
Path 87 | total_timesteps 802.
Path 88 | total_timesteps 808.
Path 89 | total_timesteps 815.
Path 90 | total_timesteps 824.
Path 91 | total_timesteps 833.
Path 92 | total_timesteps 840.
Path 93 | total_timesteps 849.
Path 94 | total_timesteps 858.
Path 95 | total_timesteps 869.
Path 96 | total_timesteps 877.
Path 97 | total_timesteps 885.
Path 98 | total_timesteps 894.
Path 99 | total_timesteps 907.
Path 100 | total_timesteps 919.
Path 101 | total_timesteps 927.
Path 102 | total_timesteps 936.
Path 103 | total_timesteps 946.
Path 104 | total_timesteps 953.
Path 105 | total_timesteps 963.
Path 106 | total_timesteps 971.
Path 107 | total_timesteps 977.
Path 108 | total_timesteps 985.
Path 109 | total_timesteps 998.
Path 110 | total_timesteps 1005.
Path 111 | total_timesteps 1015.
Path 112 | total_timesteps 1025.
Path 113 | total_timesteps 1034.
Path 114 | total_timesteps 1043.
Path 115 | total_timesteps 1050.
Path 116 | total_timesteps 1057.
Path 117 | total_timesteps 1065.
Path 118 | total_timesteps 1073.
Path 119 | total_timesteps 1080.
Path 120 | total_timesteps 1089.
Path 121 | total_timesteps 1098.
Path 122 | total_timesteps 1105.
Path 123 | total_timesteps 1114.
Path 124 | total_timesteps 1124.
Path 125 | total_timesteps 1132.
Path 126 | total_timesteps 1151.
Path 127 | total_timesteps 1162.
Path 128 | total_timesteps 1174.
Path 129 | total_timesteps 1181.
Path 130 | total_timesteps 1188.
Path 131 | total_timesteps 1198.
Path 132 | total_timesteps 1208.
Path 133 | total_timesteps 1216.
Path 134 | total_timesteps 1225.
Path 135 | total_timesteps 1236.
Path 136 | total_timesteps 1245.
Path 137 | total_timesteps 1253.
Path 138 | total_timesteps 1261.
Path 139 | total_timesteps 1269.
Path 140 | total_timesteps 1278.
Path 141 | total_timesteps 1288.
Path 142 | total_timesteps 1295.
Path 143 | total_timesteps 1315.
Path 144 | total_timesteps 1324.
Path 145 | total_timesteps 1333.
Path 146 | total_timesteps 1343.
Path 147 | total_timesteps 1354.
Path 148 | total_timesteps 1366.
Path 149 | total_timesteps 1374.
Path 150 | total_timesteps 1384.
Path 151 | total_timesteps 1396.
Path 152 | total_timesteps 1405.
Path 153 | total_timesteps 1415.
Path 154 | total_timesteps 1424.
Path 155 | total_timesteps 1432.
Path 156 | total_timesteps 1441.
Path 157 | total_timesteps 1451.
Path 158 | total_timesteps 1459.
Path 159 | total_timesteps 1469.
Path 160 | total_timesteps 1475.
Path 161 | total_timesteps 1484.
Path 162 | total_timesteps 1491.
Path 163 | total_timesteps 1500.
Path 164 | total_timesteps 1508.
Path 165 | total_timesteps 1516.
Path 166 | total_timesteps 1523.
Path 167 | total_timesteps 1530.
Path 168 | total_timesteps 1540.
Path 169 | total_timesteps 1550.
Path 170 | total_timesteps 1557.
Path 171 | total_timesteps 1566.
Path 172 | total_timesteps 1574.
Path 173 | total_timesteps 1581.
Path 174 | total_timesteps 1594.
Path 175 | total_timesteps 1606.
Path 176 | total_timesteps 1617.
Path 177 | total_timesteps 1628.
Path 178 | total_timesteps 1636.
Path 179 | total_timesteps 1644.
Path 180 | total_timesteps 1654.
Path 181 | total_timesteps 1663.
Path 182 | total_timesteps 1671.
Path 183 | total_timesteps 1678.
Path 184 | total_timesteps 1688.
Path 185 | total_timesteps 1698.
Path 186 | total_timesteps 1712.
Path 187 | total_timesteps 1724.
Path 188 | total_timesteps 1733.
Path 189 | total_timesteps 1745.
Path 190 | total_timesteps 1753.
Path 191 | total_timesteps 1766.
Path 192 | total_timesteps 1776.
Path 193 | total_timesteps 1785.
Path 194 | total_timesteps 1794.
Path 195 | total_timesteps 1804.
Path 196 | total_timesteps 1813.
Path 197 | total_timesteps 1825.
Path 198 | total_timesteps 1832.
Path 199 | total_timesteps 1840.
Path 200 | total_timesteps 1851.
Path 201 | total_timesteps 1860.
Path 202 | total_timesteps 1871.
Path 203 | total_timesteps 1885.
Path 204 | total_timesteps 1893.
Path 205 | total_timesteps 1902.
Path 206 | total_timesteps 1912.
Path 207 | total_timesteps 1923.
Path 208 | total_timesteps 1930.
Path 209 | total_timesteps 1938.
Path 210 | total_timesteps 1946.
Path 211 | total_timesteps 1953.
Path 212 | total_timesteps 1962.
Path 213 | total_timesteps 1974.
Path 214 | total_timesteps 1982.
Path 215 | total_timesteps 1988.
Path 216 | total_timesteps 1998.
Path 217 | total_timesteps 2008.
Path 218 | total_timesteps 2014.
Path 219 | total_timesteps 2022.
Path 220 | total_timesteps 2031.
Path 221 | total_timesteps 2040.
Path 222 | total_timesteps 2051.
Path 223 | total_timesteps 2064.
Path 224 | total_timesteps 2076.
Path 225 | total_timesteps 2085.
Path 226 | total_timesteps 2093.
Path 227 | total_timesteps 2102.
Path 228 | total_timesteps 2109.
Path 229 | total_timesteps 2117.
Path 230 | total_timesteps 2128.
Path 231 | total_timesteps 2136.
Path 232 | total_timesteps 2143.
Path 233 | total_timesteps 2152.
Path 234 | total_timesteps 2161.
Path 235 | total_timesteps 2173.
Path 236 | total_timesteps 2184.
Path 237 | total_timesteps 2196.
Path 238 | total_timesteps 2204.
Path 239 | total_timesteps 2216.
Path 240 | total_timesteps 2226.
Path 241 | total_timesteps 2235.
Path 242 | total_timesteps 2243.
Path 243 | total_timesteps 2254.
Path 244 | total_timesteps 2261.
Path 245 | total_timesteps 2269.
Path 246 | total_timesteps 2278.
Path 247 | total_timesteps 2288.
Path 248 | total_timesteps 2301.
Path 249 | total_timesteps 2311.
Path 250 | total_timesteps 2320.
Path 251 | total_timesteps 2327.
Path 252 | total_timesteps 2336.
Path 253 | total_timesteps 2349.
Path 254 | total_timesteps 2359.
Path 255 | total_timesteps 2368.
Path 256 | total_timesteps 2376.
Path 257 | total_timesteps 2387.
Path 258 | total_timesteps 2399.
Path 259 | total_timesteps 2408.
Path 260 | total_timesteps 2417.
Path 261 | total_timesteps 2425.
Path 262 | total_timesteps 2433.
Path 263 | total_timesteps 2442.
Path 264 | total_timesteps 2453.
Path 265 | total_timesteps 2461.
Path 266 | total_timesteps 2471.
Path 267 | total_timesteps 2479.
Path 268 | total_timesteps 2488.
Path 269 | total_timesteps 2496.
Path 270 | total_timesteps 2506.
Path 271 | total_timesteps 2515.
Path 272 | total_timesteps 2527.
Path 273 | total_timesteps 2540.
Path 274 | total_timesteps 2547.
Path 275 | total_timesteps 2558.
Path 276 | total_timesteps 2567.
Path 277 | total_timesteps 2578.
Path 278 | total_timesteps 2587.
Path 279 | total_timesteps 2597.
Path 280 | total_timesteps 2609.
Path 281 | total_timesteps 2620.
Path 282 | total_timesteps 2630.
Path 283 | total_timesteps 2641.
Path 284 | total_timesteps 2650.
Path 285 | total_timesteps 2660.
Path 286 | total_timesteps 2667.
Path 287 | total_timesteps 2676.
Path 288 | total_timesteps 2684.
Path 289 | total_timesteps 2695.
Path 290 | total_timesteps 2706.
Path 291 | total_timesteps 2714.
Path 292 | total_timesteps 2726.
Path 293 | total_timesteps 2735.
Path 294 | total_timesteps 2743.
Path 295 | total_timesteps 2754.
Path 296 | total_timesteps 2764.
Path 297 | total_timesteps 2772.
Path 298 | total_timesteps 2784.
Path 299 | total_timesteps 2795.
Path 300 | total_timesteps 2804.
Path 301 | total_timesteps 2817.
Path 302 | total_timesteps 2828.
Path 303 | total_timesteps 2839.
Path 304 | total_timesteps 2853.
Path 305 | total_timesteps 2860.
Path 306 | total_timesteps 2869.
Path 307 | total_timesteps 2876.
Path 308 | total_timesteps 2886.
Path 309 | total_timesteps 2896.
Path 310 | total_timesteps 2903.
Path 311 | total_timesteps 2911.
Path 312 | total_timesteps 2920.
Path 313 | total_timesteps 2927.
Path 314 | total_timesteps 2937.
Path 315 | total_timesteps 2945.
Path 316 | total_timesteps 2955.
Path 317 | total_timesteps 2963.
Path 318 | total_timesteps 2971.
Path 319 | total_timesteps 2978.
Path 320 | total_timesteps 2987.
Path 321 | total_timesteps 2995.
Path 322 | total_timesteps 3006.
Path 323 | total_timesteps 3013.
Path 324 | total_timesteps 3022.
Path 325 | total_timesteps 3030.
Path 326 | total_timesteps 3041.
Path 327 | total_timesteps 3053.
Path 328 | total_timesteps 3062.
Path 329 | total_timesteps 3071.
Path 330 | total_timesteps 3082.
Path 331 | total_timesteps 3089.
Path 332 | total_timesteps 3097.
Path 333 | total_timesteps 3110.
Path 334 | total_timesteps 3125.
Path 335 | total_timesteps 3136.
Path 336 | total_timesteps 3150.
Path 337 | total_timesteps 3163.
Path 338 | total_timesteps 3174.
Path 339 | total_timesteps 3187.
Path 340 | total_timesteps 3199.
Path 341 | total_timesteps 3206.
Path 342 | total_timesteps 3219.
Path 343 | total_timesteps 3225.
Path 344 | total_timesteps 3232.
Path 345 | total_timesteps 3240.
Path 346 | total_timesteps 3249.
Path 347 | total_timesteps 3257.
Path 348 | total_timesteps 3264.
Path 349 | total_timesteps 3274.
Path 350 | total_timesteps 3281.
Path 351 | total_timesteps 3290.
Path 352 | total_timesteps 3296.
Path 353 | total_timesteps 3305.
Path 354 | total_timesteps 3314.
Path 355 | total_timesteps 3327.
Path 356 | total_timesteps 3336.
Path 357 | total_timesteps 3348.
Path 358 | total_timesteps 3356.
Path 359 | total_timesteps 3366.
Path 360 | total_timesteps 3373.
Path 361 | total_timesteps 3381.
Path 362 | total_timesteps 3391.
Path 363 | total_timesteps 3400.
Path 364 | total_timesteps 3408.
Path 365 | total_timesteps 3418.
Path 366 | total_timesteps 3427.
Path 367 | total_timesteps 3435.
Path 368 | total_timesteps 3443.
Path 369 | total_timesteps 3451.
Path 370 | total_timesteps 3463.
Path 371 | total_timesteps 3473.
Path 372 | total_timesteps 3483.
Path 373 | total_timesteps 3492.
Path 374 | total_timesteps 3503.
Path 375 | total_timesteps 3511.
Path 376 | total_timesteps 3520.
Path 377 | total_timesteps 3535.
Path 378 | total_timesteps 3542.
Path 379 | total_timesteps 3552.
Path 380 | total_timesteps 3560.
Path 381 | total_timesteps 3568.
Path 382 | total_timesteps 3580.
Path 383 | total_timesteps 3586.
Path 384 | total_timesteps 3593.
Path 385 | total_timesteps 3602.
Path 386 | total_timesteps 3611.
Path 387 | total_timesteps 3619.
Path 388 | total_timesteps 3627.
Path 389 | total_timesteps 3637.
Path 390 | total_timesteps 3649.
Path 391 | total_timesteps 3659.
Path 392 | total_timesteps 3668.
Path 393 | total_timesteps 3680.
Path 394 | total_timesteps 3691.
Path 395 | total_timesteps 3698.
Path 396 | total_timesteps 3706.
Path 397 | total_timesteps 3716.
Path 398 | total_timesteps 3723.
Path 399 | total_timesteps 3731.
Path 400 | total_timesteps 3737.
Path 401 | total_timesteps 3751.
Path 402 | total_timesteps 3758.
Path 403 | total_timesteps 3770.
Path 404 | total_timesteps 3787.
Path 405 | total_timesteps 3797.
Path 406 | total_timesteps 3810.
Path 407 | total_timesteps 3820.
Path 408 | total_timesteps 3829.
Path 409 | total_timesteps 3838.
Path 410 | total_timesteps 3846.
Path 411 | total_timesteps 3852.
Path 412 | total_timesteps 3862.
Path 413 | total_timesteps 3870.
Path 414 | total_timesteps 3880.
Path 415 | total_timesteps 3887.
Path 416 | total_timesteps 3897.
Path 417 | total_timesteps 3904.
Path 418 | total_timesteps 3913.
Path 419 | total_timesteps 3923.
Path 420 | total_timesteps 3937.
Path 421 | total_timesteps 3945.
Path 422 | total_timesteps 3955.
Path 423 | total_timesteps 3963.
Path 424 | total_timesteps 3973.
Path 425 | total_timesteps 3982.
Path 426 | total_timesteps 3990.
Path 427 | total_timesteps 4000.
Path 428 | total_timesteps 4011.
Path 429 | total_timesteps 4019.
Path 430 | total_timesteps 4027.
Path 431 | total_timesteps 4037.
Path 432 | total_timesteps 4045.
Path 433 | total_timesteps 4056.
Path 434 | total_timesteps 4064.
Path 435 | total_timesteps 4073.
Path 436 | total_timesteps 4081.
Path 437 | total_timesteps 4090.
Path 438 | total_timesteps 4098.
Path 439 | total_timesteps 4108.
Path 440 | total_timesteps 4119.
Path 441 | total_timesteps 4127.
Path 442 | total_timesteps 4142.
Path 443 | total_timesteps 4150.
Path 444 | total_timesteps 4160.
Path 445 | total_timesteps 4168.
Path 446 | total_timesteps 4180.
Path 447 | total_timesteps 4190.
Path 448 | total_timesteps 4197.
Path 449 | total_timesteps 4205.
Path 450 | total_timesteps 4213.
Path 451 | total_timesteps 4222.
Path 452 | total_timesteps 4229.
Path 453 | total_timesteps 4238.
Path 454 | total_timesteps 4246.
Path 455 | total_timesteps 4255.
Path 456 | total_timesteps 4264.
Path 457 | total_timesteps 4273.
Path 458 | total_timesteps 4282.
Path 459 | total_timesteps 4291.
Path 460 | total_timesteps 4299.
Path 461 | total_timesteps 4308.
Path 462 | total_timesteps 4315.
Path 463 | total_timesteps 4322.
Path 464 | total_timesteps 4329.
Path 465 | total_timesteps 4342.
Path 466 | total_timesteps 4350.
Path 467 | total_timesteps 4359.
Path 468 | total_timesteps 4370.
Path 469 | total_timesteps 4378.
Path 470 | total_timesteps 4389.
Path 471 | total_timesteps 4397.
Path 472 | total_timesteps 4405.
Path 473 | total_timesteps 4418.
Path 474 | total_timesteps 4425.
Path 475 | total_timesteps 4441.
Path 476 | total_timesteps 4450.
Path 477 | total_timesteps 4457.
Path 478 | total_timesteps 4465.
Path 479 | total_timesteps 4473.
Path 480 | total_timesteps 4480.
Path 481 | total_timesteps 4492.
Path 482 | total_timesteps 4499.
Path 483 | total_timesteps 4509.
Path 484 | total_timesteps 4519.
Path 485 | total_timesteps 4527.
Path 486 | total_timesteps 4534.
Path 487 | total_timesteps 4542.
Path 488 | total_timesteps 4552.
Path 489 | total_timesteps 4562.
Path 490 | total_timesteps 4571.
Path 491 | total_timesteps 4580.
Path 492 | total_timesteps 4587.
Path 493 | total_timesteps 4595.
Path 494 | total_timesteps 4610.
Path 495 | total_timesteps 4619.
Path 496 | total_timesteps 4628.
Path 497 | total_timesteps 4639.
Path 498 | total_timesteps 4649.
Path 499 | total_timesteps 4661.
Path 500 | total_timesteps 4669.
Path 501 | total_timesteps 4678.
Path 502 | total_timesteps 4686.
Path 503 | total_timesteps 4698.
Path 504 | total_timesteps 4707.
Path 505 | total_timesteps 4715.
Path 506 | total_timesteps 4723.
Path 507 | total_timesteps 4731.
Path 508 | total_timesteps 4743.
Path 509 | total_timesteps 4754.
Path 510 | total_timesteps 4763.
Path 511 | total_timesteps 4770.
Path 512 | total_timesteps 4780.
Path 513 | total_timesteps 4789.
Path 514 | total_timesteps 4798.
Path 515 | total_timesteps 4809.
Path 516 | total_timesteps 4820.
Path 517 | total_timesteps 4827.
Path 518 | total_timesteps 4838.
Path 519 | total_timesteps 4847.
Path 520 | total_timesteps 4857.
Path 521 | total_timesteps 4873.
Path 522 | total_timesteps 4885.
Path 523 | total_timesteps 4893.
Path 524 | total_timesteps 4904.
Path 525 | total_timesteps 4911.
Path 526 | total_timesteps 4921.
Path 527 | total_timesteps 4931.
Path 528 | total_timesteps 4941.
Path 529 | total_timesteps 4950.
Path 530 | total_timesteps 4962.
Path 531 | total_timesteps 4969.
Path 532 | total_timesteps 4977.
Path 533 | total_timesteps 4984.
Path 534 | total_timesteps 4994.
Path 535 | total_timesteps 5002.
Path 536 | total_timesteps 5010.
Path 537 | total_timesteps 5017.
Path 538 | total_timesteps 5024.
Path 539 | total_timesteps 5032.
Path 540 | total_timesteps 5040.
Path 541 | total_timesteps 5048.
Path 542 | total_timesteps 5056.
Path 543 | total_timesteps 5065.
Path 544 | total_timesteps 5077.
Path 545 | total_timesteps 5086.
Path 546 | total_timesteps 5095.
Path 547 | total_timesteps 5104.
Path 548 | total_timesteps 5110.
Path 549 | total_timesteps 5118.
Path 550 | total_timesteps 5126.
Path 551 | total_timesteps 5133.
Path 552 | total_timesteps 5142.
Path 553 | total_timesteps 5152.
Path 554 | total_timesteps 5160.
Path 555 | total_timesteps 5169.
Path 556 | total_timesteps 5178.
Path 557 | total_timesteps 5190.
Path 558 | total_timesteps 5198.
Path 559 | total_timesteps 5207.
Path 560 | total_timesteps 5215.
Path 561 | total_timesteps 5225.
Path 562 | total_timesteps 5236.
Path 563 | total_timesteps 5249.
Path 564 | total_timesteps 5257.
Path 565 | total_timesteps 5265.
Path 566 | total_timesteps 5275.
Path 567 | total_timesteps 5284.
Path 568 | total_timesteps 5295.
Path 569 | total_timesteps 5305.
Path 570 | total_timesteps 5314.
Path 571 | total_timesteps 5324.
Path 572 | total_timesteps 5336.
Path 573 | total_timesteps 5347.
Path 574 | total_timesteps 5356.
Path 575 | total_timesteps 5369.
Path 576 | total_timesteps 5379.
Path 577 | total_timesteps 5390.
Path 578 | total_timesteps 5402.
Path 579 | total_timesteps 5410.
Path 580 | total_timesteps 5417.
Path 581 | total_timesteps 5424.
Path 582 | total_timesteps 5433.
Path 583 | total_timesteps 5441.
Path 584 | total_timesteps 5455.
Path 585 | total_timesteps 5462.
Path 586 | total_timesteps 5468.
Path 587 | total_timesteps 5486.
Path 588 | total_timesteps 5494.
Path 589 | total_timesteps 5505.
Path 590 | total_timesteps 5512.
Path 591 | total_timesteps 5520.
Path 592 | total_timesteps 5529.
Path 593 | total_timesteps 5538.
Path 594 | total_timesteps 5546.
Path 595 | total_timesteps 5555.
Path 596 | total_timesteps 5562.
Path 597 | total_timesteps 5571.
Path 598 | total_timesteps 5582.
Path 599 | total_timesteps 5591.
Path 600 | total_timesteps 5600.
Path 601 | total_timesteps 5610.
Path 602 | total_timesteps 5618.
Path 603 | total_timesteps 5625.
Path 604 | total_timesteps 5634.
Path 605 | total_timesteps 5645.
Path 606 | total_timesteps 5653.
Path 607 | total_timesteps 5660.
Path 608 | total_timesteps 5669.
Path 609 | total_timesteps 5682.
Path 610 | total_timesteps 5689.
Path 611 | total_timesteps 5698.
Path 612 | total_timesteps 5706.
Path 613 | total_timesteps 5715.
Path 614 | total_timesteps 5724.
Path 615 | total_timesteps 5733.
Path 616 | total_timesteps 5740.
Path 617 | total_timesteps 5749.
Path 618 | total_timesteps 5758.
Path 619 | total_timesteps 5768.
Path 620 | total_timesteps 5775.
Path 621 | total_timesteps 5782.
Path 622 | total_timesteps 5789.
Path 623 | total_timesteps 5797.
Path 624 | total_timesteps 5806.
Path 625 | total_timesteps 5816.
Path 626 | total_timesteps 5825.
Path 627 | total_timesteps 5835.
Path 628 | total_timesteps 5845.
Path 629 | total_timesteps 5856.
Path 630 | total_timesteps 5867.
Path 631 | total_timesteps 5877.
Path 632 | total_timesteps 5889.
Path 633 | total_timesteps 5897.
Path 634 | total_timesteps 5905.
Path 635 | total_timesteps 5911.
Path 636 | total_timesteps 5922.
Path 637 | total_timesteps 5929.
Path 638 | total_timesteps 5937.
Path 639 | total_timesteps 5948.
Path 640 | total_timesteps 5962.
Path 641 | total_timesteps 5971.
Path 642 | total_timesteps 5980.
Path 643 | total_timesteps 5987.
Path 644 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.97    |
| Iteration     | 27       |
| MaximumReturn | -1.29    |
| MinimumReturn | -15.2    |
| TotalSamples  | 116141   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.0019081380451098084
Validation loss = 0.002003739820793271
Validation loss = 0.0020101929549127817
Validation loss = 0.001973153091967106
Validation loss = 0.0020110744517296553
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 41.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 63.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 80.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 107.
Path 12 | total_timesteps 117.
Path 13 | total_timesteps 130.
Path 14 | total_timesteps 139.
Path 15 | total_timesteps 148.
Path 16 | total_timesteps 156.
Path 17 | total_timesteps 165.
Path 18 | total_timesteps 172.
Path 19 | total_timesteps 180.
Path 20 | total_timesteps 188.
Path 21 | total_timesteps 197.
Path 22 | total_timesteps 211.
Path 23 | total_timesteps 219.
Path 24 | total_timesteps 228.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 247.
Path 27 | total_timesteps 255.
Path 28 | total_timesteps 263.
Path 29 | total_timesteps 270.
Path 30 | total_timesteps 280.
Path 31 | total_timesteps 289.
Path 32 | total_timesteps 298.
Path 33 | total_timesteps 306.
Path 34 | total_timesteps 315.
Path 35 | total_timesteps 326.
Path 36 | total_timesteps 333.
Path 37 | total_timesteps 344.
Path 38 | total_timesteps 356.
Path 39 | total_timesteps 363.
Path 40 | total_timesteps 372.
Path 41 | total_timesteps 383.
Path 42 | total_timesteps 395.
Path 43 | total_timesteps 403.
Path 44 | total_timesteps 409.
Path 45 | total_timesteps 417.
Path 46 | total_timesteps 426.
Path 47 | total_timesteps 433.
Path 48 | total_timesteps 441.
Path 49 | total_timesteps 449.
Path 50 | total_timesteps 460.
Path 51 | total_timesteps 470.
Path 52 | total_timesteps 480.
Path 53 | total_timesteps 492.
Path 54 | total_timesteps 502.
Path 55 | total_timesteps 529.
Path 56 | total_timesteps 536.
Path 57 | total_timesteps 544.
Path 58 | total_timesteps 554.
Path 59 | total_timesteps 563.
Path 60 | total_timesteps 574.
Path 61 | total_timesteps 583.
Path 62 | total_timesteps 591.
Path 63 | total_timesteps 601.
Path 64 | total_timesteps 610.
Path 65 | total_timesteps 617.
Path 66 | total_timesteps 623.
Path 67 | total_timesteps 632.
Path 68 | total_timesteps 642.
Path 69 | total_timesteps 650.
Path 70 | total_timesteps 658.
Path 71 | total_timesteps 671.
Path 72 | total_timesteps 679.
Path 73 | total_timesteps 691.
Path 74 | total_timesteps 701.
Path 75 | total_timesteps 710.
Path 76 | total_timesteps 721.
Path 77 | total_timesteps 730.
Path 78 | total_timesteps 739.
Path 79 | total_timesteps 750.
Path 80 | total_timesteps 758.
Path 81 | total_timesteps 766.
Path 82 | total_timesteps 776.
Path 83 | total_timesteps 785.
Path 84 | total_timesteps 797.
Path 85 | total_timesteps 807.
Path 86 | total_timesteps 818.
Path 87 | total_timesteps 827.
Path 88 | total_timesteps 841.
Path 89 | total_timesteps 856.
Path 90 | total_timesteps 866.
Path 91 | total_timesteps 874.
Path 92 | total_timesteps 882.
Path 93 | total_timesteps 890.
Path 94 | total_timesteps 900.
Path 95 | total_timesteps 909.
Path 96 | total_timesteps 917.
Path 97 | total_timesteps 925.
Path 98 | total_timesteps 932.
Path 99 | total_timesteps 938.
Path 100 | total_timesteps 947.
Path 101 | total_timesteps 960.
Path 102 | total_timesteps 967.
Path 103 | total_timesteps 974.
Path 104 | total_timesteps 982.
Path 105 | total_timesteps 990.
Path 106 | total_timesteps 998.
Path 107 | total_timesteps 1006.
Path 108 | total_timesteps 1014.
Path 109 | total_timesteps 1022.
Path 110 | total_timesteps 1031.
Path 111 | total_timesteps 1038.
Path 112 | total_timesteps 1048.
Path 113 | total_timesteps 1057.
Path 114 | total_timesteps 1072.
Path 115 | total_timesteps 1081.
Path 116 | total_timesteps 1092.
Path 117 | total_timesteps 1100.
Path 118 | total_timesteps 1108.
Path 119 | total_timesteps 1117.
Path 120 | total_timesteps 1124.
Path 121 | total_timesteps 1132.
Path 122 | total_timesteps 1139.
Path 123 | total_timesteps 1148.
Path 124 | total_timesteps 1161.
Path 125 | total_timesteps 1168.
Path 126 | total_timesteps 1178.
Path 127 | total_timesteps 1185.
Path 128 | total_timesteps 1192.
Path 129 | total_timesteps 1202.
Path 130 | total_timesteps 1213.
Path 131 | total_timesteps 1221.
Path 132 | total_timesteps 1228.
Path 133 | total_timesteps 1238.
Path 134 | total_timesteps 1249.
Path 135 | total_timesteps 1256.
Path 136 | total_timesteps 1264.
Path 137 | total_timesteps 1275.
Path 138 | total_timesteps 1282.
Path 139 | total_timesteps 1290.
Path 140 | total_timesteps 1301.
Path 141 | total_timesteps 1311.
Path 142 | total_timesteps 1318.
Path 143 | total_timesteps 1326.
Path 144 | total_timesteps 1333.
Path 145 | total_timesteps 1340.
Path 146 | total_timesteps 1355.
Path 147 | total_timesteps 1362.
Path 148 | total_timesteps 1370.
Path 149 | total_timesteps 1382.
Path 150 | total_timesteps 1391.
Path 151 | total_timesteps 1400.
Path 152 | total_timesteps 1413.
Path 153 | total_timesteps 1422.
Path 154 | total_timesteps 1430.
Path 155 | total_timesteps 1443.
Path 156 | total_timesteps 1455.
Path 157 | total_timesteps 1464.
Path 158 | total_timesteps 1471.
Path 159 | total_timesteps 1478.
Path 160 | total_timesteps 1485.
Path 161 | total_timesteps 1495.
Path 162 | total_timesteps 1502.
Path 163 | total_timesteps 1514.
Path 164 | total_timesteps 1523.
Path 165 | total_timesteps 1529.
Path 166 | total_timesteps 1539.
Path 167 | total_timesteps 1546.
Path 168 | total_timesteps 1559.
Path 169 | total_timesteps 1569.
Path 170 | total_timesteps 1577.
Path 171 | total_timesteps 1585.
Path 172 | total_timesteps 1593.
Path 173 | total_timesteps 1600.
Path 174 | total_timesteps 1611.
Path 175 | total_timesteps 1628.
Path 176 | total_timesteps 1636.
Path 177 | total_timesteps 1645.
Path 178 | total_timesteps 1654.
Path 179 | total_timesteps 1661.
Path 180 | total_timesteps 1671.
Path 181 | total_timesteps 1680.
Path 182 | total_timesteps 1688.
Path 183 | total_timesteps 1696.
Path 184 | total_timesteps 1707.
Path 185 | total_timesteps 1715.
Path 186 | total_timesteps 1726.
Path 187 | total_timesteps 1735.
Path 188 | total_timesteps 1745.
Path 189 | total_timesteps 1752.
Path 190 | total_timesteps 1759.
Path 191 | total_timesteps 1766.
Path 192 | total_timesteps 1773.
Path 193 | total_timesteps 1781.
Path 194 | total_timesteps 1790.
Path 195 | total_timesteps 1797.
Path 196 | total_timesteps 1804.
Path 197 | total_timesteps 1815.
Path 198 | total_timesteps 1824.
Path 199 | total_timesteps 1837.
Path 200 | total_timesteps 1844.
Path 201 | total_timesteps 1856.
Path 202 | total_timesteps 1864.
Path 203 | total_timesteps 1872.
Path 204 | total_timesteps 1879.
Path 205 | total_timesteps 1887.
Path 206 | total_timesteps 1902.
Path 207 | total_timesteps 1912.
Path 208 | total_timesteps 1922.
Path 209 | total_timesteps 1930.
Path 210 | total_timesteps 1938.
Path 211 | total_timesteps 1948.
Path 212 | total_timesteps 1957.
Path 213 | total_timesteps 1967.
Path 214 | total_timesteps 1978.
Path 215 | total_timesteps 1986.
Path 216 | total_timesteps 1992.
Path 217 | total_timesteps 1999.
Path 218 | total_timesteps 2008.
Path 219 | total_timesteps 2020.
Path 220 | total_timesteps 2033.
Path 221 | total_timesteps 2043.
Path 222 | total_timesteps 2052.
Path 223 | total_timesteps 2060.
Path 224 | total_timesteps 2068.
Path 225 | total_timesteps 2079.
Path 226 | total_timesteps 2089.
Path 227 | total_timesteps 2101.
Path 228 | total_timesteps 2110.
Path 229 | total_timesteps 2119.
Path 230 | total_timesteps 2128.
Path 231 | total_timesteps 2136.
Path 232 | total_timesteps 2143.
Path 233 | total_timesteps 2151.
Path 234 | total_timesteps 2160.
Path 235 | total_timesteps 2169.
Path 236 | total_timesteps 2176.
Path 237 | total_timesteps 2184.
Path 238 | total_timesteps 2193.
Path 239 | total_timesteps 2203.
Path 240 | total_timesteps 2212.
Path 241 | total_timesteps 2220.
Path 242 | total_timesteps 2228.
Path 243 | total_timesteps 2236.
Path 244 | total_timesteps 2245.
Path 245 | total_timesteps 2254.
Path 246 | total_timesteps 2265.
Path 247 | total_timesteps 2274.
Path 248 | total_timesteps 2282.
Path 249 | total_timesteps 2291.
Path 250 | total_timesteps 2298.
Path 251 | total_timesteps 2305.
Path 252 | total_timesteps 2311.
Path 253 | total_timesteps 2319.
Path 254 | total_timesteps 2328.
Path 255 | total_timesteps 2335.
Path 256 | total_timesteps 2341.
Path 257 | total_timesteps 2351.
Path 258 | total_timesteps 2359.
Path 259 | total_timesteps 2367.
Path 260 | total_timesteps 2375.
Path 261 | total_timesteps 2384.
Path 262 | total_timesteps 2395.
Path 263 | total_timesteps 2405.
Path 264 | total_timesteps 2416.
Path 265 | total_timesteps 2425.
Path 266 | total_timesteps 2434.
Path 267 | total_timesteps 2443.
Path 268 | total_timesteps 2452.
Path 269 | total_timesteps 2459.
Path 270 | total_timesteps 2466.
Path 271 | total_timesteps 2475.
Path 272 | total_timesteps 2488.
Path 273 | total_timesteps 2496.
Path 274 | total_timesteps 2512.
Path 275 | total_timesteps 2520.
Path 276 | total_timesteps 2531.
Path 277 | total_timesteps 2539.
Path 278 | total_timesteps 2548.
Path 279 | total_timesteps 2557.
Path 280 | total_timesteps 2565.
Path 281 | total_timesteps 2571.
Path 282 | total_timesteps 2581.
Path 283 | total_timesteps 2592.
Path 284 | total_timesteps 2601.
Path 285 | total_timesteps 2613.
Path 286 | total_timesteps 2621.
Path 287 | total_timesteps 2630.
Path 288 | total_timesteps 2639.
Path 289 | total_timesteps 2647.
Path 290 | total_timesteps 2657.
Path 291 | total_timesteps 2666.
Path 292 | total_timesteps 2677.
Path 293 | total_timesteps 2684.
Path 294 | total_timesteps 2692.
Path 295 | total_timesteps 2701.
Path 296 | total_timesteps 2709.
Path 297 | total_timesteps 2719.
Path 298 | total_timesteps 2729.
Path 299 | total_timesteps 2741.
Path 300 | total_timesteps 2750.
Path 301 | total_timesteps 2758.
Path 302 | total_timesteps 2768.
Path 303 | total_timesteps 2776.
Path 304 | total_timesteps 2785.
Path 305 | total_timesteps 2793.
Path 306 | total_timesteps 2808.
Path 307 | total_timesteps 2819.
Path 308 | total_timesteps 2830.
Path 309 | total_timesteps 2837.
Path 310 | total_timesteps 2852.
Path 311 | total_timesteps 2863.
Path 312 | total_timesteps 2872.
Path 313 | total_timesteps 2879.
Path 314 | total_timesteps 2888.
Path 315 | total_timesteps 2898.
Path 316 | total_timesteps 2906.
Path 317 | total_timesteps 2920.
Path 318 | total_timesteps 2931.
Path 319 | total_timesteps 2942.
Path 320 | total_timesteps 2950.
Path 321 | total_timesteps 2957.
Path 322 | total_timesteps 2968.
Path 323 | total_timesteps 2976.
Path 324 | total_timesteps 2985.
Path 325 | total_timesteps 2996.
Path 326 | total_timesteps 3003.
Path 327 | total_timesteps 3011.
Path 328 | total_timesteps 3019.
Path 329 | total_timesteps 3031.
Path 330 | total_timesteps 3042.
Path 331 | total_timesteps 3052.
Path 332 | total_timesteps 3059.
Path 333 | total_timesteps 3070.
Path 334 | total_timesteps 3080.
Path 335 | total_timesteps 3095.
Path 336 | total_timesteps 3104.
Path 337 | total_timesteps 3112.
Path 338 | total_timesteps 3122.
Path 339 | total_timesteps 3134.
Path 340 | total_timesteps 3142.
Path 341 | total_timesteps 3149.
Path 342 | total_timesteps 3156.
Path 343 | total_timesteps 3168.
Path 344 | total_timesteps 3180.
Path 345 | total_timesteps 3191.
Path 346 | total_timesteps 3202.
Path 347 | total_timesteps 3213.
Path 348 | total_timesteps 3226.
Path 349 | total_timesteps 3236.
Path 350 | total_timesteps 3243.
Path 351 | total_timesteps 3252.
Path 352 | total_timesteps 3263.
Path 353 | total_timesteps 3270.
Path 354 | total_timesteps 3279.
Path 355 | total_timesteps 3287.
Path 356 | total_timesteps 3310.
Path 357 | total_timesteps 3318.
Path 358 | total_timesteps 3325.
Path 359 | total_timesteps 3333.
Path 360 | total_timesteps 3341.
Path 361 | total_timesteps 3354.
Path 362 | total_timesteps 3368.
Path 363 | total_timesteps 3375.
Path 364 | total_timesteps 3386.
Path 365 | total_timesteps 3395.
Path 366 | total_timesteps 3402.
Path 367 | total_timesteps 3411.
Path 368 | total_timesteps 3419.
Path 369 | total_timesteps 3431.
Path 370 | total_timesteps 3438.
Path 371 | total_timesteps 3448.
Path 372 | total_timesteps 3462.
Path 373 | total_timesteps 3471.
Path 374 | total_timesteps 3484.
Path 375 | total_timesteps 3492.
Path 376 | total_timesteps 3501.
Path 377 | total_timesteps 3508.
Path 378 | total_timesteps 3515.
Path 379 | total_timesteps 3527.
Path 380 | total_timesteps 3537.
Path 381 | total_timesteps 3546.
Path 382 | total_timesteps 3555.
Path 383 | total_timesteps 3567.
Path 384 | total_timesteps 3573.
Path 385 | total_timesteps 3582.
Path 386 | total_timesteps 3593.
Path 387 | total_timesteps 3602.
Path 388 | total_timesteps 3609.
Path 389 | total_timesteps 3618.
Path 390 | total_timesteps 3630.
Path 391 | total_timesteps 3642.
Path 392 | total_timesteps 3654.
Path 393 | total_timesteps 3664.
Path 394 | total_timesteps 3674.
Path 395 | total_timesteps 3682.
Path 396 | total_timesteps 3692.
Path 397 | total_timesteps 3704.
Path 398 | total_timesteps 3721.
Path 399 | total_timesteps 3733.
Path 400 | total_timesteps 3742.
Path 401 | total_timesteps 3749.
Path 402 | total_timesteps 3758.
Path 403 | total_timesteps 3768.
Path 404 | total_timesteps 3775.
Path 405 | total_timesteps 3785.
Path 406 | total_timesteps 3794.
Path 407 | total_timesteps 3805.
Path 408 | total_timesteps 3820.
Path 409 | total_timesteps 3830.
Path 410 | total_timesteps 3841.
Path 411 | total_timesteps 3851.
Path 412 | total_timesteps 3866.
Path 413 | total_timesteps 3880.
Path 414 | total_timesteps 3889.
Path 415 | total_timesteps 3901.
Path 416 | total_timesteps 3909.
Path 417 | total_timesteps 3922.
Path 418 | total_timesteps 3932.
Path 419 | total_timesteps 3939.
Path 420 | total_timesteps 3946.
Path 421 | total_timesteps 3953.
Path 422 | total_timesteps 3960.
Path 423 | total_timesteps 3968.
Path 424 | total_timesteps 3975.
Path 425 | total_timesteps 3984.
Path 426 | total_timesteps 3990.
Path 427 | total_timesteps 4004.
Path 428 | total_timesteps 4017.
Path 429 | total_timesteps 4027.
Path 430 | total_timesteps 4036.
Path 431 | total_timesteps 4044.
Path 432 | total_timesteps 4055.
Path 433 | total_timesteps 4063.
Path 434 | total_timesteps 4072.
Path 435 | total_timesteps 4084.
Path 436 | total_timesteps 4095.
Path 437 | total_timesteps 4105.
Path 438 | total_timesteps 4114.
Path 439 | total_timesteps 4125.
Path 440 | total_timesteps 4136.
Path 441 | total_timesteps 4146.
Path 442 | total_timesteps 4157.
Path 443 | total_timesteps 4167.
Path 444 | total_timesteps 4175.
Path 445 | total_timesteps 4187.
Path 446 | total_timesteps 4196.
Path 447 | total_timesteps 4208.
Path 448 | total_timesteps 4218.
Path 449 | total_timesteps 4227.
Path 450 | total_timesteps 4234.
Path 451 | total_timesteps 4243.
Path 452 | total_timesteps 4252.
Path 453 | total_timesteps 4260.
Path 454 | total_timesteps 4269.
Path 455 | total_timesteps 4277.
Path 456 | total_timesteps 4286.
Path 457 | total_timesteps 4298.
Path 458 | total_timesteps 4305.
Path 459 | total_timesteps 4315.
Path 460 | total_timesteps 4325.
Path 461 | total_timesteps 4333.
Path 462 | total_timesteps 4342.
Path 463 | total_timesteps 4352.
Path 464 | total_timesteps 4358.
Path 465 | total_timesteps 4372.
Path 466 | total_timesteps 4383.
Path 467 | total_timesteps 4393.
Path 468 | total_timesteps 4404.
Path 469 | total_timesteps 4413.
Path 470 | total_timesteps 4424.
Path 471 | total_timesteps 4433.
Path 472 | total_timesteps 4441.
Path 473 | total_timesteps 4450.
Path 474 | total_timesteps 4458.
Path 475 | total_timesteps 4469.
Path 476 | total_timesteps 4480.
Path 477 | total_timesteps 4488.
Path 478 | total_timesteps 4500.
Path 479 | total_timesteps 4508.
Path 480 | total_timesteps 4516.
Path 481 | total_timesteps 4524.
Path 482 | total_timesteps 4531.
Path 483 | total_timesteps 4539.
Path 484 | total_timesteps 4548.
Path 485 | total_timesteps 4559.
Path 486 | total_timesteps 4567.
Path 487 | total_timesteps 4573.
Path 488 | total_timesteps 4584.
Path 489 | total_timesteps 4591.
Path 490 | total_timesteps 4600.
Path 491 | total_timesteps 4611.
Path 492 | total_timesteps 4624.
Path 493 | total_timesteps 4637.
Path 494 | total_timesteps 4647.
Path 495 | total_timesteps 4654.
Path 496 | total_timesteps 4666.
Path 497 | total_timesteps 4676.
Path 498 | total_timesteps 4684.
Path 499 | total_timesteps 4691.
Path 500 | total_timesteps 4701.
Path 501 | total_timesteps 4710.
Path 502 | total_timesteps 4720.
Path 503 | total_timesteps 4727.
Path 504 | total_timesteps 4739.
Path 505 | total_timesteps 4748.
Path 506 | total_timesteps 4756.
Path 507 | total_timesteps 4766.
Path 508 | total_timesteps 4778.
Path 509 | total_timesteps 4786.
Path 510 | total_timesteps 4798.
Path 511 | total_timesteps 4809.
Path 512 | total_timesteps 4818.
Path 513 | total_timesteps 4829.
Path 514 | total_timesteps 4841.
Path 515 | total_timesteps 4849.
Path 516 | total_timesteps 4859.
Path 517 | total_timesteps 4868.
Path 518 | total_timesteps 4875.
Path 519 | total_timesteps 4887.
Path 520 | total_timesteps 4897.
Path 521 | total_timesteps 4906.
Path 522 | total_timesteps 4917.
Path 523 | total_timesteps 4925.
Path 524 | total_timesteps 4934.
Path 525 | total_timesteps 4941.
Path 526 | total_timesteps 4949.
Path 527 | total_timesteps 4957.
Path 528 | total_timesteps 4965.
Path 529 | total_timesteps 4973.
Path 530 | total_timesteps 4982.
Path 531 | total_timesteps 4989.
Path 532 | total_timesteps 5000.
Path 533 | total_timesteps 5010.
Path 534 | total_timesteps 5023.
Path 535 | total_timesteps 5034.
Path 536 | total_timesteps 5044.
Path 537 | total_timesteps 5051.
Path 538 | total_timesteps 5059.
Path 539 | total_timesteps 5071.
Path 540 | total_timesteps 5082.
Path 541 | total_timesteps 5093.
Path 542 | total_timesteps 5105.
Path 543 | total_timesteps 5114.
Path 544 | total_timesteps 5122.
Path 545 | total_timesteps 5132.
Path 546 | total_timesteps 5143.
Path 547 | total_timesteps 5151.
Path 548 | total_timesteps 5160.
Path 549 | total_timesteps 5168.
Path 550 | total_timesteps 5178.
Path 551 | total_timesteps 5186.
Path 552 | total_timesteps 5200.
Path 553 | total_timesteps 5213.
Path 554 | total_timesteps 5222.
Path 555 | total_timesteps 5231.
Path 556 | total_timesteps 5240.
Path 557 | total_timesteps 5251.
Path 558 | total_timesteps 5259.
Path 559 | total_timesteps 5269.
Path 560 | total_timesteps 5276.
Path 561 | total_timesteps 5286.
Path 562 | total_timesteps 5296.
Path 563 | total_timesteps 5304.
Path 564 | total_timesteps 5315.
Path 565 | total_timesteps 5332.
Path 566 | total_timesteps 5339.
Path 567 | total_timesteps 5346.
Path 568 | total_timesteps 5356.
Path 569 | total_timesteps 5364.
Path 570 | total_timesteps 5377.
Path 571 | total_timesteps 5387.
Path 572 | total_timesteps 5395.
Path 573 | total_timesteps 5409.
Path 574 | total_timesteps 5418.
Path 575 | total_timesteps 5429.
Path 576 | total_timesteps 5436.
Path 577 | total_timesteps 5443.
Path 578 | total_timesteps 5452.
Path 579 | total_timesteps 5459.
Path 580 | total_timesteps 5466.
Path 581 | total_timesteps 5476.
Path 582 | total_timesteps 5486.
Path 583 | total_timesteps 5498.
Path 584 | total_timesteps 5507.
Path 585 | total_timesteps 5516.
Path 586 | total_timesteps 5528.
Path 587 | total_timesteps 5537.
Path 588 | total_timesteps 5545.
Path 589 | total_timesteps 5553.
Path 590 | total_timesteps 5561.
Path 591 | total_timesteps 5568.
Path 592 | total_timesteps 5578.
Path 593 | total_timesteps 5588.
Path 594 | total_timesteps 5597.
Path 595 | total_timesteps 5606.
Path 596 | total_timesteps 5619.
Path 597 | total_timesteps 5628.
Path 598 | total_timesteps 5637.
Path 599 | total_timesteps 5647.
Path 600 | total_timesteps 5659.
Path 601 | total_timesteps 5665.
Path 602 | total_timesteps 5673.
Path 603 | total_timesteps 5686.
Path 604 | total_timesteps 5696.
Path 605 | total_timesteps 5707.
Path 606 | total_timesteps 5716.
Path 607 | total_timesteps 5724.
Path 608 | total_timesteps 5730.
Path 609 | total_timesteps 5739.
Path 610 | total_timesteps 5750.
Path 611 | total_timesteps 5760.
Path 612 | total_timesteps 5771.
Path 613 | total_timesteps 5780.
Path 614 | total_timesteps 5791.
Path 615 | total_timesteps 5798.
Path 616 | total_timesteps 5808.
Path 617 | total_timesteps 5815.
Path 618 | total_timesteps 5826.
Path 619 | total_timesteps 5838.
Path 620 | total_timesteps 5849.
Path 621 | total_timesteps 5857.
Path 622 | total_timesteps 5867.
Path 623 | total_timesteps 5875.
Path 624 | total_timesteps 5888.
Path 625 | total_timesteps 5898.
Path 626 | total_timesteps 5907.
Path 627 | total_timesteps 5916.
Path 628 | total_timesteps 5925.
Path 629 | total_timesteps 5937.
Path 630 | total_timesteps 5946.
Path 631 | total_timesteps 5956.
Path 632 | total_timesteps 5966.
Path 633 | total_timesteps 5975.
Path 634 | total_timesteps 5990.
Path 635 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.92    |
| Iteration     | 28       |
| MaximumReturn | -0.82    |
| MinimumReturn | -15.9    |
| TotalSamples  | 120147   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.0019514107843860984
Validation loss = 0.002162736374884844
Validation loss = 0.0019729058258235455
Validation loss = 0.0020713540725409985
Validation loss = 0.0018854913068935275
Validation loss = 0.002024818677455187
Validation loss = 0.0019978603813797235
Validation loss = 0.0017933361232280731
Validation loss = 0.0017533915815874934
Validation loss = 0.002083639847114682
Validation loss = 0.0017502267146483064
Validation loss = 0.0017947540618479252
Validation loss = 0.0018283554818481207
Validation loss = 0.0016556925838813186
Validation loss = 0.001851811190135777
Validation loss = 0.0016545449616387486
Validation loss = 0.0018344416748732328
Validation loss = 0.0018110707169398665
Validation loss = 0.0017850040458142757
Validation loss = 0.0017859115032479167
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 60.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 82.
Path 9 | total_timesteps 91.
Path 10 | total_timesteps 97.
Path 11 | total_timesteps 103.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 124.
Path 14 | total_timesteps 131.
Path 15 | total_timesteps 137.
Path 16 | total_timesteps 144.
Path 17 | total_timesteps 153.
Path 18 | total_timesteps 162.
Path 19 | total_timesteps 170.
Path 20 | total_timesteps 182.
Path 21 | total_timesteps 193.
Path 22 | total_timesteps 203.
Path 23 | total_timesteps 216.
Path 24 | total_timesteps 224.
Path 25 | total_timesteps 233.
Path 26 | total_timesteps 243.
Path 27 | total_timesteps 255.
Path 28 | total_timesteps 265.
Path 29 | total_timesteps 273.
Path 30 | total_timesteps 283.
Path 31 | total_timesteps 293.
Path 32 | total_timesteps 305.
Path 33 | total_timesteps 313.
Path 34 | total_timesteps 323.
Path 35 | total_timesteps 332.
Path 36 | total_timesteps 339.
Path 37 | total_timesteps 350.
Path 38 | total_timesteps 360.
Path 39 | total_timesteps 368.
Path 40 | total_timesteps 378.
Path 41 | total_timesteps 387.
Path 42 | total_timesteps 396.
Path 43 | total_timesteps 407.
Path 44 | total_timesteps 415.
Path 45 | total_timesteps 427.
Path 46 | total_timesteps 436.
Path 47 | total_timesteps 445.
Path 48 | total_timesteps 453.
Path 49 | total_timesteps 464.
Path 50 | total_timesteps 474.
Path 51 | total_timesteps 481.
Path 52 | total_timesteps 488.
Path 53 | total_timesteps 500.
Path 54 | total_timesteps 508.
Path 55 | total_timesteps 517.
Path 56 | total_timesteps 523.
Path 57 | total_timesteps 534.
Path 58 | total_timesteps 543.
Path 59 | total_timesteps 552.
Path 60 | total_timesteps 561.
Path 61 | total_timesteps 569.
Path 62 | total_timesteps 582.
Path 63 | total_timesteps 591.
Path 64 | total_timesteps 599.
Path 65 | total_timesteps 607.
Path 66 | total_timesteps 619.
Path 67 | total_timesteps 626.
Path 68 | total_timesteps 634.
Path 69 | total_timesteps 642.
Path 70 | total_timesteps 652.
Path 71 | total_timesteps 659.
Path 72 | total_timesteps 667.
Path 73 | total_timesteps 675.
Path 74 | total_timesteps 683.
Path 75 | total_timesteps 691.
Path 76 | total_timesteps 701.
Path 77 | total_timesteps 714.
Path 78 | total_timesteps 724.
Path 79 | total_timesteps 731.
Path 80 | total_timesteps 741.
Path 81 | total_timesteps 751.
Path 82 | total_timesteps 762.
Path 83 | total_timesteps 771.
Path 84 | total_timesteps 781.
Path 85 | total_timesteps 794.
Path 86 | total_timesteps 801.
Path 87 | total_timesteps 809.
Path 88 | total_timesteps 820.
Path 89 | total_timesteps 827.
Path 90 | total_timesteps 838.
Path 91 | total_timesteps 846.
Path 92 | total_timesteps 855.
Path 93 | total_timesteps 863.
Path 94 | total_timesteps 873.
Path 95 | total_timesteps 884.
Path 96 | total_timesteps 892.
Path 97 | total_timesteps 901.
Path 98 | total_timesteps 909.
Path 99 | total_timesteps 919.
Path 100 | total_timesteps 927.
Path 101 | total_timesteps 937.
Path 102 | total_timesteps 946.
Path 103 | total_timesteps 955.
Path 104 | total_timesteps 963.
Path 105 | total_timesteps 971.
Path 106 | total_timesteps 980.
Path 107 | total_timesteps 989.
Path 108 | total_timesteps 998.
Path 109 | total_timesteps 1012.
Path 110 | total_timesteps 1021.
Path 111 | total_timesteps 1031.
Path 112 | total_timesteps 1041.
Path 113 | total_timesteps 1049.
Path 114 | total_timesteps 1058.
Path 115 | total_timesteps 1065.
Path 116 | total_timesteps 1072.
Path 117 | total_timesteps 1083.
Path 118 | total_timesteps 1094.
Path 119 | total_timesteps 1103.
Path 120 | total_timesteps 1111.
Path 121 | total_timesteps 1118.
Path 122 | total_timesteps 1133.
Path 123 | total_timesteps 1144.
Path 124 | total_timesteps 1151.
Path 125 | total_timesteps 1164.
Path 126 | total_timesteps 1170.
Path 127 | total_timesteps 1179.
Path 128 | total_timesteps 1188.
Path 129 | total_timesteps 1197.
Path 130 | total_timesteps 1208.
Path 131 | total_timesteps 1217.
Path 132 | total_timesteps 1227.
Path 133 | total_timesteps 1238.
Path 134 | total_timesteps 1247.
Path 135 | total_timesteps 1258.
Path 136 | total_timesteps 1273.
Path 137 | total_timesteps 1284.
Path 138 | total_timesteps 1296.
Path 139 | total_timesteps 1305.
Path 140 | total_timesteps 1316.
Path 141 | total_timesteps 1325.
Path 142 | total_timesteps 1333.
Path 143 | total_timesteps 1343.
Path 144 | total_timesteps 1351.
Path 145 | total_timesteps 1361.
Path 146 | total_timesteps 1371.
Path 147 | total_timesteps 1378.
Path 148 | total_timesteps 1389.
Path 149 | total_timesteps 1398.
Path 150 | total_timesteps 1409.
Path 151 | total_timesteps 1417.
Path 152 | total_timesteps 1427.
Path 153 | total_timesteps 1437.
Path 154 | total_timesteps 1446.
Path 155 | total_timesteps 1454.
Path 156 | total_timesteps 1462.
Path 157 | total_timesteps 1472.
Path 158 | total_timesteps 1481.
Path 159 | total_timesteps 1491.
Path 160 | total_timesteps 1502.
Path 161 | total_timesteps 1510.
Path 162 | total_timesteps 1518.
Path 163 | total_timesteps 1526.
Path 164 | total_timesteps 1536.
Path 165 | total_timesteps 1548.
Path 166 | total_timesteps 1556.
Path 167 | total_timesteps 1567.
Path 168 | total_timesteps 1575.
Path 169 | total_timesteps 1583.
Path 170 | total_timesteps 1590.
Path 171 | total_timesteps 1598.
Path 172 | total_timesteps 1608.
Path 173 | total_timesteps 1616.
Path 174 | total_timesteps 1625.
Path 175 | total_timesteps 1632.
Path 176 | total_timesteps 1641.
Path 177 | total_timesteps 1657.
Path 178 | total_timesteps 1665.
Path 179 | total_timesteps 1672.
Path 180 | total_timesteps 1682.
Path 181 | total_timesteps 1690.
Path 182 | total_timesteps 1699.
Path 183 | total_timesteps 1707.
Path 184 | total_timesteps 1716.
Path 185 | total_timesteps 1727.
Path 186 | total_timesteps 1736.
Path 187 | total_timesteps 1744.
Path 188 | total_timesteps 1754.
Path 189 | total_timesteps 1764.
Path 190 | total_timesteps 1771.
Path 191 | total_timesteps 1781.
Path 192 | total_timesteps 1788.
Path 193 | total_timesteps 1798.
Path 194 | total_timesteps 1806.
Path 195 | total_timesteps 1814.
Path 196 | total_timesteps 1821.
Path 197 | total_timesteps 1827.
Path 198 | total_timesteps 1843.
Path 199 | total_timesteps 1852.
Path 200 | total_timesteps 1862.
Path 201 | total_timesteps 1870.
Path 202 | total_timesteps 1887.
Path 203 | total_timesteps 1896.
Path 204 | total_timesteps 1904.
Path 205 | total_timesteps 1919.
Path 206 | total_timesteps 1926.
Path 207 | total_timesteps 1934.
Path 208 | total_timesteps 1940.
Path 209 | total_timesteps 1951.
Path 210 | total_timesteps 1962.
Path 211 | total_timesteps 1971.
Path 212 | total_timesteps 1979.
Path 213 | total_timesteps 1995.
Path 214 | total_timesteps 2006.
Path 215 | total_timesteps 2015.
Path 216 | total_timesteps 2022.
Path 217 | total_timesteps 2031.
Path 218 | total_timesteps 2042.
Path 219 | total_timesteps 2049.
Path 220 | total_timesteps 2058.
Path 221 | total_timesteps 2069.
Path 222 | total_timesteps 2085.
Path 223 | total_timesteps 2096.
Path 224 | total_timesteps 2105.
Path 225 | total_timesteps 2116.
Path 226 | total_timesteps 2127.
Path 227 | total_timesteps 2137.
Path 228 | total_timesteps 2148.
Path 229 | total_timesteps 2154.
Path 230 | total_timesteps 2164.
Path 231 | total_timesteps 2173.
Path 232 | total_timesteps 2184.
Path 233 | total_timesteps 2193.
Path 234 | total_timesteps 2203.
Path 235 | total_timesteps 2210.
Path 236 | total_timesteps 2219.
Path 237 | total_timesteps 2231.
Path 238 | total_timesteps 2242.
Path 239 | total_timesteps 2251.
Path 240 | total_timesteps 2263.
Path 241 | total_timesteps 2271.
Path 242 | total_timesteps 2284.
Path 243 | total_timesteps 2293.
Path 244 | total_timesteps 2303.
Path 245 | total_timesteps 2312.
Path 246 | total_timesteps 2320.
Path 247 | total_timesteps 2331.
Path 248 | total_timesteps 2340.
Path 249 | total_timesteps 2349.
Path 250 | total_timesteps 2358.
Path 251 | total_timesteps 2366.
Path 252 | total_timesteps 2378.
Path 253 | total_timesteps 2385.
Path 254 | total_timesteps 2393.
Path 255 | total_timesteps 2402.
Path 256 | total_timesteps 2409.
Path 257 | total_timesteps 2417.
Path 258 | total_timesteps 2424.
Path 259 | total_timesteps 2434.
Path 260 | total_timesteps 2442.
Path 261 | total_timesteps 2449.
Path 262 | total_timesteps 2457.
Path 263 | total_timesteps 2464.
Path 264 | total_timesteps 2472.
Path 265 | total_timesteps 2486.
Path 266 | total_timesteps 2495.
Path 267 | total_timesteps 2507.
Path 268 | total_timesteps 2516.
Path 269 | total_timesteps 2525.
Path 270 | total_timesteps 2535.
Path 271 | total_timesteps 2544.
Path 272 | total_timesteps 2552.
Path 273 | total_timesteps 2559.
Path 274 | total_timesteps 2569.
Path 275 | total_timesteps 2577.
Path 276 | total_timesteps 2590.
Path 277 | total_timesteps 2597.
Path 278 | total_timesteps 2604.
Path 279 | total_timesteps 2612.
Path 280 | total_timesteps 2624.
Path 281 | total_timesteps 2632.
Path 282 | total_timesteps 2642.
Path 283 | total_timesteps 2652.
Path 284 | total_timesteps 2664.
Path 285 | total_timesteps 2675.
Path 286 | total_timesteps 2682.
Path 287 | total_timesteps 2692.
Path 288 | total_timesteps 2702.
Path 289 | total_timesteps 2708.
Path 290 | total_timesteps 2717.
Path 291 | total_timesteps 2725.
Path 292 | total_timesteps 2734.
Path 293 | total_timesteps 2743.
Path 294 | total_timesteps 2754.
Path 295 | total_timesteps 2763.
Path 296 | total_timesteps 2772.
Path 297 | total_timesteps 2784.
Path 298 | total_timesteps 2794.
Path 299 | total_timesteps 2803.
Path 300 | total_timesteps 2811.
Path 301 | total_timesteps 2823.
Path 302 | total_timesteps 2830.
Path 303 | total_timesteps 2839.
Path 304 | total_timesteps 2847.
Path 305 | total_timesteps 2858.
Path 306 | total_timesteps 2867.
Path 307 | total_timesteps 2873.
Path 308 | total_timesteps 2882.
Path 309 | total_timesteps 2891.
Path 310 | total_timesteps 2902.
Path 311 | total_timesteps 2913.
Path 312 | total_timesteps 2922.
Path 313 | total_timesteps 2932.
Path 314 | total_timesteps 2940.
Path 315 | total_timesteps 2950.
Path 316 | total_timesteps 2965.
Path 317 | total_timesteps 2980.
Path 318 | total_timesteps 2989.
Path 319 | total_timesteps 2998.
Path 320 | total_timesteps 3005.
Path 321 | total_timesteps 3019.
Path 322 | total_timesteps 3028.
Path 323 | total_timesteps 3035.
Path 324 | total_timesteps 3045.
Path 325 | total_timesteps 3055.
Path 326 | total_timesteps 3062.
Path 327 | total_timesteps 3071.
Path 328 | total_timesteps 3079.
Path 329 | total_timesteps 3092.
Path 330 | total_timesteps 3102.
Path 331 | total_timesteps 3109.
Path 332 | total_timesteps 3117.
Path 333 | total_timesteps 3124.
Path 334 | total_timesteps 3133.
Path 335 | total_timesteps 3142.
Path 336 | total_timesteps 3154.
Path 337 | total_timesteps 3165.
Path 338 | total_timesteps 3172.
Path 339 | total_timesteps 3180.
Path 340 | total_timesteps 3188.
Path 341 | total_timesteps 3200.
Path 342 | total_timesteps 3211.
Path 343 | total_timesteps 3222.
Path 344 | total_timesteps 3233.
Path 345 | total_timesteps 3241.
Path 346 | total_timesteps 3250.
Path 347 | total_timesteps 3261.
Path 348 | total_timesteps 3272.
Path 349 | total_timesteps 3284.
Path 350 | total_timesteps 3293.
Path 351 | total_timesteps 3303.
Path 352 | total_timesteps 3311.
Path 353 | total_timesteps 3323.
Path 354 | total_timesteps 3335.
Path 355 | total_timesteps 3344.
Path 356 | total_timesteps 3351.
Path 357 | total_timesteps 3361.
Path 358 | total_timesteps 3369.
Path 359 | total_timesteps 3378.
Path 360 | total_timesteps 3387.
Path 361 | total_timesteps 3398.
Path 362 | total_timesteps 3408.
Path 363 | total_timesteps 3418.
Path 364 | total_timesteps 3427.
Path 365 | total_timesteps 3439.
Path 366 | total_timesteps 3450.
Path 367 | total_timesteps 3459.
Path 368 | total_timesteps 3472.
Path 369 | total_timesteps 3481.
Path 370 | total_timesteps 3490.
Path 371 | total_timesteps 3497.
Path 372 | total_timesteps 3505.
Path 373 | total_timesteps 3512.
Path 374 | total_timesteps 3519.
Path 375 | total_timesteps 3533.
Path 376 | total_timesteps 3542.
Path 377 | total_timesteps 3550.
Path 378 | total_timesteps 3561.
Path 379 | total_timesteps 3569.
Path 380 | total_timesteps 3576.
Path 381 | total_timesteps 3585.
Path 382 | total_timesteps 3593.
Path 383 | total_timesteps 3608.
Path 384 | total_timesteps 3622.
Path 385 | total_timesteps 3631.
Path 386 | total_timesteps 3641.
Path 387 | total_timesteps 3649.
Path 388 | total_timesteps 3657.
Path 389 | total_timesteps 3667.
Path 390 | total_timesteps 3675.
Path 391 | total_timesteps 3684.
Path 392 | total_timesteps 3694.
Path 393 | total_timesteps 3702.
Path 394 | total_timesteps 3709.
Path 395 | total_timesteps 3721.
Path 396 | total_timesteps 3729.
Path 397 | total_timesteps 3737.
Path 398 | total_timesteps 3744.
Path 399 | total_timesteps 3752.
Path 400 | total_timesteps 3760.
Path 401 | total_timesteps 3768.
Path 402 | total_timesteps 3788.
Path 403 | total_timesteps 3797.
Path 404 | total_timesteps 3804.
Path 405 | total_timesteps 3815.
Path 406 | total_timesteps 3828.
Path 407 | total_timesteps 3836.
Path 408 | total_timesteps 3845.
Path 409 | total_timesteps 3856.
Path 410 | total_timesteps 3866.
Path 411 | total_timesteps 3879.
Path 412 | total_timesteps 3888.
Path 413 | total_timesteps 3897.
Path 414 | total_timesteps 3909.
Path 415 | total_timesteps 3917.
Path 416 | total_timesteps 3927.
Path 417 | total_timesteps 3936.
Path 418 | total_timesteps 3945.
Path 419 | total_timesteps 3961.
Path 420 | total_timesteps 3971.
Path 421 | total_timesteps 3981.
Path 422 | total_timesteps 3988.
Path 423 | total_timesteps 3997.
Path 424 | total_timesteps 4007.
Path 425 | total_timesteps 4017.
Path 426 | total_timesteps 4028.
Path 427 | total_timesteps 4038.
Path 428 | total_timesteps 4045.
Path 429 | total_timesteps 4054.
Path 430 | total_timesteps 4062.
Path 431 | total_timesteps 4071.
Path 432 | total_timesteps 4079.
Path 433 | total_timesteps 4088.
Path 434 | total_timesteps 4096.
Path 435 | total_timesteps 4103.
Path 436 | total_timesteps 4114.
Path 437 | total_timesteps 4128.
Path 438 | total_timesteps 4135.
Path 439 | total_timesteps 4142.
Path 440 | total_timesteps 4150.
Path 441 | total_timesteps 4158.
Path 442 | total_timesteps 4166.
Path 443 | total_timesteps 4177.
Path 444 | total_timesteps 4185.
Path 445 | total_timesteps 4196.
Path 446 | total_timesteps 4206.
Path 447 | total_timesteps 4214.
Path 448 | total_timesteps 4221.
Path 449 | total_timesteps 4233.
Path 450 | total_timesteps 4242.
Path 451 | total_timesteps 4249.
Path 452 | total_timesteps 4259.
Path 453 | total_timesteps 4269.
Path 454 | total_timesteps 4280.
Path 455 | total_timesteps 4288.
Path 456 | total_timesteps 4297.
Path 457 | total_timesteps 4305.
Path 458 | total_timesteps 4316.
Path 459 | total_timesteps 4323.
Path 460 | total_timesteps 4330.
Path 461 | total_timesteps 4338.
Path 462 | total_timesteps 4347.
Path 463 | total_timesteps 4359.
Path 464 | total_timesteps 4367.
Path 465 | total_timesteps 4375.
Path 466 | total_timesteps 4384.
Path 467 | total_timesteps 4393.
Path 468 | total_timesteps 4400.
Path 469 | total_timesteps 4407.
Path 470 | total_timesteps 4421.
Path 471 | total_timesteps 4431.
Path 472 | total_timesteps 4442.
Path 473 | total_timesteps 4450.
Path 474 | total_timesteps 4458.
Path 475 | total_timesteps 4466.
Path 476 | total_timesteps 4473.
Path 477 | total_timesteps 4485.
Path 478 | total_timesteps 4495.
Path 479 | total_timesteps 4501.
Path 480 | total_timesteps 4513.
Path 481 | total_timesteps 4520.
Path 482 | total_timesteps 4528.
Path 483 | total_timesteps 4538.
Path 484 | total_timesteps 4546.
Path 485 | total_timesteps 4555.
Path 486 | total_timesteps 4563.
Path 487 | total_timesteps 4572.
Path 488 | total_timesteps 4581.
Path 489 | total_timesteps 4589.
Path 490 | total_timesteps 4603.
Path 491 | total_timesteps 4614.
Path 492 | total_timesteps 4622.
Path 493 | total_timesteps 4630.
Path 494 | total_timesteps 4640.
Path 495 | total_timesteps 4653.
Path 496 | total_timesteps 4663.
Path 497 | total_timesteps 4673.
Path 498 | total_timesteps 4679.
Path 499 | total_timesteps 4689.
Path 500 | total_timesteps 4698.
Path 501 | total_timesteps 4707.
Path 502 | total_timesteps 4715.
Path 503 | total_timesteps 4723.
Path 504 | total_timesteps 4738.
Path 505 | total_timesteps 4745.
Path 506 | total_timesteps 4752.
Path 507 | total_timesteps 4760.
Path 508 | total_timesteps 4771.
Path 509 | total_timesteps 4777.
Path 510 | total_timesteps 4785.
Path 511 | total_timesteps 4794.
Path 512 | total_timesteps 4803.
Path 513 | total_timesteps 4811.
Path 514 | total_timesteps 4823.
Path 515 | total_timesteps 4830.
Path 516 | total_timesteps 4841.
Path 517 | total_timesteps 4848.
Path 518 | total_timesteps 4860.
Path 519 | total_timesteps 4869.
Path 520 | total_timesteps 4883.
Path 521 | total_timesteps 4892.
Path 522 | total_timesteps 4901.
Path 523 | total_timesteps 4907.
Path 524 | total_timesteps 4917.
Path 525 | total_timesteps 4926.
Path 526 | total_timesteps 4937.
Path 527 | total_timesteps 4950.
Path 528 | total_timesteps 4959.
Path 529 | total_timesteps 4971.
Path 530 | total_timesteps 4978.
Path 531 | total_timesteps 4992.
Path 532 | total_timesteps 5004.
Path 533 | total_timesteps 5012.
Path 534 | total_timesteps 5022.
Path 535 | total_timesteps 5032.
Path 536 | total_timesteps 5042.
Path 537 | total_timesteps 5051.
Path 538 | total_timesteps 5061.
Path 539 | total_timesteps 5075.
Path 540 | total_timesteps 5083.
Path 541 | total_timesteps 5090.
Path 542 | total_timesteps 5105.
Path 543 | total_timesteps 5113.
Path 544 | total_timesteps 5122.
Path 545 | total_timesteps 5132.
Path 546 | total_timesteps 5141.
Path 547 | total_timesteps 5151.
Path 548 | total_timesteps 5159.
Path 549 | total_timesteps 5174.
Path 550 | total_timesteps 5186.
Path 551 | total_timesteps 5195.
Path 552 | total_timesteps 5204.
Path 553 | total_timesteps 5212.
Path 554 | total_timesteps 5223.
Path 555 | total_timesteps 5229.
Path 556 | total_timesteps 5239.
Path 557 | total_timesteps 5248.
Path 558 | total_timesteps 5257.
Path 559 | total_timesteps 5266.
Path 560 | total_timesteps 5275.
Path 561 | total_timesteps 5286.
Path 562 | total_timesteps 5294.
Path 563 | total_timesteps 5304.
Path 564 | total_timesteps 5312.
Path 565 | total_timesteps 5322.
Path 566 | total_timesteps 5330.
Path 567 | total_timesteps 5339.
Path 568 | total_timesteps 5349.
Path 569 | total_timesteps 5360.
Path 570 | total_timesteps 5369.
Path 571 | total_timesteps 5378.
Path 572 | total_timesteps 5386.
Path 573 | total_timesteps 5393.
Path 574 | total_timesteps 5402.
Path 575 | total_timesteps 5412.
Path 576 | total_timesteps 5423.
Path 577 | total_timesteps 5433.
Path 578 | total_timesteps 5442.
Path 579 | total_timesteps 5450.
Path 580 | total_timesteps 5459.
Path 581 | total_timesteps 5467.
Path 582 | total_timesteps 5482.
Path 583 | total_timesteps 5489.
Path 584 | total_timesteps 5498.
Path 585 | total_timesteps 5505.
Path 586 | total_timesteps 5516.
Path 587 | total_timesteps 5524.
Path 588 | total_timesteps 5531.
Path 589 | total_timesteps 5544.
Path 590 | total_timesteps 5552.
Path 591 | total_timesteps 5559.
Path 592 | total_timesteps 5567.
Path 593 | total_timesteps 5576.
Path 594 | total_timesteps 5585.
Path 595 | total_timesteps 5593.
Path 596 | total_timesteps 5600.
Path 597 | total_timesteps 5607.
Path 598 | total_timesteps 5616.
Path 599 | total_timesteps 5624.
Path 600 | total_timesteps 5631.
Path 601 | total_timesteps 5642.
Path 602 | total_timesteps 5649.
Path 603 | total_timesteps 5661.
Path 604 | total_timesteps 5669.
Path 605 | total_timesteps 5677.
Path 606 | total_timesteps 5687.
Path 607 | total_timesteps 5694.
Path 608 | total_timesteps 5703.
Path 609 | total_timesteps 5711.
Path 610 | total_timesteps 5720.
Path 611 | total_timesteps 5729.
Path 612 | total_timesteps 5736.
Path 613 | total_timesteps 5748.
Path 614 | total_timesteps 5757.
Path 615 | total_timesteps 5765.
Path 616 | total_timesteps 5774.
Path 617 | total_timesteps 5781.
Path 618 | total_timesteps 5793.
Path 619 | total_timesteps 5801.
Path 620 | total_timesteps 5808.
Path 621 | total_timesteps 5817.
Path 622 | total_timesteps 5824.
Path 623 | total_timesteps 5834.
Path 624 | total_timesteps 5842.
Path 625 | total_timesteps 5851.
Path 626 | total_timesteps 5863.
Path 627 | total_timesteps 5873.
Path 628 | total_timesteps 5882.
Path 629 | total_timesteps 5891.
Path 630 | total_timesteps 5901.
Path 631 | total_timesteps 5912.
Path 632 | total_timesteps 5921.
Path 633 | total_timesteps 5930.
Path 634 | total_timesteps 5938.
Path 635 | total_timesteps 5946.
Path 636 | total_timesteps 5956.
Path 637 | total_timesteps 5965.
Path 638 | total_timesteps 5973.
Path 639 | total_timesteps 5982.
Path 640 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.84    |
| Iteration     | 29       |
| MaximumReturn | 1.99     |
| MinimumReturn | -13.9    |
| TotalSamples  | 124147   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.0020101687405258417
Validation loss = 0.001910901046358049
Validation loss = 0.0018099166918545961
Validation loss = 0.0017756185261532664
Validation loss = 0.0017316978191956878
Validation loss = 0.0017065581632778049
Validation loss = 0.0018401355482637882
Validation loss = 0.0016819954616948962
Validation loss = 0.0017616837285459042
Validation loss = 0.001812514616176486
Validation loss = 0.0016725087771192193
Validation loss = 0.0017016996862366796
Validation loss = 0.0017188868951052427
Validation loss = 0.0018197322497144341
Validation loss = 0.001729058800265193
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 77.
Path 9 | total_timesteps 83.
Path 10 | total_timesteps 93.
Path 11 | total_timesteps 102.
Path 12 | total_timesteps 114.
Path 13 | total_timesteps 124.
Path 14 | total_timesteps 134.
Path 15 | total_timesteps 143.
Path 16 | total_timesteps 150.
Path 17 | total_timesteps 157.
Path 18 | total_timesteps 167.
Path 19 | total_timesteps 177.
Path 20 | total_timesteps 185.
Path 21 | total_timesteps 195.
Path 22 | total_timesteps 205.
Path 23 | total_timesteps 213.
Path 24 | total_timesteps 225.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 247.
Path 27 | total_timesteps 256.
Path 28 | total_timesteps 264.
Path 29 | total_timesteps 278.
Path 30 | total_timesteps 286.
Path 31 | total_timesteps 297.
Path 32 | total_timesteps 307.
Path 33 | total_timesteps 316.
Path 34 | total_timesteps 325.
Path 35 | total_timesteps 335.
Path 36 | total_timesteps 346.
Path 37 | total_timesteps 354.
Path 38 | total_timesteps 366.
Path 39 | total_timesteps 377.
Path 40 | total_timesteps 385.
Path 41 | total_timesteps 393.
Path 42 | total_timesteps 404.
Path 43 | total_timesteps 415.
Path 44 | total_timesteps 426.
Path 45 | total_timesteps 437.
Path 46 | total_timesteps 444.
Path 47 | total_timesteps 455.
Path 48 | total_timesteps 462.
Path 49 | total_timesteps 472.
Path 50 | total_timesteps 481.
Path 51 | total_timesteps 495.
Path 52 | total_timesteps 505.
Path 53 | total_timesteps 512.
Path 54 | total_timesteps 524.
Path 55 | total_timesteps 535.
Path 56 | total_timesteps 545.
Path 57 | total_timesteps 560.
Path 58 | total_timesteps 571.
Path 59 | total_timesteps 579.
Path 60 | total_timesteps 586.
Path 61 | total_timesteps 595.
Path 62 | total_timesteps 602.
Path 63 | total_timesteps 612.
Path 64 | total_timesteps 619.
Path 65 | total_timesteps 627.
Path 66 | total_timesteps 639.
Path 67 | total_timesteps 645.
Path 68 | total_timesteps 654.
Path 69 | total_timesteps 662.
Path 70 | total_timesteps 674.
Path 71 | total_timesteps 682.
Path 72 | total_timesteps 692.
Path 73 | total_timesteps 700.
Path 74 | total_timesteps 710.
Path 75 | total_timesteps 730.
Path 76 | total_timesteps 739.
Path 77 | total_timesteps 746.
Path 78 | total_timesteps 753.
Path 79 | total_timesteps 767.
Path 80 | total_timesteps 775.
Path 81 | total_timesteps 786.
Path 82 | total_timesteps 798.
Path 83 | total_timesteps 809.
Path 84 | total_timesteps 818.
Path 85 | total_timesteps 831.
Path 86 | total_timesteps 841.
Path 87 | total_timesteps 848.
Path 88 | total_timesteps 856.
Path 89 | total_timesteps 863.
Path 90 | total_timesteps 872.
Path 91 | total_timesteps 880.
Path 92 | total_timesteps 891.
Path 93 | total_timesteps 906.
Path 94 | total_timesteps 912.
Path 95 | total_timesteps 920.
Path 96 | total_timesteps 928.
Path 97 | total_timesteps 938.
Path 98 | total_timesteps 947.
Path 99 | total_timesteps 957.
Path 100 | total_timesteps 966.
Path 101 | total_timesteps 975.
Path 102 | total_timesteps 984.
Path 103 | total_timesteps 991.
Path 104 | total_timesteps 1002.
Path 105 | total_timesteps 1017.
Path 106 | total_timesteps 1028.
Path 107 | total_timesteps 1035.
Path 108 | total_timesteps 1046.
Path 109 | total_timesteps 1056.
Path 110 | total_timesteps 1063.
Path 111 | total_timesteps 1071.
Path 112 | total_timesteps 1080.
Path 113 | total_timesteps 1087.
Path 114 | total_timesteps 1097.
Path 115 | total_timesteps 1107.
Path 116 | total_timesteps 1117.
Path 117 | total_timesteps 1126.
Path 118 | total_timesteps 1134.
Path 119 | total_timesteps 1145.
Path 120 | total_timesteps 1153.
Path 121 | total_timesteps 1162.
Path 122 | total_timesteps 1177.
Path 123 | total_timesteps 1184.
Path 124 | total_timesteps 1192.
Path 125 | total_timesteps 1199.
Path 126 | total_timesteps 1207.
Path 127 | total_timesteps 1216.
Path 128 | total_timesteps 1228.
Path 129 | total_timesteps 1238.
Path 130 | total_timesteps 1247.
Path 131 | total_timesteps 1256.
Path 132 | total_timesteps 1265.
Path 133 | total_timesteps 1276.
Path 134 | total_timesteps 1287.
Path 135 | total_timesteps 1294.
Path 136 | total_timesteps 1305.
Path 137 | total_timesteps 1316.
Path 138 | total_timesteps 1327.
Path 139 | total_timesteps 1336.
Path 140 | total_timesteps 1343.
Path 141 | total_timesteps 1351.
Path 142 | total_timesteps 1360.
Path 143 | total_timesteps 1369.
Path 144 | total_timesteps 1378.
Path 145 | total_timesteps 1391.
Path 146 | total_timesteps 1399.
Path 147 | total_timesteps 1409.
Path 148 | total_timesteps 1417.
Path 149 | total_timesteps 1430.
Path 150 | total_timesteps 1438.
Path 151 | total_timesteps 1448.
Path 152 | total_timesteps 1460.
Path 153 | total_timesteps 1467.
Path 154 | total_timesteps 1473.
Path 155 | total_timesteps 1480.
Path 156 | total_timesteps 1490.
Path 157 | total_timesteps 1499.
Path 158 | total_timesteps 1507.
Path 159 | total_timesteps 1518.
Path 160 | total_timesteps 1527.
Path 161 | total_timesteps 1537.
Path 162 | total_timesteps 1552.
Path 163 | total_timesteps 1561.
Path 164 | total_timesteps 1571.
Path 165 | total_timesteps 1583.
Path 166 | total_timesteps 1593.
Path 167 | total_timesteps 1603.
Path 168 | total_timesteps 1611.
Path 169 | total_timesteps 1619.
Path 170 | total_timesteps 1629.
Path 171 | total_timesteps 1638.
Path 172 | total_timesteps 1646.
Path 173 | total_timesteps 1658.
Path 174 | total_timesteps 1668.
Path 175 | total_timesteps 1677.
Path 176 | total_timesteps 1686.
Path 177 | total_timesteps 1698.
Path 178 | total_timesteps 1707.
Path 179 | total_timesteps 1717.
Path 180 | total_timesteps 1725.
Path 181 | total_timesteps 1733.
Path 182 | total_timesteps 1744.
Path 183 | total_timesteps 1755.
Path 184 | total_timesteps 1764.
Path 185 | total_timesteps 1774.
Path 186 | total_timesteps 1786.
Path 187 | total_timesteps 1795.
Path 188 | total_timesteps 1808.
Path 189 | total_timesteps 1818.
Path 190 | total_timesteps 1829.
Path 191 | total_timesteps 1837.
Path 192 | total_timesteps 1849.
Path 193 | total_timesteps 1860.
Path 194 | total_timesteps 1867.
Path 195 | total_timesteps 1874.
Path 196 | total_timesteps 1882.
Path 197 | total_timesteps 1894.
Path 198 | total_timesteps 1905.
Path 199 | total_timesteps 1911.
Path 200 | total_timesteps 1918.
Path 201 | total_timesteps 1925.
Path 202 | total_timesteps 1936.
Path 203 | total_timesteps 1947.
Path 204 | total_timesteps 1955.
Path 205 | total_timesteps 1964.
Path 206 | total_timesteps 1980.
Path 207 | total_timesteps 1987.
Path 208 | total_timesteps 1999.
Path 209 | total_timesteps 2012.
Path 210 | total_timesteps 2021.
Path 211 | total_timesteps 2029.
Path 212 | total_timesteps 2038.
Path 213 | total_timesteps 2048.
Path 214 | total_timesteps 2057.
Path 215 | total_timesteps 2065.
Path 216 | total_timesteps 2075.
Path 217 | total_timesteps 2084.
Path 218 | total_timesteps 2093.
Path 219 | total_timesteps 2104.
Path 220 | total_timesteps 2111.
Path 221 | total_timesteps 2119.
Path 222 | total_timesteps 2130.
Path 223 | total_timesteps 2137.
Path 224 | total_timesteps 2148.
Path 225 | total_timesteps 2157.
Path 226 | total_timesteps 2166.
Path 227 | total_timesteps 2174.
Path 228 | total_timesteps 2185.
Path 229 | total_timesteps 2193.
Path 230 | total_timesteps 2202.
Path 231 | total_timesteps 2217.
Path 232 | total_timesteps 2226.
Path 233 | total_timesteps 2234.
Path 234 | total_timesteps 2242.
Path 235 | total_timesteps 2250.
Path 236 | total_timesteps 2258.
Path 237 | total_timesteps 2267.
Path 238 | total_timesteps 2279.
Path 239 | total_timesteps 2286.
Path 240 | total_timesteps 2299.
Path 241 | total_timesteps 2307.
Path 242 | total_timesteps 2317.
Path 243 | total_timesteps 2328.
Path 244 | total_timesteps 2338.
Path 245 | total_timesteps 2349.
Path 246 | total_timesteps 2357.
Path 247 | total_timesteps 2365.
Path 248 | total_timesteps 2376.
Path 249 | total_timesteps 2388.
Path 250 | total_timesteps 2397.
Path 251 | total_timesteps 2406.
Path 252 | total_timesteps 2415.
Path 253 | total_timesteps 2426.
Path 254 | total_timesteps 2437.
Path 255 | total_timesteps 2447.
Path 256 | total_timesteps 2458.
Path 257 | total_timesteps 2466.
Path 258 | total_timesteps 2478.
Path 259 | total_timesteps 2486.
Path 260 | total_timesteps 2493.
Path 261 | total_timesteps 2502.
Path 262 | total_timesteps 2511.
Path 263 | total_timesteps 2524.
Path 264 | total_timesteps 2531.
Path 265 | total_timesteps 2540.
Path 266 | total_timesteps 2548.
Path 267 | total_timesteps 2562.
Path 268 | total_timesteps 2575.
Path 269 | total_timesteps 2588.
Path 270 | total_timesteps 2601.
Path 271 | total_timesteps 2614.
Path 272 | total_timesteps 2626.
Path 273 | total_timesteps 2638.
Path 274 | total_timesteps 2645.
Path 275 | total_timesteps 2652.
Path 276 | total_timesteps 2660.
Path 277 | total_timesteps 2668.
Path 278 | total_timesteps 2677.
Path 279 | total_timesteps 2686.
Path 280 | total_timesteps 2695.
Path 281 | total_timesteps 2705.
Path 282 | total_timesteps 2714.
Path 283 | total_timesteps 2722.
Path 284 | total_timesteps 2731.
Path 285 | total_timesteps 2742.
Path 286 | total_timesteps 2749.
Path 287 | total_timesteps 2759.
Path 288 | total_timesteps 2768.
Path 289 | total_timesteps 2776.
Path 290 | total_timesteps 2786.
Path 291 | total_timesteps 2796.
Path 292 | total_timesteps 2807.
Path 293 | total_timesteps 2817.
Path 294 | total_timesteps 2824.
Path 295 | total_timesteps 2831.
Path 296 | total_timesteps 2838.
Path 297 | total_timesteps 2848.
Path 298 | total_timesteps 2860.
Path 299 | total_timesteps 2872.
Path 300 | total_timesteps 2880.
Path 301 | total_timesteps 2887.
Path 302 | total_timesteps 2894.
Path 303 | total_timesteps 2903.
Path 304 | total_timesteps 2911.
Path 305 | total_timesteps 2921.
Path 306 | total_timesteps 2928.
Path 307 | total_timesteps 2936.
Path 308 | total_timesteps 2944.
Path 309 | total_timesteps 2952.
Path 310 | total_timesteps 2962.
Path 311 | total_timesteps 2973.
Path 312 | total_timesteps 2981.
Path 313 | total_timesteps 2990.
Path 314 | total_timesteps 2999.
Path 315 | total_timesteps 3011.
Path 316 | total_timesteps 3020.
Path 317 | total_timesteps 3028.
Path 318 | total_timesteps 3036.
Path 319 | total_timesteps 3049.
Path 320 | total_timesteps 3057.
Path 321 | total_timesteps 3067.
Path 322 | total_timesteps 3078.
Path 323 | total_timesteps 3086.
Path 324 | total_timesteps 3094.
Path 325 | total_timesteps 3102.
Path 326 | total_timesteps 3109.
Path 327 | total_timesteps 3119.
Path 328 | total_timesteps 3132.
Path 329 | total_timesteps 3144.
Path 330 | total_timesteps 3152.
Path 331 | total_timesteps 3160.
Path 332 | total_timesteps 3169.
Path 333 | total_timesteps 3182.
Path 334 | total_timesteps 3195.
Path 335 | total_timesteps 3204.
Path 336 | total_timesteps 3211.
Path 337 | total_timesteps 3217.
Path 338 | total_timesteps 3228.
Path 339 | total_timesteps 3237.
Path 340 | total_timesteps 3245.
Path 341 | total_timesteps 3256.
Path 342 | total_timesteps 3264.
Path 343 | total_timesteps 3272.
Path 344 | total_timesteps 3281.
Path 345 | total_timesteps 3289.
Path 346 | total_timesteps 3299.
Path 347 | total_timesteps 3308.
Path 348 | total_timesteps 3317.
Path 349 | total_timesteps 3325.
Path 350 | total_timesteps 3335.
Path 351 | total_timesteps 3345.
Path 352 | total_timesteps 3355.
Path 353 | total_timesteps 3365.
Path 354 | total_timesteps 3374.
Path 355 | total_timesteps 3383.
Path 356 | total_timesteps 3394.
Path 357 | total_timesteps 3405.
Path 358 | total_timesteps 3416.
Path 359 | total_timesteps 3424.
Path 360 | total_timesteps 3431.
Path 361 | total_timesteps 3447.
Path 362 | total_timesteps 3455.
Path 363 | total_timesteps 3470.
Path 364 | total_timesteps 3478.
Path 365 | total_timesteps 3489.
Path 366 | total_timesteps 3497.
Path 367 | total_timesteps 3504.
Path 368 | total_timesteps 3516.
Path 369 | total_timesteps 3524.
Path 370 | total_timesteps 3531.
Path 371 | total_timesteps 3539.
Path 372 | total_timesteps 3547.
Path 373 | total_timesteps 3555.
Path 374 | total_timesteps 3563.
Path 375 | total_timesteps 3574.
Path 376 | total_timesteps 3587.
Path 377 | total_timesteps 3598.
Path 378 | total_timesteps 3605.
Path 379 | total_timesteps 3617.
Path 380 | total_timesteps 3630.
Path 381 | total_timesteps 3642.
Path 382 | total_timesteps 3656.
Path 383 | total_timesteps 3664.
Path 384 | total_timesteps 3679.
Path 385 | total_timesteps 3688.
Path 386 | total_timesteps 3700.
Path 387 | total_timesteps 3708.
Path 388 | total_timesteps 3716.
Path 389 | total_timesteps 3726.
Path 390 | total_timesteps 3735.
Path 391 | total_timesteps 3746.
Path 392 | total_timesteps 3755.
Path 393 | total_timesteps 3766.
Path 394 | total_timesteps 3775.
Path 395 | total_timesteps 3789.
Path 396 | total_timesteps 3797.
Path 397 | total_timesteps 3808.
Path 398 | total_timesteps 3818.
Path 399 | total_timesteps 3829.
Path 400 | total_timesteps 3839.
Path 401 | total_timesteps 3850.
Path 402 | total_timesteps 3860.
Path 403 | total_timesteps 3870.
Path 404 | total_timesteps 3881.
Path 405 | total_timesteps 3889.
Path 406 | total_timesteps 3897.
Path 407 | total_timesteps 3909.
Path 408 | total_timesteps 3920.
Path 409 | total_timesteps 3931.
Path 410 | total_timesteps 3950.
Path 411 | total_timesteps 3960.
Path 412 | total_timesteps 3968.
Path 413 | total_timesteps 3976.
Path 414 | total_timesteps 3984.
Path 415 | total_timesteps 3992.
Path 416 | total_timesteps 4000.
Path 417 | total_timesteps 4009.
Path 418 | total_timesteps 4019.
Path 419 | total_timesteps 4026.
Path 420 | total_timesteps 4038.
Path 421 | total_timesteps 4045.
Path 422 | total_timesteps 4054.
Path 423 | total_timesteps 4071.
Path 424 | total_timesteps 4078.
Path 425 | total_timesteps 4087.
Path 426 | total_timesteps 4093.
Path 427 | total_timesteps 4101.
Path 428 | total_timesteps 4109.
Path 429 | total_timesteps 4120.
Path 430 | total_timesteps 4131.
Path 431 | total_timesteps 4141.
Path 432 | total_timesteps 4149.
Path 433 | total_timesteps 4163.
Path 434 | total_timesteps 4171.
Path 435 | total_timesteps 4181.
Path 436 | total_timesteps 4187.
Path 437 | total_timesteps 4195.
Path 438 | total_timesteps 4206.
Path 439 | total_timesteps 4217.
Path 440 | total_timesteps 4224.
Path 441 | total_timesteps 4231.
Path 442 | total_timesteps 4241.
Path 443 | total_timesteps 4253.
Path 444 | total_timesteps 4263.
Path 445 | total_timesteps 4272.
Path 446 | total_timesteps 4286.
Path 447 | total_timesteps 4293.
Path 448 | total_timesteps 4303.
Path 449 | total_timesteps 4312.
Path 450 | total_timesteps 4323.
Path 451 | total_timesteps 4334.
Path 452 | total_timesteps 4344.
Path 453 | total_timesteps 4352.
Path 454 | total_timesteps 4360.
Path 455 | total_timesteps 4366.
Path 456 | total_timesteps 4375.
Path 457 | total_timesteps 4383.
Path 458 | total_timesteps 4390.
Path 459 | total_timesteps 4399.
Path 460 | total_timesteps 4412.
Path 461 | total_timesteps 4421.
Path 462 | total_timesteps 4428.
Path 463 | total_timesteps 4437.
Path 464 | total_timesteps 4448.
Path 465 | total_timesteps 4455.
Path 466 | total_timesteps 4467.
Path 467 | total_timesteps 4475.
Path 468 | total_timesteps 4482.
Path 469 | total_timesteps 4488.
Path 470 | total_timesteps 4499.
Path 471 | total_timesteps 4509.
Path 472 | total_timesteps 4521.
Path 473 | total_timesteps 4531.
Path 474 | total_timesteps 4540.
Path 475 | total_timesteps 4547.
Path 476 | total_timesteps 4557.
Path 477 | total_timesteps 4567.
Path 478 | total_timesteps 4580.
Path 479 | total_timesteps 4587.
Path 480 | total_timesteps 4596.
Path 481 | total_timesteps 4604.
Path 482 | total_timesteps 4610.
Path 483 | total_timesteps 4620.
Path 484 | total_timesteps 4629.
Path 485 | total_timesteps 4639.
Path 486 | total_timesteps 4647.
Path 487 | total_timesteps 4655.
Path 488 | total_timesteps 4663.
Path 489 | total_timesteps 4671.
Path 490 | total_timesteps 4680.
Path 491 | total_timesteps 4687.
Path 492 | total_timesteps 4702.
Path 493 | total_timesteps 4710.
Path 494 | total_timesteps 4718.
Path 495 | total_timesteps 4728.
Path 496 | total_timesteps 4735.
Path 497 | total_timesteps 4747.
Path 498 | total_timesteps 4756.
Path 499 | total_timesteps 4766.
Path 500 | total_timesteps 4774.
Path 501 | total_timesteps 4785.
Path 502 | total_timesteps 4793.
Path 503 | total_timesteps 4803.
Path 504 | total_timesteps 4815.
Path 505 | total_timesteps 4823.
Path 506 | total_timesteps 4835.
Path 507 | total_timesteps 4843.
Path 508 | total_timesteps 4850.
Path 509 | total_timesteps 4861.
Path 510 | total_timesteps 4870.
Path 511 | total_timesteps 4880.
Path 512 | total_timesteps 4892.
Path 513 | total_timesteps 4903.
Path 514 | total_timesteps 4915.
Path 515 | total_timesteps 4923.
Path 516 | total_timesteps 4933.
Path 517 | total_timesteps 4939.
Path 518 | total_timesteps 4953.
Path 519 | total_timesteps 4962.
Path 520 | total_timesteps 4970.
Path 521 | total_timesteps 4979.
Path 522 | total_timesteps 4986.
Path 523 | total_timesteps 4995.
Path 524 | total_timesteps 5003.
Path 525 | total_timesteps 5011.
Path 526 | total_timesteps 5019.
Path 527 | total_timesteps 5029.
Path 528 | total_timesteps 5037.
Path 529 | total_timesteps 5045.
Path 530 | total_timesteps 5058.
Path 531 | total_timesteps 5065.
Path 532 | total_timesteps 5078.
Path 533 | total_timesteps 5085.
Path 534 | total_timesteps 5092.
Path 535 | total_timesteps 5102.
Path 536 | total_timesteps 5111.
Path 537 | total_timesteps 5123.
Path 538 | total_timesteps 5130.
Path 539 | total_timesteps 5140.
Path 540 | total_timesteps 5149.
Path 541 | total_timesteps 5163.
Path 542 | total_timesteps 5174.
Path 543 | total_timesteps 5182.
Path 544 | total_timesteps 5191.
Path 545 | total_timesteps 5200.
Path 546 | total_timesteps 5209.
Path 547 | total_timesteps 5216.
Path 548 | total_timesteps 5224.
Path 549 | total_timesteps 5236.
Path 550 | total_timesteps 5244.
Path 551 | total_timesteps 5252.
Path 552 | total_timesteps 5266.
Path 553 | total_timesteps 5272.
Path 554 | total_timesteps 5282.
Path 555 | total_timesteps 5291.
Path 556 | total_timesteps 5299.
Path 557 | total_timesteps 5309.
Path 558 | total_timesteps 5318.
Path 559 | total_timesteps 5326.
Path 560 | total_timesteps 5332.
Path 561 | total_timesteps 5340.
Path 562 | total_timesteps 5347.
Path 563 | total_timesteps 5355.
Path 564 | total_timesteps 5366.
Path 565 | total_timesteps 5374.
Path 566 | total_timesteps 5383.
Path 567 | total_timesteps 5393.
Path 568 | total_timesteps 5404.
Path 569 | total_timesteps 5411.
Path 570 | total_timesteps 5418.
Path 571 | total_timesteps 5427.
Path 572 | total_timesteps 5434.
Path 573 | total_timesteps 5444.
Path 574 | total_timesteps 5453.
Path 575 | total_timesteps 5462.
Path 576 | total_timesteps 5473.
Path 577 | total_timesteps 5484.
Path 578 | total_timesteps 5493.
Path 579 | total_timesteps 5501.
Path 580 | total_timesteps 5508.
Path 581 | total_timesteps 5516.
Path 582 | total_timesteps 5525.
Path 583 | total_timesteps 5534.
Path 584 | total_timesteps 5541.
Path 585 | total_timesteps 5551.
Path 586 | total_timesteps 5559.
Path 587 | total_timesteps 5569.
Path 588 | total_timesteps 5577.
Path 589 | total_timesteps 5589.
Path 590 | total_timesteps 5599.
Path 591 | total_timesteps 5608.
Path 592 | total_timesteps 5622.
Path 593 | total_timesteps 5629.
Path 594 | total_timesteps 5637.
Path 595 | total_timesteps 5646.
Path 596 | total_timesteps 5655.
Path 597 | total_timesteps 5665.
Path 598 | total_timesteps 5672.
Path 599 | total_timesteps 5684.
Path 600 | total_timesteps 5693.
Path 601 | total_timesteps 5705.
Path 602 | total_timesteps 5712.
Path 603 | total_timesteps 5721.
Path 604 | total_timesteps 5737.
Path 605 | total_timesteps 5747.
Path 606 | total_timesteps 5756.
Path 607 | total_timesteps 5766.
Path 608 | total_timesteps 5775.
Path 609 | total_timesteps 5784.
Path 610 | total_timesteps 5792.
Path 611 | total_timesteps 5803.
Path 612 | total_timesteps 5811.
Path 613 | total_timesteps 5821.
Path 614 | total_timesteps 5829.
Path 615 | total_timesteps 5839.
Path 616 | total_timesteps 5848.
Path 617 | total_timesteps 5860.
Path 618 | total_timesteps 5870.
Path 619 | total_timesteps 5880.
Path 620 | total_timesteps 5890.
Path 621 | total_timesteps 5899.
Path 622 | total_timesteps 5906.
Path 623 | total_timesteps 5924.
Path 624 | total_timesteps 5932.
Path 625 | total_timesteps 5941.
Path 626 | total_timesteps 5950.
Path 627 | total_timesteps 5961.
Path 628 | total_timesteps 5971.
Path 629 | total_timesteps 5985.
Path 630 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.86    |
| Iteration     | 30       |
| MaximumReturn | -1.02    |
| MinimumReturn | -14.4    |
| TotalSamples  | 128149   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.001706126146018505
Validation loss = 0.0019296593964099884
Validation loss = 0.0015833370853215456
Validation loss = 0.0017695748247206211
Validation loss = 0.0017235127743333578
Validation loss = 0.0016994362231343985
Validation loss = 0.001979693304747343
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 47.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 77.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 94.
Path 11 | total_timesteps 106.
Path 12 | total_timesteps 117.
Path 13 | total_timesteps 125.
Path 14 | total_timesteps 140.
Path 15 | total_timesteps 150.
Path 16 | total_timesteps 161.
Path 17 | total_timesteps 173.
Path 18 | total_timesteps 184.
Path 19 | total_timesteps 192.
Path 20 | total_timesteps 201.
Path 21 | total_timesteps 211.
Path 22 | total_timesteps 221.
Path 23 | total_timesteps 228.
Path 24 | total_timesteps 237.
Path 25 | total_timesteps 248.
Path 26 | total_timesteps 257.
Path 27 | total_timesteps 265.
Path 28 | total_timesteps 275.
Path 29 | total_timesteps 285.
Path 30 | total_timesteps 297.
Path 31 | total_timesteps 305.
Path 32 | total_timesteps 314.
Path 33 | total_timesteps 324.
Path 34 | total_timesteps 332.
Path 35 | total_timesteps 342.
Path 36 | total_timesteps 353.
Path 37 | total_timesteps 362.
Path 38 | total_timesteps 371.
Path 39 | total_timesteps 384.
Path 40 | total_timesteps 394.
Path 41 | total_timesteps 401.
Path 42 | total_timesteps 409.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 423.
Path 45 | total_timesteps 429.
Path 46 | total_timesteps 436.
Path 47 | total_timesteps 452.
Path 48 | total_timesteps 466.
Path 49 | total_timesteps 476.
Path 50 | total_timesteps 484.
Path 51 | total_timesteps 494.
Path 52 | total_timesteps 504.
Path 53 | total_timesteps 514.
Path 54 | total_timesteps 522.
Path 55 | total_timesteps 533.
Path 56 | total_timesteps 541.
Path 57 | total_timesteps 549.
Path 58 | total_timesteps 564.
Path 59 | total_timesteps 571.
Path 60 | total_timesteps 578.
Path 61 | total_timesteps 586.
Path 62 | total_timesteps 595.
Path 63 | total_timesteps 603.
Path 64 | total_timesteps 611.
Path 65 | total_timesteps 618.
Path 66 | total_timesteps 625.
Path 67 | total_timesteps 633.
Path 68 | total_timesteps 640.
Path 69 | total_timesteps 649.
Path 70 | total_timesteps 660.
Path 71 | total_timesteps 667.
Path 72 | total_timesteps 674.
Path 73 | total_timesteps 681.
Path 74 | total_timesteps 690.
Path 75 | total_timesteps 701.
Path 76 | total_timesteps 711.
Path 77 | total_timesteps 721.
Path 78 | total_timesteps 728.
Path 79 | total_timesteps 735.
Path 80 | total_timesteps 745.
Path 81 | total_timesteps 756.
Path 82 | total_timesteps 765.
Path 83 | total_timesteps 774.
Path 84 | total_timesteps 781.
Path 85 | total_timesteps 788.
Path 86 | total_timesteps 798.
Path 87 | total_timesteps 809.
Path 88 | total_timesteps 820.
Path 89 | total_timesteps 830.
Path 90 | total_timesteps 844.
Path 91 | total_timesteps 855.
Path 92 | total_timesteps 865.
Path 93 | total_timesteps 874.
Path 94 | total_timesteps 889.
Path 95 | total_timesteps 900.
Path 96 | total_timesteps 909.
Path 97 | total_timesteps 918.
Path 98 | total_timesteps 927.
Path 99 | total_timesteps 937.
Path 100 | total_timesteps 946.
Path 101 | total_timesteps 954.
Path 102 | total_timesteps 961.
Path 103 | total_timesteps 972.
Path 104 | total_timesteps 985.
Path 105 | total_timesteps 995.
Path 106 | total_timesteps 1002.
Path 107 | total_timesteps 1018.
Path 108 | total_timesteps 1027.
Path 109 | total_timesteps 1036.
Path 110 | total_timesteps 1051.
Path 111 | total_timesteps 1058.
Path 112 | total_timesteps 1066.
Path 113 | total_timesteps 1073.
Path 114 | total_timesteps 1080.
Path 115 | total_timesteps 1086.
Path 116 | total_timesteps 1093.
Path 117 | total_timesteps 1102.
Path 118 | total_timesteps 1113.
Path 119 | total_timesteps 1122.
Path 120 | total_timesteps 1132.
Path 121 | total_timesteps 1142.
Path 122 | total_timesteps 1153.
Path 123 | total_timesteps 1159.
Path 124 | total_timesteps 1169.
Path 125 | total_timesteps 1178.
Path 126 | total_timesteps 1189.
Path 127 | total_timesteps 1201.
Path 128 | total_timesteps 1209.
Path 129 | total_timesteps 1218.
Path 130 | total_timesteps 1226.
Path 131 | total_timesteps 1234.
Path 132 | total_timesteps 1246.
Path 133 | total_timesteps 1255.
Path 134 | total_timesteps 1266.
Path 135 | total_timesteps 1276.
Path 136 | total_timesteps 1284.
Path 137 | total_timesteps 1293.
Path 138 | total_timesteps 1301.
Path 139 | total_timesteps 1310.
Path 140 | total_timesteps 1322.
Path 141 | total_timesteps 1335.
Path 142 | total_timesteps 1341.
Path 143 | total_timesteps 1352.
Path 144 | total_timesteps 1361.
Path 145 | total_timesteps 1369.
Path 146 | total_timesteps 1378.
Path 147 | total_timesteps 1386.
Path 148 | total_timesteps 1398.
Path 149 | total_timesteps 1410.
Path 150 | total_timesteps 1417.
Path 151 | total_timesteps 1424.
Path 152 | total_timesteps 1436.
Path 153 | total_timesteps 1444.
Path 154 | total_timesteps 1455.
Path 155 | total_timesteps 1465.
Path 156 | total_timesteps 1474.
Path 157 | total_timesteps 1484.
Path 158 | total_timesteps 1493.
Path 159 | total_timesteps 1499.
Path 160 | total_timesteps 1507.
Path 161 | total_timesteps 1517.
Path 162 | total_timesteps 1527.
Path 163 | total_timesteps 1538.
Path 164 | total_timesteps 1549.
Path 165 | total_timesteps 1557.
Path 166 | total_timesteps 1568.
Path 167 | total_timesteps 1579.
Path 168 | total_timesteps 1586.
Path 169 | total_timesteps 1598.
Path 170 | total_timesteps 1608.
Path 171 | total_timesteps 1616.
Path 172 | total_timesteps 1623.
Path 173 | total_timesteps 1636.
Path 174 | total_timesteps 1643.
Path 175 | total_timesteps 1650.
Path 176 | total_timesteps 1662.
Path 177 | total_timesteps 1669.
Path 178 | total_timesteps 1679.
Path 179 | total_timesteps 1686.
Path 180 | total_timesteps 1694.
Path 181 | total_timesteps 1704.
Path 182 | total_timesteps 1714.
Path 183 | total_timesteps 1725.
Path 184 | total_timesteps 1734.
Path 185 | total_timesteps 1747.
Path 186 | total_timesteps 1756.
Path 187 | total_timesteps 1765.
Path 188 | total_timesteps 1777.
Path 189 | total_timesteps 1788.
Path 190 | total_timesteps 1803.
Path 191 | total_timesteps 1811.
Path 192 | total_timesteps 1826.
Path 193 | total_timesteps 1838.
Path 194 | total_timesteps 1846.
Path 195 | total_timesteps 1854.
Path 196 | total_timesteps 1860.
Path 197 | total_timesteps 1870.
Path 198 | total_timesteps 1881.
Path 199 | total_timesteps 1890.
Path 200 | total_timesteps 1898.
Path 201 | total_timesteps 1909.
Path 202 | total_timesteps 1916.
Path 203 | total_timesteps 1926.
Path 204 | total_timesteps 1933.
Path 205 | total_timesteps 1947.
Path 206 | total_timesteps 1957.
Path 207 | total_timesteps 1966.
Path 208 | total_timesteps 1974.
Path 209 | total_timesteps 1987.
Path 210 | total_timesteps 1999.
Path 211 | total_timesteps 2010.
Path 212 | total_timesteps 2020.
Path 213 | total_timesteps 2032.
Path 214 | total_timesteps 2040.
Path 215 | total_timesteps 2047.
Path 216 | total_timesteps 2061.
Path 217 | total_timesteps 2069.
Path 218 | total_timesteps 2077.
Path 219 | total_timesteps 2086.
Path 220 | total_timesteps 2092.
Path 221 | total_timesteps 2100.
Path 222 | total_timesteps 2111.
Path 223 | total_timesteps 2120.
Path 224 | total_timesteps 2132.
Path 225 | total_timesteps 2139.
Path 226 | total_timesteps 2148.
Path 227 | total_timesteps 2156.
Path 228 | total_timesteps 2165.
Path 229 | total_timesteps 2174.
Path 230 | total_timesteps 2185.
Path 231 | total_timesteps 2194.
Path 232 | total_timesteps 2203.
Path 233 | total_timesteps 2211.
Path 234 | total_timesteps 2223.
Path 235 | total_timesteps 2232.
Path 236 | total_timesteps 2245.
Path 237 | total_timesteps 2253.
Path 238 | total_timesteps 2264.
Path 239 | total_timesteps 2274.
Path 240 | total_timesteps 2284.
Path 241 | total_timesteps 2291.
Path 242 | total_timesteps 2300.
Path 243 | total_timesteps 2309.
Path 244 | total_timesteps 2321.
Path 245 | total_timesteps 2333.
Path 246 | total_timesteps 2343.
Path 247 | total_timesteps 2351.
Path 248 | total_timesteps 2360.
Path 249 | total_timesteps 2367.
Path 250 | total_timesteps 2381.
Path 251 | total_timesteps 2391.
Path 252 | total_timesteps 2401.
Path 253 | total_timesteps 2408.
Path 254 | total_timesteps 2416.
Path 255 | total_timesteps 2424.
Path 256 | total_timesteps 2433.
Path 257 | total_timesteps 2442.
Path 258 | total_timesteps 2451.
Path 259 | total_timesteps 2464.
Path 260 | total_timesteps 2475.
Path 261 | total_timesteps 2483.
Path 262 | total_timesteps 2494.
Path 263 | total_timesteps 2502.
Path 264 | total_timesteps 2513.
Path 265 | total_timesteps 2525.
Path 266 | total_timesteps 2538.
Path 267 | total_timesteps 2546.
Path 268 | total_timesteps 2555.
Path 269 | total_timesteps 2564.
Path 270 | total_timesteps 2573.
Path 271 | total_timesteps 2580.
Path 272 | total_timesteps 2589.
Path 273 | total_timesteps 2598.
Path 274 | total_timesteps 2609.
Path 275 | total_timesteps 2617.
Path 276 | total_timesteps 2626.
Path 277 | total_timesteps 2639.
Path 278 | total_timesteps 2646.
Path 279 | total_timesteps 2657.
Path 280 | total_timesteps 2666.
Path 281 | total_timesteps 2674.
Path 282 | total_timesteps 2683.
Path 283 | total_timesteps 2693.
Path 284 | total_timesteps 2702.
Path 285 | total_timesteps 2711.
Path 286 | total_timesteps 2718.
Path 287 | total_timesteps 2727.
Path 288 | total_timesteps 2737.
Path 289 | total_timesteps 2750.
Path 290 | total_timesteps 2758.
Path 291 | total_timesteps 2767.
Path 292 | total_timesteps 2776.
Path 293 | total_timesteps 2788.
Path 294 | total_timesteps 2795.
Path 295 | total_timesteps 2803.
Path 296 | total_timesteps 2813.
Path 297 | total_timesteps 2824.
Path 298 | total_timesteps 2831.
Path 299 | total_timesteps 2839.
Path 300 | total_timesteps 2846.
Path 301 | total_timesteps 2853.
Path 302 | total_timesteps 2862.
Path 303 | total_timesteps 2869.
Path 304 | total_timesteps 2879.
Path 305 | total_timesteps 2888.
Path 306 | total_timesteps 2897.
Path 307 | total_timesteps 2905.
Path 308 | total_timesteps 2914.
Path 309 | total_timesteps 2924.
Path 310 | total_timesteps 2933.
Path 311 | total_timesteps 2946.
Path 312 | total_timesteps 2956.
Path 313 | total_timesteps 2967.
Path 314 | total_timesteps 2981.
Path 315 | total_timesteps 2990.
Path 316 | total_timesteps 2997.
Path 317 | total_timesteps 3006.
Path 318 | total_timesteps 3015.
Path 319 | total_timesteps 3023.
Path 320 | total_timesteps 3030.
Path 321 | total_timesteps 3043.
Path 322 | total_timesteps 3052.
Path 323 | total_timesteps 3061.
Path 324 | total_timesteps 3068.
Path 325 | total_timesteps 3078.
Path 326 | total_timesteps 3085.
Path 327 | total_timesteps 3094.
Path 328 | total_timesteps 3103.
Path 329 | total_timesteps 3115.
Path 330 | total_timesteps 3126.
Path 331 | total_timesteps 3137.
Path 332 | total_timesteps 3149.
Path 333 | total_timesteps 3162.
Path 334 | total_timesteps 3169.
Path 335 | total_timesteps 3176.
Path 336 | total_timesteps 3186.
Path 337 | total_timesteps 3199.
Path 338 | total_timesteps 3208.
Path 339 | total_timesteps 3219.
Path 340 | total_timesteps 3232.
Path 341 | total_timesteps 3242.
Path 342 | total_timesteps 3250.
Path 343 | total_timesteps 3259.
Path 344 | total_timesteps 3272.
Path 345 | total_timesteps 3280.
Path 346 | total_timesteps 3293.
Path 347 | total_timesteps 3300.
Path 348 | total_timesteps 3309.
Path 349 | total_timesteps 3317.
Path 350 | total_timesteps 3326.
Path 351 | total_timesteps 3335.
Path 352 | total_timesteps 3342.
Path 353 | total_timesteps 3352.
Path 354 | total_timesteps 3366.
Path 355 | total_timesteps 3377.
Path 356 | total_timesteps 3386.
Path 357 | total_timesteps 3393.
Path 358 | total_timesteps 3405.
Path 359 | total_timesteps 3412.
Path 360 | total_timesteps 3421.
Path 361 | total_timesteps 3431.
Path 362 | total_timesteps 3439.
Path 363 | total_timesteps 3449.
Path 364 | total_timesteps 3459.
Path 365 | total_timesteps 3472.
Path 366 | total_timesteps 3480.
Path 367 | total_timesteps 3491.
Path 368 | total_timesteps 3498.
Path 369 | total_timesteps 3508.
Path 370 | total_timesteps 3517.
Path 371 | total_timesteps 3533.
Path 372 | total_timesteps 3544.
Path 373 | total_timesteps 3552.
Path 374 | total_timesteps 3561.
Path 375 | total_timesteps 3568.
Path 376 | total_timesteps 3576.
Path 377 | total_timesteps 3588.
Path 378 | total_timesteps 3595.
Path 379 | total_timesteps 3607.
Path 380 | total_timesteps 3617.
Path 381 | total_timesteps 3629.
Path 382 | total_timesteps 3641.
Path 383 | total_timesteps 3650.
Path 384 | total_timesteps 3661.
Path 385 | total_timesteps 3669.
Path 386 | total_timesteps 3676.
Path 387 | total_timesteps 3687.
Path 388 | total_timesteps 3694.
Path 389 | total_timesteps 3702.
Path 390 | total_timesteps 3712.
Path 391 | total_timesteps 3721.
Path 392 | total_timesteps 3729.
Path 393 | total_timesteps 3738.
Path 394 | total_timesteps 3745.
Path 395 | total_timesteps 3753.
Path 396 | total_timesteps 3759.
Path 397 | total_timesteps 3768.
Path 398 | total_timesteps 3776.
Path 399 | total_timesteps 3785.
Path 400 | total_timesteps 3793.
Path 401 | total_timesteps 3802.
Path 402 | total_timesteps 3810.
Path 403 | total_timesteps 3818.
Path 404 | total_timesteps 3825.
Path 405 | total_timesteps 3835.
Path 406 | total_timesteps 3844.
Path 407 | total_timesteps 3852.
Path 408 | total_timesteps 3859.
Path 409 | total_timesteps 3865.
Path 410 | total_timesteps 3874.
Path 411 | total_timesteps 3881.
Path 412 | total_timesteps 3888.
Path 413 | total_timesteps 3894.
Path 414 | total_timesteps 3904.
Path 415 | total_timesteps 3912.
Path 416 | total_timesteps 3922.
Path 417 | total_timesteps 3930.
Path 418 | total_timesteps 3937.
Path 419 | total_timesteps 3950.
Path 420 | total_timesteps 3960.
Path 421 | total_timesteps 3971.
Path 422 | total_timesteps 3981.
Path 423 | total_timesteps 3988.
Path 424 | total_timesteps 3997.
Path 425 | total_timesteps 4013.
Path 426 | total_timesteps 4022.
Path 427 | total_timesteps 4034.
Path 428 | total_timesteps 4044.
Path 429 | total_timesteps 4052.
Path 430 | total_timesteps 4062.
Path 431 | total_timesteps 4073.
Path 432 | total_timesteps 4090.
Path 433 | total_timesteps 4101.
Path 434 | total_timesteps 4111.
Path 435 | total_timesteps 4117.
Path 436 | total_timesteps 4126.
Path 437 | total_timesteps 4135.
Path 438 | total_timesteps 4147.
Path 439 | total_timesteps 4157.
Path 440 | total_timesteps 4166.
Path 441 | total_timesteps 4175.
Path 442 | total_timesteps 4184.
Path 443 | total_timesteps 4197.
Path 444 | total_timesteps 4206.
Path 445 | total_timesteps 4218.
Path 446 | total_timesteps 4226.
Path 447 | total_timesteps 4234.
Path 448 | total_timesteps 4241.
Path 449 | total_timesteps 4248.
Path 450 | total_timesteps 4258.
Path 451 | total_timesteps 4267.
Path 452 | total_timesteps 4278.
Path 453 | total_timesteps 4288.
Path 454 | total_timesteps 4298.
Path 455 | total_timesteps 4304.
Path 456 | total_timesteps 4315.
Path 457 | total_timesteps 4332.
Path 458 | total_timesteps 4341.
Path 459 | total_timesteps 4350.
Path 460 | total_timesteps 4357.
Path 461 | total_timesteps 4364.
Path 462 | total_timesteps 4372.
Path 463 | total_timesteps 4384.
Path 464 | total_timesteps 4391.
Path 465 | total_timesteps 4401.
Path 466 | total_timesteps 4409.
Path 467 | total_timesteps 4420.
Path 468 | total_timesteps 4430.
Path 469 | total_timesteps 4438.
Path 470 | total_timesteps 4450.
Path 471 | total_timesteps 4461.
Path 472 | total_timesteps 4468.
Path 473 | total_timesteps 4477.
Path 474 | total_timesteps 4492.
Path 475 | total_timesteps 4501.
Path 476 | total_timesteps 4508.
Path 477 | total_timesteps 4522.
Path 478 | total_timesteps 4533.
Path 479 | total_timesteps 4545.
Path 480 | total_timesteps 4556.
Path 481 | total_timesteps 4565.
Path 482 | total_timesteps 4576.
Path 483 | total_timesteps 4587.
Path 484 | total_timesteps 4596.
Path 485 | total_timesteps 4604.
Path 486 | total_timesteps 4613.
Path 487 | total_timesteps 4621.
Path 488 | total_timesteps 4631.
Path 489 | total_timesteps 4640.
Path 490 | total_timesteps 4651.
Path 491 | total_timesteps 4660.
Path 492 | total_timesteps 4669.
Path 493 | total_timesteps 4677.
Path 494 | total_timesteps 4689.
Path 495 | total_timesteps 4703.
Path 496 | total_timesteps 4713.
Path 497 | total_timesteps 4721.
Path 498 | total_timesteps 4735.
Path 499 | total_timesteps 4743.
Path 500 | total_timesteps 4751.
Path 501 | total_timesteps 4758.
Path 502 | total_timesteps 4767.
Path 503 | total_timesteps 4776.
Path 504 | total_timesteps 4785.
Path 505 | total_timesteps 4795.
Path 506 | total_timesteps 4802.
Path 507 | total_timesteps 4815.
Path 508 | total_timesteps 4823.
Path 509 | total_timesteps 4833.
Path 510 | total_timesteps 4842.
Path 511 | total_timesteps 4855.
Path 512 | total_timesteps 4862.
Path 513 | total_timesteps 4871.
Path 514 | total_timesteps 4880.
Path 515 | total_timesteps 4888.
Path 516 | total_timesteps 4897.
Path 517 | total_timesteps 4906.
Path 518 | total_timesteps 4914.
Path 519 | total_timesteps 4923.
Path 520 | total_timesteps 4931.
Path 521 | total_timesteps 4942.
Path 522 | total_timesteps 4954.
Path 523 | total_timesteps 4964.
Path 524 | total_timesteps 4971.
Path 525 | total_timesteps 4981.
Path 526 | total_timesteps 4989.
Path 527 | total_timesteps 4999.
Path 528 | total_timesteps 5008.
Path 529 | total_timesteps 5018.
Path 530 | total_timesteps 5027.
Path 531 | total_timesteps 5035.
Path 532 | total_timesteps 5042.
Path 533 | total_timesteps 5049.
Path 534 | total_timesteps 5057.
Path 535 | total_timesteps 5067.
Path 536 | total_timesteps 5076.
Path 537 | total_timesteps 5086.
Path 538 | total_timesteps 5098.
Path 539 | total_timesteps 5105.
Path 540 | total_timesteps 5114.
Path 541 | total_timesteps 5124.
Path 542 | total_timesteps 5136.
Path 543 | total_timesteps 5148.
Path 544 | total_timesteps 5157.
Path 545 | total_timesteps 5163.
Path 546 | total_timesteps 5170.
Path 547 | total_timesteps 5185.
Path 548 | total_timesteps 5193.
Path 549 | total_timesteps 5200.
Path 550 | total_timesteps 5214.
Path 551 | total_timesteps 5221.
Path 552 | total_timesteps 5228.
Path 553 | total_timesteps 5236.
Path 554 | total_timesteps 5244.
Path 555 | total_timesteps 5255.
Path 556 | total_timesteps 5263.
Path 557 | total_timesteps 5271.
Path 558 | total_timesteps 5282.
Path 559 | total_timesteps 5293.
Path 560 | total_timesteps 5301.
Path 561 | total_timesteps 5317.
Path 562 | total_timesteps 5325.
Path 563 | total_timesteps 5335.
Path 564 | total_timesteps 5344.
Path 565 | total_timesteps 5359.
Path 566 | total_timesteps 5368.
Path 567 | total_timesteps 5377.
Path 568 | total_timesteps 5386.
Path 569 | total_timesteps 5392.
Path 570 | total_timesteps 5403.
Path 571 | total_timesteps 5418.
Path 572 | total_timesteps 5428.
Path 573 | total_timesteps 5439.
Path 574 | total_timesteps 5448.
Path 575 | total_timesteps 5455.
Path 576 | total_timesteps 5462.
Path 577 | total_timesteps 5471.
Path 578 | total_timesteps 5484.
Path 579 | total_timesteps 5491.
Path 580 | total_timesteps 5498.
Path 581 | total_timesteps 5508.
Path 582 | total_timesteps 5518.
Path 583 | total_timesteps 5530.
Path 584 | total_timesteps 5538.
Path 585 | total_timesteps 5546.
Path 586 | total_timesteps 5555.
Path 587 | total_timesteps 5567.
Path 588 | total_timesteps 5574.
Path 589 | total_timesteps 5582.
Path 590 | total_timesteps 5594.
Path 591 | total_timesteps 5601.
Path 592 | total_timesteps 5610.
Path 593 | total_timesteps 5619.
Path 594 | total_timesteps 5628.
Path 595 | total_timesteps 5638.
Path 596 | total_timesteps 5648.
Path 597 | total_timesteps 5655.
Path 598 | total_timesteps 5667.
Path 599 | total_timesteps 5675.
Path 600 | total_timesteps 5681.
Path 601 | total_timesteps 5690.
Path 602 | total_timesteps 5702.
Path 603 | total_timesteps 5709.
Path 604 | total_timesteps 5716.
Path 605 | total_timesteps 5722.
Path 606 | total_timesteps 5731.
Path 607 | total_timesteps 5738.
Path 608 | total_timesteps 5746.
Path 609 | total_timesteps 5755.
Path 610 | total_timesteps 5764.
Path 611 | total_timesteps 5774.
Path 612 | total_timesteps 5784.
Path 613 | total_timesteps 5792.
Path 614 | total_timesteps 5803.
Path 615 | total_timesteps 5816.
Path 616 | total_timesteps 5823.
Path 617 | total_timesteps 5834.
Path 618 | total_timesteps 5843.
Path 619 | total_timesteps 5850.
Path 620 | total_timesteps 5859.
Path 621 | total_timesteps 5866.
Path 622 | total_timesteps 5874.
Path 623 | total_timesteps 5886.
Path 624 | total_timesteps 5896.
Path 625 | total_timesteps 5905.
Path 626 | total_timesteps 5914.
Path 627 | total_timesteps 5928.
Path 628 | total_timesteps 5942.
Path 629 | total_timesteps 5950.
Path 630 | total_timesteps 5963.
Path 631 | total_timesteps 5971.
Path 632 | total_timesteps 5982.
Path 633 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.96    |
| Iteration     | 31       |
| MaximumReturn | -1.63    |
| MinimumReturn | -14.6    |
| TotalSamples  | 132153   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.0016964254900813103
Validation loss = 0.0017403215169906616
Validation loss = 0.001472079660743475
Validation loss = 0.001555492402985692
Validation loss = 0.0015980289317667484
Validation loss = 0.0015735127963125706
Validation loss = 0.0017087192973122
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 23.
Path 4 | total_timesteps 33.
Path 5 | total_timesteps 44.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 61.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 77.
Path 10 | total_timesteps 86.
Path 11 | total_timesteps 93.
Path 12 | total_timesteps 102.
Path 13 | total_timesteps 110.
Path 14 | total_timesteps 117.
Path 15 | total_timesteps 133.
Path 16 | total_timesteps 146.
Path 17 | total_timesteps 153.
Path 18 | total_timesteps 161.
Path 19 | total_timesteps 175.
Path 20 | total_timesteps 183.
Path 21 | total_timesteps 194.
Path 22 | total_timesteps 204.
Path 23 | total_timesteps 214.
Path 24 | total_timesteps 223.
Path 25 | total_timesteps 231.
Path 26 | total_timesteps 240.
Path 27 | total_timesteps 249.
Path 28 | total_timesteps 262.
Path 29 | total_timesteps 269.
Path 30 | total_timesteps 277.
Path 31 | total_timesteps 292.
Path 32 | total_timesteps 301.
Path 33 | total_timesteps 312.
Path 34 | total_timesteps 324.
Path 35 | total_timesteps 333.
Path 36 | total_timesteps 343.
Path 37 | total_timesteps 351.
Path 38 | total_timesteps 360.
Path 39 | total_timesteps 375.
Path 40 | total_timesteps 383.
Path 41 | total_timesteps 392.
Path 42 | total_timesteps 399.
Path 43 | total_timesteps 405.
Path 44 | total_timesteps 413.
Path 45 | total_timesteps 422.
Path 46 | total_timesteps 430.
Path 47 | total_timesteps 437.
Path 48 | total_timesteps 448.
Path 49 | total_timesteps 457.
Path 50 | total_timesteps 465.
Path 51 | total_timesteps 474.
Path 52 | total_timesteps 483.
Path 53 | total_timesteps 499.
Path 54 | total_timesteps 509.
Path 55 | total_timesteps 517.
Path 56 | total_timesteps 526.
Path 57 | total_timesteps 543.
Path 58 | total_timesteps 553.
Path 59 | total_timesteps 563.
Path 60 | total_timesteps 572.
Path 61 | total_timesteps 580.
Path 62 | total_timesteps 591.
Path 63 | total_timesteps 603.
Path 64 | total_timesteps 612.
Path 65 | total_timesteps 625.
Path 66 | total_timesteps 644.
Path 67 | total_timesteps 652.
Path 68 | total_timesteps 662.
Path 69 | total_timesteps 671.
Path 70 | total_timesteps 678.
Path 71 | total_timesteps 693.
Path 72 | total_timesteps 702.
Path 73 | total_timesteps 712.
Path 74 | total_timesteps 724.
Path 75 | total_timesteps 732.
Path 76 | total_timesteps 739.
Path 77 | total_timesteps 746.
Path 78 | total_timesteps 755.
Path 79 | total_timesteps 765.
Path 80 | total_timesteps 774.
Path 81 | total_timesteps 782.
Path 82 | total_timesteps 791.
Path 83 | total_timesteps 799.
Path 84 | total_timesteps 809.
Path 85 | total_timesteps 820.
Path 86 | total_timesteps 828.
Path 87 | total_timesteps 835.
Path 88 | total_timesteps 856.
Path 89 | total_timesteps 864.
Path 90 | total_timesteps 871.
Path 91 | total_timesteps 878.
Path 92 | total_timesteps 884.
Path 93 | total_timesteps 897.
Path 94 | total_timesteps 908.
Path 95 | total_timesteps 919.
Path 96 | total_timesteps 928.
Path 97 | total_timesteps 937.
Path 98 | total_timesteps 947.
Path 99 | total_timesteps 955.
Path 100 | total_timesteps 964.
Path 101 | total_timesteps 974.
Path 102 | total_timesteps 984.
Path 103 | total_timesteps 994.
Path 104 | total_timesteps 1007.
Path 105 | total_timesteps 1017.
Path 106 | total_timesteps 1027.
Path 107 | total_timesteps 1037.
Path 108 | total_timesteps 1050.
Path 109 | total_timesteps 1058.
Path 110 | total_timesteps 1067.
Path 111 | total_timesteps 1077.
Path 112 | total_timesteps 1087.
Path 113 | total_timesteps 1097.
Path 114 | total_timesteps 1106.
Path 115 | total_timesteps 1116.
Path 116 | total_timesteps 1124.
Path 117 | total_timesteps 1132.
Path 118 | total_timesteps 1142.
Path 119 | total_timesteps 1149.
Path 120 | total_timesteps 1157.
Path 121 | total_timesteps 1166.
Path 122 | total_timesteps 1179.
Path 123 | total_timesteps 1189.
Path 124 | total_timesteps 1196.
Path 125 | total_timesteps 1205.
Path 126 | total_timesteps 1212.
Path 127 | total_timesteps 1219.
Path 128 | total_timesteps 1228.
Path 129 | total_timesteps 1236.
Path 130 | total_timesteps 1246.
Path 131 | total_timesteps 1255.
Path 132 | total_timesteps 1262.
Path 133 | total_timesteps 1274.
Path 134 | total_timesteps 1280.
Path 135 | total_timesteps 1288.
Path 136 | total_timesteps 1295.
Path 137 | total_timesteps 1304.
Path 138 | total_timesteps 1316.
Path 139 | total_timesteps 1323.
Path 140 | total_timesteps 1337.
Path 141 | total_timesteps 1344.
Path 142 | total_timesteps 1353.
Path 143 | total_timesteps 1363.
Path 144 | total_timesteps 1376.
Path 145 | total_timesteps 1383.
Path 146 | total_timesteps 1394.
Path 147 | total_timesteps 1402.
Path 148 | total_timesteps 1409.
Path 149 | total_timesteps 1419.
Path 150 | total_timesteps 1429.
Path 151 | total_timesteps 1439.
Path 152 | total_timesteps 1447.
Path 153 | total_timesteps 1455.
Path 154 | total_timesteps 1464.
Path 155 | total_timesteps 1473.
Path 156 | total_timesteps 1485.
Path 157 | total_timesteps 1493.
Path 158 | total_timesteps 1504.
Path 159 | total_timesteps 1513.
Path 160 | total_timesteps 1520.
Path 161 | total_timesteps 1527.
Path 162 | total_timesteps 1538.
Path 163 | total_timesteps 1548.
Path 164 | total_timesteps 1557.
Path 165 | total_timesteps 1566.
Path 166 | total_timesteps 1574.
Path 167 | total_timesteps 1583.
Path 168 | total_timesteps 1591.
Path 169 | total_timesteps 1601.
Path 170 | total_timesteps 1610.
Path 171 | total_timesteps 1621.
Path 172 | total_timesteps 1633.
Path 173 | total_timesteps 1642.
Path 174 | total_timesteps 1651.
Path 175 | total_timesteps 1659.
Path 176 | total_timesteps 1671.
Path 177 | total_timesteps 1677.
Path 178 | total_timesteps 1687.
Path 179 | total_timesteps 1699.
Path 180 | total_timesteps 1707.
Path 181 | total_timesteps 1719.
Path 182 | total_timesteps 1726.
Path 183 | total_timesteps 1736.
Path 184 | total_timesteps 1744.
Path 185 | total_timesteps 1751.
Path 186 | total_timesteps 1758.
Path 187 | total_timesteps 1766.
Path 188 | total_timesteps 1777.
Path 189 | total_timesteps 1787.
Path 190 | total_timesteps 1795.
Path 191 | total_timesteps 1802.
Path 192 | total_timesteps 1814.
Path 193 | total_timesteps 1822.
Path 194 | total_timesteps 1831.
Path 195 | total_timesteps 1839.
Path 196 | total_timesteps 1856.
Path 197 | total_timesteps 1867.
Path 198 | total_timesteps 1876.
Path 199 | total_timesteps 1882.
Path 200 | total_timesteps 1892.
Path 201 | total_timesteps 1899.
Path 202 | total_timesteps 1907.
Path 203 | total_timesteps 1919.
Path 204 | total_timesteps 1927.
Path 205 | total_timesteps 1937.
Path 206 | total_timesteps 1947.
Path 207 | total_timesteps 1957.
Path 208 | total_timesteps 1965.
Path 209 | total_timesteps 1975.
Path 210 | total_timesteps 1985.
Path 211 | total_timesteps 1993.
Path 212 | total_timesteps 1999.
Path 213 | total_timesteps 2008.
Path 214 | total_timesteps 2017.
Path 215 | total_timesteps 2029.
Path 216 | total_timesteps 2038.
Path 217 | total_timesteps 2045.
Path 218 | total_timesteps 2052.
Path 219 | total_timesteps 2062.
Path 220 | total_timesteps 2074.
Path 221 | total_timesteps 2082.
Path 222 | total_timesteps 2094.
Path 223 | total_timesteps 2105.
Path 224 | total_timesteps 2115.
Path 225 | total_timesteps 2126.
Path 226 | total_timesteps 2134.
Path 227 | total_timesteps 2143.
Path 228 | total_timesteps 2153.
Path 229 | total_timesteps 2161.
Path 230 | total_timesteps 2168.
Path 231 | total_timesteps 2176.
Path 232 | total_timesteps 2187.
Path 233 | total_timesteps 2195.
Path 234 | total_timesteps 2204.
Path 235 | total_timesteps 2214.
Path 236 | total_timesteps 2223.
Path 237 | total_timesteps 2232.
Path 238 | total_timesteps 2239.
Path 239 | total_timesteps 2250.
Path 240 | total_timesteps 2259.
Path 241 | total_timesteps 2271.
Path 242 | total_timesteps 2278.
Path 243 | total_timesteps 2290.
Path 244 | total_timesteps 2316.
Path 245 | total_timesteps 2326.
Path 246 | total_timesteps 2333.
Path 247 | total_timesteps 2345.
Path 248 | total_timesteps 2355.
Path 249 | total_timesteps 2363.
Path 250 | total_timesteps 2371.
Path 251 | total_timesteps 2379.
Path 252 | total_timesteps 2387.
Path 253 | total_timesteps 2397.
Path 254 | total_timesteps 2408.
Path 255 | total_timesteps 2416.
Path 256 | total_timesteps 2424.
Path 257 | total_timesteps 2432.
Path 258 | total_timesteps 2443.
Path 259 | total_timesteps 2452.
Path 260 | total_timesteps 2460.
Path 261 | total_timesteps 2471.
Path 262 | total_timesteps 2478.
Path 263 | total_timesteps 2485.
Path 264 | total_timesteps 2500.
Path 265 | total_timesteps 2506.
Path 266 | total_timesteps 2514.
Path 267 | total_timesteps 2522.
Path 268 | total_timesteps 2530.
Path 269 | total_timesteps 2539.
Path 270 | total_timesteps 2546.
Path 271 | total_timesteps 2555.
Path 272 | total_timesteps 2564.
Path 273 | total_timesteps 2582.
Path 274 | total_timesteps 2590.
Path 275 | total_timesteps 2600.
Path 276 | total_timesteps 2606.
Path 277 | total_timesteps 2619.
Path 278 | total_timesteps 2628.
Path 279 | total_timesteps 2639.
Path 280 | total_timesteps 2650.
Path 281 | total_timesteps 2670.
Path 282 | total_timesteps 2681.
Path 283 | total_timesteps 2694.
Path 284 | total_timesteps 2704.
Path 285 | total_timesteps 2711.
Path 286 | total_timesteps 2722.
Path 287 | total_timesteps 2736.
Path 288 | total_timesteps 2748.
Path 289 | total_timesteps 2755.
Path 290 | total_timesteps 2763.
Path 291 | total_timesteps 2769.
Path 292 | total_timesteps 2778.
Path 293 | total_timesteps 2786.
Path 294 | total_timesteps 2796.
Path 295 | total_timesteps 2803.
Path 296 | total_timesteps 2814.
Path 297 | total_timesteps 2823.
Path 298 | total_timesteps 2830.
Path 299 | total_timesteps 2840.
Path 300 | total_timesteps 2847.
Path 301 | total_timesteps 2859.
Path 302 | total_timesteps 2868.
Path 303 | total_timesteps 2884.
Path 304 | total_timesteps 2895.
Path 305 | total_timesteps 2902.
Path 306 | total_timesteps 2911.
Path 307 | total_timesteps 2918.
Path 308 | total_timesteps 2926.
Path 309 | total_timesteps 2937.
Path 310 | total_timesteps 2943.
Path 311 | total_timesteps 2957.
Path 312 | total_timesteps 2968.
Path 313 | total_timesteps 2976.
Path 314 | total_timesteps 2985.
Path 315 | total_timesteps 2996.
Path 316 | total_timesteps 3005.
Path 317 | total_timesteps 3015.
Path 318 | total_timesteps 3022.
Path 319 | total_timesteps 3029.
Path 320 | total_timesteps 3041.
Path 321 | total_timesteps 3048.
Path 322 | total_timesteps 3056.
Path 323 | total_timesteps 3064.
Path 324 | total_timesteps 3074.
Path 325 | total_timesteps 3082.
Path 326 | total_timesteps 3091.
Path 327 | total_timesteps 3100.
Path 328 | total_timesteps 3109.
Path 329 | total_timesteps 3117.
Path 330 | total_timesteps 3126.
Path 331 | total_timesteps 3135.
Path 332 | total_timesteps 3146.
Path 333 | total_timesteps 3156.
Path 334 | total_timesteps 3165.
Path 335 | total_timesteps 3174.
Path 336 | total_timesteps 3182.
Path 337 | total_timesteps 3189.
Path 338 | total_timesteps 3198.
Path 339 | total_timesteps 3208.
Path 340 | total_timesteps 3217.
Path 341 | total_timesteps 3225.
Path 342 | total_timesteps 3235.
Path 343 | total_timesteps 3246.
Path 344 | total_timesteps 3252.
Path 345 | total_timesteps 3261.
Path 346 | total_timesteps 3269.
Path 347 | total_timesteps 3276.
Path 348 | total_timesteps 3285.
Path 349 | total_timesteps 3293.
Path 350 | total_timesteps 3306.
Path 351 | total_timesteps 3315.
Path 352 | total_timesteps 3323.
Path 353 | total_timesteps 3338.
Path 354 | total_timesteps 3345.
Path 355 | total_timesteps 3355.
Path 356 | total_timesteps 3363.
Path 357 | total_timesteps 3374.
Path 358 | total_timesteps 3384.
Path 359 | total_timesteps 3393.
Path 360 | total_timesteps 3402.
Path 361 | total_timesteps 3412.
Path 362 | total_timesteps 3421.
Path 363 | total_timesteps 3429.
Path 364 | total_timesteps 3441.
Path 365 | total_timesteps 3448.
Path 366 | total_timesteps 3458.
Path 367 | total_timesteps 3469.
Path 368 | total_timesteps 3481.
Path 369 | total_timesteps 3488.
Path 370 | total_timesteps 3498.
Path 371 | total_timesteps 3506.
Path 372 | total_timesteps 3516.
Path 373 | total_timesteps 3526.
Path 374 | total_timesteps 3534.
Path 375 | total_timesteps 3542.
Path 376 | total_timesteps 3549.
Path 377 | total_timesteps 3556.
Path 378 | total_timesteps 3566.
Path 379 | total_timesteps 3578.
Path 380 | total_timesteps 3587.
Path 381 | total_timesteps 3598.
Path 382 | total_timesteps 3607.
Path 383 | total_timesteps 3617.
Path 384 | total_timesteps 3626.
Path 385 | total_timesteps 3634.
Path 386 | total_timesteps 3642.
Path 387 | total_timesteps 3652.
Path 388 | total_timesteps 3662.
Path 389 | total_timesteps 3670.
Path 390 | total_timesteps 3676.
Path 391 | total_timesteps 3684.
Path 392 | total_timesteps 3694.
Path 393 | total_timesteps 3704.
Path 394 | total_timesteps 3712.
Path 395 | total_timesteps 3723.
Path 396 | total_timesteps 3731.
Path 397 | total_timesteps 3745.
Path 398 | total_timesteps 3754.
Path 399 | total_timesteps 3762.
Path 400 | total_timesteps 3770.
Path 401 | total_timesteps 3780.
Path 402 | total_timesteps 3790.
Path 403 | total_timesteps 3801.
Path 404 | total_timesteps 3808.
Path 405 | total_timesteps 3820.
Path 406 | total_timesteps 3829.
Path 407 | total_timesteps 3840.
Path 408 | total_timesteps 3850.
Path 409 | total_timesteps 3857.
Path 410 | total_timesteps 3865.
Path 411 | total_timesteps 3877.
Path 412 | total_timesteps 3890.
Path 413 | total_timesteps 3901.
Path 414 | total_timesteps 3908.
Path 415 | total_timesteps 3917.
Path 416 | total_timesteps 3930.
Path 417 | total_timesteps 3936.
Path 418 | total_timesteps 3944.
Path 419 | total_timesteps 3953.
Path 420 | total_timesteps 3961.
Path 421 | total_timesteps 3971.
Path 422 | total_timesteps 3983.
Path 423 | total_timesteps 3992.
Path 424 | total_timesteps 4004.
Path 425 | total_timesteps 4015.
Path 426 | total_timesteps 4024.
Path 427 | total_timesteps 4031.
Path 428 | total_timesteps 4042.
Path 429 | total_timesteps 4050.
Path 430 | total_timesteps 4058.
Path 431 | total_timesteps 4065.
Path 432 | total_timesteps 4077.
Path 433 | total_timesteps 4087.
Path 434 | total_timesteps 4095.
Path 435 | total_timesteps 4103.
Path 436 | total_timesteps 4113.
Path 437 | total_timesteps 4121.
Path 438 | total_timesteps 4133.
Path 439 | total_timesteps 4142.
Path 440 | total_timesteps 4153.
Path 441 | total_timesteps 4161.
Path 442 | total_timesteps 4171.
Path 443 | total_timesteps 4181.
Path 444 | total_timesteps 4188.
Path 445 | total_timesteps 4199.
Path 446 | total_timesteps 4208.
Path 447 | total_timesteps 4216.
Path 448 | total_timesteps 4233.
Path 449 | total_timesteps 4240.
Path 450 | total_timesteps 4249.
Path 451 | total_timesteps 4257.
Path 452 | total_timesteps 4265.
Path 453 | total_timesteps 4272.
Path 454 | total_timesteps 4280.
Path 455 | total_timesteps 4288.
Path 456 | total_timesteps 4297.
Path 457 | total_timesteps 4307.
Path 458 | total_timesteps 4318.
Path 459 | total_timesteps 4330.
Path 460 | total_timesteps 4336.
Path 461 | total_timesteps 4346.
Path 462 | total_timesteps 4355.
Path 463 | total_timesteps 4363.
Path 464 | total_timesteps 4374.
Path 465 | total_timesteps 4382.
Path 466 | total_timesteps 4391.
Path 467 | total_timesteps 4403.
Path 468 | total_timesteps 4412.
Path 469 | total_timesteps 4423.
Path 470 | total_timesteps 4433.
Path 471 | total_timesteps 4446.
Path 472 | total_timesteps 4454.
Path 473 | total_timesteps 4463.
Path 474 | total_timesteps 4471.
Path 475 | total_timesteps 4483.
Path 476 | total_timesteps 4496.
Path 477 | total_timesteps 4503.
Path 478 | total_timesteps 4510.
Path 479 | total_timesteps 4517.
Path 480 | total_timesteps 4529.
Path 481 | total_timesteps 4537.
Path 482 | total_timesteps 4548.
Path 483 | total_timesteps 4559.
Path 484 | total_timesteps 4567.
Path 485 | total_timesteps 4575.
Path 486 | total_timesteps 4586.
Path 487 | total_timesteps 4597.
Path 488 | total_timesteps 4607.
Path 489 | total_timesteps 4619.
Path 490 | total_timesteps 4630.
Path 491 | total_timesteps 4640.
Path 492 | total_timesteps 4651.
Path 493 | total_timesteps 4665.
Path 494 | total_timesteps 4675.
Path 495 | total_timesteps 4682.
Path 496 | total_timesteps 4691.
Path 497 | total_timesteps 4702.
Path 498 | total_timesteps 4709.
Path 499 | total_timesteps 4717.
Path 500 | total_timesteps 4724.
Path 501 | total_timesteps 4731.
Path 502 | total_timesteps 4741.
Path 503 | total_timesteps 4751.
Path 504 | total_timesteps 4758.
Path 505 | total_timesteps 4768.
Path 506 | total_timesteps 4779.
Path 507 | total_timesteps 4786.
Path 508 | total_timesteps 4806.
Path 509 | total_timesteps 4814.
Path 510 | total_timesteps 4822.
Path 511 | total_timesteps 4830.
Path 512 | total_timesteps 4840.
Path 513 | total_timesteps 4856.
Path 514 | total_timesteps 4863.
Path 515 | total_timesteps 4873.
Path 516 | total_timesteps 4883.
Path 517 | total_timesteps 4895.
Path 518 | total_timesteps 4904.
Path 519 | total_timesteps 4914.
Path 520 | total_timesteps 4922.
Path 521 | total_timesteps 4931.
Path 522 | total_timesteps 4937.
Path 523 | total_timesteps 4948.
Path 524 | total_timesteps 4959.
Path 525 | total_timesteps 4972.
Path 526 | total_timesteps 4982.
Path 527 | total_timesteps 4994.
Path 528 | total_timesteps 5002.
Path 529 | total_timesteps 5011.
Path 530 | total_timesteps 5023.
Path 531 | total_timesteps 5034.
Path 532 | total_timesteps 5053.
Path 533 | total_timesteps 5063.
Path 534 | total_timesteps 5072.
Path 535 | total_timesteps 5081.
Path 536 | total_timesteps 5089.
Path 537 | total_timesteps 5100.
Path 538 | total_timesteps 5107.
Path 539 | total_timesteps 5120.
Path 540 | total_timesteps 5127.
Path 541 | total_timesteps 5136.
Path 542 | total_timesteps 5143.
Path 543 | total_timesteps 5152.
Path 544 | total_timesteps 5159.
Path 545 | total_timesteps 5171.
Path 546 | total_timesteps 5179.
Path 547 | total_timesteps 5189.
Path 548 | total_timesteps 5198.
Path 549 | total_timesteps 5205.
Path 550 | total_timesteps 5215.
Path 551 | total_timesteps 5222.
Path 552 | total_timesteps 5231.
Path 553 | total_timesteps 5240.
Path 554 | total_timesteps 5251.
Path 555 | total_timesteps 5261.
Path 556 | total_timesteps 5270.
Path 557 | total_timesteps 5279.
Path 558 | total_timesteps 5286.
Path 559 | total_timesteps 5295.
Path 560 | total_timesteps 5305.
Path 561 | total_timesteps 5318.
Path 562 | total_timesteps 5329.
Path 563 | total_timesteps 5339.
Path 564 | total_timesteps 5348.
Path 565 | total_timesteps 5356.
Path 566 | total_timesteps 5368.
Path 567 | total_timesteps 5376.
Path 568 | total_timesteps 5386.
Path 569 | total_timesteps 5395.
Path 570 | total_timesteps 5404.
Path 571 | total_timesteps 5412.
Path 572 | total_timesteps 5423.
Path 573 | total_timesteps 5431.
Path 574 | total_timesteps 5439.
Path 575 | total_timesteps 5448.
Path 576 | total_timesteps 5456.
Path 577 | total_timesteps 5466.
Path 578 | total_timesteps 5474.
Path 579 | total_timesteps 5485.
Path 580 | total_timesteps 5493.
Path 581 | total_timesteps 5501.
Path 582 | total_timesteps 5511.
Path 583 | total_timesteps 5522.
Path 584 | total_timesteps 5530.
Path 585 | total_timesteps 5540.
Path 586 | total_timesteps 5548.
Path 587 | total_timesteps 5555.
Path 588 | total_timesteps 5564.
Path 589 | total_timesteps 5574.
Path 590 | total_timesteps 5588.
Path 591 | total_timesteps 5599.
Path 592 | total_timesteps 5606.
Path 593 | total_timesteps 5614.
Path 594 | total_timesteps 5623.
Path 595 | total_timesteps 5632.
Path 596 | total_timesteps 5645.
Path 597 | total_timesteps 5655.
Path 598 | total_timesteps 5666.
Path 599 | total_timesteps 5675.
Path 600 | total_timesteps 5684.
Path 601 | total_timesteps 5695.
Path 602 | total_timesteps 5704.
Path 603 | total_timesteps 5712.
Path 604 | total_timesteps 5721.
Path 605 | total_timesteps 5730.
Path 606 | total_timesteps 5741.
Path 607 | total_timesteps 5747.
Path 608 | total_timesteps 5759.
Path 609 | total_timesteps 5772.
Path 610 | total_timesteps 5779.
Path 611 | total_timesteps 5790.
Path 612 | total_timesteps 5797.
Path 613 | total_timesteps 5805.
Path 614 | total_timesteps 5818.
Path 615 | total_timesteps 5828.
Path 616 | total_timesteps 5835.
Path 617 | total_timesteps 5850.
Path 618 | total_timesteps 5858.
Path 619 | total_timesteps 5866.
Path 620 | total_timesteps 5875.
Path 621 | total_timesteps 5883.
Path 622 | total_timesteps 5894.
Path 623 | total_timesteps 5909.
Path 624 | total_timesteps 5918.
Path 625 | total_timesteps 5928.
Path 626 | total_timesteps 5936.
Path 627 | total_timesteps 5947.
Path 628 | total_timesteps 5955.
Path 629 | total_timesteps 5963.
Path 630 | total_timesteps 5977.
Path 631 | total_timesteps 5986.
Path 632 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.99    |
| Iteration     | 32       |
| MaximumReturn | 2.06     |
| MinimumReturn | -15.5    |
| TotalSamples  | 136155   |
----------------------------
