Logging to experiments/gym_fwalker2d/W/Mon-07-Nov-2022-10-28-41-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2231
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 33.
Path 3 | total_timesteps 47.
Path 4 | total_timesteps 67.
Path 5 | total_timesteps 89.
Path 6 | total_timesteps 106.
Path 7 | total_timesteps 121.
Path 8 | total_timesteps 133.
Path 9 | total_timesteps 149.
Path 10 | total_timesteps 165.
Path 11 | total_timesteps 184.
Path 12 | total_timesteps 195.
Path 13 | total_timesteps 211.
Path 14 | total_timesteps 235.
Path 15 | total_timesteps 260.
Path 16 | total_timesteps 290.
Path 17 | total_timesteps 313.
Path 18 | total_timesteps 327.
Path 19 | total_timesteps 348.
Path 20 | total_timesteps 391.
Path 21 | total_timesteps 415.
Path 22 | total_timesteps 426.
Path 23 | total_timesteps 448.
Path 24 | total_timesteps 485.
Path 25 | total_timesteps 513.
Path 26 | total_timesteps 554.
Path 27 | total_timesteps 568.
Path 28 | total_timesteps 587.
Path 29 | total_timesteps 605.
Path 30 | total_timesteps 618.
Path 31 | total_timesteps 631.
Path 32 | total_timesteps 663.
Path 33 | total_timesteps 699.
Path 34 | total_timesteps 721.
Path 35 | total_timesteps 743.
Path 36 | total_timesteps 776.
Path 37 | total_timesteps 798.
Path 38 | total_timesteps 820.
Path 39 | total_timesteps 835.
Path 40 | total_timesteps 850.
Path 41 | total_timesteps 872.
Path 42 | total_timesteps 890.
Path 43 | total_timesteps 915.
Path 44 | total_timesteps 929.
Path 45 | total_timesteps 942.
Path 46 | total_timesteps 954.
Path 47 | total_timesteps 980.
Path 48 | total_timesteps 1003.
Path 49 | total_timesteps 1023.
Path 50 | total_timesteps 1040.
Path 51 | total_timesteps 1063.
Path 52 | total_timesteps 1076.
Path 53 | total_timesteps 1088.
Path 54 | total_timesteps 1126.
Path 55 | total_timesteps 1138.
Path 56 | total_timesteps 1180.
Path 57 | total_timesteps 1244.
Path 58 | total_timesteps 1255.
Path 59 | total_timesteps 1287.
Path 60 | total_timesteps 1308.
Path 61 | total_timesteps 1322.
Path 62 | total_timesteps 1345.
Path 63 | total_timesteps 1382.
Path 64 | total_timesteps 1397.
Path 65 | total_timesteps 1409.
Path 66 | total_timesteps 1423.
Path 67 | total_timesteps 1438.
Path 68 | total_timesteps 1456.
Path 69 | total_timesteps 1499.
Path 70 | total_timesteps 1518.
Path 71 | total_timesteps 1540.
Path 72 | total_timesteps 1549.
Path 73 | total_timesteps 1561.
Path 74 | total_timesteps 1618.
Path 75 | total_timesteps 1639.
Path 76 | total_timesteps 1664.
Path 77 | total_timesteps 1681.
Path 78 | total_timesteps 1691.
Path 79 | total_timesteps 1716.
Path 80 | total_timesteps 1729.
Path 81 | total_timesteps 1758.
Path 82 | total_timesteps 1783.
Path 83 | total_timesteps 1796.
Path 84 | total_timesteps 1819.
Path 85 | total_timesteps 1848.
Path 86 | total_timesteps 1864.
Path 87 | total_timesteps 1877.
Path 88 | total_timesteps 1917.
Path 89 | total_timesteps 1927.
Path 90 | total_timesteps 1946.
Path 91 | total_timesteps 1968.
Path 92 | total_timesteps 1992.
Path 93 | total_timesteps 2024.
Path 94 | total_timesteps 2047.
Path 95 | total_timesteps 2066.
Path 96 | total_timesteps 2087.
Path 97 | total_timesteps 2116.
Path 98 | total_timesteps 2125.
Path 99 | total_timesteps 2141.
Path 100 | total_timesteps 2151.
Path 101 | total_timesteps 2161.
Path 102 | total_timesteps 2186.
Path 103 | total_timesteps 2211.
Path 104 | total_timesteps 2235.
Path 105 | total_timesteps 2251.
Path 106 | total_timesteps 2269.
Path 107 | total_timesteps 2306.
Path 108 | total_timesteps 2334.
Path 109 | total_timesteps 2354.
Path 110 | total_timesteps 2399.
Path 111 | total_timesteps 2420.
Path 112 | total_timesteps 2451.
Path 113 | total_timesteps 2468.
Path 114 | total_timesteps 2486.
Path 115 | total_timesteps 2501.
Path 116 | total_timesteps 2536.
Path 117 | total_timesteps 2549.
Path 118 | total_timesteps 2567.
Path 119 | total_timesteps 2583.
Path 120 | total_timesteps 2610.
Path 121 | total_timesteps 2625.
Path 122 | total_timesteps 2640.
Path 123 | total_timesteps 2662.
Path 124 | total_timesteps 2678.
Path 125 | total_timesteps 2689.
Path 126 | total_timesteps 2712.
Path 127 | total_timesteps 2739.
Path 128 | total_timesteps 2763.
Path 129 | total_timesteps 2778.
Path 130 | total_timesteps 2789.
Path 131 | total_timesteps 2816.
Path 132 | total_timesteps 2826.
Path 133 | total_timesteps 2844.
Path 134 | total_timesteps 2857.
Path 135 | total_timesteps 2864.
Path 136 | total_timesteps 2897.
Path 137 | total_timesteps 2921.
Path 138 | total_timesteps 2957.
Path 139 | total_timesteps 2983.
Path 140 | total_timesteps 2993.
Path 141 | total_timesteps 3010.
Path 142 | total_timesteps 3020.
Path 143 | total_timesteps 3029.
Path 144 | total_timesteps 3050.
Path 145 | total_timesteps 3116.
Path 146 | total_timesteps 3141.
Path 147 | total_timesteps 3158.
Path 148 | total_timesteps 3182.
Path 149 | total_timesteps 3199.
Path 150 | total_timesteps 3211.
Path 151 | total_timesteps 3240.
Path 152 | total_timesteps 3254.
Path 153 | total_timesteps 3281.
Path 154 | total_timesteps 3298.
Path 155 | total_timesteps 3315.
Path 156 | total_timesteps 3336.
Path 157 | total_timesteps 3347.
Path 158 | total_timesteps 3379.
Path 159 | total_timesteps 3419.
Path 160 | total_timesteps 3438.
Path 161 | total_timesteps 3463.
Path 162 | total_timesteps 3481.
Path 163 | total_timesteps 3493.
Path 164 | total_timesteps 3507.
Path 165 | total_timesteps 3521.
Path 166 | total_timesteps 3547.
Path 167 | total_timesteps 3563.
Path 168 | total_timesteps 3584.
Path 169 | total_timesteps 3609.
Path 170 | total_timesteps 3627.
Path 171 | total_timesteps 3654.
Path 172 | total_timesteps 3667.
Path 173 | total_timesteps 3697.
Path 174 | total_timesteps 3719.
Path 175 | total_timesteps 3736.
Path 176 | total_timesteps 3763.
Path 177 | total_timesteps 3795.
Path 178 | total_timesteps 3818.
Path 179 | total_timesteps 3835.
Path 180 | total_timesteps 3844.
Path 181 | total_timesteps 3877.
Path 182 | total_timesteps 3890.
Path 183 | total_timesteps 3912.
Path 184 | total_timesteps 3931.
Path 185 | total_timesteps 3958.
Path 186 | total_timesteps 3970.
Path 187 | total_timesteps 3982.
Path 188 | total_timesteps 4012.
Path 189 | total_timesteps 4037.
Path 190 | total_timesteps 4074.
Path 191 | total_timesteps 4087.
Path 192 | total_timesteps 4106.
Path 193 | total_timesteps 4119.
Path 194 | total_timesteps 4137.
Path 195 | total_timesteps 4151.
Path 196 | total_timesteps 4176.
Path 197 | total_timesteps 4189.
Path 198 | total_timesteps 4210.
Path 199 | total_timesteps 4231.
Path 200 | total_timesteps 4253.
Path 201 | total_timesteps 4278.
Path 202 | total_timesteps 4310.
Path 203 | total_timesteps 4325.
Path 204 | total_timesteps 4345.
Path 205 | total_timesteps 4353.
Path 206 | total_timesteps 4378.
Path 207 | total_timesteps 4388.
Path 208 | total_timesteps 4409.
Path 209 | total_timesteps 4420.
Path 210 | total_timesteps 4447.
Path 211 | total_timesteps 4459.
Path 212 | total_timesteps 4485.
Path 213 | total_timesteps 4505.
Path 214 | total_timesteps 4530.
Path 215 | total_timesteps 4547.
Path 216 | total_timesteps 4567.
Path 217 | total_timesteps 4590.
Path 218 | total_timesteps 4613.
Path 219 | total_timesteps 4628.
Path 220 | total_timesteps 4651.
Path 221 | total_timesteps 4662.
Path 222 | total_timesteps 4675.
Path 223 | total_timesteps 4710.
Path 224 | total_timesteps 4724.
Path 225 | total_timesteps 4757.
Path 226 | total_timesteps 4780.
Path 227 | total_timesteps 4815.
Path 228 | total_timesteps 4836.
Path 229 | total_timesteps 4870.
Path 230 | total_timesteps 4882.
Path 231 | total_timesteps 4901.
Path 232 | total_timesteps 4915.
Path 233 | total_timesteps 4937.
Path 234 | total_timesteps 4964.
Path 235 | total_timesteps 4983.
Path 236 | total_timesteps 5015.
Path 237 | total_timesteps 5044.
Path 238 | total_timesteps 5078.
Path 239 | total_timesteps 5097.
Path 240 | total_timesteps 5125.
Path 241 | total_timesteps 5144.
Path 242 | total_timesteps 5184.
Path 243 | total_timesteps 5200.
Path 244 | total_timesteps 5211.
Path 245 | total_timesteps 5231.
Path 246 | total_timesteps 5264.
Path 247 | total_timesteps 5283.
Path 248 | total_timesteps 5301.
Path 249 | total_timesteps 5325.
Path 250 | total_timesteps 5337.
Path 251 | total_timesteps 5353.
Path 252 | total_timesteps 5368.
Path 253 | total_timesteps 5387.
Path 254 | total_timesteps 5417.
Path 255 | total_timesteps 5432.
Path 256 | total_timesteps 5445.
Path 257 | total_timesteps 5463.
Path 258 | total_timesteps 5495.
Path 259 | total_timesteps 5509.
Path 260 | total_timesteps 5538.
Path 261 | total_timesteps 5555.
Path 262 | total_timesteps 5575.
Path 263 | total_timesteps 5594.
Path 264 | total_timesteps 5631.
Path 265 | total_timesteps 5641.
Path 266 | total_timesteps 5660.
Path 267 | total_timesteps 5679.
Path 268 | total_timesteps 5710.
Path 269 | total_timesteps 5717.
Path 270 | total_timesteps 5736.
Path 271 | total_timesteps 5779.
Path 272 | total_timesteps 5792.
Path 273 | total_timesteps 5803.
Path 274 | total_timesteps 5820.
Path 275 | total_timesteps 5841.
Path 276 | total_timesteps 5857.
Path 277 | total_timesteps 5879.
Path 278 | total_timesteps 5894.
Path 279 | total_timesteps 5903.
Path 280 | total_timesteps 5918.
Path 281 | total_timesteps 5936.
Path 282 | total_timesteps 5953.
Path 283 | total_timesteps 5977.
Path 284 | total_timesteps 5989.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.3362397253513336
Validation loss = 0.1352909505367279
Validation loss = 0.09466902166604996
Validation loss = 0.07740727812051773
Validation loss = 0.06901225447654724
Validation loss = 0.06337985396385193
Validation loss = 0.060194119811058044
Validation loss = 0.055364951491355896
Validation loss = 0.05061890929937363
Validation loss = 0.05067264288663864
Validation loss = 0.04929446056485176
Validation loss = 0.05751066282391548
Validation loss = 0.046133074909448624
Validation loss = 0.048153992742300034
Validation loss = 0.04246809333562851
Validation loss = 0.04333331063389778
Validation loss = 0.04208303242921829
Validation loss = 0.04449809342622757
Validation loss = 0.03984919190406799
Validation loss = 0.04141325503587723
Validation loss = 0.04172462224960327
Validation loss = 0.05894237384200096
Validation loss = 0.040513623505830765
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 33.
Path 2 | total_timesteps 46.
Path 3 | total_timesteps 68.
Path 4 | total_timesteps 97.
Path 5 | total_timesteps 117.
Path 6 | total_timesteps 137.
Path 7 | total_timesteps 157.
Path 8 | total_timesteps 185.
Path 9 | total_timesteps 205.
Path 10 | total_timesteps 233.
Path 11 | total_timesteps 277.
Path 12 | total_timesteps 308.
Path 13 | total_timesteps 340.
Path 14 | total_timesteps 371.
Path 15 | total_timesteps 394.
Path 16 | total_timesteps 423.
Path 17 | total_timesteps 448.
Path 18 | total_timesteps 505.
Path 19 | total_timesteps 530.
Path 20 | total_timesteps 549.
Path 21 | total_timesteps 592.
Path 22 | total_timesteps 622.
Path 23 | total_timesteps 647.
Path 24 | total_timesteps 680.
Path 25 | total_timesteps 703.
Path 26 | total_timesteps 728.
Path 27 | total_timesteps 772.
Path 28 | total_timesteps 798.
Path 29 | total_timesteps 813.
Path 30 | total_timesteps 828.
Path 31 | total_timesteps 844.
Path 32 | total_timesteps 874.
Path 33 | total_timesteps 907.
Path 34 | total_timesteps 916.
Path 35 | total_timesteps 943.
Path 36 | total_timesteps 971.
Path 37 | total_timesteps 989.
Path 38 | total_timesteps 1021.
Path 39 | total_timesteps 1037.
Path 40 | total_timesteps 1074.
Path 41 | total_timesteps 1104.
Path 42 | total_timesteps 1115.
Path 43 | total_timesteps 1132.
Path 44 | total_timesteps 1158.
Path 45 | total_timesteps 1206.
Path 46 | total_timesteps 1227.
Path 47 | total_timesteps 1238.
Path 48 | total_timesteps 1255.
Path 49 | total_timesteps 1266.
Path 50 | total_timesteps 1291.
Path 51 | total_timesteps 1303.
Path 52 | total_timesteps 1317.
Path 53 | total_timesteps 1350.
Path 54 | total_timesteps 1383.
Path 55 | total_timesteps 1406.
Path 56 | total_timesteps 1421.
Path 57 | total_timesteps 1438.
Path 58 | total_timesteps 1457.
Path 59 | total_timesteps 1481.
Path 60 | total_timesteps 1496.
Path 61 | total_timesteps 1505.
Path 62 | total_timesteps 1525.
Path 63 | total_timesteps 1543.
Path 64 | total_timesteps 1609.
Path 65 | total_timesteps 1634.
Path 66 | total_timesteps 1657.
Path 67 | total_timesteps 1687.
Path 68 | total_timesteps 1702.
Path 69 | total_timesteps 1720.
Path 70 | total_timesteps 1739.
Path 71 | total_timesteps 1752.
Path 72 | total_timesteps 1809.
Path 73 | total_timesteps 1820.
Path 74 | total_timesteps 1870.
Path 75 | total_timesteps 1885.
Path 76 | total_timesteps 1928.
Path 77 | total_timesteps 1959.
Path 78 | total_timesteps 1975.
Path 79 | total_timesteps 1987.
Path 80 | total_timesteps 2007.
Path 81 | total_timesteps 2025.
Path 82 | total_timesteps 2060.
Path 83 | total_timesteps 2090.
Path 84 | total_timesteps 2108.
Path 85 | total_timesteps 2144.
Path 86 | total_timesteps 2195.
Path 87 | total_timesteps 2224.
Path 88 | total_timesteps 2245.
Path 89 | total_timesteps 2270.
Path 90 | total_timesteps 2280.
Path 91 | total_timesteps 2311.
Path 92 | total_timesteps 2346.
Path 93 | total_timesteps 2354.
Path 94 | total_timesteps 2369.
Path 95 | total_timesteps 2391.
Path 96 | total_timesteps 2409.
Path 97 | total_timesteps 2425.
Path 98 | total_timesteps 2438.
Path 99 | total_timesteps 2458.
Path 100 | total_timesteps 2470.
Path 101 | total_timesteps 2506.
Path 102 | total_timesteps 2543.
Path 103 | total_timesteps 2564.
Path 104 | total_timesteps 2586.
Path 105 | total_timesteps 2614.
Path 106 | total_timesteps 2645.
Path 107 | total_timesteps 2683.
Path 108 | total_timesteps 2703.
Path 109 | total_timesteps 2721.
Path 110 | total_timesteps 2740.
Path 111 | total_timesteps 2756.
Path 112 | total_timesteps 2777.
Path 113 | total_timesteps 2821.
Path 114 | total_timesteps 2841.
Path 115 | total_timesteps 2880.
Path 116 | total_timesteps 2910.
Path 117 | total_timesteps 2954.
Path 118 | total_timesteps 3003.
Path 119 | total_timesteps 3019.
Path 120 | total_timesteps 3050.
Path 121 | total_timesteps 3071.
Path 122 | total_timesteps 3099.
Path 123 | total_timesteps 3128.
Path 124 | total_timesteps 3136.
Path 125 | total_timesteps 3154.
Path 126 | total_timesteps 3174.
Path 127 | total_timesteps 3211.
Path 128 | total_timesteps 3233.
Path 129 | total_timesteps 3260.
Path 130 | total_timesteps 3295.
Path 131 | total_timesteps 3326.
Path 132 | total_timesteps 3348.
Path 133 | total_timesteps 3369.
Path 134 | total_timesteps 3393.
Path 135 | total_timesteps 3413.
Path 136 | total_timesteps 3459.
Path 137 | total_timesteps 3482.
Path 138 | total_timesteps 3510.
Path 139 | total_timesteps 3527.
Path 140 | total_timesteps 3551.
Path 141 | total_timesteps 3561.
Path 142 | total_timesteps 3589.
Path 143 | total_timesteps 3611.
Path 144 | total_timesteps 3644.
Path 145 | total_timesteps 3678.
Path 146 | total_timesteps 3700.
Path 147 | total_timesteps 3714.
Path 148 | total_timesteps 3734.
Path 149 | total_timesteps 3751.
Path 150 | total_timesteps 3776.
Path 151 | total_timesteps 3817.
Path 152 | total_timesteps 3851.
Path 153 | total_timesteps 3871.
Path 154 | total_timesteps 3910.
Path 155 | total_timesteps 3955.
Path 156 | total_timesteps 3977.
Path 157 | total_timesteps 3985.
Path 158 | total_timesteps 4014.
Path 159 | total_timesteps 4046.
Path 160 | total_timesteps 4067.
Path 161 | total_timesteps 4080.
Path 162 | total_timesteps 4120.
Path 163 | total_timesteps 4170.
Path 164 | total_timesteps 4191.
Path 165 | total_timesteps 4206.
Path 166 | total_timesteps 4225.
Path 167 | total_timesteps 4245.
Path 168 | total_timesteps 4262.
Path 169 | total_timesteps 4300.
Path 170 | total_timesteps 4322.
Path 171 | total_timesteps 4341.
Path 172 | total_timesteps 4355.
Path 173 | total_timesteps 4391.
Path 174 | total_timesteps 4410.
Path 175 | total_timesteps 4451.
Path 176 | total_timesteps 4477.
Path 177 | total_timesteps 4493.
Path 178 | total_timesteps 4510.
Path 179 | total_timesteps 4545.
Path 180 | total_timesteps 4575.
Path 181 | total_timesteps 4598.
Path 182 | total_timesteps 4623.
Path 183 | total_timesteps 4642.
Path 184 | total_timesteps 4655.
Path 185 | total_timesteps 4676.
Path 186 | total_timesteps 4712.
Path 187 | total_timesteps 4727.
Path 188 | total_timesteps 4757.
Path 189 | total_timesteps 4792.
Path 190 | total_timesteps 4819.
Path 191 | total_timesteps 4839.
Path 192 | total_timesteps 4864.
Path 193 | total_timesteps 4886.
Path 194 | total_timesteps 4897.
Path 195 | total_timesteps 4920.
Path 196 | total_timesteps 4944.
Path 197 | total_timesteps 4964.
Path 198 | total_timesteps 5002.
Path 199 | total_timesteps 5034.
Path 200 | total_timesteps 5074.
Path 201 | total_timesteps 5107.
Path 202 | total_timesteps 5122.
Path 203 | total_timesteps 5157.
Path 204 | total_timesteps 5176.
Path 205 | total_timesteps 5201.
Path 206 | total_timesteps 5224.
Path 207 | total_timesteps 5248.
Path 208 | total_timesteps 5273.
Path 209 | total_timesteps 5318.
Path 210 | total_timesteps 5329.
Path 211 | total_timesteps 5348.
Path 212 | total_timesteps 5370.
Path 213 | total_timesteps 5407.
Path 214 | total_timesteps 5428.
Path 215 | total_timesteps 5459.
Path 216 | total_timesteps 5488.
Path 217 | total_timesteps 5526.
Path 218 | total_timesteps 5550.
Path 219 | total_timesteps 5574.
Path 220 | total_timesteps 5596.
Path 221 | total_timesteps 5616.
Path 222 | total_timesteps 5647.
Path 223 | total_timesteps 5673.
Path 224 | total_timesteps 5704.
Path 225 | total_timesteps 5727.
Path 226 | total_timesteps 5757.
Path 227 | total_timesteps 5769.
Path 228 | total_timesteps 5796.
Path 229 | total_timesteps 5812.
Path 230 | total_timesteps 5838.
Path 231 | total_timesteps 5875.
Path 232 | total_timesteps 5906.
Path 233 | total_timesteps 5918.
Path 234 | total_timesteps 5954.
Path 235 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.82    |
| Iteration     | 0        |
| MaximumReturn | 9.67     |
| MinimumReturn | -26.3    |
| TotalSamples  | 8025     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.09645739197731018
Validation loss = 0.06773631274700165
Validation loss = 0.05396351218223572
Validation loss = 0.049624066799879074
Validation loss = 0.04647620767354965
Validation loss = 0.043820157647132874
Validation loss = 0.044155053794384
Validation loss = 0.04091745987534523
Validation loss = 0.04086398333311081
Validation loss = 0.041923295706510544
Validation loss = 0.04133253172039986
Validation loss = 0.04027014598250389
Validation loss = 0.043062079697847366
Validation loss = 0.037279896438121796
Validation loss = 0.040602151304483414
Validation loss = 0.039462827146053314
Validation loss = 0.03783600777387619
Validation loss = 0.03976709395647049
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 46.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 75.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 94.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 115.
Path 12 | total_timesteps 123.
Path 13 | total_timesteps 131.
Path 14 | total_timesteps 144.
Path 15 | total_timesteps 154.
Path 16 | total_timesteps 166.
Path 17 | total_timesteps 182.
Path 18 | total_timesteps 192.
Path 19 | total_timesteps 206.
Path 20 | total_timesteps 222.
Path 21 | total_timesteps 231.
Path 22 | total_timesteps 241.
Path 23 | total_timesteps 249.
Path 24 | total_timesteps 260.
Path 25 | total_timesteps 271.
Path 26 | total_timesteps 284.
Path 27 | total_timesteps 302.
Path 28 | total_timesteps 314.
Path 29 | total_timesteps 323.
Path 30 | total_timesteps 333.
Path 31 | total_timesteps 344.
Path 32 | total_timesteps 363.
Path 33 | total_timesteps 375.
Path 34 | total_timesteps 385.
Path 35 | total_timesteps 399.
Path 36 | total_timesteps 413.
Path 37 | total_timesteps 423.
Path 38 | total_timesteps 433.
Path 39 | total_timesteps 446.
Path 40 | total_timesteps 456.
Path 41 | total_timesteps 467.
Path 42 | total_timesteps 483.
Path 43 | total_timesteps 491.
Path 44 | total_timesteps 499.
Path 45 | total_timesteps 508.
Path 46 | total_timesteps 518.
Path 47 | total_timesteps 525.
Path 48 | total_timesteps 542.
Path 49 | total_timesteps 553.
Path 50 | total_timesteps 560.
Path 51 | total_timesteps 575.
Path 52 | total_timesteps 584.
Path 53 | total_timesteps 595.
Path 54 | total_timesteps 612.
Path 55 | total_timesteps 620.
Path 56 | total_timesteps 629.
Path 57 | total_timesteps 637.
Path 58 | total_timesteps 645.
Path 59 | total_timesteps 657.
Path 60 | total_timesteps 670.
Path 61 | total_timesteps 677.
Path 62 | total_timesteps 690.
Path 63 | total_timesteps 700.
Path 64 | total_timesteps 712.
Path 65 | total_timesteps 721.
Path 66 | total_timesteps 730.
Path 67 | total_timesteps 741.
Path 68 | total_timesteps 769.
Path 69 | total_timesteps 781.
Path 70 | total_timesteps 791.
Path 71 | total_timesteps 804.
Path 72 | total_timesteps 824.
Path 73 | total_timesteps 832.
Path 74 | total_timesteps 842.
Path 75 | total_timesteps 851.
Path 76 | total_timesteps 873.
Path 77 | total_timesteps 880.
Path 78 | total_timesteps 891.
Path 79 | total_timesteps 901.
Path 80 | total_timesteps 917.
Path 81 | total_timesteps 929.
Path 82 | total_timesteps 936.
Path 83 | total_timesteps 948.
Path 84 | total_timesteps 956.
Path 85 | total_timesteps 971.
Path 86 | total_timesteps 984.
Path 87 | total_timesteps 1007.
Path 88 | total_timesteps 1015.
Path 89 | total_timesteps 1022.
Path 90 | total_timesteps 1040.
Path 91 | total_timesteps 1050.
Path 92 | total_timesteps 1073.
Path 93 | total_timesteps 1083.
Path 94 | total_timesteps 1094.
Path 95 | total_timesteps 1105.
Path 96 | total_timesteps 1114.
Path 97 | total_timesteps 1122.
Path 98 | total_timesteps 1131.
Path 99 | total_timesteps 1144.
Path 100 | total_timesteps 1151.
Path 101 | total_timesteps 1159.
Path 102 | total_timesteps 1169.
Path 103 | total_timesteps 1179.
Path 104 | total_timesteps 1193.
Path 105 | total_timesteps 1208.
Path 106 | total_timesteps 1218.
Path 107 | total_timesteps 1227.
Path 108 | total_timesteps 1240.
Path 109 | total_timesteps 1248.
Path 110 | total_timesteps 1260.
Path 111 | total_timesteps 1271.
Path 112 | total_timesteps 1282.
Path 113 | total_timesteps 1292.
Path 114 | total_timesteps 1301.
Path 115 | total_timesteps 1311.
Path 116 | total_timesteps 1325.
Path 117 | total_timesteps 1337.
Path 118 | total_timesteps 1349.
Path 119 | total_timesteps 1358.
Path 120 | total_timesteps 1373.
Path 121 | total_timesteps 1395.
Path 122 | total_timesteps 1406.
Path 123 | total_timesteps 1417.
Path 124 | total_timesteps 1424.
Path 125 | total_timesteps 1440.
Path 126 | total_timesteps 1449.
Path 127 | total_timesteps 1458.
Path 128 | total_timesteps 1472.
Path 129 | total_timesteps 1482.
Path 130 | total_timesteps 1489.
Path 131 | total_timesteps 1497.
Path 132 | total_timesteps 1507.
Path 133 | total_timesteps 1517.
Path 134 | total_timesteps 1526.
Path 135 | total_timesteps 1537.
Path 136 | total_timesteps 1557.
Path 137 | total_timesteps 1565.
Path 138 | total_timesteps 1575.
Path 139 | total_timesteps 1586.
Path 140 | total_timesteps 1596.
Path 141 | total_timesteps 1607.
Path 142 | total_timesteps 1630.
Path 143 | total_timesteps 1649.
Path 144 | total_timesteps 1665.
Path 145 | total_timesteps 1672.
Path 146 | total_timesteps 1686.
Path 147 | total_timesteps 1699.
Path 148 | total_timesteps 1706.
Path 149 | total_timesteps 1714.
Path 150 | total_timesteps 1726.
Path 151 | total_timesteps 1743.
Path 152 | total_timesteps 1752.
Path 153 | total_timesteps 1769.
Path 154 | total_timesteps 1780.
Path 155 | total_timesteps 1799.
Path 156 | total_timesteps 1816.
Path 157 | total_timesteps 1832.
Path 158 | total_timesteps 1842.
Path 159 | total_timesteps 1854.
Path 160 | total_timesteps 1869.
Path 161 | total_timesteps 1880.
Path 162 | total_timesteps 1894.
Path 163 | total_timesteps 1903.
Path 164 | total_timesteps 1912.
Path 165 | total_timesteps 1924.
Path 166 | total_timesteps 1939.
Path 167 | total_timesteps 1947.
Path 168 | total_timesteps 1967.
Path 169 | total_timesteps 1978.
Path 170 | total_timesteps 1990.
Path 171 | total_timesteps 1999.
Path 172 | total_timesteps 2015.
Path 173 | total_timesteps 2024.
Path 174 | total_timesteps 2041.
Path 175 | total_timesteps 2050.
Path 176 | total_timesteps 2059.
Path 177 | total_timesteps 2069.
Path 178 | total_timesteps 2082.
Path 179 | total_timesteps 2092.
Path 180 | total_timesteps 2103.
Path 181 | total_timesteps 2111.
Path 182 | total_timesteps 2120.
Path 183 | total_timesteps 2134.
Path 184 | total_timesteps 2145.
Path 185 | total_timesteps 2153.
Path 186 | total_timesteps 2164.
Path 187 | total_timesteps 2175.
Path 188 | total_timesteps 2199.
Path 189 | total_timesteps 2207.
Path 190 | total_timesteps 2217.
Path 191 | total_timesteps 2226.
Path 192 | total_timesteps 2237.
Path 193 | total_timesteps 2249.
Path 194 | total_timesteps 2259.
Path 195 | total_timesteps 2267.
Path 196 | total_timesteps 2275.
Path 197 | total_timesteps 2286.
Path 198 | total_timesteps 2295.
Path 199 | total_timesteps 2305.
Path 200 | total_timesteps 2318.
Path 201 | total_timesteps 2328.
Path 202 | total_timesteps 2335.
Path 203 | total_timesteps 2345.
Path 204 | total_timesteps 2355.
Path 205 | total_timesteps 2370.
Path 206 | total_timesteps 2379.
Path 207 | total_timesteps 2394.
Path 208 | total_timesteps 2404.
Path 209 | total_timesteps 2411.
Path 210 | total_timesteps 2424.
Path 211 | total_timesteps 2434.
Path 212 | total_timesteps 2441.
Path 213 | total_timesteps 2450.
Path 214 | total_timesteps 2459.
Path 215 | total_timesteps 2470.
Path 216 | total_timesteps 2480.
Path 217 | total_timesteps 2490.
Path 218 | total_timesteps 2500.
Path 219 | total_timesteps 2508.
Path 220 | total_timesteps 2516.
Path 221 | total_timesteps 2531.
Path 222 | total_timesteps 2542.
Path 223 | total_timesteps 2551.
Path 224 | total_timesteps 2561.
Path 225 | total_timesteps 2570.
Path 226 | total_timesteps 2584.
Path 227 | total_timesteps 2592.
Path 228 | total_timesteps 2601.
Path 229 | total_timesteps 2613.
Path 230 | total_timesteps 2624.
Path 231 | total_timesteps 2638.
Path 232 | total_timesteps 2645.
Path 233 | total_timesteps 2653.
Path 234 | total_timesteps 2663.
Path 235 | total_timesteps 2673.
Path 236 | total_timesteps 2682.
Path 237 | total_timesteps 2690.
Path 238 | total_timesteps 2700.
Path 239 | total_timesteps 2713.
Path 240 | total_timesteps 2725.
Path 241 | total_timesteps 2734.
Path 242 | total_timesteps 2742.
Path 243 | total_timesteps 2758.
Path 244 | total_timesteps 2773.
Path 245 | total_timesteps 2783.
Path 246 | total_timesteps 2792.
Path 247 | total_timesteps 2815.
Path 248 | total_timesteps 2824.
Path 249 | total_timesteps 2834.
Path 250 | total_timesteps 2845.
Path 251 | total_timesteps 2854.
Path 252 | total_timesteps 2864.
Path 253 | total_timesteps 2872.
Path 254 | total_timesteps 2883.
Path 255 | total_timesteps 2892.
Path 256 | total_timesteps 2901.
Path 257 | total_timesteps 2908.
Path 258 | total_timesteps 2919.
Path 259 | total_timesteps 2935.
Path 260 | total_timesteps 2947.
Path 261 | total_timesteps 2957.
Path 262 | total_timesteps 2974.
Path 263 | total_timesteps 2989.
Path 264 | total_timesteps 3001.
Path 265 | total_timesteps 3011.
Path 266 | total_timesteps 3023.
Path 267 | total_timesteps 3036.
Path 268 | total_timesteps 3046.
Path 269 | total_timesteps 3055.
Path 270 | total_timesteps 3067.
Path 271 | total_timesteps 3081.
Path 272 | total_timesteps 3092.
Path 273 | total_timesteps 3104.
Path 274 | total_timesteps 3114.
Path 275 | total_timesteps 3122.
Path 276 | total_timesteps 3131.
Path 277 | total_timesteps 3144.
Path 278 | total_timesteps 3157.
Path 279 | total_timesteps 3173.
Path 280 | total_timesteps 3182.
Path 281 | total_timesteps 3191.
Path 282 | total_timesteps 3203.
Path 283 | total_timesteps 3217.
Path 284 | total_timesteps 3232.
Path 285 | total_timesteps 3238.
Path 286 | total_timesteps 3250.
Path 287 | total_timesteps 3258.
Path 288 | total_timesteps 3266.
Path 289 | total_timesteps 3273.
Path 290 | total_timesteps 3288.
Path 291 | total_timesteps 3300.
Path 292 | total_timesteps 3309.
Path 293 | total_timesteps 3318.
Path 294 | total_timesteps 3326.
Path 295 | total_timesteps 3342.
Path 296 | total_timesteps 3355.
Path 297 | total_timesteps 3364.
Path 298 | total_timesteps 3376.
Path 299 | total_timesteps 3388.
Path 300 | total_timesteps 3403.
Path 301 | total_timesteps 3411.
Path 302 | total_timesteps 3426.
Path 303 | total_timesteps 3432.
Path 304 | total_timesteps 3444.
Path 305 | total_timesteps 3458.
Path 306 | total_timesteps 3470.
Path 307 | total_timesteps 3482.
Path 308 | total_timesteps 3492.
Path 309 | total_timesteps 3501.
Path 310 | total_timesteps 3515.
Path 311 | total_timesteps 3532.
Path 312 | total_timesteps 3543.
Path 313 | total_timesteps 3555.
Path 314 | total_timesteps 3568.
Path 315 | total_timesteps 3578.
Path 316 | total_timesteps 3589.
Path 317 | total_timesteps 3602.
Path 318 | total_timesteps 3612.
Path 319 | total_timesteps 3635.
Path 320 | total_timesteps 3644.
Path 321 | total_timesteps 3655.
Path 322 | total_timesteps 3664.
Path 323 | total_timesteps 3672.
Path 324 | total_timesteps 3693.
Path 325 | total_timesteps 3700.
Path 326 | total_timesteps 3711.
Path 327 | total_timesteps 3720.
Path 328 | total_timesteps 3728.
Path 329 | total_timesteps 3735.
Path 330 | total_timesteps 3748.
Path 331 | total_timesteps 3758.
Path 332 | total_timesteps 3766.
Path 333 | total_timesteps 3777.
Path 334 | total_timesteps 3784.
Path 335 | total_timesteps 3800.
Path 336 | total_timesteps 3813.
Path 337 | total_timesteps 3825.
Path 338 | total_timesteps 3841.
Path 339 | total_timesteps 3851.
Path 340 | total_timesteps 3859.
Path 341 | total_timesteps 3873.
Path 342 | total_timesteps 3881.
Path 343 | total_timesteps 3891.
Path 344 | total_timesteps 3902.
Path 345 | total_timesteps 3911.
Path 346 | total_timesteps 3921.
Path 347 | total_timesteps 3932.
Path 348 | total_timesteps 3942.
Path 349 | total_timesteps 3952.
Path 350 | total_timesteps 3962.
Path 351 | total_timesteps 3978.
Path 352 | total_timesteps 4007.
Path 353 | total_timesteps 4015.
Path 354 | total_timesteps 4025.
Path 355 | total_timesteps 4031.
Path 356 | total_timesteps 4037.
Path 357 | total_timesteps 4053.
Path 358 | total_timesteps 4070.
Path 359 | total_timesteps 4080.
Path 360 | total_timesteps 4091.
Path 361 | total_timesteps 4098.
Path 362 | total_timesteps 4107.
Path 363 | total_timesteps 4117.
Path 364 | total_timesteps 4131.
Path 365 | total_timesteps 4142.
Path 366 | total_timesteps 4154.
Path 367 | total_timesteps 4163.
Path 368 | total_timesteps 4177.
Path 369 | total_timesteps 4185.
Path 370 | total_timesteps 4194.
Path 371 | total_timesteps 4206.
Path 372 | total_timesteps 4215.
Path 373 | total_timesteps 4224.
Path 374 | total_timesteps 4239.
Path 375 | total_timesteps 4251.
Path 376 | total_timesteps 4259.
Path 377 | total_timesteps 4267.
Path 378 | total_timesteps 4285.
Path 379 | total_timesteps 4298.
Path 380 | total_timesteps 4325.
Path 381 | total_timesteps 4337.
Path 382 | total_timesteps 4351.
Path 383 | total_timesteps 4359.
Path 384 | total_timesteps 4369.
Path 385 | total_timesteps 4379.
Path 386 | total_timesteps 4390.
Path 387 | total_timesteps 4400.
Path 388 | total_timesteps 4425.
Path 389 | total_timesteps 4438.
Path 390 | total_timesteps 4446.
Path 391 | total_timesteps 4454.
Path 392 | total_timesteps 4462.
Path 393 | total_timesteps 4473.
Path 394 | total_timesteps 4481.
Path 395 | total_timesteps 4494.
Path 396 | total_timesteps 4500.
Path 397 | total_timesteps 4509.
Path 398 | total_timesteps 4521.
Path 399 | total_timesteps 4527.
Path 400 | total_timesteps 4534.
Path 401 | total_timesteps 4547.
Path 402 | total_timesteps 4560.
Path 403 | total_timesteps 4568.
Path 404 | total_timesteps 4578.
Path 405 | total_timesteps 4592.
Path 406 | total_timesteps 4601.
Path 407 | total_timesteps 4610.
Path 408 | total_timesteps 4627.
Path 409 | total_timesteps 4639.
Path 410 | total_timesteps 4648.
Path 411 | total_timesteps 4666.
Path 412 | total_timesteps 4676.
Path 413 | total_timesteps 4687.
Path 414 | total_timesteps 4693.
Path 415 | total_timesteps 4704.
Path 416 | total_timesteps 4713.
Path 417 | total_timesteps 4730.
Path 418 | total_timesteps 4739.
Path 419 | total_timesteps 4748.
Path 420 | total_timesteps 4761.
Path 421 | total_timesteps 4774.
Path 422 | total_timesteps 4781.
Path 423 | total_timesteps 4789.
Path 424 | total_timesteps 4796.
Path 425 | total_timesteps 4819.
Path 426 | total_timesteps 4827.
Path 427 | total_timesteps 4840.
Path 428 | total_timesteps 4856.
Path 429 | total_timesteps 4867.
Path 430 | total_timesteps 4878.
Path 431 | total_timesteps 4889.
Path 432 | total_timesteps 4906.
Path 433 | total_timesteps 4917.
Path 434 | total_timesteps 4928.
Path 435 | total_timesteps 4942.
Path 436 | total_timesteps 4953.
Path 437 | total_timesteps 4964.
Path 438 | total_timesteps 4983.
Path 439 | total_timesteps 4994.
Path 440 | total_timesteps 5004.
Path 441 | total_timesteps 5010.
Path 442 | total_timesteps 5018.
Path 443 | total_timesteps 5025.
Path 444 | total_timesteps 5039.
Path 445 | total_timesteps 5050.
Path 446 | total_timesteps 5062.
Path 447 | total_timesteps 5072.
Path 448 | total_timesteps 5081.
Path 449 | total_timesteps 5090.
Path 450 | total_timesteps 5101.
Path 451 | total_timesteps 5107.
Path 452 | total_timesteps 5117.
Path 453 | total_timesteps 5129.
Path 454 | total_timesteps 5142.
Path 455 | total_timesteps 5154.
Path 456 | total_timesteps 5161.
Path 457 | total_timesteps 5171.
Path 458 | total_timesteps 5179.
Path 459 | total_timesteps 5188.
Path 460 | total_timesteps 5196.
Path 461 | total_timesteps 5206.
Path 462 | total_timesteps 5219.
Path 463 | total_timesteps 5243.
Path 464 | total_timesteps 5258.
Path 465 | total_timesteps 5269.
Path 466 | total_timesteps 5279.
Path 467 | total_timesteps 5288.
Path 468 | total_timesteps 5296.
Path 469 | total_timesteps 5304.
Path 470 | total_timesteps 5325.
Path 471 | total_timesteps 5339.
Path 472 | total_timesteps 5350.
Path 473 | total_timesteps 5360.
Path 474 | total_timesteps 5375.
Path 475 | total_timesteps 5385.
Path 476 | total_timesteps 5394.
Path 477 | total_timesteps 5405.
Path 478 | total_timesteps 5417.
Path 479 | total_timesteps 5428.
Path 480 | total_timesteps 5440.
Path 481 | total_timesteps 5450.
Path 482 | total_timesteps 5462.
Path 483 | total_timesteps 5471.
Path 484 | total_timesteps 5478.
Path 485 | total_timesteps 5486.
Path 486 | total_timesteps 5498.
Path 487 | total_timesteps 5511.
Path 488 | total_timesteps 5520.
Path 489 | total_timesteps 5538.
Path 490 | total_timesteps 5545.
Path 491 | total_timesteps 5557.
Path 492 | total_timesteps 5581.
Path 493 | total_timesteps 5593.
Path 494 | total_timesteps 5602.
Path 495 | total_timesteps 5614.
Path 496 | total_timesteps 5622.
Path 497 | total_timesteps 5634.
Path 498 | total_timesteps 5644.
Path 499 | total_timesteps 5657.
Path 500 | total_timesteps 5674.
Path 501 | total_timesteps 5682.
Path 502 | total_timesteps 5691.
Path 503 | total_timesteps 5710.
Path 504 | total_timesteps 5720.
Path 505 | total_timesteps 5729.
Path 506 | total_timesteps 5739.
Path 507 | total_timesteps 5747.
Path 508 | total_timesteps 5755.
Path 509 | total_timesteps 5764.
Path 510 | total_timesteps 5776.
Path 511 | total_timesteps 5788.
Path 512 | total_timesteps 5796.
Path 513 | total_timesteps 5803.
Path 514 | total_timesteps 5818.
Path 515 | total_timesteps 5839.
Path 516 | total_timesteps 5846.
Path 517 | total_timesteps 5857.
Path 518 | total_timesteps 5867.
Path 519 | total_timesteps 5881.
Path 520 | total_timesteps 5892.
Path 521 | total_timesteps 5904.
Path 522 | total_timesteps 5923.
Path 523 | total_timesteps 5937.
Path 524 | total_timesteps 5947.
Path 525 | total_timesteps 5955.
Path 526 | total_timesteps 5975.
Path 527 | total_timesteps 5988.
Path 528 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.42    |
| Iteration     | 1        |
| MaximumReturn | 0.837    |
| MinimumReturn | -22.1    |
| TotalSamples  | 12031    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.05719893053174019
Validation loss = 0.027735697105526924
Validation loss = 0.028812706470489502
Validation loss = 0.027744973078370094
Validation loss = 0.026471061632037163
Validation loss = 0.028901593759655952
Validation loss = 0.026867687702178955
Validation loss = 0.026799580082297325
Validation loss = 0.02385190688073635
Validation loss = 0.03172281011939049
Validation loss = 0.02604273520410061
Validation loss = 0.023714430630207062
Validation loss = 0.024385197088122368
Validation loss = 0.02522129751741886
Validation loss = 0.024942325428128242
Validation loss = 0.023782780393958092
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 13.
Path 3 | total_timesteps 20.
Path 4 | total_timesteps 26.
Path 5 | total_timesteps 38.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 57.
Path 8 | total_timesteps 66.
Path 9 | total_timesteps 73.
Path 10 | total_timesteps 81.
Path 11 | total_timesteps 89.
Path 12 | total_timesteps 97.
Path 13 | total_timesteps 105.
Path 14 | total_timesteps 113.
Path 15 | total_timesteps 123.
Path 16 | total_timesteps 131.
Path 17 | total_timesteps 138.
Path 18 | total_timesteps 148.
Path 19 | total_timesteps 156.
Path 20 | total_timesteps 163.
Path 21 | total_timesteps 173.
Path 22 | total_timesteps 181.
Path 23 | total_timesteps 187.
Path 24 | total_timesteps 195.
Path 25 | total_timesteps 205.
Path 26 | total_timesteps 221.
Path 27 | total_timesteps 228.
Path 28 | total_timesteps 239.
Path 29 | total_timesteps 249.
Path 30 | total_timesteps 262.
Path 31 | total_timesteps 269.
Path 32 | total_timesteps 276.
Path 33 | total_timesteps 284.
Path 34 | total_timesteps 290.
Path 35 | total_timesteps 298.
Path 36 | total_timesteps 306.
Path 37 | total_timesteps 313.
Path 38 | total_timesteps 322.
Path 39 | total_timesteps 333.
Path 40 | total_timesteps 342.
Path 41 | total_timesteps 355.
Path 42 | total_timesteps 363.
Path 43 | total_timesteps 370.
Path 44 | total_timesteps 377.
Path 45 | total_timesteps 387.
Path 46 | total_timesteps 397.
Path 47 | total_timesteps 409.
Path 48 | total_timesteps 418.
Path 49 | total_timesteps 427.
Path 50 | total_timesteps 441.
Path 51 | total_timesteps 449.
Path 52 | total_timesteps 457.
Path 53 | total_timesteps 465.
Path 54 | total_timesteps 471.
Path 55 | total_timesteps 477.
Path 56 | total_timesteps 486.
Path 57 | total_timesteps 493.
Path 58 | total_timesteps 500.
Path 59 | total_timesteps 508.
Path 60 | total_timesteps 517.
Path 61 | total_timesteps 527.
Path 62 | total_timesteps 535.
Path 63 | total_timesteps 547.
Path 64 | total_timesteps 555.
Path 65 | total_timesteps 564.
Path 66 | total_timesteps 573.
Path 67 | total_timesteps 580.
Path 68 | total_timesteps 589.
Path 69 | total_timesteps 604.
Path 70 | total_timesteps 612.
Path 71 | total_timesteps 621.
Path 72 | total_timesteps 630.
Path 73 | total_timesteps 641.
Path 74 | total_timesteps 649.
Path 75 | total_timesteps 657.
Path 76 | total_timesteps 668.
Path 77 | total_timesteps 678.
Path 78 | total_timesteps 699.
Path 79 | total_timesteps 709.
Path 80 | total_timesteps 717.
Path 81 | total_timesteps 727.
Path 82 | total_timesteps 737.
Path 83 | total_timesteps 745.
Path 84 | total_timesteps 752.
Path 85 | total_timesteps 763.
Path 86 | total_timesteps 772.
Path 87 | total_timesteps 781.
Path 88 | total_timesteps 789.
Path 89 | total_timesteps 800.
Path 90 | total_timesteps 807.
Path 91 | total_timesteps 819.
Path 92 | total_timesteps 831.
Path 93 | total_timesteps 839.
Path 94 | total_timesteps 850.
Path 95 | total_timesteps 860.
Path 96 | total_timesteps 867.
Path 97 | total_timesteps 875.
Path 98 | total_timesteps 882.
Path 99 | total_timesteps 890.
Path 100 | total_timesteps 901.
Path 101 | total_timesteps 908.
Path 102 | total_timesteps 917.
Path 103 | total_timesteps 925.
Path 104 | total_timesteps 935.
Path 105 | total_timesteps 942.
Path 106 | total_timesteps 949.
Path 107 | total_timesteps 959.
Path 108 | total_timesteps 968.
Path 109 | total_timesteps 978.
Path 110 | total_timesteps 985.
Path 111 | total_timesteps 998.
Path 112 | total_timesteps 1008.
Path 113 | total_timesteps 1016.
Path 114 | total_timesteps 1024.
Path 115 | total_timesteps 1032.
Path 116 | total_timesteps 1042.
Path 117 | total_timesteps 1050.
Path 118 | total_timesteps 1058.
Path 119 | total_timesteps 1064.
Path 120 | total_timesteps 1073.
Path 121 | total_timesteps 1081.
Path 122 | total_timesteps 1092.
Path 123 | total_timesteps 1100.
Path 124 | total_timesteps 1108.
Path 125 | total_timesteps 1116.
Path 126 | total_timesteps 1123.
Path 127 | total_timesteps 1134.
Path 128 | total_timesteps 1143.
Path 129 | total_timesteps 1154.
Path 130 | total_timesteps 1162.
Path 131 | total_timesteps 1171.
Path 132 | total_timesteps 1179.
Path 133 | total_timesteps 1188.
Path 134 | total_timesteps 1197.
Path 135 | total_timesteps 1206.
Path 136 | total_timesteps 1217.
Path 137 | total_timesteps 1227.
Path 138 | total_timesteps 1239.
Path 139 | total_timesteps 1249.
Path 140 | total_timesteps 1259.
Path 141 | total_timesteps 1266.
Path 142 | total_timesteps 1273.
Path 143 | total_timesteps 1281.
Path 144 | total_timesteps 1287.
Path 145 | total_timesteps 1294.
Path 146 | total_timesteps 1301.
Path 147 | total_timesteps 1312.
Path 148 | total_timesteps 1322.
Path 149 | total_timesteps 1329.
Path 150 | total_timesteps 1338.
Path 151 | total_timesteps 1352.
Path 152 | total_timesteps 1363.
Path 153 | total_timesteps 1371.
Path 154 | total_timesteps 1382.
Path 155 | total_timesteps 1392.
Path 156 | total_timesteps 1403.
Path 157 | total_timesteps 1411.
Path 158 | total_timesteps 1417.
Path 159 | total_timesteps 1429.
Path 160 | total_timesteps 1439.
Path 161 | total_timesteps 1448.
Path 162 | total_timesteps 1457.
Path 163 | total_timesteps 1466.
Path 164 | total_timesteps 1473.
Path 165 | total_timesteps 1482.
Path 166 | total_timesteps 1489.
Path 167 | total_timesteps 1497.
Path 168 | total_timesteps 1507.
Path 169 | total_timesteps 1516.
Path 170 | total_timesteps 1525.
Path 171 | total_timesteps 1532.
Path 172 | total_timesteps 1542.
Path 173 | total_timesteps 1548.
Path 174 | total_timesteps 1555.
Path 175 | total_timesteps 1562.
Path 176 | total_timesteps 1570.
Path 177 | total_timesteps 1577.
Path 178 | total_timesteps 1587.
Path 179 | total_timesteps 1594.
Path 180 | total_timesteps 1602.
Path 181 | total_timesteps 1609.
Path 182 | total_timesteps 1619.
Path 183 | total_timesteps 1631.
Path 184 | total_timesteps 1638.
Path 185 | total_timesteps 1645.
Path 186 | total_timesteps 1654.
Path 187 | total_timesteps 1661.
Path 188 | total_timesteps 1670.
Path 189 | total_timesteps 1677.
Path 190 | total_timesteps 1685.
Path 191 | total_timesteps 1693.
Path 192 | total_timesteps 1704.
Path 193 | total_timesteps 1719.
Path 194 | total_timesteps 1727.
Path 195 | total_timesteps 1733.
Path 196 | total_timesteps 1742.
Path 197 | total_timesteps 1749.
Path 198 | total_timesteps 1759.
Path 199 | total_timesteps 1767.
Path 200 | total_timesteps 1774.
Path 201 | total_timesteps 1783.
Path 202 | total_timesteps 1789.
Path 203 | total_timesteps 1795.
Path 204 | total_timesteps 1806.
Path 205 | total_timesteps 1816.
Path 206 | total_timesteps 1823.
Path 207 | total_timesteps 1831.
Path 208 | total_timesteps 1839.
Path 209 | total_timesteps 1846.
Path 210 | total_timesteps 1856.
Path 211 | total_timesteps 1864.
Path 212 | total_timesteps 1873.
Path 213 | total_timesteps 1881.
Path 214 | total_timesteps 1892.
Path 215 | total_timesteps 1900.
Path 216 | total_timesteps 1908.
Path 217 | total_timesteps 1916.
Path 218 | total_timesteps 1923.
Path 219 | total_timesteps 1932.
Path 220 | total_timesteps 1939.
Path 221 | total_timesteps 1945.
Path 222 | total_timesteps 1956.
Path 223 | total_timesteps 1973.
Path 224 | total_timesteps 1980.
Path 225 | total_timesteps 1991.
Path 226 | total_timesteps 1999.
Path 227 | total_timesteps 2006.
Path 228 | total_timesteps 2014.
Path 229 | total_timesteps 2027.
Path 230 | total_timesteps 2034.
Path 231 | total_timesteps 2040.
Path 232 | total_timesteps 2048.
Path 233 | total_timesteps 2056.
Path 234 | total_timesteps 2062.
Path 235 | total_timesteps 2076.
Path 236 | total_timesteps 2086.
Path 237 | total_timesteps 2093.
Path 238 | total_timesteps 2102.
Path 239 | total_timesteps 2110.
Path 240 | total_timesteps 2120.
Path 241 | total_timesteps 2132.
Path 242 | total_timesteps 2140.
Path 243 | total_timesteps 2147.
Path 244 | total_timesteps 2155.
Path 245 | total_timesteps 2162.
Path 246 | total_timesteps 2170.
Path 247 | total_timesteps 2176.
Path 248 | total_timesteps 2184.
Path 249 | total_timesteps 2192.
Path 250 | total_timesteps 2200.
Path 251 | total_timesteps 2209.
Path 252 | total_timesteps 2218.
Path 253 | total_timesteps 2226.
Path 254 | total_timesteps 2234.
Path 255 | total_timesteps 2243.
Path 256 | total_timesteps 2252.
Path 257 | total_timesteps 2259.
Path 258 | total_timesteps 2266.
Path 259 | total_timesteps 2274.
Path 260 | total_timesteps 2291.
Path 261 | total_timesteps 2300.
Path 262 | total_timesteps 2308.
Path 263 | total_timesteps 2316.
Path 264 | total_timesteps 2324.
Path 265 | total_timesteps 2332.
Path 266 | total_timesteps 2340.
Path 267 | total_timesteps 2349.
Path 268 | total_timesteps 2357.
Path 269 | total_timesteps 2367.
Path 270 | total_timesteps 2376.
Path 271 | total_timesteps 2385.
Path 272 | total_timesteps 2395.
Path 273 | total_timesteps 2402.
Path 274 | total_timesteps 2411.
Path 275 | total_timesteps 2418.
Path 276 | total_timesteps 2425.
Path 277 | total_timesteps 2436.
Path 278 | total_timesteps 2445.
Path 279 | total_timesteps 2459.
Path 280 | total_timesteps 2467.
Path 281 | total_timesteps 2477.
Path 282 | total_timesteps 2484.
Path 283 | total_timesteps 2491.
Path 284 | total_timesteps 2498.
Path 285 | total_timesteps 2505.
Path 286 | total_timesteps 2514.
Path 287 | total_timesteps 2526.
Path 288 | total_timesteps 2536.
Path 289 | total_timesteps 2547.
Path 290 | total_timesteps 2558.
Path 291 | total_timesteps 2570.
Path 292 | total_timesteps 2580.
Path 293 | total_timesteps 2589.
Path 294 | total_timesteps 2597.
Path 295 | total_timesteps 2606.
Path 296 | total_timesteps 2613.
Path 297 | total_timesteps 2621.
Path 298 | total_timesteps 2628.
Path 299 | total_timesteps 2636.
Path 300 | total_timesteps 2645.
Path 301 | total_timesteps 2654.
Path 302 | total_timesteps 2662.
Path 303 | total_timesteps 2671.
Path 304 | total_timesteps 2690.
Path 305 | total_timesteps 2698.
Path 306 | total_timesteps 2705.
Path 307 | total_timesteps 2712.
Path 308 | total_timesteps 2720.
Path 309 | total_timesteps 2729.
Path 310 | total_timesteps 2738.
Path 311 | total_timesteps 2745.
Path 312 | total_timesteps 2753.
Path 313 | total_timesteps 2761.
Path 314 | total_timesteps 2773.
Path 315 | total_timesteps 2780.
Path 316 | total_timesteps 2788.
Path 317 | total_timesteps 2808.
Path 318 | total_timesteps 2815.
Path 319 | total_timesteps 2825.
Path 320 | total_timesteps 2833.
Path 321 | total_timesteps 2840.
Path 322 | total_timesteps 2849.
Path 323 | total_timesteps 2857.
Path 324 | total_timesteps 2864.
Path 325 | total_timesteps 2871.
Path 326 | total_timesteps 2882.
Path 327 | total_timesteps 2893.
Path 328 | total_timesteps 2902.
Path 329 | total_timesteps 2911.
Path 330 | total_timesteps 2920.
Path 331 | total_timesteps 2927.
Path 332 | total_timesteps 2933.
Path 333 | total_timesteps 2939.
Path 334 | total_timesteps 2946.
Path 335 | total_timesteps 2960.
Path 336 | total_timesteps 2968.
Path 337 | total_timesteps 2981.
Path 338 | total_timesteps 2993.
Path 339 | total_timesteps 3000.
Path 340 | total_timesteps 3009.
Path 341 | total_timesteps 3018.
Path 342 | total_timesteps 3026.
Path 343 | total_timesteps 3033.
Path 344 | total_timesteps 3041.
Path 345 | total_timesteps 3052.
Path 346 | total_timesteps 3062.
Path 347 | total_timesteps 3072.
Path 348 | total_timesteps 3079.
Path 349 | total_timesteps 3087.
Path 350 | total_timesteps 3098.
Path 351 | total_timesteps 3108.
Path 352 | total_timesteps 3117.
Path 353 | total_timesteps 3124.
Path 354 | total_timesteps 3134.
Path 355 | total_timesteps 3144.
Path 356 | total_timesteps 3151.
Path 357 | total_timesteps 3159.
Path 358 | total_timesteps 3167.
Path 359 | total_timesteps 3175.
Path 360 | total_timesteps 3184.
Path 361 | total_timesteps 3197.
Path 362 | total_timesteps 3206.
Path 363 | total_timesteps 3214.
Path 364 | total_timesteps 3221.
Path 365 | total_timesteps 3229.
Path 366 | total_timesteps 3237.
Path 367 | total_timesteps 3248.
Path 368 | total_timesteps 3259.
Path 369 | total_timesteps 3268.
Path 370 | total_timesteps 3276.
Path 371 | total_timesteps 3283.
Path 372 | total_timesteps 3290.
Path 373 | total_timesteps 3297.
Path 374 | total_timesteps 3312.
Path 375 | total_timesteps 3320.
Path 376 | total_timesteps 3330.
Path 377 | total_timesteps 3336.
Path 378 | total_timesteps 3343.
Path 379 | total_timesteps 3353.
Path 380 | total_timesteps 3360.
Path 381 | total_timesteps 3369.
Path 382 | total_timesteps 3377.
Path 383 | total_timesteps 3388.
Path 384 | total_timesteps 3397.
Path 385 | total_timesteps 3406.
Path 386 | total_timesteps 3415.
Path 387 | total_timesteps 3431.
Path 388 | total_timesteps 3449.
Path 389 | total_timesteps 3456.
Path 390 | total_timesteps 3466.
Path 391 | total_timesteps 3472.
Path 392 | total_timesteps 3478.
Path 393 | total_timesteps 3485.
Path 394 | total_timesteps 3494.
Path 395 | total_timesteps 3501.
Path 396 | total_timesteps 3507.
Path 397 | total_timesteps 3515.
Path 398 | total_timesteps 3523.
Path 399 | total_timesteps 3532.
Path 400 | total_timesteps 3540.
Path 401 | total_timesteps 3546.
Path 402 | total_timesteps 3555.
Path 403 | total_timesteps 3562.
Path 404 | total_timesteps 3570.
Path 405 | total_timesteps 3579.
Path 406 | total_timesteps 3590.
Path 407 | total_timesteps 3597.
Path 408 | total_timesteps 3608.
Path 409 | total_timesteps 3619.
Path 410 | total_timesteps 3627.
Path 411 | total_timesteps 3633.
Path 412 | total_timesteps 3640.
Path 413 | total_timesteps 3647.
Path 414 | total_timesteps 3653.
Path 415 | total_timesteps 3661.
Path 416 | total_timesteps 3669.
Path 417 | total_timesteps 3679.
Path 418 | total_timesteps 3686.
Path 419 | total_timesteps 3695.
Path 420 | total_timesteps 3702.
Path 421 | total_timesteps 3709.
Path 422 | total_timesteps 3716.
Path 423 | total_timesteps 3730.
Path 424 | total_timesteps 3738.
Path 425 | total_timesteps 3748.
Path 426 | total_timesteps 3758.
Path 427 | total_timesteps 3770.
Path 428 | total_timesteps 3779.
Path 429 | total_timesteps 3788.
Path 430 | total_timesteps 3794.
Path 431 | total_timesteps 3804.
Path 432 | total_timesteps 3811.
Path 433 | total_timesteps 3820.
Path 434 | total_timesteps 3831.
Path 435 | total_timesteps 3840.
Path 436 | total_timesteps 3854.
Path 437 | total_timesteps 3864.
Path 438 | total_timesteps 3873.
Path 439 | total_timesteps 3883.
Path 440 | total_timesteps 3890.
Path 441 | total_timesteps 3897.
Path 442 | total_timesteps 3905.
Path 443 | total_timesteps 3912.
Path 444 | total_timesteps 3921.
Path 445 | total_timesteps 3933.
Path 446 | total_timesteps 3941.
Path 447 | total_timesteps 3948.
Path 448 | total_timesteps 3956.
Path 449 | total_timesteps 3965.
Path 450 | total_timesteps 3973.
Path 451 | total_timesteps 3980.
Path 452 | total_timesteps 3989.
Path 453 | total_timesteps 3998.
Path 454 | total_timesteps 4014.
Path 455 | total_timesteps 4022.
Path 456 | total_timesteps 4034.
Path 457 | total_timesteps 4043.
Path 458 | total_timesteps 4051.
Path 459 | total_timesteps 4059.
Path 460 | total_timesteps 4069.
Path 461 | total_timesteps 4076.
Path 462 | total_timesteps 4085.
Path 463 | total_timesteps 4094.
Path 464 | total_timesteps 4104.
Path 465 | total_timesteps 4115.
Path 466 | total_timesteps 4125.
Path 467 | total_timesteps 4132.
Path 468 | total_timesteps 4139.
Path 469 | total_timesteps 4148.
Path 470 | total_timesteps 4162.
Path 471 | total_timesteps 4171.
Path 472 | total_timesteps 4180.
Path 473 | total_timesteps 4187.
Path 474 | total_timesteps 4196.
Path 475 | total_timesteps 4207.
Path 476 | total_timesteps 4213.
Path 477 | total_timesteps 4220.
Path 478 | total_timesteps 4229.
Path 479 | total_timesteps 4237.
Path 480 | total_timesteps 4249.
Path 481 | total_timesteps 4259.
Path 482 | total_timesteps 4268.
Path 483 | total_timesteps 4277.
Path 484 | total_timesteps 4285.
Path 485 | total_timesteps 4292.
Path 486 | total_timesteps 4300.
Path 487 | total_timesteps 4313.
Path 488 | total_timesteps 4322.
Path 489 | total_timesteps 4331.
Path 490 | total_timesteps 4343.
Path 491 | total_timesteps 4351.
Path 492 | total_timesteps 4358.
Path 493 | total_timesteps 4364.
Path 494 | total_timesteps 4370.
Path 495 | total_timesteps 4382.
Path 496 | total_timesteps 4389.
Path 497 | total_timesteps 4398.
Path 498 | total_timesteps 4404.
Path 499 | total_timesteps 4413.
Path 500 | total_timesteps 4424.
Path 501 | total_timesteps 4432.
Path 502 | total_timesteps 4440.
Path 503 | total_timesteps 4457.
Path 504 | total_timesteps 4467.
Path 505 | total_timesteps 4482.
Path 506 | total_timesteps 4491.
Path 507 | total_timesteps 4501.
Path 508 | total_timesteps 4512.
Path 509 | total_timesteps 4522.
Path 510 | total_timesteps 4531.
Path 511 | total_timesteps 4539.
Path 512 | total_timesteps 4548.
Path 513 | total_timesteps 4557.
Path 514 | total_timesteps 4564.
Path 515 | total_timesteps 4575.
Path 516 | total_timesteps 4581.
Path 517 | total_timesteps 4589.
Path 518 | total_timesteps 4597.
Path 519 | total_timesteps 4604.
Path 520 | total_timesteps 4613.
Path 521 | total_timesteps 4624.
Path 522 | total_timesteps 4631.
Path 523 | total_timesteps 4642.
Path 524 | total_timesteps 4649.
Path 525 | total_timesteps 4662.
Path 526 | total_timesteps 4674.
Path 527 | total_timesteps 4682.
Path 528 | total_timesteps 4688.
Path 529 | total_timesteps 4696.
Path 530 | total_timesteps 4702.
Path 531 | total_timesteps 4711.
Path 532 | total_timesteps 4719.
Path 533 | total_timesteps 4728.
Path 534 | total_timesteps 4735.
Path 535 | total_timesteps 4748.
Path 536 | total_timesteps 4765.
Path 537 | total_timesteps 4773.
Path 538 | total_timesteps 4780.
Path 539 | total_timesteps 4789.
Path 540 | total_timesteps 4796.
Path 541 | total_timesteps 4802.
Path 542 | total_timesteps 4810.
Path 543 | total_timesteps 4818.
Path 544 | total_timesteps 4825.
Path 545 | total_timesteps 4832.
Path 546 | total_timesteps 4839.
Path 547 | total_timesteps 4850.
Path 548 | total_timesteps 4857.
Path 549 | total_timesteps 4865.
Path 550 | total_timesteps 4873.
Path 551 | total_timesteps 4881.
Path 552 | total_timesteps 4890.
Path 553 | total_timesteps 4904.
Path 554 | total_timesteps 4913.
Path 555 | total_timesteps 4922.
Path 556 | total_timesteps 4930.
Path 557 | total_timesteps 4939.
Path 558 | total_timesteps 4947.
Path 559 | total_timesteps 4953.
Path 560 | total_timesteps 4961.
Path 561 | total_timesteps 4971.
Path 562 | total_timesteps 4983.
Path 563 | total_timesteps 4989.
Path 564 | total_timesteps 5006.
Path 565 | total_timesteps 5015.
Path 566 | total_timesteps 5022.
Path 567 | total_timesteps 5030.
Path 568 | total_timesteps 5038.
Path 569 | total_timesteps 5046.
Path 570 | total_timesteps 5055.
Path 571 | total_timesteps 5062.
Path 572 | total_timesteps 5069.
Path 573 | total_timesteps 5076.
Path 574 | total_timesteps 5086.
Path 575 | total_timesteps 5093.
Path 576 | total_timesteps 5104.
Path 577 | total_timesteps 5112.
Path 578 | total_timesteps 5122.
Path 579 | total_timesteps 5133.
Path 580 | total_timesteps 5141.
Path 581 | total_timesteps 5150.
Path 582 | total_timesteps 5158.
Path 583 | total_timesteps 5168.
Path 584 | total_timesteps 5174.
Path 585 | total_timesteps 5182.
Path 586 | total_timesteps 5190.
Path 587 | total_timesteps 5197.
Path 588 | total_timesteps 5209.
Path 589 | total_timesteps 5219.
Path 590 | total_timesteps 5225.
Path 591 | total_timesteps 5235.
Path 592 | total_timesteps 5244.
Path 593 | total_timesteps 5254.
Path 594 | total_timesteps 5262.
Path 595 | total_timesteps 5269.
Path 596 | total_timesteps 5276.
Path 597 | total_timesteps 5286.
Path 598 | total_timesteps 5292.
Path 599 | total_timesteps 5303.
Path 600 | total_timesteps 5312.
Path 601 | total_timesteps 5323.
Path 602 | total_timesteps 5336.
Path 603 | total_timesteps 5344.
Path 604 | total_timesteps 5351.
Path 605 | total_timesteps 5358.
Path 606 | total_timesteps 5364.
Path 607 | total_timesteps 5371.
Path 608 | total_timesteps 5378.
Path 609 | total_timesteps 5387.
Path 610 | total_timesteps 5397.
Path 611 | total_timesteps 5410.
Path 612 | total_timesteps 5425.
Path 613 | total_timesteps 5431.
Path 614 | total_timesteps 5440.
Path 615 | total_timesteps 5447.
Path 616 | total_timesteps 5456.
Path 617 | total_timesteps 5465.
Path 618 | total_timesteps 5475.
Path 619 | total_timesteps 5481.
Path 620 | total_timesteps 5488.
Path 621 | total_timesteps 5496.
Path 622 | total_timesteps 5503.
Path 623 | total_timesteps 5510.
Path 624 | total_timesteps 5517.
Path 625 | total_timesteps 5523.
Path 626 | total_timesteps 5531.
Path 627 | total_timesteps 5543.
Path 628 | total_timesteps 5552.
Path 629 | total_timesteps 5559.
Path 630 | total_timesteps 5567.
Path 631 | total_timesteps 5574.
Path 632 | total_timesteps 5582.
Path 633 | total_timesteps 5593.
Path 634 | total_timesteps 5599.
Path 635 | total_timesteps 5608.
Path 636 | total_timesteps 5616.
Path 637 | total_timesteps 5626.
Path 638 | total_timesteps 5632.
Path 639 | total_timesteps 5639.
Path 640 | total_timesteps 5646.
Path 641 | total_timesteps 5654.
Path 642 | total_timesteps 5661.
Path 643 | total_timesteps 5672.
Path 644 | total_timesteps 5685.
Path 645 | total_timesteps 5696.
Path 646 | total_timesteps 5706.
Path 647 | total_timesteps 5713.
Path 648 | total_timesteps 5721.
Path 649 | total_timesteps 5727.
Path 650 | total_timesteps 5737.
Path 651 | total_timesteps 5745.
Path 652 | total_timesteps 5752.
Path 653 | total_timesteps 5761.
Path 654 | total_timesteps 5775.
Path 655 | total_timesteps 5785.
Path 656 | total_timesteps 5793.
Path 657 | total_timesteps 5802.
Path 658 | total_timesteps 5818.
Path 659 | total_timesteps 5828.
Path 660 | total_timesteps 5836.
Path 661 | total_timesteps 5844.
Path 662 | total_timesteps 5855.
Path 663 | total_timesteps 5865.
Path 664 | total_timesteps 5873.
Path 665 | total_timesteps 5887.
Path 666 | total_timesteps 5894.
Path 667 | total_timesteps 5903.
Path 668 | total_timesteps 5913.
Path 669 | total_timesteps 5921.
Path 670 | total_timesteps 5928.
Path 671 | total_timesteps 5937.
Path 672 | total_timesteps 5944.
Path 673 | total_timesteps 5950.
Path 674 | total_timesteps 5957.
Path 675 | total_timesteps 5967.
Path 676 | total_timesteps 5978.
Path 677 | total_timesteps 5985.
Path 678 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.77    |
| Iteration     | 2        |
| MaximumReturn | 0.975    |
| MinimumReturn | -16.3    |
| TotalSamples  | 16032    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.03360016644001007
Validation loss = 0.018099278211593628
Validation loss = 0.01860804110765457
Validation loss = 0.01782141625881195
Validation loss = 0.018367018550634384
Validation loss = 0.0208579171448946
Validation loss = 0.016644222661852837
Validation loss = 0.018698371946811676
Validation loss = 0.016664257273077965
Validation loss = 0.016543161123991013
Validation loss = 0.017010949552059174
Validation loss = 0.017054252326488495
Validation loss = 0.0159483402967453
Validation loss = 0.018313337117433548
Validation loss = 0.01599719375371933
Validation loss = 0.015605520457029343
Validation loss = 0.015552759170532227
Validation loss = 0.016685251146554947
Validation loss = 0.018393561244010925
Validation loss = 0.01602477952837944
Validation loss = 0.016925912350416183
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 44.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 61.
Path 8 | total_timesteps 67.
Path 9 | total_timesteps 75.
Path 10 | total_timesteps 84.
Path 11 | total_timesteps 93.
Path 12 | total_timesteps 101.
Path 13 | total_timesteps 111.
Path 14 | total_timesteps 118.
Path 15 | total_timesteps 125.
Path 16 | total_timesteps 134.
Path 17 | total_timesteps 142.
Path 18 | total_timesteps 154.
Path 19 | total_timesteps 164.
Path 20 | total_timesteps 172.
Path 21 | total_timesteps 180.
Path 22 | total_timesteps 187.
Path 23 | total_timesteps 197.
Path 24 | total_timesteps 205.
Path 25 | total_timesteps 214.
Path 26 | total_timesteps 223.
Path 27 | total_timesteps 234.
Path 28 | total_timesteps 241.
Path 29 | total_timesteps 250.
Path 30 | total_timesteps 256.
Path 31 | total_timesteps 263.
Path 32 | total_timesteps 272.
Path 33 | total_timesteps 279.
Path 34 | total_timesteps 287.
Path 35 | total_timesteps 294.
Path 36 | total_timesteps 303.
Path 37 | total_timesteps 314.
Path 38 | total_timesteps 324.
Path 39 | total_timesteps 334.
Path 40 | total_timesteps 343.
Path 41 | total_timesteps 350.
Path 42 | total_timesteps 358.
Path 43 | total_timesteps 366.
Path 44 | total_timesteps 373.
Path 45 | total_timesteps 381.
Path 46 | total_timesteps 388.
Path 47 | total_timesteps 394.
Path 48 | total_timesteps 402.
Path 49 | total_timesteps 411.
Path 50 | total_timesteps 419.
Path 51 | total_timesteps 427.
Path 52 | total_timesteps 435.
Path 53 | total_timesteps 444.
Path 54 | total_timesteps 452.
Path 55 | total_timesteps 463.
Path 56 | total_timesteps 471.
Path 57 | total_timesteps 479.
Path 58 | total_timesteps 488.
Path 59 | total_timesteps 498.
Path 60 | total_timesteps 506.
Path 61 | total_timesteps 514.
Path 62 | total_timesteps 523.
Path 63 | total_timesteps 529.
Path 64 | total_timesteps 536.
Path 65 | total_timesteps 546.
Path 66 | total_timesteps 552.
Path 67 | total_timesteps 560.
Path 68 | total_timesteps 567.
Path 69 | total_timesteps 578.
Path 70 | total_timesteps 588.
Path 71 | total_timesteps 597.
Path 72 | total_timesteps 606.
Path 73 | total_timesteps 615.
Path 74 | total_timesteps 628.
Path 75 | total_timesteps 635.
Path 76 | total_timesteps 646.
Path 77 | total_timesteps 657.
Path 78 | total_timesteps 667.
Path 79 | total_timesteps 677.
Path 80 | total_timesteps 698.
Path 81 | total_timesteps 710.
Path 82 | total_timesteps 717.
Path 83 | total_timesteps 727.
Path 84 | total_timesteps 739.
Path 85 | total_timesteps 746.
Path 86 | total_timesteps 760.
Path 87 | total_timesteps 767.
Path 88 | total_timesteps 776.
Path 89 | total_timesteps 787.
Path 90 | total_timesteps 793.
Path 91 | total_timesteps 801.
Path 92 | total_timesteps 809.
Path 93 | total_timesteps 819.
Path 94 | total_timesteps 825.
Path 95 | total_timesteps 835.
Path 96 | total_timesteps 845.
Path 97 | total_timesteps 858.
Path 98 | total_timesteps 867.
Path 99 | total_timesteps 874.
Path 100 | total_timesteps 882.
Path 101 | total_timesteps 890.
Path 102 | total_timesteps 898.
Path 103 | total_timesteps 905.
Path 104 | total_timesteps 911.
Path 105 | total_timesteps 918.
Path 106 | total_timesteps 926.
Path 107 | total_timesteps 934.
Path 108 | total_timesteps 941.
Path 109 | total_timesteps 951.
Path 110 | total_timesteps 960.
Path 111 | total_timesteps 967.
Path 112 | total_timesteps 979.
Path 113 | total_timesteps 988.
Path 114 | total_timesteps 1000.
Path 115 | total_timesteps 1008.
Path 116 | total_timesteps 1019.
Path 117 | total_timesteps 1028.
Path 118 | total_timesteps 1037.
Path 119 | total_timesteps 1044.
Path 120 | total_timesteps 1054.
Path 121 | total_timesteps 1060.
Path 122 | total_timesteps 1074.
Path 123 | total_timesteps 1083.
Path 124 | total_timesteps 1093.
Path 125 | total_timesteps 1103.
Path 126 | total_timesteps 1111.
Path 127 | total_timesteps 1118.
Path 128 | total_timesteps 1126.
Path 129 | total_timesteps 1136.
Path 130 | total_timesteps 1144.
Path 131 | total_timesteps 1152.
Path 132 | total_timesteps 1159.
Path 133 | total_timesteps 1166.
Path 134 | total_timesteps 1176.
Path 135 | total_timesteps 1186.
Path 136 | total_timesteps 1194.
Path 137 | total_timesteps 1201.
Path 138 | total_timesteps 1209.
Path 139 | total_timesteps 1216.
Path 140 | total_timesteps 1226.
Path 141 | total_timesteps 1233.
Path 142 | total_timesteps 1244.
Path 143 | total_timesteps 1262.
Path 144 | total_timesteps 1272.
Path 145 | total_timesteps 1282.
Path 146 | total_timesteps 1291.
Path 147 | total_timesteps 1298.
Path 148 | total_timesteps 1310.
Path 149 | total_timesteps 1323.
Path 150 | total_timesteps 1334.
Path 151 | total_timesteps 1341.
Path 152 | total_timesteps 1349.
Path 153 | total_timesteps 1357.
Path 154 | total_timesteps 1370.
Path 155 | total_timesteps 1379.
Path 156 | total_timesteps 1388.
Path 157 | total_timesteps 1394.
Path 158 | total_timesteps 1401.
Path 159 | total_timesteps 1409.
Path 160 | total_timesteps 1416.
Path 161 | total_timesteps 1423.
Path 162 | total_timesteps 1430.
Path 163 | total_timesteps 1437.
Path 164 | total_timesteps 1446.
Path 165 | total_timesteps 1454.
Path 166 | total_timesteps 1462.
Path 167 | total_timesteps 1473.
Path 168 | total_timesteps 1481.
Path 169 | total_timesteps 1488.
Path 170 | total_timesteps 1496.
Path 171 | total_timesteps 1508.
Path 172 | total_timesteps 1519.
Path 173 | total_timesteps 1528.
Path 174 | total_timesteps 1539.
Path 175 | total_timesteps 1547.
Path 176 | total_timesteps 1554.
Path 177 | total_timesteps 1568.
Path 178 | total_timesteps 1575.
Path 179 | total_timesteps 1582.
Path 180 | total_timesteps 1590.
Path 181 | total_timesteps 1602.
Path 182 | total_timesteps 1609.
Path 183 | total_timesteps 1615.
Path 184 | total_timesteps 1622.
Path 185 | total_timesteps 1630.
Path 186 | total_timesteps 1639.
Path 187 | total_timesteps 1645.
Path 188 | total_timesteps 1652.
Path 189 | total_timesteps 1661.
Path 190 | total_timesteps 1670.
Path 191 | total_timesteps 1680.
Path 192 | total_timesteps 1691.
Path 193 | total_timesteps 1698.
Path 194 | total_timesteps 1710.
Path 195 | total_timesteps 1718.
Path 196 | total_timesteps 1725.
Path 197 | total_timesteps 1732.
Path 198 | total_timesteps 1739.
Path 199 | total_timesteps 1750.
Path 200 | total_timesteps 1759.
Path 201 | total_timesteps 1769.
Path 202 | total_timesteps 1777.
Path 203 | total_timesteps 1784.
Path 204 | total_timesteps 1790.
Path 205 | total_timesteps 1799.
Path 206 | total_timesteps 1813.
Path 207 | total_timesteps 1820.
Path 208 | total_timesteps 1832.
Path 209 | total_timesteps 1840.
Path 210 | total_timesteps 1847.
Path 211 | total_timesteps 1858.
Path 212 | total_timesteps 1866.
Path 213 | total_timesteps 1875.
Path 214 | total_timesteps 1885.
Path 215 | total_timesteps 1900.
Path 216 | total_timesteps 1908.
Path 217 | total_timesteps 1915.
Path 218 | total_timesteps 1926.
Path 219 | total_timesteps 1932.
Path 220 | total_timesteps 1939.
Path 221 | total_timesteps 1949.
Path 222 | total_timesteps 1957.
Path 223 | total_timesteps 1966.
Path 224 | total_timesteps 1979.
Path 225 | total_timesteps 1987.
Path 226 | total_timesteps 1998.
Path 227 | total_timesteps 2004.
Path 228 | total_timesteps 2012.
Path 229 | total_timesteps 2020.
Path 230 | total_timesteps 2027.
Path 231 | total_timesteps 2033.
Path 232 | total_timesteps 2040.
Path 233 | total_timesteps 2048.
Path 234 | total_timesteps 2056.
Path 235 | total_timesteps 2062.
Path 236 | total_timesteps 2071.
Path 237 | total_timesteps 2078.
Path 238 | total_timesteps 2086.
Path 239 | total_timesteps 2094.
Path 240 | total_timesteps 2106.
Path 241 | total_timesteps 2114.
Path 242 | total_timesteps 2124.
Path 243 | total_timesteps 2132.
Path 244 | total_timesteps 2138.
Path 245 | total_timesteps 2144.
Path 246 | total_timesteps 2154.
Path 247 | total_timesteps 2167.
Path 248 | total_timesteps 2174.
Path 249 | total_timesteps 2183.
Path 250 | total_timesteps 2192.
Path 251 | total_timesteps 2200.
Path 252 | total_timesteps 2208.
Path 253 | total_timesteps 2215.
Path 254 | total_timesteps 2223.
Path 255 | total_timesteps 2230.
Path 256 | total_timesteps 2240.
Path 257 | total_timesteps 2249.
Path 258 | total_timesteps 2262.
Path 259 | total_timesteps 2272.
Path 260 | total_timesteps 2281.
Path 261 | total_timesteps 2290.
Path 262 | total_timesteps 2297.
Path 263 | total_timesteps 2305.
Path 264 | total_timesteps 2314.
Path 265 | total_timesteps 2324.
Path 266 | total_timesteps 2331.
Path 267 | total_timesteps 2338.
Path 268 | total_timesteps 2348.
Path 269 | total_timesteps 2357.
Path 270 | total_timesteps 2364.
Path 271 | total_timesteps 2371.
Path 272 | total_timesteps 2377.
Path 273 | total_timesteps 2385.
Path 274 | total_timesteps 2394.
Path 275 | total_timesteps 2403.
Path 276 | total_timesteps 2411.
Path 277 | total_timesteps 2418.
Path 278 | total_timesteps 2425.
Path 279 | total_timesteps 2434.
Path 280 | total_timesteps 2443.
Path 281 | total_timesteps 2453.
Path 282 | total_timesteps 2464.
Path 283 | total_timesteps 2474.
Path 284 | total_timesteps 2482.
Path 285 | total_timesteps 2488.
Path 286 | total_timesteps 2498.
Path 287 | total_timesteps 2509.
Path 288 | total_timesteps 2516.
Path 289 | total_timesteps 2523.
Path 290 | total_timesteps 2532.
Path 291 | total_timesteps 2541.
Path 292 | total_timesteps 2549.
Path 293 | total_timesteps 2555.
Path 294 | total_timesteps 2566.
Path 295 | total_timesteps 2575.
Path 296 | total_timesteps 2582.
Path 297 | total_timesteps 2594.
Path 298 | total_timesteps 2602.
Path 299 | total_timesteps 2609.
Path 300 | total_timesteps 2620.
Path 301 | total_timesteps 2629.
Path 302 | total_timesteps 2637.
Path 303 | total_timesteps 2645.
Path 304 | total_timesteps 2656.
Path 305 | total_timesteps 2664.
Path 306 | total_timesteps 2678.
Path 307 | total_timesteps 2687.
Path 308 | total_timesteps 2693.
Path 309 | total_timesteps 2701.
Path 310 | total_timesteps 2711.
Path 311 | total_timesteps 2719.
Path 312 | total_timesteps 2729.
Path 313 | total_timesteps 2740.
Path 314 | total_timesteps 2750.
Path 315 | total_timesteps 2763.
Path 316 | total_timesteps 2773.
Path 317 | total_timesteps 2784.
Path 318 | total_timesteps 2795.
Path 319 | total_timesteps 2810.
Path 320 | total_timesteps 2817.
Path 321 | total_timesteps 2824.
Path 322 | total_timesteps 2834.
Path 323 | total_timesteps 2844.
Path 324 | total_timesteps 2856.
Path 325 | total_timesteps 2862.
Path 326 | total_timesteps 2871.
Path 327 | total_timesteps 2884.
Path 328 | total_timesteps 2892.
Path 329 | total_timesteps 2904.
Path 330 | total_timesteps 2914.
Path 331 | total_timesteps 2922.
Path 332 | total_timesteps 2928.
Path 333 | total_timesteps 2940.
Path 334 | total_timesteps 2946.
Path 335 | total_timesteps 2953.
Path 336 | total_timesteps 2962.
Path 337 | total_timesteps 2973.
Path 338 | total_timesteps 2980.
Path 339 | total_timesteps 2990.
Path 340 | total_timesteps 3002.
Path 341 | total_timesteps 3010.
Path 342 | total_timesteps 3017.
Path 343 | total_timesteps 3025.
Path 344 | total_timesteps 3035.
Path 345 | total_timesteps 3044.
Path 346 | total_timesteps 3054.
Path 347 | total_timesteps 3069.
Path 348 | total_timesteps 3080.
Path 349 | total_timesteps 3091.
Path 350 | total_timesteps 3102.
Path 351 | total_timesteps 3112.
Path 352 | total_timesteps 3122.
Path 353 | total_timesteps 3130.
Path 354 | total_timesteps 3141.
Path 355 | total_timesteps 3148.
Path 356 | total_timesteps 3155.
Path 357 | total_timesteps 3162.
Path 358 | total_timesteps 3168.
Path 359 | total_timesteps 3180.
Path 360 | total_timesteps 3188.
Path 361 | total_timesteps 3203.
Path 362 | total_timesteps 3215.
Path 363 | total_timesteps 3225.
Path 364 | total_timesteps 3231.
Path 365 | total_timesteps 3239.
Path 366 | total_timesteps 3249.
Path 367 | total_timesteps 3259.
Path 368 | total_timesteps 3270.
Path 369 | total_timesteps 3277.
Path 370 | total_timesteps 3287.
Path 371 | total_timesteps 3297.
Path 372 | total_timesteps 3303.
Path 373 | total_timesteps 3310.
Path 374 | total_timesteps 3317.
Path 375 | total_timesteps 3325.
Path 376 | total_timesteps 3332.
Path 377 | total_timesteps 3339.
Path 378 | total_timesteps 3350.
Path 379 | total_timesteps 3359.
Path 380 | total_timesteps 3368.
Path 381 | total_timesteps 3377.
Path 382 | total_timesteps 3384.
Path 383 | total_timesteps 3397.
Path 384 | total_timesteps 3405.
Path 385 | total_timesteps 3415.
Path 386 | total_timesteps 3427.
Path 387 | total_timesteps 3434.
Path 388 | total_timesteps 3444.
Path 389 | total_timesteps 3452.
Path 390 | total_timesteps 3461.
Path 391 | total_timesteps 3470.
Path 392 | total_timesteps 3481.
Path 393 | total_timesteps 3493.
Path 394 | total_timesteps 3505.
Path 395 | total_timesteps 3516.
Path 396 | total_timesteps 3525.
Path 397 | total_timesteps 3531.
Path 398 | total_timesteps 3540.
Path 399 | total_timesteps 3550.
Path 400 | total_timesteps 3563.
Path 401 | total_timesteps 3571.
Path 402 | total_timesteps 3581.
Path 403 | total_timesteps 3587.
Path 404 | total_timesteps 3594.
Path 405 | total_timesteps 3601.
Path 406 | total_timesteps 3608.
Path 407 | total_timesteps 3616.
Path 408 | total_timesteps 3625.
Path 409 | total_timesteps 3632.
Path 410 | total_timesteps 3638.
Path 411 | total_timesteps 3645.
Path 412 | total_timesteps 3653.
Path 413 | total_timesteps 3665.
Path 414 | total_timesteps 3673.
Path 415 | total_timesteps 3683.
Path 416 | total_timesteps 3690.
Path 417 | total_timesteps 3698.
Path 418 | total_timesteps 3706.
Path 419 | total_timesteps 3723.
Path 420 | total_timesteps 3731.
Path 421 | total_timesteps 3738.
Path 422 | total_timesteps 3747.
Path 423 | total_timesteps 3756.
Path 424 | total_timesteps 3763.
Path 425 | total_timesteps 3774.
Path 426 | total_timesteps 3785.
Path 427 | total_timesteps 3793.
Path 428 | total_timesteps 3802.
Path 429 | total_timesteps 3813.
Path 430 | total_timesteps 3827.
Path 431 | total_timesteps 3837.
Path 432 | total_timesteps 3847.
Path 433 | total_timesteps 3855.
Path 434 | total_timesteps 3863.
Path 435 | total_timesteps 3878.
Path 436 | total_timesteps 3886.
Path 437 | total_timesteps 3901.
Path 438 | total_timesteps 3907.
Path 439 | total_timesteps 3916.
Path 440 | total_timesteps 3924.
Path 441 | total_timesteps 3934.
Path 442 | total_timesteps 3945.
Path 443 | total_timesteps 3952.
Path 444 | total_timesteps 3962.
Path 445 | total_timesteps 3970.
Path 446 | total_timesteps 3980.
Path 447 | total_timesteps 3990.
Path 448 | total_timesteps 3998.
Path 449 | total_timesteps 4006.
Path 450 | total_timesteps 4015.
Path 451 | total_timesteps 4023.
Path 452 | total_timesteps 4030.
Path 453 | total_timesteps 4039.
Path 454 | total_timesteps 4049.
Path 455 | total_timesteps 4057.
Path 456 | total_timesteps 4066.
Path 457 | total_timesteps 4074.
Path 458 | total_timesteps 4082.
Path 459 | total_timesteps 4088.
Path 460 | total_timesteps 4097.
Path 461 | total_timesteps 4103.
Path 462 | total_timesteps 4112.
Path 463 | total_timesteps 4119.
Path 464 | total_timesteps 4131.
Path 465 | total_timesteps 4139.
Path 466 | total_timesteps 4147.
Path 467 | total_timesteps 4153.
Path 468 | total_timesteps 4161.
Path 469 | total_timesteps 4168.
Path 470 | total_timesteps 4177.
Path 471 | total_timesteps 4189.
Path 472 | total_timesteps 4198.
Path 473 | total_timesteps 4207.
Path 474 | total_timesteps 4214.
Path 475 | total_timesteps 4221.
Path 476 | total_timesteps 4230.
Path 477 | total_timesteps 4237.
Path 478 | total_timesteps 4243.
Path 479 | total_timesteps 4256.
Path 480 | total_timesteps 4263.
Path 481 | total_timesteps 4271.
Path 482 | total_timesteps 4279.
Path 483 | total_timesteps 4285.
Path 484 | total_timesteps 4294.
Path 485 | total_timesteps 4300.
Path 486 | total_timesteps 4310.
Path 487 | total_timesteps 4317.
Path 488 | total_timesteps 4325.
Path 489 | total_timesteps 4332.
Path 490 | total_timesteps 4340.
Path 491 | total_timesteps 4350.
Path 492 | total_timesteps 4360.
Path 493 | total_timesteps 4367.
Path 494 | total_timesteps 4375.
Path 495 | total_timesteps 4382.
Path 496 | total_timesteps 4394.
Path 497 | total_timesteps 4404.
Path 498 | total_timesteps 4411.
Path 499 | total_timesteps 4420.
Path 500 | total_timesteps 4430.
Path 501 | total_timesteps 4436.
Path 502 | total_timesteps 4445.
Path 503 | total_timesteps 4454.
Path 504 | total_timesteps 4464.
Path 505 | total_timesteps 4473.
Path 506 | total_timesteps 4483.
Path 507 | total_timesteps 4490.
Path 508 | total_timesteps 4500.
Path 509 | total_timesteps 4506.
Path 510 | total_timesteps 4515.
Path 511 | total_timesteps 4523.
Path 512 | total_timesteps 4531.
Path 513 | total_timesteps 4544.
Path 514 | total_timesteps 4553.
Path 515 | total_timesteps 4562.
Path 516 | total_timesteps 4570.
Path 517 | total_timesteps 4576.
Path 518 | total_timesteps 4582.
Path 519 | total_timesteps 4595.
Path 520 | total_timesteps 4605.
Path 521 | total_timesteps 4612.
Path 522 | total_timesteps 4621.
Path 523 | total_timesteps 4638.
Path 524 | total_timesteps 4653.
Path 525 | total_timesteps 4661.
Path 526 | total_timesteps 4670.
Path 527 | total_timesteps 4677.
Path 528 | total_timesteps 4685.
Path 529 | total_timesteps 4696.
Path 530 | total_timesteps 4704.
Path 531 | total_timesteps 4714.
Path 532 | total_timesteps 4721.
Path 533 | total_timesteps 4729.
Path 534 | total_timesteps 4737.
Path 535 | total_timesteps 4744.
Path 536 | total_timesteps 4753.
Path 537 | total_timesteps 4763.
Path 538 | total_timesteps 4779.
Path 539 | total_timesteps 4787.
Path 540 | total_timesteps 4795.
Path 541 | total_timesteps 4803.
Path 542 | total_timesteps 4810.
Path 543 | total_timesteps 4818.
Path 544 | total_timesteps 4826.
Path 545 | total_timesteps 4832.
Path 546 | total_timesteps 4841.
Path 547 | total_timesteps 4852.
Path 548 | total_timesteps 4862.
Path 549 | total_timesteps 4871.
Path 550 | total_timesteps 4879.
Path 551 | total_timesteps 4889.
Path 552 | total_timesteps 4896.
Path 553 | total_timesteps 4907.
Path 554 | total_timesteps 4917.
Path 555 | total_timesteps 4924.
Path 556 | total_timesteps 4932.
Path 557 | total_timesteps 4939.
Path 558 | total_timesteps 4951.
Path 559 | total_timesteps 4962.
Path 560 | total_timesteps 4970.
Path 561 | total_timesteps 4979.
Path 562 | total_timesteps 4990.
Path 563 | total_timesteps 4999.
Path 564 | total_timesteps 5006.
Path 565 | total_timesteps 5013.
Path 566 | total_timesteps 5026.
Path 567 | total_timesteps 5036.
Path 568 | total_timesteps 5046.
Path 569 | total_timesteps 5053.
Path 570 | total_timesteps 5060.
Path 571 | total_timesteps 5068.
Path 572 | total_timesteps 5079.
Path 573 | total_timesteps 5088.
Path 574 | total_timesteps 5097.
Path 575 | total_timesteps 5107.
Path 576 | total_timesteps 5116.
Path 577 | total_timesteps 5125.
Path 578 | total_timesteps 5131.
Path 579 | total_timesteps 5139.
Path 580 | total_timesteps 5152.
Path 581 | total_timesteps 5160.
Path 582 | total_timesteps 5175.
Path 583 | total_timesteps 5185.
Path 584 | total_timesteps 5200.
Path 585 | total_timesteps 5208.
Path 586 | total_timesteps 5216.
Path 587 | total_timesteps 5225.
Path 588 | total_timesteps 5233.
Path 589 | total_timesteps 5242.
Path 590 | total_timesteps 5249.
Path 591 | total_timesteps 5257.
Path 592 | total_timesteps 5274.
Path 593 | total_timesteps 5282.
Path 594 | total_timesteps 5290.
Path 595 | total_timesteps 5299.
Path 596 | total_timesteps 5308.
Path 597 | total_timesteps 5314.
Path 598 | total_timesteps 5324.
Path 599 | total_timesteps 5331.
Path 600 | total_timesteps 5339.
Path 601 | total_timesteps 5346.
Path 602 | total_timesteps 5353.
Path 603 | total_timesteps 5361.
Path 604 | total_timesteps 5368.
Path 605 | total_timesteps 5376.
Path 606 | total_timesteps 5387.
Path 607 | total_timesteps 5394.
Path 608 | total_timesteps 5401.
Path 609 | total_timesteps 5410.
Path 610 | total_timesteps 5417.
Path 611 | total_timesteps 5428.
Path 612 | total_timesteps 5437.
Path 613 | total_timesteps 5446.
Path 614 | total_timesteps 5456.
Path 615 | total_timesteps 5465.
Path 616 | total_timesteps 5475.
Path 617 | total_timesteps 5485.
Path 618 | total_timesteps 5494.
Path 619 | total_timesteps 5501.
Path 620 | total_timesteps 5512.
Path 621 | total_timesteps 5519.
Path 622 | total_timesteps 5526.
Path 623 | total_timesteps 5538.
Path 624 | total_timesteps 5544.
Path 625 | total_timesteps 5550.
Path 626 | total_timesteps 5567.
Path 627 | total_timesteps 5574.
Path 628 | total_timesteps 5581.
Path 629 | total_timesteps 5590.
Path 630 | total_timesteps 5597.
Path 631 | total_timesteps 5604.
Path 632 | total_timesteps 5615.
Path 633 | total_timesteps 5623.
Path 634 | total_timesteps 5633.
Path 635 | total_timesteps 5642.
Path 636 | total_timesteps 5649.
Path 637 | total_timesteps 5656.
Path 638 | total_timesteps 5665.
Path 639 | total_timesteps 5672.
Path 640 | total_timesteps 5684.
Path 641 | total_timesteps 5694.
Path 642 | total_timesteps 5703.
Path 643 | total_timesteps 5710.
Path 644 | total_timesteps 5718.
Path 645 | total_timesteps 5730.
Path 646 | total_timesteps 5736.
Path 647 | total_timesteps 5745.
Path 648 | total_timesteps 5755.
Path 649 | total_timesteps 5763.
Path 650 | total_timesteps 5770.
Path 651 | total_timesteps 5782.
Path 652 | total_timesteps 5791.
Path 653 | total_timesteps 5806.
Path 654 | total_timesteps 5815.
Path 655 | total_timesteps 5824.
Path 656 | total_timesteps 5831.
Path 657 | total_timesteps 5839.
Path 658 | total_timesteps 5848.
Path 659 | total_timesteps 5856.
Path 660 | total_timesteps 5864.
Path 661 | total_timesteps 5872.
Path 662 | total_timesteps 5882.
Path 663 | total_timesteps 5891.
Path 664 | total_timesteps 5906.
Path 665 | total_timesteps 5922.
Path 666 | total_timesteps 5930.
Path 667 | total_timesteps 5937.
Path 668 | total_timesteps 5948.
Path 669 | total_timesteps 5958.
Path 670 | total_timesteps 5966.
Path 671 | total_timesteps 5975.
Path 672 | total_timesteps 5982.
Path 673 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.98    |
| Iteration     | 3        |
| MaximumReturn | -1.62    |
| MinimumReturn | -17.6    |
| TotalSamples  | 20038    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.02170431986451149
Validation loss = 0.012245623394846916
Validation loss = 0.013159302994608879
Validation loss = 0.013488766737282276
Validation loss = 0.01375650055706501
Validation loss = 0.014690035954117775
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 73.
Path 9 | total_timesteps 82.
Path 10 | total_timesteps 95.
Path 11 | total_timesteps 103.
Path 12 | total_timesteps 113.
Path 13 | total_timesteps 122.
Path 14 | total_timesteps 133.
Path 15 | total_timesteps 142.
Path 16 | total_timesteps 154.
Path 17 | total_timesteps 160.
Path 18 | total_timesteps 170.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 189.
Path 21 | total_timesteps 201.
Path 22 | total_timesteps 209.
Path 23 | total_timesteps 221.
Path 24 | total_timesteps 229.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 245.
Path 27 | total_timesteps 254.
Path 28 | total_timesteps 267.
Path 29 | total_timesteps 276.
Path 30 | total_timesteps 283.
Path 31 | total_timesteps 292.
Path 32 | total_timesteps 300.
Path 33 | total_timesteps 311.
Path 34 | total_timesteps 319.
Path 35 | total_timesteps 327.
Path 36 | total_timesteps 337.
Path 37 | total_timesteps 345.
Path 38 | total_timesteps 354.
Path 39 | total_timesteps 362.
Path 40 | total_timesteps 375.
Path 41 | total_timesteps 382.
Path 42 | total_timesteps 389.
Path 43 | total_timesteps 400.
Path 44 | total_timesteps 408.
Path 45 | total_timesteps 415.
Path 46 | total_timesteps 423.
Path 47 | total_timesteps 436.
Path 48 | total_timesteps 443.
Path 49 | total_timesteps 452.
Path 50 | total_timesteps 459.
Path 51 | total_timesteps 467.
Path 52 | total_timesteps 474.
Path 53 | total_timesteps 482.
Path 54 | total_timesteps 488.
Path 55 | total_timesteps 498.
Path 56 | total_timesteps 505.
Path 57 | total_timesteps 512.
Path 58 | total_timesteps 520.
Path 59 | total_timesteps 527.
Path 60 | total_timesteps 538.
Path 61 | total_timesteps 550.
Path 62 | total_timesteps 560.
Path 63 | total_timesteps 567.
Path 64 | total_timesteps 573.
Path 65 | total_timesteps 582.
Path 66 | total_timesteps 595.
Path 67 | total_timesteps 602.
Path 68 | total_timesteps 611.
Path 69 | total_timesteps 618.
Path 70 | total_timesteps 629.
Path 71 | total_timesteps 638.
Path 72 | total_timesteps 648.
Path 73 | total_timesteps 657.
Path 74 | total_timesteps 664.
Path 75 | total_timesteps 672.
Path 76 | total_timesteps 683.
Path 77 | total_timesteps 691.
Path 78 | total_timesteps 698.
Path 79 | total_timesteps 705.
Path 80 | total_timesteps 714.
Path 81 | total_timesteps 722.
Path 82 | total_timesteps 729.
Path 83 | total_timesteps 735.
Path 84 | total_timesteps 741.
Path 85 | total_timesteps 759.
Path 86 | total_timesteps 767.
Path 87 | total_timesteps 775.
Path 88 | total_timesteps 784.
Path 89 | total_timesteps 797.
Path 90 | total_timesteps 806.
Path 91 | total_timesteps 814.
Path 92 | total_timesteps 821.
Path 93 | total_timesteps 828.
Path 94 | total_timesteps 837.
Path 95 | total_timesteps 846.
Path 96 | total_timesteps 856.
Path 97 | total_timesteps 868.
Path 98 | total_timesteps 875.
Path 99 | total_timesteps 882.
Path 100 | total_timesteps 891.
Path 101 | total_timesteps 898.
Path 102 | total_timesteps 907.
Path 103 | total_timesteps 914.
Path 104 | total_timesteps 921.
Path 105 | total_timesteps 934.
Path 106 | total_timesteps 943.
Path 107 | total_timesteps 958.
Path 108 | total_timesteps 966.
Path 109 | total_timesteps 973.
Path 110 | total_timesteps 984.
Path 111 | total_timesteps 994.
Path 112 | total_timesteps 1001.
Path 113 | total_timesteps 1008.
Path 114 | total_timesteps 1020.
Path 115 | total_timesteps 1027.
Path 116 | total_timesteps 1033.
Path 117 | total_timesteps 1044.
Path 118 | total_timesteps 1050.
Path 119 | total_timesteps 1060.
Path 120 | total_timesteps 1067.
Path 121 | total_timesteps 1076.
Path 122 | total_timesteps 1083.
Path 123 | total_timesteps 1090.
Path 124 | total_timesteps 1101.
Path 125 | total_timesteps 1109.
Path 126 | total_timesteps 1120.
Path 127 | total_timesteps 1130.
Path 128 | total_timesteps 1137.
Path 129 | total_timesteps 1150.
Path 130 | total_timesteps 1158.
Path 131 | total_timesteps 1168.
Path 132 | total_timesteps 1176.
Path 133 | total_timesteps 1184.
Path 134 | total_timesteps 1193.
Path 135 | total_timesteps 1202.
Path 136 | total_timesteps 1210.
Path 137 | total_timesteps 1217.
Path 138 | total_timesteps 1225.
Path 139 | total_timesteps 1233.
Path 140 | total_timesteps 1241.
Path 141 | total_timesteps 1250.
Path 142 | total_timesteps 1261.
Path 143 | total_timesteps 1270.
Path 144 | total_timesteps 1279.
Path 145 | total_timesteps 1287.
Path 146 | total_timesteps 1296.
Path 147 | total_timesteps 1303.
Path 148 | total_timesteps 1312.
Path 149 | total_timesteps 1321.
Path 150 | total_timesteps 1332.
Path 151 | total_timesteps 1341.
Path 152 | total_timesteps 1349.
Path 153 | total_timesteps 1356.
Path 154 | total_timesteps 1364.
Path 155 | total_timesteps 1373.
Path 156 | total_timesteps 1382.
Path 157 | total_timesteps 1391.
Path 158 | total_timesteps 1397.
Path 159 | total_timesteps 1403.
Path 160 | total_timesteps 1412.
Path 161 | total_timesteps 1420.
Path 162 | total_timesteps 1434.
Path 163 | total_timesteps 1440.
Path 164 | total_timesteps 1453.
Path 165 | total_timesteps 1461.
Path 166 | total_timesteps 1470.
Path 167 | total_timesteps 1476.
Path 168 | total_timesteps 1485.
Path 169 | total_timesteps 1493.
Path 170 | total_timesteps 1500.
Path 171 | total_timesteps 1507.
Path 172 | total_timesteps 1516.
Path 173 | total_timesteps 1523.
Path 174 | total_timesteps 1531.
Path 175 | total_timesteps 1538.
Path 176 | total_timesteps 1544.
Path 177 | total_timesteps 1553.
Path 178 | total_timesteps 1560.
Path 179 | total_timesteps 1567.
Path 180 | total_timesteps 1574.
Path 181 | total_timesteps 1580.
Path 182 | total_timesteps 1589.
Path 183 | total_timesteps 1595.
Path 184 | total_timesteps 1608.
Path 185 | total_timesteps 1616.
Path 186 | total_timesteps 1623.
Path 187 | total_timesteps 1632.
Path 188 | total_timesteps 1640.
Path 189 | total_timesteps 1646.
Path 190 | total_timesteps 1653.
Path 191 | total_timesteps 1662.
Path 192 | total_timesteps 1669.
Path 193 | total_timesteps 1677.
Path 194 | total_timesteps 1685.
Path 195 | total_timesteps 1695.
Path 196 | total_timesteps 1707.
Path 197 | total_timesteps 1714.
Path 198 | total_timesteps 1724.
Path 199 | total_timesteps 1735.
Path 200 | total_timesteps 1742.
Path 201 | total_timesteps 1753.
Path 202 | total_timesteps 1763.
Path 203 | total_timesteps 1772.
Path 204 | total_timesteps 1780.
Path 205 | total_timesteps 1794.
Path 206 | total_timesteps 1802.
Path 207 | total_timesteps 1809.
Path 208 | total_timesteps 1817.
Path 209 | total_timesteps 1825.
Path 210 | total_timesteps 1835.
Path 211 | total_timesteps 1843.
Path 212 | total_timesteps 1850.
Path 213 | total_timesteps 1863.
Path 214 | total_timesteps 1874.
Path 215 | total_timesteps 1882.
Path 216 | total_timesteps 1891.
Path 217 | total_timesteps 1903.
Path 218 | total_timesteps 1911.
Path 219 | total_timesteps 1919.
Path 220 | total_timesteps 1926.
Path 221 | total_timesteps 1934.
Path 222 | total_timesteps 1944.
Path 223 | total_timesteps 1952.
Path 224 | total_timesteps 1958.
Path 225 | total_timesteps 1970.
Path 226 | total_timesteps 1989.
Path 227 | total_timesteps 1999.
Path 228 | total_timesteps 2010.
Path 229 | total_timesteps 2018.
Path 230 | total_timesteps 2026.
Path 231 | total_timesteps 2034.
Path 232 | total_timesteps 2044.
Path 233 | total_timesteps 2051.
Path 234 | total_timesteps 2059.
Path 235 | total_timesteps 2065.
Path 236 | total_timesteps 2074.
Path 237 | total_timesteps 2087.
Path 238 | total_timesteps 2096.
Path 239 | total_timesteps 2105.
Path 240 | total_timesteps 2115.
Path 241 | total_timesteps 2125.
Path 242 | total_timesteps 2133.
Path 243 | total_timesteps 2141.
Path 244 | total_timesteps 2149.
Path 245 | total_timesteps 2158.
Path 246 | total_timesteps 2168.
Path 247 | total_timesteps 2176.
Path 248 | total_timesteps 2183.
Path 249 | total_timesteps 2193.
Path 250 | total_timesteps 2201.
Path 251 | total_timesteps 2210.
Path 252 | total_timesteps 2219.
Path 253 | total_timesteps 2229.
Path 254 | total_timesteps 2239.
Path 255 | total_timesteps 2248.
Path 256 | total_timesteps 2258.
Path 257 | total_timesteps 2267.
Path 258 | total_timesteps 2277.
Path 259 | total_timesteps 2285.
Path 260 | total_timesteps 2293.
Path 261 | total_timesteps 2303.
Path 262 | total_timesteps 2311.
Path 263 | total_timesteps 2320.
Path 264 | total_timesteps 2331.
Path 265 | total_timesteps 2343.
Path 266 | total_timesteps 2352.
Path 267 | total_timesteps 2360.
Path 268 | total_timesteps 2368.
Path 269 | total_timesteps 2377.
Path 270 | total_timesteps 2385.
Path 271 | total_timesteps 2396.
Path 272 | total_timesteps 2406.
Path 273 | total_timesteps 2413.
Path 274 | total_timesteps 2427.
Path 275 | total_timesteps 2436.
Path 276 | total_timesteps 2444.
Path 277 | total_timesteps 2454.
Path 278 | total_timesteps 2461.
Path 279 | total_timesteps 2474.
Path 280 | total_timesteps 2482.
Path 281 | total_timesteps 2491.
Path 282 | total_timesteps 2499.
Path 283 | total_timesteps 2510.
Path 284 | total_timesteps 2520.
Path 285 | total_timesteps 2530.
Path 286 | total_timesteps 2537.
Path 287 | total_timesteps 2550.
Path 288 | total_timesteps 2560.
Path 289 | total_timesteps 2570.
Path 290 | total_timesteps 2579.
Path 291 | total_timesteps 2586.
Path 292 | total_timesteps 2595.
Path 293 | total_timesteps 2601.
Path 294 | total_timesteps 2608.
Path 295 | total_timesteps 2619.
Path 296 | total_timesteps 2628.
Path 297 | total_timesteps 2636.
Path 298 | total_timesteps 2645.
Path 299 | total_timesteps 2652.
Path 300 | total_timesteps 2662.
Path 301 | total_timesteps 2669.
Path 302 | total_timesteps 2684.
Path 303 | total_timesteps 2693.
Path 304 | total_timesteps 2702.
Path 305 | total_timesteps 2715.
Path 306 | total_timesteps 2722.
Path 307 | total_timesteps 2732.
Path 308 | total_timesteps 2740.
Path 309 | total_timesteps 2750.
Path 310 | total_timesteps 2757.
Path 311 | total_timesteps 2764.
Path 312 | total_timesteps 2772.
Path 313 | total_timesteps 2779.
Path 314 | total_timesteps 2786.
Path 315 | total_timesteps 2798.
Path 316 | total_timesteps 2806.
Path 317 | total_timesteps 2815.
Path 318 | total_timesteps 2823.
Path 319 | total_timesteps 2834.
Path 320 | total_timesteps 2840.
Path 321 | total_timesteps 2853.
Path 322 | total_timesteps 2861.
Path 323 | total_timesteps 2872.
Path 324 | total_timesteps 2882.
Path 325 | total_timesteps 2889.
Path 326 | total_timesteps 2898.
Path 327 | total_timesteps 2906.
Path 328 | total_timesteps 2920.
Path 329 | total_timesteps 2928.
Path 330 | total_timesteps 2936.
Path 331 | total_timesteps 2944.
Path 332 | total_timesteps 2951.
Path 333 | total_timesteps 2964.
Path 334 | total_timesteps 2974.
Path 335 | total_timesteps 2983.
Path 336 | total_timesteps 2993.
Path 337 | total_timesteps 2999.
Path 338 | total_timesteps 3006.
Path 339 | total_timesteps 3016.
Path 340 | total_timesteps 3028.
Path 341 | total_timesteps 3037.
Path 342 | total_timesteps 3044.
Path 343 | total_timesteps 3051.
Path 344 | total_timesteps 3059.
Path 345 | total_timesteps 3067.
Path 346 | total_timesteps 3074.
Path 347 | total_timesteps 3084.
Path 348 | total_timesteps 3092.
Path 349 | total_timesteps 3098.
Path 350 | total_timesteps 3106.
Path 351 | total_timesteps 3113.
Path 352 | total_timesteps 3123.
Path 353 | total_timesteps 3131.
Path 354 | total_timesteps 3139.
Path 355 | total_timesteps 3145.
Path 356 | total_timesteps 3154.
Path 357 | total_timesteps 3162.
Path 358 | total_timesteps 3171.
Path 359 | total_timesteps 3179.
Path 360 | total_timesteps 3187.
Path 361 | total_timesteps 3196.
Path 362 | total_timesteps 3206.
Path 363 | total_timesteps 3213.
Path 364 | total_timesteps 3225.
Path 365 | total_timesteps 3232.
Path 366 | total_timesteps 3242.
Path 367 | total_timesteps 3250.
Path 368 | total_timesteps 3257.
Path 369 | total_timesteps 3264.
Path 370 | total_timesteps 3275.
Path 371 | total_timesteps 3283.
Path 372 | total_timesteps 3290.
Path 373 | total_timesteps 3300.
Path 374 | total_timesteps 3307.
Path 375 | total_timesteps 3317.
Path 376 | total_timesteps 3325.
Path 377 | total_timesteps 3337.
Path 378 | total_timesteps 3344.
Path 379 | total_timesteps 3356.
Path 380 | total_timesteps 3364.
Path 381 | total_timesteps 3372.
Path 382 | total_timesteps 3378.
Path 383 | total_timesteps 3386.
Path 384 | total_timesteps 3393.
Path 385 | total_timesteps 3399.
Path 386 | total_timesteps 3409.
Path 387 | total_timesteps 3417.
Path 388 | total_timesteps 3424.
Path 389 | total_timesteps 3433.
Path 390 | total_timesteps 3442.
Path 391 | total_timesteps 3449.
Path 392 | total_timesteps 3457.
Path 393 | total_timesteps 3467.
Path 394 | total_timesteps 3478.
Path 395 | total_timesteps 3485.
Path 396 | total_timesteps 3498.
Path 397 | total_timesteps 3506.
Path 398 | total_timesteps 3516.
Path 399 | total_timesteps 3524.
Path 400 | total_timesteps 3533.
Path 401 | total_timesteps 3541.
Path 402 | total_timesteps 3560.
Path 403 | total_timesteps 3569.
Path 404 | total_timesteps 3576.
Path 405 | total_timesteps 3583.
Path 406 | total_timesteps 3592.
Path 407 | total_timesteps 3598.
Path 408 | total_timesteps 3608.
Path 409 | total_timesteps 3621.
Path 410 | total_timesteps 3631.
Path 411 | total_timesteps 3642.
Path 412 | total_timesteps 3652.
Path 413 | total_timesteps 3660.
Path 414 | total_timesteps 3671.
Path 415 | total_timesteps 3682.
Path 416 | total_timesteps 3691.
Path 417 | total_timesteps 3700.
Path 418 | total_timesteps 3709.
Path 419 | total_timesteps 3717.
Path 420 | total_timesteps 3724.
Path 421 | total_timesteps 3731.
Path 422 | total_timesteps 3740.
Path 423 | total_timesteps 3751.
Path 424 | total_timesteps 3760.
Path 425 | total_timesteps 3767.
Path 426 | total_timesteps 3775.
Path 427 | total_timesteps 3787.
Path 428 | total_timesteps 3794.
Path 429 | total_timesteps 3805.
Path 430 | total_timesteps 3814.
Path 431 | total_timesteps 3822.
Path 432 | total_timesteps 3829.
Path 433 | total_timesteps 3841.
Path 434 | total_timesteps 3852.
Path 435 | total_timesteps 3862.
Path 436 | total_timesteps 3870.
Path 437 | total_timesteps 3886.
Path 438 | total_timesteps 3894.
Path 439 | total_timesteps 3902.
Path 440 | total_timesteps 3911.
Path 441 | total_timesteps 3921.
Path 442 | total_timesteps 3931.
Path 443 | total_timesteps 3942.
Path 444 | total_timesteps 3951.
Path 445 | total_timesteps 3958.
Path 446 | total_timesteps 3969.
Path 447 | total_timesteps 3982.
Path 448 | total_timesteps 3989.
Path 449 | total_timesteps 4002.
Path 450 | total_timesteps 4009.
Path 451 | total_timesteps 4016.
Path 452 | total_timesteps 4026.
Path 453 | total_timesteps 4033.
Path 454 | total_timesteps 4042.
Path 455 | total_timesteps 4051.
Path 456 | total_timesteps 4057.
Path 457 | total_timesteps 4069.
Path 458 | total_timesteps 4077.
Path 459 | total_timesteps 4085.
Path 460 | total_timesteps 4093.
Path 461 | total_timesteps 4103.
Path 462 | total_timesteps 4111.
Path 463 | total_timesteps 4120.
Path 464 | total_timesteps 4134.
Path 465 | total_timesteps 4147.
Path 466 | total_timesteps 4156.
Path 467 | total_timesteps 4165.
Path 468 | total_timesteps 4174.
Path 469 | total_timesteps 4181.
Path 470 | total_timesteps 4189.
Path 471 | total_timesteps 4196.
Path 472 | total_timesteps 4205.
Path 473 | total_timesteps 4211.
Path 474 | total_timesteps 4221.
Path 475 | total_timesteps 4232.
Path 476 | total_timesteps 4242.
Path 477 | total_timesteps 4250.
Path 478 | total_timesteps 4258.
Path 479 | total_timesteps 4267.
Path 480 | total_timesteps 4275.
Path 481 | total_timesteps 4288.
Path 482 | total_timesteps 4303.
Path 483 | total_timesteps 4310.
Path 484 | total_timesteps 4317.
Path 485 | total_timesteps 4324.
Path 486 | total_timesteps 4333.
Path 487 | total_timesteps 4340.
Path 488 | total_timesteps 4347.
Path 489 | total_timesteps 4359.
Path 490 | total_timesteps 4366.
Path 491 | total_timesteps 4373.
Path 492 | total_timesteps 4381.
Path 493 | total_timesteps 4392.
Path 494 | total_timesteps 4401.
Path 495 | total_timesteps 4408.
Path 496 | total_timesteps 4417.
Path 497 | total_timesteps 4424.
Path 498 | total_timesteps 4432.
Path 499 | total_timesteps 4441.
Path 500 | total_timesteps 4450.
Path 501 | total_timesteps 4458.
Path 502 | total_timesteps 4465.
Path 503 | total_timesteps 4476.
Path 504 | total_timesteps 4488.
Path 505 | total_timesteps 4495.
Path 506 | total_timesteps 4504.
Path 507 | total_timesteps 4517.
Path 508 | total_timesteps 4525.
Path 509 | total_timesteps 4533.
Path 510 | total_timesteps 4542.
Path 511 | total_timesteps 4551.
Path 512 | total_timesteps 4559.
Path 513 | total_timesteps 4571.
Path 514 | total_timesteps 4578.
Path 515 | total_timesteps 4586.
Path 516 | total_timesteps 4594.
Path 517 | total_timesteps 4603.
Path 518 | total_timesteps 4613.
Path 519 | total_timesteps 4620.
Path 520 | total_timesteps 4628.
Path 521 | total_timesteps 4637.
Path 522 | total_timesteps 4643.
Path 523 | total_timesteps 4652.
Path 524 | total_timesteps 4661.
Path 525 | total_timesteps 4671.
Path 526 | total_timesteps 4681.
Path 527 | total_timesteps 4687.
Path 528 | total_timesteps 4699.
Path 529 | total_timesteps 4707.
Path 530 | total_timesteps 4721.
Path 531 | total_timesteps 4727.
Path 532 | total_timesteps 4743.
Path 533 | total_timesteps 4753.
Path 534 | total_timesteps 4760.
Path 535 | total_timesteps 4768.
Path 536 | total_timesteps 4780.
Path 537 | total_timesteps 4791.
Path 538 | total_timesteps 4799.
Path 539 | total_timesteps 4805.
Path 540 | total_timesteps 4813.
Path 541 | total_timesteps 4821.
Path 542 | total_timesteps 4832.
Path 543 | total_timesteps 4842.
Path 544 | total_timesteps 4849.
Path 545 | total_timesteps 4858.
Path 546 | total_timesteps 4866.
Path 547 | total_timesteps 4872.
Path 548 | total_timesteps 4881.
Path 549 | total_timesteps 4888.
Path 550 | total_timesteps 4896.
Path 551 | total_timesteps 4904.
Path 552 | total_timesteps 4914.
Path 553 | total_timesteps 4928.
Path 554 | total_timesteps 4935.
Path 555 | total_timesteps 4941.
Path 556 | total_timesteps 4950.
Path 557 | total_timesteps 4959.
Path 558 | total_timesteps 4971.
Path 559 | total_timesteps 4981.
Path 560 | total_timesteps 4991.
Path 561 | total_timesteps 5000.
Path 562 | total_timesteps 5007.
Path 563 | total_timesteps 5015.
Path 564 | total_timesteps 5022.
Path 565 | total_timesteps 5029.
Path 566 | total_timesteps 5042.
Path 567 | total_timesteps 5050.
Path 568 | total_timesteps 5059.
Path 569 | total_timesteps 5067.
Path 570 | total_timesteps 5075.
Path 571 | total_timesteps 5082.
Path 572 | total_timesteps 5091.
Path 573 | total_timesteps 5100.
Path 574 | total_timesteps 5108.
Path 575 | total_timesteps 5122.
Path 576 | total_timesteps 5132.
Path 577 | total_timesteps 5143.
Path 578 | total_timesteps 5151.
Path 579 | total_timesteps 5159.
Path 580 | total_timesteps 5171.
Path 581 | total_timesteps 5180.
Path 582 | total_timesteps 5196.
Path 583 | total_timesteps 5208.
Path 584 | total_timesteps 5217.
Path 585 | total_timesteps 5226.
Path 586 | total_timesteps 5236.
Path 587 | total_timesteps 5245.
Path 588 | total_timesteps 5259.
Path 589 | total_timesteps 5268.
Path 590 | total_timesteps 5276.
Path 591 | total_timesteps 5286.
Path 592 | total_timesteps 5296.
Path 593 | total_timesteps 5303.
Path 594 | total_timesteps 5313.
Path 595 | total_timesteps 5321.
Path 596 | total_timesteps 5329.
Path 597 | total_timesteps 5337.
Path 598 | total_timesteps 5343.
Path 599 | total_timesteps 5352.
Path 600 | total_timesteps 5361.
Path 601 | total_timesteps 5370.
Path 602 | total_timesteps 5378.
Path 603 | total_timesteps 5388.
Path 604 | total_timesteps 5394.
Path 605 | total_timesteps 5403.
Path 606 | total_timesteps 5411.
Path 607 | total_timesteps 5419.
Path 608 | total_timesteps 5428.
Path 609 | total_timesteps 5435.
Path 610 | total_timesteps 5446.
Path 611 | total_timesteps 5457.
Path 612 | total_timesteps 5465.
Path 613 | total_timesteps 5474.
Path 614 | total_timesteps 5483.
Path 615 | total_timesteps 5492.
Path 616 | total_timesteps 5500.
Path 617 | total_timesteps 5512.
Path 618 | total_timesteps 5519.
Path 619 | total_timesteps 5525.
Path 620 | total_timesteps 5533.
Path 621 | total_timesteps 5542.
Path 622 | total_timesteps 5552.
Path 623 | total_timesteps 5560.
Path 624 | total_timesteps 5572.
Path 625 | total_timesteps 5582.
Path 626 | total_timesteps 5591.
Path 627 | total_timesteps 5600.
Path 628 | total_timesteps 5611.
Path 629 | total_timesteps 5620.
Path 630 | total_timesteps 5630.
Path 631 | total_timesteps 5639.
Path 632 | total_timesteps 5648.
Path 633 | total_timesteps 5657.
Path 634 | total_timesteps 5664.
Path 635 | total_timesteps 5673.
Path 636 | total_timesteps 5683.
Path 637 | total_timesteps 5693.
Path 638 | total_timesteps 5701.
Path 639 | total_timesteps 5708.
Path 640 | total_timesteps 5722.
Path 641 | total_timesteps 5732.
Path 642 | total_timesteps 5739.
Path 643 | total_timesteps 5748.
Path 644 | total_timesteps 5758.
Path 645 | total_timesteps 5767.
Path 646 | total_timesteps 5778.
Path 647 | total_timesteps 5799.
Path 648 | total_timesteps 5809.
Path 649 | total_timesteps 5817.
Path 650 | total_timesteps 5826.
Path 651 | total_timesteps 5835.
Path 652 | total_timesteps 5844.
Path 653 | total_timesteps 5851.
Path 654 | total_timesteps 5859.
Path 655 | total_timesteps 5866.
Path 656 | total_timesteps 5873.
Path 657 | total_timesteps 5880.
Path 658 | total_timesteps 5888.
Path 659 | total_timesteps 5896.
Path 660 | total_timesteps 5903.
Path 661 | total_timesteps 5910.
Path 662 | total_timesteps 5920.
Path 663 | total_timesteps 5927.
Path 664 | total_timesteps 5936.
Path 665 | total_timesteps 5946.
Path 666 | total_timesteps 5954.
Path 667 | total_timesteps 5960.
Path 668 | total_timesteps 5966.
Path 669 | total_timesteps 5972.
Path 670 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.88    |
| Iteration     | 4        |
| MaximumReturn | -0.532   |
| MinimumReturn | -20.3    |
| TotalSamples  | 24039    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.0180771816521883
Validation loss = 0.011371034197509289
Validation loss = 0.011504952795803547
Validation loss = 0.011878385208547115
Validation loss = 0.00997747853398323
Validation loss = 0.010894707404077053
Validation loss = 0.010689468123018742
Validation loss = 0.01013960037380457
Validation loss = 0.00982915423810482
Validation loss = 0.011050116270780563
Validation loss = 0.009405585005879402
Validation loss = 0.010302416980266571
Validation loss = 0.010858412832021713
Validation loss = 0.01787453331053257
Validation loss = 0.009523402899503708
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 41.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 59.
Path 8 | total_timesteps 67.
Path 9 | total_timesteps 74.
Path 10 | total_timesteps 86.
Path 11 | total_timesteps 94.
Path 12 | total_timesteps 105.
Path 13 | total_timesteps 113.
Path 14 | total_timesteps 127.
Path 15 | total_timesteps 135.
Path 16 | total_timesteps 143.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 169.
Path 20 | total_timesteps 179.
Path 21 | total_timesteps 192.
Path 22 | total_timesteps 201.
Path 23 | total_timesteps 207.
Path 24 | total_timesteps 217.
Path 25 | total_timesteps 224.
Path 26 | total_timesteps 231.
Path 27 | total_timesteps 239.
Path 28 | total_timesteps 246.
Path 29 | total_timesteps 255.
Path 30 | total_timesteps 263.
Path 31 | total_timesteps 274.
Path 32 | total_timesteps 282.
Path 33 | total_timesteps 292.
Path 34 | total_timesteps 299.
Path 35 | total_timesteps 308.
Path 36 | total_timesteps 320.
Path 37 | total_timesteps 330.
Path 38 | total_timesteps 338.
Path 39 | total_timesteps 350.
Path 40 | total_timesteps 360.
Path 41 | total_timesteps 371.
Path 42 | total_timesteps 379.
Path 43 | total_timesteps 387.
Path 44 | total_timesteps 394.
Path 45 | total_timesteps 400.
Path 46 | total_timesteps 408.
Path 47 | total_timesteps 418.
Path 48 | total_timesteps 424.
Path 49 | total_timesteps 431.
Path 50 | total_timesteps 440.
Path 51 | total_timesteps 449.
Path 52 | total_timesteps 457.
Path 53 | total_timesteps 472.
Path 54 | total_timesteps 480.
Path 55 | total_timesteps 486.
Path 56 | total_timesteps 499.
Path 57 | total_timesteps 512.
Path 58 | total_timesteps 521.
Path 59 | total_timesteps 531.
Path 60 | total_timesteps 539.
Path 61 | total_timesteps 547.
Path 62 | total_timesteps 558.
Path 63 | total_timesteps 569.
Path 64 | total_timesteps 576.
Path 65 | total_timesteps 583.
Path 66 | total_timesteps 597.
Path 67 | total_timesteps 605.
Path 68 | total_timesteps 612.
Path 69 | total_timesteps 622.
Path 70 | total_timesteps 629.
Path 71 | total_timesteps 637.
Path 72 | total_timesteps 644.
Path 73 | total_timesteps 652.
Path 74 | total_timesteps 660.
Path 75 | total_timesteps 669.
Path 76 | total_timesteps 682.
Path 77 | total_timesteps 690.
Path 78 | total_timesteps 698.
Path 79 | total_timesteps 707.
Path 80 | total_timesteps 715.
Path 81 | total_timesteps 722.
Path 82 | total_timesteps 740.
Path 83 | total_timesteps 748.
Path 84 | total_timesteps 755.
Path 85 | total_timesteps 763.
Path 86 | total_timesteps 772.
Path 87 | total_timesteps 780.
Path 88 | total_timesteps 792.
Path 89 | total_timesteps 799.
Path 90 | total_timesteps 810.
Path 91 | total_timesteps 820.
Path 92 | total_timesteps 828.
Path 93 | total_timesteps 835.
Path 94 | total_timesteps 843.
Path 95 | total_timesteps 850.
Path 96 | total_timesteps 859.
Path 97 | total_timesteps 866.
Path 98 | total_timesteps 877.
Path 99 | total_timesteps 884.
Path 100 | total_timesteps 893.
Path 101 | total_timesteps 900.
Path 102 | total_timesteps 908.
Path 103 | total_timesteps 916.
Path 104 | total_timesteps 925.
Path 105 | total_timesteps 933.
Path 106 | total_timesteps 939.
Path 107 | total_timesteps 946.
Path 108 | total_timesteps 956.
Path 109 | total_timesteps 964.
Path 110 | total_timesteps 973.
Path 111 | total_timesteps 986.
Path 112 | total_timesteps 993.
Path 113 | total_timesteps 1002.
Path 114 | total_timesteps 1018.
Path 115 | total_timesteps 1028.
Path 116 | total_timesteps 1041.
Path 117 | total_timesteps 1049.
Path 118 | total_timesteps 1058.
Path 119 | total_timesteps 1072.
Path 120 | total_timesteps 1081.
Path 121 | total_timesteps 1087.
Path 122 | total_timesteps 1100.
Path 123 | total_timesteps 1107.
Path 124 | total_timesteps 1120.
Path 125 | total_timesteps 1129.
Path 126 | total_timesteps 1135.
Path 127 | total_timesteps 1148.
Path 128 | total_timesteps 1155.
Path 129 | total_timesteps 1166.
Path 130 | total_timesteps 1174.
Path 131 | total_timesteps 1181.
Path 132 | total_timesteps 1190.
Path 133 | total_timesteps 1197.
Path 134 | total_timesteps 1205.
Path 135 | total_timesteps 1214.
Path 136 | total_timesteps 1221.
Path 137 | total_timesteps 1229.
Path 138 | total_timesteps 1237.
Path 139 | total_timesteps 1247.
Path 140 | total_timesteps 1258.
Path 141 | total_timesteps 1266.
Path 142 | total_timesteps 1273.
Path 143 | total_timesteps 1279.
Path 144 | total_timesteps 1292.
Path 145 | total_timesteps 1300.
Path 146 | total_timesteps 1307.
Path 147 | total_timesteps 1314.
Path 148 | total_timesteps 1321.
Path 149 | total_timesteps 1334.
Path 150 | total_timesteps 1342.
Path 151 | total_timesteps 1353.
Path 152 | total_timesteps 1367.
Path 153 | total_timesteps 1374.
Path 154 | total_timesteps 1385.
Path 155 | total_timesteps 1392.
Path 156 | total_timesteps 1402.
Path 157 | total_timesteps 1412.
Path 158 | total_timesteps 1420.
Path 159 | total_timesteps 1432.
Path 160 | total_timesteps 1444.
Path 161 | total_timesteps 1455.
Path 162 | total_timesteps 1464.
Path 163 | total_timesteps 1471.
Path 164 | total_timesteps 1484.
Path 165 | total_timesteps 1496.
Path 166 | total_timesteps 1506.
Path 167 | total_timesteps 1520.
Path 168 | total_timesteps 1529.
Path 169 | total_timesteps 1536.
Path 170 | total_timesteps 1548.
Path 171 | total_timesteps 1558.
Path 172 | total_timesteps 1574.
Path 173 | total_timesteps 1583.
Path 174 | total_timesteps 1592.
Path 175 | total_timesteps 1604.
Path 176 | total_timesteps 1613.
Path 177 | total_timesteps 1626.
Path 178 | total_timesteps 1634.
Path 179 | total_timesteps 1641.
Path 180 | total_timesteps 1648.
Path 181 | total_timesteps 1658.
Path 182 | total_timesteps 1667.
Path 183 | total_timesteps 1676.
Path 184 | total_timesteps 1685.
Path 185 | total_timesteps 1698.
Path 186 | total_timesteps 1707.
Path 187 | total_timesteps 1715.
Path 188 | total_timesteps 1725.
Path 189 | total_timesteps 1732.
Path 190 | total_timesteps 1744.
Path 191 | total_timesteps 1752.
Path 192 | total_timesteps 1760.
Path 193 | total_timesteps 1771.
Path 194 | total_timesteps 1778.
Path 195 | total_timesteps 1785.
Path 196 | total_timesteps 1795.
Path 197 | total_timesteps 1802.
Path 198 | total_timesteps 1811.
Path 199 | total_timesteps 1818.
Path 200 | total_timesteps 1825.
Path 201 | total_timesteps 1834.
Path 202 | total_timesteps 1842.
Path 203 | total_timesteps 1848.
Path 204 | total_timesteps 1858.
Path 205 | total_timesteps 1867.
Path 206 | total_timesteps 1877.
Path 207 | total_timesteps 1885.
Path 208 | total_timesteps 1896.
Path 209 | total_timesteps 1906.
Path 210 | total_timesteps 1913.
Path 211 | total_timesteps 1925.
Path 212 | total_timesteps 1938.
Path 213 | total_timesteps 1945.
Path 214 | total_timesteps 1955.
Path 215 | total_timesteps 1965.
Path 216 | total_timesteps 1971.
Path 217 | total_timesteps 1977.
Path 218 | total_timesteps 1987.
Path 219 | total_timesteps 1995.
Path 220 | total_timesteps 2011.
Path 221 | total_timesteps 2019.
Path 222 | total_timesteps 2029.
Path 223 | total_timesteps 2041.
Path 224 | total_timesteps 2051.
Path 225 | total_timesteps 2059.
Path 226 | total_timesteps 2071.
Path 227 | total_timesteps 2081.
Path 228 | total_timesteps 2094.
Path 229 | total_timesteps 2103.
Path 230 | total_timesteps 2109.
Path 231 | total_timesteps 2119.
Path 232 | total_timesteps 2127.
Path 233 | total_timesteps 2136.
Path 234 | total_timesteps 2143.
Path 235 | total_timesteps 2151.
Path 236 | total_timesteps 2158.
Path 237 | total_timesteps 2166.
Path 238 | total_timesteps 2175.
Path 239 | total_timesteps 2182.
Path 240 | total_timesteps 2188.
Path 241 | total_timesteps 2194.
Path 242 | total_timesteps 2203.
Path 243 | total_timesteps 2213.
Path 244 | total_timesteps 2222.
Path 245 | total_timesteps 2233.
Path 246 | total_timesteps 2243.
Path 247 | total_timesteps 2249.
Path 248 | total_timesteps 2258.
Path 249 | total_timesteps 2268.
Path 250 | total_timesteps 2277.
Path 251 | total_timesteps 2287.
Path 252 | total_timesteps 2295.
Path 253 | total_timesteps 2305.
Path 254 | total_timesteps 2314.
Path 255 | total_timesteps 2322.
Path 256 | total_timesteps 2331.
Path 257 | total_timesteps 2340.
Path 258 | total_timesteps 2348.
Path 259 | total_timesteps 2355.
Path 260 | total_timesteps 2366.
Path 261 | total_timesteps 2372.
Path 262 | total_timesteps 2386.
Path 263 | total_timesteps 2393.
Path 264 | total_timesteps 2400.
Path 265 | total_timesteps 2407.
Path 266 | total_timesteps 2413.
Path 267 | total_timesteps 2422.
Path 268 | total_timesteps 2430.
Path 269 | total_timesteps 2439.
Path 270 | total_timesteps 2449.
Path 271 | total_timesteps 2457.
Path 272 | total_timesteps 2466.
Path 273 | total_timesteps 2474.
Path 274 | total_timesteps 2481.
Path 275 | total_timesteps 2491.
Path 276 | total_timesteps 2498.
Path 277 | total_timesteps 2505.
Path 278 | total_timesteps 2512.
Path 279 | total_timesteps 2521.
Path 280 | total_timesteps 2529.
Path 281 | total_timesteps 2536.
Path 282 | total_timesteps 2545.
Path 283 | total_timesteps 2555.
Path 284 | total_timesteps 2570.
Path 285 | total_timesteps 2578.
Path 286 | total_timesteps 2586.
Path 287 | total_timesteps 2594.
Path 288 | total_timesteps 2608.
Path 289 | total_timesteps 2616.
Path 290 | total_timesteps 2624.
Path 291 | total_timesteps 2635.
Path 292 | total_timesteps 2643.
Path 293 | total_timesteps 2651.
Path 294 | total_timesteps 2658.
Path 295 | total_timesteps 2669.
Path 296 | total_timesteps 2680.
Path 297 | total_timesteps 2687.
Path 298 | total_timesteps 2694.
Path 299 | total_timesteps 2703.
Path 300 | total_timesteps 2711.
Path 301 | total_timesteps 2720.
Path 302 | total_timesteps 2727.
Path 303 | total_timesteps 2735.
Path 304 | total_timesteps 2744.
Path 305 | total_timesteps 2751.
Path 306 | total_timesteps 2760.
Path 307 | total_timesteps 2769.
Path 308 | total_timesteps 2776.
Path 309 | total_timesteps 2798.
Path 310 | total_timesteps 2806.
Path 311 | total_timesteps 2815.
Path 312 | total_timesteps 2827.
Path 313 | total_timesteps 2834.
Path 314 | total_timesteps 2840.
Path 315 | total_timesteps 2850.
Path 316 | total_timesteps 2856.
Path 317 | total_timesteps 2866.
Path 318 | total_timesteps 2882.
Path 319 | total_timesteps 2891.
Path 320 | total_timesteps 2898.
Path 321 | total_timesteps 2905.
Path 322 | total_timesteps 2912.
Path 323 | total_timesteps 2919.
Path 324 | total_timesteps 2929.
Path 325 | total_timesteps 2937.
Path 326 | total_timesteps 2952.
Path 327 | total_timesteps 2965.
Path 328 | total_timesteps 2973.
Path 329 | total_timesteps 2981.
Path 330 | total_timesteps 2989.
Path 331 | total_timesteps 3000.
Path 332 | total_timesteps 3012.
Path 333 | total_timesteps 3019.
Path 334 | total_timesteps 3028.
Path 335 | total_timesteps 3043.
Path 336 | total_timesteps 3053.
Path 337 | total_timesteps 3063.
Path 338 | total_timesteps 3072.
Path 339 | total_timesteps 3080.
Path 340 | total_timesteps 3087.
Path 341 | total_timesteps 3099.
Path 342 | total_timesteps 3112.
Path 343 | total_timesteps 3119.
Path 344 | total_timesteps 3125.
Path 345 | total_timesteps 3133.
Path 346 | total_timesteps 3143.
Path 347 | total_timesteps 3150.
Path 348 | total_timesteps 3158.
Path 349 | total_timesteps 3166.
Path 350 | total_timesteps 3174.
Path 351 | total_timesteps 3184.
Path 352 | total_timesteps 3195.
Path 353 | total_timesteps 3206.
Path 354 | total_timesteps 3215.
Path 355 | total_timesteps 3222.
Path 356 | total_timesteps 3231.
Path 357 | total_timesteps 3239.
Path 358 | total_timesteps 3249.
Path 359 | total_timesteps 3259.
Path 360 | total_timesteps 3268.
Path 361 | total_timesteps 3275.
Path 362 | total_timesteps 3288.
Path 363 | total_timesteps 3296.
Path 364 | total_timesteps 3306.
Path 365 | total_timesteps 3316.
Path 366 | total_timesteps 3322.
Path 367 | total_timesteps 3329.
Path 368 | total_timesteps 3336.
Path 369 | total_timesteps 3343.
Path 370 | total_timesteps 3352.
Path 371 | total_timesteps 3361.
Path 372 | total_timesteps 3368.
Path 373 | total_timesteps 3379.
Path 374 | total_timesteps 3388.
Path 375 | total_timesteps 3395.
Path 376 | total_timesteps 3405.
Path 377 | total_timesteps 3413.
Path 378 | total_timesteps 3423.
Path 379 | total_timesteps 3435.
Path 380 | total_timesteps 3450.
Path 381 | total_timesteps 3460.
Path 382 | total_timesteps 3470.
Path 383 | total_timesteps 3477.
Path 384 | total_timesteps 3484.
Path 385 | total_timesteps 3491.
Path 386 | total_timesteps 3500.
Path 387 | total_timesteps 3507.
Path 388 | total_timesteps 3516.
Path 389 | total_timesteps 3529.
Path 390 | total_timesteps 3543.
Path 391 | total_timesteps 3551.
Path 392 | total_timesteps 3561.
Path 393 | total_timesteps 3569.
Path 394 | total_timesteps 3575.
Path 395 | total_timesteps 3585.
Path 396 | total_timesteps 3593.
Path 397 | total_timesteps 3602.
Path 398 | total_timesteps 3610.
Path 399 | total_timesteps 3617.
Path 400 | total_timesteps 3625.
Path 401 | total_timesteps 3634.
Path 402 | total_timesteps 3641.
Path 403 | total_timesteps 3654.
Path 404 | total_timesteps 3660.
Path 405 | total_timesteps 3671.
Path 406 | total_timesteps 3678.
Path 407 | total_timesteps 3686.
Path 408 | total_timesteps 3695.
Path 409 | total_timesteps 3702.
Path 410 | total_timesteps 3709.
Path 411 | total_timesteps 3716.
Path 412 | total_timesteps 3725.
Path 413 | total_timesteps 3732.
Path 414 | total_timesteps 3739.
Path 415 | total_timesteps 3747.
Path 416 | total_timesteps 3754.
Path 417 | total_timesteps 3760.
Path 418 | total_timesteps 3771.
Path 419 | total_timesteps 3780.
Path 420 | total_timesteps 3790.
Path 421 | total_timesteps 3797.
Path 422 | total_timesteps 3806.
Path 423 | total_timesteps 3815.
Path 424 | total_timesteps 3823.
Path 425 | total_timesteps 3834.
Path 426 | total_timesteps 3843.
Path 427 | total_timesteps 3852.
Path 428 | total_timesteps 3859.
Path 429 | total_timesteps 3869.
Path 430 | total_timesteps 3877.
Path 431 | total_timesteps 3890.
Path 432 | total_timesteps 3897.
Path 433 | total_timesteps 3915.
Path 434 | total_timesteps 3924.
Path 435 | total_timesteps 3931.
Path 436 | total_timesteps 3943.
Path 437 | total_timesteps 3952.
Path 438 | total_timesteps 3961.
Path 439 | total_timesteps 3967.
Path 440 | total_timesteps 3974.
Path 441 | total_timesteps 3981.
Path 442 | total_timesteps 3989.
Path 443 | total_timesteps 3997.
Path 444 | total_timesteps 4006.
Path 445 | total_timesteps 4014.
Path 446 | total_timesteps 4029.
Path 447 | total_timesteps 4038.
Path 448 | total_timesteps 4045.
Path 449 | total_timesteps 4053.
Path 450 | total_timesteps 4070.
Path 451 | total_timesteps 4078.
Path 452 | total_timesteps 4091.
Path 453 | total_timesteps 4099.
Path 454 | total_timesteps 4114.
Path 455 | total_timesteps 4121.
Path 456 | total_timesteps 4132.
Path 457 | total_timesteps 4140.
Path 458 | total_timesteps 4150.
Path 459 | total_timesteps 4156.
Path 460 | total_timesteps 4163.
Path 461 | total_timesteps 4172.
Path 462 | total_timesteps 4185.
Path 463 | total_timesteps 4193.
Path 464 | total_timesteps 4207.
Path 465 | total_timesteps 4218.
Path 466 | total_timesteps 4225.
Path 467 | total_timesteps 4233.
Path 468 | total_timesteps 4241.
Path 469 | total_timesteps 4249.
Path 470 | total_timesteps 4257.
Path 471 | total_timesteps 4266.
Path 472 | total_timesteps 4278.
Path 473 | total_timesteps 4286.
Path 474 | total_timesteps 4294.
Path 475 | total_timesteps 4305.
Path 476 | total_timesteps 4312.
Path 477 | total_timesteps 4319.
Path 478 | total_timesteps 4328.
Path 479 | total_timesteps 4340.
Path 480 | total_timesteps 4348.
Path 481 | total_timesteps 4355.
Path 482 | total_timesteps 4363.
Path 483 | total_timesteps 4371.
Path 484 | total_timesteps 4378.
Path 485 | total_timesteps 4387.
Path 486 | total_timesteps 4397.
Path 487 | total_timesteps 4407.
Path 488 | total_timesteps 4417.
Path 489 | total_timesteps 4425.
Path 490 | total_timesteps 4436.
Path 491 | total_timesteps 4443.
Path 492 | total_timesteps 4452.
Path 493 | total_timesteps 4464.
Path 494 | total_timesteps 4471.
Path 495 | total_timesteps 4483.
Path 496 | total_timesteps 4490.
Path 497 | total_timesteps 4498.
Path 498 | total_timesteps 4508.
Path 499 | total_timesteps 4514.
Path 500 | total_timesteps 4521.
Path 501 | total_timesteps 4530.
Path 502 | total_timesteps 4537.
Path 503 | total_timesteps 4544.
Path 504 | total_timesteps 4557.
Path 505 | total_timesteps 4565.
Path 506 | total_timesteps 4576.
Path 507 | total_timesteps 4591.
Path 508 | total_timesteps 4602.
Path 509 | total_timesteps 4612.
Path 510 | total_timesteps 4621.
Path 511 | total_timesteps 4640.
Path 512 | total_timesteps 4648.
Path 513 | total_timesteps 4656.
Path 514 | total_timesteps 4670.
Path 515 | total_timesteps 4680.
Path 516 | total_timesteps 4691.
Path 517 | total_timesteps 4699.
Path 518 | total_timesteps 4710.
Path 519 | total_timesteps 4717.
Path 520 | total_timesteps 4725.
Path 521 | total_timesteps 4734.
Path 522 | total_timesteps 4743.
Path 523 | total_timesteps 4751.
Path 524 | total_timesteps 4759.
Path 525 | total_timesteps 4769.
Path 526 | total_timesteps 4779.
Path 527 | total_timesteps 4791.
Path 528 | total_timesteps 4800.
Path 529 | total_timesteps 4810.
Path 530 | total_timesteps 4820.
Path 531 | total_timesteps 4828.
Path 532 | total_timesteps 4835.
Path 533 | total_timesteps 4844.
Path 534 | total_timesteps 4855.
Path 535 | total_timesteps 4863.
Path 536 | total_timesteps 4878.
Path 537 | total_timesteps 4886.
Path 538 | total_timesteps 4893.
Path 539 | total_timesteps 4904.
Path 540 | total_timesteps 4914.
Path 541 | total_timesteps 4924.
Path 542 | total_timesteps 4937.
Path 543 | total_timesteps 4947.
Path 544 | total_timesteps 4957.
Path 545 | total_timesteps 4968.
Path 546 | total_timesteps 4977.
Path 547 | total_timesteps 4985.
Path 548 | total_timesteps 4994.
Path 549 | total_timesteps 5004.
Path 550 | total_timesteps 5013.
Path 551 | total_timesteps 5019.
Path 552 | total_timesteps 5027.
Path 553 | total_timesteps 5037.
Path 554 | total_timesteps 5045.
Path 555 | total_timesteps 5053.
Path 556 | total_timesteps 5060.
Path 557 | total_timesteps 5070.
Path 558 | total_timesteps 5078.
Path 559 | total_timesteps 5087.
Path 560 | total_timesteps 5096.
Path 561 | total_timesteps 5105.
Path 562 | total_timesteps 5112.
Path 563 | total_timesteps 5120.
Path 564 | total_timesteps 5132.
Path 565 | total_timesteps 5139.
Path 566 | total_timesteps 5151.
Path 567 | total_timesteps 5159.
Path 568 | total_timesteps 5166.
Path 569 | total_timesteps 5174.
Path 570 | total_timesteps 5182.
Path 571 | total_timesteps 5189.
Path 572 | total_timesteps 5196.
Path 573 | total_timesteps 5207.
Path 574 | total_timesteps 5216.
Path 575 | total_timesteps 5224.
Path 576 | total_timesteps 5233.
Path 577 | total_timesteps 5240.
Path 578 | total_timesteps 5250.
Path 579 | total_timesteps 5261.
Path 580 | total_timesteps 5269.
Path 581 | total_timesteps 5278.
Path 582 | total_timesteps 5286.
Path 583 | total_timesteps 5294.
Path 584 | total_timesteps 5307.
Path 585 | total_timesteps 5315.
Path 586 | total_timesteps 5323.
Path 587 | total_timesteps 5334.
Path 588 | total_timesteps 5344.
Path 589 | total_timesteps 5354.
Path 590 | total_timesteps 5361.
Path 591 | total_timesteps 5370.
Path 592 | total_timesteps 5380.
Path 593 | total_timesteps 5387.
Path 594 | total_timesteps 5395.
Path 595 | total_timesteps 5403.
Path 596 | total_timesteps 5415.
Path 597 | total_timesteps 5424.
Path 598 | total_timesteps 5431.
Path 599 | total_timesteps 5442.
Path 600 | total_timesteps 5450.
Path 601 | total_timesteps 5458.
Path 602 | total_timesteps 5466.
Path 603 | total_timesteps 5474.
Path 604 | total_timesteps 5480.
Path 605 | total_timesteps 5486.
Path 606 | total_timesteps 5493.
Path 607 | total_timesteps 5502.
Path 608 | total_timesteps 5509.
Path 609 | total_timesteps 5515.
Path 610 | total_timesteps 5522.
Path 611 | total_timesteps 5531.
Path 612 | total_timesteps 5538.
Path 613 | total_timesteps 5545.
Path 614 | total_timesteps 5554.
Path 615 | total_timesteps 5564.
Path 616 | total_timesteps 5573.
Path 617 | total_timesteps 5580.
Path 618 | total_timesteps 5586.
Path 619 | total_timesteps 5596.
Path 620 | total_timesteps 5604.
Path 621 | total_timesteps 5612.
Path 622 | total_timesteps 5622.
Path 623 | total_timesteps 5631.
Path 624 | total_timesteps 5639.
Path 625 | total_timesteps 5648.
Path 626 | total_timesteps 5660.
Path 627 | total_timesteps 5669.
Path 628 | total_timesteps 5677.
Path 629 | total_timesteps 5686.
Path 630 | total_timesteps 5699.
Path 631 | total_timesteps 5709.
Path 632 | total_timesteps 5718.
Path 633 | total_timesteps 5728.
Path 634 | total_timesteps 5736.
Path 635 | total_timesteps 5746.
Path 636 | total_timesteps 5754.
Path 637 | total_timesteps 5762.
Path 638 | total_timesteps 5776.
Path 639 | total_timesteps 5783.
Path 640 | total_timesteps 5792.
Path 641 | total_timesteps 5803.
Path 642 | total_timesteps 5811.
Path 643 | total_timesteps 5821.
Path 644 | total_timesteps 5827.
Path 645 | total_timesteps 5837.
Path 646 | total_timesteps 5844.
Path 647 | total_timesteps 5855.
Path 648 | total_timesteps 5864.
Path 649 | total_timesteps 5874.
Path 650 | total_timesteps 5880.
Path 651 | total_timesteps 5887.
Path 652 | total_timesteps 5895.
Path 653 | total_timesteps 5904.
Path 654 | total_timesteps 5912.
Path 655 | total_timesteps 5926.
Path 656 | total_timesteps 5937.
Path 657 | total_timesteps 5944.
Path 658 | total_timesteps 5953.
Path 659 | total_timesteps 5967.
Path 660 | total_timesteps 5975.
Path 661 | total_timesteps 5983.
Path 662 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.9     |
| Iteration     | 5        |
| MaximumReturn | -0.107   |
| MinimumReturn | -18.5    |
| TotalSamples  | 28039    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.010524011217057705
Validation loss = 0.009527266025543213
Validation loss = 0.009033622220158577
Validation loss = 0.009593784809112549
Validation loss = 0.008387329056859016
Validation loss = 0.008791105821728706
Validation loss = 0.008962131105363369
Validation loss = 0.010443734005093575
Validation loss = 0.008137842640280724
Validation loss = 0.008106712251901627
Validation loss = 0.009062605910003185
Validation loss = 0.008473383262753487
Validation loss = 0.008616038598120213
Validation loss = 0.007516148500144482
Validation loss = 0.007542856968939304
Validation loss = 0.009052759036421776
Validation loss = 0.009052163921296597
Validation loss = 0.008310082368552685
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 14.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 40.
Path 6 | total_timesteps 47.
Path 7 | total_timesteps 56.
Path 8 | total_timesteps 63.
Path 9 | total_timesteps 70.
Path 10 | total_timesteps 78.
Path 11 | total_timesteps 87.
Path 12 | total_timesteps 94.
Path 13 | total_timesteps 101.
Path 14 | total_timesteps 108.
Path 15 | total_timesteps 115.
Path 16 | total_timesteps 124.
Path 17 | total_timesteps 131.
Path 18 | total_timesteps 137.
Path 19 | total_timesteps 145.
Path 20 | total_timesteps 154.
Path 21 | total_timesteps 161.
Path 22 | total_timesteps 170.
Path 23 | total_timesteps 181.
Path 24 | total_timesteps 190.
Path 25 | total_timesteps 198.
Path 26 | total_timesteps 208.
Path 27 | total_timesteps 219.
Path 28 | total_timesteps 228.
Path 29 | total_timesteps 236.
Path 30 | total_timesteps 246.
Path 31 | total_timesteps 257.
Path 32 | total_timesteps 272.
Path 33 | total_timesteps 279.
Path 34 | total_timesteps 291.
Path 35 | total_timesteps 298.
Path 36 | total_timesteps 305.
Path 37 | total_timesteps 312.
Path 38 | total_timesteps 323.
Path 39 | total_timesteps 331.
Path 40 | total_timesteps 337.
Path 41 | total_timesteps 345.
Path 42 | total_timesteps 358.
Path 43 | total_timesteps 366.
Path 44 | total_timesteps 372.
Path 45 | total_timesteps 379.
Path 46 | total_timesteps 389.
Path 47 | total_timesteps 400.
Path 48 | total_timesteps 417.
Path 49 | total_timesteps 424.
Path 50 | total_timesteps 433.
Path 51 | total_timesteps 444.
Path 52 | total_timesteps 456.
Path 53 | total_timesteps 463.
Path 54 | total_timesteps 470.
Path 55 | total_timesteps 481.
Path 56 | total_timesteps 488.
Path 57 | total_timesteps 496.
Path 58 | total_timesteps 504.
Path 59 | total_timesteps 517.
Path 60 | total_timesteps 526.
Path 61 | total_timesteps 533.
Path 62 | total_timesteps 544.
Path 63 | total_timesteps 553.
Path 64 | total_timesteps 561.
Path 65 | total_timesteps 570.
Path 66 | total_timesteps 581.
Path 67 | total_timesteps 594.
Path 68 | total_timesteps 602.
Path 69 | total_timesteps 612.
Path 70 | total_timesteps 619.
Path 71 | total_timesteps 625.
Path 72 | total_timesteps 634.
Path 73 | total_timesteps 643.
Path 74 | total_timesteps 650.
Path 75 | total_timesteps 656.
Path 76 | total_timesteps 669.
Path 77 | total_timesteps 678.
Path 78 | total_timesteps 684.
Path 79 | total_timesteps 691.
Path 80 | total_timesteps 702.
Path 81 | total_timesteps 713.
Path 82 | total_timesteps 726.
Path 83 | total_timesteps 733.
Path 84 | total_timesteps 741.
Path 85 | total_timesteps 751.
Path 86 | total_timesteps 759.
Path 87 | total_timesteps 767.
Path 88 | total_timesteps 780.
Path 89 | total_timesteps 790.
Path 90 | total_timesteps 799.
Path 91 | total_timesteps 807.
Path 92 | total_timesteps 820.
Path 93 | total_timesteps 830.
Path 94 | total_timesteps 837.
Path 95 | total_timesteps 844.
Path 96 | total_timesteps 853.
Path 97 | total_timesteps 862.
Path 98 | total_timesteps 872.
Path 99 | total_timesteps 879.
Path 100 | total_timesteps 896.
Path 101 | total_timesteps 904.
Path 102 | total_timesteps 915.
Path 103 | total_timesteps 922.
Path 104 | total_timesteps 934.
Path 105 | total_timesteps 944.
Path 106 | total_timesteps 953.
Path 107 | total_timesteps 960.
Path 108 | total_timesteps 967.
Path 109 | total_timesteps 975.
Path 110 | total_timesteps 982.
Path 111 | total_timesteps 990.
Path 112 | total_timesteps 1002.
Path 113 | total_timesteps 1009.
Path 114 | total_timesteps 1018.
Path 115 | total_timesteps 1027.
Path 116 | total_timesteps 1035.
Path 117 | total_timesteps 1044.
Path 118 | total_timesteps 1053.
Path 119 | total_timesteps 1060.
Path 120 | total_timesteps 1069.
Path 121 | total_timesteps 1080.
Path 122 | total_timesteps 1087.
Path 123 | total_timesteps 1098.
Path 124 | total_timesteps 1105.
Path 125 | total_timesteps 1113.
Path 126 | total_timesteps 1121.
Path 127 | total_timesteps 1133.
Path 128 | total_timesteps 1142.
Path 129 | total_timesteps 1149.
Path 130 | total_timesteps 1160.
Path 131 | total_timesteps 1166.
Path 132 | total_timesteps 1175.
Path 133 | total_timesteps 1183.
Path 134 | total_timesteps 1193.
Path 135 | total_timesteps 1203.
Path 136 | total_timesteps 1210.
Path 137 | total_timesteps 1217.
Path 138 | total_timesteps 1225.
Path 139 | total_timesteps 1235.
Path 140 | total_timesteps 1245.
Path 141 | total_timesteps 1254.
Path 142 | total_timesteps 1263.
Path 143 | total_timesteps 1272.
Path 144 | total_timesteps 1279.
Path 145 | total_timesteps 1288.
Path 146 | total_timesteps 1299.
Path 147 | total_timesteps 1306.
Path 148 | total_timesteps 1315.
Path 149 | total_timesteps 1322.
Path 150 | total_timesteps 1328.
Path 151 | total_timesteps 1337.
Path 152 | total_timesteps 1344.
Path 153 | total_timesteps 1352.
Path 154 | total_timesteps 1362.
Path 155 | total_timesteps 1371.
Path 156 | total_timesteps 1377.
Path 157 | total_timesteps 1387.
Path 158 | total_timesteps 1401.
Path 159 | total_timesteps 1409.
Path 160 | total_timesteps 1421.
Path 161 | total_timesteps 1431.
Path 162 | total_timesteps 1441.
Path 163 | total_timesteps 1452.
Path 164 | total_timesteps 1459.
Path 165 | total_timesteps 1468.
Path 166 | total_timesteps 1475.
Path 167 | total_timesteps 1484.
Path 168 | total_timesteps 1492.
Path 169 | total_timesteps 1501.
Path 170 | total_timesteps 1508.
Path 171 | total_timesteps 1518.
Path 172 | total_timesteps 1531.
Path 173 | total_timesteps 1539.
Path 174 | total_timesteps 1546.
Path 175 | total_timesteps 1555.
Path 176 | total_timesteps 1563.
Path 177 | total_timesteps 1571.
Path 178 | total_timesteps 1579.
Path 179 | total_timesteps 1588.
Path 180 | total_timesteps 1599.
Path 181 | total_timesteps 1606.
Path 182 | total_timesteps 1613.
Path 183 | total_timesteps 1623.
Path 184 | total_timesteps 1630.
Path 185 | total_timesteps 1641.
Path 186 | total_timesteps 1652.
Path 187 | total_timesteps 1660.
Path 188 | total_timesteps 1667.
Path 189 | total_timesteps 1674.
Path 190 | total_timesteps 1680.
Path 191 | total_timesteps 1686.
Path 192 | total_timesteps 1695.
Path 193 | total_timesteps 1704.
Path 194 | total_timesteps 1711.
Path 195 | total_timesteps 1721.
Path 196 | total_timesteps 1727.
Path 197 | total_timesteps 1736.
Path 198 | total_timesteps 1747.
Path 199 | total_timesteps 1758.
Path 200 | total_timesteps 1765.
Path 201 | total_timesteps 1771.
Path 202 | total_timesteps 1779.
Path 203 | total_timesteps 1786.
Path 204 | total_timesteps 1793.
Path 205 | total_timesteps 1799.
Path 206 | total_timesteps 1808.
Path 207 | total_timesteps 1825.
Path 208 | total_timesteps 1833.
Path 209 | total_timesteps 1842.
Path 210 | total_timesteps 1849.
Path 211 | total_timesteps 1858.
Path 212 | total_timesteps 1866.
Path 213 | total_timesteps 1874.
Path 214 | total_timesteps 1883.
Path 215 | total_timesteps 1893.
Path 216 | total_timesteps 1905.
Path 217 | total_timesteps 1912.
Path 218 | total_timesteps 1921.
Path 219 | total_timesteps 1929.
Path 220 | total_timesteps 1936.
Path 221 | total_timesteps 1943.
Path 222 | total_timesteps 1953.
Path 223 | total_timesteps 1959.
Path 224 | total_timesteps 1969.
Path 225 | total_timesteps 1978.
Path 226 | total_timesteps 1989.
Path 227 | total_timesteps 2000.
Path 228 | total_timesteps 2007.
Path 229 | total_timesteps 2015.
Path 230 | total_timesteps 2023.
Path 231 | total_timesteps 2041.
Path 232 | total_timesteps 2048.
Path 233 | total_timesteps 2056.
Path 234 | total_timesteps 2064.
Path 235 | total_timesteps 2072.
Path 236 | total_timesteps 2085.
Path 237 | total_timesteps 2092.
Path 238 | total_timesteps 2100.
Path 239 | total_timesteps 2107.
Path 240 | total_timesteps 2115.
Path 241 | total_timesteps 2126.
Path 242 | total_timesteps 2135.
Path 243 | total_timesteps 2145.
Path 244 | total_timesteps 2152.
Path 245 | total_timesteps 2160.
Path 246 | total_timesteps 2169.
Path 247 | total_timesteps 2178.
Path 248 | total_timesteps 2184.
Path 249 | total_timesteps 2193.
Path 250 | total_timesteps 2200.
Path 251 | total_timesteps 2209.
Path 252 | total_timesteps 2216.
Path 253 | total_timesteps 2227.
Path 254 | total_timesteps 2237.
Path 255 | total_timesteps 2244.
Path 256 | total_timesteps 2251.
Path 257 | total_timesteps 2258.
Path 258 | total_timesteps 2266.
Path 259 | total_timesteps 2274.
Path 260 | total_timesteps 2283.
Path 261 | total_timesteps 2291.
Path 262 | total_timesteps 2300.
Path 263 | total_timesteps 2308.
Path 264 | total_timesteps 2315.
Path 265 | total_timesteps 2325.
Path 266 | total_timesteps 2335.
Path 267 | total_timesteps 2346.
Path 268 | total_timesteps 2353.
Path 269 | total_timesteps 2364.
Path 270 | total_timesteps 2370.
Path 271 | total_timesteps 2380.
Path 272 | total_timesteps 2389.
Path 273 | total_timesteps 2401.
Path 274 | total_timesteps 2409.
Path 275 | total_timesteps 2421.
Path 276 | total_timesteps 2429.
Path 277 | total_timesteps 2436.
Path 278 | total_timesteps 2446.
Path 279 | total_timesteps 2454.
Path 280 | total_timesteps 2461.
Path 281 | total_timesteps 2470.
Path 282 | total_timesteps 2478.
Path 283 | total_timesteps 2486.
Path 284 | total_timesteps 2498.
Path 285 | total_timesteps 2512.
Path 286 | total_timesteps 2522.
Path 287 | total_timesteps 2529.
Path 288 | total_timesteps 2537.
Path 289 | total_timesteps 2547.
Path 290 | total_timesteps 2555.
Path 291 | total_timesteps 2563.
Path 292 | total_timesteps 2571.
Path 293 | total_timesteps 2580.
Path 294 | total_timesteps 2589.
Path 295 | total_timesteps 2597.
Path 296 | total_timesteps 2607.
Path 297 | total_timesteps 2614.
Path 298 | total_timesteps 2625.
Path 299 | total_timesteps 2636.
Path 300 | total_timesteps 2642.
Path 301 | total_timesteps 2651.
Path 302 | total_timesteps 2670.
Path 303 | total_timesteps 2679.
Path 304 | total_timesteps 2688.
Path 305 | total_timesteps 2696.
Path 306 | total_timesteps 2703.
Path 307 | total_timesteps 2713.
Path 308 | total_timesteps 2721.
Path 309 | total_timesteps 2733.
Path 310 | total_timesteps 2745.
Path 311 | total_timesteps 2751.
Path 312 | total_timesteps 2762.
Path 313 | total_timesteps 2770.
Path 314 | total_timesteps 2779.
Path 315 | total_timesteps 2788.
Path 316 | total_timesteps 2796.
Path 317 | total_timesteps 2806.
Path 318 | total_timesteps 2816.
Path 319 | total_timesteps 2831.
Path 320 | total_timesteps 2838.
Path 321 | total_timesteps 2849.
Path 322 | total_timesteps 2858.
Path 323 | total_timesteps 2870.
Path 324 | total_timesteps 2877.
Path 325 | total_timesteps 2885.
Path 326 | total_timesteps 2892.
Path 327 | total_timesteps 2902.
Path 328 | total_timesteps 2909.
Path 329 | total_timesteps 2918.
Path 330 | total_timesteps 2931.
Path 331 | total_timesteps 2937.
Path 332 | total_timesteps 2944.
Path 333 | total_timesteps 2952.
Path 334 | total_timesteps 2962.
Path 335 | total_timesteps 2970.
Path 336 | total_timesteps 2977.
Path 337 | total_timesteps 2984.
Path 338 | total_timesteps 2992.
Path 339 | total_timesteps 3005.
Path 340 | total_timesteps 3011.
Path 341 | total_timesteps 3020.
Path 342 | total_timesteps 3028.
Path 343 | total_timesteps 3036.
Path 344 | total_timesteps 3043.
Path 345 | total_timesteps 3054.
Path 346 | total_timesteps 3066.
Path 347 | total_timesteps 3075.
Path 348 | total_timesteps 3084.
Path 349 | total_timesteps 3093.
Path 350 | total_timesteps 3101.
Path 351 | total_timesteps 3110.
Path 352 | total_timesteps 3119.
Path 353 | total_timesteps 3130.
Path 354 | total_timesteps 3139.
Path 355 | total_timesteps 3153.
Path 356 | total_timesteps 3168.
Path 357 | total_timesteps 3174.
Path 358 | total_timesteps 3185.
Path 359 | total_timesteps 3194.
Path 360 | total_timesteps 3202.
Path 361 | total_timesteps 3213.
Path 362 | total_timesteps 3224.
Path 363 | total_timesteps 3231.
Path 364 | total_timesteps 3241.
Path 365 | total_timesteps 3249.
Path 366 | total_timesteps 3256.
Path 367 | total_timesteps 3263.
Path 368 | total_timesteps 3270.
Path 369 | total_timesteps 3279.
Path 370 | total_timesteps 3287.
Path 371 | total_timesteps 3298.
Path 372 | total_timesteps 3311.
Path 373 | total_timesteps 3332.
Path 374 | total_timesteps 3340.
Path 375 | total_timesteps 3349.
Path 376 | total_timesteps 3355.
Path 377 | total_timesteps 3362.
Path 378 | total_timesteps 3373.
Path 379 | total_timesteps 3383.
Path 380 | total_timesteps 3395.
Path 381 | total_timesteps 3402.
Path 382 | total_timesteps 3408.
Path 383 | total_timesteps 3416.
Path 384 | total_timesteps 3427.
Path 385 | total_timesteps 3434.
Path 386 | total_timesteps 3441.
Path 387 | total_timesteps 3448.
Path 388 | total_timesteps 3455.
Path 389 | total_timesteps 3464.
Path 390 | total_timesteps 3471.
Path 391 | total_timesteps 3482.
Path 392 | total_timesteps 3490.
Path 393 | total_timesteps 3499.
Path 394 | total_timesteps 3514.
Path 395 | total_timesteps 3523.
Path 396 | total_timesteps 3530.
Path 397 | total_timesteps 3538.
Path 398 | total_timesteps 3548.
Path 399 | total_timesteps 3555.
Path 400 | total_timesteps 3563.
Path 401 | total_timesteps 3571.
Path 402 | total_timesteps 3580.
Path 403 | total_timesteps 3587.
Path 404 | total_timesteps 3596.
Path 405 | total_timesteps 3603.
Path 406 | total_timesteps 3611.
Path 407 | total_timesteps 3618.
Path 408 | total_timesteps 3626.
Path 409 | total_timesteps 3633.
Path 410 | total_timesteps 3642.
Path 411 | total_timesteps 3656.
Path 412 | total_timesteps 3665.
Path 413 | total_timesteps 3676.
Path 414 | total_timesteps 3687.
Path 415 | total_timesteps 3697.
Path 416 | total_timesteps 3712.
Path 417 | total_timesteps 3718.
Path 418 | total_timesteps 3726.
Path 419 | total_timesteps 3734.
Path 420 | total_timesteps 3742.
Path 421 | total_timesteps 3749.
Path 422 | total_timesteps 3758.
Path 423 | total_timesteps 3768.
Path 424 | total_timesteps 3775.
Path 425 | total_timesteps 3783.
Path 426 | total_timesteps 3789.
Path 427 | total_timesteps 3796.
Path 428 | total_timesteps 3805.
Path 429 | total_timesteps 3814.
Path 430 | total_timesteps 3823.
Path 431 | total_timesteps 3836.
Path 432 | total_timesteps 3844.
Path 433 | total_timesteps 3851.
Path 434 | total_timesteps 3860.
Path 435 | total_timesteps 3866.
Path 436 | total_timesteps 3875.
Path 437 | total_timesteps 3883.
Path 438 | total_timesteps 3893.
Path 439 | total_timesteps 3905.
Path 440 | total_timesteps 3913.
Path 441 | total_timesteps 3925.
Path 442 | total_timesteps 3935.
Path 443 | total_timesteps 3946.
Path 444 | total_timesteps 3955.
Path 445 | total_timesteps 3966.
Path 446 | total_timesteps 3973.
Path 447 | total_timesteps 3980.
Path 448 | total_timesteps 3988.
Path 449 | total_timesteps 3994.
Path 450 | total_timesteps 4002.
Path 451 | total_timesteps 4008.
Path 452 | total_timesteps 4014.
Path 453 | total_timesteps 4023.
Path 454 | total_timesteps 4031.
Path 455 | total_timesteps 4044.
Path 456 | total_timesteps 4051.
Path 457 | total_timesteps 4059.
Path 458 | total_timesteps 4065.
Path 459 | total_timesteps 4072.
Path 460 | total_timesteps 4082.
Path 461 | total_timesteps 4090.
Path 462 | total_timesteps 4105.
Path 463 | total_timesteps 4118.
Path 464 | total_timesteps 4128.
Path 465 | total_timesteps 4137.
Path 466 | total_timesteps 4144.
Path 467 | total_timesteps 4152.
Path 468 | total_timesteps 4175.
Path 469 | total_timesteps 4182.
Path 470 | total_timesteps 4190.
Path 471 | total_timesteps 4196.
Path 472 | total_timesteps 4207.
Path 473 | total_timesteps 4215.
Path 474 | total_timesteps 4224.
Path 475 | total_timesteps 4233.
Path 476 | total_timesteps 4240.
Path 477 | total_timesteps 4247.
Path 478 | total_timesteps 4256.
Path 479 | total_timesteps 4266.
Path 480 | total_timesteps 4274.
Path 481 | total_timesteps 4281.
Path 482 | total_timesteps 4290.
Path 483 | total_timesteps 4305.
Path 484 | total_timesteps 4312.
Path 485 | total_timesteps 4325.
Path 486 | total_timesteps 4332.
Path 487 | total_timesteps 4342.
Path 488 | total_timesteps 4349.
Path 489 | total_timesteps 4359.
Path 490 | total_timesteps 4367.
Path 491 | total_timesteps 4377.
Path 492 | total_timesteps 4384.
Path 493 | total_timesteps 4394.
Path 494 | total_timesteps 4407.
Path 495 | total_timesteps 4419.
Path 496 | total_timesteps 4429.
Path 497 | total_timesteps 4436.
Path 498 | total_timesteps 4443.
Path 499 | total_timesteps 4450.
Path 500 | total_timesteps 4456.
Path 501 | total_timesteps 4463.
Path 502 | total_timesteps 4475.
Path 503 | total_timesteps 4486.
Path 504 | total_timesteps 4492.
Path 505 | total_timesteps 4501.
Path 506 | total_timesteps 4509.
Path 507 | total_timesteps 4519.
Path 508 | total_timesteps 4528.
Path 509 | total_timesteps 4534.
Path 510 | total_timesteps 4541.
Path 511 | total_timesteps 4552.
Path 512 | total_timesteps 4562.
Path 513 | total_timesteps 4569.
Path 514 | total_timesteps 4578.
Path 515 | total_timesteps 4585.
Path 516 | total_timesteps 4594.
Path 517 | total_timesteps 4601.
Path 518 | total_timesteps 4613.
Path 519 | total_timesteps 4620.
Path 520 | total_timesteps 4635.
Path 521 | total_timesteps 4646.
Path 522 | total_timesteps 4654.
Path 523 | total_timesteps 4663.
Path 524 | total_timesteps 4672.
Path 525 | total_timesteps 4681.
Path 526 | total_timesteps 4688.
Path 527 | total_timesteps 4695.
Path 528 | total_timesteps 4710.
Path 529 | total_timesteps 4720.
Path 530 | total_timesteps 4727.
Path 531 | total_timesteps 4733.
Path 532 | total_timesteps 4741.
Path 533 | total_timesteps 4749.
Path 534 | total_timesteps 4760.
Path 535 | total_timesteps 4768.
Path 536 | total_timesteps 4776.
Path 537 | total_timesteps 4783.
Path 538 | total_timesteps 4792.
Path 539 | total_timesteps 4800.
Path 540 | total_timesteps 4810.
Path 541 | total_timesteps 4817.
Path 542 | total_timesteps 4825.
Path 543 | total_timesteps 4835.
Path 544 | total_timesteps 4847.
Path 545 | total_timesteps 4859.
Path 546 | total_timesteps 4869.
Path 547 | total_timesteps 4878.
Path 548 | total_timesteps 4889.
Path 549 | total_timesteps 4897.
Path 550 | total_timesteps 4904.
Path 551 | total_timesteps 4912.
Path 552 | total_timesteps 4921.
Path 553 | total_timesteps 4927.
Path 554 | total_timesteps 4938.
Path 555 | total_timesteps 4951.
Path 556 | total_timesteps 4958.
Path 557 | total_timesteps 4968.
Path 558 | total_timesteps 4975.
Path 559 | total_timesteps 4982.
Path 560 | total_timesteps 4992.
Path 561 | total_timesteps 4998.
Path 562 | total_timesteps 5007.
Path 563 | total_timesteps 5014.
Path 564 | total_timesteps 5023.
Path 565 | total_timesteps 5031.
Path 566 | total_timesteps 5038.
Path 567 | total_timesteps 5047.
Path 568 | total_timesteps 5054.
Path 569 | total_timesteps 5063.
Path 570 | total_timesteps 5070.
Path 571 | total_timesteps 5076.
Path 572 | total_timesteps 5082.
Path 573 | total_timesteps 5090.
Path 574 | total_timesteps 5102.
Path 575 | total_timesteps 5110.
Path 576 | total_timesteps 5127.
Path 577 | total_timesteps 5139.
Path 578 | total_timesteps 5149.
Path 579 | total_timesteps 5156.
Path 580 | total_timesteps 5165.
Path 581 | total_timesteps 5174.
Path 582 | total_timesteps 5181.
Path 583 | total_timesteps 5188.
Path 584 | total_timesteps 5195.
Path 585 | total_timesteps 5206.
Path 586 | total_timesteps 5224.
Path 587 | total_timesteps 5232.
Path 588 | total_timesteps 5241.
Path 589 | total_timesteps 5249.
Path 590 | total_timesteps 5256.
Path 591 | total_timesteps 5266.
Path 592 | total_timesteps 5275.
Path 593 | total_timesteps 5281.
Path 594 | total_timesteps 5288.
Path 595 | total_timesteps 5296.
Path 596 | total_timesteps 5306.
Path 597 | total_timesteps 5315.
Path 598 | total_timesteps 5324.
Path 599 | total_timesteps 5338.
Path 600 | total_timesteps 5345.
Path 601 | total_timesteps 5354.
Path 602 | total_timesteps 5360.
Path 603 | total_timesteps 5368.
Path 604 | total_timesteps 5376.
Path 605 | total_timesteps 5382.
Path 606 | total_timesteps 5392.
Path 607 | total_timesteps 5399.
Path 608 | total_timesteps 5408.
Path 609 | total_timesteps 5416.
Path 610 | total_timesteps 5424.
Path 611 | total_timesteps 5431.
Path 612 | total_timesteps 5443.
Path 613 | total_timesteps 5451.
Path 614 | total_timesteps 5457.
Path 615 | total_timesteps 5465.
Path 616 | total_timesteps 5476.
Path 617 | total_timesteps 5483.
Path 618 | total_timesteps 5491.
Path 619 | total_timesteps 5498.
Path 620 | total_timesteps 5509.
Path 621 | total_timesteps 5519.
Path 622 | total_timesteps 5526.
Path 623 | total_timesteps 5536.
Path 624 | total_timesteps 5553.
Path 625 | total_timesteps 5560.
Path 626 | total_timesteps 5568.
Path 627 | total_timesteps 5578.
Path 628 | total_timesteps 5589.
Path 629 | total_timesteps 5596.
Path 630 | total_timesteps 5605.
Path 631 | total_timesteps 5612.
Path 632 | total_timesteps 5620.
Path 633 | total_timesteps 5627.
Path 634 | total_timesteps 5634.
Path 635 | total_timesteps 5641.
Path 636 | total_timesteps 5654.
Path 637 | total_timesteps 5661.
Path 638 | total_timesteps 5675.
Path 639 | total_timesteps 5684.
Path 640 | total_timesteps 5693.
Path 641 | total_timesteps 5701.
Path 642 | total_timesteps 5708.
Path 643 | total_timesteps 5719.
Path 644 | total_timesteps 5730.
Path 645 | total_timesteps 5737.
Path 646 | total_timesteps 5744.
Path 647 | total_timesteps 5754.
Path 648 | total_timesteps 5763.
Path 649 | total_timesteps 5770.
Path 650 | total_timesteps 5779.
Path 651 | total_timesteps 5787.
Path 652 | total_timesteps 5794.
Path 653 | total_timesteps 5803.
Path 654 | total_timesteps 5811.
Path 655 | total_timesteps 5821.
Path 656 | total_timesteps 5829.
Path 657 | total_timesteps 5843.
Path 658 | total_timesteps 5850.
Path 659 | total_timesteps 5858.
Path 660 | total_timesteps 5871.
Path 661 | total_timesteps 5879.
Path 662 | total_timesteps 5886.
Path 663 | total_timesteps 5899.
Path 664 | total_timesteps 5908.
Path 665 | total_timesteps 5916.
Path 666 | total_timesteps 5924.
Path 667 | total_timesteps 5931.
Path 668 | total_timesteps 5938.
Path 669 | total_timesteps 5946.
Path 670 | total_timesteps 5954.
Path 671 | total_timesteps 5964.
Path 672 | total_timesteps 5972.
Path 673 | total_timesteps 5980.
Path 674 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.81    |
| Iteration     | 6        |
| MaximumReturn | 1.04     |
| MinimumReturn | -19.7    |
| TotalSamples  | 32039    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.007665600161999464
Validation loss = 0.006826536729931831
Validation loss = 0.008268531411886215
Validation loss = 0.0076999845914542675
Validation loss = 0.00704915588721633
Validation loss = 0.007308166474103928
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 38.
Path 4 | total_timesteps 45.
Path 5 | total_timesteps 55.
Path 6 | total_timesteps 65.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 104.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 122.
Path 14 | total_timesteps 128.
Path 15 | total_timesteps 136.
Path 16 | total_timesteps 144.
Path 17 | total_timesteps 150.
Path 18 | total_timesteps 157.
Path 19 | total_timesteps 165.
Path 20 | total_timesteps 173.
Path 21 | total_timesteps 180.
Path 22 | total_timesteps 189.
Path 23 | total_timesteps 196.
Path 24 | total_timesteps 205.
Path 25 | total_timesteps 212.
Path 26 | total_timesteps 218.
Path 27 | total_timesteps 227.
Path 28 | total_timesteps 240.
Path 29 | total_timesteps 247.
Path 30 | total_timesteps 257.
Path 31 | total_timesteps 264.
Path 32 | total_timesteps 275.
Path 33 | total_timesteps 284.
Path 34 | total_timesteps 292.
Path 35 | total_timesteps 299.
Path 36 | total_timesteps 311.
Path 37 | total_timesteps 320.
Path 38 | total_timesteps 327.
Path 39 | total_timesteps 333.
Path 40 | total_timesteps 342.
Path 41 | total_timesteps 348.
Path 42 | total_timesteps 358.
Path 43 | total_timesteps 367.
Path 44 | total_timesteps 375.
Path 45 | total_timesteps 385.
Path 46 | total_timesteps 393.
Path 47 | total_timesteps 400.
Path 48 | total_timesteps 410.
Path 49 | total_timesteps 420.
Path 50 | total_timesteps 427.
Path 51 | total_timesteps 436.
Path 52 | total_timesteps 443.
Path 53 | total_timesteps 451.
Path 54 | total_timesteps 461.
Path 55 | total_timesteps 468.
Path 56 | total_timesteps 481.
Path 57 | total_timesteps 489.
Path 58 | total_timesteps 498.
Path 59 | total_timesteps 508.
Path 60 | total_timesteps 515.
Path 61 | total_timesteps 522.
Path 62 | total_timesteps 531.
Path 63 | total_timesteps 542.
Path 64 | total_timesteps 549.
Path 65 | total_timesteps 556.
Path 66 | total_timesteps 564.
Path 67 | total_timesteps 572.
Path 68 | total_timesteps 579.
Path 69 | total_timesteps 588.
Path 70 | total_timesteps 594.
Path 71 | total_timesteps 601.
Path 72 | total_timesteps 610.
Path 73 | total_timesteps 618.
Path 74 | total_timesteps 627.
Path 75 | total_timesteps 634.
Path 76 | total_timesteps 645.
Path 77 | total_timesteps 654.
Path 78 | total_timesteps 666.
Path 79 | total_timesteps 673.
Path 80 | total_timesteps 680.
Path 81 | total_timesteps 689.
Path 82 | total_timesteps 696.
Path 83 | total_timesteps 707.
Path 84 | total_timesteps 720.
Path 85 | total_timesteps 726.
Path 86 | total_timesteps 736.
Path 87 | total_timesteps 743.
Path 88 | total_timesteps 751.
Path 89 | total_timesteps 757.
Path 90 | total_timesteps 765.
Path 91 | total_timesteps 771.
Path 92 | total_timesteps 778.
Path 93 | total_timesteps 787.
Path 94 | total_timesteps 794.
Path 95 | total_timesteps 800.
Path 96 | total_timesteps 811.
Path 97 | total_timesteps 818.
Path 98 | total_timesteps 830.
Path 99 | total_timesteps 838.
Path 100 | total_timesteps 846.
Path 101 | total_timesteps 853.
Path 102 | total_timesteps 863.
Path 103 | total_timesteps 871.
Path 104 | total_timesteps 879.
Path 105 | total_timesteps 886.
Path 106 | total_timesteps 898.
Path 107 | total_timesteps 909.
Path 108 | total_timesteps 920.
Path 109 | total_timesteps 926.
Path 110 | total_timesteps 935.
Path 111 | total_timesteps 944.
Path 112 | total_timesteps 952.
Path 113 | total_timesteps 961.
Path 114 | total_timesteps 971.
Path 115 | total_timesteps 978.
Path 116 | total_timesteps 986.
Path 117 | total_timesteps 994.
Path 118 | total_timesteps 1006.
Path 119 | total_timesteps 1015.
Path 120 | total_timesteps 1023.
Path 121 | total_timesteps 1032.
Path 122 | total_timesteps 1045.
Path 123 | total_timesteps 1052.
Path 124 | total_timesteps 1063.
Path 125 | total_timesteps 1071.
Path 126 | total_timesteps 1078.
Path 127 | total_timesteps 1088.
Path 128 | total_timesteps 1094.
Path 129 | total_timesteps 1103.
Path 130 | total_timesteps 1110.
Path 131 | total_timesteps 1117.
Path 132 | total_timesteps 1126.
Path 133 | total_timesteps 1134.
Path 134 | total_timesteps 1144.
Path 135 | total_timesteps 1150.
Path 136 | total_timesteps 1159.
Path 137 | total_timesteps 1169.
Path 138 | total_timesteps 1179.
Path 139 | total_timesteps 1191.
Path 140 | total_timesteps 1198.
Path 141 | total_timesteps 1211.
Path 142 | total_timesteps 1223.
Path 143 | total_timesteps 1230.
Path 144 | total_timesteps 1240.
Path 145 | total_timesteps 1246.
Path 146 | total_timesteps 1256.
Path 147 | total_timesteps 1263.
Path 148 | total_timesteps 1271.
Path 149 | total_timesteps 1279.
Path 150 | total_timesteps 1287.
Path 151 | total_timesteps 1299.
Path 152 | total_timesteps 1307.
Path 153 | total_timesteps 1314.
Path 154 | total_timesteps 1323.
Path 155 | total_timesteps 1330.
Path 156 | total_timesteps 1341.
Path 157 | total_timesteps 1352.
Path 158 | total_timesteps 1364.
Path 159 | total_timesteps 1376.
Path 160 | total_timesteps 1385.
Path 161 | total_timesteps 1394.
Path 162 | total_timesteps 1403.
Path 163 | total_timesteps 1410.
Path 164 | total_timesteps 1427.
Path 165 | total_timesteps 1435.
Path 166 | total_timesteps 1445.
Path 167 | total_timesteps 1456.
Path 168 | total_timesteps 1464.
Path 169 | total_timesteps 1481.
Path 170 | total_timesteps 1499.
Path 171 | total_timesteps 1508.
Path 172 | total_timesteps 1515.
Path 173 | total_timesteps 1524.
Path 174 | total_timesteps 1532.
Path 175 | total_timesteps 1543.
Path 176 | total_timesteps 1550.
Path 177 | total_timesteps 1563.
Path 178 | total_timesteps 1570.
Path 179 | total_timesteps 1579.
Path 180 | total_timesteps 1586.
Path 181 | total_timesteps 1601.
Path 182 | total_timesteps 1608.
Path 183 | total_timesteps 1615.
Path 184 | total_timesteps 1631.
Path 185 | total_timesteps 1642.
Path 186 | total_timesteps 1651.
Path 187 | total_timesteps 1662.
Path 188 | total_timesteps 1673.
Path 189 | total_timesteps 1684.
Path 190 | total_timesteps 1691.
Path 191 | total_timesteps 1699.
Path 192 | total_timesteps 1705.
Path 193 | total_timesteps 1713.
Path 194 | total_timesteps 1721.
Path 195 | total_timesteps 1732.
Path 196 | total_timesteps 1747.
Path 197 | total_timesteps 1754.
Path 198 | total_timesteps 1761.
Path 199 | total_timesteps 1768.
Path 200 | total_timesteps 1782.
Path 201 | total_timesteps 1792.
Path 202 | total_timesteps 1799.
Path 203 | total_timesteps 1809.
Path 204 | total_timesteps 1821.
Path 205 | total_timesteps 1831.
Path 206 | total_timesteps 1839.
Path 207 | total_timesteps 1848.
Path 208 | total_timesteps 1855.
Path 209 | total_timesteps 1861.
Path 210 | total_timesteps 1871.
Path 211 | total_timesteps 1878.
Path 212 | total_timesteps 1887.
Path 213 | total_timesteps 1896.
Path 214 | total_timesteps 1904.
Path 215 | total_timesteps 1910.
Path 216 | total_timesteps 1917.
Path 217 | total_timesteps 1926.
Path 218 | total_timesteps 1934.
Path 219 | total_timesteps 1943.
Path 220 | total_timesteps 1953.
Path 221 | total_timesteps 1962.
Path 222 | total_timesteps 1969.
Path 223 | total_timesteps 1978.
Path 224 | total_timesteps 1987.
Path 225 | total_timesteps 1996.
Path 226 | total_timesteps 2011.
Path 227 | total_timesteps 2019.
Path 228 | total_timesteps 2028.
Path 229 | total_timesteps 2036.
Path 230 | total_timesteps 2044.
Path 231 | total_timesteps 2052.
Path 232 | total_timesteps 2061.
Path 233 | total_timesteps 2070.
Path 234 | total_timesteps 2085.
Path 235 | total_timesteps 2092.
Path 236 | total_timesteps 2099.
Path 237 | total_timesteps 2106.
Path 238 | total_timesteps 2112.
Path 239 | total_timesteps 2121.
Path 240 | total_timesteps 2128.
Path 241 | total_timesteps 2134.
Path 242 | total_timesteps 2141.
Path 243 | total_timesteps 2149.
Path 244 | total_timesteps 2157.
Path 245 | total_timesteps 2166.
Path 246 | total_timesteps 2175.
Path 247 | total_timesteps 2186.
Path 248 | total_timesteps 2197.
Path 249 | total_timesteps 2204.
Path 250 | total_timesteps 2211.
Path 251 | total_timesteps 2221.
Path 252 | total_timesteps 2230.
Path 253 | total_timesteps 2237.
Path 254 | total_timesteps 2244.
Path 255 | total_timesteps 2251.
Path 256 | total_timesteps 2265.
Path 257 | total_timesteps 2274.
Path 258 | total_timesteps 2285.
Path 259 | total_timesteps 2298.
Path 260 | total_timesteps 2310.
Path 261 | total_timesteps 2321.
Path 262 | total_timesteps 2328.
Path 263 | total_timesteps 2342.
Path 264 | total_timesteps 2352.
Path 265 | total_timesteps 2362.
Path 266 | total_timesteps 2370.
Path 267 | total_timesteps 2378.
Path 268 | total_timesteps 2387.
Path 269 | total_timesteps 2397.
Path 270 | total_timesteps 2406.
Path 271 | total_timesteps 2416.
Path 272 | total_timesteps 2429.
Path 273 | total_timesteps 2440.
Path 274 | total_timesteps 2448.
Path 275 | total_timesteps 2454.
Path 276 | total_timesteps 2464.
Path 277 | total_timesteps 2483.
Path 278 | total_timesteps 2493.
Path 279 | total_timesteps 2504.
Path 280 | total_timesteps 2510.
Path 281 | total_timesteps 2517.
Path 282 | total_timesteps 2526.
Path 283 | total_timesteps 2542.
Path 284 | total_timesteps 2553.
Path 285 | total_timesteps 2563.
Path 286 | total_timesteps 2570.
Path 287 | total_timesteps 2580.
Path 288 | total_timesteps 2590.
Path 289 | total_timesteps 2600.
Path 290 | total_timesteps 2617.
Path 291 | total_timesteps 2625.
Path 292 | total_timesteps 2632.
Path 293 | total_timesteps 2642.
Path 294 | total_timesteps 2651.
Path 295 | total_timesteps 2659.
Path 296 | total_timesteps 2667.
Path 297 | total_timesteps 2676.
Path 298 | total_timesteps 2683.
Path 299 | total_timesteps 2690.
Path 300 | total_timesteps 2698.
Path 301 | total_timesteps 2707.
Path 302 | total_timesteps 2717.
Path 303 | total_timesteps 2726.
Path 304 | total_timesteps 2733.
Path 305 | total_timesteps 2741.
Path 306 | total_timesteps 2751.
Path 307 | total_timesteps 2758.
Path 308 | total_timesteps 2766.
Path 309 | total_timesteps 2773.
Path 310 | total_timesteps 2780.
Path 311 | total_timesteps 2789.
Path 312 | total_timesteps 2796.
Path 313 | total_timesteps 2813.
Path 314 | total_timesteps 2823.
Path 315 | total_timesteps 2832.
Path 316 | total_timesteps 2841.
Path 317 | total_timesteps 2848.
Path 318 | total_timesteps 2857.
Path 319 | total_timesteps 2866.
Path 320 | total_timesteps 2874.
Path 321 | total_timesteps 2882.
Path 322 | total_timesteps 2890.
Path 323 | total_timesteps 2899.
Path 324 | total_timesteps 2907.
Path 325 | total_timesteps 2916.
Path 326 | total_timesteps 2923.
Path 327 | total_timesteps 2929.
Path 328 | total_timesteps 2935.
Path 329 | total_timesteps 2943.
Path 330 | total_timesteps 2951.
Path 331 | total_timesteps 2963.
Path 332 | total_timesteps 2972.
Path 333 | total_timesteps 2979.
Path 334 | total_timesteps 2990.
Path 335 | total_timesteps 2996.
Path 336 | total_timesteps 3004.
Path 337 | total_timesteps 3013.
Path 338 | total_timesteps 3021.
Path 339 | total_timesteps 3029.
Path 340 | total_timesteps 3035.
Path 341 | total_timesteps 3042.
Path 342 | total_timesteps 3049.
Path 343 | total_timesteps 3056.
Path 344 | total_timesteps 3062.
Path 345 | total_timesteps 3068.
Path 346 | total_timesteps 3079.
Path 347 | total_timesteps 3091.
Path 348 | total_timesteps 3102.
Path 349 | total_timesteps 3108.
Path 350 | total_timesteps 3115.
Path 351 | total_timesteps 3126.
Path 352 | total_timesteps 3136.
Path 353 | total_timesteps 3143.
Path 354 | total_timesteps 3151.
Path 355 | total_timesteps 3157.
Path 356 | total_timesteps 3165.
Path 357 | total_timesteps 3179.
Path 358 | total_timesteps 3188.
Path 359 | total_timesteps 3199.
Path 360 | total_timesteps 3207.
Path 361 | total_timesteps 3216.
Path 362 | total_timesteps 3225.
Path 363 | total_timesteps 3232.
Path 364 | total_timesteps 3240.
Path 365 | total_timesteps 3249.
Path 366 | total_timesteps 3258.
Path 367 | total_timesteps 3265.
Path 368 | total_timesteps 3272.
Path 369 | total_timesteps 3284.
Path 370 | total_timesteps 3298.
Path 371 | total_timesteps 3305.
Path 372 | total_timesteps 3313.
Path 373 | total_timesteps 3321.
Path 374 | total_timesteps 3332.
Path 375 | total_timesteps 3339.
Path 376 | total_timesteps 3351.
Path 377 | total_timesteps 3358.
Path 378 | total_timesteps 3370.
Path 379 | total_timesteps 3378.
Path 380 | total_timesteps 3388.
Path 381 | total_timesteps 3396.
Path 382 | total_timesteps 3403.
Path 383 | total_timesteps 3410.
Path 384 | total_timesteps 3421.
Path 385 | total_timesteps 3430.
Path 386 | total_timesteps 3438.
Path 387 | total_timesteps 3444.
Path 388 | total_timesteps 3454.
Path 389 | total_timesteps 3460.
Path 390 | total_timesteps 3466.
Path 391 | total_timesteps 3473.
Path 392 | total_timesteps 3481.
Path 393 | total_timesteps 3488.
Path 394 | total_timesteps 3496.
Path 395 | total_timesteps 3503.
Path 396 | total_timesteps 3510.
Path 397 | total_timesteps 3516.
Path 398 | total_timesteps 3523.
Path 399 | total_timesteps 3529.
Path 400 | total_timesteps 3538.
Path 401 | total_timesteps 3544.
Path 402 | total_timesteps 3551.
Path 403 | total_timesteps 3560.
Path 404 | total_timesteps 3572.
Path 405 | total_timesteps 3580.
Path 406 | total_timesteps 3587.
Path 407 | total_timesteps 3598.
Path 408 | total_timesteps 3606.
Path 409 | total_timesteps 3616.
Path 410 | total_timesteps 3623.
Path 411 | total_timesteps 3632.
Path 412 | total_timesteps 3641.
Path 413 | total_timesteps 3649.
Path 414 | total_timesteps 3656.
Path 415 | total_timesteps 3667.
Path 416 | total_timesteps 3674.
Path 417 | total_timesteps 3681.
Path 418 | total_timesteps 3689.
Path 419 | total_timesteps 3702.
Path 420 | total_timesteps 3712.
Path 421 | total_timesteps 3724.
Path 422 | total_timesteps 3731.
Path 423 | total_timesteps 3741.
Path 424 | total_timesteps 3748.
Path 425 | total_timesteps 3756.
Path 426 | total_timesteps 3766.
Path 427 | total_timesteps 3776.
Path 428 | total_timesteps 3783.
Path 429 | total_timesteps 3791.
Path 430 | total_timesteps 3800.
Path 431 | total_timesteps 3813.
Path 432 | total_timesteps 3824.
Path 433 | total_timesteps 3831.
Path 434 | total_timesteps 3840.
Path 435 | total_timesteps 3848.
Path 436 | total_timesteps 3857.
Path 437 | total_timesteps 3865.
Path 438 | total_timesteps 3875.
Path 439 | total_timesteps 3883.
Path 440 | total_timesteps 3893.
Path 441 | total_timesteps 3903.
Path 442 | total_timesteps 3911.
Path 443 | total_timesteps 3919.
Path 444 | total_timesteps 3926.
Path 445 | total_timesteps 3936.
Path 446 | total_timesteps 3944.
Path 447 | total_timesteps 3954.
Path 448 | total_timesteps 3963.
Path 449 | total_timesteps 3971.
Path 450 | total_timesteps 3979.
Path 451 | total_timesteps 3991.
Path 452 | total_timesteps 4001.
Path 453 | total_timesteps 4009.
Path 454 | total_timesteps 4017.
Path 455 | total_timesteps 4025.
Path 456 | total_timesteps 4033.
Path 457 | total_timesteps 4041.
Path 458 | total_timesteps 4050.
Path 459 | total_timesteps 4061.
Path 460 | total_timesteps 4075.
Path 461 | total_timesteps 4082.
Path 462 | total_timesteps 4096.
Path 463 | total_timesteps 4104.
Path 464 | total_timesteps 4111.
Path 465 | total_timesteps 4119.
Path 466 | total_timesteps 4126.
Path 467 | total_timesteps 4133.
Path 468 | total_timesteps 4140.
Path 469 | total_timesteps 4147.
Path 470 | total_timesteps 4154.
Path 471 | total_timesteps 4163.
Path 472 | total_timesteps 4170.
Path 473 | total_timesteps 4178.
Path 474 | total_timesteps 4186.
Path 475 | total_timesteps 4193.
Path 476 | total_timesteps 4200.
Path 477 | total_timesteps 4208.
Path 478 | total_timesteps 4217.
Path 479 | total_timesteps 4227.
Path 480 | total_timesteps 4240.
Path 481 | total_timesteps 4247.
Path 482 | total_timesteps 4256.
Path 483 | total_timesteps 4267.
Path 484 | total_timesteps 4276.
Path 485 | total_timesteps 4286.
Path 486 | total_timesteps 4295.
Path 487 | total_timesteps 4302.
Path 488 | total_timesteps 4315.
Path 489 | total_timesteps 4322.
Path 490 | total_timesteps 4329.
Path 491 | total_timesteps 4338.
Path 492 | total_timesteps 4346.
Path 493 | total_timesteps 4354.
Path 494 | total_timesteps 4363.
Path 495 | total_timesteps 4373.
Path 496 | total_timesteps 4380.
Path 497 | total_timesteps 4389.
Path 498 | total_timesteps 4400.
Path 499 | total_timesteps 4408.
Path 500 | total_timesteps 4416.
Path 501 | total_timesteps 4424.
Path 502 | total_timesteps 4430.
Path 503 | total_timesteps 4437.
Path 504 | total_timesteps 4450.
Path 505 | total_timesteps 4457.
Path 506 | total_timesteps 4464.
Path 507 | total_timesteps 4471.
Path 508 | total_timesteps 4480.
Path 509 | total_timesteps 4487.
Path 510 | total_timesteps 4495.
Path 511 | total_timesteps 4504.
Path 512 | total_timesteps 4511.
Path 513 | total_timesteps 4519.
Path 514 | total_timesteps 4534.
Path 515 | total_timesteps 4543.
Path 516 | total_timesteps 4550.
Path 517 | total_timesteps 4558.
Path 518 | total_timesteps 4572.
Path 519 | total_timesteps 4580.
Path 520 | total_timesteps 4587.
Path 521 | total_timesteps 4596.
Path 522 | total_timesteps 4604.
Path 523 | total_timesteps 4613.
Path 524 | total_timesteps 4619.
Path 525 | total_timesteps 4627.
Path 526 | total_timesteps 4636.
Path 527 | total_timesteps 4646.
Path 528 | total_timesteps 4653.
Path 529 | total_timesteps 4662.
Path 530 | total_timesteps 4669.
Path 531 | total_timesteps 4675.
Path 532 | total_timesteps 4688.
Path 533 | total_timesteps 4699.
Path 534 | total_timesteps 4711.
Path 535 | total_timesteps 4720.
Path 536 | total_timesteps 4730.
Path 537 | total_timesteps 4737.
Path 538 | total_timesteps 4749.
Path 539 | total_timesteps 4756.
Path 540 | total_timesteps 4764.
Path 541 | total_timesteps 4775.
Path 542 | total_timesteps 4783.
Path 543 | total_timesteps 4792.
Path 544 | total_timesteps 4801.
Path 545 | total_timesteps 4810.
Path 546 | total_timesteps 4818.
Path 547 | total_timesteps 4828.
Path 548 | total_timesteps 4836.
Path 549 | total_timesteps 4842.
Path 550 | total_timesteps 4848.
Path 551 | total_timesteps 4857.
Path 552 | total_timesteps 4865.
Path 553 | total_timesteps 4873.
Path 554 | total_timesteps 4879.
Path 555 | total_timesteps 4888.
Path 556 | total_timesteps 4896.
Path 557 | total_timesteps 4903.
Path 558 | total_timesteps 4910.
Path 559 | total_timesteps 4920.
Path 560 | total_timesteps 4929.
Path 561 | total_timesteps 4939.
Path 562 | total_timesteps 4946.
Path 563 | total_timesteps 4955.
Path 564 | total_timesteps 4962.
Path 565 | total_timesteps 4972.
Path 566 | total_timesteps 4984.
Path 567 | total_timesteps 4995.
Path 568 | total_timesteps 5004.
Path 569 | total_timesteps 5014.
Path 570 | total_timesteps 5024.
Path 571 | total_timesteps 5033.
Path 572 | total_timesteps 5039.
Path 573 | total_timesteps 5045.
Path 574 | total_timesteps 5057.
Path 575 | total_timesteps 5068.
Path 576 | total_timesteps 5080.
Path 577 | total_timesteps 5087.
Path 578 | total_timesteps 5095.
Path 579 | total_timesteps 5103.
Path 580 | total_timesteps 5110.
Path 581 | total_timesteps 5120.
Path 582 | total_timesteps 5127.
Path 583 | total_timesteps 5139.
Path 584 | total_timesteps 5148.
Path 585 | total_timesteps 5155.
Path 586 | total_timesteps 5165.
Path 587 | total_timesteps 5174.
Path 588 | total_timesteps 5183.
Path 589 | total_timesteps 5191.
Path 590 | total_timesteps 5199.
Path 591 | total_timesteps 5207.
Path 592 | total_timesteps 5213.
Path 593 | total_timesteps 5221.
Path 594 | total_timesteps 5229.
Path 595 | total_timesteps 5239.
Path 596 | total_timesteps 5250.
Path 597 | total_timesteps 5264.
Path 598 | total_timesteps 5271.
Path 599 | total_timesteps 5282.
Path 600 | total_timesteps 5288.
Path 601 | total_timesteps 5296.
Path 602 | total_timesteps 5304.
Path 603 | total_timesteps 5310.
Path 604 | total_timesteps 5319.
Path 605 | total_timesteps 5326.
Path 606 | total_timesteps 5335.
Path 607 | total_timesteps 5348.
Path 608 | total_timesteps 5360.
Path 609 | total_timesteps 5371.
Path 610 | total_timesteps 5383.
Path 611 | total_timesteps 5390.
Path 612 | total_timesteps 5398.
Path 613 | total_timesteps 5405.
Path 614 | total_timesteps 5413.
Path 615 | total_timesteps 5424.
Path 616 | total_timesteps 5430.
Path 617 | total_timesteps 5437.
Path 618 | total_timesteps 5444.
Path 619 | total_timesteps 5451.
Path 620 | total_timesteps 5458.
Path 621 | total_timesteps 5465.
Path 622 | total_timesteps 5474.
Path 623 | total_timesteps 5481.
Path 624 | total_timesteps 5489.
Path 625 | total_timesteps 5498.
Path 626 | total_timesteps 5505.
Path 627 | total_timesteps 5513.
Path 628 | total_timesteps 5521.
Path 629 | total_timesteps 5528.
Path 630 | total_timesteps 5542.
Path 631 | total_timesteps 5553.
Path 632 | total_timesteps 5560.
Path 633 | total_timesteps 5581.
Path 634 | total_timesteps 5589.
Path 635 | total_timesteps 5599.
Path 636 | total_timesteps 5606.
Path 637 | total_timesteps 5618.
Path 638 | total_timesteps 5627.
Path 639 | total_timesteps 5635.
Path 640 | total_timesteps 5642.
Path 641 | total_timesteps 5651.
Path 642 | total_timesteps 5660.
Path 643 | total_timesteps 5666.
Path 644 | total_timesteps 5674.
Path 645 | total_timesteps 5682.
Path 646 | total_timesteps 5689.
Path 647 | total_timesteps 5696.
Path 648 | total_timesteps 5706.
Path 649 | total_timesteps 5716.
Path 650 | total_timesteps 5728.
Path 651 | total_timesteps 5736.
Path 652 | total_timesteps 5746.
Path 653 | total_timesteps 5755.
Path 654 | total_timesteps 5762.
Path 655 | total_timesteps 5769.
Path 656 | total_timesteps 5778.
Path 657 | total_timesteps 5784.
Path 658 | total_timesteps 5792.
Path 659 | total_timesteps 5802.
Path 660 | total_timesteps 5810.
Path 661 | total_timesteps 5818.
Path 662 | total_timesteps 5825.
Path 663 | total_timesteps 5833.
Path 664 | total_timesteps 5841.
Path 665 | total_timesteps 5850.
Path 666 | total_timesteps 5857.
Path 667 | total_timesteps 5865.
Path 668 | total_timesteps 5874.
Path 669 | total_timesteps 5882.
Path 670 | total_timesteps 5889.
Path 671 | total_timesteps 5898.
Path 672 | total_timesteps 5906.
Path 673 | total_timesteps 5913.
Path 674 | total_timesteps 5920.
Path 675 | total_timesteps 5928.
Path 676 | total_timesteps 5938.
Path 677 | total_timesteps 5948.
Path 678 | total_timesteps 5958.
Path 679 | total_timesteps 5964.
Path 680 | total_timesteps 5975.
Path 681 | total_timesteps 5983.
Path 682 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.65    |
| Iteration     | 7        |
| MaximumReturn | 0.065    |
| MinimumReturn | -14.4    |
| TotalSamples  | 36041    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.0072039091028273106
Validation loss = 0.006765605416148901
Validation loss = 0.006072892341762781
Validation loss = 0.006158575881272554
Validation loss = 0.006068510934710503
Validation loss = 0.006751103326678276
Validation loss = 0.006157654337584972
Validation loss = 0.006068236194550991
Validation loss = 0.005876177456229925
Validation loss = 0.006323277484625578
Validation loss = 0.006943304557353258
Validation loss = 0.0061043016612529755
Validation loss = 0.005869487300515175
Validation loss = 0.006471136584877968
Validation loss = 0.006046263035386801
Validation loss = 0.006032894365489483
Validation loss = 0.005651820916682482
Validation loss = 0.005972275510430336
Validation loss = 0.005427663680166006
Validation loss = 0.006892894394695759
Validation loss = 0.005880505312234163
Validation loss = 0.006059839390218258
Validation loss = 0.006298981606960297
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 64.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 78.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 94.
Path 12 | total_timesteps 105.
Path 13 | total_timesteps 114.
Path 14 | total_timesteps 122.
Path 15 | total_timesteps 132.
Path 16 | total_timesteps 141.
Path 17 | total_timesteps 149.
Path 18 | total_timesteps 165.
Path 19 | total_timesteps 174.
Path 20 | total_timesteps 187.
Path 21 | total_timesteps 197.
Path 22 | total_timesteps 213.
Path 23 | total_timesteps 221.
Path 24 | total_timesteps 229.
Path 25 | total_timesteps 238.
Path 26 | total_timesteps 244.
Path 27 | total_timesteps 263.
Path 28 | total_timesteps 271.
Path 29 | total_timesteps 278.
Path 30 | total_timesteps 287.
Path 31 | total_timesteps 293.
Path 32 | total_timesteps 305.
Path 33 | total_timesteps 316.
Path 34 | total_timesteps 327.
Path 35 | total_timesteps 334.
Path 36 | total_timesteps 347.
Path 37 | total_timesteps 355.
Path 38 | total_timesteps 362.
Path 39 | total_timesteps 373.
Path 40 | total_timesteps 381.
Path 41 | total_timesteps 389.
Path 42 | total_timesteps 396.
Path 43 | total_timesteps 406.
Path 44 | total_timesteps 414.
Path 45 | total_timesteps 423.
Path 46 | total_timesteps 431.
Path 47 | total_timesteps 443.
Path 48 | total_timesteps 451.
Path 49 | total_timesteps 460.
Path 50 | total_timesteps 466.
Path 51 | total_timesteps 473.
Path 52 | total_timesteps 481.
Path 53 | total_timesteps 490.
Path 54 | total_timesteps 501.
Path 55 | total_timesteps 511.
Path 56 | total_timesteps 520.
Path 57 | total_timesteps 530.
Path 58 | total_timesteps 537.
Path 59 | total_timesteps 547.
Path 60 | total_timesteps 554.
Path 61 | total_timesteps 563.
Path 62 | total_timesteps 570.
Path 63 | total_timesteps 584.
Path 64 | total_timesteps 593.
Path 65 | total_timesteps 601.
Path 66 | total_timesteps 609.
Path 67 | total_timesteps 617.
Path 68 | total_timesteps 625.
Path 69 | total_timesteps 635.
Path 70 | total_timesteps 642.
Path 71 | total_timesteps 652.
Path 72 | total_timesteps 660.
Path 73 | total_timesteps 670.
Path 74 | total_timesteps 677.
Path 75 | total_timesteps 689.
Path 76 | total_timesteps 698.
Path 77 | total_timesteps 704.
Path 78 | total_timesteps 713.
Path 79 | total_timesteps 724.
Path 80 | total_timesteps 731.
Path 81 | total_timesteps 741.
Path 82 | total_timesteps 752.
Path 83 | total_timesteps 764.
Path 84 | total_timesteps 771.
Path 85 | total_timesteps 779.
Path 86 | total_timesteps 788.
Path 87 | total_timesteps 794.
Path 88 | total_timesteps 804.
Path 89 | total_timesteps 815.
Path 90 | total_timesteps 827.
Path 91 | total_timesteps 837.
Path 92 | total_timesteps 844.
Path 93 | total_timesteps 851.
Path 94 | total_timesteps 858.
Path 95 | total_timesteps 865.
Path 96 | total_timesteps 877.
Path 97 | total_timesteps 884.
Path 98 | total_timesteps 895.
Path 99 | total_timesteps 902.
Path 100 | total_timesteps 913.
Path 101 | total_timesteps 921.
Path 102 | total_timesteps 929.
Path 103 | total_timesteps 937.
Path 104 | total_timesteps 945.
Path 105 | total_timesteps 952.
Path 106 | total_timesteps 958.
Path 107 | total_timesteps 966.
Path 108 | total_timesteps 973.
Path 109 | total_timesteps 980.
Path 110 | total_timesteps 987.
Path 111 | total_timesteps 997.
Path 112 | total_timesteps 1008.
Path 113 | total_timesteps 1018.
Path 114 | total_timesteps 1028.
Path 115 | total_timesteps 1036.
Path 116 | total_timesteps 1047.
Path 117 | total_timesteps 1055.
Path 118 | total_timesteps 1064.
Path 119 | total_timesteps 1073.
Path 120 | total_timesteps 1082.
Path 121 | total_timesteps 1090.
Path 122 | total_timesteps 1096.
Path 123 | total_timesteps 1107.
Path 124 | total_timesteps 1114.
Path 125 | total_timesteps 1123.
Path 126 | total_timesteps 1133.
Path 127 | total_timesteps 1142.
Path 128 | total_timesteps 1149.
Path 129 | total_timesteps 1158.
Path 130 | total_timesteps 1170.
Path 131 | total_timesteps 1176.
Path 132 | total_timesteps 1187.
Path 133 | total_timesteps 1193.
Path 134 | total_timesteps 1205.
Path 135 | total_timesteps 1214.
Path 136 | total_timesteps 1223.
Path 137 | total_timesteps 1231.
Path 138 | total_timesteps 1239.
Path 139 | total_timesteps 1249.
Path 140 | total_timesteps 1255.
Path 141 | total_timesteps 1263.
Path 142 | total_timesteps 1274.
Path 143 | total_timesteps 1281.
Path 144 | total_timesteps 1291.
Path 145 | total_timesteps 1298.
Path 146 | total_timesteps 1307.
Path 147 | total_timesteps 1315.
Path 148 | total_timesteps 1325.
Path 149 | total_timesteps 1336.
Path 150 | total_timesteps 1346.
Path 151 | total_timesteps 1355.
Path 152 | total_timesteps 1365.
Path 153 | total_timesteps 1372.
Path 154 | total_timesteps 1378.
Path 155 | total_timesteps 1385.
Path 156 | total_timesteps 1392.
Path 157 | total_timesteps 1399.
Path 158 | total_timesteps 1413.
Path 159 | total_timesteps 1421.
Path 160 | total_timesteps 1428.
Path 161 | total_timesteps 1435.
Path 162 | total_timesteps 1446.
Path 163 | total_timesteps 1454.
Path 164 | total_timesteps 1461.
Path 165 | total_timesteps 1469.
Path 166 | total_timesteps 1479.
Path 167 | total_timesteps 1489.
Path 168 | total_timesteps 1497.
Path 169 | total_timesteps 1506.
Path 170 | total_timesteps 1516.
Path 171 | total_timesteps 1527.
Path 172 | total_timesteps 1535.
Path 173 | total_timesteps 1544.
Path 174 | total_timesteps 1553.
Path 175 | total_timesteps 1560.
Path 176 | total_timesteps 1570.
Path 177 | total_timesteps 1578.
Path 178 | total_timesteps 1588.
Path 179 | total_timesteps 1595.
Path 180 | total_timesteps 1604.
Path 181 | total_timesteps 1614.
Path 182 | total_timesteps 1623.
Path 183 | total_timesteps 1633.
Path 184 | total_timesteps 1641.
Path 185 | total_timesteps 1654.
Path 186 | total_timesteps 1662.
Path 187 | total_timesteps 1675.
Path 188 | total_timesteps 1681.
Path 189 | total_timesteps 1688.
Path 190 | total_timesteps 1698.
Path 191 | total_timesteps 1707.
Path 192 | total_timesteps 1718.
Path 193 | total_timesteps 1733.
Path 194 | total_timesteps 1748.
Path 195 | total_timesteps 1758.
Path 196 | total_timesteps 1765.
Path 197 | total_timesteps 1772.
Path 198 | total_timesteps 1781.
Path 199 | total_timesteps 1791.
Path 200 | total_timesteps 1798.
Path 201 | total_timesteps 1808.
Path 202 | total_timesteps 1815.
Path 203 | total_timesteps 1825.
Path 204 | total_timesteps 1834.
Path 205 | total_timesteps 1844.
Path 206 | total_timesteps 1854.
Path 207 | total_timesteps 1863.
Path 208 | total_timesteps 1870.
Path 209 | total_timesteps 1880.
Path 210 | total_timesteps 1888.
Path 211 | total_timesteps 1896.
Path 212 | total_timesteps 1907.
Path 213 | total_timesteps 1915.
Path 214 | total_timesteps 1925.
Path 215 | total_timesteps 1940.
Path 216 | total_timesteps 1948.
Path 217 | total_timesteps 1958.
Path 218 | total_timesteps 1967.
Path 219 | total_timesteps 1977.
Path 220 | total_timesteps 1986.
Path 221 | total_timesteps 1993.
Path 222 | total_timesteps 2002.
Path 223 | total_timesteps 2009.
Path 224 | total_timesteps 2016.
Path 225 | total_timesteps 2022.
Path 226 | total_timesteps 2037.
Path 227 | total_timesteps 2045.
Path 228 | total_timesteps 2051.
Path 229 | total_timesteps 2058.
Path 230 | total_timesteps 2067.
Path 231 | total_timesteps 2076.
Path 232 | total_timesteps 2086.
Path 233 | total_timesteps 2096.
Path 234 | total_timesteps 2105.
Path 235 | total_timesteps 2114.
Path 236 | total_timesteps 2127.
Path 237 | total_timesteps 2134.
Path 238 | total_timesteps 2147.
Path 239 | total_timesteps 2154.
Path 240 | total_timesteps 2168.
Path 241 | total_timesteps 2180.
Path 242 | total_timesteps 2194.
Path 243 | total_timesteps 2202.
Path 244 | total_timesteps 2211.
Path 245 | total_timesteps 2219.
Path 246 | total_timesteps 2226.
Path 247 | total_timesteps 2234.
Path 248 | total_timesteps 2243.
Path 249 | total_timesteps 2251.
Path 250 | total_timesteps 2263.
Path 251 | total_timesteps 2270.
Path 252 | total_timesteps 2278.
Path 253 | total_timesteps 2285.
Path 254 | total_timesteps 2292.
Path 255 | total_timesteps 2301.
Path 256 | total_timesteps 2309.
Path 257 | total_timesteps 2317.
Path 258 | total_timesteps 2329.
Path 259 | total_timesteps 2345.
Path 260 | total_timesteps 2352.
Path 261 | total_timesteps 2359.
Path 262 | total_timesteps 2366.
Path 263 | total_timesteps 2373.
Path 264 | total_timesteps 2386.
Path 265 | total_timesteps 2398.
Path 266 | total_timesteps 2404.
Path 267 | total_timesteps 2410.
Path 268 | total_timesteps 2418.
Path 269 | total_timesteps 2426.
Path 270 | total_timesteps 2435.
Path 271 | total_timesteps 2442.
Path 272 | total_timesteps 2454.
Path 273 | total_timesteps 2462.
Path 274 | total_timesteps 2470.
Path 275 | total_timesteps 2478.
Path 276 | total_timesteps 2488.
Path 277 | total_timesteps 2497.
Path 278 | total_timesteps 2507.
Path 279 | total_timesteps 2514.
Path 280 | total_timesteps 2522.
Path 281 | total_timesteps 2533.
Path 282 | total_timesteps 2540.
Path 283 | total_timesteps 2548.
Path 284 | total_timesteps 2557.
Path 285 | total_timesteps 2564.
Path 286 | total_timesteps 2571.
Path 287 | total_timesteps 2581.
Path 288 | total_timesteps 2608.
Path 289 | total_timesteps 2615.
Path 290 | total_timesteps 2627.
Path 291 | total_timesteps 2635.
Path 292 | total_timesteps 2643.
Path 293 | total_timesteps 2653.
Path 294 | total_timesteps 2659.
Path 295 | total_timesteps 2667.
Path 296 | total_timesteps 2679.
Path 297 | total_timesteps 2685.
Path 298 | total_timesteps 2697.
Path 299 | total_timesteps 2707.
Path 300 | total_timesteps 2714.
Path 301 | total_timesteps 2720.
Path 302 | total_timesteps 2728.
Path 303 | total_timesteps 2738.
Path 304 | total_timesteps 2746.
Path 305 | total_timesteps 2759.
Path 306 | total_timesteps 2767.
Path 307 | total_timesteps 2775.
Path 308 | total_timesteps 2788.
Path 309 | total_timesteps 2795.
Path 310 | total_timesteps 2805.
Path 311 | total_timesteps 2812.
Path 312 | total_timesteps 2826.
Path 313 | total_timesteps 2838.
Path 314 | total_timesteps 2847.
Path 315 | total_timesteps 2854.
Path 316 | total_timesteps 2863.
Path 317 | total_timesteps 2871.
Path 318 | total_timesteps 2878.
Path 319 | total_timesteps 2890.
Path 320 | total_timesteps 2898.
Path 321 | total_timesteps 2907.
Path 322 | total_timesteps 2915.
Path 323 | total_timesteps 2923.
Path 324 | total_timesteps 2934.
Path 325 | total_timesteps 2943.
Path 326 | total_timesteps 2951.
Path 327 | total_timesteps 2959.
Path 328 | total_timesteps 2967.
Path 329 | total_timesteps 2975.
Path 330 | total_timesteps 2982.
Path 331 | total_timesteps 2991.
Path 332 | total_timesteps 3001.
Path 333 | total_timesteps 3008.
Path 334 | total_timesteps 3017.
Path 335 | total_timesteps 3026.
Path 336 | total_timesteps 3034.
Path 337 | total_timesteps 3043.
Path 338 | total_timesteps 3053.
Path 339 | total_timesteps 3061.
Path 340 | total_timesteps 3067.
Path 341 | total_timesteps 3076.
Path 342 | total_timesteps 3084.
Path 343 | total_timesteps 3094.
Path 344 | total_timesteps 3106.
Path 345 | total_timesteps 3116.
Path 346 | total_timesteps 3123.
Path 347 | total_timesteps 3132.
Path 348 | total_timesteps 3139.
Path 349 | total_timesteps 3148.
Path 350 | total_timesteps 3156.
Path 351 | total_timesteps 3169.
Path 352 | total_timesteps 3181.
Path 353 | total_timesteps 3190.
Path 354 | total_timesteps 3200.
Path 355 | total_timesteps 3209.
Path 356 | total_timesteps 3218.
Path 357 | total_timesteps 3226.
Path 358 | total_timesteps 3233.
Path 359 | total_timesteps 3240.
Path 360 | total_timesteps 3248.
Path 361 | total_timesteps 3261.
Path 362 | total_timesteps 3270.
Path 363 | total_timesteps 3278.
Path 364 | total_timesteps 3284.
Path 365 | total_timesteps 3291.
Path 366 | total_timesteps 3297.
Path 367 | total_timesteps 3307.
Path 368 | total_timesteps 3319.
Path 369 | total_timesteps 3326.
Path 370 | total_timesteps 3333.
Path 371 | total_timesteps 3340.
Path 372 | total_timesteps 3351.
Path 373 | total_timesteps 3364.
Path 374 | total_timesteps 3376.
Path 375 | total_timesteps 3384.
Path 376 | total_timesteps 3391.
Path 377 | total_timesteps 3401.
Path 378 | total_timesteps 3409.
Path 379 | total_timesteps 3420.
Path 380 | total_timesteps 3427.
Path 381 | total_timesteps 3435.
Path 382 | total_timesteps 3445.
Path 383 | total_timesteps 3457.
Path 384 | total_timesteps 3466.
Path 385 | total_timesteps 3477.
Path 386 | total_timesteps 3491.
Path 387 | total_timesteps 3504.
Path 388 | total_timesteps 3510.
Path 389 | total_timesteps 3517.
Path 390 | total_timesteps 3524.
Path 391 | total_timesteps 3531.
Path 392 | total_timesteps 3539.
Path 393 | total_timesteps 3549.
Path 394 | total_timesteps 3558.
Path 395 | total_timesteps 3566.
Path 396 | total_timesteps 3577.
Path 397 | total_timesteps 3583.
Path 398 | total_timesteps 3593.
Path 399 | total_timesteps 3602.
Path 400 | total_timesteps 3611.
Path 401 | total_timesteps 3618.
Path 402 | total_timesteps 3627.
Path 403 | total_timesteps 3634.
Path 404 | total_timesteps 3649.
Path 405 | total_timesteps 3656.
Path 406 | total_timesteps 3665.
Path 407 | total_timesteps 3673.
Path 408 | total_timesteps 3680.
Path 409 | total_timesteps 3688.
Path 410 | total_timesteps 3697.
Path 411 | total_timesteps 3706.
Path 412 | total_timesteps 3715.
Path 413 | total_timesteps 3723.
Path 414 | total_timesteps 3732.
Path 415 | total_timesteps 3738.
Path 416 | total_timesteps 3745.
Path 417 | total_timesteps 3752.
Path 418 | total_timesteps 3760.
Path 419 | total_timesteps 3768.
Path 420 | total_timesteps 3778.
Path 421 | total_timesteps 3787.
Path 422 | total_timesteps 3794.
Path 423 | total_timesteps 3807.
Path 424 | total_timesteps 3815.
Path 425 | total_timesteps 3824.
Path 426 | total_timesteps 3834.
Path 427 | total_timesteps 3842.
Path 428 | total_timesteps 3854.
Path 429 | total_timesteps 3862.
Path 430 | total_timesteps 3870.
Path 431 | total_timesteps 3879.
Path 432 | total_timesteps 3888.
Path 433 | total_timesteps 3904.
Path 434 | total_timesteps 3918.
Path 435 | total_timesteps 3927.
Path 436 | total_timesteps 3935.
Path 437 | total_timesteps 3944.
Path 438 | total_timesteps 3951.
Path 439 | total_timesteps 3961.
Path 440 | total_timesteps 3970.
Path 441 | total_timesteps 3978.
Path 442 | total_timesteps 3985.
Path 443 | total_timesteps 3993.
Path 444 | total_timesteps 4000.
Path 445 | total_timesteps 4007.
Path 446 | total_timesteps 4016.
Path 447 | total_timesteps 4023.
Path 448 | total_timesteps 4033.
Path 449 | total_timesteps 4042.
Path 450 | total_timesteps 4054.
Path 451 | total_timesteps 4068.
Path 452 | total_timesteps 4075.
Path 453 | total_timesteps 4087.
Path 454 | total_timesteps 4095.
Path 455 | total_timesteps 4106.
Path 456 | total_timesteps 4114.
Path 457 | total_timesteps 4127.
Path 458 | total_timesteps 4136.
Path 459 | total_timesteps 4144.
Path 460 | total_timesteps 4157.
Path 461 | total_timesteps 4165.
Path 462 | total_timesteps 4172.
Path 463 | total_timesteps 4184.
Path 464 | total_timesteps 4196.
Path 465 | total_timesteps 4202.
Path 466 | total_timesteps 4214.
Path 467 | total_timesteps 4225.
Path 468 | total_timesteps 4235.
Path 469 | total_timesteps 4243.
Path 470 | total_timesteps 4249.
Path 471 | total_timesteps 4258.
Path 472 | total_timesteps 4267.
Path 473 | total_timesteps 4274.
Path 474 | total_timesteps 4288.
Path 475 | total_timesteps 4296.
Path 476 | total_timesteps 4304.
Path 477 | total_timesteps 4312.
Path 478 | total_timesteps 4322.
Path 479 | total_timesteps 4329.
Path 480 | total_timesteps 4341.
Path 481 | total_timesteps 4351.
Path 482 | total_timesteps 4359.
Path 483 | total_timesteps 4367.
Path 484 | total_timesteps 4374.
Path 485 | total_timesteps 4381.
Path 486 | total_timesteps 4388.
Path 487 | total_timesteps 4396.
Path 488 | total_timesteps 4404.
Path 489 | total_timesteps 4413.
Path 490 | total_timesteps 4421.
Path 491 | total_timesteps 4430.
Path 492 | total_timesteps 4437.
Path 493 | total_timesteps 4444.
Path 494 | total_timesteps 4455.
Path 495 | total_timesteps 4463.
Path 496 | total_timesteps 4473.
Path 497 | total_timesteps 4488.
Path 498 | total_timesteps 4494.
Path 499 | total_timesteps 4503.
Path 500 | total_timesteps 4513.
Path 501 | total_timesteps 4525.
Path 502 | total_timesteps 4536.
Path 503 | total_timesteps 4547.
Path 504 | total_timesteps 4553.
Path 505 | total_timesteps 4562.
Path 506 | total_timesteps 4572.
Path 507 | total_timesteps 4579.
Path 508 | total_timesteps 4586.
Path 509 | total_timesteps 4598.
Path 510 | total_timesteps 4607.
Path 511 | total_timesteps 4620.
Path 512 | total_timesteps 4627.
Path 513 | total_timesteps 4635.
Path 514 | total_timesteps 4649.
Path 515 | total_timesteps 4658.
Path 516 | total_timesteps 4666.
Path 517 | total_timesteps 4675.
Path 518 | total_timesteps 4683.
Path 519 | total_timesteps 4690.
Path 520 | total_timesteps 4696.
Path 521 | total_timesteps 4705.
Path 522 | total_timesteps 4713.
Path 523 | total_timesteps 4719.
Path 524 | total_timesteps 4729.
Path 525 | total_timesteps 4738.
Path 526 | total_timesteps 4747.
Path 527 | total_timesteps 4756.
Path 528 | total_timesteps 4764.
Path 529 | total_timesteps 4772.
Path 530 | total_timesteps 4782.
Path 531 | total_timesteps 4790.
Path 532 | total_timesteps 4805.
Path 533 | total_timesteps 4814.
Path 534 | total_timesteps 4820.
Path 535 | total_timesteps 4830.
Path 536 | total_timesteps 4843.
Path 537 | total_timesteps 4854.
Path 538 | total_timesteps 4863.
Path 539 | total_timesteps 4870.
Path 540 | total_timesteps 4886.
Path 541 | total_timesteps 4895.
Path 542 | total_timesteps 4906.
Path 543 | total_timesteps 4915.
Path 544 | total_timesteps 4926.
Path 545 | total_timesteps 4934.
Path 546 | total_timesteps 4941.
Path 547 | total_timesteps 4950.
Path 548 | total_timesteps 4962.
Path 549 | total_timesteps 4971.
Path 550 | total_timesteps 4980.
Path 551 | total_timesteps 4987.
Path 552 | total_timesteps 5001.
Path 553 | total_timesteps 5011.
Path 554 | total_timesteps 5021.
Path 555 | total_timesteps 5027.
Path 556 | total_timesteps 5033.
Path 557 | total_timesteps 5043.
Path 558 | total_timesteps 5054.
Path 559 | total_timesteps 5061.
Path 560 | total_timesteps 5072.
Path 561 | total_timesteps 5079.
Path 562 | total_timesteps 5089.
Path 563 | total_timesteps 5096.
Path 564 | total_timesteps 5103.
Path 565 | total_timesteps 5110.
Path 566 | total_timesteps 5120.
Path 567 | total_timesteps 5128.
Path 568 | total_timesteps 5138.
Path 569 | total_timesteps 5146.
Path 570 | total_timesteps 5154.
Path 571 | total_timesteps 5165.
Path 572 | total_timesteps 5172.
Path 573 | total_timesteps 5179.
Path 574 | total_timesteps 5186.
Path 575 | total_timesteps 5195.
Path 576 | total_timesteps 5201.
Path 577 | total_timesteps 5209.
Path 578 | total_timesteps 5219.
Path 579 | total_timesteps 5226.
Path 580 | total_timesteps 5239.
Path 581 | total_timesteps 5250.
Path 582 | total_timesteps 5258.
Path 583 | total_timesteps 5273.
Path 584 | total_timesteps 5283.
Path 585 | total_timesteps 5290.
Path 586 | total_timesteps 5300.
Path 587 | total_timesteps 5308.
Path 588 | total_timesteps 5319.
Path 589 | total_timesteps 5328.
Path 590 | total_timesteps 5339.
Path 591 | total_timesteps 5345.
Path 592 | total_timesteps 5354.
Path 593 | total_timesteps 5362.
Path 594 | total_timesteps 5370.
Path 595 | total_timesteps 5379.
Path 596 | total_timesteps 5388.
Path 597 | total_timesteps 5399.
Path 598 | total_timesteps 5412.
Path 599 | total_timesteps 5423.
Path 600 | total_timesteps 5431.
Path 601 | total_timesteps 5439.
Path 602 | total_timesteps 5447.
Path 603 | total_timesteps 5455.
Path 604 | total_timesteps 5463.
Path 605 | total_timesteps 5475.
Path 606 | total_timesteps 5483.
Path 607 | total_timesteps 5489.
Path 608 | total_timesteps 5505.
Path 609 | total_timesteps 5519.
Path 610 | total_timesteps 5527.
Path 611 | total_timesteps 5534.
Path 612 | total_timesteps 5542.
Path 613 | total_timesteps 5550.
Path 614 | total_timesteps 5560.
Path 615 | total_timesteps 5569.
Path 616 | total_timesteps 5576.
Path 617 | total_timesteps 5583.
Path 618 | total_timesteps 5590.
Path 619 | total_timesteps 5597.
Path 620 | total_timesteps 5607.
Path 621 | total_timesteps 5625.
Path 622 | total_timesteps 5639.
Path 623 | total_timesteps 5648.
Path 624 | total_timesteps 5654.
Path 625 | total_timesteps 5662.
Path 626 | total_timesteps 5670.
Path 627 | total_timesteps 5683.
Path 628 | total_timesteps 5689.
Path 629 | total_timesteps 5698.
Path 630 | total_timesteps 5707.
Path 631 | total_timesteps 5715.
Path 632 | total_timesteps 5723.
Path 633 | total_timesteps 5730.
Path 634 | total_timesteps 5743.
Path 635 | total_timesteps 5752.
Path 636 | total_timesteps 5761.
Path 637 | total_timesteps 5767.
Path 638 | total_timesteps 5778.
Path 639 | total_timesteps 5786.
Path 640 | total_timesteps 5799.
Path 641 | total_timesteps 5807.
Path 642 | total_timesteps 5816.
Path 643 | total_timesteps 5827.
Path 644 | total_timesteps 5835.
Path 645 | total_timesteps 5842.
Path 646 | total_timesteps 5848.
Path 647 | total_timesteps 5856.
Path 648 | total_timesteps 5863.
Path 649 | total_timesteps 5875.
Path 650 | total_timesteps 5884.
Path 651 | total_timesteps 5891.
Path 652 | total_timesteps 5898.
Path 653 | total_timesteps 5905.
Path 654 | total_timesteps 5912.
Path 655 | total_timesteps 5924.
Path 656 | total_timesteps 5931.
Path 657 | total_timesteps 5945.
Path 658 | total_timesteps 5954.
Path 659 | total_timesteps 5960.
Path 660 | total_timesteps 5966.
Path 661 | total_timesteps 5974.
Path 662 | total_timesteps 5984.
Path 663 | total_timesteps 5991.
Path 664 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.78    |
| Iteration     | 8        |
| MaximumReturn | -1.03    |
| MinimumReturn | -16.7    |
| TotalSamples  | 40044    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.0053443643264472485
Validation loss = 0.005463392473757267
Validation loss = 0.005414520390331745
Validation loss = 0.005109794903546572
Validation loss = 0.004840515553951263
Validation loss = 0.0064724101684987545
Validation loss = 0.005589189939200878
Validation loss = 0.004963630810379982
Validation loss = 0.008006202057003975
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 14.
Path 3 | total_timesteps 21.
Path 4 | total_timesteps 29.
Path 5 | total_timesteps 38.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 57.
Path 8 | total_timesteps 68.
Path 9 | total_timesteps 75.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 95.
Path 12 | total_timesteps 104.
Path 13 | total_timesteps 111.
Path 14 | total_timesteps 122.
Path 15 | total_timesteps 129.
Path 16 | total_timesteps 142.
Path 17 | total_timesteps 151.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 169.
Path 20 | total_timesteps 177.
Path 21 | total_timesteps 185.
Path 22 | total_timesteps 191.
Path 23 | total_timesteps 200.
Path 24 | total_timesteps 208.
Path 25 | total_timesteps 216.
Path 26 | total_timesteps 224.
Path 27 | total_timesteps 237.
Path 28 | total_timesteps 245.
Path 29 | total_timesteps 253.
Path 30 | total_timesteps 260.
Path 31 | total_timesteps 270.
Path 32 | total_timesteps 276.
Path 33 | total_timesteps 284.
Path 34 | total_timesteps 290.
Path 35 | total_timesteps 299.
Path 36 | total_timesteps 307.
Path 37 | total_timesteps 314.
Path 38 | total_timesteps 326.
Path 39 | total_timesteps 334.
Path 40 | total_timesteps 342.
Path 41 | total_timesteps 351.
Path 42 | total_timesteps 362.
Path 43 | total_timesteps 372.
Path 44 | total_timesteps 379.
Path 45 | total_timesteps 387.
Path 46 | total_timesteps 398.
Path 47 | total_timesteps 410.
Path 48 | total_timesteps 418.
Path 49 | total_timesteps 425.
Path 50 | total_timesteps 436.
Path 51 | total_timesteps 445.
Path 52 | total_timesteps 453.
Path 53 | total_timesteps 464.
Path 54 | total_timesteps 471.
Path 55 | total_timesteps 477.
Path 56 | total_timesteps 487.
Path 57 | total_timesteps 495.
Path 58 | total_timesteps 504.
Path 59 | total_timesteps 517.
Path 60 | total_timesteps 535.
Path 61 | total_timesteps 543.
Path 62 | total_timesteps 551.
Path 63 | total_timesteps 559.
Path 64 | total_timesteps 569.
Path 65 | total_timesteps 584.
Path 66 | total_timesteps 598.
Path 67 | total_timesteps 607.
Path 68 | total_timesteps 619.
Path 69 | total_timesteps 628.
Path 70 | total_timesteps 635.
Path 71 | total_timesteps 643.
Path 72 | total_timesteps 661.
Path 73 | total_timesteps 669.
Path 74 | total_timesteps 675.
Path 75 | total_timesteps 683.
Path 76 | total_timesteps 698.
Path 77 | total_timesteps 709.
Path 78 | total_timesteps 717.
Path 79 | total_timesteps 724.
Path 80 | total_timesteps 731.
Path 81 | total_timesteps 740.
Path 82 | total_timesteps 748.
Path 83 | total_timesteps 756.
Path 84 | total_timesteps 764.
Path 85 | total_timesteps 774.
Path 86 | total_timesteps 782.
Path 87 | total_timesteps 790.
Path 88 | total_timesteps 799.
Path 89 | total_timesteps 807.
Path 90 | total_timesteps 818.
Path 91 | total_timesteps 827.
Path 92 | total_timesteps 835.
Path 93 | total_timesteps 842.
Path 94 | total_timesteps 853.
Path 95 | total_timesteps 859.
Path 96 | total_timesteps 867.
Path 97 | total_timesteps 877.
Path 98 | total_timesteps 887.
Path 99 | total_timesteps 894.
Path 100 | total_timesteps 902.
Path 101 | total_timesteps 912.
Path 102 | total_timesteps 926.
Path 103 | total_timesteps 942.
Path 104 | total_timesteps 953.
Path 105 | total_timesteps 962.
Path 106 | total_timesteps 971.
Path 107 | total_timesteps 977.
Path 108 | total_timesteps 991.
Path 109 | total_timesteps 1000.
Path 110 | total_timesteps 1007.
Path 111 | total_timesteps 1013.
Path 112 | total_timesteps 1023.
Path 113 | total_timesteps 1031.
Path 114 | total_timesteps 1038.
Path 115 | total_timesteps 1046.
Path 116 | total_timesteps 1057.
Path 117 | total_timesteps 1064.
Path 118 | total_timesteps 1074.
Path 119 | total_timesteps 1082.
Path 120 | total_timesteps 1089.
Path 121 | total_timesteps 1097.
Path 122 | total_timesteps 1109.
Path 123 | total_timesteps 1116.
Path 124 | total_timesteps 1125.
Path 125 | total_timesteps 1133.
Path 126 | total_timesteps 1140.
Path 127 | total_timesteps 1149.
Path 128 | total_timesteps 1162.
Path 129 | total_timesteps 1171.
Path 130 | total_timesteps 1182.
Path 131 | total_timesteps 1193.
Path 132 | total_timesteps 1202.
Path 133 | total_timesteps 1211.
Path 134 | total_timesteps 1220.
Path 135 | total_timesteps 1228.
Path 136 | total_timesteps 1237.
Path 137 | total_timesteps 1248.
Path 138 | total_timesteps 1257.
Path 139 | total_timesteps 1268.
Path 140 | total_timesteps 1274.
Path 141 | total_timesteps 1283.
Path 142 | total_timesteps 1295.
Path 143 | total_timesteps 1301.
Path 144 | total_timesteps 1309.
Path 145 | total_timesteps 1319.
Path 146 | total_timesteps 1327.
Path 147 | total_timesteps 1334.
Path 148 | total_timesteps 1345.
Path 149 | total_timesteps 1352.
Path 150 | total_timesteps 1362.
Path 151 | total_timesteps 1370.
Path 152 | total_timesteps 1378.
Path 153 | total_timesteps 1385.
Path 154 | total_timesteps 1396.
Path 155 | total_timesteps 1407.
Path 156 | total_timesteps 1417.
Path 157 | total_timesteps 1424.
Path 158 | total_timesteps 1433.
Path 159 | total_timesteps 1444.
Path 160 | total_timesteps 1451.
Path 161 | total_timesteps 1458.
Path 162 | total_timesteps 1466.
Path 163 | total_timesteps 1474.
Path 164 | total_timesteps 1483.
Path 165 | total_timesteps 1490.
Path 166 | total_timesteps 1497.
Path 167 | total_timesteps 1505.
Path 168 | total_timesteps 1512.
Path 169 | total_timesteps 1524.
Path 170 | total_timesteps 1536.
Path 171 | total_timesteps 1546.
Path 172 | total_timesteps 1554.
Path 173 | total_timesteps 1562.
Path 174 | total_timesteps 1570.
Path 175 | total_timesteps 1581.
Path 176 | total_timesteps 1591.
Path 177 | total_timesteps 1602.
Path 178 | total_timesteps 1611.
Path 179 | total_timesteps 1618.
Path 180 | total_timesteps 1625.
Path 181 | total_timesteps 1631.
Path 182 | total_timesteps 1641.
Path 183 | total_timesteps 1650.
Path 184 | total_timesteps 1658.
Path 185 | total_timesteps 1665.
Path 186 | total_timesteps 1673.
Path 187 | total_timesteps 1682.
Path 188 | total_timesteps 1693.
Path 189 | total_timesteps 1702.
Path 190 | total_timesteps 1710.
Path 191 | total_timesteps 1719.
Path 192 | total_timesteps 1730.
Path 193 | total_timesteps 1740.
Path 194 | total_timesteps 1748.
Path 195 | total_timesteps 1758.
Path 196 | total_timesteps 1765.
Path 197 | total_timesteps 1776.
Path 198 | total_timesteps 1784.
Path 199 | total_timesteps 1793.
Path 200 | total_timesteps 1802.
Path 201 | total_timesteps 1809.
Path 202 | total_timesteps 1815.
Path 203 | total_timesteps 1827.
Path 204 | total_timesteps 1834.
Path 205 | total_timesteps 1842.
Path 206 | total_timesteps 1849.
Path 207 | total_timesteps 1859.
Path 208 | total_timesteps 1866.
Path 209 | total_timesteps 1875.
Path 210 | total_timesteps 1881.
Path 211 | total_timesteps 1888.
Path 212 | total_timesteps 1898.
Path 213 | total_timesteps 1906.
Path 214 | total_timesteps 1913.
Path 215 | total_timesteps 1922.
Path 216 | total_timesteps 1931.
Path 217 | total_timesteps 1938.
Path 218 | total_timesteps 1947.
Path 219 | total_timesteps 1954.
Path 220 | total_timesteps 1963.
Path 221 | total_timesteps 1972.
Path 222 | total_timesteps 1983.
Path 223 | total_timesteps 1991.
Path 224 | total_timesteps 2002.
Path 225 | total_timesteps 2011.
Path 226 | total_timesteps 2020.
Path 227 | total_timesteps 2034.
Path 228 | total_timesteps 2044.
Path 229 | total_timesteps 2051.
Path 230 | total_timesteps 2059.
Path 231 | total_timesteps 2067.
Path 232 | total_timesteps 2073.
Path 233 | total_timesteps 2084.
Path 234 | total_timesteps 2092.
Path 235 | total_timesteps 2100.
Path 236 | total_timesteps 2107.
Path 237 | total_timesteps 2115.
Path 238 | total_timesteps 2123.
Path 239 | total_timesteps 2130.
Path 240 | total_timesteps 2141.
Path 241 | total_timesteps 2154.
Path 242 | total_timesteps 2166.
Path 243 | total_timesteps 2181.
Path 244 | total_timesteps 2189.
Path 245 | total_timesteps 2200.
Path 246 | total_timesteps 2210.
Path 247 | total_timesteps 2220.
Path 248 | total_timesteps 2229.
Path 249 | total_timesteps 2241.
Path 250 | total_timesteps 2248.
Path 251 | total_timesteps 2258.
Path 252 | total_timesteps 2265.
Path 253 | total_timesteps 2272.
Path 254 | total_timesteps 2286.
Path 255 | total_timesteps 2297.
Path 256 | total_timesteps 2307.
Path 257 | total_timesteps 2316.
Path 258 | total_timesteps 2324.
Path 259 | total_timesteps 2332.
Path 260 | total_timesteps 2341.
Path 261 | total_timesteps 2353.
Path 262 | total_timesteps 2359.
Path 263 | total_timesteps 2368.
Path 264 | total_timesteps 2375.
Path 265 | total_timesteps 2385.
Path 266 | total_timesteps 2394.
Path 267 | total_timesteps 2405.
Path 268 | total_timesteps 2413.
Path 269 | total_timesteps 2423.
Path 270 | total_timesteps 2441.
Path 271 | total_timesteps 2455.
Path 272 | total_timesteps 2463.
Path 273 | total_timesteps 2473.
Path 274 | total_timesteps 2482.
Path 275 | total_timesteps 2490.
Path 276 | total_timesteps 2502.
Path 277 | total_timesteps 2509.
Path 278 | total_timesteps 2516.
Path 279 | total_timesteps 2524.
Path 280 | total_timesteps 2531.
Path 281 | total_timesteps 2543.
Path 282 | total_timesteps 2549.
Path 283 | total_timesteps 2560.
Path 284 | total_timesteps 2566.
Path 285 | total_timesteps 2575.
Path 286 | total_timesteps 2583.
Path 287 | total_timesteps 2590.
Path 288 | total_timesteps 2600.
Path 289 | total_timesteps 2607.
Path 290 | total_timesteps 2614.
Path 291 | total_timesteps 2627.
Path 292 | total_timesteps 2636.
Path 293 | total_timesteps 2645.
Path 294 | total_timesteps 2653.
Path 295 | total_timesteps 2663.
Path 296 | total_timesteps 2677.
Path 297 | total_timesteps 2684.
Path 298 | total_timesteps 2695.
Path 299 | total_timesteps 2708.
Path 300 | total_timesteps 2716.
Path 301 | total_timesteps 2724.
Path 302 | total_timesteps 2730.
Path 303 | total_timesteps 2743.
Path 304 | total_timesteps 2758.
Path 305 | total_timesteps 2770.
Path 306 | total_timesteps 2779.
Path 307 | total_timesteps 2786.
Path 308 | total_timesteps 2795.
Path 309 | total_timesteps 2805.
Path 310 | total_timesteps 2814.
Path 311 | total_timesteps 2826.
Path 312 | total_timesteps 2835.
Path 313 | total_timesteps 2846.
Path 314 | total_timesteps 2857.
Path 315 | total_timesteps 2869.
Path 316 | total_timesteps 2876.
Path 317 | total_timesteps 2883.
Path 318 | total_timesteps 2891.
Path 319 | total_timesteps 2904.
Path 320 | total_timesteps 2912.
Path 321 | total_timesteps 2919.
Path 322 | total_timesteps 2929.
Path 323 | total_timesteps 2943.
Path 324 | total_timesteps 2951.
Path 325 | total_timesteps 2960.
Path 326 | total_timesteps 2969.
Path 327 | total_timesteps 2976.
Path 328 | total_timesteps 2985.
Path 329 | total_timesteps 2994.
Path 330 | total_timesteps 3003.
Path 331 | total_timesteps 3010.
Path 332 | total_timesteps 3018.
Path 333 | total_timesteps 3025.
Path 334 | total_timesteps 3032.
Path 335 | total_timesteps 3041.
Path 336 | total_timesteps 3055.
Path 337 | total_timesteps 3062.
Path 338 | total_timesteps 3071.
Path 339 | total_timesteps 3079.
Path 340 | total_timesteps 3088.
Path 341 | total_timesteps 3102.
Path 342 | total_timesteps 3110.
Path 343 | total_timesteps 3121.
Path 344 | total_timesteps 3129.
Path 345 | total_timesteps 3138.
Path 346 | total_timesteps 3150.
Path 347 | total_timesteps 3159.
Path 348 | total_timesteps 3169.
Path 349 | total_timesteps 3176.
Path 350 | total_timesteps 3186.
Path 351 | total_timesteps 3193.
Path 352 | total_timesteps 3203.
Path 353 | total_timesteps 3216.
Path 354 | total_timesteps 3224.
Path 355 | total_timesteps 3234.
Path 356 | total_timesteps 3244.
Path 357 | total_timesteps 3252.
Path 358 | total_timesteps 3266.
Path 359 | total_timesteps 3277.
Path 360 | total_timesteps 3284.
Path 361 | total_timesteps 3294.
Path 362 | total_timesteps 3304.
Path 363 | total_timesteps 3315.
Path 364 | total_timesteps 3332.
Path 365 | total_timesteps 3338.
Path 366 | total_timesteps 3347.
Path 367 | total_timesteps 3356.
Path 368 | total_timesteps 3363.
Path 369 | total_timesteps 3373.
Path 370 | total_timesteps 3381.
Path 371 | total_timesteps 3395.
Path 372 | total_timesteps 3402.
Path 373 | total_timesteps 3411.
Path 374 | total_timesteps 3420.
Path 375 | total_timesteps 3428.
Path 376 | total_timesteps 3436.
Path 377 | total_timesteps 3443.
Path 378 | total_timesteps 3452.
Path 379 | total_timesteps 3459.
Path 380 | total_timesteps 3470.
Path 381 | total_timesteps 3477.
Path 382 | total_timesteps 3486.
Path 383 | total_timesteps 3498.
Path 384 | total_timesteps 3507.
Path 385 | total_timesteps 3515.
Path 386 | total_timesteps 3522.
Path 387 | total_timesteps 3534.
Path 388 | total_timesteps 3542.
Path 389 | total_timesteps 3550.
Path 390 | total_timesteps 3557.
Path 391 | total_timesteps 3565.
Path 392 | total_timesteps 3572.
Path 393 | total_timesteps 3586.
Path 394 | total_timesteps 3593.
Path 395 | total_timesteps 3600.
Path 396 | total_timesteps 3609.
Path 397 | total_timesteps 3621.
Path 398 | total_timesteps 3628.
Path 399 | total_timesteps 3636.
Path 400 | total_timesteps 3645.
Path 401 | total_timesteps 3652.
Path 402 | total_timesteps 3661.
Path 403 | total_timesteps 3671.
Path 404 | total_timesteps 3678.
Path 405 | total_timesteps 3689.
Path 406 | total_timesteps 3701.
Path 407 | total_timesteps 3708.
Path 408 | total_timesteps 3715.
Path 409 | total_timesteps 3723.
Path 410 | total_timesteps 3731.
Path 411 | total_timesteps 3739.
Path 412 | total_timesteps 3748.
Path 413 | total_timesteps 3755.
Path 414 | total_timesteps 3765.
Path 415 | total_timesteps 3771.
Path 416 | total_timesteps 3779.
Path 417 | total_timesteps 3792.
Path 418 | total_timesteps 3803.
Path 419 | total_timesteps 3813.
Path 420 | total_timesteps 3823.
Path 421 | total_timesteps 3832.
Path 422 | total_timesteps 3839.
Path 423 | total_timesteps 3851.
Path 424 | total_timesteps 3859.
Path 425 | total_timesteps 3872.
Path 426 | total_timesteps 3879.
Path 427 | total_timesteps 3887.
Path 428 | total_timesteps 3894.
Path 429 | total_timesteps 3902.
Path 430 | total_timesteps 3914.
Path 431 | total_timesteps 3920.
Path 432 | total_timesteps 3928.
Path 433 | total_timesteps 3939.
Path 434 | total_timesteps 3948.
Path 435 | total_timesteps 3959.
Path 436 | total_timesteps 3971.
Path 437 | total_timesteps 3981.
Path 438 | total_timesteps 3988.
Path 439 | total_timesteps 3996.
Path 440 | total_timesteps 4007.
Path 441 | total_timesteps 4014.
Path 442 | total_timesteps 4031.
Path 443 | total_timesteps 4038.
Path 444 | total_timesteps 4048.
Path 445 | total_timesteps 4056.
Path 446 | total_timesteps 4063.
Path 447 | total_timesteps 4071.
Path 448 | total_timesteps 4084.
Path 449 | total_timesteps 4094.
Path 450 | total_timesteps 4105.
Path 451 | total_timesteps 4117.
Path 452 | total_timesteps 4124.
Path 453 | total_timesteps 4133.
Path 454 | total_timesteps 4149.
Path 455 | total_timesteps 4157.
Path 456 | total_timesteps 4167.
Path 457 | total_timesteps 4174.
Path 458 | total_timesteps 4180.
Path 459 | total_timesteps 4188.
Path 460 | total_timesteps 4194.
Path 461 | total_timesteps 4202.
Path 462 | total_timesteps 4210.
Path 463 | total_timesteps 4223.
Path 464 | total_timesteps 4230.
Path 465 | total_timesteps 4237.
Path 466 | total_timesteps 4245.
Path 467 | total_timesteps 4258.
Path 468 | total_timesteps 4266.
Path 469 | total_timesteps 4278.
Path 470 | total_timesteps 4286.
Path 471 | total_timesteps 4294.
Path 472 | total_timesteps 4300.
Path 473 | total_timesteps 4307.
Path 474 | total_timesteps 4319.
Path 475 | total_timesteps 4329.
Path 476 | total_timesteps 4341.
Path 477 | total_timesteps 4349.
Path 478 | total_timesteps 4355.
Path 479 | total_timesteps 4362.
Path 480 | total_timesteps 4368.
Path 481 | total_timesteps 4379.
Path 482 | total_timesteps 4386.
Path 483 | total_timesteps 4394.
Path 484 | total_timesteps 4402.
Path 485 | total_timesteps 4408.
Path 486 | total_timesteps 4416.
Path 487 | total_timesteps 4426.
Path 488 | total_timesteps 4434.
Path 489 | total_timesteps 4444.
Path 490 | total_timesteps 4453.
Path 491 | total_timesteps 4463.
Path 492 | total_timesteps 4470.
Path 493 | total_timesteps 4477.
Path 494 | total_timesteps 4488.
Path 495 | total_timesteps 4496.
Path 496 | total_timesteps 4504.
Path 497 | total_timesteps 4516.
Path 498 | total_timesteps 4523.
Path 499 | total_timesteps 4533.
Path 500 | total_timesteps 4548.
Path 501 | total_timesteps 4556.
Path 502 | total_timesteps 4569.
Path 503 | total_timesteps 4576.
Path 504 | total_timesteps 4590.
Path 505 | total_timesteps 4597.
Path 506 | total_timesteps 4605.
Path 507 | total_timesteps 4614.
Path 508 | total_timesteps 4622.
Path 509 | total_timesteps 4631.
Path 510 | total_timesteps 4645.
Path 511 | total_timesteps 4653.
Path 512 | total_timesteps 4666.
Path 513 | total_timesteps 4676.
Path 514 | total_timesteps 4683.
Path 515 | total_timesteps 4695.
Path 516 | total_timesteps 4707.
Path 517 | total_timesteps 4713.
Path 518 | total_timesteps 4720.
Path 519 | total_timesteps 4730.
Path 520 | total_timesteps 4740.
Path 521 | total_timesteps 4747.
Path 522 | total_timesteps 4755.
Path 523 | total_timesteps 4771.
Path 524 | total_timesteps 4782.
Path 525 | total_timesteps 4789.
Path 526 | total_timesteps 4800.
Path 527 | total_timesteps 4808.
Path 528 | total_timesteps 4817.
Path 529 | total_timesteps 4824.
Path 530 | total_timesteps 4833.
Path 531 | total_timesteps 4853.
Path 532 | total_timesteps 4860.
Path 533 | total_timesteps 4874.
Path 534 | total_timesteps 4882.
Path 535 | total_timesteps 4889.
Path 536 | total_timesteps 4896.
Path 537 | total_timesteps 4904.
Path 538 | total_timesteps 4910.
Path 539 | total_timesteps 4921.
Path 540 | total_timesteps 4928.
Path 541 | total_timesteps 4936.
Path 542 | total_timesteps 4947.
Path 543 | total_timesteps 4953.
Path 544 | total_timesteps 4965.
Path 545 | total_timesteps 4972.
Path 546 | total_timesteps 4979.
Path 547 | total_timesteps 4988.
Path 548 | total_timesteps 4995.
Path 549 | total_timesteps 5005.
Path 550 | total_timesteps 5012.
Path 551 | total_timesteps 5021.
Path 552 | total_timesteps 5028.
Path 553 | total_timesteps 5037.
Path 554 | total_timesteps 5045.
Path 555 | total_timesteps 5055.
Path 556 | total_timesteps 5062.
Path 557 | total_timesteps 5070.
Path 558 | total_timesteps 5078.
Path 559 | total_timesteps 5085.
Path 560 | total_timesteps 5094.
Path 561 | total_timesteps 5108.
Path 562 | total_timesteps 5120.
Path 563 | total_timesteps 5127.
Path 564 | total_timesteps 5134.
Path 565 | total_timesteps 5141.
Path 566 | total_timesteps 5152.
Path 567 | total_timesteps 5161.
Path 568 | total_timesteps 5169.
Path 569 | total_timesteps 5178.
Path 570 | total_timesteps 5185.
Path 571 | total_timesteps 5192.
Path 572 | total_timesteps 5200.
Path 573 | total_timesteps 5208.
Path 574 | total_timesteps 5217.
Path 575 | total_timesteps 5225.
Path 576 | total_timesteps 5236.
Path 577 | total_timesteps 5243.
Path 578 | total_timesteps 5252.
Path 579 | total_timesteps 5260.
Path 580 | total_timesteps 5266.
Path 581 | total_timesteps 5275.
Path 582 | total_timesteps 5285.
Path 583 | total_timesteps 5301.
Path 584 | total_timesteps 5308.
Path 585 | total_timesteps 5316.
Path 586 | total_timesteps 5323.
Path 587 | total_timesteps 5331.
Path 588 | total_timesteps 5338.
Path 589 | total_timesteps 5346.
Path 590 | total_timesteps 5353.
Path 591 | total_timesteps 5362.
Path 592 | total_timesteps 5369.
Path 593 | total_timesteps 5382.
Path 594 | total_timesteps 5390.
Path 595 | total_timesteps 5397.
Path 596 | total_timesteps 5407.
Path 597 | total_timesteps 5416.
Path 598 | total_timesteps 5426.
Path 599 | total_timesteps 5434.
Path 600 | total_timesteps 5443.
Path 601 | total_timesteps 5450.
Path 602 | total_timesteps 5457.
Path 603 | total_timesteps 5466.
Path 604 | total_timesteps 5474.
Path 605 | total_timesteps 5487.
Path 606 | total_timesteps 5498.
Path 607 | total_timesteps 5512.
Path 608 | total_timesteps 5525.
Path 609 | total_timesteps 5538.
Path 610 | total_timesteps 5549.
Path 611 | total_timesteps 5561.
Path 612 | total_timesteps 5569.
Path 613 | total_timesteps 5577.
Path 614 | total_timesteps 5583.
Path 615 | total_timesteps 5594.
Path 616 | total_timesteps 5603.
Path 617 | total_timesteps 5613.
Path 618 | total_timesteps 5622.
Path 619 | total_timesteps 5630.
Path 620 | total_timesteps 5643.
Path 621 | total_timesteps 5651.
Path 622 | total_timesteps 5661.
Path 623 | total_timesteps 5676.
Path 624 | total_timesteps 5685.
Path 625 | total_timesteps 5693.
Path 626 | total_timesteps 5703.
Path 627 | total_timesteps 5714.
Path 628 | total_timesteps 5721.
Path 629 | total_timesteps 5728.
Path 630 | total_timesteps 5742.
Path 631 | total_timesteps 5752.
Path 632 | total_timesteps 5760.
Path 633 | total_timesteps 5769.
Path 634 | total_timesteps 5776.
Path 635 | total_timesteps 5783.
Path 636 | total_timesteps 5790.
Path 637 | total_timesteps 5796.
Path 638 | total_timesteps 5804.
Path 639 | total_timesteps 5813.
Path 640 | total_timesteps 5821.
Path 641 | total_timesteps 5828.
Path 642 | total_timesteps 5838.
Path 643 | total_timesteps 5847.
Path 644 | total_timesteps 5855.
Path 645 | total_timesteps 5865.
Path 646 | total_timesteps 5874.
Path 647 | total_timesteps 5883.
Path 648 | total_timesteps 5892.
Path 649 | total_timesteps 5900.
Path 650 | total_timesteps 5907.
Path 651 | total_timesteps 5915.
Path 652 | total_timesteps 5923.
Path 653 | total_timesteps 5934.
Path 654 | total_timesteps 5941.
Path 655 | total_timesteps 5950.
Path 656 | total_timesteps 5961.
Path 657 | total_timesteps 5970.
Path 658 | total_timesteps 5979.
Path 659 | total_timesteps 5987.
Path 660 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.56    |
| Iteration     | 9        |
| MaximumReturn | 0.964    |
| MinimumReturn | -20.3    |
| TotalSamples  | 44044    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.004577652085572481
Validation loss = 0.005061673931777477
Validation loss = 0.005164642818272114
Validation loss = 0.004942939151078463
Validation loss = 0.004826363176107407
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 47.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 64.
Path 8 | total_timesteps 72.
Path 9 | total_timesteps 79.
Path 10 | total_timesteps 86.
Path 11 | total_timesteps 94.
Path 12 | total_timesteps 103.
Path 13 | total_timesteps 111.
Path 14 | total_timesteps 125.
Path 15 | total_timesteps 131.
Path 16 | total_timesteps 139.
Path 17 | total_timesteps 150.
Path 18 | total_timesteps 159.
Path 19 | total_timesteps 170.
Path 20 | total_timesteps 177.
Path 21 | total_timesteps 184.
Path 22 | total_timesteps 192.
Path 23 | total_timesteps 199.
Path 24 | total_timesteps 206.
Path 25 | total_timesteps 212.
Path 26 | total_timesteps 220.
Path 27 | total_timesteps 226.
Path 28 | total_timesteps 233.
Path 29 | total_timesteps 240.
Path 30 | total_timesteps 253.
Path 31 | total_timesteps 261.
Path 32 | total_timesteps 270.
Path 33 | total_timesteps 279.
Path 34 | total_timesteps 286.
Path 35 | total_timesteps 293.
Path 36 | total_timesteps 300.
Path 37 | total_timesteps 307.
Path 38 | total_timesteps 315.
Path 39 | total_timesteps 322.
Path 40 | total_timesteps 329.
Path 41 | total_timesteps 340.
Path 42 | total_timesteps 351.
Path 43 | total_timesteps 360.
Path 44 | total_timesteps 368.
Path 45 | total_timesteps 375.
Path 46 | total_timesteps 381.
Path 47 | total_timesteps 391.
Path 48 | total_timesteps 403.
Path 49 | total_timesteps 415.
Path 50 | total_timesteps 423.
Path 51 | total_timesteps 431.
Path 52 | total_timesteps 443.
Path 53 | total_timesteps 450.
Path 54 | total_timesteps 461.
Path 55 | total_timesteps 473.
Path 56 | total_timesteps 485.
Path 57 | total_timesteps 495.
Path 58 | total_timesteps 502.
Path 59 | total_timesteps 511.
Path 60 | total_timesteps 520.
Path 61 | total_timesteps 530.
Path 62 | total_timesteps 537.
Path 63 | total_timesteps 543.
Path 64 | total_timesteps 553.
Path 65 | total_timesteps 562.
Path 66 | total_timesteps 572.
Path 67 | total_timesteps 580.
Path 68 | total_timesteps 592.
Path 69 | total_timesteps 604.
Path 70 | total_timesteps 622.
Path 71 | total_timesteps 631.
Path 72 | total_timesteps 639.
Path 73 | total_timesteps 647.
Path 74 | total_timesteps 655.
Path 75 | total_timesteps 671.
Path 76 | total_timesteps 679.
Path 77 | total_timesteps 690.
Path 78 | total_timesteps 697.
Path 79 | total_timesteps 704.
Path 80 | total_timesteps 715.
Path 81 | total_timesteps 725.
Path 82 | total_timesteps 734.
Path 83 | total_timesteps 746.
Path 84 | total_timesteps 757.
Path 85 | total_timesteps 767.
Path 86 | total_timesteps 775.
Path 87 | total_timesteps 782.
Path 88 | total_timesteps 790.
Path 89 | total_timesteps 800.
Path 90 | total_timesteps 808.
Path 91 | total_timesteps 815.
Path 92 | total_timesteps 822.
Path 93 | total_timesteps 831.
Path 94 | total_timesteps 841.
Path 95 | total_timesteps 855.
Path 96 | total_timesteps 861.
Path 97 | total_timesteps 872.
Path 98 | total_timesteps 881.
Path 99 | total_timesteps 890.
Path 100 | total_timesteps 905.
Path 101 | total_timesteps 913.
Path 102 | total_timesteps 920.
Path 103 | total_timesteps 931.
Path 104 | total_timesteps 938.
Path 105 | total_timesteps 950.
Path 106 | total_timesteps 959.
Path 107 | total_timesteps 970.
Path 108 | total_timesteps 978.
Path 109 | total_timesteps 984.
Path 110 | total_timesteps 994.
Path 111 | total_timesteps 1004.
Path 112 | total_timesteps 1016.
Path 113 | total_timesteps 1023.
Path 114 | total_timesteps 1034.
Path 115 | total_timesteps 1046.
Path 116 | total_timesteps 1057.
Path 117 | total_timesteps 1071.
Path 118 | total_timesteps 1080.
Path 119 | total_timesteps 1090.
Path 120 | total_timesteps 1100.
Path 121 | total_timesteps 1108.
Path 122 | total_timesteps 1115.
Path 123 | total_timesteps 1124.
Path 124 | total_timesteps 1132.
Path 125 | total_timesteps 1142.
Path 126 | total_timesteps 1152.
Path 127 | total_timesteps 1163.
Path 128 | total_timesteps 1171.
Path 129 | total_timesteps 1178.
Path 130 | total_timesteps 1185.
Path 131 | total_timesteps 1193.
Path 132 | total_timesteps 1200.
Path 133 | total_timesteps 1207.
Path 134 | total_timesteps 1220.
Path 135 | total_timesteps 1227.
Path 136 | total_timesteps 1236.
Path 137 | total_timesteps 1244.
Path 138 | total_timesteps 1256.
Path 139 | total_timesteps 1268.
Path 140 | total_timesteps 1277.
Path 141 | total_timesteps 1286.
Path 142 | total_timesteps 1294.
Path 143 | total_timesteps 1303.
Path 144 | total_timesteps 1311.
Path 145 | total_timesteps 1322.
Path 146 | total_timesteps 1328.
Path 147 | total_timesteps 1335.
Path 148 | total_timesteps 1344.
Path 149 | total_timesteps 1353.
Path 150 | total_timesteps 1363.
Path 151 | total_timesteps 1371.
Path 152 | total_timesteps 1378.
Path 153 | total_timesteps 1387.
Path 154 | total_timesteps 1396.
Path 155 | total_timesteps 1405.
Path 156 | total_timesteps 1414.
Path 157 | total_timesteps 1421.
Path 158 | total_timesteps 1431.
Path 159 | total_timesteps 1438.
Path 160 | total_timesteps 1446.
Path 161 | total_timesteps 1454.
Path 162 | total_timesteps 1467.
Path 163 | total_timesteps 1475.
Path 164 | total_timesteps 1483.
Path 165 | total_timesteps 1492.
Path 166 | total_timesteps 1502.
Path 167 | total_timesteps 1511.
Path 168 | total_timesteps 1519.
Path 169 | total_timesteps 1527.
Path 170 | total_timesteps 1535.
Path 171 | total_timesteps 1543.
Path 172 | total_timesteps 1553.
Path 173 | total_timesteps 1566.
Path 174 | total_timesteps 1575.
Path 175 | total_timesteps 1587.
Path 176 | total_timesteps 1596.
Path 177 | total_timesteps 1605.
Path 178 | total_timesteps 1613.
Path 179 | total_timesteps 1622.
Path 180 | total_timesteps 1628.
Path 181 | total_timesteps 1636.
Path 182 | total_timesteps 1645.
Path 183 | total_timesteps 1654.
Path 184 | total_timesteps 1663.
Path 185 | total_timesteps 1670.
Path 186 | total_timesteps 1683.
Path 187 | total_timesteps 1691.
Path 188 | total_timesteps 1700.
Path 189 | total_timesteps 1708.
Path 190 | total_timesteps 1715.
Path 191 | total_timesteps 1723.
Path 192 | total_timesteps 1730.
Path 193 | total_timesteps 1737.
Path 194 | total_timesteps 1746.
Path 195 | total_timesteps 1758.
Path 196 | total_timesteps 1767.
Path 197 | total_timesteps 1773.
Path 198 | total_timesteps 1786.
Path 199 | total_timesteps 1797.
Path 200 | total_timesteps 1805.
Path 201 | total_timesteps 1818.
Path 202 | total_timesteps 1825.
Path 203 | total_timesteps 1834.
Path 204 | total_timesteps 1840.
Path 205 | total_timesteps 1846.
Path 206 | total_timesteps 1854.
Path 207 | total_timesteps 1861.
Path 208 | total_timesteps 1875.
Path 209 | total_timesteps 1886.
Path 210 | total_timesteps 1899.
Path 211 | total_timesteps 1907.
Path 212 | total_timesteps 1916.
Path 213 | total_timesteps 1923.
Path 214 | total_timesteps 1930.
Path 215 | total_timesteps 1938.
Path 216 | total_timesteps 1958.
Path 217 | total_timesteps 1968.
Path 218 | total_timesteps 1979.
Path 219 | total_timesteps 1991.
Path 220 | total_timesteps 1999.
Path 221 | total_timesteps 2007.
Path 222 | total_timesteps 2018.
Path 223 | total_timesteps 2029.
Path 224 | total_timesteps 2038.
Path 225 | total_timesteps 2053.
Path 226 | total_timesteps 2060.
Path 227 | total_timesteps 2068.
Path 228 | total_timesteps 2077.
Path 229 | total_timesteps 2086.
Path 230 | total_timesteps 2095.
Path 231 | total_timesteps 2102.
Path 232 | total_timesteps 2113.
Path 233 | total_timesteps 2121.
Path 234 | total_timesteps 2130.
Path 235 | total_timesteps 2138.
Path 236 | total_timesteps 2147.
Path 237 | total_timesteps 2154.
Path 238 | total_timesteps 2169.
Path 239 | total_timesteps 2182.
Path 240 | total_timesteps 2195.
Path 241 | total_timesteps 2203.
Path 242 | total_timesteps 2214.
Path 243 | total_timesteps 2222.
Path 244 | total_timesteps 2231.
Path 245 | total_timesteps 2240.
Path 246 | total_timesteps 2254.
Path 247 | total_timesteps 2266.
Path 248 | total_timesteps 2280.
Path 249 | total_timesteps 2289.
Path 250 | total_timesteps 2296.
Path 251 | total_timesteps 2305.
Path 252 | total_timesteps 2314.
Path 253 | total_timesteps 2323.
Path 254 | total_timesteps 2331.
Path 255 | total_timesteps 2341.
Path 256 | total_timesteps 2352.
Path 257 | total_timesteps 2360.
Path 258 | total_timesteps 2368.
Path 259 | total_timesteps 2376.
Path 260 | total_timesteps 2385.
Path 261 | total_timesteps 2398.
Path 262 | total_timesteps 2407.
Path 263 | total_timesteps 2421.
Path 264 | total_timesteps 2428.
Path 265 | total_timesteps 2441.
Path 266 | total_timesteps 2450.
Path 267 | total_timesteps 2457.
Path 268 | total_timesteps 2464.
Path 269 | total_timesteps 2473.
Path 270 | total_timesteps 2480.
Path 271 | total_timesteps 2487.
Path 272 | total_timesteps 2505.
Path 273 | total_timesteps 2513.
Path 274 | total_timesteps 2522.
Path 275 | total_timesteps 2531.
Path 276 | total_timesteps 2541.
Path 277 | total_timesteps 2551.
Path 278 | total_timesteps 2561.
Path 279 | total_timesteps 2569.
Path 280 | total_timesteps 2582.
Path 281 | total_timesteps 2592.
Path 282 | total_timesteps 2602.
Path 283 | total_timesteps 2615.
Path 284 | total_timesteps 2625.
Path 285 | total_timesteps 2634.
Path 286 | total_timesteps 2645.
Path 287 | total_timesteps 2656.
Path 288 | total_timesteps 2666.
Path 289 | total_timesteps 2673.
Path 290 | total_timesteps 2684.
Path 291 | total_timesteps 2694.
Path 292 | total_timesteps 2702.
Path 293 | total_timesteps 2708.
Path 294 | total_timesteps 2717.
Path 295 | total_timesteps 2724.
Path 296 | total_timesteps 2735.
Path 297 | total_timesteps 2742.
Path 298 | total_timesteps 2751.
Path 299 | total_timesteps 2764.
Path 300 | total_timesteps 2773.
Path 301 | total_timesteps 2781.
Path 302 | total_timesteps 2789.
Path 303 | total_timesteps 2813.
Path 304 | total_timesteps 2820.
Path 305 | total_timesteps 2829.
Path 306 | total_timesteps 2839.
Path 307 | total_timesteps 2852.
Path 308 | total_timesteps 2862.
Path 309 | total_timesteps 2872.
Path 310 | total_timesteps 2880.
Path 311 | total_timesteps 2889.
Path 312 | total_timesteps 2899.
Path 313 | total_timesteps 2905.
Path 314 | total_timesteps 2914.
Path 315 | total_timesteps 2921.
Path 316 | total_timesteps 2929.
Path 317 | total_timesteps 2935.
Path 318 | total_timesteps 2947.
Path 319 | total_timesteps 2960.
Path 320 | total_timesteps 2970.
Path 321 | total_timesteps 2977.
Path 322 | total_timesteps 2984.
Path 323 | total_timesteps 2996.
Path 324 | total_timesteps 3009.
Path 325 | total_timesteps 3017.
Path 326 | total_timesteps 3028.
Path 327 | total_timesteps 3037.
Path 328 | total_timesteps 3048.
Path 329 | total_timesteps 3059.
Path 330 | total_timesteps 3068.
Path 331 | total_timesteps 3076.
Path 332 | total_timesteps 3085.
Path 333 | total_timesteps 3092.
Path 334 | total_timesteps 3099.
Path 335 | total_timesteps 3106.
Path 336 | total_timesteps 3113.
Path 337 | total_timesteps 3131.
Path 338 | total_timesteps 3138.
Path 339 | total_timesteps 3147.
Path 340 | total_timesteps 3153.
Path 341 | total_timesteps 3161.
Path 342 | total_timesteps 3169.
Path 343 | total_timesteps 3177.
Path 344 | total_timesteps 3191.
Path 345 | total_timesteps 3203.
Path 346 | total_timesteps 3213.
Path 347 | total_timesteps 3222.
Path 348 | total_timesteps 3231.
Path 349 | total_timesteps 3239.
Path 350 | total_timesteps 3249.
Path 351 | total_timesteps 3262.
Path 352 | total_timesteps 3273.
Path 353 | total_timesteps 3286.
Path 354 | total_timesteps 3295.
Path 355 | total_timesteps 3303.
Path 356 | total_timesteps 3311.
Path 357 | total_timesteps 3319.
Path 358 | total_timesteps 3328.
Path 359 | total_timesteps 3337.
Path 360 | total_timesteps 3347.
Path 361 | total_timesteps 3356.
Path 362 | total_timesteps 3364.
Path 363 | total_timesteps 3375.
Path 364 | total_timesteps 3383.
Path 365 | total_timesteps 3390.
Path 366 | total_timesteps 3398.
Path 367 | total_timesteps 3404.
Path 368 | total_timesteps 3414.
Path 369 | total_timesteps 3420.
Path 370 | total_timesteps 3429.
Path 371 | total_timesteps 3437.
Path 372 | total_timesteps 3446.
Path 373 | total_timesteps 3459.
Path 374 | total_timesteps 3468.
Path 375 | total_timesteps 3478.
Path 376 | total_timesteps 3492.
Path 377 | total_timesteps 3501.
Path 378 | total_timesteps 3509.
Path 379 | total_timesteps 3516.
Path 380 | total_timesteps 3525.
Path 381 | total_timesteps 3537.
Path 382 | total_timesteps 3545.
Path 383 | total_timesteps 3553.
Path 384 | total_timesteps 3560.
Path 385 | total_timesteps 3566.
Path 386 | total_timesteps 3577.
Path 387 | total_timesteps 3586.
Path 388 | total_timesteps 3596.
Path 389 | total_timesteps 3614.
Path 390 | total_timesteps 3623.
Path 391 | total_timesteps 3632.
Path 392 | total_timesteps 3640.
Path 393 | total_timesteps 3655.
Path 394 | total_timesteps 3666.
Path 395 | total_timesteps 3677.
Path 396 | total_timesteps 3689.
Path 397 | total_timesteps 3697.
Path 398 | total_timesteps 3706.
Path 399 | total_timesteps 3715.
Path 400 | total_timesteps 3722.
Path 401 | total_timesteps 3729.
Path 402 | total_timesteps 3737.
Path 403 | total_timesteps 3746.
Path 404 | total_timesteps 3752.
Path 405 | total_timesteps 3762.
Path 406 | total_timesteps 3769.
Path 407 | total_timesteps 3776.
Path 408 | total_timesteps 3790.
Path 409 | total_timesteps 3797.
Path 410 | total_timesteps 3805.
Path 411 | total_timesteps 3821.
Path 412 | total_timesteps 3832.
Path 413 | total_timesteps 3840.
Path 414 | total_timesteps 3846.
Path 415 | total_timesteps 3856.
Path 416 | total_timesteps 3863.
Path 417 | total_timesteps 3870.
Path 418 | total_timesteps 3877.
Path 419 | total_timesteps 3884.
Path 420 | total_timesteps 3892.
Path 421 | total_timesteps 3899.
Path 422 | total_timesteps 3907.
Path 423 | total_timesteps 3917.
Path 424 | total_timesteps 3927.
Path 425 | total_timesteps 3935.
Path 426 | total_timesteps 3946.
Path 427 | total_timesteps 3955.
Path 428 | total_timesteps 3961.
Path 429 | total_timesteps 3970.
Path 430 | total_timesteps 3979.
Path 431 | total_timesteps 3987.
Path 432 | total_timesteps 3995.
Path 433 | total_timesteps 4004.
Path 434 | total_timesteps 4013.
Path 435 | total_timesteps 4024.
Path 436 | total_timesteps 4031.
Path 437 | total_timesteps 4043.
Path 438 | total_timesteps 4053.
Path 439 | total_timesteps 4061.
Path 440 | total_timesteps 4070.
Path 441 | total_timesteps 4082.
Path 442 | total_timesteps 4090.
Path 443 | total_timesteps 4096.
Path 444 | total_timesteps 4106.
Path 445 | total_timesteps 4113.
Path 446 | total_timesteps 4122.
Path 447 | total_timesteps 4129.
Path 448 | total_timesteps 4138.
Path 449 | total_timesteps 4147.
Path 450 | total_timesteps 4155.
Path 451 | total_timesteps 4166.
Path 452 | total_timesteps 4174.
Path 453 | total_timesteps 4182.
Path 454 | total_timesteps 4190.
Path 455 | total_timesteps 4199.
Path 456 | total_timesteps 4213.
Path 457 | total_timesteps 4222.
Path 458 | total_timesteps 4229.
Path 459 | total_timesteps 4238.
Path 460 | total_timesteps 4244.
Path 461 | total_timesteps 4252.
Path 462 | total_timesteps 4262.
Path 463 | total_timesteps 4271.
Path 464 | total_timesteps 4281.
Path 465 | total_timesteps 4294.
Path 466 | total_timesteps 4302.
Path 467 | total_timesteps 4309.
Path 468 | total_timesteps 4318.
Path 469 | total_timesteps 4334.
Path 470 | total_timesteps 4342.
Path 471 | total_timesteps 4349.
Path 472 | total_timesteps 4357.
Path 473 | total_timesteps 4366.
Path 474 | total_timesteps 4376.
Path 475 | total_timesteps 4388.
Path 476 | total_timesteps 4395.
Path 477 | total_timesteps 4405.
Path 478 | total_timesteps 4418.
Path 479 | total_timesteps 4425.
Path 480 | total_timesteps 4436.
Path 481 | total_timesteps 4443.
Path 482 | total_timesteps 4451.
Path 483 | total_timesteps 4460.
Path 484 | total_timesteps 4466.
Path 485 | total_timesteps 4475.
Path 486 | total_timesteps 4483.
Path 487 | total_timesteps 4491.
Path 488 | total_timesteps 4501.
Path 489 | total_timesteps 4512.
Path 490 | total_timesteps 4524.
Path 491 | total_timesteps 4535.
Path 492 | total_timesteps 4543.
Path 493 | total_timesteps 4549.
Path 494 | total_timesteps 4556.
Path 495 | total_timesteps 4567.
Path 496 | total_timesteps 4573.
Path 497 | total_timesteps 4582.
Path 498 | total_timesteps 4594.
Path 499 | total_timesteps 4603.
Path 500 | total_timesteps 4610.
Path 501 | total_timesteps 4623.
Path 502 | total_timesteps 4634.
Path 503 | total_timesteps 4641.
Path 504 | total_timesteps 4648.
Path 505 | total_timesteps 4655.
Path 506 | total_timesteps 4661.
Path 507 | total_timesteps 4668.
Path 508 | total_timesteps 4679.
Path 509 | total_timesteps 4685.
Path 510 | total_timesteps 4693.
Path 511 | total_timesteps 4702.
Path 512 | total_timesteps 4708.
Path 513 | total_timesteps 4715.
Path 514 | total_timesteps 4722.
Path 515 | total_timesteps 4732.
Path 516 | total_timesteps 4739.
Path 517 | total_timesteps 4748.
Path 518 | total_timesteps 4756.
Path 519 | total_timesteps 4770.
Path 520 | total_timesteps 4782.
Path 521 | total_timesteps 4788.
Path 522 | total_timesteps 4799.
Path 523 | total_timesteps 4809.
Path 524 | total_timesteps 4817.
Path 525 | total_timesteps 4825.
Path 526 | total_timesteps 4831.
Path 527 | total_timesteps 4843.
Path 528 | total_timesteps 4851.
Path 529 | total_timesteps 4858.
Path 530 | total_timesteps 4870.
Path 531 | total_timesteps 4878.
Path 532 | total_timesteps 4887.
Path 533 | total_timesteps 4895.
Path 534 | total_timesteps 4902.
Path 535 | total_timesteps 4911.
Path 536 | total_timesteps 4920.
Path 537 | total_timesteps 4932.
Path 538 | total_timesteps 4942.
Path 539 | total_timesteps 4952.
Path 540 | total_timesteps 4963.
Path 541 | total_timesteps 4974.
Path 542 | total_timesteps 4988.
Path 543 | total_timesteps 4995.
Path 544 | total_timesteps 5010.
Path 545 | total_timesteps 5018.
Path 546 | total_timesteps 5028.
Path 547 | total_timesteps 5035.
Path 548 | total_timesteps 5049.
Path 549 | total_timesteps 5058.
Path 550 | total_timesteps 5066.
Path 551 | total_timesteps 5074.
Path 552 | total_timesteps 5082.
Path 553 | total_timesteps 5090.
Path 554 | total_timesteps 5096.
Path 555 | total_timesteps 5104.
Path 556 | total_timesteps 5118.
Path 557 | total_timesteps 5130.
Path 558 | total_timesteps 5146.
Path 559 | total_timesteps 5153.
Path 560 | total_timesteps 5162.
Path 561 | total_timesteps 5169.
Path 562 | total_timesteps 5180.
Path 563 | total_timesteps 5188.
Path 564 | total_timesteps 5204.
Path 565 | total_timesteps 5212.
Path 566 | total_timesteps 5221.
Path 567 | total_timesteps 5228.
Path 568 | total_timesteps 5236.
Path 569 | total_timesteps 5243.
Path 570 | total_timesteps 5252.
Path 571 | total_timesteps 5261.
Path 572 | total_timesteps 5270.
Path 573 | total_timesteps 5291.
Path 574 | total_timesteps 5300.
Path 575 | total_timesteps 5309.
Path 576 | total_timesteps 5320.
Path 577 | total_timesteps 5335.
Path 578 | total_timesteps 5343.
Path 579 | total_timesteps 5350.
Path 580 | total_timesteps 5358.
Path 581 | total_timesteps 5365.
Path 582 | total_timesteps 5381.
Path 583 | total_timesteps 5391.
Path 584 | total_timesteps 5406.
Path 585 | total_timesteps 5417.
Path 586 | total_timesteps 5423.
Path 587 | total_timesteps 5433.
Path 588 | total_timesteps 5442.
Path 589 | total_timesteps 5449.
Path 590 | total_timesteps 5458.
Path 591 | total_timesteps 5470.
Path 592 | total_timesteps 5477.
Path 593 | total_timesteps 5486.
Path 594 | total_timesteps 5500.
Path 595 | total_timesteps 5506.
Path 596 | total_timesteps 5516.
Path 597 | total_timesteps 5524.
Path 598 | total_timesteps 5530.
Path 599 | total_timesteps 5536.
Path 600 | total_timesteps 5545.
Path 601 | total_timesteps 5557.
Path 602 | total_timesteps 5565.
Path 603 | total_timesteps 5571.
Path 604 | total_timesteps 5578.
Path 605 | total_timesteps 5587.
Path 606 | total_timesteps 5594.
Path 607 | total_timesteps 5603.
Path 608 | total_timesteps 5611.
Path 609 | total_timesteps 5618.
Path 610 | total_timesteps 5626.
Path 611 | total_timesteps 5634.
Path 612 | total_timesteps 5646.
Path 613 | total_timesteps 5658.
Path 614 | total_timesteps 5665.
Path 615 | total_timesteps 5676.
Path 616 | total_timesteps 5686.
Path 617 | total_timesteps 5693.
Path 618 | total_timesteps 5705.
Path 619 | total_timesteps 5721.
Path 620 | total_timesteps 5734.
Path 621 | total_timesteps 5740.
Path 622 | total_timesteps 5747.
Path 623 | total_timesteps 5754.
Path 624 | total_timesteps 5762.
Path 625 | total_timesteps 5770.
Path 626 | total_timesteps 5778.
Path 627 | total_timesteps 5787.
Path 628 | total_timesteps 5793.
Path 629 | total_timesteps 5801.
Path 630 | total_timesteps 5813.
Path 631 | total_timesteps 5821.
Path 632 | total_timesteps 5828.
Path 633 | total_timesteps 5834.
Path 634 | total_timesteps 5843.
Path 635 | total_timesteps 5864.
Path 636 | total_timesteps 5876.
Path 637 | total_timesteps 5882.
Path 638 | total_timesteps 5894.
Path 639 | total_timesteps 5904.
Path 640 | total_timesteps 5915.
Path 641 | total_timesteps 5924.
Path 642 | total_timesteps 5931.
Path 643 | total_timesteps 5942.
Path 644 | total_timesteps 5948.
Path 645 | total_timesteps 5958.
Path 646 | total_timesteps 5966.
Path 647 | total_timesteps 5973.
Path 648 | total_timesteps 5981.
Path 649 | total_timesteps 5990.
Path 650 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.8     |
| Iteration     | 10       |
| MaximumReturn | -1.3     |
| MinimumReturn | -20.3    |
| TotalSamples  | 48048    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.0044106487184762955
Validation loss = 0.0043197316117584705
Validation loss = 0.00443462235853076
Validation loss = 0.0043573323637247086
Validation loss = 0.004520692862570286
Validation loss = 0.00469041895121336
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 41.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 61.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 78.
Path 10 | total_timesteps 86.
Path 11 | total_timesteps 93.
Path 12 | total_timesteps 102.
Path 13 | total_timesteps 112.
Path 14 | total_timesteps 124.
Path 15 | total_timesteps 132.
Path 16 | total_timesteps 142.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 161.
Path 19 | total_timesteps 170.
Path 20 | total_timesteps 176.
Path 21 | total_timesteps 191.
Path 22 | total_timesteps 201.
Path 23 | total_timesteps 210.
Path 24 | total_timesteps 217.
Path 25 | total_timesteps 226.
Path 26 | total_timesteps 234.
Path 27 | total_timesteps 245.
Path 28 | total_timesteps 252.
Path 29 | total_timesteps 260.
Path 30 | total_timesteps 270.
Path 31 | total_timesteps 280.
Path 32 | total_timesteps 288.
Path 33 | total_timesteps 296.
Path 34 | total_timesteps 303.
Path 35 | total_timesteps 310.
Path 36 | total_timesteps 318.
Path 37 | total_timesteps 329.
Path 38 | total_timesteps 341.
Path 39 | total_timesteps 348.
Path 40 | total_timesteps 359.
Path 41 | total_timesteps 368.
Path 42 | total_timesteps 376.
Path 43 | total_timesteps 383.
Path 44 | total_timesteps 393.
Path 45 | total_timesteps 404.
Path 46 | total_timesteps 414.
Path 47 | total_timesteps 421.
Path 48 | total_timesteps 427.
Path 49 | total_timesteps 434.
Path 50 | total_timesteps 451.
Path 51 | total_timesteps 457.
Path 52 | total_timesteps 463.
Path 53 | total_timesteps 470.
Path 54 | total_timesteps 476.
Path 55 | total_timesteps 483.
Path 56 | total_timesteps 491.
Path 57 | total_timesteps 500.
Path 58 | total_timesteps 510.
Path 59 | total_timesteps 518.
Path 60 | total_timesteps 528.
Path 61 | total_timesteps 534.
Path 62 | total_timesteps 548.
Path 63 | total_timesteps 557.
Path 64 | total_timesteps 565.
Path 65 | total_timesteps 573.
Path 66 | total_timesteps 580.
Path 67 | total_timesteps 590.
Path 68 | total_timesteps 598.
Path 69 | total_timesteps 605.
Path 70 | total_timesteps 613.
Path 71 | total_timesteps 620.
Path 72 | total_timesteps 626.
Path 73 | total_timesteps 634.
Path 74 | total_timesteps 640.
Path 75 | total_timesteps 647.
Path 76 | total_timesteps 657.
Path 77 | total_timesteps 665.
Path 78 | total_timesteps 677.
Path 79 | total_timesteps 685.
Path 80 | total_timesteps 699.
Path 81 | total_timesteps 706.
Path 82 | total_timesteps 715.
Path 83 | total_timesteps 727.
Path 84 | total_timesteps 736.
Path 85 | total_timesteps 747.
Path 86 | total_timesteps 758.
Path 87 | total_timesteps 766.
Path 88 | total_timesteps 776.
Path 89 | total_timesteps 783.
Path 90 | total_timesteps 789.
Path 91 | total_timesteps 799.
Path 92 | total_timesteps 810.
Path 93 | total_timesteps 820.
Path 94 | total_timesteps 828.
Path 95 | total_timesteps 840.
Path 96 | total_timesteps 848.
Path 97 | total_timesteps 856.
Path 98 | total_timesteps 865.
Path 99 | total_timesteps 873.
Path 100 | total_timesteps 879.
Path 101 | total_timesteps 885.
Path 102 | total_timesteps 894.
Path 103 | total_timesteps 901.
Path 104 | total_timesteps 907.
Path 105 | total_timesteps 913.
Path 106 | total_timesteps 926.
Path 107 | total_timesteps 935.
Path 108 | total_timesteps 944.
Path 109 | total_timesteps 950.
Path 110 | total_timesteps 957.
Path 111 | total_timesteps 964.
Path 112 | total_timesteps 974.
Path 113 | total_timesteps 983.
Path 114 | total_timesteps 991.
Path 115 | total_timesteps 1000.
Path 116 | total_timesteps 1007.
Path 117 | total_timesteps 1014.
Path 118 | total_timesteps 1021.
Path 119 | total_timesteps 1030.
Path 120 | total_timesteps 1039.
Path 121 | total_timesteps 1048.
Path 122 | total_timesteps 1055.
Path 123 | total_timesteps 1062.
Path 124 | total_timesteps 1070.
Path 125 | total_timesteps 1078.
Path 126 | total_timesteps 1085.
Path 127 | total_timesteps 1092.
Path 128 | total_timesteps 1099.
Path 129 | total_timesteps 1111.
Path 130 | total_timesteps 1119.
Path 131 | total_timesteps 1127.
Path 132 | total_timesteps 1135.
Path 133 | total_timesteps 1144.
Path 134 | total_timesteps 1151.
Path 135 | total_timesteps 1158.
Path 136 | total_timesteps 1165.
Path 137 | total_timesteps 1172.
Path 138 | total_timesteps 1181.
Path 139 | total_timesteps 1188.
Path 140 | total_timesteps 1197.
Path 141 | total_timesteps 1206.
Path 142 | total_timesteps 1212.
Path 143 | total_timesteps 1220.
Path 144 | total_timesteps 1227.
Path 145 | total_timesteps 1237.
Path 146 | total_timesteps 1247.
Path 147 | total_timesteps 1257.
Path 148 | total_timesteps 1266.
Path 149 | total_timesteps 1278.
Path 150 | total_timesteps 1284.
Path 151 | total_timesteps 1294.
Path 152 | total_timesteps 1305.
Path 153 | total_timesteps 1314.
Path 154 | total_timesteps 1321.
Path 155 | total_timesteps 1328.
Path 156 | total_timesteps 1340.
Path 157 | total_timesteps 1347.
Path 158 | total_timesteps 1355.
Path 159 | total_timesteps 1362.
Path 160 | total_timesteps 1370.
Path 161 | total_timesteps 1380.
Path 162 | total_timesteps 1387.
Path 163 | total_timesteps 1395.
Path 164 | total_timesteps 1413.
Path 165 | total_timesteps 1423.
Path 166 | total_timesteps 1432.
Path 167 | total_timesteps 1443.
Path 168 | total_timesteps 1455.
Path 169 | total_timesteps 1462.
Path 170 | total_timesteps 1471.
Path 171 | total_timesteps 1479.
Path 172 | total_timesteps 1486.
Path 173 | total_timesteps 1498.
Path 174 | total_timesteps 1504.
Path 175 | total_timesteps 1512.
Path 176 | total_timesteps 1521.
Path 177 | total_timesteps 1532.
Path 178 | total_timesteps 1540.
Path 179 | total_timesteps 1546.
Path 180 | total_timesteps 1554.
Path 181 | total_timesteps 1563.
Path 182 | total_timesteps 1569.
Path 183 | total_timesteps 1578.
Path 184 | total_timesteps 1587.
Path 185 | total_timesteps 1597.
Path 186 | total_timesteps 1614.
Path 187 | total_timesteps 1622.
Path 188 | total_timesteps 1631.
Path 189 | total_timesteps 1639.
Path 190 | total_timesteps 1648.
Path 191 | total_timesteps 1657.
Path 192 | total_timesteps 1665.
Path 193 | total_timesteps 1673.
Path 194 | total_timesteps 1680.
Path 195 | total_timesteps 1687.
Path 196 | total_timesteps 1698.
Path 197 | total_timesteps 1704.
Path 198 | total_timesteps 1712.
Path 199 | total_timesteps 1719.
Path 200 | total_timesteps 1726.
Path 201 | total_timesteps 1734.
Path 202 | total_timesteps 1743.
Path 203 | total_timesteps 1752.
Path 204 | total_timesteps 1760.
Path 205 | total_timesteps 1768.
Path 206 | total_timesteps 1778.
Path 207 | total_timesteps 1787.
Path 208 | total_timesteps 1795.
Path 209 | total_timesteps 1804.
Path 210 | total_timesteps 1811.
Path 211 | total_timesteps 1822.
Path 212 | total_timesteps 1831.
Path 213 | total_timesteps 1839.
Path 214 | total_timesteps 1848.
Path 215 | total_timesteps 1861.
Path 216 | total_timesteps 1867.
Path 217 | total_timesteps 1876.
Path 218 | total_timesteps 1883.
Path 219 | total_timesteps 1892.
Path 220 | total_timesteps 1899.
Path 221 | total_timesteps 1906.
Path 222 | total_timesteps 1918.
Path 223 | total_timesteps 1925.
Path 224 | total_timesteps 1934.
Path 225 | total_timesteps 1940.
Path 226 | total_timesteps 1947.
Path 227 | total_timesteps 1961.
Path 228 | total_timesteps 1970.
Path 229 | total_timesteps 1979.
Path 230 | total_timesteps 1987.
Path 231 | total_timesteps 1996.
Path 232 | total_timesteps 2004.
Path 233 | total_timesteps 2010.
Path 234 | total_timesteps 2018.
Path 235 | total_timesteps 2025.
Path 236 | total_timesteps 2044.
Path 237 | total_timesteps 2053.
Path 238 | total_timesteps 2060.
Path 239 | total_timesteps 2070.
Path 240 | total_timesteps 2082.
Path 241 | total_timesteps 2090.
Path 242 | total_timesteps 2099.
Path 243 | total_timesteps 2105.
Path 244 | total_timesteps 2115.
Path 245 | total_timesteps 2124.
Path 246 | total_timesteps 2130.
Path 247 | total_timesteps 2147.
Path 248 | total_timesteps 2155.
Path 249 | total_timesteps 2163.
Path 250 | total_timesteps 2174.
Path 251 | total_timesteps 2186.
Path 252 | total_timesteps 2192.
Path 253 | total_timesteps 2202.
Path 254 | total_timesteps 2213.
Path 255 | total_timesteps 2222.
Path 256 | total_timesteps 2229.
Path 257 | total_timesteps 2236.
Path 258 | total_timesteps 2243.
Path 259 | total_timesteps 2253.
Path 260 | total_timesteps 2261.
Path 261 | total_timesteps 2268.
Path 262 | total_timesteps 2278.
Path 263 | total_timesteps 2286.
Path 264 | total_timesteps 2292.
Path 265 | total_timesteps 2304.
Path 266 | total_timesteps 2315.
Path 267 | total_timesteps 2324.
Path 268 | total_timesteps 2333.
Path 269 | total_timesteps 2341.
Path 270 | total_timesteps 2348.
Path 271 | total_timesteps 2354.
Path 272 | total_timesteps 2360.
Path 273 | total_timesteps 2366.
Path 274 | total_timesteps 2376.
Path 275 | total_timesteps 2385.
Path 276 | total_timesteps 2394.
Path 277 | total_timesteps 2401.
Path 278 | total_timesteps 2409.
Path 279 | total_timesteps 2419.
Path 280 | total_timesteps 2426.
Path 281 | total_timesteps 2434.
Path 282 | total_timesteps 2441.
Path 283 | total_timesteps 2447.
Path 284 | total_timesteps 2457.
Path 285 | total_timesteps 2466.
Path 286 | total_timesteps 2474.
Path 287 | total_timesteps 2482.
Path 288 | total_timesteps 2490.
Path 289 | total_timesteps 2496.
Path 290 | total_timesteps 2502.
Path 291 | total_timesteps 2512.
Path 292 | total_timesteps 2520.
Path 293 | total_timesteps 2531.
Path 294 | total_timesteps 2540.
Path 295 | total_timesteps 2549.
Path 296 | total_timesteps 2556.
Path 297 | total_timesteps 2569.
Path 298 | total_timesteps 2577.
Path 299 | total_timesteps 2586.
Path 300 | total_timesteps 2593.
Path 301 | total_timesteps 2604.
Path 302 | total_timesteps 2611.
Path 303 | total_timesteps 2618.
Path 304 | total_timesteps 2626.
Path 305 | total_timesteps 2637.
Path 306 | total_timesteps 2644.
Path 307 | total_timesteps 2650.
Path 308 | total_timesteps 2658.
Path 309 | total_timesteps 2667.
Path 310 | total_timesteps 2677.
Path 311 | total_timesteps 2686.
Path 312 | total_timesteps 2692.
Path 313 | total_timesteps 2701.
Path 314 | total_timesteps 2707.
Path 315 | total_timesteps 2718.
Path 316 | total_timesteps 2729.
Path 317 | total_timesteps 2738.
Path 318 | total_timesteps 2744.
Path 319 | total_timesteps 2751.
Path 320 | total_timesteps 2759.
Path 321 | total_timesteps 2770.
Path 322 | total_timesteps 2779.
Path 323 | total_timesteps 2785.
Path 324 | total_timesteps 2791.
Path 325 | total_timesteps 2800.
Path 326 | total_timesteps 2816.
Path 327 | total_timesteps 2824.
Path 328 | total_timesteps 2836.
Path 329 | total_timesteps 2844.
Path 330 | total_timesteps 2850.
Path 331 | total_timesteps 2859.
Path 332 | total_timesteps 2866.
Path 333 | total_timesteps 2873.
Path 334 | total_timesteps 2882.
Path 335 | total_timesteps 2890.
Path 336 | total_timesteps 2898.
Path 337 | total_timesteps 2905.
Path 338 | total_timesteps 2915.
Path 339 | total_timesteps 2923.
Path 340 | total_timesteps 2930.
Path 341 | total_timesteps 2938.
Path 342 | total_timesteps 2945.
Path 343 | total_timesteps 2952.
Path 344 | total_timesteps 2959.
Path 345 | total_timesteps 2968.
Path 346 | total_timesteps 2977.
Path 347 | total_timesteps 2987.
Path 348 | total_timesteps 2996.
Path 349 | total_timesteps 3005.
Path 350 | total_timesteps 3012.
Path 351 | total_timesteps 3020.
Path 352 | total_timesteps 3027.
Path 353 | total_timesteps 3037.
Path 354 | total_timesteps 3044.
Path 355 | total_timesteps 3053.
Path 356 | total_timesteps 3065.
Path 357 | total_timesteps 3073.
Path 358 | total_timesteps 3081.
Path 359 | total_timesteps 3088.
Path 360 | total_timesteps 3095.
Path 361 | total_timesteps 3109.
Path 362 | total_timesteps 3117.
Path 363 | total_timesteps 3126.
Path 364 | total_timesteps 3136.
Path 365 | total_timesteps 3143.
Path 366 | total_timesteps 3149.
Path 367 | total_timesteps 3156.
Path 368 | total_timesteps 3163.
Path 369 | total_timesteps 3170.
Path 370 | total_timesteps 3177.
Path 371 | total_timesteps 3183.
Path 372 | total_timesteps 3194.
Path 373 | total_timesteps 3203.
Path 374 | total_timesteps 3211.
Path 375 | total_timesteps 3219.
Path 376 | total_timesteps 3227.
Path 377 | total_timesteps 3237.
Path 378 | total_timesteps 3245.
Path 379 | total_timesteps 3252.
Path 380 | total_timesteps 3260.
Path 381 | total_timesteps 3269.
Path 382 | total_timesteps 3276.
Path 383 | total_timesteps 3290.
Path 384 | total_timesteps 3302.
Path 385 | total_timesteps 3308.
Path 386 | total_timesteps 3317.
Path 387 | total_timesteps 3324.
Path 388 | total_timesteps 3330.
Path 389 | total_timesteps 3337.
Path 390 | total_timesteps 3346.
Path 391 | total_timesteps 3362.
Path 392 | total_timesteps 3371.
Path 393 | total_timesteps 3378.
Path 394 | total_timesteps 3385.
Path 395 | total_timesteps 3392.
Path 396 | total_timesteps 3401.
Path 397 | total_timesteps 3409.
Path 398 | total_timesteps 3419.
Path 399 | total_timesteps 3428.
Path 400 | total_timesteps 3435.
Path 401 | total_timesteps 3444.
Path 402 | total_timesteps 3452.
Path 403 | total_timesteps 3462.
Path 404 | total_timesteps 3469.
Path 405 | total_timesteps 3476.
Path 406 | total_timesteps 3482.
Path 407 | total_timesteps 3491.
Path 408 | total_timesteps 3503.
Path 409 | total_timesteps 3512.
Path 410 | total_timesteps 3524.
Path 411 | total_timesteps 3532.
Path 412 | total_timesteps 3539.
Path 413 | total_timesteps 3549.
Path 414 | total_timesteps 3558.
Path 415 | total_timesteps 3568.
Path 416 | total_timesteps 3579.
Path 417 | total_timesteps 3585.
Path 418 | total_timesteps 3597.
Path 419 | total_timesteps 3605.
Path 420 | total_timesteps 3612.
Path 421 | total_timesteps 3621.
Path 422 | total_timesteps 3630.
Path 423 | total_timesteps 3636.
Path 424 | total_timesteps 3643.
Path 425 | total_timesteps 3649.
Path 426 | total_timesteps 3662.
Path 427 | total_timesteps 3671.
Path 428 | total_timesteps 3679.
Path 429 | total_timesteps 3687.
Path 430 | total_timesteps 3694.
Path 431 | total_timesteps 3700.
Path 432 | total_timesteps 3710.
Path 433 | total_timesteps 3719.
Path 434 | total_timesteps 3726.
Path 435 | total_timesteps 3734.
Path 436 | total_timesteps 3743.
Path 437 | total_timesteps 3749.
Path 438 | total_timesteps 3757.
Path 439 | total_timesteps 3765.
Path 440 | total_timesteps 3773.
Path 441 | total_timesteps 3782.
Path 442 | total_timesteps 3793.
Path 443 | total_timesteps 3801.
Path 444 | total_timesteps 3809.
Path 445 | total_timesteps 3816.
Path 446 | total_timesteps 3826.
Path 447 | total_timesteps 3834.
Path 448 | total_timesteps 3844.
Path 449 | total_timesteps 3854.
Path 450 | total_timesteps 3864.
Path 451 | total_timesteps 3873.
Path 452 | total_timesteps 3880.
Path 453 | total_timesteps 3889.
Path 454 | total_timesteps 3899.
Path 455 | total_timesteps 3909.
Path 456 | total_timesteps 3917.
Path 457 | total_timesteps 3926.
Path 458 | total_timesteps 3933.
Path 459 | total_timesteps 3945.
Path 460 | total_timesteps 3955.
Path 461 | total_timesteps 3961.
Path 462 | total_timesteps 3970.
Path 463 | total_timesteps 3980.
Path 464 | total_timesteps 3989.
Path 465 | total_timesteps 3997.
Path 466 | total_timesteps 4004.
Path 467 | total_timesteps 4011.
Path 468 | total_timesteps 4020.
Path 469 | total_timesteps 4030.
Path 470 | total_timesteps 4036.
Path 471 | total_timesteps 4046.
Path 472 | total_timesteps 4053.
Path 473 | total_timesteps 4065.
Path 474 | total_timesteps 4073.
Path 475 | total_timesteps 4080.
Path 476 | total_timesteps 4087.
Path 477 | total_timesteps 4094.
Path 478 | total_timesteps 4107.
Path 479 | total_timesteps 4117.
Path 480 | total_timesteps 4125.
Path 481 | total_timesteps 4135.
Path 482 | total_timesteps 4142.
Path 483 | total_timesteps 4151.
Path 484 | total_timesteps 4158.
Path 485 | total_timesteps 4166.
Path 486 | total_timesteps 4175.
Path 487 | total_timesteps 4185.
Path 488 | total_timesteps 4195.
Path 489 | total_timesteps 4201.
Path 490 | total_timesteps 4209.
Path 491 | total_timesteps 4220.
Path 492 | total_timesteps 4228.
Path 493 | total_timesteps 4235.
Path 494 | total_timesteps 4249.
Path 495 | total_timesteps 4256.
Path 496 | total_timesteps 4264.
Path 497 | total_timesteps 4273.
Path 498 | total_timesteps 4280.
Path 499 | total_timesteps 4287.
Path 500 | total_timesteps 4297.
Path 501 | total_timesteps 4306.
Path 502 | total_timesteps 4316.
Path 503 | total_timesteps 4326.
Path 504 | total_timesteps 4332.
Path 505 | total_timesteps 4340.
Path 506 | total_timesteps 4348.
Path 507 | total_timesteps 4355.
Path 508 | total_timesteps 4364.
Path 509 | total_timesteps 4372.
Path 510 | total_timesteps 4381.
Path 511 | total_timesteps 4392.
Path 512 | total_timesteps 4402.
Path 513 | total_timesteps 4410.
Path 514 | total_timesteps 4421.
Path 515 | total_timesteps 4429.
Path 516 | total_timesteps 4436.
Path 517 | total_timesteps 4444.
Path 518 | total_timesteps 4454.
Path 519 | total_timesteps 4463.
Path 520 | total_timesteps 4472.
Path 521 | total_timesteps 4481.
Path 522 | total_timesteps 4490.
Path 523 | total_timesteps 4499.
Path 524 | total_timesteps 4506.
Path 525 | total_timesteps 4514.
Path 526 | total_timesteps 4526.
Path 527 | total_timesteps 4533.
Path 528 | total_timesteps 4544.
Path 529 | total_timesteps 4551.
Path 530 | total_timesteps 4558.
Path 531 | total_timesteps 4568.
Path 532 | total_timesteps 4576.
Path 533 | total_timesteps 4586.
Path 534 | total_timesteps 4592.
Path 535 | total_timesteps 4601.
Path 536 | total_timesteps 4608.
Path 537 | total_timesteps 4614.
Path 538 | total_timesteps 4620.
Path 539 | total_timesteps 4629.
Path 540 | total_timesteps 4635.
Path 541 | total_timesteps 4642.
Path 542 | total_timesteps 4651.
Path 543 | total_timesteps 4660.
Path 544 | total_timesteps 4667.
Path 545 | total_timesteps 4673.
Path 546 | total_timesteps 4680.
Path 547 | total_timesteps 4688.
Path 548 | total_timesteps 4697.
Path 549 | total_timesteps 4704.
Path 550 | total_timesteps 4710.
Path 551 | total_timesteps 4717.
Path 552 | total_timesteps 4727.
Path 553 | total_timesteps 4734.
Path 554 | total_timesteps 4740.
Path 555 | total_timesteps 4747.
Path 556 | total_timesteps 4755.
Path 557 | total_timesteps 4762.
Path 558 | total_timesteps 4771.
Path 559 | total_timesteps 4779.
Path 560 | total_timesteps 4786.
Path 561 | total_timesteps 4802.
Path 562 | total_timesteps 4808.
Path 563 | total_timesteps 4816.
Path 564 | total_timesteps 4823.
Path 565 | total_timesteps 4830.
Path 566 | total_timesteps 4838.
Path 567 | total_timesteps 4846.
Path 568 | total_timesteps 4856.
Path 569 | total_timesteps 4870.
Path 570 | total_timesteps 4879.
Path 571 | total_timesteps 4886.
Path 572 | total_timesteps 4892.
Path 573 | total_timesteps 4903.
Path 574 | total_timesteps 4911.
Path 575 | total_timesteps 4921.
Path 576 | total_timesteps 4936.
Path 577 | total_timesteps 4946.
Path 578 | total_timesteps 4955.
Path 579 | total_timesteps 4962.
Path 580 | total_timesteps 4974.
Path 581 | total_timesteps 4986.
Path 582 | total_timesteps 4994.
Path 583 | total_timesteps 5003.
Path 584 | total_timesteps 5013.
Path 585 | total_timesteps 5021.
Path 586 | total_timesteps 5029.
Path 587 | total_timesteps 5037.
Path 588 | total_timesteps 5048.
Path 589 | total_timesteps 5057.
Path 590 | total_timesteps 5068.
Path 591 | total_timesteps 5076.
Path 592 | total_timesteps 5085.
Path 593 | total_timesteps 5096.
Path 594 | total_timesteps 5104.
Path 595 | total_timesteps 5113.
Path 596 | total_timesteps 5124.
Path 597 | total_timesteps 5132.
Path 598 | total_timesteps 5139.
Path 599 | total_timesteps 5148.
Path 600 | total_timesteps 5155.
Path 601 | total_timesteps 5163.
Path 602 | total_timesteps 5172.
Path 603 | total_timesteps 5180.
Path 604 | total_timesteps 5187.
Path 605 | total_timesteps 5197.
Path 606 | total_timesteps 5205.
Path 607 | total_timesteps 5213.
Path 608 | total_timesteps 5223.
Path 609 | total_timesteps 5234.
Path 610 | total_timesteps 5246.
Path 611 | total_timesteps 5258.
Path 612 | total_timesteps 5265.
Path 613 | total_timesteps 5276.
Path 614 | total_timesteps 5286.
Path 615 | total_timesteps 5292.
Path 616 | total_timesteps 5299.
Path 617 | total_timesteps 5306.
Path 618 | total_timesteps 5316.
Path 619 | total_timesteps 5327.
Path 620 | total_timesteps 5334.
Path 621 | total_timesteps 5343.
Path 622 | total_timesteps 5351.
Path 623 | total_timesteps 5360.
Path 624 | total_timesteps 5369.
Path 625 | total_timesteps 5378.
Path 626 | total_timesteps 5384.
Path 627 | total_timesteps 5390.
Path 628 | total_timesteps 5400.
Path 629 | total_timesteps 5410.
Path 630 | total_timesteps 5419.
Path 631 | total_timesteps 5427.
Path 632 | total_timesteps 5441.
Path 633 | total_timesteps 5448.
Path 634 | total_timesteps 5456.
Path 635 | total_timesteps 5462.
Path 636 | total_timesteps 5471.
Path 637 | total_timesteps 5478.
Path 638 | total_timesteps 5485.
Path 639 | total_timesteps 5491.
Path 640 | total_timesteps 5498.
Path 641 | total_timesteps 5508.
Path 642 | total_timesteps 5519.
Path 643 | total_timesteps 5526.
Path 644 | total_timesteps 5533.
Path 645 | total_timesteps 5541.
Path 646 | total_timesteps 5547.
Path 647 | total_timesteps 5557.
Path 648 | total_timesteps 5567.
Path 649 | total_timesteps 5573.
Path 650 | total_timesteps 5584.
Path 651 | total_timesteps 5593.
Path 652 | total_timesteps 5600.
Path 653 | total_timesteps 5606.
Path 654 | total_timesteps 5615.
Path 655 | total_timesteps 5623.
Path 656 | total_timesteps 5631.
Path 657 | total_timesteps 5650.
Path 658 | total_timesteps 5658.
Path 659 | total_timesteps 5666.
Path 660 | total_timesteps 5675.
Path 661 | total_timesteps 5683.
Path 662 | total_timesteps 5691.
Path 663 | total_timesteps 5703.
Path 664 | total_timesteps 5710.
Path 665 | total_timesteps 5720.
Path 666 | total_timesteps 5732.
Path 667 | total_timesteps 5739.
Path 668 | total_timesteps 5747.
Path 669 | total_timesteps 5758.
Path 670 | total_timesteps 5764.
Path 671 | total_timesteps 5772.
Path 672 | total_timesteps 5779.
Path 673 | total_timesteps 5793.
Path 674 | total_timesteps 5805.
Path 675 | total_timesteps 5813.
Path 676 | total_timesteps 5832.
Path 677 | total_timesteps 5840.
Path 678 | total_timesteps 5850.
Path 679 | total_timesteps 5857.
Path 680 | total_timesteps 5867.
Path 681 | total_timesteps 5877.
Path 682 | total_timesteps 5890.
Path 683 | total_timesteps 5898.
Path 684 | total_timesteps 5904.
Path 685 | total_timesteps 5915.
Path 686 | total_timesteps 5926.
Path 687 | total_timesteps 5937.
Path 688 | total_timesteps 5944.
Path 689 | total_timesteps 5953.
Path 690 | total_timesteps 5964.
Path 691 | total_timesteps 5972.
Path 692 | total_timesteps 5982.
Path 693 | total_timesteps 5990.
Path 694 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.43    |
| Iteration     | 11       |
| MaximumReturn | 1.37     |
| MinimumReturn | -15.6    |
| TotalSamples  | 52054    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.004340187646448612
Validation loss = 0.004290202632546425
Validation loss = 0.003747023642063141
Validation loss = 0.004765971563756466
Validation loss = 0.003871964756399393
Validation loss = 0.003912203945219517
Validation loss = 0.0038895888719707727
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 45.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 99.
Path 11 | total_timesteps 108.
Path 12 | total_timesteps 114.
Path 13 | total_timesteps 122.
Path 14 | total_timesteps 131.
Path 15 | total_timesteps 138.
Path 16 | total_timesteps 145.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 159.
Path 19 | total_timesteps 168.
Path 20 | total_timesteps 176.
Path 21 | total_timesteps 184.
Path 22 | total_timesteps 191.
Path 23 | total_timesteps 199.
Path 24 | total_timesteps 211.
Path 25 | total_timesteps 220.
Path 26 | total_timesteps 227.
Path 27 | total_timesteps 237.
Path 28 | total_timesteps 243.
Path 29 | total_timesteps 252.
Path 30 | total_timesteps 259.
Path 31 | total_timesteps 271.
Path 32 | total_timesteps 279.
Path 33 | total_timesteps 286.
Path 34 | total_timesteps 294.
Path 35 | total_timesteps 304.
Path 36 | total_timesteps 314.
Path 37 | total_timesteps 323.
Path 38 | total_timesteps 329.
Path 39 | total_timesteps 338.
Path 40 | total_timesteps 346.
Path 41 | total_timesteps 352.
Path 42 | total_timesteps 360.
Path 43 | total_timesteps 368.
Path 44 | total_timesteps 376.
Path 45 | total_timesteps 383.
Path 46 | total_timesteps 392.
Path 47 | total_timesteps 401.
Path 48 | total_timesteps 408.
Path 49 | total_timesteps 416.
Path 50 | total_timesteps 425.
Path 51 | total_timesteps 431.
Path 52 | total_timesteps 443.
Path 53 | total_timesteps 451.
Path 54 | total_timesteps 459.
Path 55 | total_timesteps 467.
Path 56 | total_timesteps 476.
Path 57 | total_timesteps 485.
Path 58 | total_timesteps 495.
Path 59 | total_timesteps 503.
Path 60 | total_timesteps 510.
Path 61 | total_timesteps 520.
Path 62 | total_timesteps 526.
Path 63 | total_timesteps 535.
Path 64 | total_timesteps 545.
Path 65 | total_timesteps 553.
Path 66 | total_timesteps 560.
Path 67 | total_timesteps 570.
Path 68 | total_timesteps 580.
Path 69 | total_timesteps 586.
Path 70 | total_timesteps 592.
Path 71 | total_timesteps 600.
Path 72 | total_timesteps 607.
Path 73 | total_timesteps 614.
Path 74 | total_timesteps 622.
Path 75 | total_timesteps 629.
Path 76 | total_timesteps 637.
Path 77 | total_timesteps 646.
Path 78 | total_timesteps 653.
Path 79 | total_timesteps 661.
Path 80 | total_timesteps 668.
Path 81 | total_timesteps 675.
Path 82 | total_timesteps 683.
Path 83 | total_timesteps 693.
Path 84 | total_timesteps 699.
Path 85 | total_timesteps 707.
Path 86 | total_timesteps 715.
Path 87 | total_timesteps 728.
Path 88 | total_timesteps 736.
Path 89 | total_timesteps 743.
Path 90 | total_timesteps 754.
Path 91 | total_timesteps 763.
Path 92 | total_timesteps 772.
Path 93 | total_timesteps 781.
Path 94 | total_timesteps 789.
Path 95 | total_timesteps 797.
Path 96 | total_timesteps 804.
Path 97 | total_timesteps 813.
Path 98 | total_timesteps 821.
Path 99 | total_timesteps 828.
Path 100 | total_timesteps 835.
Path 101 | total_timesteps 847.
Path 102 | total_timesteps 855.
Path 103 | total_timesteps 862.
Path 104 | total_timesteps 871.
Path 105 | total_timesteps 878.
Path 106 | total_timesteps 893.
Path 107 | total_timesteps 900.
Path 108 | total_timesteps 906.
Path 109 | total_timesteps 915.
Path 110 | total_timesteps 922.
Path 111 | total_timesteps 929.
Path 112 | total_timesteps 940.
Path 113 | total_timesteps 948.
Path 114 | total_timesteps 954.
Path 115 | total_timesteps 960.
Path 116 | total_timesteps 967.
Path 117 | total_timesteps 973.
Path 118 | total_timesteps 984.
Path 119 | total_timesteps 996.
Path 120 | total_timesteps 1004.
Path 121 | total_timesteps 1014.
Path 122 | total_timesteps 1021.
Path 123 | total_timesteps 1030.
Path 124 | total_timesteps 1040.
Path 125 | total_timesteps 1048.
Path 126 | total_timesteps 1056.
Path 127 | total_timesteps 1064.
Path 128 | total_timesteps 1072.
Path 129 | total_timesteps 1080.
Path 130 | total_timesteps 1087.
Path 131 | total_timesteps 1096.
Path 132 | total_timesteps 1105.
Path 133 | total_timesteps 1113.
Path 134 | total_timesteps 1120.
Path 135 | total_timesteps 1129.
Path 136 | total_timesteps 1136.
Path 137 | total_timesteps 1144.
Path 138 | total_timesteps 1151.
Path 139 | total_timesteps 1158.
Path 140 | total_timesteps 1168.
Path 141 | total_timesteps 1181.
Path 142 | total_timesteps 1187.
Path 143 | total_timesteps 1194.
Path 144 | total_timesteps 1201.
Path 145 | total_timesteps 1212.
Path 146 | total_timesteps 1221.
Path 147 | total_timesteps 1227.
Path 148 | total_timesteps 1238.
Path 149 | total_timesteps 1250.
Path 150 | total_timesteps 1256.
Path 151 | total_timesteps 1265.
Path 152 | total_timesteps 1275.
Path 153 | total_timesteps 1286.
Path 154 | total_timesteps 1294.
Path 155 | total_timesteps 1303.
Path 156 | total_timesteps 1315.
Path 157 | total_timesteps 1324.
Path 158 | total_timesteps 1332.
Path 159 | total_timesteps 1340.
Path 160 | total_timesteps 1349.
Path 161 | total_timesteps 1357.
Path 162 | total_timesteps 1365.
Path 163 | total_timesteps 1373.
Path 164 | total_timesteps 1382.
Path 165 | total_timesteps 1391.
Path 166 | total_timesteps 1399.
Path 167 | total_timesteps 1407.
Path 168 | total_timesteps 1415.
Path 169 | total_timesteps 1422.
Path 170 | total_timesteps 1434.
Path 171 | total_timesteps 1441.
Path 172 | total_timesteps 1450.
Path 173 | total_timesteps 1460.
Path 174 | total_timesteps 1471.
Path 175 | total_timesteps 1478.
Path 176 | total_timesteps 1485.
Path 177 | total_timesteps 1492.
Path 178 | total_timesteps 1502.
Path 179 | total_timesteps 1510.
Path 180 | total_timesteps 1519.
Path 181 | total_timesteps 1527.
Path 182 | total_timesteps 1534.
Path 183 | total_timesteps 1544.
Path 184 | total_timesteps 1551.
Path 185 | total_timesteps 1561.
Path 186 | total_timesteps 1572.
Path 187 | total_timesteps 1578.
Path 188 | total_timesteps 1589.
Path 189 | total_timesteps 1595.
Path 190 | total_timesteps 1605.
Path 191 | total_timesteps 1614.
Path 192 | total_timesteps 1623.
Path 193 | total_timesteps 1630.
Path 194 | total_timesteps 1637.
Path 195 | total_timesteps 1644.
Path 196 | total_timesteps 1652.
Path 197 | total_timesteps 1659.
Path 198 | total_timesteps 1669.
Path 199 | total_timesteps 1677.
Path 200 | total_timesteps 1691.
Path 201 | total_timesteps 1700.
Path 202 | total_timesteps 1710.
Path 203 | total_timesteps 1717.
Path 204 | total_timesteps 1726.
Path 205 | total_timesteps 1735.
Path 206 | total_timesteps 1742.
Path 207 | total_timesteps 1750.
Path 208 | total_timesteps 1758.
Path 209 | total_timesteps 1767.
Path 210 | total_timesteps 1778.
Path 211 | total_timesteps 1785.
Path 212 | total_timesteps 1794.
Path 213 | total_timesteps 1802.
Path 214 | total_timesteps 1813.
Path 215 | total_timesteps 1820.
Path 216 | total_timesteps 1829.
Path 217 | total_timesteps 1835.
Path 218 | total_timesteps 1845.
Path 219 | total_timesteps 1852.
Path 220 | total_timesteps 1863.
Path 221 | total_timesteps 1870.
Path 222 | total_timesteps 1882.
Path 223 | total_timesteps 1890.
Path 224 | total_timesteps 1898.
Path 225 | total_timesteps 1908.
Path 226 | total_timesteps 1918.
Path 227 | total_timesteps 1925.
Path 228 | total_timesteps 1932.
Path 229 | total_timesteps 1938.
Path 230 | total_timesteps 1945.
Path 231 | total_timesteps 1956.
Path 232 | total_timesteps 1964.
Path 233 | total_timesteps 1975.
Path 234 | total_timesteps 1984.
Path 235 | total_timesteps 1992.
Path 236 | total_timesteps 2000.
Path 237 | total_timesteps 2006.
Path 238 | total_timesteps 2017.
Path 239 | total_timesteps 2026.
Path 240 | total_timesteps 2033.
Path 241 | total_timesteps 2040.
Path 242 | total_timesteps 2046.
Path 243 | total_timesteps 2055.
Path 244 | total_timesteps 2062.
Path 245 | total_timesteps 2069.
Path 246 | total_timesteps 2079.
Path 247 | total_timesteps 2088.
Path 248 | total_timesteps 2096.
Path 249 | total_timesteps 2102.
Path 250 | total_timesteps 2111.
Path 251 | total_timesteps 2118.
Path 252 | total_timesteps 2127.
Path 253 | total_timesteps 2134.
Path 254 | total_timesteps 2142.
Path 255 | total_timesteps 2153.
Path 256 | total_timesteps 2163.
Path 257 | total_timesteps 2169.
Path 258 | total_timesteps 2177.
Path 259 | total_timesteps 2189.
Path 260 | total_timesteps 2196.
Path 261 | total_timesteps 2206.
Path 262 | total_timesteps 2213.
Path 263 | total_timesteps 2219.
Path 264 | total_timesteps 2227.
Path 265 | total_timesteps 2236.
Path 266 | total_timesteps 2243.
Path 267 | total_timesteps 2250.
Path 268 | total_timesteps 2260.
Path 269 | total_timesteps 2268.
Path 270 | total_timesteps 2275.
Path 271 | total_timesteps 2283.
Path 272 | total_timesteps 2293.
Path 273 | total_timesteps 2300.
Path 274 | total_timesteps 2307.
Path 275 | total_timesteps 2313.
Path 276 | total_timesteps 2323.
Path 277 | total_timesteps 2334.
Path 278 | total_timesteps 2340.
Path 279 | total_timesteps 2348.
Path 280 | total_timesteps 2355.
Path 281 | total_timesteps 2365.
Path 282 | total_timesteps 2372.
Path 283 | total_timesteps 2385.
Path 284 | total_timesteps 2398.
Path 285 | total_timesteps 2407.
Path 286 | total_timesteps 2416.
Path 287 | total_timesteps 2424.
Path 288 | total_timesteps 2433.
Path 289 | total_timesteps 2441.
Path 290 | total_timesteps 2453.
Path 291 | total_timesteps 2463.
Path 292 | total_timesteps 2473.
Path 293 | total_timesteps 2482.
Path 294 | total_timesteps 2494.
Path 295 | total_timesteps 2501.
Path 296 | total_timesteps 2517.
Path 297 | total_timesteps 2527.
Path 298 | total_timesteps 2535.
Path 299 | total_timesteps 2542.
Path 300 | total_timesteps 2553.
Path 301 | total_timesteps 2559.
Path 302 | total_timesteps 2567.
Path 303 | total_timesteps 2579.
Path 304 | total_timesteps 2585.
Path 305 | total_timesteps 2597.
Path 306 | total_timesteps 2607.
Path 307 | total_timesteps 2614.
Path 308 | total_timesteps 2622.
Path 309 | total_timesteps 2631.
Path 310 | total_timesteps 2638.
Path 311 | total_timesteps 2653.
Path 312 | total_timesteps 2661.
Path 313 | total_timesteps 2671.
Path 314 | total_timesteps 2678.
Path 315 | total_timesteps 2685.
Path 316 | total_timesteps 2692.
Path 317 | total_timesteps 2699.
Path 318 | total_timesteps 2707.
Path 319 | total_timesteps 2714.
Path 320 | total_timesteps 2724.
Path 321 | total_timesteps 2732.
Path 322 | total_timesteps 2738.
Path 323 | total_timesteps 2749.
Path 324 | total_timesteps 2756.
Path 325 | total_timesteps 2763.
Path 326 | total_timesteps 2771.
Path 327 | total_timesteps 2779.
Path 328 | total_timesteps 2786.
Path 329 | total_timesteps 2794.
Path 330 | total_timesteps 2802.
Path 331 | total_timesteps 2808.
Path 332 | total_timesteps 2815.
Path 333 | total_timesteps 2822.
Path 334 | total_timesteps 2830.
Path 335 | total_timesteps 2840.
Path 336 | total_timesteps 2847.
Path 337 | total_timesteps 2858.
Path 338 | total_timesteps 2865.
Path 339 | total_timesteps 2871.
Path 340 | total_timesteps 2878.
Path 341 | total_timesteps 2886.
Path 342 | total_timesteps 2893.
Path 343 | total_timesteps 2905.
Path 344 | total_timesteps 2913.
Path 345 | total_timesteps 2921.
Path 346 | total_timesteps 2929.
Path 347 | total_timesteps 2935.
Path 348 | total_timesteps 2946.
Path 349 | total_timesteps 2954.
Path 350 | total_timesteps 2970.
Path 351 | total_timesteps 2977.
Path 352 | total_timesteps 2985.
Path 353 | total_timesteps 2992.
Path 354 | total_timesteps 3000.
Path 355 | total_timesteps 3007.
Path 356 | total_timesteps 3014.
Path 357 | total_timesteps 3024.
Path 358 | total_timesteps 3031.
Path 359 | total_timesteps 3038.
Path 360 | total_timesteps 3045.
Path 361 | total_timesteps 3055.
Path 362 | total_timesteps 3064.
Path 363 | total_timesteps 3072.
Path 364 | total_timesteps 3080.
Path 365 | total_timesteps 3087.
Path 366 | total_timesteps 3097.
Path 367 | total_timesteps 3105.
Path 368 | total_timesteps 3114.
Path 369 | total_timesteps 3122.
Path 370 | total_timesteps 3130.
Path 371 | total_timesteps 3137.
Path 372 | total_timesteps 3146.
Path 373 | total_timesteps 3153.
Path 374 | total_timesteps 3160.
Path 375 | total_timesteps 3167.
Path 376 | total_timesteps 3174.
Path 377 | total_timesteps 3184.
Path 378 | total_timesteps 3193.
Path 379 | total_timesteps 3200.
Path 380 | total_timesteps 3209.
Path 381 | total_timesteps 3218.
Path 382 | total_timesteps 3227.
Path 383 | total_timesteps 3237.
Path 384 | total_timesteps 3244.
Path 385 | total_timesteps 3254.
Path 386 | total_timesteps 3262.
Path 387 | total_timesteps 3271.
Path 388 | total_timesteps 3278.
Path 389 | total_timesteps 3286.
Path 390 | total_timesteps 3293.
Path 391 | total_timesteps 3300.
Path 392 | total_timesteps 3310.
Path 393 | total_timesteps 3321.
Path 394 | total_timesteps 3330.
Path 395 | total_timesteps 3343.
Path 396 | total_timesteps 3351.
Path 397 | total_timesteps 3359.
Path 398 | total_timesteps 3366.
Path 399 | total_timesteps 3374.
Path 400 | total_timesteps 3381.
Path 401 | total_timesteps 3394.
Path 402 | total_timesteps 3403.
Path 403 | total_timesteps 3411.
Path 404 | total_timesteps 3419.
Path 405 | total_timesteps 3427.
Path 406 | total_timesteps 3434.
Path 407 | total_timesteps 3442.
Path 408 | total_timesteps 3449.
Path 409 | total_timesteps 3459.
Path 410 | total_timesteps 3466.
Path 411 | total_timesteps 3474.
Path 412 | total_timesteps 3483.
Path 413 | total_timesteps 3491.
Path 414 | total_timesteps 3500.
Path 415 | total_timesteps 3510.
Path 416 | total_timesteps 3519.
Path 417 | total_timesteps 3526.
Path 418 | total_timesteps 3536.
Path 419 | total_timesteps 3544.
Path 420 | total_timesteps 3552.
Path 421 | total_timesteps 3561.
Path 422 | total_timesteps 3570.
Path 423 | total_timesteps 3578.
Path 424 | total_timesteps 3584.
Path 425 | total_timesteps 3591.
Path 426 | total_timesteps 3601.
Path 427 | total_timesteps 3608.
Path 428 | total_timesteps 3615.
Path 429 | total_timesteps 3621.
Path 430 | total_timesteps 3628.
Path 431 | total_timesteps 3635.
Path 432 | total_timesteps 3643.
Path 433 | total_timesteps 3649.
Path 434 | total_timesteps 3655.
Path 435 | total_timesteps 3662.
Path 436 | total_timesteps 3670.
Path 437 | total_timesteps 3679.
Path 438 | total_timesteps 3687.
Path 439 | total_timesteps 3695.
Path 440 | total_timesteps 3702.
Path 441 | total_timesteps 3709.
Path 442 | total_timesteps 3718.
Path 443 | total_timesteps 3725.
Path 444 | total_timesteps 3737.
Path 445 | total_timesteps 3745.
Path 446 | total_timesteps 3752.
Path 447 | total_timesteps 3760.
Path 448 | total_timesteps 3769.
Path 449 | total_timesteps 3776.
Path 450 | total_timesteps 3783.
Path 451 | total_timesteps 3790.
Path 452 | total_timesteps 3803.
Path 453 | total_timesteps 3810.
Path 454 | total_timesteps 3816.
Path 455 | total_timesteps 3825.
Path 456 | total_timesteps 3832.
Path 457 | total_timesteps 3840.
Path 458 | total_timesteps 3850.
Path 459 | total_timesteps 3858.
Path 460 | total_timesteps 3864.
Path 461 | total_timesteps 3874.
Path 462 | total_timesteps 3886.
Path 463 | total_timesteps 3896.
Path 464 | total_timesteps 3904.
Path 465 | total_timesteps 3913.
Path 466 | total_timesteps 3921.
Path 467 | total_timesteps 3932.
Path 468 | total_timesteps 3940.
Path 469 | total_timesteps 3950.
Path 470 | total_timesteps 3957.
Path 471 | total_timesteps 3967.
Path 472 | total_timesteps 3974.
Path 473 | total_timesteps 3981.
Path 474 | total_timesteps 3989.
Path 475 | total_timesteps 3997.
Path 476 | total_timesteps 4008.
Path 477 | total_timesteps 4015.
Path 478 | total_timesteps 4025.
Path 479 | total_timesteps 4037.
Path 480 | total_timesteps 4046.
Path 481 | total_timesteps 4053.
Path 482 | total_timesteps 4060.
Path 483 | total_timesteps 4068.
Path 484 | total_timesteps 4077.
Path 485 | total_timesteps 4084.
Path 486 | total_timesteps 4092.
Path 487 | total_timesteps 4102.
Path 488 | total_timesteps 4111.
Path 489 | total_timesteps 4121.
Path 490 | total_timesteps 4128.
Path 491 | total_timesteps 4136.
Path 492 | total_timesteps 4142.
Path 493 | total_timesteps 4150.
Path 494 | total_timesteps 4161.
Path 495 | total_timesteps 4169.
Path 496 | total_timesteps 4177.
Path 497 | total_timesteps 4184.
Path 498 | total_timesteps 4195.
Path 499 | total_timesteps 4206.
Path 500 | total_timesteps 4213.
Path 501 | total_timesteps 4220.
Path 502 | total_timesteps 4232.
Path 503 | total_timesteps 4241.
Path 504 | total_timesteps 4249.
Path 505 | total_timesteps 4256.
Path 506 | total_timesteps 4264.
Path 507 | total_timesteps 4275.
Path 508 | total_timesteps 4284.
Path 509 | total_timesteps 4291.
Path 510 | total_timesteps 4299.
Path 511 | total_timesteps 4308.
Path 512 | total_timesteps 4318.
Path 513 | total_timesteps 4325.
Path 514 | total_timesteps 4335.
Path 515 | total_timesteps 4344.
Path 516 | total_timesteps 4351.
Path 517 | total_timesteps 4360.
Path 518 | total_timesteps 4366.
Path 519 | total_timesteps 4377.
Path 520 | total_timesteps 4384.
Path 521 | total_timesteps 4394.
Path 522 | total_timesteps 4401.
Path 523 | total_timesteps 4410.
Path 524 | total_timesteps 4418.
Path 525 | total_timesteps 4426.
Path 526 | total_timesteps 4434.
Path 527 | total_timesteps 4443.
Path 528 | total_timesteps 4451.
Path 529 | total_timesteps 4459.
Path 530 | total_timesteps 4469.
Path 531 | total_timesteps 4476.
Path 532 | total_timesteps 4485.
Path 533 | total_timesteps 4492.
Path 534 | total_timesteps 4500.
Path 535 | total_timesteps 4507.
Path 536 | total_timesteps 4522.
Path 537 | total_timesteps 4529.
Path 538 | total_timesteps 4536.
Path 539 | total_timesteps 4542.
Path 540 | total_timesteps 4550.
Path 541 | total_timesteps 4559.
Path 542 | total_timesteps 4569.
Path 543 | total_timesteps 4576.
Path 544 | total_timesteps 4583.
Path 545 | total_timesteps 4590.
Path 546 | total_timesteps 4597.
Path 547 | total_timesteps 4605.
Path 548 | total_timesteps 4616.
Path 549 | total_timesteps 4624.
Path 550 | total_timesteps 4630.
Path 551 | total_timesteps 4640.
Path 552 | total_timesteps 4648.
Path 553 | total_timesteps 4657.
Path 554 | total_timesteps 4665.
Path 555 | total_timesteps 4673.
Path 556 | total_timesteps 4681.
Path 557 | total_timesteps 4688.
Path 558 | total_timesteps 4696.
Path 559 | total_timesteps 4707.
Path 560 | total_timesteps 4714.
Path 561 | total_timesteps 4723.
Path 562 | total_timesteps 4734.
Path 563 | total_timesteps 4742.
Path 564 | total_timesteps 4749.
Path 565 | total_timesteps 4757.
Path 566 | total_timesteps 4765.
Path 567 | total_timesteps 4776.
Path 568 | total_timesteps 4784.
Path 569 | total_timesteps 4791.
Path 570 | total_timesteps 4799.
Path 571 | total_timesteps 4806.
Path 572 | total_timesteps 4815.
Path 573 | total_timesteps 4822.
Path 574 | total_timesteps 4836.
Path 575 | total_timesteps 4843.
Path 576 | total_timesteps 4852.
Path 577 | total_timesteps 4860.
Path 578 | total_timesteps 4869.
Path 579 | total_timesteps 4875.
Path 580 | total_timesteps 4882.
Path 581 | total_timesteps 4892.
Path 582 | total_timesteps 4898.
Path 583 | total_timesteps 4907.
Path 584 | total_timesteps 4917.
Path 585 | total_timesteps 4924.
Path 586 | total_timesteps 4933.
Path 587 | total_timesteps 4943.
Path 588 | total_timesteps 4950.
Path 589 | total_timesteps 4958.
Path 590 | total_timesteps 4967.
Path 591 | total_timesteps 4974.
Path 592 | total_timesteps 4981.
Path 593 | total_timesteps 4989.
Path 594 | total_timesteps 4995.
Path 595 | total_timesteps 5004.
Path 596 | total_timesteps 5015.
Path 597 | total_timesteps 5026.
Path 598 | total_timesteps 5032.
Path 599 | total_timesteps 5044.
Path 600 | total_timesteps 5051.
Path 601 | total_timesteps 5058.
Path 602 | total_timesteps 5069.
Path 603 | total_timesteps 5077.
Path 604 | total_timesteps 5085.
Path 605 | total_timesteps 5093.
Path 606 | total_timesteps 5103.
Path 607 | total_timesteps 5111.
Path 608 | total_timesteps 5121.
Path 609 | total_timesteps 5128.
Path 610 | total_timesteps 5140.
Path 611 | total_timesteps 5148.
Path 612 | total_timesteps 5159.
Path 613 | total_timesteps 5166.
Path 614 | total_timesteps 5176.
Path 615 | total_timesteps 5183.
Path 616 | total_timesteps 5190.
Path 617 | total_timesteps 5198.
Path 618 | total_timesteps 5206.
Path 619 | total_timesteps 5214.
Path 620 | total_timesteps 5222.
Path 621 | total_timesteps 5232.
Path 622 | total_timesteps 5238.
Path 623 | total_timesteps 5247.
Path 624 | total_timesteps 5253.
Path 625 | total_timesteps 5259.
Path 626 | total_timesteps 5265.
Path 627 | total_timesteps 5272.
Path 628 | total_timesteps 5280.
Path 629 | total_timesteps 5289.
Path 630 | total_timesteps 5297.
Path 631 | total_timesteps 5303.
Path 632 | total_timesteps 5310.
Path 633 | total_timesteps 5317.
Path 634 | total_timesteps 5324.
Path 635 | total_timesteps 5330.
Path 636 | total_timesteps 5339.
Path 637 | total_timesteps 5346.
Path 638 | total_timesteps 5359.
Path 639 | total_timesteps 5368.
Path 640 | total_timesteps 5375.
Path 641 | total_timesteps 5382.
Path 642 | total_timesteps 5392.
Path 643 | total_timesteps 5403.
Path 644 | total_timesteps 5410.
Path 645 | total_timesteps 5417.
Path 646 | total_timesteps 5427.
Path 647 | total_timesteps 5435.
Path 648 | total_timesteps 5442.
Path 649 | total_timesteps 5453.
Path 650 | total_timesteps 5460.
Path 651 | total_timesteps 5467.
Path 652 | total_timesteps 5475.
Path 653 | total_timesteps 5488.
Path 654 | total_timesteps 5497.
Path 655 | total_timesteps 5504.
Path 656 | total_timesteps 5511.
Path 657 | total_timesteps 5519.
Path 658 | total_timesteps 5532.
Path 659 | total_timesteps 5541.
Path 660 | total_timesteps 5549.
Path 661 | total_timesteps 5559.
Path 662 | total_timesteps 5567.
Path 663 | total_timesteps 5574.
Path 664 | total_timesteps 5586.
Path 665 | total_timesteps 5593.
Path 666 | total_timesteps 5600.
Path 667 | total_timesteps 5606.
Path 668 | total_timesteps 5614.
Path 669 | total_timesteps 5621.
Path 670 | total_timesteps 5629.
Path 671 | total_timesteps 5636.
Path 672 | total_timesteps 5642.
Path 673 | total_timesteps 5650.
Path 674 | total_timesteps 5662.
Path 675 | total_timesteps 5670.
Path 676 | total_timesteps 5677.
Path 677 | total_timesteps 5684.
Path 678 | total_timesteps 5691.
Path 679 | total_timesteps 5700.
Path 680 | total_timesteps 5709.
Path 681 | total_timesteps 5715.
Path 682 | total_timesteps 5722.
Path 683 | total_timesteps 5729.
Path 684 | total_timesteps 5736.
Path 685 | total_timesteps 5743.
Path 686 | total_timesteps 5751.
Path 687 | total_timesteps 5758.
Path 688 | total_timesteps 5764.
Path 689 | total_timesteps 5772.
Path 690 | total_timesteps 5782.
Path 691 | total_timesteps 5790.
Path 692 | total_timesteps 5798.
Path 693 | total_timesteps 5805.
Path 694 | total_timesteps 5811.
Path 695 | total_timesteps 5818.
Path 696 | total_timesteps 5827.
Path 697 | total_timesteps 5836.
Path 698 | total_timesteps 5845.
Path 699 | total_timesteps 5854.
Path 700 | total_timesteps 5861.
Path 701 | total_timesteps 5867.
Path 702 | total_timesteps 5876.
Path 703 | total_timesteps 5884.
Path 704 | total_timesteps 5891.
Path 705 | total_timesteps 5899.
Path 706 | total_timesteps 5906.
Path 707 | total_timesteps 5913.
Path 708 | total_timesteps 5924.
Path 709 | total_timesteps 5932.
Path 710 | total_timesteps 5938.
Path 711 | total_timesteps 5946.
Path 712 | total_timesteps 5955.
Path 713 | total_timesteps 5964.
Path 714 | total_timesteps 5972.
Path 715 | total_timesteps 5979.
Path 716 | total_timesteps 5986.
Path 717 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.34    |
| Iteration     | 12       |
| MaximumReturn | -0.303   |
| MinimumReturn | -14      |
| TotalSamples  | 56057    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.003630150808021426
Validation loss = 0.0036717255134135485
Validation loss = 0.0035373754799365997
Validation loss = 0.0038913313765078783
Validation loss = 0.003314367262646556
Validation loss = 0.0033488960471004248
Validation loss = 0.003576903836801648
Validation loss = 0.003685904433950782
Validation loss = 0.0033644940704107285
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 21.
Path 4 | total_timesteps 27.
Path 5 | total_timesteps 37.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 58.
Path 8 | total_timesteps 65.
Path 9 | total_timesteps 74.
Path 10 | total_timesteps 82.
Path 11 | total_timesteps 91.
Path 12 | total_timesteps 100.
Path 13 | total_timesteps 107.
Path 14 | total_timesteps 116.
Path 15 | total_timesteps 123.
Path 16 | total_timesteps 133.
Path 17 | total_timesteps 146.
Path 18 | total_timesteps 154.
Path 19 | total_timesteps 163.
Path 20 | total_timesteps 172.
Path 21 | total_timesteps 179.
Path 22 | total_timesteps 186.
Path 23 | total_timesteps 197.
Path 24 | total_timesteps 204.
Path 25 | total_timesteps 218.
Path 26 | total_timesteps 227.
Path 27 | total_timesteps 235.
Path 28 | total_timesteps 243.
Path 29 | total_timesteps 253.
Path 30 | total_timesteps 260.
Path 31 | total_timesteps 267.
Path 32 | total_timesteps 274.
Path 33 | total_timesteps 281.
Path 34 | total_timesteps 290.
Path 35 | total_timesteps 298.
Path 36 | total_timesteps 306.
Path 37 | total_timesteps 313.
Path 38 | total_timesteps 319.
Path 39 | total_timesteps 327.
Path 40 | total_timesteps 334.
Path 41 | total_timesteps 344.
Path 42 | total_timesteps 352.
Path 43 | total_timesteps 359.
Path 44 | total_timesteps 368.
Path 45 | total_timesteps 377.
Path 46 | total_timesteps 389.
Path 47 | total_timesteps 396.
Path 48 | total_timesteps 403.
Path 49 | total_timesteps 411.
Path 50 | total_timesteps 420.
Path 51 | total_timesteps 430.
Path 52 | total_timesteps 437.
Path 53 | total_timesteps 445.
Path 54 | total_timesteps 458.
Path 55 | total_timesteps 468.
Path 56 | total_timesteps 476.
Path 57 | total_timesteps 484.
Path 58 | total_timesteps 491.
Path 59 | total_timesteps 498.
Path 60 | total_timesteps 505.
Path 61 | total_timesteps 518.
Path 62 | total_timesteps 527.
Path 63 | total_timesteps 536.
Path 64 | total_timesteps 545.
Path 65 | total_timesteps 553.
Path 66 | total_timesteps 559.
Path 67 | total_timesteps 566.
Path 68 | total_timesteps 575.
Path 69 | total_timesteps 581.
Path 70 | total_timesteps 591.
Path 71 | total_timesteps 598.
Path 72 | total_timesteps 604.
Path 73 | total_timesteps 612.
Path 74 | total_timesteps 619.
Path 75 | total_timesteps 627.
Path 76 | total_timesteps 635.
Path 77 | total_timesteps 643.
Path 78 | total_timesteps 650.
Path 79 | total_timesteps 657.
Path 80 | total_timesteps 663.
Path 81 | total_timesteps 673.
Path 82 | total_timesteps 687.
Path 83 | total_timesteps 695.
Path 84 | total_timesteps 702.
Path 85 | total_timesteps 711.
Path 86 | total_timesteps 718.
Path 87 | total_timesteps 728.
Path 88 | total_timesteps 735.
Path 89 | total_timesteps 746.
Path 90 | total_timesteps 753.
Path 91 | total_timesteps 761.
Path 92 | total_timesteps 771.
Path 93 | total_timesteps 778.
Path 94 | total_timesteps 786.
Path 95 | total_timesteps 797.
Path 96 | total_timesteps 807.
Path 97 | total_timesteps 813.
Path 98 | total_timesteps 823.
Path 99 | total_timesteps 830.
Path 100 | total_timesteps 837.
Path 101 | total_timesteps 849.
Path 102 | total_timesteps 858.
Path 103 | total_timesteps 865.
Path 104 | total_timesteps 871.
Path 105 | total_timesteps 879.
Path 106 | total_timesteps 887.
Path 107 | total_timesteps 895.
Path 108 | total_timesteps 906.
Path 109 | total_timesteps 914.
Path 110 | total_timesteps 925.
Path 111 | total_timesteps 932.
Path 112 | total_timesteps 942.
Path 113 | total_timesteps 950.
Path 114 | total_timesteps 957.
Path 115 | total_timesteps 968.
Path 116 | total_timesteps 975.
Path 117 | total_timesteps 984.
Path 118 | total_timesteps 996.
Path 119 | total_timesteps 1002.
Path 120 | total_timesteps 1012.
Path 121 | total_timesteps 1019.
Path 122 | total_timesteps 1029.
Path 123 | total_timesteps 1039.
Path 124 | total_timesteps 1047.
Path 125 | total_timesteps 1060.
Path 126 | total_timesteps 1067.
Path 127 | total_timesteps 1076.
Path 128 | total_timesteps 1086.
Path 129 | total_timesteps 1098.
Path 130 | total_timesteps 1107.
Path 131 | total_timesteps 1121.
Path 132 | total_timesteps 1128.
Path 133 | total_timesteps 1135.
Path 134 | total_timesteps 1142.
Path 135 | total_timesteps 1151.
Path 136 | total_timesteps 1158.
Path 137 | total_timesteps 1168.
Path 138 | total_timesteps 1176.
Path 139 | total_timesteps 1184.
Path 140 | total_timesteps 1193.
Path 141 | total_timesteps 1203.
Path 142 | total_timesteps 1212.
Path 143 | total_timesteps 1219.
Path 144 | total_timesteps 1226.
Path 145 | total_timesteps 1233.
Path 146 | total_timesteps 1240.
Path 147 | total_timesteps 1249.
Path 148 | total_timesteps 1255.
Path 149 | total_timesteps 1265.
Path 150 | total_timesteps 1272.
Path 151 | total_timesteps 1279.
Path 152 | total_timesteps 1289.
Path 153 | total_timesteps 1297.
Path 154 | total_timesteps 1306.
Path 155 | total_timesteps 1314.
Path 156 | total_timesteps 1327.
Path 157 | total_timesteps 1335.
Path 158 | total_timesteps 1349.
Path 159 | total_timesteps 1363.
Path 160 | total_timesteps 1370.
Path 161 | total_timesteps 1377.
Path 162 | total_timesteps 1390.
Path 163 | total_timesteps 1399.
Path 164 | total_timesteps 1410.
Path 165 | total_timesteps 1420.
Path 166 | total_timesteps 1429.
Path 167 | total_timesteps 1439.
Path 168 | total_timesteps 1446.
Path 169 | total_timesteps 1452.
Path 170 | total_timesteps 1461.
Path 171 | total_timesteps 1468.
Path 172 | total_timesteps 1476.
Path 173 | total_timesteps 1485.
Path 174 | total_timesteps 1493.
Path 175 | total_timesteps 1500.
Path 176 | total_timesteps 1508.
Path 177 | total_timesteps 1520.
Path 178 | total_timesteps 1527.
Path 179 | total_timesteps 1537.
Path 180 | total_timesteps 1547.
Path 181 | total_timesteps 1555.
Path 182 | total_timesteps 1564.
Path 183 | total_timesteps 1573.
Path 184 | total_timesteps 1584.
Path 185 | total_timesteps 1593.
Path 186 | total_timesteps 1600.
Path 187 | total_timesteps 1607.
Path 188 | total_timesteps 1615.
Path 189 | total_timesteps 1622.
Path 190 | total_timesteps 1628.
Path 191 | total_timesteps 1640.
Path 192 | total_timesteps 1647.
Path 193 | total_timesteps 1659.
Path 194 | total_timesteps 1668.
Path 195 | total_timesteps 1675.
Path 196 | total_timesteps 1684.
Path 197 | total_timesteps 1693.
Path 198 | total_timesteps 1702.
Path 199 | total_timesteps 1709.
Path 200 | total_timesteps 1717.
Path 201 | total_timesteps 1727.
Path 202 | total_timesteps 1737.
Path 203 | total_timesteps 1746.
Path 204 | total_timesteps 1753.
Path 205 | total_timesteps 1762.
Path 206 | total_timesteps 1769.
Path 207 | total_timesteps 1777.
Path 208 | total_timesteps 1783.
Path 209 | total_timesteps 1790.
Path 210 | total_timesteps 1797.
Path 211 | total_timesteps 1804.
Path 212 | total_timesteps 1812.
Path 213 | total_timesteps 1820.
Path 214 | total_timesteps 1830.
Path 215 | total_timesteps 1841.
Path 216 | total_timesteps 1847.
Path 217 | total_timesteps 1855.
Path 218 | total_timesteps 1870.
Path 219 | total_timesteps 1876.
Path 220 | total_timesteps 1888.
Path 221 | total_timesteps 1896.
Path 222 | total_timesteps 1904.
Path 223 | total_timesteps 1914.
Path 224 | total_timesteps 1921.
Path 225 | total_timesteps 1930.
Path 226 | total_timesteps 1939.
Path 227 | total_timesteps 1947.
Path 228 | total_timesteps 1958.
Path 229 | total_timesteps 1966.
Path 230 | total_timesteps 1976.
Path 231 | total_timesteps 1984.
Path 232 | total_timesteps 1992.
Path 233 | total_timesteps 1999.
Path 234 | total_timesteps 2006.
Path 235 | total_timesteps 2013.
Path 236 | total_timesteps 2021.
Path 237 | total_timesteps 2027.
Path 238 | total_timesteps 2035.
Path 239 | total_timesteps 2045.
Path 240 | total_timesteps 2052.
Path 241 | total_timesteps 2060.
Path 242 | total_timesteps 2068.
Path 243 | total_timesteps 2075.
Path 244 | total_timesteps 2085.
Path 245 | total_timesteps 2095.
Path 246 | total_timesteps 2103.
Path 247 | total_timesteps 2115.
Path 248 | total_timesteps 2122.
Path 249 | total_timesteps 2135.
Path 250 | total_timesteps 2142.
Path 251 | total_timesteps 2151.
Path 252 | total_timesteps 2159.
Path 253 | total_timesteps 2168.
Path 254 | total_timesteps 2176.
Path 255 | total_timesteps 2188.
Path 256 | total_timesteps 2195.
Path 257 | total_timesteps 2203.
Path 258 | total_timesteps 2211.
Path 259 | total_timesteps 2218.
Path 260 | total_timesteps 2226.
Path 261 | total_timesteps 2232.
Path 262 | total_timesteps 2239.
Path 263 | total_timesteps 2247.
Path 264 | total_timesteps 2255.
Path 265 | total_timesteps 2264.
Path 266 | total_timesteps 2271.
Path 267 | total_timesteps 2279.
Path 268 | total_timesteps 2287.
Path 269 | total_timesteps 2296.
Path 270 | total_timesteps 2305.
Path 271 | total_timesteps 2318.
Path 272 | total_timesteps 2328.
Path 273 | total_timesteps 2335.
Path 274 | total_timesteps 2343.
Path 275 | total_timesteps 2354.
Path 276 | total_timesteps 2361.
Path 277 | total_timesteps 2371.
Path 278 | total_timesteps 2378.
Path 279 | total_timesteps 2387.
Path 280 | total_timesteps 2394.
Path 281 | total_timesteps 2402.
Path 282 | total_timesteps 2408.
Path 283 | total_timesteps 2415.
Path 284 | total_timesteps 2426.
Path 285 | total_timesteps 2435.
Path 286 | total_timesteps 2442.
Path 287 | total_timesteps 2449.
Path 288 | total_timesteps 2457.
Path 289 | total_timesteps 2464.
Path 290 | total_timesteps 2473.
Path 291 | total_timesteps 2480.
Path 292 | total_timesteps 2487.
Path 293 | total_timesteps 2496.
Path 294 | total_timesteps 2503.
Path 295 | total_timesteps 2511.
Path 296 | total_timesteps 2521.
Path 297 | total_timesteps 2532.
Path 298 | total_timesteps 2538.
Path 299 | total_timesteps 2546.
Path 300 | total_timesteps 2553.
Path 301 | total_timesteps 2561.
Path 302 | total_timesteps 2567.
Path 303 | total_timesteps 2573.
Path 304 | total_timesteps 2581.
Path 305 | total_timesteps 2588.
Path 306 | total_timesteps 2594.
Path 307 | total_timesteps 2601.
Path 308 | total_timesteps 2611.
Path 309 | total_timesteps 2620.
Path 310 | total_timesteps 2627.
Path 311 | total_timesteps 2635.
Path 312 | total_timesteps 2641.
Path 313 | total_timesteps 2650.
Path 314 | total_timesteps 2657.
Path 315 | total_timesteps 2666.
Path 316 | total_timesteps 2675.
Path 317 | total_timesteps 2683.
Path 318 | total_timesteps 2691.
Path 319 | total_timesteps 2699.
Path 320 | total_timesteps 2706.
Path 321 | total_timesteps 2719.
Path 322 | total_timesteps 2727.
Path 323 | total_timesteps 2737.
Path 324 | total_timesteps 2743.
Path 325 | total_timesteps 2750.
Path 326 | total_timesteps 2757.
Path 327 | total_timesteps 2764.
Path 328 | total_timesteps 2775.
Path 329 | total_timesteps 2781.
Path 330 | total_timesteps 2788.
Path 331 | total_timesteps 2795.
Path 332 | total_timesteps 2804.
Path 333 | total_timesteps 2811.
Path 334 | total_timesteps 2818.
Path 335 | total_timesteps 2825.
Path 336 | total_timesteps 2832.
Path 337 | total_timesteps 2838.
Path 338 | total_timesteps 2844.
Path 339 | total_timesteps 2851.
Path 340 | total_timesteps 2859.
Path 341 | total_timesteps 2867.
Path 342 | total_timesteps 2874.
Path 343 | total_timesteps 2881.
Path 344 | total_timesteps 2887.
Path 345 | total_timesteps 2893.
Path 346 | total_timesteps 2901.
Path 347 | total_timesteps 2909.
Path 348 | total_timesteps 2916.
Path 349 | total_timesteps 2924.
Path 350 | total_timesteps 2937.
Path 351 | total_timesteps 2943.
Path 352 | total_timesteps 2950.
Path 353 | total_timesteps 2958.
Path 354 | total_timesteps 2966.
Path 355 | total_timesteps 2976.
Path 356 | total_timesteps 2984.
Path 357 | total_timesteps 2993.
Path 358 | total_timesteps 3003.
Path 359 | total_timesteps 3010.
Path 360 | total_timesteps 3020.
Path 361 | total_timesteps 3026.
Path 362 | total_timesteps 3035.
Path 363 | total_timesteps 3042.
Path 364 | total_timesteps 3049.
Path 365 | total_timesteps 3055.
Path 366 | total_timesteps 3063.
Path 367 | total_timesteps 3072.
Path 368 | total_timesteps 3082.
Path 369 | total_timesteps 3094.
Path 370 | total_timesteps 3101.
Path 371 | total_timesteps 3109.
Path 372 | total_timesteps 3116.
Path 373 | total_timesteps 3123.
Path 374 | total_timesteps 3130.
Path 375 | total_timesteps 3142.
Path 376 | total_timesteps 3151.
Path 377 | total_timesteps 3162.
Path 378 | total_timesteps 3171.
Path 379 | total_timesteps 3177.
Path 380 | total_timesteps 3184.
Path 381 | total_timesteps 3194.
Path 382 | total_timesteps 3204.
Path 383 | total_timesteps 3211.
Path 384 | total_timesteps 3219.
Path 385 | total_timesteps 3227.
Path 386 | total_timesteps 3233.
Path 387 | total_timesteps 3242.
Path 388 | total_timesteps 3249.
Path 389 | total_timesteps 3258.
Path 390 | total_timesteps 3267.
Path 391 | total_timesteps 3279.
Path 392 | total_timesteps 3287.
Path 393 | total_timesteps 3294.
Path 394 | total_timesteps 3300.
Path 395 | total_timesteps 3307.
Path 396 | total_timesteps 3314.
Path 397 | total_timesteps 3322.
Path 398 | total_timesteps 3329.
Path 399 | total_timesteps 3336.
Path 400 | total_timesteps 3344.
Path 401 | total_timesteps 3355.
Path 402 | total_timesteps 3362.
Path 403 | total_timesteps 3369.
Path 404 | total_timesteps 3376.
Path 405 | total_timesteps 3384.
Path 406 | total_timesteps 3391.
Path 407 | total_timesteps 3398.
Path 408 | total_timesteps 3405.
Path 409 | total_timesteps 3414.
Path 410 | total_timesteps 3423.
Path 411 | total_timesteps 3434.
Path 412 | total_timesteps 3441.
Path 413 | total_timesteps 3448.
Path 414 | total_timesteps 3457.
Path 415 | total_timesteps 3464.
Path 416 | total_timesteps 3471.
Path 417 | total_timesteps 3482.
Path 418 | total_timesteps 3490.
Path 419 | total_timesteps 3497.
Path 420 | total_timesteps 3508.
Path 421 | total_timesteps 3516.
Path 422 | total_timesteps 3525.
Path 423 | total_timesteps 3534.
Path 424 | total_timesteps 3542.
Path 425 | total_timesteps 3550.
Path 426 | total_timesteps 3556.
Path 427 | total_timesteps 3564.
Path 428 | total_timesteps 3572.
Path 429 | total_timesteps 3579.
Path 430 | total_timesteps 3585.
Path 431 | total_timesteps 3593.
Path 432 | total_timesteps 3600.
Path 433 | total_timesteps 3607.
Path 434 | total_timesteps 3614.
Path 435 | total_timesteps 3624.
Path 436 | total_timesteps 3631.
Path 437 | total_timesteps 3640.
Path 438 | total_timesteps 3648.
Path 439 | total_timesteps 3658.
Path 440 | total_timesteps 3669.
Path 441 | total_timesteps 3679.
Path 442 | total_timesteps 3685.
Path 443 | total_timesteps 3692.
Path 444 | total_timesteps 3701.
Path 445 | total_timesteps 3708.
Path 446 | total_timesteps 3721.
Path 447 | total_timesteps 3733.
Path 448 | total_timesteps 3744.
Path 449 | total_timesteps 3751.
Path 450 | total_timesteps 3760.
Path 451 | total_timesteps 3768.
Path 452 | total_timesteps 3775.
Path 453 | total_timesteps 3784.
Path 454 | total_timesteps 3790.
Path 455 | total_timesteps 3798.
Path 456 | total_timesteps 3807.
Path 457 | total_timesteps 3814.
Path 458 | total_timesteps 3824.
Path 459 | total_timesteps 3839.
Path 460 | total_timesteps 3846.
Path 461 | total_timesteps 3854.
Path 462 | total_timesteps 3862.
Path 463 | total_timesteps 3871.
Path 464 | total_timesteps 3877.
Path 465 | total_timesteps 3886.
Path 466 | total_timesteps 3898.
Path 467 | total_timesteps 3905.
Path 468 | total_timesteps 3913.
Path 469 | total_timesteps 3924.
Path 470 | total_timesteps 3932.
Path 471 | total_timesteps 3938.
Path 472 | total_timesteps 3949.
Path 473 | total_timesteps 3956.
Path 474 | total_timesteps 3963.
Path 475 | total_timesteps 3970.
Path 476 | total_timesteps 3978.
Path 477 | total_timesteps 3985.
Path 478 | total_timesteps 3993.
Path 479 | total_timesteps 4002.
Path 480 | total_timesteps 4010.
Path 481 | total_timesteps 4019.
Path 482 | total_timesteps 4026.
Path 483 | total_timesteps 4033.
Path 484 | total_timesteps 4043.
Path 485 | total_timesteps 4052.
Path 486 | total_timesteps 4061.
Path 487 | total_timesteps 4070.
Path 488 | total_timesteps 4077.
Path 489 | total_timesteps 4084.
Path 490 | total_timesteps 4094.
Path 491 | total_timesteps 4102.
Path 492 | total_timesteps 4110.
Path 493 | total_timesteps 4121.
Path 494 | total_timesteps 4129.
Path 495 | total_timesteps 4141.
Path 496 | total_timesteps 4149.
Path 497 | total_timesteps 4157.
Path 498 | total_timesteps 4165.
Path 499 | total_timesteps 4178.
Path 500 | total_timesteps 4186.
Path 501 | total_timesteps 4196.
Path 502 | total_timesteps 4204.
Path 503 | total_timesteps 4213.
Path 504 | total_timesteps 4220.
Path 505 | total_timesteps 4227.
Path 506 | total_timesteps 4236.
Path 507 | total_timesteps 4244.
Path 508 | total_timesteps 4253.
Path 509 | total_timesteps 4262.
Path 510 | total_timesteps 4270.
Path 511 | total_timesteps 4277.
Path 512 | total_timesteps 4283.
Path 513 | total_timesteps 4289.
Path 514 | total_timesteps 4296.
Path 515 | total_timesteps 4303.
Path 516 | total_timesteps 4313.
Path 517 | total_timesteps 4323.
Path 518 | total_timesteps 4331.
Path 519 | total_timesteps 4338.
Path 520 | total_timesteps 4346.
Path 521 | total_timesteps 4354.
Path 522 | total_timesteps 4361.
Path 523 | total_timesteps 4370.
Path 524 | total_timesteps 4382.
Path 525 | total_timesteps 4391.
Path 526 | total_timesteps 4402.
Path 527 | total_timesteps 4408.
Path 528 | total_timesteps 4414.
Path 529 | total_timesteps 4426.
Path 530 | total_timesteps 4435.
Path 531 | total_timesteps 4442.
Path 532 | total_timesteps 4450.
Path 533 | total_timesteps 4458.
Path 534 | total_timesteps 4466.
Path 535 | total_timesteps 4480.
Path 536 | total_timesteps 4486.
Path 537 | total_timesteps 4494.
Path 538 | total_timesteps 4503.
Path 539 | total_timesteps 4510.
Path 540 | total_timesteps 4517.
Path 541 | total_timesteps 4526.
Path 542 | total_timesteps 4536.
Path 543 | total_timesteps 4543.
Path 544 | total_timesteps 4550.
Path 545 | total_timesteps 4558.
Path 546 | total_timesteps 4566.
Path 547 | total_timesteps 4577.
Path 548 | total_timesteps 4588.
Path 549 | total_timesteps 4595.
Path 550 | total_timesteps 4602.
Path 551 | total_timesteps 4610.
Path 552 | total_timesteps 4617.
Path 553 | total_timesteps 4631.
Path 554 | total_timesteps 4641.
Path 555 | total_timesteps 4651.
Path 556 | total_timesteps 4660.
Path 557 | total_timesteps 4667.
Path 558 | total_timesteps 4673.
Path 559 | total_timesteps 4681.
Path 560 | total_timesteps 4689.
Path 561 | total_timesteps 4702.
Path 562 | total_timesteps 4714.
Path 563 | total_timesteps 4722.
Path 564 | total_timesteps 4736.
Path 565 | total_timesteps 4744.
Path 566 | total_timesteps 4750.
Path 567 | total_timesteps 4758.
Path 568 | total_timesteps 4766.
Path 569 | total_timesteps 4772.
Path 570 | total_timesteps 4779.
Path 571 | total_timesteps 4786.
Path 572 | total_timesteps 4793.
Path 573 | total_timesteps 4800.
Path 574 | total_timesteps 4810.
Path 575 | total_timesteps 4817.
Path 576 | total_timesteps 4825.
Path 577 | total_timesteps 4834.
Path 578 | total_timesteps 4843.
Path 579 | total_timesteps 4852.
Path 580 | total_timesteps 4861.
Path 581 | total_timesteps 4869.
Path 582 | total_timesteps 4878.
Path 583 | total_timesteps 4886.
Path 584 | total_timesteps 4895.
Path 585 | total_timesteps 4905.
Path 586 | total_timesteps 4921.
Path 587 | total_timesteps 4928.
Path 588 | total_timesteps 4935.
Path 589 | total_timesteps 4944.
Path 590 | total_timesteps 4950.
Path 591 | total_timesteps 4957.
Path 592 | total_timesteps 4964.
Path 593 | total_timesteps 4972.
Path 594 | total_timesteps 4983.
Path 595 | total_timesteps 4991.
Path 596 | total_timesteps 4998.
Path 597 | total_timesteps 5004.
Path 598 | total_timesteps 5010.
Path 599 | total_timesteps 5017.
Path 600 | total_timesteps 5025.
Path 601 | total_timesteps 5033.
Path 602 | total_timesteps 5041.
Path 603 | total_timesteps 5049.
Path 604 | total_timesteps 5058.
Path 605 | total_timesteps 5067.
Path 606 | total_timesteps 5076.
Path 607 | total_timesteps 5085.
Path 608 | total_timesteps 5091.
Path 609 | total_timesteps 5098.
Path 610 | total_timesteps 5107.
Path 611 | total_timesteps 5117.
Path 612 | total_timesteps 5125.
Path 613 | total_timesteps 5138.
Path 614 | total_timesteps 5144.
Path 615 | total_timesteps 5150.
Path 616 | total_timesteps 5158.
Path 617 | total_timesteps 5167.
Path 618 | total_timesteps 5174.
Path 619 | total_timesteps 5180.
Path 620 | total_timesteps 5187.
Path 621 | total_timesteps 5196.
Path 622 | total_timesteps 5203.
Path 623 | total_timesteps 5211.
Path 624 | total_timesteps 5218.
Path 625 | total_timesteps 5226.
Path 626 | total_timesteps 5236.
Path 627 | total_timesteps 5246.
Path 628 | total_timesteps 5254.
Path 629 | total_timesteps 5264.
Path 630 | total_timesteps 5274.
Path 631 | total_timesteps 5283.
Path 632 | total_timesteps 5290.
Path 633 | total_timesteps 5296.
Path 634 | total_timesteps 5303.
Path 635 | total_timesteps 5312.
Path 636 | total_timesteps 5318.
Path 637 | total_timesteps 5324.
Path 638 | total_timesteps 5332.
Path 639 | total_timesteps 5340.
Path 640 | total_timesteps 5350.
Path 641 | total_timesteps 5358.
Path 642 | total_timesteps 5365.
Path 643 | total_timesteps 5374.
Path 644 | total_timesteps 5383.
Path 645 | total_timesteps 5390.
Path 646 | total_timesteps 5397.
Path 647 | total_timesteps 5403.
Path 648 | total_timesteps 5414.
Path 649 | total_timesteps 5422.
Path 650 | total_timesteps 5432.
Path 651 | total_timesteps 5442.
Path 652 | total_timesteps 5449.
Path 653 | total_timesteps 5459.
Path 654 | total_timesteps 5467.
Path 655 | total_timesteps 5478.
Path 656 | total_timesteps 5488.
Path 657 | total_timesteps 5495.
Path 658 | total_timesteps 5502.
Path 659 | total_timesteps 5508.
Path 660 | total_timesteps 5514.
Path 661 | total_timesteps 5524.
Path 662 | total_timesteps 5535.
Path 663 | total_timesteps 5544.
Path 664 | total_timesteps 5553.
Path 665 | total_timesteps 5563.
Path 666 | total_timesteps 5574.
Path 667 | total_timesteps 5581.
Path 668 | total_timesteps 5590.
Path 669 | total_timesteps 5597.
Path 670 | total_timesteps 5604.
Path 671 | total_timesteps 5615.
Path 672 | total_timesteps 5622.
Path 673 | total_timesteps 5629.
Path 674 | total_timesteps 5639.
Path 675 | total_timesteps 5647.
Path 676 | total_timesteps 5655.
Path 677 | total_timesteps 5663.
Path 678 | total_timesteps 5671.
Path 679 | total_timesteps 5677.
Path 680 | total_timesteps 5684.
Path 681 | total_timesteps 5695.
Path 682 | total_timesteps 5702.
Path 683 | total_timesteps 5709.
Path 684 | total_timesteps 5715.
Path 685 | total_timesteps 5725.
Path 686 | total_timesteps 5731.
Path 687 | total_timesteps 5737.
Path 688 | total_timesteps 5745.
Path 689 | total_timesteps 5755.
Path 690 | total_timesteps 5762.
Path 691 | total_timesteps 5772.
Path 692 | total_timesteps 5780.
Path 693 | total_timesteps 5786.
Path 694 | total_timesteps 5794.
Path 695 | total_timesteps 5800.
Path 696 | total_timesteps 5809.
Path 697 | total_timesteps 5819.
Path 698 | total_timesteps 5828.
Path 699 | total_timesteps 5837.
Path 700 | total_timesteps 5845.
Path 701 | total_timesteps 5854.
Path 702 | total_timesteps 5863.
Path 703 | total_timesteps 5870.
Path 704 | total_timesteps 5879.
Path 705 | total_timesteps 5888.
Path 706 | total_timesteps 5894.
Path 707 | total_timesteps 5905.
Path 708 | total_timesteps 5916.
Path 709 | total_timesteps 5926.
Path 710 | total_timesteps 5934.
Path 711 | total_timesteps 5948.
Path 712 | total_timesteps 5955.
Path 713 | total_timesteps 5964.
Path 714 | total_timesteps 5974.
Path 715 | total_timesteps 5981.
Path 716 | total_timesteps 5991.
Path 717 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.58    |
| Iteration     | 13       |
| MaximumReturn | -1.2     |
| MinimumReturn | -17.8    |
| TotalSamples  | 60059    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.003857552772387862
Validation loss = 0.0033141546882689
Validation loss = 0.003278113668784499
Validation loss = 0.0043952250853180885
Validation loss = 0.0031860023736953735
Validation loss = 0.003359589958563447
Validation loss = 0.0029687066562473774
Validation loss = 0.003649376332759857
Validation loss = 0.0037515866570174694
Validation loss = 0.0032340448815375566
Validation loss = 0.003201315412297845
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 44.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 61.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 75.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 92.
Path 11 | total_timesteps 99.
Path 12 | total_timesteps 107.
Path 13 | total_timesteps 113.
Path 14 | total_timesteps 120.
Path 15 | total_timesteps 129.
Path 16 | total_timesteps 146.
Path 17 | total_timesteps 153.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 167.
Path 20 | total_timesteps 175.
Path 21 | total_timesteps 184.
Path 22 | total_timesteps 190.
Path 23 | total_timesteps 199.
Path 24 | total_timesteps 209.
Path 25 | total_timesteps 215.
Path 26 | total_timesteps 223.
Path 27 | total_timesteps 230.
Path 28 | total_timesteps 237.
Path 29 | total_timesteps 244.
Path 30 | total_timesteps 255.
Path 31 | total_timesteps 263.
Path 32 | total_timesteps 272.
Path 33 | total_timesteps 281.
Path 34 | total_timesteps 290.
Path 35 | total_timesteps 299.
Path 36 | total_timesteps 312.
Path 37 | total_timesteps 321.
Path 38 | total_timesteps 335.
Path 39 | total_timesteps 343.
Path 40 | total_timesteps 350.
Path 41 | total_timesteps 359.
Path 42 | total_timesteps 368.
Path 43 | total_timesteps 376.
Path 44 | total_timesteps 384.
Path 45 | total_timesteps 391.
Path 46 | total_timesteps 400.
Path 47 | total_timesteps 410.
Path 48 | total_timesteps 417.
Path 49 | total_timesteps 424.
Path 50 | total_timesteps 434.
Path 51 | total_timesteps 442.
Path 52 | total_timesteps 449.
Path 53 | total_timesteps 457.
Path 54 | total_timesteps 463.
Path 55 | total_timesteps 473.
Path 56 | total_timesteps 483.
Path 57 | total_timesteps 490.
Path 58 | total_timesteps 500.
Path 59 | total_timesteps 509.
Path 60 | total_timesteps 520.
Path 61 | total_timesteps 527.
Path 62 | total_timesteps 534.
Path 63 | total_timesteps 548.
Path 64 | total_timesteps 556.
Path 65 | total_timesteps 563.
Path 66 | total_timesteps 573.
Path 67 | total_timesteps 581.
Path 68 | total_timesteps 590.
Path 69 | total_timesteps 598.
Path 70 | total_timesteps 605.
Path 71 | total_timesteps 613.
Path 72 | total_timesteps 619.
Path 73 | total_timesteps 628.
Path 74 | total_timesteps 635.
Path 75 | total_timesteps 642.
Path 76 | total_timesteps 649.
Path 77 | total_timesteps 658.
Path 78 | total_timesteps 666.
Path 79 | total_timesteps 674.
Path 80 | total_timesteps 681.
Path 81 | total_timesteps 688.
Path 82 | total_timesteps 694.
Path 83 | total_timesteps 703.
Path 84 | total_timesteps 712.
Path 85 | total_timesteps 719.
Path 86 | total_timesteps 726.
Path 87 | total_timesteps 738.
Path 88 | total_timesteps 746.
Path 89 | total_timesteps 758.
Path 90 | total_timesteps 765.
Path 91 | total_timesteps 772.
Path 92 | total_timesteps 781.
Path 93 | total_timesteps 789.
Path 94 | total_timesteps 798.
Path 95 | total_timesteps 805.
Path 96 | total_timesteps 816.
Path 97 | total_timesteps 824.
Path 98 | total_timesteps 831.
Path 99 | total_timesteps 840.
Path 100 | total_timesteps 849.
Path 101 | total_timesteps 863.
Path 102 | total_timesteps 869.
Path 103 | total_timesteps 880.
Path 104 | total_timesteps 887.
Path 105 | total_timesteps 895.
Path 106 | total_timesteps 902.
Path 107 | total_timesteps 915.
Path 108 | total_timesteps 925.
Path 109 | total_timesteps 933.
Path 110 | total_timesteps 941.
Path 111 | total_timesteps 949.
Path 112 | total_timesteps 957.
Path 113 | total_timesteps 965.
Path 114 | total_timesteps 973.
Path 115 | total_timesteps 983.
Path 116 | total_timesteps 993.
Path 117 | total_timesteps 1000.
Path 118 | total_timesteps 1007.
Path 119 | total_timesteps 1015.
Path 120 | total_timesteps 1024.
Path 121 | total_timesteps 1034.
Path 122 | total_timesteps 1044.
Path 123 | total_timesteps 1052.
Path 124 | total_timesteps 1061.
Path 125 | total_timesteps 1069.
Path 126 | total_timesteps 1080.
Path 127 | total_timesteps 1090.
Path 128 | total_timesteps 1099.
Path 129 | total_timesteps 1111.
Path 130 | total_timesteps 1119.
Path 131 | total_timesteps 1126.
Path 132 | total_timesteps 1135.
Path 133 | total_timesteps 1144.
Path 134 | total_timesteps 1154.
Path 135 | total_timesteps 1162.
Path 136 | total_timesteps 1174.
Path 137 | total_timesteps 1182.
Path 138 | total_timesteps 1189.
Path 139 | total_timesteps 1196.
Path 140 | total_timesteps 1204.
Path 141 | total_timesteps 1212.
Path 142 | total_timesteps 1220.
Path 143 | total_timesteps 1227.
Path 144 | total_timesteps 1234.
Path 145 | total_timesteps 1240.
Path 146 | total_timesteps 1246.
Path 147 | total_timesteps 1252.
Path 148 | total_timesteps 1261.
Path 149 | total_timesteps 1269.
Path 150 | total_timesteps 1278.
Path 151 | total_timesteps 1285.
Path 152 | total_timesteps 1294.
Path 153 | total_timesteps 1300.
Path 154 | total_timesteps 1307.
Path 155 | total_timesteps 1315.
Path 156 | total_timesteps 1322.
Path 157 | total_timesteps 1331.
Path 158 | total_timesteps 1339.
Path 159 | total_timesteps 1347.
Path 160 | total_timesteps 1354.
Path 161 | total_timesteps 1363.
Path 162 | total_timesteps 1373.
Path 163 | total_timesteps 1379.
Path 164 | total_timesteps 1386.
Path 165 | total_timesteps 1395.
Path 166 | total_timesteps 1403.
Path 167 | total_timesteps 1409.
Path 168 | total_timesteps 1417.
Path 169 | total_timesteps 1426.
Path 170 | total_timesteps 1433.
Path 171 | total_timesteps 1440.
Path 172 | total_timesteps 1448.
Path 173 | total_timesteps 1457.
Path 174 | total_timesteps 1466.
Path 175 | total_timesteps 1474.
Path 176 | total_timesteps 1482.
Path 177 | total_timesteps 1497.
Path 178 | total_timesteps 1507.
Path 179 | total_timesteps 1515.
Path 180 | total_timesteps 1525.
Path 181 | total_timesteps 1534.
Path 182 | total_timesteps 1540.
Path 183 | total_timesteps 1550.
Path 184 | total_timesteps 1557.
Path 185 | total_timesteps 1565.
Path 186 | total_timesteps 1574.
Path 187 | total_timesteps 1583.
Path 188 | total_timesteps 1600.
Path 189 | total_timesteps 1611.
Path 190 | total_timesteps 1618.
Path 191 | total_timesteps 1629.
Path 192 | total_timesteps 1636.
Path 193 | total_timesteps 1649.
Path 194 | total_timesteps 1658.
Path 195 | total_timesteps 1666.
Path 196 | total_timesteps 1673.
Path 197 | total_timesteps 1682.
Path 198 | total_timesteps 1692.
Path 199 | total_timesteps 1702.
Path 200 | total_timesteps 1711.
Path 201 | total_timesteps 1719.
Path 202 | total_timesteps 1728.
Path 203 | total_timesteps 1737.
Path 204 | total_timesteps 1750.
Path 205 | total_timesteps 1758.
Path 206 | total_timesteps 1764.
Path 207 | total_timesteps 1771.
Path 208 | total_timesteps 1780.
Path 209 | total_timesteps 1787.
Path 210 | total_timesteps 1797.
Path 211 | total_timesteps 1804.
Path 212 | total_timesteps 1811.
Path 213 | total_timesteps 1819.
Path 214 | total_timesteps 1826.
Path 215 | total_timesteps 1836.
Path 216 | total_timesteps 1843.
Path 217 | total_timesteps 1850.
Path 218 | total_timesteps 1858.
Path 219 | total_timesteps 1867.
Path 220 | total_timesteps 1875.
Path 221 | total_timesteps 1881.
Path 222 | total_timesteps 1890.
Path 223 | total_timesteps 1897.
Path 224 | total_timesteps 1903.
Path 225 | total_timesteps 1910.
Path 226 | total_timesteps 1917.
Path 227 | total_timesteps 1926.
Path 228 | total_timesteps 1937.
Path 229 | total_timesteps 1944.
Path 230 | total_timesteps 1954.
Path 231 | total_timesteps 1966.
Path 232 | total_timesteps 1974.
Path 233 | total_timesteps 1981.
Path 234 | total_timesteps 1990.
Path 235 | total_timesteps 2005.
Path 236 | total_timesteps 2011.
Path 237 | total_timesteps 2022.
Path 238 | total_timesteps 2028.
Path 239 | total_timesteps 2034.
Path 240 | total_timesteps 2040.
Path 241 | total_timesteps 2047.
Path 242 | total_timesteps 2055.
Path 243 | total_timesteps 2074.
Path 244 | total_timesteps 2080.
Path 245 | total_timesteps 2088.
Path 246 | total_timesteps 2095.
Path 247 | total_timesteps 2100.
Path 248 | total_timesteps 2107.
Path 249 | total_timesteps 2114.
Path 250 | total_timesteps 2120.
Path 251 | total_timesteps 2128.
Path 252 | total_timesteps 2136.
Path 253 | total_timesteps 2143.
Path 254 | total_timesteps 2153.
Path 255 | total_timesteps 2161.
Path 256 | total_timesteps 2176.
Path 257 | total_timesteps 2184.
Path 258 | total_timesteps 2192.
Path 259 | total_timesteps 2202.
Path 260 | total_timesteps 2213.
Path 261 | total_timesteps 2221.
Path 262 | total_timesteps 2227.
Path 263 | total_timesteps 2233.
Path 264 | total_timesteps 2239.
Path 265 | total_timesteps 2248.
Path 266 | total_timesteps 2255.
Path 267 | total_timesteps 2262.
Path 268 | total_timesteps 2271.
Path 269 | total_timesteps 2280.
Path 270 | total_timesteps 2288.
Path 271 | total_timesteps 2294.
Path 272 | total_timesteps 2301.
Path 273 | total_timesteps 2314.
Path 274 | total_timesteps 2323.
Path 275 | total_timesteps 2335.
Path 276 | total_timesteps 2346.
Path 277 | total_timesteps 2352.
Path 278 | total_timesteps 2361.
Path 279 | total_timesteps 2369.
Path 280 | total_timesteps 2379.
Path 281 | total_timesteps 2388.
Path 282 | total_timesteps 2398.
Path 283 | total_timesteps 2407.
Path 284 | total_timesteps 2416.
Path 285 | total_timesteps 2423.
Path 286 | total_timesteps 2431.
Path 287 | total_timesteps 2440.
Path 288 | total_timesteps 2446.
Path 289 | total_timesteps 2455.
Path 290 | total_timesteps 2463.
Path 291 | total_timesteps 2472.
Path 292 | total_timesteps 2478.
Path 293 | total_timesteps 2485.
Path 294 | total_timesteps 2491.
Path 295 | total_timesteps 2498.
Path 296 | total_timesteps 2507.
Path 297 | total_timesteps 2518.
Path 298 | total_timesteps 2527.
Path 299 | total_timesteps 2533.
Path 300 | total_timesteps 2543.
Path 301 | total_timesteps 2552.
Path 302 | total_timesteps 2563.
Path 303 | total_timesteps 2571.
Path 304 | total_timesteps 2579.
Path 305 | total_timesteps 2587.
Path 306 | total_timesteps 2596.
Path 307 | total_timesteps 2604.
Path 308 | total_timesteps 2611.
Path 309 | total_timesteps 2619.
Path 310 | total_timesteps 2625.
Path 311 | total_timesteps 2636.
Path 312 | total_timesteps 2644.
Path 313 | total_timesteps 2652.
Path 314 | total_timesteps 2662.
Path 315 | total_timesteps 2669.
Path 316 | total_timesteps 2680.
Path 317 | total_timesteps 2688.
Path 318 | total_timesteps 2696.
Path 319 | total_timesteps 2704.
Path 320 | total_timesteps 2713.
Path 321 | total_timesteps 2721.
Path 322 | total_timesteps 2728.
Path 323 | total_timesteps 2735.
Path 324 | total_timesteps 2747.
Path 325 | total_timesteps 2755.
Path 326 | total_timesteps 2762.
Path 327 | total_timesteps 2771.
Path 328 | total_timesteps 2782.
Path 329 | total_timesteps 2792.
Path 330 | total_timesteps 2802.
Path 331 | total_timesteps 2813.
Path 332 | total_timesteps 2823.
Path 333 | total_timesteps 2833.
Path 334 | total_timesteps 2842.
Path 335 | total_timesteps 2850.
Path 336 | total_timesteps 2858.
Path 337 | total_timesteps 2866.
Path 338 | total_timesteps 2877.
Path 339 | total_timesteps 2884.
Path 340 | total_timesteps 2890.
Path 341 | total_timesteps 2897.
Path 342 | total_timesteps 2905.
Path 343 | total_timesteps 2915.
Path 344 | total_timesteps 2922.
Path 345 | total_timesteps 2930.
Path 346 | total_timesteps 2938.
Path 347 | total_timesteps 2945.
Path 348 | total_timesteps 2951.
Path 349 | total_timesteps 2961.
Path 350 | total_timesteps 2969.
Path 351 | total_timesteps 2977.
Path 352 | total_timesteps 2984.
Path 353 | total_timesteps 2992.
Path 354 | total_timesteps 2999.
Path 355 | total_timesteps 3009.
Path 356 | total_timesteps 3016.
Path 357 | total_timesteps 3027.
Path 358 | total_timesteps 3035.
Path 359 | total_timesteps 3043.
Path 360 | total_timesteps 3052.
Path 361 | total_timesteps 3062.
Path 362 | total_timesteps 3071.
Path 363 | total_timesteps 3081.
Path 364 | total_timesteps 3090.
Path 365 | total_timesteps 3098.
Path 366 | total_timesteps 3110.
Path 367 | total_timesteps 3117.
Path 368 | total_timesteps 3126.
Path 369 | total_timesteps 3135.
Path 370 | total_timesteps 3144.
Path 371 | total_timesteps 3154.
Path 372 | total_timesteps 3167.
Path 373 | total_timesteps 3175.
Path 374 | total_timesteps 3184.
Path 375 | total_timesteps 3191.
Path 376 | total_timesteps 3200.
Path 377 | total_timesteps 3209.
Path 378 | total_timesteps 3216.
Path 379 | total_timesteps 3225.
Path 380 | total_timesteps 3235.
Path 381 | total_timesteps 3243.
Path 382 | total_timesteps 3250.
Path 383 | total_timesteps 3263.
Path 384 | total_timesteps 3281.
Path 385 | total_timesteps 3290.
Path 386 | total_timesteps 3298.
Path 387 | total_timesteps 3306.
Path 388 | total_timesteps 3314.
Path 389 | total_timesteps 3323.
Path 390 | total_timesteps 3331.
Path 391 | total_timesteps 3338.
Path 392 | total_timesteps 3350.
Path 393 | total_timesteps 3357.
Path 394 | total_timesteps 3363.
Path 395 | total_timesteps 3371.
Path 396 | total_timesteps 3380.
Path 397 | total_timesteps 3388.
Path 398 | total_timesteps 3402.
Path 399 | total_timesteps 3410.
Path 400 | total_timesteps 3421.
Path 401 | total_timesteps 3429.
Path 402 | total_timesteps 3438.
Path 403 | total_timesteps 3444.
Path 404 | total_timesteps 3451.
Path 405 | total_timesteps 3457.
Path 406 | total_timesteps 3466.
Path 407 | total_timesteps 3474.
Path 408 | total_timesteps 3483.
Path 409 | total_timesteps 3491.
Path 410 | total_timesteps 3501.
Path 411 | total_timesteps 3511.
Path 412 | total_timesteps 3520.
Path 413 | total_timesteps 3528.
Path 414 | total_timesteps 3536.
Path 415 | total_timesteps 3542.
Path 416 | total_timesteps 3550.
Path 417 | total_timesteps 3557.
Path 418 | total_timesteps 3567.
Path 419 | total_timesteps 3576.
Path 420 | total_timesteps 3584.
Path 421 | total_timesteps 3592.
Path 422 | total_timesteps 3602.
Path 423 | total_timesteps 3608.
Path 424 | total_timesteps 3615.
Path 425 | total_timesteps 3623.
Path 426 | total_timesteps 3634.
Path 427 | total_timesteps 3643.
Path 428 | total_timesteps 3651.
Path 429 | total_timesteps 3658.
Path 430 | total_timesteps 3668.
Path 431 | total_timesteps 3676.
Path 432 | total_timesteps 3684.
Path 433 | total_timesteps 3693.
Path 434 | total_timesteps 3701.
Path 435 | total_timesteps 3708.
Path 436 | total_timesteps 3715.
Path 437 | total_timesteps 3724.
Path 438 | total_timesteps 3731.
Path 439 | total_timesteps 3741.
Path 440 | total_timesteps 3749.
Path 441 | total_timesteps 3756.
Path 442 | total_timesteps 3764.
Path 443 | total_timesteps 3772.
Path 444 | total_timesteps 3783.
Path 445 | total_timesteps 3791.
Path 446 | total_timesteps 3799.
Path 447 | total_timesteps 3805.
Path 448 | total_timesteps 3815.
Path 449 | total_timesteps 3823.
Path 450 | total_timesteps 3831.
Path 451 | total_timesteps 3843.
Path 452 | total_timesteps 3858.
Path 453 | total_timesteps 3866.
Path 454 | total_timesteps 3875.
Path 455 | total_timesteps 3882.
Path 456 | total_timesteps 3891.
Path 457 | total_timesteps 3901.
Path 458 | total_timesteps 3908.
Path 459 | total_timesteps 3918.
Path 460 | total_timesteps 3924.
Path 461 | total_timesteps 3932.
Path 462 | total_timesteps 3939.
Path 463 | total_timesteps 3947.
Path 464 | total_timesteps 3954.
Path 465 | total_timesteps 3961.
Path 466 | total_timesteps 3968.
Path 467 | total_timesteps 3976.
Path 468 | total_timesteps 3991.
Path 469 | total_timesteps 4002.
Path 470 | total_timesteps 4012.
Path 471 | total_timesteps 4019.
Path 472 | total_timesteps 4028.
Path 473 | total_timesteps 4036.
Path 474 | total_timesteps 4043.
Path 475 | total_timesteps 4052.
Path 476 | total_timesteps 4061.
Path 477 | total_timesteps 4075.
Path 478 | total_timesteps 4082.
Path 479 | total_timesteps 4091.
Path 480 | total_timesteps 4102.
Path 481 | total_timesteps 4109.
Path 482 | total_timesteps 4119.
Path 483 | total_timesteps 4128.
Path 484 | total_timesteps 4140.
Path 485 | total_timesteps 4149.
Path 486 | total_timesteps 4156.
Path 487 | total_timesteps 4163.
Path 488 | total_timesteps 4175.
Path 489 | total_timesteps 4181.
Path 490 | total_timesteps 4188.
Path 491 | total_timesteps 4203.
Path 492 | total_timesteps 4211.
Path 493 | total_timesteps 4218.
Path 494 | total_timesteps 4230.
Path 495 | total_timesteps 4238.
Path 496 | total_timesteps 4247.
Path 497 | total_timesteps 4257.
Path 498 | total_timesteps 4267.
Path 499 | total_timesteps 4275.
Path 500 | total_timesteps 4282.
Path 501 | total_timesteps 4295.
Path 502 | total_timesteps 4302.
Path 503 | total_timesteps 4314.
Path 504 | total_timesteps 4322.
Path 505 | total_timesteps 4333.
Path 506 | total_timesteps 4341.
Path 507 | total_timesteps 4357.
Path 508 | total_timesteps 4364.
Path 509 | total_timesteps 4376.
Path 510 | total_timesteps 4384.
Path 511 | total_timesteps 4390.
Path 512 | total_timesteps 4398.
Path 513 | total_timesteps 4404.
Path 514 | total_timesteps 4410.
Path 515 | total_timesteps 4421.
Path 516 | total_timesteps 4430.
Path 517 | total_timesteps 4440.
Path 518 | total_timesteps 4448.
Path 519 | total_timesteps 4455.
Path 520 | total_timesteps 4464.
Path 521 | total_timesteps 4472.
Path 522 | total_timesteps 4482.
Path 523 | total_timesteps 4489.
Path 524 | total_timesteps 4496.
Path 525 | total_timesteps 4503.
Path 526 | total_timesteps 4511.
Path 527 | total_timesteps 4521.
Path 528 | total_timesteps 4532.
Path 529 | total_timesteps 4539.
Path 530 | total_timesteps 4546.
Path 531 | total_timesteps 4555.
Path 532 | total_timesteps 4566.
Path 533 | total_timesteps 4575.
Path 534 | total_timesteps 4581.
Path 535 | total_timesteps 4589.
Path 536 | total_timesteps 4596.
Path 537 | total_timesteps 4605.
Path 538 | total_timesteps 4612.
Path 539 | total_timesteps 4620.
Path 540 | total_timesteps 4627.
Path 541 | total_timesteps 4637.
Path 542 | total_timesteps 4644.
Path 543 | total_timesteps 4651.
Path 544 | total_timesteps 4658.
Path 545 | total_timesteps 4666.
Path 546 | total_timesteps 4672.
Path 547 | total_timesteps 4680.
Path 548 | total_timesteps 4691.
Path 549 | total_timesteps 4700.
Path 550 | total_timesteps 4710.
Path 551 | total_timesteps 4718.
Path 552 | total_timesteps 4727.
Path 553 | total_timesteps 4737.
Path 554 | total_timesteps 4747.
Path 555 | total_timesteps 4759.
Path 556 | total_timesteps 4767.
Path 557 | total_timesteps 4776.
Path 558 | total_timesteps 4783.
Path 559 | total_timesteps 4798.
Path 560 | total_timesteps 4807.
Path 561 | total_timesteps 4813.
Path 562 | total_timesteps 4823.
Path 563 | total_timesteps 4831.
Path 564 | total_timesteps 4838.
Path 565 | total_timesteps 4846.
Path 566 | total_timesteps 4854.
Path 567 | total_timesteps 4862.
Path 568 | total_timesteps 4872.
Path 569 | total_timesteps 4881.
Path 570 | total_timesteps 4889.
Path 571 | total_timesteps 4896.
Path 572 | total_timesteps 4904.
Path 573 | total_timesteps 4911.
Path 574 | total_timesteps 4919.
Path 575 | total_timesteps 4929.
Path 576 | total_timesteps 4941.
Path 577 | total_timesteps 4952.
Path 578 | total_timesteps 4961.
Path 579 | total_timesteps 4968.
Path 580 | total_timesteps 4977.
Path 581 | total_timesteps 4987.
Path 582 | total_timesteps 4995.
Path 583 | total_timesteps 5005.
Path 584 | total_timesteps 5012.
Path 585 | total_timesteps 5021.
Path 586 | total_timesteps 5028.
Path 587 | total_timesteps 5034.
Path 588 | total_timesteps 5041.
Path 589 | total_timesteps 5049.
Path 590 | total_timesteps 5057.
Path 591 | total_timesteps 5079.
Path 592 | total_timesteps 5087.
Path 593 | total_timesteps 5096.
Path 594 | total_timesteps 5106.
Path 595 | total_timesteps 5113.
Path 596 | total_timesteps 5120.
Path 597 | total_timesteps 5128.
Path 598 | total_timesteps 5136.
Path 599 | total_timesteps 5143.
Path 600 | total_timesteps 5152.
Path 601 | total_timesteps 5159.
Path 602 | total_timesteps 5168.
Path 603 | total_timesteps 5179.
Path 604 | total_timesteps 5188.
Path 605 | total_timesteps 5194.
Path 606 | total_timesteps 5200.
Path 607 | total_timesteps 5209.
Path 608 | total_timesteps 5218.
Path 609 | total_timesteps 5230.
Path 610 | total_timesteps 5237.
Path 611 | total_timesteps 5245.
Path 612 | total_timesteps 5256.
Path 613 | total_timesteps 5263.
Path 614 | total_timesteps 5273.
Path 615 | total_timesteps 5281.
Path 616 | total_timesteps 5288.
Path 617 | total_timesteps 5296.
Path 618 | total_timesteps 5306.
Path 619 | total_timesteps 5315.
Path 620 | total_timesteps 5323.
Path 621 | total_timesteps 5335.
Path 622 | total_timesteps 5342.
Path 623 | total_timesteps 5352.
Path 624 | total_timesteps 5358.
Path 625 | total_timesteps 5365.
Path 626 | total_timesteps 5372.
Path 627 | total_timesteps 5380.
Path 628 | total_timesteps 5390.
Path 629 | total_timesteps 5398.
Path 630 | total_timesteps 5408.
Path 631 | total_timesteps 5419.
Path 632 | total_timesteps 5429.
Path 633 | total_timesteps 5436.
Path 634 | total_timesteps 5444.
Path 635 | total_timesteps 5451.
Path 636 | total_timesteps 5464.
Path 637 | total_timesteps 5472.
Path 638 | total_timesteps 5481.
Path 639 | total_timesteps 5492.
Path 640 | total_timesteps 5501.
Path 641 | total_timesteps 5510.
Path 642 | total_timesteps 5517.
Path 643 | total_timesteps 5528.
Path 644 | total_timesteps 5536.
Path 645 | total_timesteps 5542.
Path 646 | total_timesteps 5552.
Path 647 | total_timesteps 5559.
Path 648 | total_timesteps 5568.
Path 649 | total_timesteps 5577.
Path 650 | total_timesteps 5583.
Path 651 | total_timesteps 5590.
Path 652 | total_timesteps 5597.
Path 653 | total_timesteps 5607.
Path 654 | total_timesteps 5616.
Path 655 | total_timesteps 5624.
Path 656 | total_timesteps 5631.
Path 657 | total_timesteps 5638.
Path 658 | total_timesteps 5647.
Path 659 | total_timesteps 5657.
Path 660 | total_timesteps 5665.
Path 661 | total_timesteps 5674.
Path 662 | total_timesteps 5682.
Path 663 | total_timesteps 5689.
Path 664 | total_timesteps 5696.
Path 665 | total_timesteps 5703.
Path 666 | total_timesteps 5711.
Path 667 | total_timesteps 5721.
Path 668 | total_timesteps 5730.
Path 669 | total_timesteps 5736.
Path 670 | total_timesteps 5748.
Path 671 | total_timesteps 5757.
Path 672 | total_timesteps 5770.
Path 673 | total_timesteps 5777.
Path 674 | total_timesteps 5787.
Path 675 | total_timesteps 5794.
Path 676 | total_timesteps 5801.
Path 677 | total_timesteps 5808.
Path 678 | total_timesteps 5816.
Path 679 | total_timesteps 5823.
Path 680 | total_timesteps 5833.
Path 681 | total_timesteps 5842.
Path 682 | total_timesteps 5850.
Path 683 | total_timesteps 5864.
Path 684 | total_timesteps 5873.
Path 685 | total_timesteps 5881.
Path 686 | total_timesteps 5888.
Path 687 | total_timesteps 5896.
Path 688 | total_timesteps 5905.
Path 689 | total_timesteps 5918.
Path 690 | total_timesteps 5925.
Path 691 | total_timesteps 5933.
Path 692 | total_timesteps 5941.
Path 693 | total_timesteps 5949.
Path 694 | total_timesteps 5962.
Path 695 | total_timesteps 5973.
Path 696 | total_timesteps 5985.
Path 697 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.69    |
| Iteration     | 14       |
| MaximumReturn | 2.23     |
| MinimumReturn | -15.1    |
| TotalSamples  | 64061    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.003490322269499302
Validation loss = 0.0033353990875184536
Validation loss = 0.0030437414534389973
Validation loss = 0.0032571456395089626
Validation loss = 0.003254376817494631
Validation loss = 0.0027287600096315145
Validation loss = 0.0027039488777518272
Validation loss = 0.0030258032493293285
Validation loss = 0.00282593653537333
Validation loss = 0.003443450666964054
Validation loss = 0.0028901458717882633
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 23.
Path 4 | total_timesteps 32.
Path 5 | total_timesteps 42.
Path 6 | total_timesteps 50.
Path 7 | total_timesteps 58.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 86.
Path 10 | total_timesteps 94.
Path 11 | total_timesteps 102.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 118.
Path 14 | total_timesteps 127.
Path 15 | total_timesteps 135.
Path 16 | total_timesteps 142.
Path 17 | total_timesteps 148.
Path 18 | total_timesteps 156.
Path 19 | total_timesteps 164.
Path 20 | total_timesteps 174.
Path 21 | total_timesteps 182.
Path 22 | total_timesteps 191.
Path 23 | total_timesteps 198.
Path 24 | total_timesteps 205.
Path 25 | total_timesteps 214.
Path 26 | total_timesteps 220.
Path 27 | total_timesteps 229.
Path 28 | total_timesteps 237.
Path 29 | total_timesteps 246.
Path 30 | total_timesteps 257.
Path 31 | total_timesteps 264.
Path 32 | total_timesteps 272.
Path 33 | total_timesteps 279.
Path 34 | total_timesteps 286.
Path 35 | total_timesteps 294.
Path 36 | total_timesteps 302.
Path 37 | total_timesteps 309.
Path 38 | total_timesteps 318.
Path 39 | total_timesteps 328.
Path 40 | total_timesteps 335.
Path 41 | total_timesteps 342.
Path 42 | total_timesteps 348.
Path 43 | total_timesteps 357.
Path 44 | total_timesteps 365.
Path 45 | total_timesteps 372.
Path 46 | total_timesteps 380.
Path 47 | total_timesteps 387.
Path 48 | total_timesteps 395.
Path 49 | total_timesteps 402.
Path 50 | total_timesteps 410.
Path 51 | total_timesteps 417.
Path 52 | total_timesteps 424.
Path 53 | total_timesteps 431.
Path 54 | total_timesteps 442.
Path 55 | total_timesteps 448.
Path 56 | total_timesteps 455.
Path 57 | total_timesteps 462.
Path 58 | total_timesteps 479.
Path 59 | total_timesteps 485.
Path 60 | total_timesteps 492.
Path 61 | total_timesteps 507.
Path 62 | total_timesteps 514.
Path 63 | total_timesteps 525.
Path 64 | total_timesteps 533.
Path 65 | total_timesteps 540.
Path 66 | total_timesteps 551.
Path 67 | total_timesteps 559.
Path 68 | total_timesteps 567.
Path 69 | total_timesteps 576.
Path 70 | total_timesteps 584.
Path 71 | total_timesteps 594.
Path 72 | total_timesteps 603.
Path 73 | total_timesteps 609.
Path 74 | total_timesteps 616.
Path 75 | total_timesteps 622.
Path 76 | total_timesteps 633.
Path 77 | total_timesteps 641.
Path 78 | total_timesteps 650.
Path 79 | total_timesteps 659.
Path 80 | total_timesteps 667.
Path 81 | total_timesteps 677.
Path 82 | total_timesteps 686.
Path 83 | total_timesteps 696.
Path 84 | total_timesteps 709.
Path 85 | total_timesteps 715.
Path 86 | total_timesteps 722.
Path 87 | total_timesteps 731.
Path 88 | total_timesteps 740.
Path 89 | total_timesteps 752.
Path 90 | total_timesteps 760.
Path 91 | total_timesteps 768.
Path 92 | total_timesteps 777.
Path 93 | total_timesteps 786.
Path 94 | total_timesteps 796.
Path 95 | total_timesteps 805.
Path 96 | total_timesteps 817.
Path 97 | total_timesteps 826.
Path 98 | total_timesteps 833.
Path 99 | total_timesteps 840.
Path 100 | total_timesteps 849.
Path 101 | total_timesteps 861.
Path 102 | total_timesteps 868.
Path 103 | total_timesteps 874.
Path 104 | total_timesteps 881.
Path 105 | total_timesteps 890.
Path 106 | total_timesteps 899.
Path 107 | total_timesteps 908.
Path 108 | total_timesteps 916.
Path 109 | total_timesteps 922.
Path 110 | total_timesteps 932.
Path 111 | total_timesteps 938.
Path 112 | total_timesteps 945.
Path 113 | total_timesteps 954.
Path 114 | total_timesteps 961.
Path 115 | total_timesteps 968.
Path 116 | total_timesteps 976.
Path 117 | total_timesteps 989.
Path 118 | total_timesteps 997.
Path 119 | total_timesteps 1005.
Path 120 | total_timesteps 1019.
Path 121 | total_timesteps 1028.
Path 122 | total_timesteps 1035.
Path 123 | total_timesteps 1046.
Path 124 | total_timesteps 1053.
Path 125 | total_timesteps 1061.
Path 126 | total_timesteps 1070.
Path 127 | total_timesteps 1079.
Path 128 | total_timesteps 1089.
Path 129 | total_timesteps 1098.
Path 130 | total_timesteps 1108.
Path 131 | total_timesteps 1114.
Path 132 | total_timesteps 1121.
Path 133 | total_timesteps 1128.
Path 134 | total_timesteps 1136.
Path 135 | total_timesteps 1144.
Path 136 | total_timesteps 1153.
Path 137 | total_timesteps 1169.
Path 138 | total_timesteps 1175.
Path 139 | total_timesteps 1183.
Path 140 | total_timesteps 1192.
Path 141 | total_timesteps 1200.
Path 142 | total_timesteps 1207.
Path 143 | total_timesteps 1215.
Path 144 | total_timesteps 1222.
Path 145 | total_timesteps 1235.
Path 146 | total_timesteps 1242.
Path 147 | total_timesteps 1250.
Path 148 | total_timesteps 1256.
Path 149 | total_timesteps 1266.
Path 150 | total_timesteps 1277.
Path 151 | total_timesteps 1284.
Path 152 | total_timesteps 1294.
Path 153 | total_timesteps 1302.
Path 154 | total_timesteps 1314.
Path 155 | total_timesteps 1325.
Path 156 | total_timesteps 1333.
Path 157 | total_timesteps 1341.
Path 158 | total_timesteps 1349.
Path 159 | total_timesteps 1357.
Path 160 | total_timesteps 1366.
Path 161 | total_timesteps 1374.
Path 162 | total_timesteps 1384.
Path 163 | total_timesteps 1391.
Path 164 | total_timesteps 1400.
Path 165 | total_timesteps 1410.
Path 166 | total_timesteps 1418.
Path 167 | total_timesteps 1425.
Path 168 | total_timesteps 1431.
Path 169 | total_timesteps 1438.
Path 170 | total_timesteps 1446.
Path 171 | total_timesteps 1457.
Path 172 | total_timesteps 1467.
Path 173 | total_timesteps 1480.
Path 174 | total_timesteps 1487.
Path 175 | total_timesteps 1493.
Path 176 | total_timesteps 1505.
Path 177 | total_timesteps 1513.
Path 178 | total_timesteps 1521.
Path 179 | total_timesteps 1528.
Path 180 | total_timesteps 1535.
Path 181 | total_timesteps 1544.
Path 182 | total_timesteps 1554.
Path 183 | total_timesteps 1561.
Path 184 | total_timesteps 1569.
Path 185 | total_timesteps 1575.
Path 186 | total_timesteps 1584.
Path 187 | total_timesteps 1592.
Path 188 | total_timesteps 1600.
Path 189 | total_timesteps 1615.
Path 190 | total_timesteps 1626.
Path 191 | total_timesteps 1633.
Path 192 | total_timesteps 1640.
Path 193 | total_timesteps 1647.
Path 194 | total_timesteps 1654.
Path 195 | total_timesteps 1663.
Path 196 | total_timesteps 1672.
Path 197 | total_timesteps 1680.
Path 198 | total_timesteps 1690.
Path 199 | total_timesteps 1696.
Path 200 | total_timesteps 1703.
Path 201 | total_timesteps 1709.
Path 202 | total_timesteps 1719.
Path 203 | total_timesteps 1728.
Path 204 | total_timesteps 1736.
Path 205 | total_timesteps 1747.
Path 206 | total_timesteps 1753.
Path 207 | total_timesteps 1761.
Path 208 | total_timesteps 1773.
Path 209 | total_timesteps 1780.
Path 210 | total_timesteps 1788.
Path 211 | total_timesteps 1797.
Path 212 | total_timesteps 1805.
Path 213 | total_timesteps 1812.
Path 214 | total_timesteps 1819.
Path 215 | total_timesteps 1826.
Path 216 | total_timesteps 1833.
Path 217 | total_timesteps 1840.
Path 218 | total_timesteps 1850.
Path 219 | total_timesteps 1858.
Path 220 | total_timesteps 1868.
Path 221 | total_timesteps 1876.
Path 222 | total_timesteps 1885.
Path 223 | total_timesteps 1897.
Path 224 | total_timesteps 1906.
Path 225 | total_timesteps 1916.
Path 226 | total_timesteps 1927.
Path 227 | total_timesteps 1933.
Path 228 | total_timesteps 1939.
Path 229 | total_timesteps 1951.
Path 230 | total_timesteps 1958.
Path 231 | total_timesteps 1970.
Path 232 | total_timesteps 1977.
Path 233 | total_timesteps 1986.
Path 234 | total_timesteps 1994.
Path 235 | total_timesteps 2002.
Path 236 | total_timesteps 2014.
Path 237 | total_timesteps 2021.
Path 238 | total_timesteps 2032.
Path 239 | total_timesteps 2044.
Path 240 | total_timesteps 2055.
Path 241 | total_timesteps 2064.
Path 242 | total_timesteps 2073.
Path 243 | total_timesteps 2080.
Path 244 | total_timesteps 2087.
Path 245 | total_timesteps 2095.
Path 246 | total_timesteps 2103.
Path 247 | total_timesteps 2109.
Path 248 | total_timesteps 2116.
Path 249 | total_timesteps 2123.
Path 250 | total_timesteps 2135.
Path 251 | total_timesteps 2142.
Path 252 | total_timesteps 2150.
Path 253 | total_timesteps 2158.
Path 254 | total_timesteps 2165.
Path 255 | total_timesteps 2173.
Path 256 | total_timesteps 2182.
Path 257 | total_timesteps 2189.
Path 258 | total_timesteps 2197.
Path 259 | total_timesteps 2207.
Path 260 | total_timesteps 2214.
Path 261 | total_timesteps 2224.
Path 262 | total_timesteps 2233.
Path 263 | total_timesteps 2241.
Path 264 | total_timesteps 2249.
Path 265 | total_timesteps 2258.
Path 266 | total_timesteps 2267.
Path 267 | total_timesteps 2279.
Path 268 | total_timesteps 2286.
Path 269 | total_timesteps 2296.
Path 270 | total_timesteps 2306.
Path 271 | total_timesteps 2315.
Path 272 | total_timesteps 2323.
Path 273 | total_timesteps 2331.
Path 274 | total_timesteps 2343.
Path 275 | total_timesteps 2350.
Path 276 | total_timesteps 2360.
Path 277 | total_timesteps 2369.
Path 278 | total_timesteps 2379.
Path 279 | total_timesteps 2387.
Path 280 | total_timesteps 2400.
Path 281 | total_timesteps 2407.
Path 282 | total_timesteps 2418.
Path 283 | total_timesteps 2429.
Path 284 | total_timesteps 2437.
Path 285 | total_timesteps 2445.
Path 286 | total_timesteps 2454.
Path 287 | total_timesteps 2461.
Path 288 | total_timesteps 2469.
Path 289 | total_timesteps 2479.
Path 290 | total_timesteps 2487.
Path 291 | total_timesteps 2500.
Path 292 | total_timesteps 2509.
Path 293 | total_timesteps 2517.
Path 294 | total_timesteps 2525.
Path 295 | total_timesteps 2536.
Path 296 | total_timesteps 2544.
Path 297 | total_timesteps 2555.
Path 298 | total_timesteps 2563.
Path 299 | total_timesteps 2571.
Path 300 | total_timesteps 2579.
Path 301 | total_timesteps 2589.
Path 302 | total_timesteps 2595.
Path 303 | total_timesteps 2601.
Path 304 | total_timesteps 2609.
Path 305 | total_timesteps 2616.
Path 306 | total_timesteps 2623.
Path 307 | total_timesteps 2629.
Path 308 | total_timesteps 2635.
Path 309 | total_timesteps 2645.
Path 310 | total_timesteps 2654.
Path 311 | total_timesteps 2664.
Path 312 | total_timesteps 2677.
Path 313 | total_timesteps 2686.
Path 314 | total_timesteps 2693.
Path 315 | total_timesteps 2701.
Path 316 | total_timesteps 2711.
Path 317 | total_timesteps 2718.
Path 318 | total_timesteps 2727.
Path 319 | total_timesteps 2737.
Path 320 | total_timesteps 2746.
Path 321 | total_timesteps 2754.
Path 322 | total_timesteps 2761.
Path 323 | total_timesteps 2769.
Path 324 | total_timesteps 2777.
Path 325 | total_timesteps 2784.
Path 326 | total_timesteps 2797.
Path 327 | total_timesteps 2805.
Path 328 | total_timesteps 2814.
Path 329 | total_timesteps 2824.
Path 330 | total_timesteps 2833.
Path 331 | total_timesteps 2843.
Path 332 | total_timesteps 2850.
Path 333 | total_timesteps 2859.
Path 334 | total_timesteps 2867.
Path 335 | total_timesteps 2875.
Path 336 | total_timesteps 2884.
Path 337 | total_timesteps 2892.
Path 338 | total_timesteps 2905.
Path 339 | total_timesteps 2913.
Path 340 | total_timesteps 2921.
Path 341 | total_timesteps 2931.
Path 342 | total_timesteps 2938.
Path 343 | total_timesteps 2945.
Path 344 | total_timesteps 2955.
Path 345 | total_timesteps 2962.
Path 346 | total_timesteps 2968.
Path 347 | total_timesteps 2976.
Path 348 | total_timesteps 2985.
Path 349 | total_timesteps 2991.
Path 350 | total_timesteps 3001.
Path 351 | total_timesteps 3008.
Path 352 | total_timesteps 3015.
Path 353 | total_timesteps 3023.
Path 354 | total_timesteps 3031.
Path 355 | total_timesteps 3039.
Path 356 | total_timesteps 3046.
Path 357 | total_timesteps 3054.
Path 358 | total_timesteps 3064.
Path 359 | total_timesteps 3073.
Path 360 | total_timesteps 3081.
Path 361 | total_timesteps 3092.
Path 362 | total_timesteps 3099.
Path 363 | total_timesteps 3108.
Path 364 | total_timesteps 3116.
Path 365 | total_timesteps 3123.
Path 366 | total_timesteps 3135.
Path 367 | total_timesteps 3143.
Path 368 | total_timesteps 3153.
Path 369 | total_timesteps 3164.
Path 370 | total_timesteps 3172.
Path 371 | total_timesteps 3183.
Path 372 | total_timesteps 3190.
Path 373 | total_timesteps 3196.
Path 374 | total_timesteps 3205.
Path 375 | total_timesteps 3213.
Path 376 | total_timesteps 3226.
Path 377 | total_timesteps 3235.
Path 378 | total_timesteps 3242.
Path 379 | total_timesteps 3248.
Path 380 | total_timesteps 3254.
Path 381 | total_timesteps 3262.
Path 382 | total_timesteps 3272.
Path 383 | total_timesteps 3283.
Path 384 | total_timesteps 3290.
Path 385 | total_timesteps 3303.
Path 386 | total_timesteps 3310.
Path 387 | total_timesteps 3318.
Path 388 | total_timesteps 3325.
Path 389 | total_timesteps 3331.
Path 390 | total_timesteps 3338.
Path 391 | total_timesteps 3347.
Path 392 | total_timesteps 3355.
Path 393 | total_timesteps 3364.
Path 394 | total_timesteps 3371.
Path 395 | total_timesteps 3379.
Path 396 | total_timesteps 3388.
Path 397 | total_timesteps 3397.
Path 398 | total_timesteps 3403.
Path 399 | total_timesteps 3412.
Path 400 | total_timesteps 3419.
Path 401 | total_timesteps 3427.
Path 402 | total_timesteps 3436.
Path 403 | total_timesteps 3443.
Path 404 | total_timesteps 3450.
Path 405 | total_timesteps 3456.
Path 406 | total_timesteps 3463.
Path 407 | total_timesteps 3470.
Path 408 | total_timesteps 3477.
Path 409 | total_timesteps 3486.
Path 410 | total_timesteps 3494.
Path 411 | total_timesteps 3502.
Path 412 | total_timesteps 3512.
Path 413 | total_timesteps 3520.
Path 414 | total_timesteps 3528.
Path 415 | total_timesteps 3538.
Path 416 | total_timesteps 3547.
Path 417 | total_timesteps 3555.
Path 418 | total_timesteps 3564.
Path 419 | total_timesteps 3570.
Path 420 | total_timesteps 3578.
Path 421 | total_timesteps 3587.
Path 422 | total_timesteps 3594.
Path 423 | total_timesteps 3600.
Path 424 | total_timesteps 3610.
Path 425 | total_timesteps 3619.
Path 426 | total_timesteps 3625.
Path 427 | total_timesteps 3633.
Path 428 | total_timesteps 3639.
Path 429 | total_timesteps 3648.
Path 430 | total_timesteps 3656.
Path 431 | total_timesteps 3665.
Path 432 | total_timesteps 3672.
Path 433 | total_timesteps 3679.
Path 434 | total_timesteps 3688.
Path 435 | total_timesteps 3696.
Path 436 | total_timesteps 3703.
Path 437 | total_timesteps 3710.
Path 438 | total_timesteps 3716.
Path 439 | total_timesteps 3724.
Path 440 | total_timesteps 3732.
Path 441 | total_timesteps 3739.
Path 442 | total_timesteps 3750.
Path 443 | total_timesteps 3758.
Path 444 | total_timesteps 3771.
Path 445 | total_timesteps 3780.
Path 446 | total_timesteps 3789.
Path 447 | total_timesteps 3795.
Path 448 | total_timesteps 3804.
Path 449 | total_timesteps 3811.
Path 450 | total_timesteps 3818.
Path 451 | total_timesteps 3833.
Path 452 | total_timesteps 3845.
Path 453 | total_timesteps 3854.
Path 454 | total_timesteps 3860.
Path 455 | total_timesteps 3869.
Path 456 | total_timesteps 3878.
Path 457 | total_timesteps 3886.
Path 458 | total_timesteps 3893.
Path 459 | total_timesteps 3900.
Path 460 | total_timesteps 3908.
Path 461 | total_timesteps 3916.
Path 462 | total_timesteps 3924.
Path 463 | total_timesteps 3932.
Path 464 | total_timesteps 3939.
Path 465 | total_timesteps 3948.
Path 466 | total_timesteps 3955.
Path 467 | total_timesteps 3965.
Path 468 | total_timesteps 3975.
Path 469 | total_timesteps 3983.
Path 470 | total_timesteps 3990.
Path 471 | total_timesteps 4000.
Path 472 | total_timesteps 4007.
Path 473 | total_timesteps 4014.
Path 474 | total_timesteps 4022.
Path 475 | total_timesteps 4029.
Path 476 | total_timesteps 4040.
Path 477 | total_timesteps 4048.
Path 478 | total_timesteps 4058.
Path 479 | total_timesteps 4070.
Path 480 | total_timesteps 4081.
Path 481 | total_timesteps 4088.
Path 482 | total_timesteps 4096.
Path 483 | total_timesteps 4105.
Path 484 | total_timesteps 4112.
Path 485 | total_timesteps 4124.
Path 486 | total_timesteps 4137.
Path 487 | total_timesteps 4147.
Path 488 | total_timesteps 4154.
Path 489 | total_timesteps 4166.
Path 490 | total_timesteps 4177.
Path 491 | total_timesteps 4185.
Path 492 | total_timesteps 4194.
Path 493 | total_timesteps 4201.
Path 494 | total_timesteps 4213.
Path 495 | total_timesteps 4220.
Path 496 | total_timesteps 4229.
Path 497 | total_timesteps 4238.
Path 498 | total_timesteps 4247.
Path 499 | total_timesteps 4255.
Path 500 | total_timesteps 4263.
Path 501 | total_timesteps 4271.
Path 502 | total_timesteps 4281.
Path 503 | total_timesteps 4289.
Path 504 | total_timesteps 4296.
Path 505 | total_timesteps 4302.
Path 506 | total_timesteps 4311.
Path 507 | total_timesteps 4320.
Path 508 | total_timesteps 4330.
Path 509 | total_timesteps 4344.
Path 510 | total_timesteps 4355.
Path 511 | total_timesteps 4362.
Path 512 | total_timesteps 4369.
Path 513 | total_timesteps 4376.
Path 514 | total_timesteps 4384.
Path 515 | total_timesteps 4395.
Path 516 | total_timesteps 4405.
Path 517 | total_timesteps 4413.
Path 518 | total_timesteps 4425.
Path 519 | total_timesteps 4434.
Path 520 | total_timesteps 4443.
Path 521 | total_timesteps 4451.
Path 522 | total_timesteps 4461.
Path 523 | total_timesteps 4473.
Path 524 | total_timesteps 4480.
Path 525 | total_timesteps 4487.
Path 526 | total_timesteps 4494.
Path 527 | total_timesteps 4501.
Path 528 | total_timesteps 4508.
Path 529 | total_timesteps 4519.
Path 530 | total_timesteps 4527.
Path 531 | total_timesteps 4534.
Path 532 | total_timesteps 4540.
Path 533 | total_timesteps 4549.
Path 534 | total_timesteps 4558.
Path 535 | total_timesteps 4565.
Path 536 | total_timesteps 4578.
Path 537 | total_timesteps 4586.
Path 538 | total_timesteps 4597.
Path 539 | total_timesteps 4606.
Path 540 | total_timesteps 4617.
Path 541 | total_timesteps 4625.
Path 542 | total_timesteps 4635.
Path 543 | total_timesteps 4646.
Path 544 | total_timesteps 4659.
Path 545 | total_timesteps 4668.
Path 546 | total_timesteps 4676.
Path 547 | total_timesteps 4684.
Path 548 | total_timesteps 4691.
Path 549 | total_timesteps 4698.
Path 550 | total_timesteps 4705.
Path 551 | total_timesteps 4718.
Path 552 | total_timesteps 4728.
Path 553 | total_timesteps 4735.
Path 554 | total_timesteps 4743.
Path 555 | total_timesteps 4750.
Path 556 | total_timesteps 4756.
Path 557 | total_timesteps 4767.
Path 558 | total_timesteps 4775.
Path 559 | total_timesteps 4782.
Path 560 | total_timesteps 4791.
Path 561 | total_timesteps 4797.
Path 562 | total_timesteps 4806.
Path 563 | total_timesteps 4815.
Path 564 | total_timesteps 4824.
Path 565 | total_timesteps 4832.
Path 566 | total_timesteps 4839.
Path 567 | total_timesteps 4849.
Path 568 | total_timesteps 4858.
Path 569 | total_timesteps 4869.
Path 570 | total_timesteps 4881.
Path 571 | total_timesteps 4892.
Path 572 | total_timesteps 4899.
Path 573 | total_timesteps 4909.
Path 574 | total_timesteps 4916.
Path 575 | total_timesteps 4923.
Path 576 | total_timesteps 4932.
Path 577 | total_timesteps 4944.
Path 578 | total_timesteps 4951.
Path 579 | total_timesteps 4957.
Path 580 | total_timesteps 4965.
Path 581 | total_timesteps 4973.
Path 582 | total_timesteps 4979.
Path 583 | total_timesteps 4987.
Path 584 | total_timesteps 4995.
Path 585 | total_timesteps 5002.
Path 586 | total_timesteps 5010.
Path 587 | total_timesteps 5022.
Path 588 | total_timesteps 5030.
Path 589 | total_timesteps 5042.
Path 590 | total_timesteps 5052.
Path 591 | total_timesteps 5060.
Path 592 | total_timesteps 5069.
Path 593 | total_timesteps 5078.
Path 594 | total_timesteps 5088.
Path 595 | total_timesteps 5096.
Path 596 | total_timesteps 5106.
Path 597 | total_timesteps 5113.
Path 598 | total_timesteps 5120.
Path 599 | total_timesteps 5128.
Path 600 | total_timesteps 5139.
Path 601 | total_timesteps 5147.
Path 602 | total_timesteps 5154.
Path 603 | total_timesteps 5160.
Path 604 | total_timesteps 5167.
Path 605 | total_timesteps 5177.
Path 606 | total_timesteps 5184.
Path 607 | total_timesteps 5191.
Path 608 | total_timesteps 5198.
Path 609 | total_timesteps 5207.
Path 610 | total_timesteps 5218.
Path 611 | total_timesteps 5229.
Path 612 | total_timesteps 5237.
Path 613 | total_timesteps 5246.
Path 614 | total_timesteps 5253.
Path 615 | total_timesteps 5261.
Path 616 | total_timesteps 5275.
Path 617 | total_timesteps 5282.
Path 618 | total_timesteps 5289.
Path 619 | total_timesteps 5296.
Path 620 | total_timesteps 5303.
Path 621 | total_timesteps 5311.
Path 622 | total_timesteps 5324.
Path 623 | total_timesteps 5334.
Path 624 | total_timesteps 5340.
Path 625 | total_timesteps 5348.
Path 626 | total_timesteps 5359.
Path 627 | total_timesteps 5367.
Path 628 | total_timesteps 5374.
Path 629 | total_timesteps 5380.
Path 630 | total_timesteps 5390.
Path 631 | total_timesteps 5403.
Path 632 | total_timesteps 5410.
Path 633 | total_timesteps 5418.
Path 634 | total_timesteps 5427.
Path 635 | total_timesteps 5441.
Path 636 | total_timesteps 5452.
Path 637 | total_timesteps 5458.
Path 638 | total_timesteps 5464.
Path 639 | total_timesteps 5475.
Path 640 | total_timesteps 5484.
Path 641 | total_timesteps 5490.
Path 642 | total_timesteps 5500.
Path 643 | total_timesteps 5508.
Path 644 | total_timesteps 5517.
Path 645 | total_timesteps 5525.
Path 646 | total_timesteps 5535.
Path 647 | total_timesteps 5544.
Path 648 | total_timesteps 5552.
Path 649 | total_timesteps 5559.
Path 650 | total_timesteps 5571.
Path 651 | total_timesteps 5580.
Path 652 | total_timesteps 5588.
Path 653 | total_timesteps 5597.
Path 654 | total_timesteps 5608.
Path 655 | total_timesteps 5616.
Path 656 | total_timesteps 5624.
Path 657 | total_timesteps 5634.
Path 658 | total_timesteps 5641.
Path 659 | total_timesteps 5655.
Path 660 | total_timesteps 5665.
Path 661 | total_timesteps 5674.
Path 662 | total_timesteps 5686.
Path 663 | total_timesteps 5694.
Path 664 | total_timesteps 5702.
Path 665 | total_timesteps 5711.
Path 666 | total_timesteps 5718.
Path 667 | total_timesteps 5725.
Path 668 | total_timesteps 5733.
Path 669 | total_timesteps 5745.
Path 670 | total_timesteps 5752.
Path 671 | total_timesteps 5764.
Path 672 | total_timesteps 5771.
Path 673 | total_timesteps 5780.
Path 674 | total_timesteps 5788.
Path 675 | total_timesteps 5799.
Path 676 | total_timesteps 5805.
Path 677 | total_timesteps 5812.
Path 678 | total_timesteps 5821.
Path 679 | total_timesteps 5834.
Path 680 | total_timesteps 5840.
Path 681 | total_timesteps 5847.
Path 682 | total_timesteps 5853.
Path 683 | total_timesteps 5860.
Path 684 | total_timesteps 5866.
Path 685 | total_timesteps 5874.
Path 686 | total_timesteps 5882.
Path 687 | total_timesteps 5890.
Path 688 | total_timesteps 5897.
Path 689 | total_timesteps 5906.
Path 690 | total_timesteps 5912.
Path 691 | total_timesteps 5923.
Path 692 | total_timesteps 5932.
Path 693 | total_timesteps 5940.
Path 694 | total_timesteps 5949.
Path 695 | total_timesteps 5960.
Path 696 | total_timesteps 5969.
Path 697 | total_timesteps 5978.
Path 698 | total_timesteps 5984.
Path 699 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.64    |
| Iteration     | 15       |
| MaximumReturn | -1.94    |
| MinimumReturn | -14.4    |
| TotalSamples  | 68061    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.00271604279987514
Validation loss = 0.0031617770437151194
Validation loss = 0.002628934569656849
Validation loss = 0.003381883492693305
Validation loss = 0.002978493459522724
Validation loss = 0.002691091736778617
Validation loss = 0.003240574849769473
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 15.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 31.
Path 5 | total_timesteps 41.
Path 6 | total_timesteps 48.
Path 7 | total_timesteps 55.
Path 8 | total_timesteps 61.
Path 9 | total_timesteps 69.
Path 10 | total_timesteps 76.
Path 11 | total_timesteps 85.
Path 12 | total_timesteps 95.
Path 13 | total_timesteps 103.
Path 14 | total_timesteps 111.
Path 15 | total_timesteps 120.
Path 16 | total_timesteps 128.
Path 17 | total_timesteps 138.
Path 18 | total_timesteps 145.
Path 19 | total_timesteps 152.
Path 20 | total_timesteps 164.
Path 21 | total_timesteps 176.
Path 22 | total_timesteps 186.
Path 23 | total_timesteps 195.
Path 24 | total_timesteps 202.
Path 25 | total_timesteps 214.
Path 26 | total_timesteps 223.
Path 27 | total_timesteps 232.
Path 28 | total_timesteps 239.
Path 29 | total_timesteps 246.
Path 30 | total_timesteps 252.
Path 31 | total_timesteps 259.
Path 32 | total_timesteps 267.
Path 33 | total_timesteps 273.
Path 34 | total_timesteps 283.
Path 35 | total_timesteps 289.
Path 36 | total_timesteps 300.
Path 37 | total_timesteps 309.
Path 38 | total_timesteps 321.
Path 39 | total_timesteps 327.
Path 40 | total_timesteps 336.
Path 41 | total_timesteps 343.
Path 42 | total_timesteps 351.
Path 43 | total_timesteps 358.
Path 44 | total_timesteps 367.
Path 45 | total_timesteps 373.
Path 46 | total_timesteps 382.
Path 47 | total_timesteps 388.
Path 48 | total_timesteps 398.
Path 49 | total_timesteps 406.
Path 50 | total_timesteps 412.
Path 51 | total_timesteps 422.
Path 52 | total_timesteps 431.
Path 53 | total_timesteps 439.
Path 54 | total_timesteps 446.
Path 55 | total_timesteps 455.
Path 56 | total_timesteps 461.
Path 57 | total_timesteps 468.
Path 58 | total_timesteps 476.
Path 59 | total_timesteps 485.
Path 60 | total_timesteps 494.
Path 61 | total_timesteps 501.
Path 62 | total_timesteps 511.
Path 63 | total_timesteps 522.
Path 64 | total_timesteps 529.
Path 65 | total_timesteps 541.
Path 66 | total_timesteps 549.
Path 67 | total_timesteps 560.
Path 68 | total_timesteps 567.
Path 69 | total_timesteps 575.
Path 70 | total_timesteps 597.
Path 71 | total_timesteps 603.
Path 72 | total_timesteps 613.
Path 73 | total_timesteps 625.
Path 74 | total_timesteps 635.
Path 75 | total_timesteps 643.
Path 76 | total_timesteps 650.
Path 77 | total_timesteps 657.
Path 78 | total_timesteps 664.
Path 79 | total_timesteps 677.
Path 80 | total_timesteps 684.
Path 81 | total_timesteps 691.
Path 82 | total_timesteps 700.
Path 83 | total_timesteps 709.
Path 84 | total_timesteps 716.
Path 85 | total_timesteps 723.
Path 86 | total_timesteps 731.
Path 87 | total_timesteps 739.
Path 88 | total_timesteps 747.
Path 89 | total_timesteps 754.
Path 90 | total_timesteps 762.
Path 91 | total_timesteps 770.
Path 92 | total_timesteps 778.
Path 93 | total_timesteps 785.
Path 94 | total_timesteps 791.
Path 95 | total_timesteps 798.
Path 96 | total_timesteps 806.
Path 97 | total_timesteps 814.
Path 98 | total_timesteps 823.
Path 99 | total_timesteps 831.
Path 100 | total_timesteps 838.
Path 101 | total_timesteps 845.
Path 102 | total_timesteps 856.
Path 103 | total_timesteps 863.
Path 104 | total_timesteps 871.
Path 105 | total_timesteps 879.
Path 106 | total_timesteps 887.
Path 107 | total_timesteps 900.
Path 108 | total_timesteps 910.
Path 109 | total_timesteps 918.
Path 110 | total_timesteps 926.
Path 111 | total_timesteps 935.
Path 112 | total_timesteps 943.
Path 113 | total_timesteps 951.
Path 114 | total_timesteps 958.
Path 115 | total_timesteps 967.
Path 116 | total_timesteps 977.
Path 117 | total_timesteps 985.
Path 118 | total_timesteps 992.
Path 119 | total_timesteps 999.
Path 120 | total_timesteps 1006.
Path 121 | total_timesteps 1016.
Path 122 | total_timesteps 1023.
Path 123 | total_timesteps 1031.
Path 124 | total_timesteps 1041.
Path 125 | total_timesteps 1051.
Path 126 | total_timesteps 1059.
Path 127 | total_timesteps 1080.
Path 128 | total_timesteps 1087.
Path 129 | total_timesteps 1095.
Path 130 | total_timesteps 1103.
Path 131 | total_timesteps 1113.
Path 132 | total_timesteps 1122.
Path 133 | total_timesteps 1135.
Path 134 | total_timesteps 1145.
Path 135 | total_timesteps 1152.
Path 136 | total_timesteps 1158.
Path 137 | total_timesteps 1164.
Path 138 | total_timesteps 1173.
Path 139 | total_timesteps 1180.
Path 140 | total_timesteps 1190.
Path 141 | total_timesteps 1196.
Path 142 | total_timesteps 1204.
Path 143 | total_timesteps 1212.
Path 144 | total_timesteps 1218.
Path 145 | total_timesteps 1225.
Path 146 | total_timesteps 1231.
Path 147 | total_timesteps 1239.
Path 148 | total_timesteps 1246.
Path 149 | total_timesteps 1254.
Path 150 | total_timesteps 1263.
Path 151 | total_timesteps 1271.
Path 152 | total_timesteps 1281.
Path 153 | total_timesteps 1289.
Path 154 | total_timesteps 1298.
Path 155 | total_timesteps 1305.
Path 156 | total_timesteps 1312.
Path 157 | total_timesteps 1319.
Path 158 | total_timesteps 1327.
Path 159 | total_timesteps 1336.
Path 160 | total_timesteps 1344.
Path 161 | total_timesteps 1351.
Path 162 | total_timesteps 1358.
Path 163 | total_timesteps 1370.
Path 164 | total_timesteps 1377.
Path 165 | total_timesteps 1388.
Path 166 | total_timesteps 1395.
Path 167 | total_timesteps 1404.
Path 168 | total_timesteps 1410.
Path 169 | total_timesteps 1417.
Path 170 | total_timesteps 1426.
Path 171 | total_timesteps 1432.
Path 172 | total_timesteps 1442.
Path 173 | total_timesteps 1451.
Path 174 | total_timesteps 1459.
Path 175 | total_timesteps 1466.
Path 176 | total_timesteps 1474.
Path 177 | total_timesteps 1480.
Path 178 | total_timesteps 1488.
Path 179 | total_timesteps 1499.
Path 180 | total_timesteps 1507.
Path 181 | total_timesteps 1513.
Path 182 | total_timesteps 1521.
Path 183 | total_timesteps 1529.
Path 184 | total_timesteps 1539.
Path 185 | total_timesteps 1550.
Path 186 | total_timesteps 1558.
Path 187 | total_timesteps 1568.
Path 188 | total_timesteps 1575.
Path 189 | total_timesteps 1584.
Path 190 | total_timesteps 1591.
Path 191 | total_timesteps 1598.
Path 192 | total_timesteps 1606.
Path 193 | total_timesteps 1614.
Path 194 | total_timesteps 1622.
Path 195 | total_timesteps 1629.
Path 196 | total_timesteps 1638.
Path 197 | total_timesteps 1648.
Path 198 | total_timesteps 1656.
Path 199 | total_timesteps 1677.
Path 200 | total_timesteps 1684.
Path 201 | total_timesteps 1692.
Path 202 | total_timesteps 1700.
Path 203 | total_timesteps 1708.
Path 204 | total_timesteps 1717.
Path 205 | total_timesteps 1724.
Path 206 | total_timesteps 1734.
Path 207 | total_timesteps 1742.
Path 208 | total_timesteps 1750.
Path 209 | total_timesteps 1760.
Path 210 | total_timesteps 1768.
Path 211 | total_timesteps 1777.
Path 212 | total_timesteps 1788.
Path 213 | total_timesteps 1795.
Path 214 | total_timesteps 1805.
Path 215 | total_timesteps 1814.
Path 216 | total_timesteps 1821.
Path 217 | total_timesteps 1832.
Path 218 | total_timesteps 1840.
Path 219 | total_timesteps 1847.
Path 220 | total_timesteps 1856.
Path 221 | total_timesteps 1866.
Path 222 | total_timesteps 1876.
Path 223 | total_timesteps 1882.
Path 224 | total_timesteps 1893.
Path 225 | total_timesteps 1901.
Path 226 | total_timesteps 1908.
Path 227 | total_timesteps 1915.
Path 228 | total_timesteps 1923.
Path 229 | total_timesteps 1931.
Path 230 | total_timesteps 1939.
Path 231 | total_timesteps 1948.
Path 232 | total_timesteps 1955.
Path 233 | total_timesteps 1962.
Path 234 | total_timesteps 1968.
Path 235 | total_timesteps 1978.
Path 236 | total_timesteps 1984.
Path 237 | total_timesteps 1994.
Path 238 | total_timesteps 2003.
Path 239 | total_timesteps 2011.
Path 240 | total_timesteps 2018.
Path 241 | total_timesteps 2024.
Path 242 | total_timesteps 2033.
Path 243 | total_timesteps 2040.
Path 244 | total_timesteps 2051.
Path 245 | total_timesteps 2057.
Path 246 | total_timesteps 2064.
Path 247 | total_timesteps 2073.
Path 248 | total_timesteps 2079.
Path 249 | total_timesteps 2086.
Path 250 | total_timesteps 2095.
Path 251 | total_timesteps 2102.
Path 252 | total_timesteps 2108.
Path 253 | total_timesteps 2118.
Path 254 | total_timesteps 2129.
Path 255 | total_timesteps 2138.
Path 256 | total_timesteps 2145.
Path 257 | total_timesteps 2156.
Path 258 | total_timesteps 2162.
Path 259 | total_timesteps 2168.
Path 260 | total_timesteps 2175.
Path 261 | total_timesteps 2185.
Path 262 | total_timesteps 2194.
Path 263 | total_timesteps 2201.
Path 264 | total_timesteps 2213.
Path 265 | total_timesteps 2224.
Path 266 | total_timesteps 2231.
Path 267 | total_timesteps 2239.
Path 268 | total_timesteps 2248.
Path 269 | total_timesteps 2257.
Path 270 | total_timesteps 2264.
Path 271 | total_timesteps 2275.
Path 272 | total_timesteps 2284.
Path 273 | total_timesteps 2294.
Path 274 | total_timesteps 2301.
Path 275 | total_timesteps 2308.
Path 276 | total_timesteps 2316.
Path 277 | total_timesteps 2324.
Path 278 | total_timesteps 2331.
Path 279 | total_timesteps 2338.
Path 280 | total_timesteps 2345.
Path 281 | total_timesteps 2354.
Path 282 | total_timesteps 2363.
Path 283 | total_timesteps 2370.
Path 284 | total_timesteps 2383.
Path 285 | total_timesteps 2400.
Path 286 | total_timesteps 2409.
Path 287 | total_timesteps 2415.
Path 288 | total_timesteps 2424.
Path 289 | total_timesteps 2431.
Path 290 | total_timesteps 2440.
Path 291 | total_timesteps 2449.
Path 292 | total_timesteps 2455.
Path 293 | total_timesteps 2462.
Path 294 | total_timesteps 2473.
Path 295 | total_timesteps 2479.
Path 296 | total_timesteps 2485.
Path 297 | total_timesteps 2493.
Path 298 | total_timesteps 2503.
Path 299 | total_timesteps 2510.
Path 300 | total_timesteps 2521.
Path 301 | total_timesteps 2528.
Path 302 | total_timesteps 2535.
Path 303 | total_timesteps 2543.
Path 304 | total_timesteps 2552.
Path 305 | total_timesteps 2559.
Path 306 | total_timesteps 2570.
Path 307 | total_timesteps 2576.
Path 308 | total_timesteps 2583.
Path 309 | total_timesteps 2591.
Path 310 | total_timesteps 2599.
Path 311 | total_timesteps 2607.
Path 312 | total_timesteps 2614.
Path 313 | total_timesteps 2623.
Path 314 | total_timesteps 2632.
Path 315 | total_timesteps 2646.
Path 316 | total_timesteps 2653.
Path 317 | total_timesteps 2664.
Path 318 | total_timesteps 2670.
Path 319 | total_timesteps 2676.
Path 320 | total_timesteps 2684.
Path 321 | total_timesteps 2693.
Path 322 | total_timesteps 2702.
Path 323 | total_timesteps 2712.
Path 324 | total_timesteps 2722.
Path 325 | total_timesteps 2730.
Path 326 | total_timesteps 2737.
Path 327 | total_timesteps 2744.
Path 328 | total_timesteps 2751.
Path 329 | total_timesteps 2758.
Path 330 | total_timesteps 2765.
Path 331 | total_timesteps 2774.
Path 332 | total_timesteps 2782.
Path 333 | total_timesteps 2790.
Path 334 | total_timesteps 2799.
Path 335 | total_timesteps 2806.
Path 336 | total_timesteps 2818.
Path 337 | total_timesteps 2826.
Path 338 | total_timesteps 2836.
Path 339 | total_timesteps 2846.
Path 340 | total_timesteps 2860.
Path 341 | total_timesteps 2870.
Path 342 | total_timesteps 2877.
Path 343 | total_timesteps 2887.
Path 344 | total_timesteps 2893.
Path 345 | total_timesteps 2904.
Path 346 | total_timesteps 2912.
Path 347 | total_timesteps 2919.
Path 348 | total_timesteps 2928.
Path 349 | total_timesteps 2936.
Path 350 | total_timesteps 2944.
Path 351 | total_timesteps 2952.
Path 352 | total_timesteps 2960.
Path 353 | total_timesteps 2969.
Path 354 | total_timesteps 2979.
Path 355 | total_timesteps 2994.
Path 356 | total_timesteps 3001.
Path 357 | total_timesteps 3010.
Path 358 | total_timesteps 3017.
Path 359 | total_timesteps 3026.
Path 360 | total_timesteps 3033.
Path 361 | total_timesteps 3040.
Path 362 | total_timesteps 3047.
Path 363 | total_timesteps 3056.
Path 364 | total_timesteps 3063.
Path 365 | total_timesteps 3072.
Path 366 | total_timesteps 3081.
Path 367 | total_timesteps 3094.
Path 368 | total_timesteps 3101.
Path 369 | total_timesteps 3108.
Path 370 | total_timesteps 3119.
Path 371 | total_timesteps 3126.
Path 372 | total_timesteps 3134.
Path 373 | total_timesteps 3141.
Path 374 | total_timesteps 3149.
Path 375 | total_timesteps 3155.
Path 376 | total_timesteps 3163.
Path 377 | total_timesteps 3171.
Path 378 | total_timesteps 3178.
Path 379 | total_timesteps 3186.
Path 380 | total_timesteps 3194.
Path 381 | total_timesteps 3203.
Path 382 | total_timesteps 3216.
Path 383 | total_timesteps 3226.
Path 384 | total_timesteps 3236.
Path 385 | total_timesteps 3244.
Path 386 | total_timesteps 3251.
Path 387 | total_timesteps 3259.
Path 388 | total_timesteps 3267.
Path 389 | total_timesteps 3273.
Path 390 | total_timesteps 3283.
Path 391 | total_timesteps 3290.
Path 392 | total_timesteps 3302.
Path 393 | total_timesteps 3311.
Path 394 | total_timesteps 3317.
Path 395 | total_timesteps 3328.
Path 396 | total_timesteps 3335.
Path 397 | total_timesteps 3343.
Path 398 | total_timesteps 3350.
Path 399 | total_timesteps 3356.
Path 400 | total_timesteps 3362.
Path 401 | total_timesteps 3372.
Path 402 | total_timesteps 3378.
Path 403 | total_timesteps 3387.
Path 404 | total_timesteps 3397.
Path 405 | total_timesteps 3409.
Path 406 | total_timesteps 3416.
Path 407 | total_timesteps 3423.
Path 408 | total_timesteps 3430.
Path 409 | total_timesteps 3438.
Path 410 | total_timesteps 3446.
Path 411 | total_timesteps 3454.
Path 412 | total_timesteps 3460.
Path 413 | total_timesteps 3467.
Path 414 | total_timesteps 3476.
Path 415 | total_timesteps 3485.
Path 416 | total_timesteps 3493.
Path 417 | total_timesteps 3499.
Path 418 | total_timesteps 3507.
Path 419 | total_timesteps 3514.
Path 420 | total_timesteps 3520.
Path 421 | total_timesteps 3530.
Path 422 | total_timesteps 3539.
Path 423 | total_timesteps 3546.
Path 424 | total_timesteps 3555.
Path 425 | total_timesteps 3567.
Path 426 | total_timesteps 3575.
Path 427 | total_timesteps 3582.
Path 428 | total_timesteps 3591.
Path 429 | total_timesteps 3603.
Path 430 | total_timesteps 3618.
Path 431 | total_timesteps 3626.
Path 432 | total_timesteps 3636.
Path 433 | total_timesteps 3648.
Path 434 | total_timesteps 3657.
Path 435 | total_timesteps 3666.
Path 436 | total_timesteps 3673.
Path 437 | total_timesteps 3681.
Path 438 | total_timesteps 3688.
Path 439 | total_timesteps 3698.
Path 440 | total_timesteps 3707.
Path 441 | total_timesteps 3715.
Path 442 | total_timesteps 3724.
Path 443 | total_timesteps 3733.
Path 444 | total_timesteps 3742.
Path 445 | total_timesteps 3750.
Path 446 | total_timesteps 3758.
Path 447 | total_timesteps 3765.
Path 448 | total_timesteps 3773.
Path 449 | total_timesteps 3781.
Path 450 | total_timesteps 3789.
Path 451 | total_timesteps 3796.
Path 452 | total_timesteps 3803.
Path 453 | total_timesteps 3812.
Path 454 | total_timesteps 3819.
Path 455 | total_timesteps 3826.
Path 456 | total_timesteps 3842.
Path 457 | total_timesteps 3852.
Path 458 | total_timesteps 3859.
Path 459 | total_timesteps 3872.
Path 460 | total_timesteps 3878.
Path 461 | total_timesteps 3887.
Path 462 | total_timesteps 3894.
Path 463 | total_timesteps 3901.
Path 464 | total_timesteps 3907.
Path 465 | total_timesteps 3918.
Path 466 | total_timesteps 3929.
Path 467 | total_timesteps 3939.
Path 468 | total_timesteps 3947.
Path 469 | total_timesteps 3955.
Path 470 | total_timesteps 3963.
Path 471 | total_timesteps 3970.
Path 472 | total_timesteps 3981.
Path 473 | total_timesteps 3989.
Path 474 | total_timesteps 3996.
Path 475 | total_timesteps 4006.
Path 476 | total_timesteps 4013.
Path 477 | total_timesteps 4023.
Path 478 | total_timesteps 4030.
Path 479 | total_timesteps 4037.
Path 480 | total_timesteps 4045.
Path 481 | total_timesteps 4052.
Path 482 | total_timesteps 4062.
Path 483 | total_timesteps 4073.
Path 484 | total_timesteps 4081.
Path 485 | total_timesteps 4088.
Path 486 | total_timesteps 4095.
Path 487 | total_timesteps 4103.
Path 488 | total_timesteps 4114.
Path 489 | total_timesteps 4127.
Path 490 | total_timesteps 4136.
Path 491 | total_timesteps 4146.
Path 492 | total_timesteps 4154.
Path 493 | total_timesteps 4162.
Path 494 | total_timesteps 4172.
Path 495 | total_timesteps 4179.
Path 496 | total_timesteps 4187.
Path 497 | total_timesteps 4195.
Path 498 | total_timesteps 4202.
Path 499 | total_timesteps 4214.
Path 500 | total_timesteps 4227.
Path 501 | total_timesteps 4238.
Path 502 | total_timesteps 4246.
Path 503 | total_timesteps 4254.
Path 504 | total_timesteps 4262.
Path 505 | total_timesteps 4272.
Path 506 | total_timesteps 4280.
Path 507 | total_timesteps 4289.
Path 508 | total_timesteps 4296.
Path 509 | total_timesteps 4304.
Path 510 | total_timesteps 4312.
Path 511 | total_timesteps 4319.
Path 512 | total_timesteps 4330.
Path 513 | total_timesteps 4340.
Path 514 | total_timesteps 4349.
Path 515 | total_timesteps 4355.
Path 516 | total_timesteps 4361.
Path 517 | total_timesteps 4369.
Path 518 | total_timesteps 4378.
Path 519 | total_timesteps 4386.
Path 520 | total_timesteps 4393.
Path 521 | total_timesteps 4405.
Path 522 | total_timesteps 4414.
Path 523 | total_timesteps 4422.
Path 524 | total_timesteps 4429.
Path 525 | total_timesteps 4435.
Path 526 | total_timesteps 4442.
Path 527 | total_timesteps 4450.
Path 528 | total_timesteps 4458.
Path 529 | total_timesteps 4465.
Path 530 | total_timesteps 4473.
Path 531 | total_timesteps 4481.
Path 532 | total_timesteps 4497.
Path 533 | total_timesteps 4504.
Path 534 | total_timesteps 4511.
Path 535 | total_timesteps 4518.
Path 536 | total_timesteps 4529.
Path 537 | total_timesteps 4537.
Path 538 | total_timesteps 4544.
Path 539 | total_timesteps 4551.
Path 540 | total_timesteps 4559.
Path 541 | total_timesteps 4570.
Path 542 | total_timesteps 4578.
Path 543 | total_timesteps 4587.
Path 544 | total_timesteps 4593.
Path 545 | total_timesteps 4603.
Path 546 | total_timesteps 4610.
Path 547 | total_timesteps 4620.
Path 548 | total_timesteps 4626.
Path 549 | total_timesteps 4636.
Path 550 | total_timesteps 4643.
Path 551 | total_timesteps 4651.
Path 552 | total_timesteps 4662.
Path 553 | total_timesteps 4669.
Path 554 | total_timesteps 4678.
Path 555 | total_timesteps 4685.
Path 556 | total_timesteps 4692.
Path 557 | total_timesteps 4700.
Path 558 | total_timesteps 4707.
Path 559 | total_timesteps 4715.
Path 560 | total_timesteps 4722.
Path 561 | total_timesteps 4734.
Path 562 | total_timesteps 4742.
Path 563 | total_timesteps 4752.
Path 564 | total_timesteps 4761.
Path 565 | total_timesteps 4768.
Path 566 | total_timesteps 4777.
Path 567 | total_timesteps 4788.
Path 568 | total_timesteps 4796.
Path 569 | total_timesteps 4805.
Path 570 | total_timesteps 4814.
Path 571 | total_timesteps 4824.
Path 572 | total_timesteps 4834.
Path 573 | total_timesteps 4842.
Path 574 | total_timesteps 4849.
Path 575 | total_timesteps 4858.
Path 576 | total_timesteps 4867.
Path 577 | total_timesteps 4874.
Path 578 | total_timesteps 4880.
Path 579 | total_timesteps 4887.
Path 580 | total_timesteps 4896.
Path 581 | total_timesteps 4904.
Path 582 | total_timesteps 4912.
Path 583 | total_timesteps 4919.
Path 584 | total_timesteps 4928.
Path 585 | total_timesteps 4935.
Path 586 | total_timesteps 4944.
Path 587 | total_timesteps 4953.
Path 588 | total_timesteps 4964.
Path 589 | total_timesteps 4971.
Path 590 | total_timesteps 4978.
Path 591 | total_timesteps 4985.
Path 592 | total_timesteps 4992.
Path 593 | total_timesteps 5000.
Path 594 | total_timesteps 5009.
Path 595 | total_timesteps 5016.
Path 596 | total_timesteps 5023.
Path 597 | total_timesteps 5031.
Path 598 | total_timesteps 5038.
Path 599 | total_timesteps 5050.
Path 600 | total_timesteps 5058.
Path 601 | total_timesteps 5067.
Path 602 | total_timesteps 5076.
Path 603 | total_timesteps 5082.
Path 604 | total_timesteps 5089.
Path 605 | total_timesteps 5096.
Path 606 | total_timesteps 5105.
Path 607 | total_timesteps 5111.
Path 608 | total_timesteps 5119.
Path 609 | total_timesteps 5134.
Path 610 | total_timesteps 5142.
Path 611 | total_timesteps 5148.
Path 612 | total_timesteps 5158.
Path 613 | total_timesteps 5167.
Path 614 | total_timesteps 5175.
Path 615 | total_timesteps 5183.
Path 616 | total_timesteps 5192.
Path 617 | total_timesteps 5203.
Path 618 | total_timesteps 5212.
Path 619 | total_timesteps 5219.
Path 620 | total_timesteps 5234.
Path 621 | total_timesteps 5241.
Path 622 | total_timesteps 5248.
Path 623 | total_timesteps 5255.
Path 624 | total_timesteps 5265.
Path 625 | total_timesteps 5274.
Path 626 | total_timesteps 5281.
Path 627 | total_timesteps 5289.
Path 628 | total_timesteps 5297.
Path 629 | total_timesteps 5305.
Path 630 | total_timesteps 5313.
Path 631 | total_timesteps 5329.
Path 632 | total_timesteps 5336.
Path 633 | total_timesteps 5345.
Path 634 | total_timesteps 5353.
Path 635 | total_timesteps 5363.
Path 636 | total_timesteps 5374.
Path 637 | total_timesteps 5382.
Path 638 | total_timesteps 5388.
Path 639 | total_timesteps 5395.
Path 640 | total_timesteps 5408.
Path 641 | total_timesteps 5416.
Path 642 | total_timesteps 5423.
Path 643 | total_timesteps 5431.
Path 644 | total_timesteps 5439.
Path 645 | total_timesteps 5446.
Path 646 | total_timesteps 5454.
Path 647 | total_timesteps 5460.
Path 648 | total_timesteps 5468.
Path 649 | total_timesteps 5476.
Path 650 | total_timesteps 5488.
Path 651 | total_timesteps 5494.
Path 652 | total_timesteps 5507.
Path 653 | total_timesteps 5517.
Path 654 | total_timesteps 5524.
Path 655 | total_timesteps 5533.
Path 656 | total_timesteps 5541.
Path 657 | total_timesteps 5549.
Path 658 | total_timesteps 5557.
Path 659 | total_timesteps 5565.
Path 660 | total_timesteps 5574.
Path 661 | total_timesteps 5580.
Path 662 | total_timesteps 5588.
Path 663 | total_timesteps 5599.
Path 664 | total_timesteps 5606.
Path 665 | total_timesteps 5614.
Path 666 | total_timesteps 5622.
Path 667 | total_timesteps 5630.
Path 668 | total_timesteps 5638.
Path 669 | total_timesteps 5645.
Path 670 | total_timesteps 5655.
Path 671 | total_timesteps 5662.
Path 672 | total_timesteps 5672.
Path 673 | total_timesteps 5679.
Path 674 | total_timesteps 5688.
Path 675 | total_timesteps 5699.
Path 676 | total_timesteps 5707.
Path 677 | total_timesteps 5714.
Path 678 | total_timesteps 5721.
Path 679 | total_timesteps 5728.
Path 680 | total_timesteps 5735.
Path 681 | total_timesteps 5742.
Path 682 | total_timesteps 5749.
Path 683 | total_timesteps 5760.
Path 684 | total_timesteps 5774.
Path 685 | total_timesteps 5788.
Path 686 | total_timesteps 5796.
Path 687 | total_timesteps 5804.
Path 688 | total_timesteps 5811.
Path 689 | total_timesteps 5817.
Path 690 | total_timesteps 5827.
Path 691 | total_timesteps 5834.
Path 692 | total_timesteps 5840.
Path 693 | total_timesteps 5848.
Path 694 | total_timesteps 5855.
Path 695 | total_timesteps 5861.
Path 696 | total_timesteps 5870.
Path 697 | total_timesteps 5878.
Path 698 | total_timesteps 5884.
Path 699 | total_timesteps 5894.
Path 700 | total_timesteps 5904.
Path 701 | total_timesteps 5915.
Path 702 | total_timesteps 5922.
Path 703 | total_timesteps 5928.
Path 704 | total_timesteps 5935.
Path 705 | total_timesteps 5942.
Path 706 | total_timesteps 5949.
Path 707 | total_timesteps 5955.
Path 708 | total_timesteps 5963.
Path 709 | total_timesteps 5972.
Path 710 | total_timesteps 5979.
Path 711 | total_timesteps 5991.
Path 712 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.68    |
| Iteration     | 16       |
| MaximumReturn | -0.549   |
| MinimumReturn | -18.4    |
| TotalSamples  | 72067    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.0026266484055668116
Validation loss = 0.002698668744415045
Validation loss = 0.002560428576543927
Validation loss = 0.0026400459464639425
Validation loss = 0.002508479868993163
Validation loss = 0.0023564593866467476
Validation loss = 0.0029627836775034666
Validation loss = 0.00252386718057096
Validation loss = 0.00253665610216558
Validation loss = 0.0028192379977554083
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 44.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 60.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 79.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 95.
Path 12 | total_timesteps 104.
Path 13 | total_timesteps 113.
Path 14 | total_timesteps 123.
Path 15 | total_timesteps 131.
Path 16 | total_timesteps 140.
Path 17 | total_timesteps 148.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 173.
Path 20 | total_timesteps 184.
Path 21 | total_timesteps 192.
Path 22 | total_timesteps 201.
Path 23 | total_timesteps 208.
Path 24 | total_timesteps 215.
Path 25 | total_timesteps 222.
Path 26 | total_timesteps 232.
Path 27 | total_timesteps 244.
Path 28 | total_timesteps 251.
Path 29 | total_timesteps 262.
Path 30 | total_timesteps 270.
Path 31 | total_timesteps 282.
Path 32 | total_timesteps 289.
Path 33 | total_timesteps 300.
Path 34 | total_timesteps 309.
Path 35 | total_timesteps 318.
Path 36 | total_timesteps 326.
Path 37 | total_timesteps 334.
Path 38 | total_timesteps 343.
Path 39 | total_timesteps 350.
Path 40 | total_timesteps 358.
Path 41 | total_timesteps 366.
Path 42 | total_timesteps 373.
Path 43 | total_timesteps 383.
Path 44 | total_timesteps 392.
Path 45 | total_timesteps 402.
Path 46 | total_timesteps 412.
Path 47 | total_timesteps 419.
Path 48 | total_timesteps 425.
Path 49 | total_timesteps 433.
Path 50 | total_timesteps 441.
Path 51 | total_timesteps 448.
Path 52 | total_timesteps 456.
Path 53 | total_timesteps 463.
Path 54 | total_timesteps 476.
Path 55 | total_timesteps 485.
Path 56 | total_timesteps 492.
Path 57 | total_timesteps 499.
Path 58 | total_timesteps 506.
Path 59 | total_timesteps 518.
Path 60 | total_timesteps 525.
Path 61 | total_timesteps 535.
Path 62 | total_timesteps 542.
Path 63 | total_timesteps 550.
Path 64 | total_timesteps 562.
Path 65 | total_timesteps 569.
Path 66 | total_timesteps 576.
Path 67 | total_timesteps 584.
Path 68 | total_timesteps 593.
Path 69 | total_timesteps 599.
Path 70 | total_timesteps 610.
Path 71 | total_timesteps 617.
Path 72 | total_timesteps 626.
Path 73 | total_timesteps 637.
Path 74 | total_timesteps 645.
Path 75 | total_timesteps 656.
Path 76 | total_timesteps 668.
Path 77 | total_timesteps 679.
Path 78 | total_timesteps 686.
Path 79 | total_timesteps 693.
Path 80 | total_timesteps 700.
Path 81 | total_timesteps 711.
Path 82 | total_timesteps 717.
Path 83 | total_timesteps 724.
Path 84 | total_timesteps 735.
Path 85 | total_timesteps 744.
Path 86 | total_timesteps 751.
Path 87 | total_timesteps 759.
Path 88 | total_timesteps 770.
Path 89 | total_timesteps 778.
Path 90 | total_timesteps 786.
Path 91 | total_timesteps 796.
Path 92 | total_timesteps 809.
Path 93 | total_timesteps 818.
Path 94 | total_timesteps 826.
Path 95 | total_timesteps 835.
Path 96 | total_timesteps 847.
Path 97 | total_timesteps 857.
Path 98 | total_timesteps 865.
Path 99 | total_timesteps 875.
Path 100 | total_timesteps 883.
Path 101 | total_timesteps 895.
Path 102 | total_timesteps 904.
Path 103 | total_timesteps 912.
Path 104 | total_timesteps 922.
Path 105 | total_timesteps 931.
Path 106 | total_timesteps 939.
Path 107 | total_timesteps 946.
Path 108 | total_timesteps 956.
Path 109 | total_timesteps 964.
Path 110 | total_timesteps 972.
Path 111 | total_timesteps 982.
Path 112 | total_timesteps 994.
Path 113 | total_timesteps 1002.
Path 114 | total_timesteps 1009.
Path 115 | total_timesteps 1017.
Path 116 | total_timesteps 1023.
Path 117 | total_timesteps 1031.
Path 118 | total_timesteps 1041.
Path 119 | total_timesteps 1049.
Path 120 | total_timesteps 1060.
Path 121 | total_timesteps 1068.
Path 122 | total_timesteps 1090.
Path 123 | total_timesteps 1098.
Path 124 | total_timesteps 1105.
Path 125 | total_timesteps 1115.
Path 126 | total_timesteps 1122.
Path 127 | total_timesteps 1133.
Path 128 | total_timesteps 1150.
Path 129 | total_timesteps 1157.
Path 130 | total_timesteps 1165.
Path 131 | total_timesteps 1171.
Path 132 | total_timesteps 1180.
Path 133 | total_timesteps 1187.
Path 134 | total_timesteps 1196.
Path 135 | total_timesteps 1204.
Path 136 | total_timesteps 1212.
Path 137 | total_timesteps 1221.
Path 138 | total_timesteps 1230.
Path 139 | total_timesteps 1237.
Path 140 | total_timesteps 1245.
Path 141 | total_timesteps 1260.
Path 142 | total_timesteps 1269.
Path 143 | total_timesteps 1281.
Path 144 | total_timesteps 1293.
Path 145 | total_timesteps 1303.
Path 146 | total_timesteps 1309.
Path 147 | total_timesteps 1316.
Path 148 | total_timesteps 1329.
Path 149 | total_timesteps 1338.
Path 150 | total_timesteps 1349.
Path 151 | total_timesteps 1356.
Path 152 | total_timesteps 1366.
Path 153 | total_timesteps 1376.
Path 154 | total_timesteps 1383.
Path 155 | total_timesteps 1392.
Path 156 | total_timesteps 1402.
Path 157 | total_timesteps 1409.
Path 158 | total_timesteps 1416.
Path 159 | total_timesteps 1425.
Path 160 | total_timesteps 1436.
Path 161 | total_timesteps 1445.
Path 162 | total_timesteps 1454.
Path 163 | total_timesteps 1462.
Path 164 | total_timesteps 1469.
Path 165 | total_timesteps 1477.
Path 166 | total_timesteps 1486.
Path 167 | total_timesteps 1495.
Path 168 | total_timesteps 1503.
Path 169 | total_timesteps 1510.
Path 170 | total_timesteps 1519.
Path 171 | total_timesteps 1526.
Path 172 | total_timesteps 1534.
Path 173 | total_timesteps 1542.
Path 174 | total_timesteps 1550.
Path 175 | total_timesteps 1559.
Path 176 | total_timesteps 1568.
Path 177 | total_timesteps 1576.
Path 178 | total_timesteps 1582.
Path 179 | total_timesteps 1591.
Path 180 | total_timesteps 1600.
Path 181 | total_timesteps 1607.
Path 182 | total_timesteps 1617.
Path 183 | total_timesteps 1626.
Path 184 | total_timesteps 1634.
Path 185 | total_timesteps 1643.
Path 186 | total_timesteps 1652.
Path 187 | total_timesteps 1659.
Path 188 | total_timesteps 1666.
Path 189 | total_timesteps 1673.
Path 190 | total_timesteps 1681.
Path 191 | total_timesteps 1690.
Path 192 | total_timesteps 1699.
Path 193 | total_timesteps 1706.
Path 194 | total_timesteps 1714.
Path 195 | total_timesteps 1721.
Path 196 | total_timesteps 1735.
Path 197 | total_timesteps 1745.
Path 198 | total_timesteps 1755.
Path 199 | total_timesteps 1767.
Path 200 | total_timesteps 1778.
Path 201 | total_timesteps 1785.
Path 202 | total_timesteps 1796.
Path 203 | total_timesteps 1805.
Path 204 | total_timesteps 1812.
Path 205 | total_timesteps 1826.
Path 206 | total_timesteps 1834.
Path 207 | total_timesteps 1844.
Path 208 | total_timesteps 1854.
Path 209 | total_timesteps 1865.
Path 210 | total_timesteps 1872.
Path 211 | total_timesteps 1880.
Path 212 | total_timesteps 1891.
Path 213 | total_timesteps 1899.
Path 214 | total_timesteps 1912.
Path 215 | total_timesteps 1919.
Path 216 | total_timesteps 1927.
Path 217 | total_timesteps 1939.
Path 218 | total_timesteps 1953.
Path 219 | total_timesteps 1962.
Path 220 | total_timesteps 1969.
Path 221 | total_timesteps 1979.
Path 222 | total_timesteps 1987.
Path 223 | total_timesteps 1999.
Path 224 | total_timesteps 2005.
Path 225 | total_timesteps 2014.
Path 226 | total_timesteps 2022.
Path 227 | total_timesteps 2029.
Path 228 | total_timesteps 2036.
Path 229 | total_timesteps 2043.
Path 230 | total_timesteps 2051.
Path 231 | total_timesteps 2060.
Path 232 | total_timesteps 2072.
Path 233 | total_timesteps 2079.
Path 234 | total_timesteps 2088.
Path 235 | total_timesteps 2096.
Path 236 | total_timesteps 2106.
Path 237 | total_timesteps 2113.
Path 238 | total_timesteps 2122.
Path 239 | total_timesteps 2133.
Path 240 | total_timesteps 2143.
Path 241 | total_timesteps 2152.
Path 242 | total_timesteps 2160.
Path 243 | total_timesteps 2168.
Path 244 | total_timesteps 2175.
Path 245 | total_timesteps 2183.
Path 246 | total_timesteps 2191.
Path 247 | total_timesteps 2201.
Path 248 | total_timesteps 2210.
Path 249 | total_timesteps 2216.
Path 250 | total_timesteps 2230.
Path 251 | total_timesteps 2237.
Path 252 | total_timesteps 2246.
Path 253 | total_timesteps 2256.
Path 254 | total_timesteps 2263.
Path 255 | total_timesteps 2271.
Path 256 | total_timesteps 2278.
Path 257 | total_timesteps 2290.
Path 258 | total_timesteps 2297.
Path 259 | total_timesteps 2308.
Path 260 | total_timesteps 2315.
Path 261 | total_timesteps 2323.
Path 262 | total_timesteps 2334.
Path 263 | total_timesteps 2342.
Path 264 | total_timesteps 2349.
Path 265 | total_timesteps 2361.
Path 266 | total_timesteps 2368.
Path 267 | total_timesteps 2379.
Path 268 | total_timesteps 2388.
Path 269 | total_timesteps 2397.
Path 270 | total_timesteps 2405.
Path 271 | total_timesteps 2414.
Path 272 | total_timesteps 2424.
Path 273 | total_timesteps 2430.
Path 274 | total_timesteps 2445.
Path 275 | total_timesteps 2453.
Path 276 | total_timesteps 2462.
Path 277 | total_timesteps 2472.
Path 278 | total_timesteps 2485.
Path 279 | total_timesteps 2495.
Path 280 | total_timesteps 2503.
Path 281 | total_timesteps 2510.
Path 282 | total_timesteps 2518.
Path 283 | total_timesteps 2526.
Path 284 | total_timesteps 2537.
Path 285 | total_timesteps 2544.
Path 286 | total_timesteps 2551.
Path 287 | total_timesteps 2565.
Path 288 | total_timesteps 2573.
Path 289 | total_timesteps 2581.
Path 290 | total_timesteps 2592.
Path 291 | total_timesteps 2601.
Path 292 | total_timesteps 2609.
Path 293 | total_timesteps 2617.
Path 294 | total_timesteps 2625.
Path 295 | total_timesteps 2636.
Path 296 | total_timesteps 2645.
Path 297 | total_timesteps 2659.
Path 298 | total_timesteps 2668.
Path 299 | total_timesteps 2676.
Path 300 | total_timesteps 2683.
Path 301 | total_timesteps 2693.
Path 302 | total_timesteps 2702.
Path 303 | total_timesteps 2710.
Path 304 | total_timesteps 2721.
Path 305 | total_timesteps 2731.
Path 306 | total_timesteps 2743.
Path 307 | total_timesteps 2749.
Path 308 | total_timesteps 2762.
Path 309 | total_timesteps 2770.
Path 310 | total_timesteps 2777.
Path 311 | total_timesteps 2786.
Path 312 | total_timesteps 2794.
Path 313 | total_timesteps 2802.
Path 314 | total_timesteps 2810.
Path 315 | total_timesteps 2817.
Path 316 | total_timesteps 2824.
Path 317 | total_timesteps 2832.
Path 318 | total_timesteps 2840.
Path 319 | total_timesteps 2847.
Path 320 | total_timesteps 2854.
Path 321 | total_timesteps 2862.
Path 322 | total_timesteps 2872.
Path 323 | total_timesteps 2881.
Path 324 | total_timesteps 2888.
Path 325 | total_timesteps 2895.
Path 326 | total_timesteps 2905.
Path 327 | total_timesteps 2913.
Path 328 | total_timesteps 2920.
Path 329 | total_timesteps 2930.
Path 330 | total_timesteps 2940.
Path 331 | total_timesteps 2949.
Path 332 | total_timesteps 2960.
Path 333 | total_timesteps 2969.
Path 334 | total_timesteps 2977.
Path 335 | total_timesteps 2987.
Path 336 | total_timesteps 2996.
Path 337 | total_timesteps 3007.
Path 338 | total_timesteps 3015.
Path 339 | total_timesteps 3022.
Path 340 | total_timesteps 3033.
Path 341 | total_timesteps 3039.
Path 342 | total_timesteps 3047.
Path 343 | total_timesteps 3055.
Path 344 | total_timesteps 3064.
Path 345 | total_timesteps 3075.
Path 346 | total_timesteps 3084.
Path 347 | total_timesteps 3093.
Path 348 | total_timesteps 3101.
Path 349 | total_timesteps 3113.
Path 350 | total_timesteps 3122.
Path 351 | total_timesteps 3132.
Path 352 | total_timesteps 3139.
Path 353 | total_timesteps 3149.
Path 354 | total_timesteps 3155.
Path 355 | total_timesteps 3165.
Path 356 | total_timesteps 3176.
Path 357 | total_timesteps 3184.
Path 358 | total_timesteps 3191.
Path 359 | total_timesteps 3199.
Path 360 | total_timesteps 3210.
Path 361 | total_timesteps 3219.
Path 362 | total_timesteps 3229.
Path 363 | total_timesteps 3240.
Path 364 | total_timesteps 3248.
Path 365 | total_timesteps 3255.
Path 366 | total_timesteps 3264.
Path 367 | total_timesteps 3273.
Path 368 | total_timesteps 3281.
Path 369 | total_timesteps 3287.
Path 370 | total_timesteps 3298.
Path 371 | total_timesteps 3305.
Path 372 | total_timesteps 3315.
Path 373 | total_timesteps 3328.
Path 374 | total_timesteps 3337.
Path 375 | total_timesteps 3353.
Path 376 | total_timesteps 3361.
Path 377 | total_timesteps 3371.
Path 378 | total_timesteps 3380.
Path 379 | total_timesteps 3389.
Path 380 | total_timesteps 3401.
Path 381 | total_timesteps 3408.
Path 382 | total_timesteps 3420.
Path 383 | total_timesteps 3427.
Path 384 | total_timesteps 3435.
Path 385 | total_timesteps 3446.
Path 386 | total_timesteps 3455.
Path 387 | total_timesteps 3462.
Path 388 | total_timesteps 3469.
Path 389 | total_timesteps 3479.
Path 390 | total_timesteps 3491.
Path 391 | total_timesteps 3500.
Path 392 | total_timesteps 3507.
Path 393 | total_timesteps 3516.
Path 394 | total_timesteps 3524.
Path 395 | total_timesteps 3538.
Path 396 | total_timesteps 3547.
Path 397 | total_timesteps 3557.
Path 398 | total_timesteps 3565.
Path 399 | total_timesteps 3572.
Path 400 | total_timesteps 3579.
Path 401 | total_timesteps 3590.
Path 402 | total_timesteps 3601.
Path 403 | total_timesteps 3607.
Path 404 | total_timesteps 3615.
Path 405 | total_timesteps 3621.
Path 406 | total_timesteps 3630.
Path 407 | total_timesteps 3640.
Path 408 | total_timesteps 3651.
Path 409 | total_timesteps 3659.
Path 410 | total_timesteps 3669.
Path 411 | total_timesteps 3677.
Path 412 | total_timesteps 3687.
Path 413 | total_timesteps 3696.
Path 414 | total_timesteps 3705.
Path 415 | total_timesteps 3711.
Path 416 | total_timesteps 3717.
Path 417 | total_timesteps 3724.
Path 418 | total_timesteps 3736.
Path 419 | total_timesteps 3744.
Path 420 | total_timesteps 3752.
Path 421 | total_timesteps 3763.
Path 422 | total_timesteps 3772.
Path 423 | total_timesteps 3779.
Path 424 | total_timesteps 3787.
Path 425 | total_timesteps 3796.
Path 426 | total_timesteps 3808.
Path 427 | total_timesteps 3817.
Path 428 | total_timesteps 3824.
Path 429 | total_timesteps 3832.
Path 430 | total_timesteps 3843.
Path 431 | total_timesteps 3852.
Path 432 | total_timesteps 3858.
Path 433 | total_timesteps 3867.
Path 434 | total_timesteps 3874.
Path 435 | total_timesteps 3884.
Path 436 | total_timesteps 3892.
Path 437 | total_timesteps 3900.
Path 438 | total_timesteps 3909.
Path 439 | total_timesteps 3918.
Path 440 | total_timesteps 3930.
Path 441 | total_timesteps 3940.
Path 442 | total_timesteps 3947.
Path 443 | total_timesteps 3958.
Path 444 | total_timesteps 3965.
Path 445 | total_timesteps 3973.
Path 446 | total_timesteps 3981.
Path 447 | total_timesteps 3988.
Path 448 | total_timesteps 3998.
Path 449 | total_timesteps 4008.
Path 450 | total_timesteps 4017.
Path 451 | total_timesteps 4024.
Path 452 | total_timesteps 4037.
Path 453 | total_timesteps 4044.
Path 454 | total_timesteps 4061.
Path 455 | total_timesteps 4071.
Path 456 | total_timesteps 4083.
Path 457 | total_timesteps 4091.
Path 458 | total_timesteps 4098.
Path 459 | total_timesteps 4104.
Path 460 | total_timesteps 4112.
Path 461 | total_timesteps 4120.
Path 462 | total_timesteps 4128.
Path 463 | total_timesteps 4136.
Path 464 | total_timesteps 4144.
Path 465 | total_timesteps 4153.
Path 466 | total_timesteps 4161.
Path 467 | total_timesteps 4168.
Path 468 | total_timesteps 4176.
Path 469 | total_timesteps 4184.
Path 470 | total_timesteps 4201.
Path 471 | total_timesteps 4210.
Path 472 | total_timesteps 4218.
Path 473 | total_timesteps 4225.
Path 474 | total_timesteps 4234.
Path 475 | total_timesteps 4243.
Path 476 | total_timesteps 4250.
Path 477 | total_timesteps 4256.
Path 478 | total_timesteps 4266.
Path 479 | total_timesteps 4275.
Path 480 | total_timesteps 4284.
Path 481 | total_timesteps 4296.
Path 482 | total_timesteps 4305.
Path 483 | total_timesteps 4314.
Path 484 | total_timesteps 4322.
Path 485 | total_timesteps 4336.
Path 486 | total_timesteps 4344.
Path 487 | total_timesteps 4354.
Path 488 | total_timesteps 4361.
Path 489 | total_timesteps 4369.
Path 490 | total_timesteps 4376.
Path 491 | total_timesteps 4384.
Path 492 | total_timesteps 4392.
Path 493 | total_timesteps 4399.
Path 494 | total_timesteps 4408.
Path 495 | total_timesteps 4414.
Path 496 | total_timesteps 4423.
Path 497 | total_timesteps 4431.
Path 498 | total_timesteps 4438.
Path 499 | total_timesteps 4447.
Path 500 | total_timesteps 4459.
Path 501 | total_timesteps 4466.
Path 502 | total_timesteps 4474.
Path 503 | total_timesteps 4488.
Path 504 | total_timesteps 4497.
Path 505 | total_timesteps 4504.
Path 506 | total_timesteps 4523.
Path 507 | total_timesteps 4531.
Path 508 | total_timesteps 4543.
Path 509 | total_timesteps 4556.
Path 510 | total_timesteps 4567.
Path 511 | total_timesteps 4574.
Path 512 | total_timesteps 4584.
Path 513 | total_timesteps 4591.
Path 514 | total_timesteps 4612.
Path 515 | total_timesteps 4621.
Path 516 | total_timesteps 4630.
Path 517 | total_timesteps 4642.
Path 518 | total_timesteps 4653.
Path 519 | total_timesteps 4662.
Path 520 | total_timesteps 4669.
Path 521 | total_timesteps 4678.
Path 522 | total_timesteps 4689.
Path 523 | total_timesteps 4697.
Path 524 | total_timesteps 4704.
Path 525 | total_timesteps 4713.
Path 526 | total_timesteps 4722.
Path 527 | total_timesteps 4732.
Path 528 | total_timesteps 4739.
Path 529 | total_timesteps 4749.
Path 530 | total_timesteps 4764.
Path 531 | total_timesteps 4774.
Path 532 | total_timesteps 4782.
Path 533 | total_timesteps 4790.
Path 534 | total_timesteps 4798.
Path 535 | total_timesteps 4810.
Path 536 | total_timesteps 4819.
Path 537 | total_timesteps 4831.
Path 538 | total_timesteps 4840.
Path 539 | total_timesteps 4849.
Path 540 | total_timesteps 4864.
Path 541 | total_timesteps 4874.
Path 542 | total_timesteps 4882.
Path 543 | total_timesteps 4889.
Path 544 | total_timesteps 4897.
Path 545 | total_timesteps 4908.
Path 546 | total_timesteps 4915.
Path 547 | total_timesteps 4923.
Path 548 | total_timesteps 4944.
Path 549 | total_timesteps 4953.
Path 550 | total_timesteps 4960.
Path 551 | total_timesteps 4968.
Path 552 | total_timesteps 4976.
Path 553 | total_timesteps 4983.
Path 554 | total_timesteps 4994.
Path 555 | total_timesteps 5002.
Path 556 | total_timesteps 5013.
Path 557 | total_timesteps 5022.
Path 558 | total_timesteps 5031.
Path 559 | total_timesteps 5039.
Path 560 | total_timesteps 5049.
Path 561 | total_timesteps 5057.
Path 562 | total_timesteps 5069.
Path 563 | total_timesteps 5081.
Path 564 | total_timesteps 5087.
Path 565 | total_timesteps 5095.
Path 566 | total_timesteps 5102.
Path 567 | total_timesteps 5113.
Path 568 | total_timesteps 5121.
Path 569 | total_timesteps 5129.
Path 570 | total_timesteps 5137.
Path 571 | total_timesteps 5145.
Path 572 | total_timesteps 5154.
Path 573 | total_timesteps 5166.
Path 574 | total_timesteps 5177.
Path 575 | total_timesteps 5187.
Path 576 | total_timesteps 5194.
Path 577 | total_timesteps 5205.
Path 578 | total_timesteps 5213.
Path 579 | total_timesteps 5226.
Path 580 | total_timesteps 5235.
Path 581 | total_timesteps 5242.
Path 582 | total_timesteps 5250.
Path 583 | total_timesteps 5261.
Path 584 | total_timesteps 5270.
Path 585 | total_timesteps 5279.
Path 586 | total_timesteps 5288.
Path 587 | total_timesteps 5294.
Path 588 | total_timesteps 5305.
Path 589 | total_timesteps 5316.
Path 590 | total_timesteps 5325.
Path 591 | total_timesteps 5332.
Path 592 | total_timesteps 5339.
Path 593 | total_timesteps 5351.
Path 594 | total_timesteps 5363.
Path 595 | total_timesteps 5374.
Path 596 | total_timesteps 5383.
Path 597 | total_timesteps 5395.
Path 598 | total_timesteps 5402.
Path 599 | total_timesteps 5412.
Path 600 | total_timesteps 5419.
Path 601 | total_timesteps 5436.
Path 602 | total_timesteps 5444.
Path 603 | total_timesteps 5457.
Path 604 | total_timesteps 5467.
Path 605 | total_timesteps 5479.
Path 606 | total_timesteps 5487.
Path 607 | total_timesteps 5496.
Path 608 | total_timesteps 5504.
Path 609 | total_timesteps 5512.
Path 610 | total_timesteps 5523.
Path 611 | total_timesteps 5532.
Path 612 | total_timesteps 5541.
Path 613 | total_timesteps 5548.
Path 614 | total_timesteps 5556.
Path 615 | total_timesteps 5564.
Path 616 | total_timesteps 5571.
Path 617 | total_timesteps 5580.
Path 618 | total_timesteps 5586.
Path 619 | total_timesteps 5593.
Path 620 | total_timesteps 5599.
Path 621 | total_timesteps 5609.
Path 622 | total_timesteps 5618.
Path 623 | total_timesteps 5626.
Path 624 | total_timesteps 5635.
Path 625 | total_timesteps 5643.
Path 626 | total_timesteps 5650.
Path 627 | total_timesteps 5658.
Path 628 | total_timesteps 5667.
Path 629 | total_timesteps 5674.
Path 630 | total_timesteps 5680.
Path 631 | total_timesteps 5688.
Path 632 | total_timesteps 5698.
Path 633 | total_timesteps 5710.
Path 634 | total_timesteps 5717.
Path 635 | total_timesteps 5724.
Path 636 | total_timesteps 5732.
Path 637 | total_timesteps 5746.
Path 638 | total_timesteps 5753.
Path 639 | total_timesteps 5760.
Path 640 | total_timesteps 5771.
Path 641 | total_timesteps 5778.
Path 642 | total_timesteps 5785.
Path 643 | total_timesteps 5792.
Path 644 | total_timesteps 5802.
Path 645 | total_timesteps 5809.
Path 646 | total_timesteps 5816.
Path 647 | total_timesteps 5823.
Path 648 | total_timesteps 5831.
Path 649 | total_timesteps 5842.
Path 650 | total_timesteps 5851.
Path 651 | total_timesteps 5862.
Path 652 | total_timesteps 5871.
Path 653 | total_timesteps 5878.
Path 654 | total_timesteps 5894.
Path 655 | total_timesteps 5906.
Path 656 | total_timesteps 5913.
Path 657 | total_timesteps 5922.
Path 658 | total_timesteps 5931.
Path 659 | total_timesteps 5939.
Path 660 | total_timesteps 5953.
Path 661 | total_timesteps 5961.
Path 662 | total_timesteps 5970.
Path 663 | total_timesteps 5980.
Path 664 | total_timesteps 5986.
Path 665 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.99    |
| Iteration     | 17       |
| MaximumReturn | 0.969    |
| MinimumReturn | -17.6    |
| TotalSamples  | 76069    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.0025804557371884584
Validation loss = 0.0024589020758867264
Validation loss = 0.0031003050971776247
Validation loss = 0.002375214360654354
Validation loss = 0.0023896535858511925
Validation loss = 0.0026873161550611258
Validation loss = 0.002461821772158146
Validation loss = 0.0023913190234452486
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 51.
Path 7 | total_timesteps 59.
Path 8 | total_timesteps 66.
Path 9 | total_timesteps 78.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 99.
Path 12 | total_timesteps 106.
Path 13 | total_timesteps 115.
Path 14 | total_timesteps 134.
Path 15 | total_timesteps 142.
Path 16 | total_timesteps 150.
Path 17 | total_timesteps 163.
Path 18 | total_timesteps 170.
Path 19 | total_timesteps 179.
Path 20 | total_timesteps 187.
Path 21 | total_timesteps 194.
Path 22 | total_timesteps 205.
Path 23 | total_timesteps 215.
Path 24 | total_timesteps 225.
Path 25 | total_timesteps 235.
Path 26 | total_timesteps 243.
Path 27 | total_timesteps 252.
Path 28 | total_timesteps 261.
Path 29 | total_timesteps 268.
Path 30 | total_timesteps 279.
Path 31 | total_timesteps 289.
Path 32 | total_timesteps 296.
Path 33 | total_timesteps 302.
Path 34 | total_timesteps 310.
Path 35 | total_timesteps 316.
Path 36 | total_timesteps 325.
Path 37 | total_timesteps 335.
Path 38 | total_timesteps 343.
Path 39 | total_timesteps 354.
Path 40 | total_timesteps 367.
Path 41 | total_timesteps 374.
Path 42 | total_timesteps 382.
Path 43 | total_timesteps 396.
Path 44 | total_timesteps 404.
Path 45 | total_timesteps 413.
Path 46 | total_timesteps 421.
Path 47 | total_timesteps 435.
Path 48 | total_timesteps 442.
Path 49 | total_timesteps 452.
Path 50 | total_timesteps 461.
Path 51 | total_timesteps 468.
Path 52 | total_timesteps 475.
Path 53 | total_timesteps 482.
Path 54 | total_timesteps 494.
Path 55 | total_timesteps 501.
Path 56 | total_timesteps 512.
Path 57 | total_timesteps 523.
Path 58 | total_timesteps 535.
Path 59 | total_timesteps 544.
Path 60 | total_timesteps 552.
Path 61 | total_timesteps 559.
Path 62 | total_timesteps 565.
Path 63 | total_timesteps 576.
Path 64 | total_timesteps 587.
Path 65 | total_timesteps 595.
Path 66 | total_timesteps 604.
Path 67 | total_timesteps 613.
Path 68 | total_timesteps 623.
Path 69 | total_timesteps 631.
Path 70 | total_timesteps 638.
Path 71 | total_timesteps 646.
Path 72 | total_timesteps 652.
Path 73 | total_timesteps 663.
Path 74 | total_timesteps 673.
Path 75 | total_timesteps 682.
Path 76 | total_timesteps 689.
Path 77 | total_timesteps 700.
Path 78 | total_timesteps 709.
Path 79 | total_timesteps 719.
Path 80 | total_timesteps 726.
Path 81 | total_timesteps 732.
Path 82 | total_timesteps 742.
Path 83 | total_timesteps 750.
Path 84 | total_timesteps 757.
Path 85 | total_timesteps 766.
Path 86 | total_timesteps 775.
Path 87 | total_timesteps 782.
Path 88 | total_timesteps 789.
Path 89 | total_timesteps 798.
Path 90 | total_timesteps 808.
Path 91 | total_timesteps 816.
Path 92 | total_timesteps 825.
Path 93 | total_timesteps 832.
Path 94 | total_timesteps 840.
Path 95 | total_timesteps 852.
Path 96 | total_timesteps 861.
Path 97 | total_timesteps 869.
Path 98 | total_timesteps 876.
Path 99 | total_timesteps 885.
Path 100 | total_timesteps 901.
Path 101 | total_timesteps 912.
Path 102 | total_timesteps 924.
Path 103 | total_timesteps 931.
Path 104 | total_timesteps 939.
Path 105 | total_timesteps 948.
Path 106 | total_timesteps 956.
Path 107 | total_timesteps 964.
Path 108 | total_timesteps 973.
Path 109 | total_timesteps 984.
Path 110 | total_timesteps 992.
Path 111 | total_timesteps 998.
Path 112 | total_timesteps 1007.
Path 113 | total_timesteps 1016.
Path 114 | total_timesteps 1025.
Path 115 | total_timesteps 1032.
Path 116 | total_timesteps 1043.
Path 117 | total_timesteps 1052.
Path 118 | total_timesteps 1063.
Path 119 | total_timesteps 1072.
Path 120 | total_timesteps 1079.
Path 121 | total_timesteps 1089.
Path 122 | total_timesteps 1098.
Path 123 | total_timesteps 1108.
Path 124 | total_timesteps 1115.
Path 125 | total_timesteps 1125.
Path 126 | total_timesteps 1131.
Path 127 | total_timesteps 1138.
Path 128 | total_timesteps 1154.
Path 129 | total_timesteps 1161.
Path 130 | total_timesteps 1170.
Path 131 | total_timesteps 1179.
Path 132 | total_timesteps 1186.
Path 133 | total_timesteps 1194.
Path 134 | total_timesteps 1200.
Path 135 | total_timesteps 1213.
Path 136 | total_timesteps 1221.
Path 137 | total_timesteps 1228.
Path 138 | total_timesteps 1240.
Path 139 | total_timesteps 1247.
Path 140 | total_timesteps 1257.
Path 141 | total_timesteps 1267.
Path 142 | total_timesteps 1276.
Path 143 | total_timesteps 1285.
Path 144 | total_timesteps 1294.
Path 145 | total_timesteps 1302.
Path 146 | total_timesteps 1310.
Path 147 | total_timesteps 1318.
Path 148 | total_timesteps 1327.
Path 149 | total_timesteps 1337.
Path 150 | total_timesteps 1347.
Path 151 | total_timesteps 1357.
Path 152 | total_timesteps 1368.
Path 153 | total_timesteps 1377.
Path 154 | total_timesteps 1384.
Path 155 | total_timesteps 1392.
Path 156 | total_timesteps 1400.
Path 157 | total_timesteps 1406.
Path 158 | total_timesteps 1414.
Path 159 | total_timesteps 1422.
Path 160 | total_timesteps 1430.
Path 161 | total_timesteps 1438.
Path 162 | total_timesteps 1446.
Path 163 | total_timesteps 1454.
Path 164 | total_timesteps 1461.
Path 165 | total_timesteps 1469.
Path 166 | total_timesteps 1481.
Path 167 | total_timesteps 1489.
Path 168 | total_timesteps 1498.
Path 169 | total_timesteps 1508.
Path 170 | total_timesteps 1516.
Path 171 | total_timesteps 1523.
Path 172 | total_timesteps 1539.
Path 173 | total_timesteps 1549.
Path 174 | total_timesteps 1558.
Path 175 | total_timesteps 1565.
Path 176 | total_timesteps 1573.
Path 177 | total_timesteps 1582.
Path 178 | total_timesteps 1591.
Path 179 | total_timesteps 1602.
Path 180 | total_timesteps 1611.
Path 181 | total_timesteps 1620.
Path 182 | total_timesteps 1628.
Path 183 | total_timesteps 1636.
Path 184 | total_timesteps 1644.
Path 185 | total_timesteps 1659.
Path 186 | total_timesteps 1673.
Path 187 | total_timesteps 1681.
Path 188 | total_timesteps 1690.
Path 189 | total_timesteps 1699.
Path 190 | total_timesteps 1709.
Path 191 | total_timesteps 1717.
Path 192 | total_timesteps 1730.
Path 193 | total_timesteps 1739.
Path 194 | total_timesteps 1747.
Path 195 | total_timesteps 1756.
Path 196 | total_timesteps 1769.
Path 197 | total_timesteps 1784.
Path 198 | total_timesteps 1792.
Path 199 | total_timesteps 1800.
Path 200 | total_timesteps 1809.
Path 201 | total_timesteps 1820.
Path 202 | total_timesteps 1828.
Path 203 | total_timesteps 1835.
Path 204 | total_timesteps 1843.
Path 205 | total_timesteps 1854.
Path 206 | total_timesteps 1865.
Path 207 | total_timesteps 1873.
Path 208 | total_timesteps 1881.
Path 209 | total_timesteps 1889.
Path 210 | total_timesteps 1900.
Path 211 | total_timesteps 1910.
Path 212 | total_timesteps 1917.
Path 213 | total_timesteps 1924.
Path 214 | total_timesteps 1935.
Path 215 | total_timesteps 1942.
Path 216 | total_timesteps 1949.
Path 217 | total_timesteps 1957.
Path 218 | total_timesteps 1971.
Path 219 | total_timesteps 1979.
Path 220 | total_timesteps 1985.
Path 221 | total_timesteps 1993.
Path 222 | total_timesteps 2000.
Path 223 | total_timesteps 2008.
Path 224 | total_timesteps 2018.
Path 225 | total_timesteps 2028.
Path 226 | total_timesteps 2038.
Path 227 | total_timesteps 2045.
Path 228 | total_timesteps 2054.
Path 229 | total_timesteps 2066.
Path 230 | total_timesteps 2077.
Path 231 | total_timesteps 2086.
Path 232 | total_timesteps 2095.
Path 233 | total_timesteps 2110.
Path 234 | total_timesteps 2121.
Path 235 | total_timesteps 2130.
Path 236 | total_timesteps 2145.
Path 237 | total_timesteps 2153.
Path 238 | total_timesteps 2161.
Path 239 | total_timesteps 2168.
Path 240 | total_timesteps 2176.
Path 241 | total_timesteps 2184.
Path 242 | total_timesteps 2193.
Path 243 | total_timesteps 2209.
Path 244 | total_timesteps 2219.
Path 245 | total_timesteps 2228.
Path 246 | total_timesteps 2241.
Path 247 | total_timesteps 2249.
Path 248 | total_timesteps 2261.
Path 249 | total_timesteps 2271.
Path 250 | total_timesteps 2279.
Path 251 | total_timesteps 2286.
Path 252 | total_timesteps 2295.
Path 253 | total_timesteps 2304.
Path 254 | total_timesteps 2311.
Path 255 | total_timesteps 2321.
Path 256 | total_timesteps 2330.
Path 257 | total_timesteps 2343.
Path 258 | total_timesteps 2351.
Path 259 | total_timesteps 2359.
Path 260 | total_timesteps 2367.
Path 261 | total_timesteps 2375.
Path 262 | total_timesteps 2384.
Path 263 | total_timesteps 2394.
Path 264 | total_timesteps 2402.
Path 265 | total_timesteps 2409.
Path 266 | total_timesteps 2416.
Path 267 | total_timesteps 2423.
Path 268 | total_timesteps 2432.
Path 269 | total_timesteps 2439.
Path 270 | total_timesteps 2447.
Path 271 | total_timesteps 2455.
Path 272 | total_timesteps 2466.
Path 273 | total_timesteps 2473.
Path 274 | total_timesteps 2481.
Path 275 | total_timesteps 2493.
Path 276 | total_timesteps 2501.
Path 277 | total_timesteps 2510.
Path 278 | total_timesteps 2520.
Path 279 | total_timesteps 2529.
Path 280 | total_timesteps 2538.
Path 281 | total_timesteps 2549.
Path 282 | total_timesteps 2559.
Path 283 | total_timesteps 2566.
Path 284 | total_timesteps 2578.
Path 285 | total_timesteps 2584.
Path 286 | total_timesteps 2591.
Path 287 | total_timesteps 2601.
Path 288 | total_timesteps 2608.
Path 289 | total_timesteps 2619.
Path 290 | total_timesteps 2628.
Path 291 | total_timesteps 2640.
Path 292 | total_timesteps 2650.
Path 293 | total_timesteps 2658.
Path 294 | total_timesteps 2665.
Path 295 | total_timesteps 2672.
Path 296 | total_timesteps 2683.
Path 297 | total_timesteps 2689.
Path 298 | total_timesteps 2696.
Path 299 | total_timesteps 2706.
Path 300 | total_timesteps 2720.
Path 301 | total_timesteps 2728.
Path 302 | total_timesteps 2738.
Path 303 | total_timesteps 2745.
Path 304 | total_timesteps 2753.
Path 305 | total_timesteps 2760.
Path 306 | total_timesteps 2767.
Path 307 | total_timesteps 2777.
Path 308 | total_timesteps 2785.
Path 309 | total_timesteps 2793.
Path 310 | total_timesteps 2802.
Path 311 | total_timesteps 2811.
Path 312 | total_timesteps 2820.
Path 313 | total_timesteps 2830.
Path 314 | total_timesteps 2839.
Path 315 | total_timesteps 2851.
Path 316 | total_timesteps 2859.
Path 317 | total_timesteps 2867.
Path 318 | total_timesteps 2876.
Path 319 | total_timesteps 2887.
Path 320 | total_timesteps 2895.
Path 321 | total_timesteps 2901.
Path 322 | total_timesteps 2908.
Path 323 | total_timesteps 2922.
Path 324 | total_timesteps 2934.
Path 325 | total_timesteps 2943.
Path 326 | total_timesteps 2950.
Path 327 | total_timesteps 2957.
Path 328 | total_timesteps 2963.
Path 329 | total_timesteps 2972.
Path 330 | total_timesteps 2981.
Path 331 | total_timesteps 2991.
Path 332 | total_timesteps 3000.
Path 333 | total_timesteps 3008.
Path 334 | total_timesteps 3015.
Path 335 | total_timesteps 3027.
Path 336 | total_timesteps 3037.
Path 337 | total_timesteps 3045.
Path 338 | total_timesteps 3054.
Path 339 | total_timesteps 3063.
Path 340 | total_timesteps 3072.
Path 341 | total_timesteps 3078.
Path 342 | total_timesteps 3085.
Path 343 | total_timesteps 3096.
Path 344 | total_timesteps 3103.
Path 345 | total_timesteps 3111.
Path 346 | total_timesteps 3118.
Path 347 | total_timesteps 3128.
Path 348 | total_timesteps 3138.
Path 349 | total_timesteps 3149.
Path 350 | total_timesteps 3157.
Path 351 | total_timesteps 3164.
Path 352 | total_timesteps 3174.
Path 353 | total_timesteps 3182.
Path 354 | total_timesteps 3195.
Path 355 | total_timesteps 3202.
Path 356 | total_timesteps 3208.
Path 357 | total_timesteps 3218.
Path 358 | total_timesteps 3227.
Path 359 | total_timesteps 3234.
Path 360 | total_timesteps 3245.
Path 361 | total_timesteps 3252.
Path 362 | total_timesteps 3261.
Path 363 | total_timesteps 3270.
Path 364 | total_timesteps 3278.
Path 365 | total_timesteps 3286.
Path 366 | total_timesteps 3294.
Path 367 | total_timesteps 3301.
Path 368 | total_timesteps 3311.
Path 369 | total_timesteps 3323.
Path 370 | total_timesteps 3333.
Path 371 | total_timesteps 3340.
Path 372 | total_timesteps 3350.
Path 373 | total_timesteps 3359.
Path 374 | total_timesteps 3367.
Path 375 | total_timesteps 3374.
Path 376 | total_timesteps 3381.
Path 377 | total_timesteps 3390.
Path 378 | total_timesteps 3398.
Path 379 | total_timesteps 3407.
Path 380 | total_timesteps 3417.
Path 381 | total_timesteps 3430.
Path 382 | total_timesteps 3437.
Path 383 | total_timesteps 3444.
Path 384 | total_timesteps 3456.
Path 385 | total_timesteps 3475.
Path 386 | total_timesteps 3483.
Path 387 | total_timesteps 3491.
Path 388 | total_timesteps 3498.
Path 389 | total_timesteps 3509.
Path 390 | total_timesteps 3527.
Path 391 | total_timesteps 3535.
Path 392 | total_timesteps 3542.
Path 393 | total_timesteps 3551.
Path 394 | total_timesteps 3560.
Path 395 | total_timesteps 3570.
Path 396 | total_timesteps 3577.
Path 397 | total_timesteps 3585.
Path 398 | total_timesteps 3592.
Path 399 | total_timesteps 3601.
Path 400 | total_timesteps 3608.
Path 401 | total_timesteps 3619.
Path 402 | total_timesteps 3631.
Path 403 | total_timesteps 3638.
Path 404 | total_timesteps 3647.
Path 405 | total_timesteps 3655.
Path 406 | total_timesteps 3661.
Path 407 | total_timesteps 3670.
Path 408 | total_timesteps 3682.
Path 409 | total_timesteps 3694.
Path 410 | total_timesteps 3702.
Path 411 | total_timesteps 3710.
Path 412 | total_timesteps 3717.
Path 413 | total_timesteps 3725.
Path 414 | total_timesteps 3734.
Path 415 | total_timesteps 3742.
Path 416 | total_timesteps 3752.
Path 417 | total_timesteps 3762.
Path 418 | total_timesteps 3770.
Path 419 | total_timesteps 3779.
Path 420 | total_timesteps 3786.
Path 421 | total_timesteps 3792.
Path 422 | total_timesteps 3805.
Path 423 | total_timesteps 3813.
Path 424 | total_timesteps 3819.
Path 425 | total_timesteps 3829.
Path 426 | total_timesteps 3841.
Path 427 | total_timesteps 3853.
Path 428 | total_timesteps 3860.
Path 429 | total_timesteps 3868.
Path 430 | total_timesteps 3876.
Path 431 | total_timesteps 3889.
Path 432 | total_timesteps 3900.
Path 433 | total_timesteps 3912.
Path 434 | total_timesteps 3922.
Path 435 | total_timesteps 3931.
Path 436 | total_timesteps 3938.
Path 437 | total_timesteps 3948.
Path 438 | total_timesteps 3960.
Path 439 | total_timesteps 3969.
Path 440 | total_timesteps 3978.
Path 441 | total_timesteps 3985.
Path 442 | total_timesteps 3993.
Path 443 | total_timesteps 4000.
Path 444 | total_timesteps 4007.
Path 445 | total_timesteps 4016.
Path 446 | total_timesteps 4022.
Path 447 | total_timesteps 4029.
Path 448 | total_timesteps 4039.
Path 449 | total_timesteps 4047.
Path 450 | total_timesteps 4056.
Path 451 | total_timesteps 4066.
Path 452 | total_timesteps 4074.
Path 453 | total_timesteps 4083.
Path 454 | total_timesteps 4092.
Path 455 | total_timesteps 4099.
Path 456 | total_timesteps 4105.
Path 457 | total_timesteps 4112.
Path 458 | total_timesteps 4124.
Path 459 | total_timesteps 4132.
Path 460 | total_timesteps 4140.
Path 461 | total_timesteps 4149.
Path 462 | total_timesteps 4159.
Path 463 | total_timesteps 4167.
Path 464 | total_timesteps 4176.
Path 465 | total_timesteps 4184.
Path 466 | total_timesteps 4192.
Path 467 | total_timesteps 4201.
Path 468 | total_timesteps 4210.
Path 469 | total_timesteps 4217.
Path 470 | total_timesteps 4230.
Path 471 | total_timesteps 4238.
Path 472 | total_timesteps 4247.
Path 473 | total_timesteps 4254.
Path 474 | total_timesteps 4265.
Path 475 | total_timesteps 4276.
Path 476 | total_timesteps 4285.
Path 477 | total_timesteps 4295.
Path 478 | total_timesteps 4302.
Path 479 | total_timesteps 4310.
Path 480 | total_timesteps 4324.
Path 481 | total_timesteps 4334.
Path 482 | total_timesteps 4343.
Path 483 | total_timesteps 4349.
Path 484 | total_timesteps 4358.
Path 485 | total_timesteps 4371.
Path 486 | total_timesteps 4378.
Path 487 | total_timesteps 4388.
Path 488 | total_timesteps 4397.
Path 489 | total_timesteps 4404.
Path 490 | total_timesteps 4411.
Path 491 | total_timesteps 4424.
Path 492 | total_timesteps 4432.
Path 493 | total_timesteps 4439.
Path 494 | total_timesteps 4448.
Path 495 | total_timesteps 4461.
Path 496 | total_timesteps 4471.
Path 497 | total_timesteps 4479.
Path 498 | total_timesteps 4489.
Path 499 | total_timesteps 4499.
Path 500 | total_timesteps 4512.
Path 501 | total_timesteps 4524.
Path 502 | total_timesteps 4540.
Path 503 | total_timesteps 4546.
Path 504 | total_timesteps 4557.
Path 505 | total_timesteps 4565.
Path 506 | total_timesteps 4573.
Path 507 | total_timesteps 4581.
Path 508 | total_timesteps 4592.
Path 509 | total_timesteps 4602.
Path 510 | total_timesteps 4609.
Path 511 | total_timesteps 4618.
Path 512 | total_timesteps 4626.
Path 513 | total_timesteps 4634.
Path 514 | total_timesteps 4642.
Path 515 | total_timesteps 4653.
Path 516 | total_timesteps 4660.
Path 517 | total_timesteps 4667.
Path 518 | total_timesteps 4677.
Path 519 | total_timesteps 4691.
Path 520 | total_timesteps 4704.
Path 521 | total_timesteps 4716.
Path 522 | total_timesteps 4723.
Path 523 | total_timesteps 4730.
Path 524 | total_timesteps 4741.
Path 525 | total_timesteps 4748.
Path 526 | total_timesteps 4757.
Path 527 | total_timesteps 4768.
Path 528 | total_timesteps 4780.
Path 529 | total_timesteps 4792.
Path 530 | total_timesteps 4802.
Path 531 | total_timesteps 4809.
Path 532 | total_timesteps 4821.
Path 533 | total_timesteps 4830.
Path 534 | total_timesteps 4840.
Path 535 | total_timesteps 4847.
Path 536 | total_timesteps 4854.
Path 537 | total_timesteps 4861.
Path 538 | total_timesteps 4868.
Path 539 | total_timesteps 4876.
Path 540 | total_timesteps 4890.
Path 541 | total_timesteps 4898.
Path 542 | total_timesteps 4905.
Path 543 | total_timesteps 4918.
Path 544 | total_timesteps 4927.
Path 545 | total_timesteps 4936.
Path 546 | total_timesteps 4943.
Path 547 | total_timesteps 4951.
Path 548 | total_timesteps 4958.
Path 549 | total_timesteps 4967.
Path 550 | total_timesteps 4976.
Path 551 | total_timesteps 4991.
Path 552 | total_timesteps 4998.
Path 553 | total_timesteps 5004.
Path 554 | total_timesteps 5012.
Path 555 | total_timesteps 5019.
Path 556 | total_timesteps 5026.
Path 557 | total_timesteps 5036.
Path 558 | total_timesteps 5045.
Path 559 | total_timesteps 5059.
Path 560 | total_timesteps 5066.
Path 561 | total_timesteps 5073.
Path 562 | total_timesteps 5080.
Path 563 | total_timesteps 5088.
Path 564 | total_timesteps 5097.
Path 565 | total_timesteps 5107.
Path 566 | total_timesteps 5119.
Path 567 | total_timesteps 5125.
Path 568 | total_timesteps 5133.
Path 569 | total_timesteps 5141.
Path 570 | total_timesteps 5148.
Path 571 | total_timesteps 5157.
Path 572 | total_timesteps 5163.
Path 573 | total_timesteps 5172.
Path 574 | total_timesteps 5179.
Path 575 | total_timesteps 5187.
Path 576 | total_timesteps 5194.
Path 577 | total_timesteps 5201.
Path 578 | total_timesteps 5207.
Path 579 | total_timesteps 5213.
Path 580 | total_timesteps 5222.
Path 581 | total_timesteps 5231.
Path 582 | total_timesteps 5240.
Path 583 | total_timesteps 5247.
Path 584 | total_timesteps 5256.
Path 585 | total_timesteps 5265.
Path 586 | total_timesteps 5271.
Path 587 | total_timesteps 5278.
Path 588 | total_timesteps 5286.
Path 589 | total_timesteps 5293.
Path 590 | total_timesteps 5300.
Path 591 | total_timesteps 5311.
Path 592 | total_timesteps 5322.
Path 593 | total_timesteps 5334.
Path 594 | total_timesteps 5343.
Path 595 | total_timesteps 5351.
Path 596 | total_timesteps 5359.
Path 597 | total_timesteps 5372.
Path 598 | total_timesteps 5378.
Path 599 | total_timesteps 5389.
Path 600 | total_timesteps 5399.
Path 601 | total_timesteps 5406.
Path 602 | total_timesteps 5413.
Path 603 | total_timesteps 5421.
Path 604 | total_timesteps 5428.
Path 605 | total_timesteps 5436.
Path 606 | total_timesteps 5446.
Path 607 | total_timesteps 5459.
Path 608 | total_timesteps 5475.
Path 609 | total_timesteps 5485.
Path 610 | total_timesteps 5492.
Path 611 | total_timesteps 5500.
Path 612 | total_timesteps 5508.
Path 613 | total_timesteps 5516.
Path 614 | total_timesteps 5522.
Path 615 | total_timesteps 5530.
Path 616 | total_timesteps 5540.
Path 617 | total_timesteps 5548.
Path 618 | total_timesteps 5554.
Path 619 | total_timesteps 5561.
Path 620 | total_timesteps 5571.
Path 621 | total_timesteps 5579.
Path 622 | total_timesteps 5589.
Path 623 | total_timesteps 5595.
Path 624 | total_timesteps 5601.
Path 625 | total_timesteps 5610.
Path 626 | total_timesteps 5616.
Path 627 | total_timesteps 5625.
Path 628 | total_timesteps 5635.
Path 629 | total_timesteps 5643.
Path 630 | total_timesteps 5660.
Path 631 | total_timesteps 5667.
Path 632 | total_timesteps 5678.
Path 633 | total_timesteps 5685.
Path 634 | total_timesteps 5692.
Path 635 | total_timesteps 5703.
Path 636 | total_timesteps 5710.
Path 637 | total_timesteps 5719.
Path 638 | total_timesteps 5726.
Path 639 | total_timesteps 5733.
Path 640 | total_timesteps 5741.
Path 641 | total_timesteps 5748.
Path 642 | total_timesteps 5754.
Path 643 | total_timesteps 5765.
Path 644 | total_timesteps 5772.
Path 645 | total_timesteps 5781.
Path 646 | total_timesteps 5787.
Path 647 | total_timesteps 5794.
Path 648 | total_timesteps 5801.
Path 649 | total_timesteps 5810.
Path 650 | total_timesteps 5818.
Path 651 | total_timesteps 5825.
Path 652 | total_timesteps 5835.
Path 653 | total_timesteps 5843.
Path 654 | total_timesteps 5851.
Path 655 | total_timesteps 5859.
Path 656 | total_timesteps 5866.
Path 657 | total_timesteps 5874.
Path 658 | total_timesteps 5885.
Path 659 | total_timesteps 5894.
Path 660 | total_timesteps 5902.
Path 661 | total_timesteps 5911.
Path 662 | total_timesteps 5919.
Path 663 | total_timesteps 5927.
Path 664 | total_timesteps 5935.
Path 665 | total_timesteps 5944.
Path 666 | total_timesteps 5951.
Path 667 | total_timesteps 5960.
Path 668 | total_timesteps 5968.
Path 669 | total_timesteps 5978.
Path 670 | total_timesteps 5988.
Path 671 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.88    |
| Iteration     | 18       |
| MaximumReturn | -1.12    |
| MinimumReturn | -19.8    |
| TotalSamples  | 80073    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.0024149413220584393
Validation loss = 0.0025956195313483477
Validation loss = 0.002360046375542879
Validation loss = 0.0022468611132353544
Validation loss = 0.0021583414636552334
Validation loss = 0.002348842564970255
Validation loss = 0.002152304397895932
Validation loss = 0.0025238157249987125
Validation loss = 0.0020363950170576572
Validation loss = 0.0024543381296098232
Validation loss = 0.0020903684198856354
Validation loss = 0.0024807420559227467
Validation loss = 0.002133650705218315
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 45.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 64.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 78.
Path 9 | total_timesteps 86.
Path 10 | total_timesteps 95.
Path 11 | total_timesteps 104.
Path 12 | total_timesteps 111.
Path 13 | total_timesteps 119.
Path 14 | total_timesteps 133.
Path 15 | total_timesteps 142.
Path 16 | total_timesteps 152.
Path 17 | total_timesteps 161.
Path 18 | total_timesteps 172.
Path 19 | total_timesteps 178.
Path 20 | total_timesteps 187.
Path 21 | total_timesteps 196.
Path 22 | total_timesteps 208.
Path 23 | total_timesteps 217.
Path 24 | total_timesteps 225.
Path 25 | total_timesteps 233.
Path 26 | total_timesteps 241.
Path 27 | total_timesteps 252.
Path 28 | total_timesteps 261.
Path 29 | total_timesteps 275.
Path 30 | total_timesteps 284.
Path 31 | total_timesteps 295.
Path 32 | total_timesteps 304.
Path 33 | total_timesteps 311.
Path 34 | total_timesteps 323.
Path 35 | total_timesteps 329.
Path 36 | total_timesteps 336.
Path 37 | total_timesteps 346.
Path 38 | total_timesteps 359.
Path 39 | total_timesteps 367.
Path 40 | total_timesteps 375.
Path 41 | total_timesteps 384.
Path 42 | total_timesteps 392.
Path 43 | total_timesteps 399.
Path 44 | total_timesteps 407.
Path 45 | total_timesteps 416.
Path 46 | total_timesteps 425.
Path 47 | total_timesteps 434.
Path 48 | total_timesteps 444.
Path 49 | total_timesteps 453.
Path 50 | total_timesteps 460.
Path 51 | total_timesteps 468.
Path 52 | total_timesteps 477.
Path 53 | total_timesteps 487.
Path 54 | total_timesteps 500.
Path 55 | total_timesteps 510.
Path 56 | total_timesteps 519.
Path 57 | total_timesteps 528.
Path 58 | total_timesteps 536.
Path 59 | total_timesteps 544.
Path 60 | total_timesteps 551.
Path 61 | total_timesteps 562.
Path 62 | total_timesteps 570.
Path 63 | total_timesteps 578.
Path 64 | total_timesteps 588.
Path 65 | total_timesteps 598.
Path 66 | total_timesteps 607.
Path 67 | total_timesteps 615.
Path 68 | total_timesteps 622.
Path 69 | total_timesteps 628.
Path 70 | total_timesteps 635.
Path 71 | total_timesteps 645.
Path 72 | total_timesteps 656.
Path 73 | total_timesteps 667.
Path 74 | total_timesteps 674.
Path 75 | total_timesteps 682.
Path 76 | total_timesteps 690.
Path 77 | total_timesteps 697.
Path 78 | total_timesteps 705.
Path 79 | total_timesteps 715.
Path 80 | total_timesteps 722.
Path 81 | total_timesteps 730.
Path 82 | total_timesteps 739.
Path 83 | total_timesteps 754.
Path 84 | total_timesteps 760.
Path 85 | total_timesteps 769.
Path 86 | total_timesteps 777.
Path 87 | total_timesteps 787.
Path 88 | total_timesteps 793.
Path 89 | total_timesteps 802.
Path 90 | total_timesteps 813.
Path 91 | total_timesteps 819.
Path 92 | total_timesteps 827.
Path 93 | total_timesteps 835.
Path 94 | total_timesteps 848.
Path 95 | total_timesteps 859.
Path 96 | total_timesteps 868.
Path 97 | total_timesteps 879.
Path 98 | total_timesteps 889.
Path 99 | total_timesteps 901.
Path 100 | total_timesteps 908.
Path 101 | total_timesteps 918.
Path 102 | total_timesteps 926.
Path 103 | total_timesteps 936.
Path 104 | total_timesteps 945.
Path 105 | total_timesteps 952.
Path 106 | total_timesteps 959.
Path 107 | total_timesteps 968.
Path 108 | total_timesteps 977.
Path 109 | total_timesteps 988.
Path 110 | total_timesteps 997.
Path 111 | total_timesteps 1007.
Path 112 | total_timesteps 1016.
Path 113 | total_timesteps 1023.
Path 114 | total_timesteps 1031.
Path 115 | total_timesteps 1040.
Path 116 | total_timesteps 1047.
Path 117 | total_timesteps 1055.
Path 118 | total_timesteps 1065.
Path 119 | total_timesteps 1078.
Path 120 | total_timesteps 1087.
Path 121 | total_timesteps 1093.
Path 122 | total_timesteps 1103.
Path 123 | total_timesteps 1110.
Path 124 | total_timesteps 1117.
Path 125 | total_timesteps 1125.
Path 126 | total_timesteps 1136.
Path 127 | total_timesteps 1147.
Path 128 | total_timesteps 1156.
Path 129 | total_timesteps 1167.
Path 130 | total_timesteps 1174.
Path 131 | total_timesteps 1183.
Path 132 | total_timesteps 1193.
Path 133 | total_timesteps 1200.
Path 134 | total_timesteps 1207.
Path 135 | total_timesteps 1221.
Path 136 | total_timesteps 1227.
Path 137 | total_timesteps 1235.
Path 138 | total_timesteps 1242.
Path 139 | total_timesteps 1258.
Path 140 | total_timesteps 1267.
Path 141 | total_timesteps 1276.
Path 142 | total_timesteps 1286.
Path 143 | total_timesteps 1294.
Path 144 | total_timesteps 1301.
Path 145 | total_timesteps 1313.
Path 146 | total_timesteps 1320.
Path 147 | total_timesteps 1329.
Path 148 | total_timesteps 1338.
Path 149 | total_timesteps 1345.
Path 150 | total_timesteps 1353.
Path 151 | total_timesteps 1360.
Path 152 | total_timesteps 1368.
Path 153 | total_timesteps 1377.
Path 154 | total_timesteps 1386.
Path 155 | total_timesteps 1393.
Path 156 | total_timesteps 1405.
Path 157 | total_timesteps 1413.
Path 158 | total_timesteps 1422.
Path 159 | total_timesteps 1432.
Path 160 | total_timesteps 1440.
Path 161 | total_timesteps 1447.
Path 162 | total_timesteps 1456.
Path 163 | total_timesteps 1470.
Path 164 | total_timesteps 1484.
Path 165 | total_timesteps 1495.
Path 166 | total_timesteps 1503.
Path 167 | total_timesteps 1510.
Path 168 | total_timesteps 1519.
Path 169 | total_timesteps 1527.
Path 170 | total_timesteps 1534.
Path 171 | total_timesteps 1543.
Path 172 | total_timesteps 1551.
Path 173 | total_timesteps 1560.
Path 174 | total_timesteps 1569.
Path 175 | total_timesteps 1577.
Path 176 | total_timesteps 1588.
Path 177 | total_timesteps 1598.
Path 178 | total_timesteps 1608.
Path 179 | total_timesteps 1619.
Path 180 | total_timesteps 1628.
Path 181 | total_timesteps 1637.
Path 182 | total_timesteps 1646.
Path 183 | total_timesteps 1655.
Path 184 | total_timesteps 1664.
Path 185 | total_timesteps 1675.
Path 186 | total_timesteps 1686.
Path 187 | total_timesteps 1692.
Path 188 | total_timesteps 1699.
Path 189 | total_timesteps 1706.
Path 190 | total_timesteps 1715.
Path 191 | total_timesteps 1722.
Path 192 | total_timesteps 1730.
Path 193 | total_timesteps 1737.
Path 194 | total_timesteps 1747.
Path 195 | total_timesteps 1756.
Path 196 | total_timesteps 1763.
Path 197 | total_timesteps 1771.
Path 198 | total_timesteps 1778.
Path 199 | total_timesteps 1785.
Path 200 | total_timesteps 1793.
Path 201 | total_timesteps 1805.
Path 202 | total_timesteps 1815.
Path 203 | total_timesteps 1825.
Path 204 | total_timesteps 1832.
Path 205 | total_timesteps 1838.
Path 206 | total_timesteps 1845.
Path 207 | total_timesteps 1856.
Path 208 | total_timesteps 1866.
Path 209 | total_timesteps 1874.
Path 210 | total_timesteps 1886.
Path 211 | total_timesteps 1894.
Path 212 | total_timesteps 1903.
Path 213 | total_timesteps 1917.
Path 214 | total_timesteps 1926.
Path 215 | total_timesteps 1936.
Path 216 | total_timesteps 1946.
Path 217 | total_timesteps 1955.
Path 218 | total_timesteps 1964.
Path 219 | total_timesteps 1973.
Path 220 | total_timesteps 1980.
Path 221 | total_timesteps 1990.
Path 222 | total_timesteps 2001.
Path 223 | total_timesteps 2008.
Path 224 | total_timesteps 2016.
Path 225 | total_timesteps 2032.
Path 226 | total_timesteps 2040.
Path 227 | total_timesteps 2049.
Path 228 | total_timesteps 2063.
Path 229 | total_timesteps 2069.
Path 230 | total_timesteps 2081.
Path 231 | total_timesteps 2090.
Path 232 | total_timesteps 2099.
Path 233 | total_timesteps 2108.
Path 234 | total_timesteps 2117.
Path 235 | total_timesteps 2124.
Path 236 | total_timesteps 2134.
Path 237 | total_timesteps 2142.
Path 238 | total_timesteps 2150.
Path 239 | total_timesteps 2159.
Path 240 | total_timesteps 2167.
Path 241 | total_timesteps 2177.
Path 242 | total_timesteps 2185.
Path 243 | total_timesteps 2193.
Path 244 | total_timesteps 2203.
Path 245 | total_timesteps 2213.
Path 246 | total_timesteps 2222.
Path 247 | total_timesteps 2231.
Path 248 | total_timesteps 2238.
Path 249 | total_timesteps 2245.
Path 250 | total_timesteps 2254.
Path 251 | total_timesteps 2261.
Path 252 | total_timesteps 2268.
Path 253 | total_timesteps 2276.
Path 254 | total_timesteps 2286.
Path 255 | total_timesteps 2294.
Path 256 | total_timesteps 2301.
Path 257 | total_timesteps 2309.
Path 258 | total_timesteps 2321.
Path 259 | total_timesteps 2331.
Path 260 | total_timesteps 2338.
Path 261 | total_timesteps 2346.
Path 262 | total_timesteps 2360.
Path 263 | total_timesteps 2370.
Path 264 | total_timesteps 2380.
Path 265 | total_timesteps 2388.
Path 266 | total_timesteps 2394.
Path 267 | total_timesteps 2402.
Path 268 | total_timesteps 2413.
Path 269 | total_timesteps 2422.
Path 270 | total_timesteps 2434.
Path 271 | total_timesteps 2445.
Path 272 | total_timesteps 2455.
Path 273 | total_timesteps 2463.
Path 274 | total_timesteps 2474.
Path 275 | total_timesteps 2482.
Path 276 | total_timesteps 2494.
Path 277 | total_timesteps 2503.
Path 278 | total_timesteps 2515.
Path 279 | total_timesteps 2524.
Path 280 | total_timesteps 2532.
Path 281 | total_timesteps 2540.
Path 282 | total_timesteps 2546.
Path 283 | total_timesteps 2553.
Path 284 | total_timesteps 2565.
Path 285 | total_timesteps 2572.
Path 286 | total_timesteps 2581.
Path 287 | total_timesteps 2588.
Path 288 | total_timesteps 2597.
Path 289 | total_timesteps 2605.
Path 290 | total_timesteps 2615.
Path 291 | total_timesteps 2624.
Path 292 | total_timesteps 2633.
Path 293 | total_timesteps 2643.
Path 294 | total_timesteps 2650.
Path 295 | total_timesteps 2658.
Path 296 | total_timesteps 2668.
Path 297 | total_timesteps 2681.
Path 298 | total_timesteps 2691.
Path 299 | total_timesteps 2702.
Path 300 | total_timesteps 2710.
Path 301 | total_timesteps 2719.
Path 302 | total_timesteps 2729.
Path 303 | total_timesteps 2737.
Path 304 | total_timesteps 2744.
Path 305 | total_timesteps 2751.
Path 306 | total_timesteps 2758.
Path 307 | total_timesteps 2766.
Path 308 | total_timesteps 2776.
Path 309 | total_timesteps 2785.
Path 310 | total_timesteps 2793.
Path 311 | total_timesteps 2803.
Path 312 | total_timesteps 2811.
Path 313 | total_timesteps 2818.
Path 314 | total_timesteps 2825.
Path 315 | total_timesteps 2833.
Path 316 | total_timesteps 2844.
Path 317 | total_timesteps 2850.
Path 318 | total_timesteps 2860.
Path 319 | total_timesteps 2871.
Path 320 | total_timesteps 2880.
Path 321 | total_timesteps 2889.
Path 322 | total_timesteps 2899.
Path 323 | total_timesteps 2905.
Path 324 | total_timesteps 2915.
Path 325 | total_timesteps 2924.
Path 326 | total_timesteps 2936.
Path 327 | total_timesteps 2945.
Path 328 | total_timesteps 2952.
Path 329 | total_timesteps 2959.
Path 330 | total_timesteps 2968.
Path 331 | total_timesteps 2976.
Path 332 | total_timesteps 2985.
Path 333 | total_timesteps 2992.
Path 334 | total_timesteps 2999.
Path 335 | total_timesteps 3007.
Path 336 | total_timesteps 3014.
Path 337 | total_timesteps 3021.
Path 338 | total_timesteps 3032.
Path 339 | total_timesteps 3040.
Path 340 | total_timesteps 3051.
Path 341 | total_timesteps 3059.
Path 342 | total_timesteps 3066.
Path 343 | total_timesteps 3075.
Path 344 | total_timesteps 3084.
Path 345 | total_timesteps 3092.
Path 346 | total_timesteps 3100.
Path 347 | total_timesteps 3107.
Path 348 | total_timesteps 3118.
Path 349 | total_timesteps 3125.
Path 350 | total_timesteps 3134.
Path 351 | total_timesteps 3142.
Path 352 | total_timesteps 3149.
Path 353 | total_timesteps 3155.
Path 354 | total_timesteps 3164.
Path 355 | total_timesteps 3175.
Path 356 | total_timesteps 3189.
Path 357 | total_timesteps 3199.
Path 358 | total_timesteps 3209.
Path 359 | total_timesteps 3216.
Path 360 | total_timesteps 3232.
Path 361 | total_timesteps 3239.
Path 362 | total_timesteps 3248.
Path 363 | total_timesteps 3256.
Path 364 | total_timesteps 3263.
Path 365 | total_timesteps 3271.
Path 366 | total_timesteps 3280.
Path 367 | total_timesteps 3290.
Path 368 | total_timesteps 3298.
Path 369 | total_timesteps 3308.
Path 370 | total_timesteps 3318.
Path 371 | total_timesteps 3326.
Path 372 | total_timesteps 3336.
Path 373 | total_timesteps 3344.
Path 374 | total_timesteps 3351.
Path 375 | total_timesteps 3362.
Path 376 | total_timesteps 3370.
Path 377 | total_timesteps 3378.
Path 378 | total_timesteps 3396.
Path 379 | total_timesteps 3404.
Path 380 | total_timesteps 3412.
Path 381 | total_timesteps 3419.
Path 382 | total_timesteps 3426.
Path 383 | total_timesteps 3435.
Path 384 | total_timesteps 3442.
Path 385 | total_timesteps 3449.
Path 386 | total_timesteps 3456.
Path 387 | total_timesteps 3467.
Path 388 | total_timesteps 3476.
Path 389 | total_timesteps 3485.
Path 390 | total_timesteps 3495.
Path 391 | total_timesteps 3501.
Path 392 | total_timesteps 3507.
Path 393 | total_timesteps 3515.
Path 394 | total_timesteps 3521.
Path 395 | total_timesteps 3528.
Path 396 | total_timesteps 3540.
Path 397 | total_timesteps 3549.
Path 398 | total_timesteps 3559.
Path 399 | total_timesteps 3567.
Path 400 | total_timesteps 3577.
Path 401 | total_timesteps 3588.
Path 402 | total_timesteps 3598.
Path 403 | total_timesteps 3607.
Path 404 | total_timesteps 3618.
Path 405 | total_timesteps 3628.
Path 406 | total_timesteps 3634.
Path 407 | total_timesteps 3641.
Path 408 | total_timesteps 3647.
Path 409 | total_timesteps 3661.
Path 410 | total_timesteps 3670.
Path 411 | total_timesteps 3682.
Path 412 | total_timesteps 3690.
Path 413 | total_timesteps 3697.
Path 414 | total_timesteps 3704.
Path 415 | total_timesteps 3713.
Path 416 | total_timesteps 3721.
Path 417 | total_timesteps 3731.
Path 418 | total_timesteps 3741.
Path 419 | total_timesteps 3749.
Path 420 | total_timesteps 3755.
Path 421 | total_timesteps 3766.
Path 422 | total_timesteps 3776.
Path 423 | total_timesteps 3784.
Path 424 | total_timesteps 3791.
Path 425 | total_timesteps 3802.
Path 426 | total_timesteps 3811.
Path 427 | total_timesteps 3819.
Path 428 | total_timesteps 3828.
Path 429 | total_timesteps 3837.
Path 430 | total_timesteps 3861.
Path 431 | total_timesteps 3869.
Path 432 | total_timesteps 3879.
Path 433 | total_timesteps 3887.
Path 434 | total_timesteps 3896.
Path 435 | total_timesteps 3903.
Path 436 | total_timesteps 3910.
Path 437 | total_timesteps 3917.
Path 438 | total_timesteps 3925.
Path 439 | total_timesteps 3934.
Path 440 | total_timesteps 3946.
Path 441 | total_timesteps 3957.
Path 442 | total_timesteps 3965.
Path 443 | total_timesteps 3975.
Path 444 | total_timesteps 3982.
Path 445 | total_timesteps 3991.
Path 446 | total_timesteps 4002.
Path 447 | total_timesteps 4015.
Path 448 | total_timesteps 4025.
Path 449 | total_timesteps 4035.
Path 450 | total_timesteps 4041.
Path 451 | total_timesteps 4050.
Path 452 | total_timesteps 4058.
Path 453 | total_timesteps 4066.
Path 454 | total_timesteps 4076.
Path 455 | total_timesteps 4088.
Path 456 | total_timesteps 4095.
Path 457 | total_timesteps 4103.
Path 458 | total_timesteps 4113.
Path 459 | total_timesteps 4122.
Path 460 | total_timesteps 4130.
Path 461 | total_timesteps 4138.
Path 462 | total_timesteps 4147.
Path 463 | total_timesteps 4154.
Path 464 | total_timesteps 4161.
Path 465 | total_timesteps 4169.
Path 466 | total_timesteps 4175.
Path 467 | total_timesteps 4188.
Path 468 | total_timesteps 4199.
Path 469 | total_timesteps 4206.
Path 470 | total_timesteps 4217.
Path 471 | total_timesteps 4227.
Path 472 | total_timesteps 4237.
Path 473 | total_timesteps 4251.
Path 474 | total_timesteps 4260.
Path 475 | total_timesteps 4267.
Path 476 | total_timesteps 4276.
Path 477 | total_timesteps 4285.
Path 478 | total_timesteps 4293.
Path 479 | total_timesteps 4300.
Path 480 | total_timesteps 4308.
Path 481 | total_timesteps 4316.
Path 482 | total_timesteps 4334.
Path 483 | total_timesteps 4345.
Path 484 | total_timesteps 4352.
Path 485 | total_timesteps 4363.
Path 486 | total_timesteps 4372.
Path 487 | total_timesteps 4380.
Path 488 | total_timesteps 4387.
Path 489 | total_timesteps 4397.
Path 490 | total_timesteps 4406.
Path 491 | total_timesteps 4413.
Path 492 | total_timesteps 4420.
Path 493 | total_timesteps 4429.
Path 494 | total_timesteps 4437.
Path 495 | total_timesteps 4445.
Path 496 | total_timesteps 4452.
Path 497 | total_timesteps 4462.
Path 498 | total_timesteps 4470.
Path 499 | total_timesteps 4478.
Path 500 | total_timesteps 4485.
Path 501 | total_timesteps 4492.
Path 502 | total_timesteps 4500.
Path 503 | total_timesteps 4507.
Path 504 | total_timesteps 4516.
Path 505 | total_timesteps 4523.
Path 506 | total_timesteps 4533.
Path 507 | total_timesteps 4540.
Path 508 | total_timesteps 4548.
Path 509 | total_timesteps 4560.
Path 510 | total_timesteps 4568.
Path 511 | total_timesteps 4579.
Path 512 | total_timesteps 4586.
Path 513 | total_timesteps 4600.
Path 514 | total_timesteps 4610.
Path 515 | total_timesteps 4619.
Path 516 | total_timesteps 4626.
Path 517 | total_timesteps 4633.
Path 518 | total_timesteps 4641.
Path 519 | total_timesteps 4651.
Path 520 | total_timesteps 4661.
Path 521 | total_timesteps 4670.
Path 522 | total_timesteps 4678.
Path 523 | total_timesteps 4686.
Path 524 | total_timesteps 4695.
Path 525 | total_timesteps 4704.
Path 526 | total_timesteps 4714.
Path 527 | total_timesteps 4721.
Path 528 | total_timesteps 4729.
Path 529 | total_timesteps 4742.
Path 530 | total_timesteps 4750.
Path 531 | total_timesteps 4757.
Path 532 | total_timesteps 4767.
Path 533 | total_timesteps 4780.
Path 534 | total_timesteps 4792.
Path 535 | total_timesteps 4804.
Path 536 | total_timesteps 4813.
Path 537 | total_timesteps 4826.
Path 538 | total_timesteps 4835.
Path 539 | total_timesteps 4844.
Path 540 | total_timesteps 4851.
Path 541 | total_timesteps 4858.
Path 542 | total_timesteps 4867.
Path 543 | total_timesteps 4875.
Path 544 | total_timesteps 4886.
Path 545 | total_timesteps 4895.
Path 546 | total_timesteps 4906.
Path 547 | total_timesteps 4914.
Path 548 | total_timesteps 4921.
Path 549 | total_timesteps 4929.
Path 550 | total_timesteps 4937.
Path 551 | total_timesteps 4947.
Path 552 | total_timesteps 4960.
Path 553 | total_timesteps 4971.
Path 554 | total_timesteps 4980.
Path 555 | total_timesteps 4989.
Path 556 | total_timesteps 4996.
Path 557 | total_timesteps 5003.
Path 558 | total_timesteps 5011.
Path 559 | total_timesteps 5020.
Path 560 | total_timesteps 5030.
Path 561 | total_timesteps 5038.
Path 562 | total_timesteps 5047.
Path 563 | total_timesteps 5057.
Path 564 | total_timesteps 5065.
Path 565 | total_timesteps 5075.
Path 566 | total_timesteps 5082.
Path 567 | total_timesteps 5089.
Path 568 | total_timesteps 5098.
Path 569 | total_timesteps 5110.
Path 570 | total_timesteps 5117.
Path 571 | total_timesteps 5125.
Path 572 | total_timesteps 5134.
Path 573 | total_timesteps 5142.
Path 574 | total_timesteps 5149.
Path 575 | total_timesteps 5158.
Path 576 | total_timesteps 5165.
Path 577 | total_timesteps 5179.
Path 578 | total_timesteps 5187.
Path 579 | total_timesteps 5204.
Path 580 | total_timesteps 5212.
Path 581 | total_timesteps 5220.
Path 582 | total_timesteps 5227.
Path 583 | total_timesteps 5238.
Path 584 | total_timesteps 5245.
Path 585 | total_timesteps 5254.
Path 586 | total_timesteps 5270.
Path 587 | total_timesteps 5282.
Path 588 | total_timesteps 5288.
Path 589 | total_timesteps 5296.
Path 590 | total_timesteps 5307.
Path 591 | total_timesteps 5316.
Path 592 | total_timesteps 5323.
Path 593 | total_timesteps 5334.
Path 594 | total_timesteps 5345.
Path 595 | total_timesteps 5352.
Path 596 | total_timesteps 5359.
Path 597 | total_timesteps 5367.
Path 598 | total_timesteps 5382.
Path 599 | total_timesteps 5390.
Path 600 | total_timesteps 5400.
Path 601 | total_timesteps 5407.
Path 602 | total_timesteps 5418.
Path 603 | total_timesteps 5426.
Path 604 | total_timesteps 5433.
Path 605 | total_timesteps 5442.
Path 606 | total_timesteps 5449.
Path 607 | total_timesteps 5458.
Path 608 | total_timesteps 5465.
Path 609 | total_timesteps 5477.
Path 610 | total_timesteps 5486.
Path 611 | total_timesteps 5497.
Path 612 | total_timesteps 5506.
Path 613 | total_timesteps 5517.
Path 614 | total_timesteps 5524.
Path 615 | total_timesteps 5531.
Path 616 | total_timesteps 5537.
Path 617 | total_timesteps 5544.
Path 618 | total_timesteps 5551.
Path 619 | total_timesteps 5557.
Path 620 | total_timesteps 5569.
Path 621 | total_timesteps 5578.
Path 622 | total_timesteps 5586.
Path 623 | total_timesteps 5595.
Path 624 | total_timesteps 5602.
Path 625 | total_timesteps 5612.
Path 626 | total_timesteps 5622.
Path 627 | total_timesteps 5631.
Path 628 | total_timesteps 5639.
Path 629 | total_timesteps 5648.
Path 630 | total_timesteps 5655.
Path 631 | total_timesteps 5666.
Path 632 | total_timesteps 5678.
Path 633 | total_timesteps 5688.
Path 634 | total_timesteps 5697.
Path 635 | total_timesteps 5707.
Path 636 | total_timesteps 5714.
Path 637 | total_timesteps 5723.
Path 638 | total_timesteps 5729.
Path 639 | total_timesteps 5736.
Path 640 | total_timesteps 5745.
Path 641 | total_timesteps 5753.
Path 642 | total_timesteps 5766.
Path 643 | total_timesteps 5772.
Path 644 | total_timesteps 5781.
Path 645 | total_timesteps 5789.
Path 646 | total_timesteps 5798.
Path 647 | total_timesteps 5807.
Path 648 | total_timesteps 5818.
Path 649 | total_timesteps 5826.
Path 650 | total_timesteps 5835.
Path 651 | total_timesteps 5843.
Path 652 | total_timesteps 5852.
Path 653 | total_timesteps 5865.
Path 654 | total_timesteps 5872.
Path 655 | total_timesteps 5878.
Path 656 | total_timesteps 5887.
Path 657 | total_timesteps 5893.
Path 658 | total_timesteps 5906.
Path 659 | total_timesteps 5918.
Path 660 | total_timesteps 5925.
Path 661 | total_timesteps 5933.
Path 662 | total_timesteps 5941.
Path 663 | total_timesteps 5950.
Path 664 | total_timesteps 5956.
Path 665 | total_timesteps 5967.
Path 666 | total_timesteps 5983.
Path 667 | total_timesteps 5993.
Path 668 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.05    |
| Iteration     | 19       |
| MaximumReturn | -1.73    |
| MinimumReturn | -17.7    |
| TotalSamples  | 84078    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.0028968271799385548
Validation loss = 0.0021964760962873697
Validation loss = 0.0021695310715585947
Validation loss = 0.0020950976759195328
Validation loss = 0.002088779117912054
Validation loss = 0.0021758233197033405
Validation loss = 0.0023744627833366394
Validation loss = 0.0021052416414022446
Validation loss = 0.0021957014687359333
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 50.
Path 7 | total_timesteps 58.
Path 8 | total_timesteps 70.
Path 9 | total_timesteps 81.
Path 10 | total_timesteps 88.
Path 11 | total_timesteps 97.
Path 12 | total_timesteps 105.
Path 13 | total_timesteps 112.
Path 14 | total_timesteps 120.
Path 15 | total_timesteps 128.
Path 16 | total_timesteps 136.
Path 17 | total_timesteps 146.
Path 18 | total_timesteps 154.
Path 19 | total_timesteps 160.
Path 20 | total_timesteps 168.
Path 21 | total_timesteps 178.
Path 22 | total_timesteps 187.
Path 23 | total_timesteps 195.
Path 24 | total_timesteps 203.
Path 25 | total_timesteps 212.
Path 26 | total_timesteps 221.
Path 27 | total_timesteps 230.
Path 28 | total_timesteps 240.
Path 29 | total_timesteps 246.
Path 30 | total_timesteps 254.
Path 31 | total_timesteps 263.
Path 32 | total_timesteps 273.
Path 33 | total_timesteps 280.
Path 34 | total_timesteps 291.
Path 35 | total_timesteps 298.
Path 36 | total_timesteps 307.
Path 37 | total_timesteps 316.
Path 38 | total_timesteps 322.
Path 39 | total_timesteps 328.
Path 40 | total_timesteps 336.
Path 41 | total_timesteps 345.
Path 42 | total_timesteps 352.
Path 43 | total_timesteps 360.
Path 44 | total_timesteps 368.
Path 45 | total_timesteps 379.
Path 46 | total_timesteps 386.
Path 47 | total_timesteps 393.
Path 48 | total_timesteps 408.
Path 49 | total_timesteps 417.
Path 50 | total_timesteps 434.
Path 51 | total_timesteps 445.
Path 52 | total_timesteps 454.
Path 53 | total_timesteps 464.
Path 54 | total_timesteps 473.
Path 55 | total_timesteps 488.
Path 56 | total_timesteps 499.
Path 57 | total_timesteps 506.
Path 58 | total_timesteps 515.
Path 59 | total_timesteps 524.
Path 60 | total_timesteps 532.
Path 61 | total_timesteps 543.
Path 62 | total_timesteps 550.
Path 63 | total_timesteps 557.
Path 64 | total_timesteps 565.
Path 65 | total_timesteps 577.
Path 66 | total_timesteps 584.
Path 67 | total_timesteps 594.
Path 68 | total_timesteps 607.
Path 69 | total_timesteps 616.
Path 70 | total_timesteps 622.
Path 71 | total_timesteps 631.
Path 72 | total_timesteps 639.
Path 73 | total_timesteps 647.
Path 74 | total_timesteps 654.
Path 75 | total_timesteps 662.
Path 76 | total_timesteps 674.
Path 77 | total_timesteps 683.
Path 78 | total_timesteps 693.
Path 79 | total_timesteps 700.
Path 80 | total_timesteps 709.
Path 81 | total_timesteps 720.
Path 82 | total_timesteps 728.
Path 83 | total_timesteps 736.
Path 84 | total_timesteps 743.
Path 85 | total_timesteps 750.
Path 86 | total_timesteps 762.
Path 87 | total_timesteps 770.
Path 88 | total_timesteps 778.
Path 89 | total_timesteps 785.
Path 90 | total_timesteps 795.
Path 91 | total_timesteps 804.
Path 92 | total_timesteps 813.
Path 93 | total_timesteps 821.
Path 94 | total_timesteps 828.
Path 95 | total_timesteps 836.
Path 96 | total_timesteps 845.
Path 97 | total_timesteps 857.
Path 98 | total_timesteps 865.
Path 99 | total_timesteps 873.
Path 100 | total_timesteps 881.
Path 101 | total_timesteps 891.
Path 102 | total_timesteps 902.
Path 103 | total_timesteps 912.
Path 104 | total_timesteps 925.
Path 105 | total_timesteps 936.
Path 106 | total_timesteps 948.
Path 107 | total_timesteps 955.
Path 108 | total_timesteps 964.
Path 109 | total_timesteps 971.
Path 110 | total_timesteps 982.
Path 111 | total_timesteps 989.
Path 112 | total_timesteps 1001.
Path 113 | total_timesteps 1011.
Path 114 | total_timesteps 1023.
Path 115 | total_timesteps 1037.
Path 116 | total_timesteps 1044.
Path 117 | total_timesteps 1055.
Path 118 | total_timesteps 1063.
Path 119 | total_timesteps 1078.
Path 120 | total_timesteps 1087.
Path 121 | total_timesteps 1094.
Path 122 | total_timesteps 1100.
Path 123 | total_timesteps 1108.
Path 124 | total_timesteps 1118.
Path 125 | total_timesteps 1129.
Path 126 | total_timesteps 1135.
Path 127 | total_timesteps 1145.
Path 128 | total_timesteps 1155.
Path 129 | total_timesteps 1162.
Path 130 | total_timesteps 1171.
Path 131 | total_timesteps 1182.
Path 132 | total_timesteps 1190.
Path 133 | total_timesteps 1196.
Path 134 | total_timesteps 1203.
Path 135 | total_timesteps 1211.
Path 136 | total_timesteps 1228.
Path 137 | total_timesteps 1241.
Path 138 | total_timesteps 1250.
Path 139 | total_timesteps 1260.
Path 140 | total_timesteps 1266.
Path 141 | total_timesteps 1273.
Path 142 | total_timesteps 1282.
Path 143 | total_timesteps 1292.
Path 144 | total_timesteps 1300.
Path 145 | total_timesteps 1311.
Path 146 | total_timesteps 1320.
Path 147 | total_timesteps 1331.
Path 148 | total_timesteps 1338.
Path 149 | total_timesteps 1344.
Path 150 | total_timesteps 1350.
Path 151 | total_timesteps 1360.
Path 152 | total_timesteps 1369.
Path 153 | total_timesteps 1380.
Path 154 | total_timesteps 1388.
Path 155 | total_timesteps 1398.
Path 156 | total_timesteps 1405.
Path 157 | total_timesteps 1413.
Path 158 | total_timesteps 1420.
Path 159 | total_timesteps 1428.
Path 160 | total_timesteps 1437.
Path 161 | total_timesteps 1445.
Path 162 | total_timesteps 1453.
Path 163 | total_timesteps 1467.
Path 164 | total_timesteps 1480.
Path 165 | total_timesteps 1487.
Path 166 | total_timesteps 1503.
Path 167 | total_timesteps 1511.
Path 168 | total_timesteps 1518.
Path 169 | total_timesteps 1526.
Path 170 | total_timesteps 1536.
Path 171 | total_timesteps 1543.
Path 172 | total_timesteps 1551.
Path 173 | total_timesteps 1559.
Path 174 | total_timesteps 1565.
Path 175 | total_timesteps 1579.
Path 176 | total_timesteps 1587.
Path 177 | total_timesteps 1594.
Path 178 | total_timesteps 1605.
Path 179 | total_timesteps 1612.
Path 180 | total_timesteps 1625.
Path 181 | total_timesteps 1633.
Path 182 | total_timesteps 1645.
Path 183 | total_timesteps 1654.
Path 184 | total_timesteps 1664.
Path 185 | total_timesteps 1672.
Path 186 | total_timesteps 1680.
Path 187 | total_timesteps 1687.
Path 188 | total_timesteps 1698.
Path 189 | total_timesteps 1709.
Path 190 | total_timesteps 1716.
Path 191 | total_timesteps 1722.
Path 192 | total_timesteps 1730.
Path 193 | total_timesteps 1739.
Path 194 | total_timesteps 1746.
Path 195 | total_timesteps 1753.
Path 196 | total_timesteps 1761.
Path 197 | total_timesteps 1774.
Path 198 | total_timesteps 1781.
Path 199 | total_timesteps 1789.
Path 200 | total_timesteps 1799.
Path 201 | total_timesteps 1807.
Path 202 | total_timesteps 1818.
Path 203 | total_timesteps 1824.
Path 204 | total_timesteps 1834.
Path 205 | total_timesteps 1841.
Path 206 | total_timesteps 1848.
Path 207 | total_timesteps 1856.
Path 208 | total_timesteps 1863.
Path 209 | total_timesteps 1871.
Path 210 | total_timesteps 1878.
Path 211 | total_timesteps 1885.
Path 212 | total_timesteps 1896.
Path 213 | total_timesteps 1903.
Path 214 | total_timesteps 1915.
Path 215 | total_timesteps 1924.
Path 216 | total_timesteps 1930.
Path 217 | total_timesteps 1945.
Path 218 | total_timesteps 1954.
Path 219 | total_timesteps 1964.
Path 220 | total_timesteps 1974.
Path 221 | total_timesteps 1982.
Path 222 | total_timesteps 1991.
Path 223 | total_timesteps 2016.
Path 224 | total_timesteps 2024.
Path 225 | total_timesteps 2031.
Path 226 | total_timesteps 2044.
Path 227 | total_timesteps 2052.
Path 228 | total_timesteps 2060.
Path 229 | total_timesteps 2069.
Path 230 | total_timesteps 2080.
Path 231 | total_timesteps 2088.
Path 232 | total_timesteps 2096.
Path 233 | total_timesteps 2103.
Path 234 | total_timesteps 2112.
Path 235 | total_timesteps 2119.
Path 236 | total_timesteps 2129.
Path 237 | total_timesteps 2138.
Path 238 | total_timesteps 2145.
Path 239 | total_timesteps 2155.
Path 240 | total_timesteps 2162.
Path 241 | total_timesteps 2170.
Path 242 | total_timesteps 2179.
Path 243 | total_timesteps 2190.
Path 244 | total_timesteps 2199.
Path 245 | total_timesteps 2207.
Path 246 | total_timesteps 2215.
Path 247 | total_timesteps 2225.
Path 248 | total_timesteps 2234.
Path 249 | total_timesteps 2243.
Path 250 | total_timesteps 2250.
Path 251 | total_timesteps 2259.
Path 252 | total_timesteps 2267.
Path 253 | total_timesteps 2274.
Path 254 | total_timesteps 2290.
Path 255 | total_timesteps 2297.
Path 256 | total_timesteps 2304.
Path 257 | total_timesteps 2314.
Path 258 | total_timesteps 2321.
Path 259 | total_timesteps 2328.
Path 260 | total_timesteps 2336.
Path 261 | total_timesteps 2344.
Path 262 | total_timesteps 2353.
Path 263 | total_timesteps 2362.
Path 264 | total_timesteps 2370.
Path 265 | total_timesteps 2376.
Path 266 | total_timesteps 2384.
Path 267 | total_timesteps 2393.
Path 268 | total_timesteps 2402.
Path 269 | total_timesteps 2413.
Path 270 | total_timesteps 2421.
Path 271 | total_timesteps 2427.
Path 272 | total_timesteps 2436.
Path 273 | total_timesteps 2443.
Path 274 | total_timesteps 2453.
Path 275 | total_timesteps 2460.
Path 276 | total_timesteps 2468.
Path 277 | total_timesteps 2476.
Path 278 | total_timesteps 2484.
Path 279 | total_timesteps 2490.
Path 280 | total_timesteps 2496.
Path 281 | total_timesteps 2504.
Path 282 | total_timesteps 2514.
Path 283 | total_timesteps 2521.
Path 284 | total_timesteps 2532.
Path 285 | total_timesteps 2540.
Path 286 | total_timesteps 2547.
Path 287 | total_timesteps 2557.
Path 288 | total_timesteps 2564.
Path 289 | total_timesteps 2574.
Path 290 | total_timesteps 2581.
Path 291 | total_timesteps 2588.
Path 292 | total_timesteps 2597.
Path 293 | total_timesteps 2605.
Path 294 | total_timesteps 2617.
Path 295 | total_timesteps 2624.
Path 296 | total_timesteps 2632.
Path 297 | total_timesteps 2641.
Path 298 | total_timesteps 2647.
Path 299 | total_timesteps 2655.
Path 300 | total_timesteps 2662.
Path 301 | total_timesteps 2673.
Path 302 | total_timesteps 2680.
Path 303 | total_timesteps 2688.
Path 304 | total_timesteps 2697.
Path 305 | total_timesteps 2704.
Path 306 | total_timesteps 2712.
Path 307 | total_timesteps 2720.
Path 308 | total_timesteps 2729.
Path 309 | total_timesteps 2740.
Path 310 | total_timesteps 2746.
Path 311 | total_timesteps 2757.
Path 312 | total_timesteps 2765.
Path 313 | total_timesteps 2773.
Path 314 | total_timesteps 2783.
Path 315 | total_timesteps 2791.
Path 316 | total_timesteps 2798.
Path 317 | total_timesteps 2811.
Path 318 | total_timesteps 2818.
Path 319 | total_timesteps 2827.
Path 320 | total_timesteps 2837.
Path 321 | total_timesteps 2847.
Path 322 | total_timesteps 2854.
Path 323 | total_timesteps 2864.
Path 324 | total_timesteps 2877.
Path 325 | total_timesteps 2883.
Path 326 | total_timesteps 2890.
Path 327 | total_timesteps 2898.
Path 328 | total_timesteps 2910.
Path 329 | total_timesteps 2918.
Path 330 | total_timesteps 2929.
Path 331 | total_timesteps 2936.
Path 332 | total_timesteps 2943.
Path 333 | total_timesteps 2951.
Path 334 | total_timesteps 2961.
Path 335 | total_timesteps 2969.
Path 336 | total_timesteps 2975.
Path 337 | total_timesteps 2983.
Path 338 | total_timesteps 2994.
Path 339 | total_timesteps 3003.
Path 340 | total_timesteps 3015.
Path 341 | total_timesteps 3022.
Path 342 | total_timesteps 3029.
Path 343 | total_timesteps 3037.
Path 344 | total_timesteps 3046.
Path 345 | total_timesteps 3054.
Path 346 | total_timesteps 3061.
Path 347 | total_timesteps 3069.
Path 348 | total_timesteps 3080.
Path 349 | total_timesteps 3087.
Path 350 | total_timesteps 3096.
Path 351 | total_timesteps 3103.
Path 352 | total_timesteps 3111.
Path 353 | total_timesteps 3119.
Path 354 | total_timesteps 3128.
Path 355 | total_timesteps 3137.
Path 356 | total_timesteps 3144.
Path 357 | total_timesteps 3153.
Path 358 | total_timesteps 3173.
Path 359 | total_timesteps 3187.
Path 360 | total_timesteps 3200.
Path 361 | total_timesteps 3208.
Path 362 | total_timesteps 3215.
Path 363 | total_timesteps 3222.
Path 364 | total_timesteps 3230.
Path 365 | total_timesteps 3237.
Path 366 | total_timesteps 3245.
Path 367 | total_timesteps 3252.
Path 368 | total_timesteps 3262.
Path 369 | total_timesteps 3270.
Path 370 | total_timesteps 3279.
Path 371 | total_timesteps 3290.
Path 372 | total_timesteps 3300.
Path 373 | total_timesteps 3307.
Path 374 | total_timesteps 3316.
Path 375 | total_timesteps 3323.
Path 376 | total_timesteps 3334.
Path 377 | total_timesteps 3342.
Path 378 | total_timesteps 3349.
Path 379 | total_timesteps 3357.
Path 380 | total_timesteps 3364.
Path 381 | total_timesteps 3371.
Path 382 | total_timesteps 3379.
Path 383 | total_timesteps 3386.
Path 384 | total_timesteps 3392.
Path 385 | total_timesteps 3400.
Path 386 | total_timesteps 3409.
Path 387 | total_timesteps 3417.
Path 388 | total_timesteps 3424.
Path 389 | total_timesteps 3433.
Path 390 | total_timesteps 3442.
Path 391 | total_timesteps 3452.
Path 392 | total_timesteps 3463.
Path 393 | total_timesteps 3470.
Path 394 | total_timesteps 3479.
Path 395 | total_timesteps 3491.
Path 396 | total_timesteps 3498.
Path 397 | total_timesteps 3507.
Path 398 | total_timesteps 3521.
Path 399 | total_timesteps 3529.
Path 400 | total_timesteps 3536.
Path 401 | total_timesteps 3543.
Path 402 | total_timesteps 3553.
Path 403 | total_timesteps 3561.
Path 404 | total_timesteps 3569.
Path 405 | total_timesteps 3580.
Path 406 | total_timesteps 3587.
Path 407 | total_timesteps 3597.
Path 408 | total_timesteps 3604.
Path 409 | total_timesteps 3616.
Path 410 | total_timesteps 3626.
Path 411 | total_timesteps 3635.
Path 412 | total_timesteps 3642.
Path 413 | total_timesteps 3649.
Path 414 | total_timesteps 3661.
Path 415 | total_timesteps 3669.
Path 416 | total_timesteps 3675.
Path 417 | total_timesteps 3683.
Path 418 | total_timesteps 3695.
Path 419 | total_timesteps 3702.
Path 420 | total_timesteps 3709.
Path 421 | total_timesteps 3719.
Path 422 | total_timesteps 3725.
Path 423 | total_timesteps 3736.
Path 424 | total_timesteps 3743.
Path 425 | total_timesteps 3753.
Path 426 | total_timesteps 3761.
Path 427 | total_timesteps 3771.
Path 428 | total_timesteps 3778.
Path 429 | total_timesteps 3786.
Path 430 | total_timesteps 3792.
Path 431 | total_timesteps 3803.
Path 432 | total_timesteps 3813.
Path 433 | total_timesteps 3820.
Path 434 | total_timesteps 3827.
Path 435 | total_timesteps 3835.
Path 436 | total_timesteps 3842.
Path 437 | total_timesteps 3851.
Path 438 | total_timesteps 3866.
Path 439 | total_timesteps 3872.
Path 440 | total_timesteps 3882.
Path 441 | total_timesteps 3892.
Path 442 | total_timesteps 3901.
Path 443 | total_timesteps 3911.
Path 444 | total_timesteps 3918.
Path 445 | total_timesteps 3924.
Path 446 | total_timesteps 3933.
Path 447 | total_timesteps 3939.
Path 448 | total_timesteps 3947.
Path 449 | total_timesteps 3955.
Path 450 | total_timesteps 3963.
Path 451 | total_timesteps 3969.
Path 452 | total_timesteps 3978.
Path 453 | total_timesteps 3985.
Path 454 | total_timesteps 3992.
Path 455 | total_timesteps 4002.
Path 456 | total_timesteps 4011.
Path 457 | total_timesteps 4019.
Path 458 | total_timesteps 4027.
Path 459 | total_timesteps 4034.
Path 460 | total_timesteps 4045.
Path 461 | total_timesteps 4060.
Path 462 | total_timesteps 4069.
Path 463 | total_timesteps 4077.
Path 464 | total_timesteps 4086.
Path 465 | total_timesteps 4094.
Path 466 | total_timesteps 4103.
Path 467 | total_timesteps 4109.
Path 468 | total_timesteps 4116.
Path 469 | total_timesteps 4123.
Path 470 | total_timesteps 4132.
Path 471 | total_timesteps 4141.
Path 472 | total_timesteps 4149.
Path 473 | total_timesteps 4157.
Path 474 | total_timesteps 4174.
Path 475 | total_timesteps 4182.
Path 476 | total_timesteps 4193.
Path 477 | total_timesteps 4202.
Path 478 | total_timesteps 4210.
Path 479 | total_timesteps 4218.
Path 480 | total_timesteps 4226.
Path 481 | total_timesteps 4238.
Path 482 | total_timesteps 4245.
Path 483 | total_timesteps 4256.
Path 484 | total_timesteps 4267.
Path 485 | total_timesteps 4274.
Path 486 | total_timesteps 4280.
Path 487 | total_timesteps 4288.
Path 488 | total_timesteps 4298.
Path 489 | total_timesteps 4306.
Path 490 | total_timesteps 4315.
Path 491 | total_timesteps 4322.
Path 492 | total_timesteps 4331.
Path 493 | total_timesteps 4338.
Path 494 | total_timesteps 4344.
Path 495 | total_timesteps 4351.
Path 496 | total_timesteps 4358.
Path 497 | total_timesteps 4367.
Path 498 | total_timesteps 4382.
Path 499 | total_timesteps 4390.
Path 500 | total_timesteps 4398.
Path 501 | total_timesteps 4407.
Path 502 | total_timesteps 4415.
Path 503 | total_timesteps 4426.
Path 504 | total_timesteps 4435.
Path 505 | total_timesteps 4442.
Path 506 | total_timesteps 4451.
Path 507 | total_timesteps 4460.
Path 508 | total_timesteps 4467.
Path 509 | total_timesteps 4474.
Path 510 | total_timesteps 4485.
Path 511 | total_timesteps 4496.
Path 512 | total_timesteps 4503.
Path 513 | total_timesteps 4511.
Path 514 | total_timesteps 4519.
Path 515 | total_timesteps 4534.
Path 516 | total_timesteps 4545.
Path 517 | total_timesteps 4557.
Path 518 | total_timesteps 4564.
Path 519 | total_timesteps 4576.
Path 520 | total_timesteps 4588.
Path 521 | total_timesteps 4599.
Path 522 | total_timesteps 4605.
Path 523 | total_timesteps 4614.
Path 524 | total_timesteps 4622.
Path 525 | total_timesteps 4631.
Path 526 | total_timesteps 4638.
Path 527 | total_timesteps 4649.
Path 528 | total_timesteps 4660.
Path 529 | total_timesteps 4668.
Path 530 | total_timesteps 4676.
Path 531 | total_timesteps 4688.
Path 532 | total_timesteps 4696.
Path 533 | total_timesteps 4706.
Path 534 | total_timesteps 4715.
Path 535 | total_timesteps 4723.
Path 536 | total_timesteps 4731.
Path 537 | total_timesteps 4737.
Path 538 | total_timesteps 4744.
Path 539 | total_timesteps 4755.
Path 540 | total_timesteps 4763.
Path 541 | total_timesteps 4770.
Path 542 | total_timesteps 4777.
Path 543 | total_timesteps 4786.
Path 544 | total_timesteps 4795.
Path 545 | total_timesteps 4803.
Path 546 | total_timesteps 4810.
Path 547 | total_timesteps 4819.
Path 548 | total_timesteps 4825.
Path 549 | total_timesteps 4832.
Path 550 | total_timesteps 4841.
Path 551 | total_timesteps 4849.
Path 552 | total_timesteps 4858.
Path 553 | total_timesteps 4866.
Path 554 | total_timesteps 4878.
Path 555 | total_timesteps 4886.
Path 556 | total_timesteps 4898.
Path 557 | total_timesteps 4906.
Path 558 | total_timesteps 4912.
Path 559 | total_timesteps 4920.
Path 560 | total_timesteps 4929.
Path 561 | total_timesteps 4940.
Path 562 | total_timesteps 4950.
Path 563 | total_timesteps 4958.
Path 564 | total_timesteps 4971.
Path 565 | total_timesteps 4981.
Path 566 | total_timesteps 4991.
Path 567 | total_timesteps 4997.
Path 568 | total_timesteps 5007.
Path 569 | total_timesteps 5014.
Path 570 | total_timesteps 5023.
Path 571 | total_timesteps 5029.
Path 572 | total_timesteps 5037.
Path 573 | total_timesteps 5048.
Path 574 | total_timesteps 5058.
Path 575 | total_timesteps 5068.
Path 576 | total_timesteps 5077.
Path 577 | total_timesteps 5084.
Path 578 | total_timesteps 5091.
Path 579 | total_timesteps 5098.
Path 580 | total_timesteps 5105.
Path 581 | total_timesteps 5114.
Path 582 | total_timesteps 5124.
Path 583 | total_timesteps 5133.
Path 584 | total_timesteps 5144.
Path 585 | total_timesteps 5153.
Path 586 | total_timesteps 5162.
Path 587 | total_timesteps 5170.
Path 588 | total_timesteps 5178.
Path 589 | total_timesteps 5186.
Path 590 | total_timesteps 5194.
Path 591 | total_timesteps 5201.
Path 592 | total_timesteps 5209.
Path 593 | total_timesteps 5218.
Path 594 | total_timesteps 5225.
Path 595 | total_timesteps 5232.
Path 596 | total_timesteps 5239.
Path 597 | total_timesteps 5248.
Path 598 | total_timesteps 5259.
Path 599 | total_timesteps 5267.
Path 600 | total_timesteps 5277.
Path 601 | total_timesteps 5287.
Path 602 | total_timesteps 5294.
Path 603 | total_timesteps 5301.
Path 604 | total_timesteps 5307.
Path 605 | total_timesteps 5314.
Path 606 | total_timesteps 5326.
Path 607 | total_timesteps 5333.
Path 608 | total_timesteps 5342.
Path 609 | total_timesteps 5353.
Path 610 | total_timesteps 5361.
Path 611 | total_timesteps 5368.
Path 612 | total_timesteps 5378.
Path 613 | total_timesteps 5386.
Path 614 | total_timesteps 5395.
Path 615 | total_timesteps 5403.
Path 616 | total_timesteps 5415.
Path 617 | total_timesteps 5423.
Path 618 | total_timesteps 5436.
Path 619 | total_timesteps 5443.
Path 620 | total_timesteps 5450.
Path 621 | total_timesteps 5457.
Path 622 | total_timesteps 5465.
Path 623 | total_timesteps 5474.
Path 624 | total_timesteps 5480.
Path 625 | total_timesteps 5488.
Path 626 | total_timesteps 5495.
Path 627 | total_timesteps 5508.
Path 628 | total_timesteps 5516.
Path 629 | total_timesteps 5524.
Path 630 | total_timesteps 5537.
Path 631 | total_timesteps 5545.
Path 632 | total_timesteps 5552.
Path 633 | total_timesteps 5559.
Path 634 | total_timesteps 5568.
Path 635 | total_timesteps 5577.
Path 636 | total_timesteps 5585.
Path 637 | total_timesteps 5592.
Path 638 | total_timesteps 5598.
Path 639 | total_timesteps 5606.
Path 640 | total_timesteps 5618.
Path 641 | total_timesteps 5627.
Path 642 | total_timesteps 5641.
Path 643 | total_timesteps 5648.
Path 644 | total_timesteps 5656.
Path 645 | total_timesteps 5663.
Path 646 | total_timesteps 5674.
Path 647 | total_timesteps 5682.
Path 648 | total_timesteps 5692.
Path 649 | total_timesteps 5699.
Path 650 | total_timesteps 5706.
Path 651 | total_timesteps 5716.
Path 652 | total_timesteps 5725.
Path 653 | total_timesteps 5733.
Path 654 | total_timesteps 5740.
Path 655 | total_timesteps 5748.
Path 656 | total_timesteps 5755.
Path 657 | total_timesteps 5770.
Path 658 | total_timesteps 5781.
Path 659 | total_timesteps 5791.
Path 660 | total_timesteps 5801.
Path 661 | total_timesteps 5809.
Path 662 | total_timesteps 5816.
Path 663 | total_timesteps 5822.
Path 664 | total_timesteps 5830.
Path 665 | total_timesteps 5841.
Path 666 | total_timesteps 5848.
Path 667 | total_timesteps 5855.
Path 668 | total_timesteps 5863.
Path 669 | total_timesteps 5871.
Path 670 | total_timesteps 5880.
Path 671 | total_timesteps 5888.
Path 672 | total_timesteps 5898.
Path 673 | total_timesteps 5908.
Path 674 | total_timesteps 5915.
Path 675 | total_timesteps 5923.
Path 676 | total_timesteps 5930.
Path 677 | total_timesteps 5938.
Path 678 | total_timesteps 5946.
Path 679 | total_timesteps 5955.
Path 680 | total_timesteps 5962.
Path 681 | total_timesteps 5971.
Path 682 | total_timesteps 5979.
Path 683 | total_timesteps 5990.
Path 684 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.03    |
| Iteration     | 20       |
| MaximumReturn | 0.647    |
| MinimumReturn | -17.4    |
| TotalSamples  | 88082    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.0021101476158946753
Validation loss = 0.0019296082900837064
Validation loss = 0.0022226169239729643
Validation loss = 0.0020384208764880896
Validation loss = 0.001996767707169056
Validation loss = 0.001946627045981586
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 53.
Path 7 | total_timesteps 62.
Path 8 | total_timesteps 73.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 91.
Path 11 | total_timesteps 98.
Path 12 | total_timesteps 108.
Path 13 | total_timesteps 117.
Path 14 | total_timesteps 125.
Path 15 | total_timesteps 134.
Path 16 | total_timesteps 143.
Path 17 | total_timesteps 149.
Path 18 | total_timesteps 156.
Path 19 | total_timesteps 164.
Path 20 | total_timesteps 172.
Path 21 | total_timesteps 181.
Path 22 | total_timesteps 188.
Path 23 | total_timesteps 194.
Path 24 | total_timesteps 202.
Path 25 | total_timesteps 209.
Path 26 | total_timesteps 217.
Path 27 | total_timesteps 225.
Path 28 | total_timesteps 232.
Path 29 | total_timesteps 239.
Path 30 | total_timesteps 245.
Path 31 | total_timesteps 253.
Path 32 | total_timesteps 263.
Path 33 | total_timesteps 272.
Path 34 | total_timesteps 279.
Path 35 | total_timesteps 289.
Path 36 | total_timesteps 296.
Path 37 | total_timesteps 305.
Path 38 | total_timesteps 312.
Path 39 | total_timesteps 320.
Path 40 | total_timesteps 330.
Path 41 | total_timesteps 338.
Path 42 | total_timesteps 346.
Path 43 | total_timesteps 355.
Path 44 | total_timesteps 364.
Path 45 | total_timesteps 372.
Path 46 | total_timesteps 380.
Path 47 | total_timesteps 386.
Path 48 | total_timesteps 395.
Path 49 | total_timesteps 402.
Path 50 | total_timesteps 414.
Path 51 | total_timesteps 426.
Path 52 | total_timesteps 436.
Path 53 | total_timesteps 443.
Path 54 | total_timesteps 451.
Path 55 | total_timesteps 459.
Path 56 | total_timesteps 469.
Path 57 | total_timesteps 480.
Path 58 | total_timesteps 487.
Path 59 | total_timesteps 494.
Path 60 | total_timesteps 501.
Path 61 | total_timesteps 510.
Path 62 | total_timesteps 518.
Path 63 | total_timesteps 527.
Path 64 | total_timesteps 534.
Path 65 | total_timesteps 541.
Path 66 | total_timesteps 548.
Path 67 | total_timesteps 558.
Path 68 | total_timesteps 566.
Path 69 | total_timesteps 572.
Path 70 | total_timesteps 578.
Path 71 | total_timesteps 588.
Path 72 | total_timesteps 595.
Path 73 | total_timesteps 604.
Path 74 | total_timesteps 612.
Path 75 | total_timesteps 620.
Path 76 | total_timesteps 631.
Path 77 | total_timesteps 637.
Path 78 | total_timesteps 649.
Path 79 | total_timesteps 656.
Path 80 | total_timesteps 666.
Path 81 | total_timesteps 674.
Path 82 | total_timesteps 681.
Path 83 | total_timesteps 689.
Path 84 | total_timesteps 699.
Path 85 | total_timesteps 706.
Path 86 | total_timesteps 715.
Path 87 | total_timesteps 726.
Path 88 | total_timesteps 735.
Path 89 | total_timesteps 747.
Path 90 | total_timesteps 757.
Path 91 | total_timesteps 767.
Path 92 | total_timesteps 775.
Path 93 | total_timesteps 784.
Path 94 | total_timesteps 798.
Path 95 | total_timesteps 805.
Path 96 | total_timesteps 815.
Path 97 | total_timesteps 822.
Path 98 | total_timesteps 829.
Path 99 | total_timesteps 837.
Path 100 | total_timesteps 844.
Path 101 | total_timesteps 856.
Path 102 | total_timesteps 865.
Path 103 | total_timesteps 874.
Path 104 | total_timesteps 886.
Path 105 | total_timesteps 899.
Path 106 | total_timesteps 908.
Path 107 | total_timesteps 916.
Path 108 | total_timesteps 926.
Path 109 | total_timesteps 933.
Path 110 | total_timesteps 942.
Path 111 | total_timesteps 957.
Path 112 | total_timesteps 968.
Path 113 | total_timesteps 977.
Path 114 | total_timesteps 984.
Path 115 | total_timesteps 992.
Path 116 | total_timesteps 1000.
Path 117 | total_timesteps 1008.
Path 118 | total_timesteps 1016.
Path 119 | total_timesteps 1023.
Path 120 | total_timesteps 1031.
Path 121 | total_timesteps 1038.
Path 122 | total_timesteps 1045.
Path 123 | total_timesteps 1051.
Path 124 | total_timesteps 1061.
Path 125 | total_timesteps 1068.
Path 126 | total_timesteps 1077.
Path 127 | total_timesteps 1084.
Path 128 | total_timesteps 1092.
Path 129 | total_timesteps 1101.
Path 130 | total_timesteps 1108.
Path 131 | total_timesteps 1117.
Path 132 | total_timesteps 1126.
Path 133 | total_timesteps 1135.
Path 134 | total_timesteps 1145.
Path 135 | total_timesteps 1154.
Path 136 | total_timesteps 1163.
Path 137 | total_timesteps 1170.
Path 138 | total_timesteps 1177.
Path 139 | total_timesteps 1185.
Path 140 | total_timesteps 1195.
Path 141 | total_timesteps 1203.
Path 142 | total_timesteps 1211.
Path 143 | total_timesteps 1220.
Path 144 | total_timesteps 1229.
Path 145 | total_timesteps 1239.
Path 146 | total_timesteps 1247.
Path 147 | total_timesteps 1259.
Path 148 | total_timesteps 1267.
Path 149 | total_timesteps 1274.
Path 150 | total_timesteps 1282.
Path 151 | total_timesteps 1290.
Path 152 | total_timesteps 1300.
Path 153 | total_timesteps 1306.
Path 154 | total_timesteps 1314.
Path 155 | total_timesteps 1322.
Path 156 | total_timesteps 1331.
Path 157 | total_timesteps 1339.
Path 158 | total_timesteps 1346.
Path 159 | total_timesteps 1357.
Path 160 | total_timesteps 1365.
Path 161 | total_timesteps 1376.
Path 162 | total_timesteps 1386.
Path 163 | total_timesteps 1392.
Path 164 | total_timesteps 1402.
Path 165 | total_timesteps 1410.
Path 166 | total_timesteps 1417.
Path 167 | total_timesteps 1425.
Path 168 | total_timesteps 1434.
Path 169 | total_timesteps 1442.
Path 170 | total_timesteps 1449.
Path 171 | total_timesteps 1455.
Path 172 | total_timesteps 1465.
Path 173 | total_timesteps 1474.
Path 174 | total_timesteps 1482.
Path 175 | total_timesteps 1491.
Path 176 | total_timesteps 1499.
Path 177 | total_timesteps 1509.
Path 178 | total_timesteps 1518.
Path 179 | total_timesteps 1526.
Path 180 | total_timesteps 1536.
Path 181 | total_timesteps 1545.
Path 182 | total_timesteps 1552.
Path 183 | total_timesteps 1559.
Path 184 | total_timesteps 1567.
Path 185 | total_timesteps 1574.
Path 186 | total_timesteps 1581.
Path 187 | total_timesteps 1591.
Path 188 | total_timesteps 1601.
Path 189 | total_timesteps 1608.
Path 190 | total_timesteps 1615.
Path 191 | total_timesteps 1626.
Path 192 | total_timesteps 1637.
Path 193 | total_timesteps 1644.
Path 194 | total_timesteps 1651.
Path 195 | total_timesteps 1657.
Path 196 | total_timesteps 1667.
Path 197 | total_timesteps 1681.
Path 198 | total_timesteps 1689.
Path 199 | total_timesteps 1697.
Path 200 | total_timesteps 1709.
Path 201 | total_timesteps 1715.
Path 202 | total_timesteps 1726.
Path 203 | total_timesteps 1734.
Path 204 | total_timesteps 1746.
Path 205 | total_timesteps 1753.
Path 206 | total_timesteps 1763.
Path 207 | total_timesteps 1773.
Path 208 | total_timesteps 1780.
Path 209 | total_timesteps 1790.
Path 210 | total_timesteps 1797.
Path 211 | total_timesteps 1806.
Path 212 | total_timesteps 1814.
Path 213 | total_timesteps 1822.
Path 214 | total_timesteps 1832.
Path 215 | total_timesteps 1840.
Path 216 | total_timesteps 1850.
Path 217 | total_timesteps 1858.
Path 218 | total_timesteps 1869.
Path 219 | total_timesteps 1879.
Path 220 | total_timesteps 1887.
Path 221 | total_timesteps 1893.
Path 222 | total_timesteps 1901.
Path 223 | total_timesteps 1910.
Path 224 | total_timesteps 1920.
Path 225 | total_timesteps 1936.
Path 226 | total_timesteps 1946.
Path 227 | total_timesteps 1954.
Path 228 | total_timesteps 1962.
Path 229 | total_timesteps 1969.
Path 230 | total_timesteps 1979.
Path 231 | total_timesteps 1989.
Path 232 | total_timesteps 2004.
Path 233 | total_timesteps 2010.
Path 234 | total_timesteps 2016.
Path 235 | total_timesteps 2024.
Path 236 | total_timesteps 2034.
Path 237 | total_timesteps 2043.
Path 238 | total_timesteps 2051.
Path 239 | total_timesteps 2064.
Path 240 | total_timesteps 2074.
Path 241 | total_timesteps 2087.
Path 242 | total_timesteps 2095.
Path 243 | total_timesteps 2103.
Path 244 | total_timesteps 2115.
Path 245 | total_timesteps 2125.
Path 246 | total_timesteps 2132.
Path 247 | total_timesteps 2140.
Path 248 | total_timesteps 2149.
Path 249 | total_timesteps 2156.
Path 250 | total_timesteps 2165.
Path 251 | total_timesteps 2173.
Path 252 | total_timesteps 2184.
Path 253 | total_timesteps 2192.
Path 254 | total_timesteps 2200.
Path 255 | total_timesteps 2208.
Path 256 | total_timesteps 2216.
Path 257 | total_timesteps 2223.
Path 258 | total_timesteps 2233.
Path 259 | total_timesteps 2242.
Path 260 | total_timesteps 2250.
Path 261 | total_timesteps 2261.
Path 262 | total_timesteps 2270.
Path 263 | total_timesteps 2278.
Path 264 | total_timesteps 2286.
Path 265 | total_timesteps 2301.
Path 266 | total_timesteps 2310.
Path 267 | total_timesteps 2318.
Path 268 | total_timesteps 2326.
Path 269 | total_timesteps 2334.
Path 270 | total_timesteps 2344.
Path 271 | total_timesteps 2358.
Path 272 | total_timesteps 2370.
Path 273 | total_timesteps 2377.
Path 274 | total_timesteps 2384.
Path 275 | total_timesteps 2391.
Path 276 | total_timesteps 2398.
Path 277 | total_timesteps 2404.
Path 278 | total_timesteps 2415.
Path 279 | total_timesteps 2422.
Path 280 | total_timesteps 2430.
Path 281 | total_timesteps 2441.
Path 282 | total_timesteps 2450.
Path 283 | total_timesteps 2460.
Path 284 | total_timesteps 2467.
Path 285 | total_timesteps 2476.
Path 286 | total_timesteps 2485.
Path 287 | total_timesteps 2495.
Path 288 | total_timesteps 2505.
Path 289 | total_timesteps 2513.
Path 290 | total_timesteps 2528.
Path 291 | total_timesteps 2536.
Path 292 | total_timesteps 2543.
Path 293 | total_timesteps 2550.
Path 294 | total_timesteps 2562.
Path 295 | total_timesteps 2574.
Path 296 | total_timesteps 2583.
Path 297 | total_timesteps 2591.
Path 298 | total_timesteps 2600.
Path 299 | total_timesteps 2609.
Path 300 | total_timesteps 2616.
Path 301 | total_timesteps 2623.
Path 302 | total_timesteps 2638.
Path 303 | total_timesteps 2653.
Path 304 | total_timesteps 2664.
Path 305 | total_timesteps 2671.
Path 306 | total_timesteps 2679.
Path 307 | total_timesteps 2688.
Path 308 | total_timesteps 2698.
Path 309 | total_timesteps 2707.
Path 310 | total_timesteps 2713.
Path 311 | total_timesteps 2721.
Path 312 | total_timesteps 2728.
Path 313 | total_timesteps 2742.
Path 314 | total_timesteps 2749.
Path 315 | total_timesteps 2757.
Path 316 | total_timesteps 2766.
Path 317 | total_timesteps 2776.
Path 318 | total_timesteps 2789.
Path 319 | total_timesteps 2796.
Path 320 | total_timesteps 2804.
Path 321 | total_timesteps 2815.
Path 322 | total_timesteps 2823.
Path 323 | total_timesteps 2831.
Path 324 | total_timesteps 2843.
Path 325 | total_timesteps 2850.
Path 326 | total_timesteps 2857.
Path 327 | total_timesteps 2865.
Path 328 | total_timesteps 2872.
Path 329 | total_timesteps 2882.
Path 330 | total_timesteps 2893.
Path 331 | total_timesteps 2902.
Path 332 | total_timesteps 2914.
Path 333 | total_timesteps 2927.
Path 334 | total_timesteps 2935.
Path 335 | total_timesteps 2941.
Path 336 | total_timesteps 2948.
Path 337 | total_timesteps 2956.
Path 338 | total_timesteps 2965.
Path 339 | total_timesteps 2975.
Path 340 | total_timesteps 2982.
Path 341 | total_timesteps 2989.
Path 342 | total_timesteps 2999.
Path 343 | total_timesteps 3006.
Path 344 | total_timesteps 3014.
Path 345 | total_timesteps 3026.
Path 346 | total_timesteps 3033.
Path 347 | total_timesteps 3043.
Path 348 | total_timesteps 3053.
Path 349 | total_timesteps 3063.
Path 350 | total_timesteps 3072.
Path 351 | total_timesteps 3078.
Path 352 | total_timesteps 3087.
Path 353 | total_timesteps 3095.
Path 354 | total_timesteps 3103.
Path 355 | total_timesteps 3114.
Path 356 | total_timesteps 3120.
Path 357 | total_timesteps 3128.
Path 358 | total_timesteps 3140.
Path 359 | total_timesteps 3149.
Path 360 | total_timesteps 3158.
Path 361 | total_timesteps 3168.
Path 362 | total_timesteps 3177.
Path 363 | total_timesteps 3186.
Path 364 | total_timesteps 3194.
Path 365 | total_timesteps 3201.
Path 366 | total_timesteps 3207.
Path 367 | total_timesteps 3219.
Path 368 | total_timesteps 3230.
Path 369 | total_timesteps 3238.
Path 370 | total_timesteps 3249.
Path 371 | total_timesteps 3257.
Path 372 | total_timesteps 3269.
Path 373 | total_timesteps 3279.
Path 374 | total_timesteps 3288.
Path 375 | total_timesteps 3300.
Path 376 | total_timesteps 3308.
Path 377 | total_timesteps 3317.
Path 378 | total_timesteps 3325.
Path 379 | total_timesteps 3332.
Path 380 | total_timesteps 3339.
Path 381 | total_timesteps 3348.
Path 382 | total_timesteps 3355.
Path 383 | total_timesteps 3362.
Path 384 | total_timesteps 3369.
Path 385 | total_timesteps 3380.
Path 386 | total_timesteps 3388.
Path 387 | total_timesteps 3401.
Path 388 | total_timesteps 3410.
Path 389 | total_timesteps 3418.
Path 390 | total_timesteps 3427.
Path 391 | total_timesteps 3433.
Path 392 | total_timesteps 3441.
Path 393 | total_timesteps 3450.
Path 394 | total_timesteps 3461.
Path 395 | total_timesteps 3469.
Path 396 | total_timesteps 3475.
Path 397 | total_timesteps 3485.
Path 398 | total_timesteps 3494.
Path 399 | total_timesteps 3502.
Path 400 | total_timesteps 3511.
Path 401 | total_timesteps 3534.
Path 402 | total_timesteps 3540.
Path 403 | total_timesteps 3550.
Path 404 | total_timesteps 3557.
Path 405 | total_timesteps 3563.
Path 406 | total_timesteps 3570.
Path 407 | total_timesteps 3579.
Path 408 | total_timesteps 3588.
Path 409 | total_timesteps 3597.
Path 410 | total_timesteps 3604.
Path 411 | total_timesteps 3615.
Path 412 | total_timesteps 3624.
Path 413 | total_timesteps 3632.
Path 414 | total_timesteps 3645.
Path 415 | total_timesteps 3653.
Path 416 | total_timesteps 3662.
Path 417 | total_timesteps 3668.
Path 418 | total_timesteps 3675.
Path 419 | total_timesteps 3682.
Path 420 | total_timesteps 3689.
Path 421 | total_timesteps 3697.
Path 422 | total_timesteps 3707.
Path 423 | total_timesteps 3717.
Path 424 | total_timesteps 3730.
Path 425 | total_timesteps 3739.
Path 426 | total_timesteps 3746.
Path 427 | total_timesteps 3755.
Path 428 | total_timesteps 3764.
Path 429 | total_timesteps 3773.
Path 430 | total_timesteps 3781.
Path 431 | total_timesteps 3790.
Path 432 | total_timesteps 3798.
Path 433 | total_timesteps 3810.
Path 434 | total_timesteps 3818.
Path 435 | total_timesteps 3825.
Path 436 | total_timesteps 3834.
Path 437 | total_timesteps 3840.
Path 438 | total_timesteps 3850.
Path 439 | total_timesteps 3861.
Path 440 | total_timesteps 3871.
Path 441 | total_timesteps 3883.
Path 442 | total_timesteps 3895.
Path 443 | total_timesteps 3904.
Path 444 | total_timesteps 3912.
Path 445 | total_timesteps 3921.
Path 446 | total_timesteps 3929.
Path 447 | total_timesteps 3937.
Path 448 | total_timesteps 3946.
Path 449 | total_timesteps 3956.
Path 450 | total_timesteps 3965.
Path 451 | total_timesteps 3974.
Path 452 | total_timesteps 3988.
Path 453 | total_timesteps 3995.
Path 454 | total_timesteps 4004.
Path 455 | total_timesteps 4011.
Path 456 | total_timesteps 4019.
Path 457 | total_timesteps 4031.
Path 458 | total_timesteps 4040.
Path 459 | total_timesteps 4047.
Path 460 | total_timesteps 4054.
Path 461 | total_timesteps 4065.
Path 462 | total_timesteps 4073.
Path 463 | total_timesteps 4081.
Path 464 | total_timesteps 4092.
Path 465 | total_timesteps 4109.
Path 466 | total_timesteps 4115.
Path 467 | total_timesteps 4126.
Path 468 | total_timesteps 4134.
Path 469 | total_timesteps 4143.
Path 470 | total_timesteps 4151.
Path 471 | total_timesteps 4163.
Path 472 | total_timesteps 4172.
Path 473 | total_timesteps 4184.
Path 474 | total_timesteps 4191.
Path 475 | total_timesteps 4199.
Path 476 | total_timesteps 4205.
Path 477 | total_timesteps 4214.
Path 478 | total_timesteps 4221.
Path 479 | total_timesteps 4230.
Path 480 | total_timesteps 4237.
Path 481 | total_timesteps 4246.
Path 482 | total_timesteps 4255.
Path 483 | total_timesteps 4263.
Path 484 | total_timesteps 4273.
Path 485 | total_timesteps 4279.
Path 486 | total_timesteps 4287.
Path 487 | total_timesteps 4294.
Path 488 | total_timesteps 4304.
Path 489 | total_timesteps 4313.
Path 490 | total_timesteps 4323.
Path 491 | total_timesteps 4332.
Path 492 | total_timesteps 4343.
Path 493 | total_timesteps 4352.
Path 494 | total_timesteps 4364.
Path 495 | total_timesteps 4373.
Path 496 | total_timesteps 4379.
Path 497 | total_timesteps 4387.
Path 498 | total_timesteps 4396.
Path 499 | total_timesteps 4404.
Path 500 | total_timesteps 4412.
Path 501 | total_timesteps 4419.
Path 502 | total_timesteps 4426.
Path 503 | total_timesteps 4434.
Path 504 | total_timesteps 4442.
Path 505 | total_timesteps 4449.
Path 506 | total_timesteps 4455.
Path 507 | total_timesteps 4463.
Path 508 | total_timesteps 4475.
Path 509 | total_timesteps 4483.
Path 510 | total_timesteps 4491.
Path 511 | total_timesteps 4498.
Path 512 | total_timesteps 4505.
Path 513 | total_timesteps 4512.
Path 514 | total_timesteps 4519.
Path 515 | total_timesteps 4530.
Path 516 | total_timesteps 4539.
Path 517 | total_timesteps 4550.
Path 518 | total_timesteps 4560.
Path 519 | total_timesteps 4567.
Path 520 | total_timesteps 4578.
Path 521 | total_timesteps 4585.
Path 522 | total_timesteps 4595.
Path 523 | total_timesteps 4604.
Path 524 | total_timesteps 4612.
Path 525 | total_timesteps 4621.
Path 526 | total_timesteps 4631.
Path 527 | total_timesteps 4638.
Path 528 | total_timesteps 4647.
Path 529 | total_timesteps 4656.
Path 530 | total_timesteps 4665.
Path 531 | total_timesteps 4676.
Path 532 | total_timesteps 4683.
Path 533 | total_timesteps 4692.
Path 534 | total_timesteps 4699.
Path 535 | total_timesteps 4705.
Path 536 | total_timesteps 4713.
Path 537 | total_timesteps 4721.
Path 538 | total_timesteps 4730.
Path 539 | total_timesteps 4740.
Path 540 | total_timesteps 4747.
Path 541 | total_timesteps 4760.
Path 542 | total_timesteps 4767.
Path 543 | total_timesteps 4777.
Path 544 | total_timesteps 4785.
Path 545 | total_timesteps 4792.
Path 546 | total_timesteps 4801.
Path 547 | total_timesteps 4810.
Path 548 | total_timesteps 4817.
Path 549 | total_timesteps 4825.
Path 550 | total_timesteps 4835.
Path 551 | total_timesteps 4842.
Path 552 | total_timesteps 4849.
Path 553 | total_timesteps 4855.
Path 554 | total_timesteps 4863.
Path 555 | total_timesteps 4874.
Path 556 | total_timesteps 4880.
Path 557 | total_timesteps 4889.
Path 558 | total_timesteps 4897.
Path 559 | total_timesteps 4907.
Path 560 | total_timesteps 4915.
Path 561 | total_timesteps 4923.
Path 562 | total_timesteps 4935.
Path 563 | total_timesteps 4946.
Path 564 | total_timesteps 4954.
Path 565 | total_timesteps 4961.
Path 566 | total_timesteps 4969.
Path 567 | total_timesteps 4977.
Path 568 | total_timesteps 4988.
Path 569 | total_timesteps 4998.
Path 570 | total_timesteps 5007.
Path 571 | total_timesteps 5015.
Path 572 | total_timesteps 5022.
Path 573 | total_timesteps 5032.
Path 574 | total_timesteps 5043.
Path 575 | total_timesteps 5052.
Path 576 | total_timesteps 5058.
Path 577 | total_timesteps 5065.
Path 578 | total_timesteps 5072.
Path 579 | total_timesteps 5080.
Path 580 | total_timesteps 5088.
Path 581 | total_timesteps 5095.
Path 582 | total_timesteps 5104.
Path 583 | total_timesteps 5113.
Path 584 | total_timesteps 5121.
Path 585 | total_timesteps 5130.
Path 586 | total_timesteps 5138.
Path 587 | total_timesteps 5146.
Path 588 | total_timesteps 5155.
Path 589 | total_timesteps 5163.
Path 590 | total_timesteps 5172.
Path 591 | total_timesteps 5180.
Path 592 | total_timesteps 5191.
Path 593 | total_timesteps 5197.
Path 594 | total_timesteps 5208.
Path 595 | total_timesteps 5216.
Path 596 | total_timesteps 5225.
Path 597 | total_timesteps 5237.
Path 598 | total_timesteps 5244.
Path 599 | total_timesteps 5251.
Path 600 | total_timesteps 5258.
Path 601 | total_timesteps 5265.
Path 602 | total_timesteps 5273.
Path 603 | total_timesteps 5281.
Path 604 | total_timesteps 5287.
Path 605 | total_timesteps 5294.
Path 606 | total_timesteps 5300.
Path 607 | total_timesteps 5309.
Path 608 | total_timesteps 5316.
Path 609 | total_timesteps 5324.
Path 610 | total_timesteps 5332.
Path 611 | total_timesteps 5340.
Path 612 | total_timesteps 5361.
Path 613 | total_timesteps 5368.
Path 614 | total_timesteps 5380.
Path 615 | total_timesteps 5395.
Path 616 | total_timesteps 5402.
Path 617 | total_timesteps 5409.
Path 618 | total_timesteps 5418.
Path 619 | total_timesteps 5428.
Path 620 | total_timesteps 5438.
Path 621 | total_timesteps 5448.
Path 622 | total_timesteps 5455.
Path 623 | total_timesteps 5464.
Path 624 | total_timesteps 5471.
Path 625 | total_timesteps 5478.
Path 626 | total_timesteps 5486.
Path 627 | total_timesteps 5500.
Path 628 | total_timesteps 5514.
Path 629 | total_timesteps 5523.
Path 630 | total_timesteps 5530.
Path 631 | total_timesteps 5538.
Path 632 | total_timesteps 5545.
Path 633 | total_timesteps 5559.
Path 634 | total_timesteps 5566.
Path 635 | total_timesteps 5576.
Path 636 | total_timesteps 5584.
Path 637 | total_timesteps 5592.
Path 638 | total_timesteps 5604.
Path 639 | total_timesteps 5611.
Path 640 | total_timesteps 5622.
Path 641 | total_timesteps 5629.
Path 642 | total_timesteps 5644.
Path 643 | total_timesteps 5650.
Path 644 | total_timesteps 5657.
Path 645 | total_timesteps 5664.
Path 646 | total_timesteps 5671.
Path 647 | total_timesteps 5679.
Path 648 | total_timesteps 5689.
Path 649 | total_timesteps 5698.
Path 650 | total_timesteps 5705.
Path 651 | total_timesteps 5712.
Path 652 | total_timesteps 5725.
Path 653 | total_timesteps 5733.
Path 654 | total_timesteps 5739.
Path 655 | total_timesteps 5746.
Path 656 | total_timesteps 5755.
Path 657 | total_timesteps 5764.
Path 658 | total_timesteps 5777.
Path 659 | total_timesteps 5787.
Path 660 | total_timesteps 5797.
Path 661 | total_timesteps 5808.
Path 662 | total_timesteps 5817.
Path 663 | total_timesteps 5825.
Path 664 | total_timesteps 5834.
Path 665 | total_timesteps 5842.
Path 666 | total_timesteps 5851.
Path 667 | total_timesteps 5859.
Path 668 | total_timesteps 5872.
Path 669 | total_timesteps 5878.
Path 670 | total_timesteps 5885.
Path 671 | total_timesteps 5896.
Path 672 | total_timesteps 5907.
Path 673 | total_timesteps 5921.
Path 674 | total_timesteps 5932.
Path 675 | total_timesteps 5939.
Path 676 | total_timesteps 5951.
Path 677 | total_timesteps 5960.
Path 678 | total_timesteps 5970.
Path 679 | total_timesteps 5982.
Path 680 | total_timesteps 5990.
Path 681 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.88    |
| Iteration     | 21       |
| MaximumReturn | -1.52    |
| MinimumReturn | -15.2    |
| TotalSamples  | 92088    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0019512743456289172
Validation loss = 0.002100846264511347
Validation loss = 0.0018499192083254457
Validation loss = 0.0019436420407146215
Validation loss = 0.0021249335259199142
Validation loss = 0.00222692615352571
Validation loss = 0.00216308212839067
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 50.
Path 7 | total_timesteps 56.
Path 8 | total_timesteps 66.
Path 9 | total_timesteps 72.
Path 10 | total_timesteps 79.
Path 11 | total_timesteps 88.
Path 12 | total_timesteps 95.
Path 13 | total_timesteps 104.
Path 14 | total_timesteps 111.
Path 15 | total_timesteps 120.
Path 16 | total_timesteps 126.
Path 17 | total_timesteps 136.
Path 18 | total_timesteps 143.
Path 19 | total_timesteps 149.
Path 20 | total_timesteps 159.
Path 21 | total_timesteps 170.
Path 22 | total_timesteps 177.
Path 23 | total_timesteps 189.
Path 24 | total_timesteps 196.
Path 25 | total_timesteps 204.
Path 26 | total_timesteps 216.
Path 27 | total_timesteps 224.
Path 28 | total_timesteps 239.
Path 29 | total_timesteps 247.
Path 30 | total_timesteps 254.
Path 31 | total_timesteps 263.
Path 32 | total_timesteps 270.
Path 33 | total_timesteps 280.
Path 34 | total_timesteps 290.
Path 35 | total_timesteps 300.
Path 36 | total_timesteps 308.
Path 37 | total_timesteps 319.
Path 38 | total_timesteps 332.
Path 39 | total_timesteps 339.
Path 40 | total_timesteps 351.
Path 41 | total_timesteps 360.
Path 42 | total_timesteps 369.
Path 43 | total_timesteps 375.
Path 44 | total_timesteps 382.
Path 45 | total_timesteps 391.
Path 46 | total_timesteps 397.
Path 47 | total_timesteps 405.
Path 48 | total_timesteps 411.
Path 49 | total_timesteps 419.
Path 50 | total_timesteps 428.
Path 51 | total_timesteps 437.
Path 52 | total_timesteps 446.
Path 53 | total_timesteps 452.
Path 54 | total_timesteps 458.
Path 55 | total_timesteps 465.
Path 56 | total_timesteps 472.
Path 57 | total_timesteps 482.
Path 58 | total_timesteps 491.
Path 59 | total_timesteps 501.
Path 60 | total_timesteps 511.
Path 61 | total_timesteps 518.
Path 62 | total_timesteps 526.
Path 63 | total_timesteps 534.
Path 64 | total_timesteps 540.
Path 65 | total_timesteps 549.
Path 66 | total_timesteps 557.
Path 67 | total_timesteps 568.
Path 68 | total_timesteps 575.
Path 69 | total_timesteps 584.
Path 70 | total_timesteps 593.
Path 71 | total_timesteps 606.
Path 72 | total_timesteps 615.
Path 73 | total_timesteps 622.
Path 74 | total_timesteps 629.
Path 75 | total_timesteps 637.
Path 76 | total_timesteps 645.
Path 77 | total_timesteps 654.
Path 78 | total_timesteps 662.
Path 79 | total_timesteps 670.
Path 80 | total_timesteps 678.
Path 81 | total_timesteps 685.
Path 82 | total_timesteps 693.
Path 83 | total_timesteps 699.
Path 84 | total_timesteps 706.
Path 85 | total_timesteps 714.
Path 86 | total_timesteps 720.
Path 87 | total_timesteps 728.
Path 88 | total_timesteps 734.
Path 89 | total_timesteps 741.
Path 90 | total_timesteps 748.
Path 91 | total_timesteps 757.
Path 92 | total_timesteps 770.
Path 93 | total_timesteps 777.
Path 94 | total_timesteps 785.
Path 95 | total_timesteps 792.
Path 96 | total_timesteps 800.
Path 97 | total_timesteps 807.
Path 98 | total_timesteps 815.
Path 99 | total_timesteps 823.
Path 100 | total_timesteps 830.
Path 101 | total_timesteps 836.
Path 102 | total_timesteps 843.
Path 103 | total_timesteps 851.
Path 104 | total_timesteps 860.
Path 105 | total_timesteps 867.
Path 106 | total_timesteps 874.
Path 107 | total_timesteps 884.
Path 108 | total_timesteps 891.
Path 109 | total_timesteps 900.
Path 110 | total_timesteps 910.
Path 111 | total_timesteps 924.
Path 112 | total_timesteps 933.
Path 113 | total_timesteps 940.
Path 114 | total_timesteps 947.
Path 115 | total_timesteps 958.
Path 116 | total_timesteps 966.
Path 117 | total_timesteps 975.
Path 118 | total_timesteps 984.
Path 119 | total_timesteps 993.
Path 120 | total_timesteps 1005.
Path 121 | total_timesteps 1013.
Path 122 | total_timesteps 1021.
Path 123 | total_timesteps 1029.
Path 124 | total_timesteps 1040.
Path 125 | total_timesteps 1049.
Path 126 | total_timesteps 1057.
Path 127 | total_timesteps 1064.
Path 128 | total_timesteps 1075.
Path 129 | total_timesteps 1082.
Path 130 | total_timesteps 1090.
Path 131 | total_timesteps 1098.
Path 132 | total_timesteps 1108.
Path 133 | total_timesteps 1118.
Path 134 | total_timesteps 1125.
Path 135 | total_timesteps 1135.
Path 136 | total_timesteps 1144.
Path 137 | total_timesteps 1153.
Path 138 | total_timesteps 1163.
Path 139 | total_timesteps 1174.
Path 140 | total_timesteps 1184.
Path 141 | total_timesteps 1192.
Path 142 | total_timesteps 1199.
Path 143 | total_timesteps 1208.
Path 144 | total_timesteps 1220.
Path 145 | total_timesteps 1228.
Path 146 | total_timesteps 1236.
Path 147 | total_timesteps 1244.
Path 148 | total_timesteps 1252.
Path 149 | total_timesteps 1262.
Path 150 | total_timesteps 1269.
Path 151 | total_timesteps 1281.
Path 152 | total_timesteps 1290.
Path 153 | total_timesteps 1299.
Path 154 | total_timesteps 1308.
Path 155 | total_timesteps 1317.
Path 156 | total_timesteps 1329.
Path 157 | total_timesteps 1337.
Path 158 | total_timesteps 1345.
Path 159 | total_timesteps 1356.
Path 160 | total_timesteps 1363.
Path 161 | total_timesteps 1372.
Path 162 | total_timesteps 1382.
Path 163 | total_timesteps 1389.
Path 164 | total_timesteps 1396.
Path 165 | total_timesteps 1408.
Path 166 | total_timesteps 1416.
Path 167 | total_timesteps 1425.
Path 168 | total_timesteps 1436.
Path 169 | total_timesteps 1445.
Path 170 | total_timesteps 1454.
Path 171 | total_timesteps 1462.
Path 172 | total_timesteps 1470.
Path 173 | total_timesteps 1481.
Path 174 | total_timesteps 1488.
Path 175 | total_timesteps 1496.
Path 176 | total_timesteps 1505.
Path 177 | total_timesteps 1513.
Path 178 | total_timesteps 1524.
Path 179 | total_timesteps 1532.
Path 180 | total_timesteps 1542.
Path 181 | total_timesteps 1549.
Path 182 | total_timesteps 1556.
Path 183 | total_timesteps 1562.
Path 184 | total_timesteps 1570.
Path 185 | total_timesteps 1579.
Path 186 | total_timesteps 1587.
Path 187 | total_timesteps 1601.
Path 188 | total_timesteps 1610.
Path 189 | total_timesteps 1618.
Path 190 | total_timesteps 1628.
Path 191 | total_timesteps 1636.
Path 192 | total_timesteps 1649.
Path 193 | total_timesteps 1655.
Path 194 | total_timesteps 1662.
Path 195 | total_timesteps 1668.
Path 196 | total_timesteps 1675.
Path 197 | total_timesteps 1685.
Path 198 | total_timesteps 1691.
Path 199 | total_timesteps 1697.
Path 200 | total_timesteps 1706.
Path 201 | total_timesteps 1715.
Path 202 | total_timesteps 1727.
Path 203 | total_timesteps 1736.
Path 204 | total_timesteps 1749.
Path 205 | total_timesteps 1761.
Path 206 | total_timesteps 1768.
Path 207 | total_timesteps 1779.
Path 208 | total_timesteps 1785.
Path 209 | total_timesteps 1795.
Path 210 | total_timesteps 1801.
Path 211 | total_timesteps 1811.
Path 212 | total_timesteps 1821.
Path 213 | total_timesteps 1829.
Path 214 | total_timesteps 1839.
Path 215 | total_timesteps 1848.
Path 216 | total_timesteps 1857.
Path 217 | total_timesteps 1864.
Path 218 | total_timesteps 1873.
Path 219 | total_timesteps 1881.
Path 220 | total_timesteps 1887.
Path 221 | total_timesteps 1894.
Path 222 | total_timesteps 1901.
Path 223 | total_timesteps 1908.
Path 224 | total_timesteps 1915.
Path 225 | total_timesteps 1922.
Path 226 | total_timesteps 1929.
Path 227 | total_timesteps 1936.
Path 228 | total_timesteps 1944.
Path 229 | total_timesteps 1950.
Path 230 | total_timesteps 1958.
Path 231 | total_timesteps 1972.
Path 232 | total_timesteps 1982.
Path 233 | total_timesteps 1993.
Path 234 | total_timesteps 2003.
Path 235 | total_timesteps 2011.
Path 236 | total_timesteps 2021.
Path 237 | total_timesteps 2029.
Path 238 | total_timesteps 2041.
Path 239 | total_timesteps 2049.
Path 240 | total_timesteps 2056.
Path 241 | total_timesteps 2066.
Path 242 | total_timesteps 2076.
Path 243 | total_timesteps 2084.
Path 244 | total_timesteps 2093.
Path 245 | total_timesteps 2101.
Path 246 | total_timesteps 2111.
Path 247 | total_timesteps 2120.
Path 248 | total_timesteps 2130.
Path 249 | total_timesteps 2146.
Path 250 | total_timesteps 2155.
Path 251 | total_timesteps 2163.
Path 252 | total_timesteps 2173.
Path 253 | total_timesteps 2188.
Path 254 | total_timesteps 2195.
Path 255 | total_timesteps 2202.
Path 256 | total_timesteps 2211.
Path 257 | total_timesteps 2222.
Path 258 | total_timesteps 2230.
Path 259 | total_timesteps 2240.
Path 260 | total_timesteps 2248.
Path 261 | total_timesteps 2256.
Path 262 | total_timesteps 2265.
Path 263 | total_timesteps 2272.
Path 264 | total_timesteps 2284.
Path 265 | total_timesteps 2290.
Path 266 | total_timesteps 2298.
Path 267 | total_timesteps 2308.
Path 268 | total_timesteps 2318.
Path 269 | total_timesteps 2325.
Path 270 | total_timesteps 2339.
Path 271 | total_timesteps 2346.
Path 272 | total_timesteps 2354.
Path 273 | total_timesteps 2365.
Path 274 | total_timesteps 2374.
Path 275 | total_timesteps 2381.
Path 276 | total_timesteps 2388.
Path 277 | total_timesteps 2396.
Path 278 | total_timesteps 2406.
Path 279 | total_timesteps 2416.
Path 280 | total_timesteps 2426.
Path 281 | total_timesteps 2433.
Path 282 | total_timesteps 2448.
Path 283 | total_timesteps 2454.
Path 284 | total_timesteps 2462.
Path 285 | total_timesteps 2471.
Path 286 | total_timesteps 2480.
Path 287 | total_timesteps 2490.
Path 288 | total_timesteps 2498.
Path 289 | total_timesteps 2505.
Path 290 | total_timesteps 2517.
Path 291 | total_timesteps 2523.
Path 292 | total_timesteps 2532.
Path 293 | total_timesteps 2539.
Path 294 | total_timesteps 2547.
Path 295 | total_timesteps 2556.
Path 296 | total_timesteps 2563.
Path 297 | total_timesteps 2572.
Path 298 | total_timesteps 2580.
Path 299 | total_timesteps 2592.
Path 300 | total_timesteps 2599.
Path 301 | total_timesteps 2609.
Path 302 | total_timesteps 2616.
Path 303 | total_timesteps 2623.
Path 304 | total_timesteps 2631.
Path 305 | total_timesteps 2642.
Path 306 | total_timesteps 2653.
Path 307 | total_timesteps 2660.
Path 308 | total_timesteps 2668.
Path 309 | total_timesteps 2675.
Path 310 | total_timesteps 2682.
Path 311 | total_timesteps 2688.
Path 312 | total_timesteps 2699.
Path 313 | total_timesteps 2707.
Path 314 | total_timesteps 2715.
Path 315 | total_timesteps 2722.
Path 316 | total_timesteps 2730.
Path 317 | total_timesteps 2739.
Path 318 | total_timesteps 2750.
Path 319 | total_timesteps 2758.
Path 320 | total_timesteps 2772.
Path 321 | total_timesteps 2781.
Path 322 | total_timesteps 2787.
Path 323 | total_timesteps 2794.
Path 324 | total_timesteps 2802.
Path 325 | total_timesteps 2814.
Path 326 | total_timesteps 2822.
Path 327 | total_timesteps 2834.
Path 328 | total_timesteps 2842.
Path 329 | total_timesteps 2849.
Path 330 | total_timesteps 2857.
Path 331 | total_timesteps 2866.
Path 332 | total_timesteps 2877.
Path 333 | total_timesteps 2885.
Path 334 | total_timesteps 2894.
Path 335 | total_timesteps 2902.
Path 336 | total_timesteps 2909.
Path 337 | total_timesteps 2918.
Path 338 | total_timesteps 2924.
Path 339 | total_timesteps 2931.
Path 340 | total_timesteps 2938.
Path 341 | total_timesteps 2948.
Path 342 | total_timesteps 2955.
Path 343 | total_timesteps 2962.
Path 344 | total_timesteps 2968.
Path 345 | total_timesteps 2976.
Path 346 | total_timesteps 2983.
Path 347 | total_timesteps 2990.
Path 348 | total_timesteps 3004.
Path 349 | total_timesteps 3012.
Path 350 | total_timesteps 3019.
Path 351 | total_timesteps 3028.
Path 352 | total_timesteps 3034.
Path 353 | total_timesteps 3042.
Path 354 | total_timesteps 3051.
Path 355 | total_timesteps 3062.
Path 356 | total_timesteps 3068.
Path 357 | total_timesteps 3076.
Path 358 | total_timesteps 3084.
Path 359 | total_timesteps 3092.
Path 360 | total_timesteps 3100.
Path 361 | total_timesteps 3107.
Path 362 | total_timesteps 3116.
Path 363 | total_timesteps 3124.
Path 364 | total_timesteps 3133.
Path 365 | total_timesteps 3139.
Path 366 | total_timesteps 3146.
Path 367 | total_timesteps 3156.
Path 368 | total_timesteps 3169.
Path 369 | total_timesteps 3179.
Path 370 | total_timesteps 3188.
Path 371 | total_timesteps 3201.
Path 372 | total_timesteps 3209.
Path 373 | total_timesteps 3216.
Path 374 | total_timesteps 3227.
Path 375 | total_timesteps 3234.
Path 376 | total_timesteps 3244.
Path 377 | total_timesteps 3253.
Path 378 | total_timesteps 3263.
Path 379 | total_timesteps 3274.
Path 380 | total_timesteps 3283.
Path 381 | total_timesteps 3293.
Path 382 | total_timesteps 3301.
Path 383 | total_timesteps 3310.
Path 384 | total_timesteps 3317.
Path 385 | total_timesteps 3326.
Path 386 | total_timesteps 3332.
Path 387 | total_timesteps 3342.
Path 388 | total_timesteps 3351.
Path 389 | total_timesteps 3361.
Path 390 | total_timesteps 3369.
Path 391 | total_timesteps 3379.
Path 392 | total_timesteps 3389.
Path 393 | total_timesteps 3400.
Path 394 | total_timesteps 3407.
Path 395 | total_timesteps 3415.
Path 396 | total_timesteps 3421.
Path 397 | total_timesteps 3430.
Path 398 | total_timesteps 3439.
Path 399 | total_timesteps 3452.
Path 400 | total_timesteps 3463.
Path 401 | total_timesteps 3469.
Path 402 | total_timesteps 3477.
Path 403 | total_timesteps 3484.
Path 404 | total_timesteps 3492.
Path 405 | total_timesteps 3503.
Path 406 | total_timesteps 3512.
Path 407 | total_timesteps 3520.
Path 408 | total_timesteps 3526.
Path 409 | total_timesteps 3535.
Path 410 | total_timesteps 3544.
Path 411 | total_timesteps 3551.
Path 412 | total_timesteps 3565.
Path 413 | total_timesteps 3572.
Path 414 | total_timesteps 3579.
Path 415 | total_timesteps 3589.
Path 416 | total_timesteps 3599.
Path 417 | total_timesteps 3605.
Path 418 | total_timesteps 3611.
Path 419 | total_timesteps 3620.
Path 420 | total_timesteps 3627.
Path 421 | total_timesteps 3637.
Path 422 | total_timesteps 3649.
Path 423 | total_timesteps 3657.
Path 424 | total_timesteps 3664.
Path 425 | total_timesteps 3671.
Path 426 | total_timesteps 3681.
Path 427 | total_timesteps 3688.
Path 428 | total_timesteps 3698.
Path 429 | total_timesteps 3705.
Path 430 | total_timesteps 3713.
Path 431 | total_timesteps 3720.
Path 432 | total_timesteps 3729.
Path 433 | total_timesteps 3736.
Path 434 | total_timesteps 3742.
Path 435 | total_timesteps 3754.
Path 436 | total_timesteps 3764.
Path 437 | total_timesteps 3775.
Path 438 | total_timesteps 3782.
Path 439 | total_timesteps 3791.
Path 440 | total_timesteps 3802.
Path 441 | total_timesteps 3812.
Path 442 | total_timesteps 3820.
Path 443 | total_timesteps 3827.
Path 444 | total_timesteps 3836.
Path 445 | total_timesteps 3844.
Path 446 | total_timesteps 3853.
Path 447 | total_timesteps 3860.
Path 448 | total_timesteps 3868.
Path 449 | total_timesteps 3877.
Path 450 | total_timesteps 3885.
Path 451 | total_timesteps 3891.
Path 452 | total_timesteps 3902.
Path 453 | total_timesteps 3909.
Path 454 | total_timesteps 3917.
Path 455 | total_timesteps 3924.
Path 456 | total_timesteps 3931.
Path 457 | total_timesteps 3940.
Path 458 | total_timesteps 3948.
Path 459 | total_timesteps 3954.
Path 460 | total_timesteps 3963.
Path 461 | total_timesteps 3974.
Path 462 | total_timesteps 3983.
Path 463 | total_timesteps 3991.
Path 464 | total_timesteps 3998.
Path 465 | total_timesteps 4006.
Path 466 | total_timesteps 4013.
Path 467 | total_timesteps 4021.
Path 468 | total_timesteps 4030.
Path 469 | total_timesteps 4038.
Path 470 | total_timesteps 4050.
Path 471 | total_timesteps 4058.
Path 472 | total_timesteps 4065.
Path 473 | total_timesteps 4073.
Path 474 | total_timesteps 4081.
Path 475 | total_timesteps 4089.
Path 476 | total_timesteps 4097.
Path 477 | total_timesteps 4104.
Path 478 | total_timesteps 4110.
Path 479 | total_timesteps 4118.
Path 480 | total_timesteps 4125.
Path 481 | total_timesteps 4134.
Path 482 | total_timesteps 4142.
Path 483 | total_timesteps 4148.
Path 484 | total_timesteps 4157.
Path 485 | total_timesteps 4163.
Path 486 | total_timesteps 4170.
Path 487 | total_timesteps 4177.
Path 488 | total_timesteps 4183.
Path 489 | total_timesteps 4190.
Path 490 | total_timesteps 4199.
Path 491 | total_timesteps 4205.
Path 492 | total_timesteps 4212.
Path 493 | total_timesteps 4218.
Path 494 | total_timesteps 4226.
Path 495 | total_timesteps 4233.
Path 496 | total_timesteps 4241.
Path 497 | total_timesteps 4250.
Path 498 | total_timesteps 4260.
Path 499 | total_timesteps 4278.
Path 500 | total_timesteps 4287.
Path 501 | total_timesteps 4294.
Path 502 | total_timesteps 4303.
Path 503 | total_timesteps 4311.
Path 504 | total_timesteps 4317.
Path 505 | total_timesteps 4326.
Path 506 | total_timesteps 4336.
Path 507 | total_timesteps 4343.
Path 508 | total_timesteps 4351.
Path 509 | total_timesteps 4359.
Path 510 | total_timesteps 4367.
Path 511 | total_timesteps 4374.
Path 512 | total_timesteps 4383.
Path 513 | total_timesteps 4391.
Path 514 | total_timesteps 4398.
Path 515 | total_timesteps 4409.
Path 516 | total_timesteps 4420.
Path 517 | total_timesteps 4430.
Path 518 | total_timesteps 4438.
Path 519 | total_timesteps 4445.
Path 520 | total_timesteps 4452.
Path 521 | total_timesteps 4460.
Path 522 | total_timesteps 4468.
Path 523 | total_timesteps 4475.
Path 524 | total_timesteps 4482.
Path 525 | total_timesteps 4490.
Path 526 | total_timesteps 4497.
Path 527 | total_timesteps 4504.
Path 528 | total_timesteps 4512.
Path 529 | total_timesteps 4522.
Path 530 | total_timesteps 4531.
Path 531 | total_timesteps 4540.
Path 532 | total_timesteps 4557.
Path 533 | total_timesteps 4569.
Path 534 | total_timesteps 4577.
Path 535 | total_timesteps 4586.
Path 536 | total_timesteps 4596.
Path 537 | total_timesteps 4603.
Path 538 | total_timesteps 4611.
Path 539 | total_timesteps 4620.
Path 540 | total_timesteps 4630.
Path 541 | total_timesteps 4641.
Path 542 | total_timesteps 4653.
Path 543 | total_timesteps 4663.
Path 544 | total_timesteps 4671.
Path 545 | total_timesteps 4678.
Path 546 | total_timesteps 4687.
Path 547 | total_timesteps 4697.
Path 548 | total_timesteps 4706.
Path 549 | total_timesteps 4716.
Path 550 | total_timesteps 4725.
Path 551 | total_timesteps 4737.
Path 552 | total_timesteps 4743.
Path 553 | total_timesteps 4753.
Path 554 | total_timesteps 4760.
Path 555 | total_timesteps 4771.
Path 556 | total_timesteps 4781.
Path 557 | total_timesteps 4794.
Path 558 | total_timesteps 4800.
Path 559 | total_timesteps 4807.
Path 560 | total_timesteps 4815.
Path 561 | total_timesteps 4823.
Path 562 | total_timesteps 4832.
Path 563 | total_timesteps 4839.
Path 564 | total_timesteps 4847.
Path 565 | total_timesteps 4853.
Path 566 | total_timesteps 4861.
Path 567 | total_timesteps 4868.
Path 568 | total_timesteps 4876.
Path 569 | total_timesteps 4883.
Path 570 | total_timesteps 4890.
Path 571 | total_timesteps 4898.
Path 572 | total_timesteps 4907.
Path 573 | total_timesteps 4915.
Path 574 | total_timesteps 4924.
Path 575 | total_timesteps 4935.
Path 576 | total_timesteps 4946.
Path 577 | total_timesteps 4953.
Path 578 | total_timesteps 4961.
Path 579 | total_timesteps 4969.
Path 580 | total_timesteps 4976.
Path 581 | total_timesteps 4988.
Path 582 | total_timesteps 4998.
Path 583 | total_timesteps 5005.
Path 584 | total_timesteps 5013.
Path 585 | total_timesteps 5022.
Path 586 | total_timesteps 5031.
Path 587 | total_timesteps 5041.
Path 588 | total_timesteps 5049.
Path 589 | total_timesteps 5057.
Path 590 | total_timesteps 5064.
Path 591 | total_timesteps 5081.
Path 592 | total_timesteps 5088.
Path 593 | total_timesteps 5097.
Path 594 | total_timesteps 5108.
Path 595 | total_timesteps 5117.
Path 596 | total_timesteps 5124.
Path 597 | total_timesteps 5132.
Path 598 | total_timesteps 5140.
Path 599 | total_timesteps 5148.
Path 600 | total_timesteps 5157.
Path 601 | total_timesteps 5168.
Path 602 | total_timesteps 5176.
Path 603 | total_timesteps 5187.
Path 604 | total_timesteps 5193.
Path 605 | total_timesteps 5200.
Path 606 | total_timesteps 5206.
Path 607 | total_timesteps 5219.
Path 608 | total_timesteps 5226.
Path 609 | total_timesteps 5238.
Path 610 | total_timesteps 5245.
Path 611 | total_timesteps 5253.
Path 612 | total_timesteps 5261.
Path 613 | total_timesteps 5271.
Path 614 | total_timesteps 5278.
Path 615 | total_timesteps 5294.
Path 616 | total_timesteps 5301.
Path 617 | total_timesteps 5308.
Path 618 | total_timesteps 5318.
Path 619 | total_timesteps 5324.
Path 620 | total_timesteps 5331.
Path 621 | total_timesteps 5339.
Path 622 | total_timesteps 5349.
Path 623 | total_timesteps 5356.
Path 624 | total_timesteps 5368.
Path 625 | total_timesteps 5374.
Path 626 | total_timesteps 5383.
Path 627 | total_timesteps 5392.
Path 628 | total_timesteps 5399.
Path 629 | total_timesteps 5412.
Path 630 | total_timesteps 5420.
Path 631 | total_timesteps 5427.
Path 632 | total_timesteps 5439.
Path 633 | total_timesteps 5447.
Path 634 | total_timesteps 5454.
Path 635 | total_timesteps 5463.
Path 636 | total_timesteps 5473.
Path 637 | total_timesteps 5482.
Path 638 | total_timesteps 5489.
Path 639 | total_timesteps 5497.
Path 640 | total_timesteps 5504.
Path 641 | total_timesteps 5513.
Path 642 | total_timesteps 5520.
Path 643 | total_timesteps 5530.
Path 644 | total_timesteps 5541.
Path 645 | total_timesteps 5553.
Path 646 | total_timesteps 5562.
Path 647 | total_timesteps 5569.
Path 648 | total_timesteps 5577.
Path 649 | total_timesteps 5585.
Path 650 | total_timesteps 5593.
Path 651 | total_timesteps 5599.
Path 652 | total_timesteps 5609.
Path 653 | total_timesteps 5619.
Path 654 | total_timesteps 5627.
Path 655 | total_timesteps 5634.
Path 656 | total_timesteps 5643.
Path 657 | total_timesteps 5649.
Path 658 | total_timesteps 5660.
Path 659 | total_timesteps 5667.
Path 660 | total_timesteps 5676.
Path 661 | total_timesteps 5686.
Path 662 | total_timesteps 5694.
Path 663 | total_timesteps 5701.
Path 664 | total_timesteps 5710.
Path 665 | total_timesteps 5720.
Path 666 | total_timesteps 5729.
Path 667 | total_timesteps 5736.
Path 668 | total_timesteps 5746.
Path 669 | total_timesteps 5753.
Path 670 | total_timesteps 5759.
Path 671 | total_timesteps 5766.
Path 672 | total_timesteps 5774.
Path 673 | total_timesteps 5780.
Path 674 | total_timesteps 5788.
Path 675 | total_timesteps 5799.
Path 676 | total_timesteps 5809.
Path 677 | total_timesteps 5820.
Path 678 | total_timesteps 5828.
Path 679 | total_timesteps 5835.
Path 680 | total_timesteps 5842.
Path 681 | total_timesteps 5848.
Path 682 | total_timesteps 5855.
Path 683 | total_timesteps 5862.
Path 684 | total_timesteps 5870.
Path 685 | total_timesteps 5877.
Path 686 | total_timesteps 5885.
Path 687 | total_timesteps 5895.
Path 688 | total_timesteps 5903.
Path 689 | total_timesteps 5910.
Path 690 | total_timesteps 5918.
Path 691 | total_timesteps 5927.
Path 692 | total_timesteps 5938.
Path 693 | total_timesteps 5947.
Path 694 | total_timesteps 5955.
Path 695 | total_timesteps 5964.
Path 696 | total_timesteps 5977.
Path 697 | total_timesteps 5986.
Path 698 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.96    |
| Iteration     | 22       |
| MaximumReturn | -0.0208  |
| MinimumReturn | -14.5    |
| TotalSamples  | 96090    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.0021001913119107485
Validation loss = 0.0018163655186071992
Validation loss = 0.0021168431267142296
Validation loss = 0.0017899293452501297
Validation loss = 0.0017190161161124706
Validation loss = 0.001775418291799724
Validation loss = 0.0018855612725019455
Validation loss = 0.0016724526649340987
Validation loss = 0.0018464378081262112
Validation loss = 0.0019740285351872444
Validation loss = 0.0016775092808529735
Validation loss = 0.001905494718812406
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 64.
Path 8 | total_timesteps 74.
Path 9 | total_timesteps 81.
Path 10 | total_timesteps 89.
Path 11 | total_timesteps 96.
Path 12 | total_timesteps 108.
Path 13 | total_timesteps 115.
Path 14 | total_timesteps 127.
Path 15 | total_timesteps 137.
Path 16 | total_timesteps 146.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 163.
Path 19 | total_timesteps 170.
Path 20 | total_timesteps 180.
Path 21 | total_timesteps 188.
Path 22 | total_timesteps 195.
Path 23 | total_timesteps 203.
Path 24 | total_timesteps 213.
Path 25 | total_timesteps 222.
Path 26 | total_timesteps 232.
Path 27 | total_timesteps 241.
Path 28 | total_timesteps 249.
Path 29 | total_timesteps 257.
Path 30 | total_timesteps 264.
Path 31 | total_timesteps 271.
Path 32 | total_timesteps 278.
Path 33 | total_timesteps 286.
Path 34 | total_timesteps 294.
Path 35 | total_timesteps 300.
Path 36 | total_timesteps 310.
Path 37 | total_timesteps 317.
Path 38 | total_timesteps 325.
Path 39 | total_timesteps 337.
Path 40 | total_timesteps 346.
Path 41 | total_timesteps 356.
Path 42 | total_timesteps 364.
Path 43 | total_timesteps 371.
Path 44 | total_timesteps 379.
Path 45 | total_timesteps 389.
Path 46 | total_timesteps 396.
Path 47 | total_timesteps 403.
Path 48 | total_timesteps 415.
Path 49 | total_timesteps 421.
Path 50 | total_timesteps 429.
Path 51 | total_timesteps 440.
Path 52 | total_timesteps 447.
Path 53 | total_timesteps 455.
Path 54 | total_timesteps 464.
Path 55 | total_timesteps 471.
Path 56 | total_timesteps 483.
Path 57 | total_timesteps 491.
Path 58 | total_timesteps 501.
Path 59 | total_timesteps 511.
Path 60 | total_timesteps 522.
Path 61 | total_timesteps 528.
Path 62 | total_timesteps 537.
Path 63 | total_timesteps 545.
Path 64 | total_timesteps 557.
Path 65 | total_timesteps 565.
Path 66 | total_timesteps 571.
Path 67 | total_timesteps 579.
Path 68 | total_timesteps 588.
Path 69 | total_timesteps 595.
Path 70 | total_timesteps 603.
Path 71 | total_timesteps 615.
Path 72 | total_timesteps 625.
Path 73 | total_timesteps 636.
Path 74 | total_timesteps 645.
Path 75 | total_timesteps 652.
Path 76 | total_timesteps 661.
Path 77 | total_timesteps 668.
Path 78 | total_timesteps 678.
Path 79 | total_timesteps 684.
Path 80 | total_timesteps 695.
Path 81 | total_timesteps 704.
Path 82 | total_timesteps 711.
Path 83 | total_timesteps 721.
Path 84 | total_timesteps 729.
Path 85 | total_timesteps 739.
Path 86 | total_timesteps 749.
Path 87 | total_timesteps 757.
Path 88 | total_timesteps 768.
Path 89 | total_timesteps 775.
Path 90 | total_timesteps 783.
Path 91 | total_timesteps 790.
Path 92 | total_timesteps 798.
Path 93 | total_timesteps 807.
Path 94 | total_timesteps 819.
Path 95 | total_timesteps 828.
Path 96 | total_timesteps 835.
Path 97 | total_timesteps 843.
Path 98 | total_timesteps 850.
Path 99 | total_timesteps 857.
Path 100 | total_timesteps 864.
Path 101 | total_timesteps 872.
Path 102 | total_timesteps 878.
Path 103 | total_timesteps 888.
Path 104 | total_timesteps 896.
Path 105 | total_timesteps 903.
Path 106 | total_timesteps 910.
Path 107 | total_timesteps 919.
Path 108 | total_timesteps 925.
Path 109 | total_timesteps 934.
Path 110 | total_timesteps 946.
Path 111 | total_timesteps 964.
Path 112 | total_timesteps 972.
Path 113 | total_timesteps 980.
Path 114 | total_timesteps 996.
Path 115 | total_timesteps 1003.
Path 116 | total_timesteps 1009.
Path 117 | total_timesteps 1016.
Path 118 | total_timesteps 1024.
Path 119 | total_timesteps 1032.
Path 120 | total_timesteps 1040.
Path 121 | total_timesteps 1047.
Path 122 | total_timesteps 1057.
Path 123 | total_timesteps 1065.
Path 124 | total_timesteps 1074.
Path 125 | total_timesteps 1082.
Path 126 | total_timesteps 1095.
Path 127 | total_timesteps 1108.
Path 128 | total_timesteps 1115.
Path 129 | total_timesteps 1124.
Path 130 | total_timesteps 1131.
Path 131 | total_timesteps 1140.
Path 132 | total_timesteps 1148.
Path 133 | total_timesteps 1158.
Path 134 | total_timesteps 1170.
Path 135 | total_timesteps 1180.
Path 136 | total_timesteps 1188.
Path 137 | total_timesteps 1198.
Path 138 | total_timesteps 1210.
Path 139 | total_timesteps 1218.
Path 140 | total_timesteps 1228.
Path 141 | total_timesteps 1235.
Path 142 | total_timesteps 1242.
Path 143 | total_timesteps 1248.
Path 144 | total_timesteps 1256.
Path 145 | total_timesteps 1265.
Path 146 | total_timesteps 1272.
Path 147 | total_timesteps 1281.
Path 148 | total_timesteps 1289.
Path 149 | total_timesteps 1297.
Path 150 | total_timesteps 1309.
Path 151 | total_timesteps 1317.
Path 152 | total_timesteps 1324.
Path 153 | total_timesteps 1332.
Path 154 | total_timesteps 1339.
Path 155 | total_timesteps 1348.
Path 156 | total_timesteps 1359.
Path 157 | total_timesteps 1367.
Path 158 | total_timesteps 1376.
Path 159 | total_timesteps 1387.
Path 160 | total_timesteps 1394.
Path 161 | total_timesteps 1403.
Path 162 | total_timesteps 1411.
Path 163 | total_timesteps 1419.
Path 164 | total_timesteps 1426.
Path 165 | total_timesteps 1436.
Path 166 | total_timesteps 1444.
Path 167 | total_timesteps 1457.
Path 168 | total_timesteps 1464.
Path 169 | total_timesteps 1473.
Path 170 | total_timesteps 1480.
Path 171 | total_timesteps 1490.
Path 172 | total_timesteps 1499.
Path 173 | total_timesteps 1507.
Path 174 | total_timesteps 1517.
Path 175 | total_timesteps 1525.
Path 176 | total_timesteps 1531.
Path 177 | total_timesteps 1538.
Path 178 | total_timesteps 1546.
Path 179 | total_timesteps 1558.
Path 180 | total_timesteps 1566.
Path 181 | total_timesteps 1579.
Path 182 | total_timesteps 1586.
Path 183 | total_timesteps 1594.
Path 184 | total_timesteps 1602.
Path 185 | total_timesteps 1612.
Path 186 | total_timesteps 1623.
Path 187 | total_timesteps 1633.
Path 188 | total_timesteps 1642.
Path 189 | total_timesteps 1654.
Path 190 | total_timesteps 1661.
Path 191 | total_timesteps 1667.
Path 192 | total_timesteps 1677.
Path 193 | total_timesteps 1684.
Path 194 | total_timesteps 1691.
Path 195 | total_timesteps 1700.
Path 196 | total_timesteps 1707.
Path 197 | total_timesteps 1716.
Path 198 | total_timesteps 1723.
Path 199 | total_timesteps 1731.
Path 200 | total_timesteps 1738.
Path 201 | total_timesteps 1746.
Path 202 | total_timesteps 1754.
Path 203 | total_timesteps 1764.
Path 204 | total_timesteps 1776.
Path 205 | total_timesteps 1786.
Path 206 | total_timesteps 1798.
Path 207 | total_timesteps 1808.
Path 208 | total_timesteps 1816.
Path 209 | total_timesteps 1824.
Path 210 | total_timesteps 1834.
Path 211 | total_timesteps 1842.
Path 212 | total_timesteps 1849.
Path 213 | total_timesteps 1856.
Path 214 | total_timesteps 1864.
Path 215 | total_timesteps 1875.
Path 216 | total_timesteps 1887.
Path 217 | total_timesteps 1894.
Path 218 | total_timesteps 1903.
Path 219 | total_timesteps 1909.
Path 220 | total_timesteps 1920.
Path 221 | total_timesteps 1929.
Path 222 | total_timesteps 1941.
Path 223 | total_timesteps 1948.
Path 224 | total_timesteps 1955.
Path 225 | total_timesteps 1962.
Path 226 | total_timesteps 1971.
Path 227 | total_timesteps 1982.
Path 228 | total_timesteps 1990.
Path 229 | total_timesteps 2003.
Path 230 | total_timesteps 2012.
Path 231 | total_timesteps 2020.
Path 232 | total_timesteps 2026.
Path 233 | total_timesteps 2035.
Path 234 | total_timesteps 2049.
Path 235 | total_timesteps 2059.
Path 236 | total_timesteps 2071.
Path 237 | total_timesteps 2081.
Path 238 | total_timesteps 2088.
Path 239 | total_timesteps 2096.
Path 240 | total_timesteps 2104.
Path 241 | total_timesteps 2111.
Path 242 | total_timesteps 2119.
Path 243 | total_timesteps 2126.
Path 244 | total_timesteps 2133.
Path 245 | total_timesteps 2140.
Path 246 | total_timesteps 2147.
Path 247 | total_timesteps 2156.
Path 248 | total_timesteps 2163.
Path 249 | total_timesteps 2171.
Path 250 | total_timesteps 2180.
Path 251 | total_timesteps 2187.
Path 252 | total_timesteps 2193.
Path 253 | total_timesteps 2201.
Path 254 | total_timesteps 2208.
Path 255 | total_timesteps 2217.
Path 256 | total_timesteps 2225.
Path 257 | total_timesteps 2232.
Path 258 | total_timesteps 2242.
Path 259 | total_timesteps 2248.
Path 260 | total_timesteps 2256.
Path 261 | total_timesteps 2269.
Path 262 | total_timesteps 2278.
Path 263 | total_timesteps 2285.
Path 264 | total_timesteps 2294.
Path 265 | total_timesteps 2300.
Path 266 | total_timesteps 2307.
Path 267 | total_timesteps 2315.
Path 268 | total_timesteps 2322.
Path 269 | total_timesteps 2334.
Path 270 | total_timesteps 2345.
Path 271 | total_timesteps 2354.
Path 272 | total_timesteps 2363.
Path 273 | total_timesteps 2370.
Path 274 | total_timesteps 2379.
Path 275 | total_timesteps 2387.
Path 276 | total_timesteps 2398.
Path 277 | total_timesteps 2409.
Path 278 | total_timesteps 2418.
Path 279 | total_timesteps 2425.
Path 280 | total_timesteps 2433.
Path 281 | total_timesteps 2440.
Path 282 | total_timesteps 2450.
Path 283 | total_timesteps 2457.
Path 284 | total_timesteps 2463.
Path 285 | total_timesteps 2473.
Path 286 | total_timesteps 2479.
Path 287 | total_timesteps 2489.
Path 288 | total_timesteps 2500.
Path 289 | total_timesteps 2512.
Path 290 | total_timesteps 2525.
Path 291 | total_timesteps 2532.
Path 292 | total_timesteps 2538.
Path 293 | total_timesteps 2545.
Path 294 | total_timesteps 2553.
Path 295 | total_timesteps 2559.
Path 296 | total_timesteps 2566.
Path 297 | total_timesteps 2573.
Path 298 | total_timesteps 2580.
Path 299 | total_timesteps 2589.
Path 300 | total_timesteps 2596.
Path 301 | total_timesteps 2606.
Path 302 | total_timesteps 2613.
Path 303 | total_timesteps 2623.
Path 304 | total_timesteps 2632.
Path 305 | total_timesteps 2642.
Path 306 | total_timesteps 2650.
Path 307 | total_timesteps 2659.
Path 308 | total_timesteps 2669.
Path 309 | total_timesteps 2677.
Path 310 | total_timesteps 2684.
Path 311 | total_timesteps 2693.
Path 312 | total_timesteps 2706.
Path 313 | total_timesteps 2714.
Path 314 | total_timesteps 2723.
Path 315 | total_timesteps 2731.
Path 316 | total_timesteps 2742.
Path 317 | total_timesteps 2751.
Path 318 | total_timesteps 2763.
Path 319 | total_timesteps 2771.
Path 320 | total_timesteps 2780.
Path 321 | total_timesteps 2788.
Path 322 | total_timesteps 2801.
Path 323 | total_timesteps 2807.
Path 324 | total_timesteps 2814.
Path 325 | total_timesteps 2823.
Path 326 | total_timesteps 2830.
Path 327 | total_timesteps 2840.
Path 328 | total_timesteps 2848.
Path 329 | total_timesteps 2858.
Path 330 | total_timesteps 2866.
Path 331 | total_timesteps 2873.
Path 332 | total_timesteps 2880.
Path 333 | total_timesteps 2888.
Path 334 | total_timesteps 2895.
Path 335 | total_timesteps 2901.
Path 336 | total_timesteps 2912.
Path 337 | total_timesteps 2924.
Path 338 | total_timesteps 2931.
Path 339 | total_timesteps 2938.
Path 340 | total_timesteps 2945.
Path 341 | total_timesteps 2954.
Path 342 | total_timesteps 2961.
Path 343 | total_timesteps 2971.
Path 344 | total_timesteps 2977.
Path 345 | total_timesteps 2986.
Path 346 | total_timesteps 2999.
Path 347 | total_timesteps 3008.
Path 348 | total_timesteps 3016.
Path 349 | total_timesteps 3026.
Path 350 | total_timesteps 3036.
Path 351 | total_timesteps 3045.
Path 352 | total_timesteps 3054.
Path 353 | total_timesteps 3063.
Path 354 | total_timesteps 3072.
Path 355 | total_timesteps 3083.
Path 356 | total_timesteps 3092.
Path 357 | total_timesteps 3100.
Path 358 | total_timesteps 3108.
Path 359 | total_timesteps 3119.
Path 360 | total_timesteps 3129.
Path 361 | total_timesteps 3136.
Path 362 | total_timesteps 3142.
Path 363 | total_timesteps 3152.
Path 364 | total_timesteps 3162.
Path 365 | total_timesteps 3174.
Path 366 | total_timesteps 3182.
Path 367 | total_timesteps 3190.
Path 368 | total_timesteps 3197.
Path 369 | total_timesteps 3206.
Path 370 | total_timesteps 3215.
Path 371 | total_timesteps 3226.
Path 372 | total_timesteps 3236.
Path 373 | total_timesteps 3244.
Path 374 | total_timesteps 3251.
Path 375 | total_timesteps 3260.
Path 376 | total_timesteps 3271.
Path 377 | total_timesteps 3280.
Path 378 | total_timesteps 3290.
Path 379 | total_timesteps 3298.
Path 380 | total_timesteps 3308.
Path 381 | total_timesteps 3314.
Path 382 | total_timesteps 3320.
Path 383 | total_timesteps 3328.
Path 384 | total_timesteps 3337.
Path 385 | total_timesteps 3347.
Path 386 | total_timesteps 3356.
Path 387 | total_timesteps 3364.
Path 388 | total_timesteps 3374.
Path 389 | total_timesteps 3387.
Path 390 | total_timesteps 3396.
Path 391 | total_timesteps 3403.
Path 392 | total_timesteps 3412.
Path 393 | total_timesteps 3418.
Path 394 | total_timesteps 3425.
Path 395 | total_timesteps 3433.
Path 396 | total_timesteps 3444.
Path 397 | total_timesteps 3451.
Path 398 | total_timesteps 3462.
Path 399 | total_timesteps 3472.
Path 400 | total_timesteps 3480.
Path 401 | total_timesteps 3489.
Path 402 | total_timesteps 3495.
Path 403 | total_timesteps 3504.
Path 404 | total_timesteps 3512.
Path 405 | total_timesteps 3521.
Path 406 | total_timesteps 3529.
Path 407 | total_timesteps 3538.
Path 408 | total_timesteps 3546.
Path 409 | total_timesteps 3555.
Path 410 | total_timesteps 3561.
Path 411 | total_timesteps 3569.
Path 412 | total_timesteps 3576.
Path 413 | total_timesteps 3586.
Path 414 | total_timesteps 3593.
Path 415 | total_timesteps 3600.
Path 416 | total_timesteps 3611.
Path 417 | total_timesteps 3617.
Path 418 | total_timesteps 3624.
Path 419 | total_timesteps 3631.
Path 420 | total_timesteps 3637.
Path 421 | total_timesteps 3647.
Path 422 | total_timesteps 3657.
Path 423 | total_timesteps 3669.
Path 424 | total_timesteps 3679.
Path 425 | total_timesteps 3686.
Path 426 | total_timesteps 3696.
Path 427 | total_timesteps 3705.
Path 428 | total_timesteps 3715.
Path 429 | total_timesteps 3724.
Path 430 | total_timesteps 3733.
Path 431 | total_timesteps 3742.
Path 432 | total_timesteps 3749.
Path 433 | total_timesteps 3756.
Path 434 | total_timesteps 3763.
Path 435 | total_timesteps 3770.
Path 436 | total_timesteps 3778.
Path 437 | total_timesteps 3784.
Path 438 | total_timesteps 3792.
Path 439 | total_timesteps 3800.
Path 440 | total_timesteps 3807.
Path 441 | total_timesteps 3814.
Path 442 | total_timesteps 3823.
Path 443 | total_timesteps 3835.
Path 444 | total_timesteps 3843.
Path 445 | total_timesteps 3850.
Path 446 | total_timesteps 3858.
Path 447 | total_timesteps 3865.
Path 448 | total_timesteps 3872.
Path 449 | total_timesteps 3883.
Path 450 | total_timesteps 3893.
Path 451 | total_timesteps 3902.
Path 452 | total_timesteps 3911.
Path 453 | total_timesteps 3918.
Path 454 | total_timesteps 3927.
Path 455 | total_timesteps 3937.
Path 456 | total_timesteps 3944.
Path 457 | total_timesteps 3950.
Path 458 | total_timesteps 3958.
Path 459 | total_timesteps 3969.
Path 460 | total_timesteps 3976.
Path 461 | total_timesteps 3983.
Path 462 | total_timesteps 3991.
Path 463 | total_timesteps 3998.
Path 464 | total_timesteps 4011.
Path 465 | total_timesteps 4018.
Path 466 | total_timesteps 4025.
Path 467 | total_timesteps 4035.
Path 468 | total_timesteps 4041.
Path 469 | total_timesteps 4049.
Path 470 | total_timesteps 4057.
Path 471 | total_timesteps 4063.
Path 472 | total_timesteps 4073.
Path 473 | total_timesteps 4083.
Path 474 | total_timesteps 4094.
Path 475 | total_timesteps 4104.
Path 476 | total_timesteps 4111.
Path 477 | total_timesteps 4117.
Path 478 | total_timesteps 4123.
Path 479 | total_timesteps 4132.
Path 480 | total_timesteps 4142.
Path 481 | total_timesteps 4149.
Path 482 | total_timesteps 4157.
Path 483 | total_timesteps 4166.
Path 484 | total_timesteps 4174.
Path 485 | total_timesteps 4189.
Path 486 | total_timesteps 4198.
Path 487 | total_timesteps 4206.
Path 488 | total_timesteps 4213.
Path 489 | total_timesteps 4221.
Path 490 | total_timesteps 4229.
Path 491 | total_timesteps 4239.
Path 492 | total_timesteps 4249.
Path 493 | total_timesteps 4262.
Path 494 | total_timesteps 4269.
Path 495 | total_timesteps 4279.
Path 496 | total_timesteps 4288.
Path 497 | total_timesteps 4296.
Path 498 | total_timesteps 4303.
Path 499 | total_timesteps 4309.
Path 500 | total_timesteps 4316.
Path 501 | total_timesteps 4330.
Path 502 | total_timesteps 4338.
Path 503 | total_timesteps 4346.
Path 504 | total_timesteps 4361.
Path 505 | total_timesteps 4368.
Path 506 | total_timesteps 4379.
Path 507 | total_timesteps 4386.
Path 508 | total_timesteps 4395.
Path 509 | total_timesteps 4402.
Path 510 | total_timesteps 4410.
Path 511 | total_timesteps 4419.
Path 512 | total_timesteps 4427.
Path 513 | total_timesteps 4436.
Path 514 | total_timesteps 4448.
Path 515 | total_timesteps 4457.
Path 516 | total_timesteps 4463.
Path 517 | total_timesteps 4471.
Path 518 | total_timesteps 4482.
Path 519 | total_timesteps 4494.
Path 520 | total_timesteps 4500.
Path 521 | total_timesteps 4513.
Path 522 | total_timesteps 4520.
Path 523 | total_timesteps 4528.
Path 524 | total_timesteps 4542.
Path 525 | total_timesteps 4548.
Path 526 | total_timesteps 4557.
Path 527 | total_timesteps 4567.
Path 528 | total_timesteps 4575.
Path 529 | total_timesteps 4585.
Path 530 | total_timesteps 4593.
Path 531 | total_timesteps 4602.
Path 532 | total_timesteps 4611.
Path 533 | total_timesteps 4623.
Path 534 | total_timesteps 4633.
Path 535 | total_timesteps 4640.
Path 536 | total_timesteps 4649.
Path 537 | total_timesteps 4660.
Path 538 | total_timesteps 4668.
Path 539 | total_timesteps 4675.
Path 540 | total_timesteps 4685.
Path 541 | total_timesteps 4693.
Path 542 | total_timesteps 4701.
Path 543 | total_timesteps 4708.
Path 544 | total_timesteps 4717.
Path 545 | total_timesteps 4726.
Path 546 | total_timesteps 4735.
Path 547 | total_timesteps 4743.
Path 548 | total_timesteps 4751.
Path 549 | total_timesteps 4762.
Path 550 | total_timesteps 4769.
Path 551 | total_timesteps 4778.
Path 552 | total_timesteps 4785.
Path 553 | total_timesteps 4797.
Path 554 | total_timesteps 4808.
Path 555 | total_timesteps 4817.
Path 556 | total_timesteps 4825.
Path 557 | total_timesteps 4835.
Path 558 | total_timesteps 4844.
Path 559 | total_timesteps 4850.
Path 560 | total_timesteps 4861.
Path 561 | total_timesteps 4872.
Path 562 | total_timesteps 4883.
Path 563 | total_timesteps 4893.
Path 564 | total_timesteps 4903.
Path 565 | total_timesteps 4912.
Path 566 | total_timesteps 4919.
Path 567 | total_timesteps 4928.
Path 568 | total_timesteps 4938.
Path 569 | total_timesteps 4945.
Path 570 | total_timesteps 4952.
Path 571 | total_timesteps 4963.
Path 572 | total_timesteps 4971.
Path 573 | total_timesteps 4979.
Path 574 | total_timesteps 4989.
Path 575 | total_timesteps 4998.
Path 576 | total_timesteps 5005.
Path 577 | total_timesteps 5015.
Path 578 | total_timesteps 5021.
Path 579 | total_timesteps 5028.
Path 580 | total_timesteps 5036.
Path 581 | total_timesteps 5044.
Path 582 | total_timesteps 5053.
Path 583 | total_timesteps 5061.
Path 584 | total_timesteps 5068.
Path 585 | total_timesteps 5077.
Path 586 | total_timesteps 5085.
Path 587 | total_timesteps 5094.
Path 588 | total_timesteps 5103.
Path 589 | total_timesteps 5118.
Path 590 | total_timesteps 5125.
Path 591 | total_timesteps 5138.
Path 592 | total_timesteps 5150.
Path 593 | total_timesteps 5158.
Path 594 | total_timesteps 5168.
Path 595 | total_timesteps 5175.
Path 596 | total_timesteps 5182.
Path 597 | total_timesteps 5198.
Path 598 | total_timesteps 5206.
Path 599 | total_timesteps 5214.
Path 600 | total_timesteps 5224.
Path 601 | total_timesteps 5231.
Path 602 | total_timesteps 5238.
Path 603 | total_timesteps 5255.
Path 604 | total_timesteps 5261.
Path 605 | total_timesteps 5271.
Path 606 | total_timesteps 5278.
Path 607 | total_timesteps 5288.
Path 608 | total_timesteps 5297.
Path 609 | total_timesteps 5307.
Path 610 | total_timesteps 5316.
Path 611 | total_timesteps 5324.
Path 612 | total_timesteps 5332.
Path 613 | total_timesteps 5340.
Path 614 | total_timesteps 5348.
Path 615 | total_timesteps 5354.
Path 616 | total_timesteps 5363.
Path 617 | total_timesteps 5370.
Path 618 | total_timesteps 5378.
Path 619 | total_timesteps 5384.
Path 620 | total_timesteps 5391.
Path 621 | total_timesteps 5399.
Path 622 | total_timesteps 5406.
Path 623 | total_timesteps 5417.
Path 624 | total_timesteps 5427.
Path 625 | total_timesteps 5435.
Path 626 | total_timesteps 5442.
Path 627 | total_timesteps 5449.
Path 628 | total_timesteps 5458.
Path 629 | total_timesteps 5470.
Path 630 | total_timesteps 5480.
Path 631 | total_timesteps 5488.
Path 632 | total_timesteps 5495.
Path 633 | total_timesteps 5502.
Path 634 | total_timesteps 5513.
Path 635 | total_timesteps 5521.
Path 636 | total_timesteps 5528.
Path 637 | total_timesteps 5535.
Path 638 | total_timesteps 5545.
Path 639 | total_timesteps 5553.
Path 640 | total_timesteps 5565.
Path 641 | total_timesteps 5575.
Path 642 | total_timesteps 5586.
Path 643 | total_timesteps 5594.
Path 644 | total_timesteps 5603.
Path 645 | total_timesteps 5613.
Path 646 | total_timesteps 5620.
Path 647 | total_timesteps 5630.
Path 648 | total_timesteps 5642.
Path 649 | total_timesteps 5650.
Path 650 | total_timesteps 5657.
Path 651 | total_timesteps 5666.
Path 652 | total_timesteps 5674.
Path 653 | total_timesteps 5680.
Path 654 | total_timesteps 5689.
Path 655 | total_timesteps 5701.
Path 656 | total_timesteps 5708.
Path 657 | total_timesteps 5716.
Path 658 | total_timesteps 5723.
Path 659 | total_timesteps 5731.
Path 660 | total_timesteps 5739.
Path 661 | total_timesteps 5746.
Path 662 | total_timesteps 5754.
Path 663 | total_timesteps 5763.
Path 664 | total_timesteps 5773.
Path 665 | total_timesteps 5780.
Path 666 | total_timesteps 5788.
Path 667 | total_timesteps 5795.
Path 668 | total_timesteps 5803.
Path 669 | total_timesteps 5811.
Path 670 | total_timesteps 5819.
Path 671 | total_timesteps 5829.
Path 672 | total_timesteps 5836.
Path 673 | total_timesteps 5843.
Path 674 | total_timesteps 5851.
Path 675 | total_timesteps 5859.
Path 676 | total_timesteps 5868.
Path 677 | total_timesteps 5875.
Path 678 | total_timesteps 5887.
Path 679 | total_timesteps 5895.
Path 680 | total_timesteps 5904.
Path 681 | total_timesteps 5914.
Path 682 | total_timesteps 5921.
Path 683 | total_timesteps 5932.
Path 684 | total_timesteps 5942.
Path 685 | total_timesteps 5950.
Path 686 | total_timesteps 5959.
Path 687 | total_timesteps 5968.
Path 688 | total_timesteps 5976.
Path 689 | total_timesteps 5985.
Path 690 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.93    |
| Iteration     | 23       |
| MaximumReturn | -1.88    |
| MinimumReturn | -15.3    |
| TotalSamples  | 100091   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.0018349327147006989
Validation loss = 0.0018774077761918306
Validation loss = 0.0017317518359050155
Validation loss = 0.001932638231664896
Validation loss = 0.0015939305303618312
Validation loss = 0.0016980187501758337
Validation loss = 0.0017852436285465956
Validation loss = 0.0016930894926190376
Validation loss = 0.001796469558030367
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 51.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 68.
Path 8 | total_timesteps 77.
Path 9 | total_timesteps 85.
Path 10 | total_timesteps 91.
Path 11 | total_timesteps 99.
Path 12 | total_timesteps 110.
Path 13 | total_timesteps 117.
Path 14 | total_timesteps 125.
Path 15 | total_timesteps 137.
Path 16 | total_timesteps 144.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 166.
Path 20 | total_timesteps 176.
Path 21 | total_timesteps 188.
Path 22 | total_timesteps 197.
Path 23 | total_timesteps 206.
Path 24 | total_timesteps 214.
Path 25 | total_timesteps 225.
Path 26 | total_timesteps 237.
Path 27 | total_timesteps 247.
Path 28 | total_timesteps 256.
Path 29 | total_timesteps 264.
Path 30 | total_timesteps 274.
Path 31 | total_timesteps 284.
Path 32 | total_timesteps 292.
Path 33 | total_timesteps 299.
Path 34 | total_timesteps 312.
Path 35 | total_timesteps 320.
Path 36 | total_timesteps 328.
Path 37 | total_timesteps 336.
Path 38 | total_timesteps 350.
Path 39 | total_timesteps 361.
Path 40 | total_timesteps 368.
Path 41 | total_timesteps 377.
Path 42 | total_timesteps 388.
Path 43 | total_timesteps 396.
Path 44 | total_timesteps 402.
Path 45 | total_timesteps 410.
Path 46 | total_timesteps 421.
Path 47 | total_timesteps 429.
Path 48 | total_timesteps 437.
Path 49 | total_timesteps 445.
Path 50 | total_timesteps 454.
Path 51 | total_timesteps 466.
Path 52 | total_timesteps 477.
Path 53 | total_timesteps 484.
Path 54 | total_timesteps 493.
Path 55 | total_timesteps 499.
Path 56 | total_timesteps 506.
Path 57 | total_timesteps 512.
Path 58 | total_timesteps 522.
Path 59 | total_timesteps 531.
Path 60 | total_timesteps 543.
Path 61 | total_timesteps 552.
Path 62 | total_timesteps 560.
Path 63 | total_timesteps 568.
Path 64 | total_timesteps 575.
Path 65 | total_timesteps 583.
Path 66 | total_timesteps 592.
Path 67 | total_timesteps 601.
Path 68 | total_timesteps 610.
Path 69 | total_timesteps 618.
Path 70 | total_timesteps 625.
Path 71 | total_timesteps 633.
Path 72 | total_timesteps 643.
Path 73 | total_timesteps 656.
Path 74 | total_timesteps 663.
Path 75 | total_timesteps 670.
Path 76 | total_timesteps 678.
Path 77 | total_timesteps 685.
Path 78 | total_timesteps 693.
Path 79 | total_timesteps 703.
Path 80 | total_timesteps 711.
Path 81 | total_timesteps 718.
Path 82 | total_timesteps 726.
Path 83 | total_timesteps 734.
Path 84 | total_timesteps 743.
Path 85 | total_timesteps 750.
Path 86 | total_timesteps 758.
Path 87 | total_timesteps 767.
Path 88 | total_timesteps 782.
Path 89 | total_timesteps 793.
Path 90 | total_timesteps 800.
Path 91 | total_timesteps 807.
Path 92 | total_timesteps 814.
Path 93 | total_timesteps 825.
Path 94 | total_timesteps 831.
Path 95 | total_timesteps 840.
Path 96 | total_timesteps 847.
Path 97 | total_timesteps 855.
Path 98 | total_timesteps 862.
Path 99 | total_timesteps 869.
Path 100 | total_timesteps 877.
Path 101 | total_timesteps 884.
Path 102 | total_timesteps 895.
Path 103 | total_timesteps 902.
Path 104 | total_timesteps 911.
Path 105 | total_timesteps 919.
Path 106 | total_timesteps 928.
Path 107 | total_timesteps 936.
Path 108 | total_timesteps 948.
Path 109 | total_timesteps 957.
Path 110 | total_timesteps 964.
Path 111 | total_timesteps 976.
Path 112 | total_timesteps 987.
Path 113 | total_timesteps 996.
Path 114 | total_timesteps 1008.
Path 115 | total_timesteps 1016.
Path 116 | total_timesteps 1025.
Path 117 | total_timesteps 1035.
Path 118 | total_timesteps 1041.
Path 119 | total_timesteps 1049.
Path 120 | total_timesteps 1057.
Path 121 | total_timesteps 1068.
Path 122 | total_timesteps 1075.
Path 123 | total_timesteps 1084.
Path 124 | total_timesteps 1095.
Path 125 | total_timesteps 1105.
Path 126 | total_timesteps 1112.
Path 127 | total_timesteps 1121.
Path 128 | total_timesteps 1130.
Path 129 | total_timesteps 1138.
Path 130 | total_timesteps 1144.
Path 131 | total_timesteps 1151.
Path 132 | total_timesteps 1160.
Path 133 | total_timesteps 1168.
Path 134 | total_timesteps 1175.
Path 135 | total_timesteps 1182.
Path 136 | total_timesteps 1191.
Path 137 | total_timesteps 1199.
Path 138 | total_timesteps 1205.
Path 139 | total_timesteps 1213.
Path 140 | total_timesteps 1221.
Path 141 | total_timesteps 1229.
Path 142 | total_timesteps 1235.
Path 143 | total_timesteps 1241.
Path 144 | total_timesteps 1249.
Path 145 | total_timesteps 1256.
Path 146 | total_timesteps 1265.
Path 147 | total_timesteps 1272.
Path 148 | total_timesteps 1281.
Path 149 | total_timesteps 1290.
Path 150 | total_timesteps 1297.
Path 151 | total_timesteps 1306.
Path 152 | total_timesteps 1313.
Path 153 | total_timesteps 1321.
Path 154 | total_timesteps 1329.
Path 155 | total_timesteps 1339.
Path 156 | total_timesteps 1348.
Path 157 | total_timesteps 1355.
Path 158 | total_timesteps 1366.
Path 159 | total_timesteps 1373.
Path 160 | total_timesteps 1384.
Path 161 | total_timesteps 1395.
Path 162 | total_timesteps 1402.
Path 163 | total_timesteps 1413.
Path 164 | total_timesteps 1421.
Path 165 | total_timesteps 1429.
Path 166 | total_timesteps 1438.
Path 167 | total_timesteps 1446.
Path 168 | total_timesteps 1455.
Path 169 | total_timesteps 1465.
Path 170 | total_timesteps 1471.
Path 171 | total_timesteps 1480.
Path 172 | total_timesteps 1489.
Path 173 | total_timesteps 1495.
Path 174 | total_timesteps 1503.
Path 175 | total_timesteps 1511.
Path 176 | total_timesteps 1520.
Path 177 | total_timesteps 1527.
Path 178 | total_timesteps 1534.
Path 179 | total_timesteps 1543.
Path 180 | total_timesteps 1553.
Path 181 | total_timesteps 1561.
Path 182 | total_timesteps 1568.
Path 183 | total_timesteps 1575.
Path 184 | total_timesteps 1591.
Path 185 | total_timesteps 1600.
Path 186 | total_timesteps 1610.
Path 187 | total_timesteps 1618.
Path 188 | total_timesteps 1625.
Path 189 | total_timesteps 1632.
Path 190 | total_timesteps 1640.
Path 191 | total_timesteps 1649.
Path 192 | total_timesteps 1657.
Path 193 | total_timesteps 1667.
Path 194 | total_timesteps 1675.
Path 195 | total_timesteps 1682.
Path 196 | total_timesteps 1693.
Path 197 | total_timesteps 1700.
Path 198 | total_timesteps 1709.
Path 199 | total_timesteps 1719.
Path 200 | total_timesteps 1726.
Path 201 | total_timesteps 1737.
Path 202 | total_timesteps 1747.
Path 203 | total_timesteps 1753.
Path 204 | total_timesteps 1764.
Path 205 | total_timesteps 1772.
Path 206 | total_timesteps 1782.
Path 207 | total_timesteps 1790.
Path 208 | total_timesteps 1797.
Path 209 | total_timesteps 1805.
Path 210 | total_timesteps 1817.
Path 211 | total_timesteps 1825.
Path 212 | total_timesteps 1832.
Path 213 | total_timesteps 1845.
Path 214 | total_timesteps 1854.
Path 215 | total_timesteps 1861.
Path 216 | total_timesteps 1868.
Path 217 | total_timesteps 1878.
Path 218 | total_timesteps 1887.
Path 219 | total_timesteps 1895.
Path 220 | total_timesteps 1904.
Path 221 | total_timesteps 1912.
Path 222 | total_timesteps 1923.
Path 223 | total_timesteps 1935.
Path 224 | total_timesteps 1943.
Path 225 | total_timesteps 1950.
Path 226 | total_timesteps 1956.
Path 227 | total_timesteps 1966.
Path 228 | total_timesteps 1977.
Path 229 | total_timesteps 1986.
Path 230 | total_timesteps 1996.
Path 231 | total_timesteps 2005.
Path 232 | total_timesteps 2013.
Path 233 | total_timesteps 2022.
Path 234 | total_timesteps 2039.
Path 235 | total_timesteps 2048.
Path 236 | total_timesteps 2055.
Path 237 | total_timesteps 2061.
Path 238 | total_timesteps 2069.
Path 239 | total_timesteps 2076.
Path 240 | total_timesteps 2084.
Path 241 | total_timesteps 2093.
Path 242 | total_timesteps 2108.
Path 243 | total_timesteps 2115.
Path 244 | total_timesteps 2125.
Path 245 | total_timesteps 2132.
Path 246 | total_timesteps 2144.
Path 247 | total_timesteps 2154.
Path 248 | total_timesteps 2161.
Path 249 | total_timesteps 2172.
Path 250 | total_timesteps 2182.
Path 251 | total_timesteps 2190.
Path 252 | total_timesteps 2202.
Path 253 | total_timesteps 2209.
Path 254 | total_timesteps 2217.
Path 255 | total_timesteps 2225.
Path 256 | total_timesteps 2232.
Path 257 | total_timesteps 2238.
Path 258 | total_timesteps 2247.
Path 259 | total_timesteps 2253.
Path 260 | total_timesteps 2262.
Path 261 | total_timesteps 2271.
Path 262 | total_timesteps 2280.
Path 263 | total_timesteps 2287.
Path 264 | total_timesteps 2295.
Path 265 | total_timesteps 2310.
Path 266 | total_timesteps 2320.
Path 267 | total_timesteps 2329.
Path 268 | total_timesteps 2337.
Path 269 | total_timesteps 2346.
Path 270 | total_timesteps 2355.
Path 271 | total_timesteps 2362.
Path 272 | total_timesteps 2369.
Path 273 | total_timesteps 2376.
Path 274 | total_timesteps 2389.
Path 275 | total_timesteps 2396.
Path 276 | total_timesteps 2407.
Path 277 | total_timesteps 2416.
Path 278 | total_timesteps 2423.
Path 279 | total_timesteps 2433.
Path 280 | total_timesteps 2440.
Path 281 | total_timesteps 2448.
Path 282 | total_timesteps 2456.
Path 283 | total_timesteps 2469.
Path 284 | total_timesteps 2477.
Path 285 | total_timesteps 2485.
Path 286 | total_timesteps 2496.
Path 287 | total_timesteps 2503.
Path 288 | total_timesteps 2510.
Path 289 | total_timesteps 2517.
Path 290 | total_timesteps 2524.
Path 291 | total_timesteps 2533.
Path 292 | total_timesteps 2546.
Path 293 | total_timesteps 2554.
Path 294 | total_timesteps 2562.
Path 295 | total_timesteps 2569.
Path 296 | total_timesteps 2575.
Path 297 | total_timesteps 2582.
Path 298 | total_timesteps 2590.
Path 299 | total_timesteps 2597.
Path 300 | total_timesteps 2604.
Path 301 | total_timesteps 2611.
Path 302 | total_timesteps 2621.
Path 303 | total_timesteps 2629.
Path 304 | total_timesteps 2637.
Path 305 | total_timesteps 2647.
Path 306 | total_timesteps 2664.
Path 307 | total_timesteps 2675.
Path 308 | total_timesteps 2682.
Path 309 | total_timesteps 2689.
Path 310 | total_timesteps 2697.
Path 311 | total_timesteps 2705.
Path 312 | total_timesteps 2715.
Path 313 | total_timesteps 2722.
Path 314 | total_timesteps 2732.
Path 315 | total_timesteps 2744.
Path 316 | total_timesteps 2751.
Path 317 | total_timesteps 2759.
Path 318 | total_timesteps 2766.
Path 319 | total_timesteps 2777.
Path 320 | total_timesteps 2792.
Path 321 | total_timesteps 2800.
Path 322 | total_timesteps 2808.
Path 323 | total_timesteps 2820.
Path 324 | total_timesteps 2827.
Path 325 | total_timesteps 2835.
Path 326 | total_timesteps 2847.
Path 327 | total_timesteps 2854.
Path 328 | total_timesteps 2861.
Path 329 | total_timesteps 2870.
Path 330 | total_timesteps 2884.
Path 331 | total_timesteps 2892.
Path 332 | total_timesteps 2900.
Path 333 | total_timesteps 2910.
Path 334 | total_timesteps 2920.
Path 335 | total_timesteps 2927.
Path 336 | total_timesteps 2937.
Path 337 | total_timesteps 2944.
Path 338 | total_timesteps 2951.
Path 339 | total_timesteps 2959.
Path 340 | total_timesteps 2967.
Path 341 | total_timesteps 2976.
Path 342 | total_timesteps 2983.
Path 343 | total_timesteps 2996.
Path 344 | total_timesteps 3003.
Path 345 | total_timesteps 3010.
Path 346 | total_timesteps 3018.
Path 347 | total_timesteps 3031.
Path 348 | total_timesteps 3041.
Path 349 | total_timesteps 3048.
Path 350 | total_timesteps 3055.
Path 351 | total_timesteps 3065.
Path 352 | total_timesteps 3074.
Path 353 | total_timesteps 3085.
Path 354 | total_timesteps 3094.
Path 355 | total_timesteps 3102.
Path 356 | total_timesteps 3112.
Path 357 | total_timesteps 3120.
Path 358 | total_timesteps 3127.
Path 359 | total_timesteps 3135.
Path 360 | total_timesteps 3143.
Path 361 | total_timesteps 3149.
Path 362 | total_timesteps 3157.
Path 363 | total_timesteps 3165.
Path 364 | total_timesteps 3172.
Path 365 | total_timesteps 3181.
Path 366 | total_timesteps 3188.
Path 367 | total_timesteps 3195.
Path 368 | total_timesteps 3205.
Path 369 | total_timesteps 3214.
Path 370 | total_timesteps 3224.
Path 371 | total_timesteps 3235.
Path 372 | total_timesteps 3243.
Path 373 | total_timesteps 3251.
Path 374 | total_timesteps 3258.
Path 375 | total_timesteps 3266.
Path 376 | total_timesteps 3277.
Path 377 | total_timesteps 3286.
Path 378 | total_timesteps 3295.
Path 379 | total_timesteps 3305.
Path 380 | total_timesteps 3312.
Path 381 | total_timesteps 3319.
Path 382 | total_timesteps 3327.
Path 383 | total_timesteps 3336.
Path 384 | total_timesteps 3344.
Path 385 | total_timesteps 3353.
Path 386 | total_timesteps 3360.
Path 387 | total_timesteps 3367.
Path 388 | total_timesteps 3376.
Path 389 | total_timesteps 3385.
Path 390 | total_timesteps 3394.
Path 391 | total_timesteps 3402.
Path 392 | total_timesteps 3414.
Path 393 | total_timesteps 3420.
Path 394 | total_timesteps 3427.
Path 395 | total_timesteps 3434.
Path 396 | total_timesteps 3445.
Path 397 | total_timesteps 3453.
Path 398 | total_timesteps 3460.
Path 399 | total_timesteps 3470.
Path 400 | total_timesteps 3481.
Path 401 | total_timesteps 3490.
Path 402 | total_timesteps 3498.
Path 403 | total_timesteps 3507.
Path 404 | total_timesteps 3513.
Path 405 | total_timesteps 3521.
Path 406 | total_timesteps 3530.
Path 407 | total_timesteps 3538.
Path 408 | total_timesteps 3551.
Path 409 | total_timesteps 3558.
Path 410 | total_timesteps 3567.
Path 411 | total_timesteps 3581.
Path 412 | total_timesteps 3589.
Path 413 | total_timesteps 3601.
Path 414 | total_timesteps 3610.
Path 415 | total_timesteps 3622.
Path 416 | total_timesteps 3630.
Path 417 | total_timesteps 3640.
Path 418 | total_timesteps 3649.
Path 419 | total_timesteps 3657.
Path 420 | total_timesteps 3664.
Path 421 | total_timesteps 3671.
Path 422 | total_timesteps 3680.
Path 423 | total_timesteps 3689.
Path 424 | total_timesteps 3698.
Path 425 | total_timesteps 3706.
Path 426 | total_timesteps 3717.
Path 427 | total_timesteps 3728.
Path 428 | total_timesteps 3736.
Path 429 | total_timesteps 3743.
Path 430 | total_timesteps 3749.
Path 431 | total_timesteps 3756.
Path 432 | total_timesteps 3765.
Path 433 | total_timesteps 3773.
Path 434 | total_timesteps 3784.
Path 435 | total_timesteps 3791.
Path 436 | total_timesteps 3800.
Path 437 | total_timesteps 3808.
Path 438 | total_timesteps 3816.
Path 439 | total_timesteps 3823.
Path 440 | total_timesteps 3831.
Path 441 | total_timesteps 3837.
Path 442 | total_timesteps 3843.
Path 443 | total_timesteps 3857.
Path 444 | total_timesteps 3864.
Path 445 | total_timesteps 3877.
Path 446 | total_timesteps 3885.
Path 447 | total_timesteps 3892.
Path 448 | total_timesteps 3899.
Path 449 | total_timesteps 3906.
Path 450 | total_timesteps 3914.
Path 451 | total_timesteps 3920.
Path 452 | total_timesteps 3926.
Path 453 | total_timesteps 3935.
Path 454 | total_timesteps 3943.
Path 455 | total_timesteps 3951.
Path 456 | total_timesteps 3959.
Path 457 | total_timesteps 3972.
Path 458 | total_timesteps 3980.
Path 459 | total_timesteps 3988.
Path 460 | total_timesteps 3996.
Path 461 | total_timesteps 4003.
Path 462 | total_timesteps 4012.
Path 463 | total_timesteps 4023.
Path 464 | total_timesteps 4031.
Path 465 | total_timesteps 4037.
Path 466 | total_timesteps 4045.
Path 467 | total_timesteps 4054.
Path 468 | total_timesteps 4063.
Path 469 | total_timesteps 4071.
Path 470 | total_timesteps 4078.
Path 471 | total_timesteps 4087.
Path 472 | total_timesteps 4098.
Path 473 | total_timesteps 4107.
Path 474 | total_timesteps 4114.
Path 475 | total_timesteps 4125.
Path 476 | total_timesteps 4132.
Path 477 | total_timesteps 4139.
Path 478 | total_timesteps 4151.
Path 479 | total_timesteps 4158.
Path 480 | total_timesteps 4166.
Path 481 | total_timesteps 4177.
Path 482 | total_timesteps 4186.
Path 483 | total_timesteps 4194.
Path 484 | total_timesteps 4209.
Path 485 | total_timesteps 4222.
Path 486 | total_timesteps 4232.
Path 487 | total_timesteps 4239.
Path 488 | total_timesteps 4251.
Path 489 | total_timesteps 4260.
Path 490 | total_timesteps 4266.
Path 491 | total_timesteps 4274.
Path 492 | total_timesteps 4280.
Path 493 | total_timesteps 4288.
Path 494 | total_timesteps 4297.
Path 495 | total_timesteps 4308.
Path 496 | total_timesteps 4315.
Path 497 | total_timesteps 4321.
Path 498 | total_timesteps 4328.
Path 499 | total_timesteps 4337.
Path 500 | total_timesteps 4350.
Path 501 | total_timesteps 4357.
Path 502 | total_timesteps 4365.
Path 503 | total_timesteps 4374.
Path 504 | total_timesteps 4383.
Path 505 | total_timesteps 4391.
Path 506 | total_timesteps 4398.
Path 507 | total_timesteps 4405.
Path 508 | total_timesteps 4414.
Path 509 | total_timesteps 4421.
Path 510 | total_timesteps 4432.
Path 511 | total_timesteps 4443.
Path 512 | total_timesteps 4457.
Path 513 | total_timesteps 4466.
Path 514 | total_timesteps 4473.
Path 515 | total_timesteps 4480.
Path 516 | total_timesteps 4489.
Path 517 | total_timesteps 4507.
Path 518 | total_timesteps 4515.
Path 519 | total_timesteps 4524.
Path 520 | total_timesteps 4533.
Path 521 | total_timesteps 4545.
Path 522 | total_timesteps 4552.
Path 523 | total_timesteps 4559.
Path 524 | total_timesteps 4568.
Path 525 | total_timesteps 4577.
Path 526 | total_timesteps 4587.
Path 527 | total_timesteps 4595.
Path 528 | total_timesteps 4604.
Path 529 | total_timesteps 4615.
Path 530 | total_timesteps 4623.
Path 531 | total_timesteps 4632.
Path 532 | total_timesteps 4640.
Path 533 | total_timesteps 4648.
Path 534 | total_timesteps 4655.
Path 535 | total_timesteps 4664.
Path 536 | total_timesteps 4672.
Path 537 | total_timesteps 4679.
Path 538 | total_timesteps 4687.
Path 539 | total_timesteps 4697.
Path 540 | total_timesteps 4705.
Path 541 | total_timesteps 4715.
Path 542 | total_timesteps 4722.
Path 543 | total_timesteps 4730.
Path 544 | total_timesteps 4738.
Path 545 | total_timesteps 4744.
Path 546 | total_timesteps 4751.
Path 547 | total_timesteps 4761.
Path 548 | total_timesteps 4768.
Path 549 | total_timesteps 4777.
Path 550 | total_timesteps 4787.
Path 551 | total_timesteps 4798.
Path 552 | total_timesteps 4804.
Path 553 | total_timesteps 4814.
Path 554 | total_timesteps 4821.
Path 555 | total_timesteps 4830.
Path 556 | total_timesteps 4838.
Path 557 | total_timesteps 4845.
Path 558 | total_timesteps 4853.
Path 559 | total_timesteps 4861.
Path 560 | total_timesteps 4869.
Path 561 | total_timesteps 4877.
Path 562 | total_timesteps 4885.
Path 563 | total_timesteps 4893.
Path 564 | total_timesteps 4906.
Path 565 | total_timesteps 4913.
Path 566 | total_timesteps 4920.
Path 567 | total_timesteps 4928.
Path 568 | total_timesteps 4936.
Path 569 | total_timesteps 4946.
Path 570 | total_timesteps 4954.
Path 571 | total_timesteps 4962.
Path 572 | total_timesteps 4973.
Path 573 | total_timesteps 4985.
Path 574 | total_timesteps 4993.
Path 575 | total_timesteps 5006.
Path 576 | total_timesteps 5014.
Path 577 | total_timesteps 5023.
Path 578 | total_timesteps 5032.
Path 579 | total_timesteps 5040.
Path 580 | total_timesteps 5049.
Path 581 | total_timesteps 5057.
Path 582 | total_timesteps 5067.
Path 583 | total_timesteps 5076.
Path 584 | total_timesteps 5084.
Path 585 | total_timesteps 5091.
Path 586 | total_timesteps 5098.
Path 587 | total_timesteps 5111.
Path 588 | total_timesteps 5119.
Path 589 | total_timesteps 5131.
Path 590 | total_timesteps 5140.
Path 591 | total_timesteps 5147.
Path 592 | total_timesteps 5154.
Path 593 | total_timesteps 5161.
Path 594 | total_timesteps 5168.
Path 595 | total_timesteps 5176.
Path 596 | total_timesteps 5185.
Path 597 | total_timesteps 5195.
Path 598 | total_timesteps 5202.
Path 599 | total_timesteps 5208.
Path 600 | total_timesteps 5215.
Path 601 | total_timesteps 5225.
Path 602 | total_timesteps 5236.
Path 603 | total_timesteps 5243.
Path 604 | total_timesteps 5250.
Path 605 | total_timesteps 5256.
Path 606 | total_timesteps 5262.
Path 607 | total_timesteps 5271.
Path 608 | total_timesteps 5282.
Path 609 | total_timesteps 5290.
Path 610 | total_timesteps 5296.
Path 611 | total_timesteps 5303.
Path 612 | total_timesteps 5310.
Path 613 | total_timesteps 5319.
Path 614 | total_timesteps 5329.
Path 615 | total_timesteps 5336.
Path 616 | total_timesteps 5346.
Path 617 | total_timesteps 5355.
Path 618 | total_timesteps 5361.
Path 619 | total_timesteps 5368.
Path 620 | total_timesteps 5375.
Path 621 | total_timesteps 5383.
Path 622 | total_timesteps 5393.
Path 623 | total_timesteps 5406.
Path 624 | total_timesteps 5416.
Path 625 | total_timesteps 5426.
Path 626 | total_timesteps 5434.
Path 627 | total_timesteps 5443.
Path 628 | total_timesteps 5451.
Path 629 | total_timesteps 5461.
Path 630 | total_timesteps 5472.
Path 631 | total_timesteps 5480.
Path 632 | total_timesteps 5487.
Path 633 | total_timesteps 5496.
Path 634 | total_timesteps 5505.
Path 635 | total_timesteps 5516.
Path 636 | total_timesteps 5527.
Path 637 | total_timesteps 5536.
Path 638 | total_timesteps 5546.
Path 639 | total_timesteps 5552.
Path 640 | total_timesteps 5560.
Path 641 | total_timesteps 5567.
Path 642 | total_timesteps 5574.
Path 643 | total_timesteps 5583.
Path 644 | total_timesteps 5593.
Path 645 | total_timesteps 5601.
Path 646 | total_timesteps 5613.
Path 647 | total_timesteps 5622.
Path 648 | total_timesteps 5628.
Path 649 | total_timesteps 5636.
Path 650 | total_timesteps 5643.
Path 651 | total_timesteps 5651.
Path 652 | total_timesteps 5659.
Path 653 | total_timesteps 5667.
Path 654 | total_timesteps 5674.
Path 655 | total_timesteps 5683.
Path 656 | total_timesteps 5691.
Path 657 | total_timesteps 5698.
Path 658 | total_timesteps 5706.
Path 659 | total_timesteps 5717.
Path 660 | total_timesteps 5725.
Path 661 | total_timesteps 5736.
Path 662 | total_timesteps 5747.
Path 663 | total_timesteps 5757.
Path 664 | total_timesteps 5764.
Path 665 | total_timesteps 5771.
Path 666 | total_timesteps 5778.
Path 667 | total_timesteps 5789.
Path 668 | total_timesteps 5798.
Path 669 | total_timesteps 5806.
Path 670 | total_timesteps 5813.
Path 671 | total_timesteps 5823.
Path 672 | total_timesteps 5829.
Path 673 | total_timesteps 5839.
Path 674 | total_timesteps 5845.
Path 675 | total_timesteps 5854.
Path 676 | total_timesteps 5861.
Path 677 | total_timesteps 5875.
Path 678 | total_timesteps 5884.
Path 679 | total_timesteps 5891.
Path 680 | total_timesteps 5899.
Path 681 | total_timesteps 5905.
Path 682 | total_timesteps 5915.
Path 683 | total_timesteps 5923.
Path 684 | total_timesteps 5932.
Path 685 | total_timesteps 5943.
Path 686 | total_timesteps 5953.
Path 687 | total_timesteps 5959.
Path 688 | total_timesteps 5972.
Path 689 | total_timesteps 5979.
Path 690 | total_timesteps 5987.
Path 691 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.88    |
| Iteration     | 24       |
| MaximumReturn | 0.00587  |
| MinimumReturn | -17.1    |
| TotalSamples  | 104095   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.001778896083123982
Validation loss = 0.0016694192308932543
Validation loss = 0.001708613010123372
Validation loss = 0.0016691270284354687
Validation loss = 0.0016228191088885069
Validation loss = 0.00196842011064291
Validation loss = 0.0016861980548128486
Validation loss = 0.0019072388531640172
Validation loss = 0.0016759018180891871
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 17.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 33.
Path 5 | total_timesteps 40.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 55.
Path 8 | total_timesteps 62.
Path 9 | total_timesteps 69.
Path 10 | total_timesteps 76.
Path 11 | total_timesteps 85.
Path 12 | total_timesteps 94.
Path 13 | total_timesteps 101.
Path 14 | total_timesteps 109.
Path 15 | total_timesteps 117.
Path 16 | total_timesteps 127.
Path 17 | total_timesteps 135.
Path 18 | total_timesteps 141.
Path 19 | total_timesteps 148.
Path 20 | total_timesteps 156.
Path 21 | total_timesteps 165.
Path 22 | total_timesteps 178.
Path 23 | total_timesteps 185.
Path 24 | total_timesteps 195.
Path 25 | total_timesteps 204.
Path 26 | total_timesteps 212.
Path 27 | total_timesteps 222.
Path 28 | total_timesteps 229.
Path 29 | total_timesteps 236.
Path 30 | total_timesteps 245.
Path 31 | total_timesteps 254.
Path 32 | total_timesteps 261.
Path 33 | total_timesteps 272.
Path 34 | total_timesteps 279.
Path 35 | total_timesteps 286.
Path 36 | total_timesteps 295.
Path 37 | total_timesteps 308.
Path 38 | total_timesteps 316.
Path 39 | total_timesteps 323.
Path 40 | total_timesteps 333.
Path 41 | total_timesteps 341.
Path 42 | total_timesteps 347.
Path 43 | total_timesteps 355.
Path 44 | total_timesteps 372.
Path 45 | total_timesteps 379.
Path 46 | total_timesteps 385.
Path 47 | total_timesteps 394.
Path 48 | total_timesteps 403.
Path 49 | total_timesteps 425.
Path 50 | total_timesteps 432.
Path 51 | total_timesteps 440.
Path 52 | total_timesteps 448.
Path 53 | total_timesteps 456.
Path 54 | total_timesteps 468.
Path 55 | total_timesteps 478.
Path 56 | total_timesteps 486.
Path 57 | total_timesteps 493.
Path 58 | total_timesteps 501.
Path 59 | total_timesteps 509.
Path 60 | total_timesteps 520.
Path 61 | total_timesteps 527.
Path 62 | total_timesteps 535.
Path 63 | total_timesteps 543.
Path 64 | total_timesteps 553.
Path 65 | total_timesteps 561.
Path 66 | total_timesteps 569.
Path 67 | total_timesteps 578.
Path 68 | total_timesteps 584.
Path 69 | total_timesteps 591.
Path 70 | total_timesteps 601.
Path 71 | total_timesteps 608.
Path 72 | total_timesteps 614.
Path 73 | total_timesteps 622.
Path 74 | total_timesteps 632.
Path 75 | total_timesteps 642.
Path 76 | total_timesteps 650.
Path 77 | total_timesteps 656.
Path 78 | total_timesteps 671.
Path 79 | total_timesteps 685.
Path 80 | total_timesteps 697.
Path 81 | total_timesteps 704.
Path 82 | total_timesteps 711.
Path 83 | total_timesteps 720.
Path 84 | total_timesteps 727.
Path 85 | total_timesteps 735.
Path 86 | total_timesteps 743.
Path 87 | total_timesteps 752.
Path 88 | total_timesteps 760.
Path 89 | total_timesteps 768.
Path 90 | total_timesteps 776.
Path 91 | total_timesteps 786.
Path 92 | total_timesteps 796.
Path 93 | total_timesteps 804.
Path 94 | total_timesteps 811.
Path 95 | total_timesteps 819.
Path 96 | total_timesteps 830.
Path 97 | total_timesteps 837.
Path 98 | total_timesteps 844.
Path 99 | total_timesteps 850.
Path 100 | total_timesteps 858.
Path 101 | total_timesteps 866.
Path 102 | total_timesteps 879.
Path 103 | total_timesteps 890.
Path 104 | total_timesteps 898.
Path 105 | total_timesteps 906.
Path 106 | total_timesteps 914.
Path 107 | total_timesteps 921.
Path 108 | total_timesteps 933.
Path 109 | total_timesteps 948.
Path 110 | total_timesteps 955.
Path 111 | total_timesteps 966.
Path 112 | total_timesteps 974.
Path 113 | total_timesteps 981.
Path 114 | total_timesteps 988.
Path 115 | total_timesteps 999.
Path 116 | total_timesteps 1006.
Path 117 | total_timesteps 1012.
Path 118 | total_timesteps 1019.
Path 119 | total_timesteps 1027.
Path 120 | total_timesteps 1036.
Path 121 | total_timesteps 1044.
Path 122 | total_timesteps 1051.
Path 123 | total_timesteps 1059.
Path 124 | total_timesteps 1069.
Path 125 | total_timesteps 1079.
Path 126 | total_timesteps 1091.
Path 127 | total_timesteps 1098.
Path 128 | total_timesteps 1104.
Path 129 | total_timesteps 1112.
Path 130 | total_timesteps 1122.
Path 131 | total_timesteps 1130.
Path 132 | total_timesteps 1139.
Path 133 | total_timesteps 1147.
Path 134 | total_timesteps 1156.
Path 135 | total_timesteps 1165.
Path 136 | total_timesteps 1172.
Path 137 | total_timesteps 1184.
Path 138 | total_timesteps 1192.
Path 139 | total_timesteps 1199.
Path 140 | total_timesteps 1205.
Path 141 | total_timesteps 1212.
Path 142 | total_timesteps 1221.
Path 143 | total_timesteps 1229.
Path 144 | total_timesteps 1237.
Path 145 | total_timesteps 1246.
Path 146 | total_timesteps 1254.
Path 147 | total_timesteps 1265.
Path 148 | total_timesteps 1273.
Path 149 | total_timesteps 1281.
Path 150 | total_timesteps 1287.
Path 151 | total_timesteps 1298.
Path 152 | total_timesteps 1310.
Path 153 | total_timesteps 1319.
Path 154 | total_timesteps 1329.
Path 155 | total_timesteps 1336.
Path 156 | total_timesteps 1354.
Path 157 | total_timesteps 1364.
Path 158 | total_timesteps 1372.
Path 159 | total_timesteps 1380.
Path 160 | total_timesteps 1391.
Path 161 | total_timesteps 1402.
Path 162 | total_timesteps 1414.
Path 163 | total_timesteps 1424.
Path 164 | total_timesteps 1433.
Path 165 | total_timesteps 1439.
Path 166 | total_timesteps 1446.
Path 167 | total_timesteps 1457.
Path 168 | total_timesteps 1465.
Path 169 | total_timesteps 1471.
Path 170 | total_timesteps 1478.
Path 171 | total_timesteps 1486.
Path 172 | total_timesteps 1495.
Path 173 | total_timesteps 1503.
Path 174 | total_timesteps 1511.
Path 175 | total_timesteps 1519.
Path 176 | total_timesteps 1528.
Path 177 | total_timesteps 1535.
Path 178 | total_timesteps 1544.
Path 179 | total_timesteps 1557.
Path 180 | total_timesteps 1564.
Path 181 | total_timesteps 1575.
Path 182 | total_timesteps 1581.
Path 183 | total_timesteps 1592.
Path 184 | total_timesteps 1600.
Path 185 | total_timesteps 1608.
Path 186 | total_timesteps 1614.
Path 187 | total_timesteps 1627.
Path 188 | total_timesteps 1634.
Path 189 | total_timesteps 1645.
Path 190 | total_timesteps 1652.
Path 191 | total_timesteps 1663.
Path 192 | total_timesteps 1671.
Path 193 | total_timesteps 1678.
Path 194 | total_timesteps 1685.
Path 195 | total_timesteps 1691.
Path 196 | total_timesteps 1701.
Path 197 | total_timesteps 1709.
Path 198 | total_timesteps 1718.
Path 199 | total_timesteps 1726.
Path 200 | total_timesteps 1734.
Path 201 | total_timesteps 1745.
Path 202 | total_timesteps 1757.
Path 203 | total_timesteps 1770.
Path 204 | total_timesteps 1778.
Path 205 | total_timesteps 1789.
Path 206 | total_timesteps 1798.
Path 207 | total_timesteps 1804.
Path 208 | total_timesteps 1817.
Path 209 | total_timesteps 1825.
Path 210 | total_timesteps 1834.
Path 211 | total_timesteps 1842.
Path 212 | total_timesteps 1850.
Path 213 | total_timesteps 1863.
Path 214 | total_timesteps 1869.
Path 215 | total_timesteps 1875.
Path 216 | total_timesteps 1884.
Path 217 | total_timesteps 1893.
Path 218 | total_timesteps 1900.
Path 219 | total_timesteps 1907.
Path 220 | total_timesteps 1915.
Path 221 | total_timesteps 1926.
Path 222 | total_timesteps 1932.
Path 223 | total_timesteps 1944.
Path 224 | total_timesteps 1952.
Path 225 | total_timesteps 1960.
Path 226 | total_timesteps 1967.
Path 227 | total_timesteps 1974.
Path 228 | total_timesteps 1983.
Path 229 | total_timesteps 1989.
Path 230 | total_timesteps 1996.
Path 231 | total_timesteps 2005.
Path 232 | total_timesteps 2014.
Path 233 | total_timesteps 2023.
Path 234 | total_timesteps 2032.
Path 235 | total_timesteps 2041.
Path 236 | total_timesteps 2048.
Path 237 | total_timesteps 2054.
Path 238 | total_timesteps 2060.
Path 239 | total_timesteps 2068.
Path 240 | total_timesteps 2076.
Path 241 | total_timesteps 2086.
Path 242 | total_timesteps 2095.
Path 243 | total_timesteps 2102.
Path 244 | total_timesteps 2108.
Path 245 | total_timesteps 2115.
Path 246 | total_timesteps 2121.
Path 247 | total_timesteps 2129.
Path 248 | total_timesteps 2138.
Path 249 | total_timesteps 2146.
Path 250 | total_timesteps 2153.
Path 251 | total_timesteps 2163.
Path 252 | total_timesteps 2169.
Path 253 | total_timesteps 2178.
Path 254 | total_timesteps 2187.
Path 255 | total_timesteps 2198.
Path 256 | total_timesteps 2210.
Path 257 | total_timesteps 2221.
Path 258 | total_timesteps 2230.
Path 259 | total_timesteps 2244.
Path 260 | total_timesteps 2254.
Path 261 | total_timesteps 2264.
Path 262 | total_timesteps 2275.
Path 263 | total_timesteps 2283.
Path 264 | total_timesteps 2293.
Path 265 | total_timesteps 2300.
Path 266 | total_timesteps 2308.
Path 267 | total_timesteps 2317.
Path 268 | total_timesteps 2325.
Path 269 | total_timesteps 2334.
Path 270 | total_timesteps 2343.
Path 271 | total_timesteps 2352.
Path 272 | total_timesteps 2359.
Path 273 | total_timesteps 2367.
Path 274 | total_timesteps 2376.
Path 275 | total_timesteps 2383.
Path 276 | total_timesteps 2391.
Path 277 | total_timesteps 2401.
Path 278 | total_timesteps 2409.
Path 279 | total_timesteps 2416.
Path 280 | total_timesteps 2424.
Path 281 | total_timesteps 2431.
Path 282 | total_timesteps 2439.
Path 283 | total_timesteps 2446.
Path 284 | total_timesteps 2454.
Path 285 | total_timesteps 2461.
Path 286 | total_timesteps 2470.
Path 287 | total_timesteps 2476.
Path 288 | total_timesteps 2484.
Path 289 | total_timesteps 2493.
Path 290 | total_timesteps 2501.
Path 291 | total_timesteps 2510.
Path 292 | total_timesteps 2517.
Path 293 | total_timesteps 2524.
Path 294 | total_timesteps 2530.
Path 295 | total_timesteps 2540.
Path 296 | total_timesteps 2550.
Path 297 | total_timesteps 2559.
Path 298 | total_timesteps 2565.
Path 299 | total_timesteps 2571.
Path 300 | total_timesteps 2579.
Path 301 | total_timesteps 2589.
Path 302 | total_timesteps 2596.
Path 303 | total_timesteps 2603.
Path 304 | total_timesteps 2610.
Path 305 | total_timesteps 2617.
Path 306 | total_timesteps 2627.
Path 307 | total_timesteps 2634.
Path 308 | total_timesteps 2642.
Path 309 | total_timesteps 2650.
Path 310 | total_timesteps 2660.
Path 311 | total_timesteps 2666.
Path 312 | total_timesteps 2675.
Path 313 | total_timesteps 2682.
Path 314 | total_timesteps 2693.
Path 315 | total_timesteps 2702.
Path 316 | total_timesteps 2712.
Path 317 | total_timesteps 2722.
Path 318 | total_timesteps 2731.
Path 319 | total_timesteps 2739.
Path 320 | total_timesteps 2751.
Path 321 | total_timesteps 2758.
Path 322 | total_timesteps 2765.
Path 323 | total_timesteps 2775.
Path 324 | total_timesteps 2783.
Path 325 | total_timesteps 2791.
Path 326 | total_timesteps 2804.
Path 327 | total_timesteps 2811.
Path 328 | total_timesteps 2822.
Path 329 | total_timesteps 2835.
Path 330 | total_timesteps 2842.
Path 331 | total_timesteps 2849.
Path 332 | total_timesteps 2858.
Path 333 | total_timesteps 2869.
Path 334 | total_timesteps 2876.
Path 335 | total_timesteps 2883.
Path 336 | total_timesteps 2890.
Path 337 | total_timesteps 2897.
Path 338 | total_timesteps 2904.
Path 339 | total_timesteps 2911.
Path 340 | total_timesteps 2919.
Path 341 | total_timesteps 2927.
Path 342 | total_timesteps 2934.
Path 343 | total_timesteps 2942.
Path 344 | total_timesteps 2954.
Path 345 | total_timesteps 2962.
Path 346 | total_timesteps 2971.
Path 347 | total_timesteps 2984.
Path 348 | total_timesteps 2990.
Path 349 | total_timesteps 2998.
Path 350 | total_timesteps 3004.
Path 351 | total_timesteps 3017.
Path 352 | total_timesteps 3025.
Path 353 | total_timesteps 3035.
Path 354 | total_timesteps 3042.
Path 355 | total_timesteps 3049.
Path 356 | total_timesteps 3062.
Path 357 | total_timesteps 3072.
Path 358 | total_timesteps 3080.
Path 359 | total_timesteps 3087.
Path 360 | total_timesteps 3097.
Path 361 | total_timesteps 3105.
Path 362 | total_timesteps 3111.
Path 363 | total_timesteps 3119.
Path 364 | total_timesteps 3127.
Path 365 | total_timesteps 3134.
Path 366 | total_timesteps 3140.
Path 367 | total_timesteps 3149.
Path 368 | total_timesteps 3158.
Path 369 | total_timesteps 3165.
Path 370 | total_timesteps 3172.
Path 371 | total_timesteps 3179.
Path 372 | total_timesteps 3188.
Path 373 | total_timesteps 3197.
Path 374 | total_timesteps 3205.
Path 375 | total_timesteps 3218.
Path 376 | total_timesteps 3230.
Path 377 | total_timesteps 3237.
Path 378 | total_timesteps 3244.
Path 379 | total_timesteps 3250.
Path 380 | total_timesteps 3258.
Path 381 | total_timesteps 3265.
Path 382 | total_timesteps 3274.
Path 383 | total_timesteps 3281.
Path 384 | total_timesteps 3289.
Path 385 | total_timesteps 3299.
Path 386 | total_timesteps 3309.
Path 387 | total_timesteps 3317.
Path 388 | total_timesteps 3324.
Path 389 | total_timesteps 3331.
Path 390 | total_timesteps 3339.
Path 391 | total_timesteps 3348.
Path 392 | total_timesteps 3357.
Path 393 | total_timesteps 3365.
Path 394 | total_timesteps 3373.
Path 395 | total_timesteps 3381.
Path 396 | total_timesteps 3388.
Path 397 | total_timesteps 3395.
Path 398 | total_timesteps 3407.
Path 399 | total_timesteps 3415.
Path 400 | total_timesteps 3424.
Path 401 | total_timesteps 3431.
Path 402 | total_timesteps 3441.
Path 403 | total_timesteps 3453.
Path 404 | total_timesteps 3462.
Path 405 | total_timesteps 3470.
Path 406 | total_timesteps 3483.
Path 407 | total_timesteps 3490.
Path 408 | total_timesteps 3497.
Path 409 | total_timesteps 3505.
Path 410 | total_timesteps 3514.
Path 411 | total_timesteps 3521.
Path 412 | total_timesteps 3532.
Path 413 | total_timesteps 3539.
Path 414 | total_timesteps 3548.
Path 415 | total_timesteps 3556.
Path 416 | total_timesteps 3563.
Path 417 | total_timesteps 3570.
Path 418 | total_timesteps 3579.
Path 419 | total_timesteps 3586.
Path 420 | total_timesteps 3592.
Path 421 | total_timesteps 3601.
Path 422 | total_timesteps 3609.
Path 423 | total_timesteps 3617.
Path 424 | total_timesteps 3623.
Path 425 | total_timesteps 3629.
Path 426 | total_timesteps 3635.
Path 427 | total_timesteps 3644.
Path 428 | total_timesteps 3653.
Path 429 | total_timesteps 3665.
Path 430 | total_timesteps 3672.
Path 431 | total_timesteps 3678.
Path 432 | total_timesteps 3686.
Path 433 | total_timesteps 3694.
Path 434 | total_timesteps 3703.
Path 435 | total_timesteps 3711.
Path 436 | total_timesteps 3719.
Path 437 | total_timesteps 3725.
Path 438 | total_timesteps 3732.
Path 439 | total_timesteps 3739.
Path 440 | total_timesteps 3746.
Path 441 | total_timesteps 3752.
Path 442 | total_timesteps 3761.
Path 443 | total_timesteps 3768.
Path 444 | total_timesteps 3776.
Path 445 | total_timesteps 3783.
Path 446 | total_timesteps 3790.
Path 447 | total_timesteps 3800.
Path 448 | total_timesteps 3810.
Path 449 | total_timesteps 3819.
Path 450 | total_timesteps 3828.
Path 451 | total_timesteps 3838.
Path 452 | total_timesteps 3846.
Path 453 | total_timesteps 3855.
Path 454 | total_timesteps 3864.
Path 455 | total_timesteps 3874.
Path 456 | total_timesteps 3882.
Path 457 | total_timesteps 3892.
Path 458 | total_timesteps 3902.
Path 459 | total_timesteps 3909.
Path 460 | total_timesteps 3919.
Path 461 | total_timesteps 3928.
Path 462 | total_timesteps 3937.
Path 463 | total_timesteps 3944.
Path 464 | total_timesteps 3951.
Path 465 | total_timesteps 3958.
Path 466 | total_timesteps 3967.
Path 467 | total_timesteps 3974.
Path 468 | total_timesteps 3982.
Path 469 | total_timesteps 3992.
Path 470 | total_timesteps 4000.
Path 471 | total_timesteps 4007.
Path 472 | total_timesteps 4018.
Path 473 | total_timesteps 4029.
Path 474 | total_timesteps 4037.
Path 475 | total_timesteps 4048.
Path 476 | total_timesteps 4055.
Path 477 | total_timesteps 4061.
Path 478 | total_timesteps 4071.
Path 479 | total_timesteps 4085.
Path 480 | total_timesteps 4095.
Path 481 | total_timesteps 4102.
Path 482 | total_timesteps 4110.
Path 483 | total_timesteps 4120.
Path 484 | total_timesteps 4129.
Path 485 | total_timesteps 4137.
Path 486 | total_timesteps 4146.
Path 487 | total_timesteps 4154.
Path 488 | total_timesteps 4162.
Path 489 | total_timesteps 4172.
Path 490 | total_timesteps 4179.
Path 491 | total_timesteps 4186.
Path 492 | total_timesteps 4198.
Path 493 | total_timesteps 4205.
Path 494 | total_timesteps 4214.
Path 495 | total_timesteps 4222.
Path 496 | total_timesteps 4229.
Path 497 | total_timesteps 4239.
Path 498 | total_timesteps 4247.
Path 499 | total_timesteps 4255.
Path 500 | total_timesteps 4267.
Path 501 | total_timesteps 4276.
Path 502 | total_timesteps 4284.
Path 503 | total_timesteps 4292.
Path 504 | total_timesteps 4301.
Path 505 | total_timesteps 4310.
Path 506 | total_timesteps 4317.
Path 507 | total_timesteps 4325.
Path 508 | total_timesteps 4335.
Path 509 | total_timesteps 4341.
Path 510 | total_timesteps 4348.
Path 511 | total_timesteps 4359.
Path 512 | total_timesteps 4367.
Path 513 | total_timesteps 4376.
Path 514 | total_timesteps 4383.
Path 515 | total_timesteps 4390.
Path 516 | total_timesteps 4397.
Path 517 | total_timesteps 4404.
Path 518 | total_timesteps 4411.
Path 519 | total_timesteps 4422.
Path 520 | total_timesteps 4430.
Path 521 | total_timesteps 4437.
Path 522 | total_timesteps 4443.
Path 523 | total_timesteps 4452.
Path 524 | total_timesteps 4461.
Path 525 | total_timesteps 4474.
Path 526 | total_timesteps 4480.
Path 527 | total_timesteps 4487.
Path 528 | total_timesteps 4496.
Path 529 | total_timesteps 4504.
Path 530 | total_timesteps 4512.
Path 531 | total_timesteps 4519.
Path 532 | total_timesteps 4526.
Path 533 | total_timesteps 4532.
Path 534 | total_timesteps 4540.
Path 535 | total_timesteps 4548.
Path 536 | total_timesteps 4559.
Path 537 | total_timesteps 4566.
Path 538 | total_timesteps 4573.
Path 539 | total_timesteps 4583.
Path 540 | total_timesteps 4590.
Path 541 | total_timesteps 4601.
Path 542 | total_timesteps 4610.
Path 543 | total_timesteps 4617.
Path 544 | total_timesteps 4626.
Path 545 | total_timesteps 4634.
Path 546 | total_timesteps 4642.
Path 547 | total_timesteps 4648.
Path 548 | total_timesteps 4654.
Path 549 | total_timesteps 4663.
Path 550 | total_timesteps 4674.
Path 551 | total_timesteps 4682.
Path 552 | total_timesteps 4690.
Path 553 | total_timesteps 4697.
Path 554 | total_timesteps 4704.
Path 555 | total_timesteps 4715.
Path 556 | total_timesteps 4722.
Path 557 | total_timesteps 4729.
Path 558 | total_timesteps 4736.
Path 559 | total_timesteps 4743.
Path 560 | total_timesteps 4749.
Path 561 | total_timesteps 4756.
Path 562 | total_timesteps 4762.
Path 563 | total_timesteps 4772.
Path 564 | total_timesteps 4779.
Path 565 | total_timesteps 4785.
Path 566 | total_timesteps 4793.
Path 567 | total_timesteps 4801.
Path 568 | total_timesteps 4809.
Path 569 | total_timesteps 4819.
Path 570 | total_timesteps 4826.
Path 571 | total_timesteps 4834.
Path 572 | total_timesteps 4841.
Path 573 | total_timesteps 4850.
Path 574 | total_timesteps 4859.
Path 575 | total_timesteps 4865.
Path 576 | total_timesteps 4874.
Path 577 | total_timesteps 4882.
Path 578 | total_timesteps 4894.
Path 579 | total_timesteps 4902.
Path 580 | total_timesteps 4915.
Path 581 | total_timesteps 4926.
Path 582 | total_timesteps 4942.
Path 583 | total_timesteps 4949.
Path 584 | total_timesteps 4955.
Path 585 | total_timesteps 4962.
Path 586 | total_timesteps 4971.
Path 587 | total_timesteps 4979.
Path 588 | total_timesteps 4988.
Path 589 | total_timesteps 4998.
Path 590 | total_timesteps 5006.
Path 591 | total_timesteps 5017.
Path 592 | total_timesteps 5028.
Path 593 | total_timesteps 5040.
Path 594 | total_timesteps 5048.
Path 595 | total_timesteps 5055.
Path 596 | total_timesteps 5062.
Path 597 | total_timesteps 5069.
Path 598 | total_timesteps 5076.
Path 599 | total_timesteps 5083.
Path 600 | total_timesteps 5091.
Path 601 | total_timesteps 5098.
Path 602 | total_timesteps 5106.
Path 603 | total_timesteps 5114.
Path 604 | total_timesteps 5123.
Path 605 | total_timesteps 5132.
Path 606 | total_timesteps 5140.
Path 607 | total_timesteps 5148.
Path 608 | total_timesteps 5155.
Path 609 | total_timesteps 5164.
Path 610 | total_timesteps 5172.
Path 611 | total_timesteps 5180.
Path 612 | total_timesteps 5190.
Path 613 | total_timesteps 5202.
Path 614 | total_timesteps 5213.
Path 615 | total_timesteps 5220.
Path 616 | total_timesteps 5230.
Path 617 | total_timesteps 5238.
Path 618 | total_timesteps 5247.
Path 619 | total_timesteps 5257.
Path 620 | total_timesteps 5266.
Path 621 | total_timesteps 5273.
Path 622 | total_timesteps 5283.
Path 623 | total_timesteps 5291.
Path 624 | total_timesteps 5300.
Path 625 | total_timesteps 5307.
Path 626 | total_timesteps 5313.
Path 627 | total_timesteps 5321.
Path 628 | total_timesteps 5328.
Path 629 | total_timesteps 5336.
Path 630 | total_timesteps 5346.
Path 631 | total_timesteps 5353.
Path 632 | total_timesteps 5361.
Path 633 | total_timesteps 5368.
Path 634 | total_timesteps 5375.
Path 635 | total_timesteps 5383.
Path 636 | total_timesteps 5391.
Path 637 | total_timesteps 5400.
Path 638 | total_timesteps 5408.
Path 639 | total_timesteps 5418.
Path 640 | total_timesteps 5425.
Path 641 | total_timesteps 5432.
Path 642 | total_timesteps 5443.
Path 643 | total_timesteps 5451.
Path 644 | total_timesteps 5460.
Path 645 | total_timesteps 5467.
Path 646 | total_timesteps 5477.
Path 647 | total_timesteps 5484.
Path 648 | total_timesteps 5492.
Path 649 | total_timesteps 5502.
Path 650 | total_timesteps 5510.
Path 651 | total_timesteps 5518.
Path 652 | total_timesteps 5525.
Path 653 | total_timesteps 5532.
Path 654 | total_timesteps 5544.
Path 655 | total_timesteps 5552.
Path 656 | total_timesteps 5560.
Path 657 | total_timesteps 5570.
Path 658 | total_timesteps 5577.
Path 659 | total_timesteps 5586.
Path 660 | total_timesteps 5593.
Path 661 | total_timesteps 5602.
Path 662 | total_timesteps 5609.
Path 663 | total_timesteps 5622.
Path 664 | total_timesteps 5629.
Path 665 | total_timesteps 5638.
Path 666 | total_timesteps 5649.
Path 667 | total_timesteps 5663.
Path 668 | total_timesteps 5671.
Path 669 | total_timesteps 5684.
Path 670 | total_timesteps 5693.
Path 671 | total_timesteps 5700.
Path 672 | total_timesteps 5709.
Path 673 | total_timesteps 5720.
Path 674 | total_timesteps 5729.
Path 675 | total_timesteps 5737.
Path 676 | total_timesteps 5745.
Path 677 | total_timesteps 5752.
Path 678 | total_timesteps 5760.
Path 679 | total_timesteps 5771.
Path 680 | total_timesteps 5780.
Path 681 | total_timesteps 5788.
Path 682 | total_timesteps 5797.
Path 683 | total_timesteps 5804.
Path 684 | total_timesteps 5813.
Path 685 | total_timesteps 5821.
Path 686 | total_timesteps 5828.
Path 687 | total_timesteps 5835.
Path 688 | total_timesteps 5843.
Path 689 | total_timesteps 5851.
Path 690 | total_timesteps 5860.
Path 691 | total_timesteps 5869.
Path 692 | total_timesteps 5878.
Path 693 | total_timesteps 5889.
Path 694 | total_timesteps 5897.
Path 695 | total_timesteps 5904.
Path 696 | total_timesteps 5911.
Path 697 | total_timesteps 5919.
Path 698 | total_timesteps 5928.
Path 699 | total_timesteps 5935.
Path 700 | total_timesteps 5946.
Path 701 | total_timesteps 5953.
Path 702 | total_timesteps 5961.
Path 703 | total_timesteps 5968.
Path 704 | total_timesteps 5976.
Path 705 | total_timesteps 5984.
Path 706 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.09    |
| Iteration     | 25       |
| MaximumReturn | 0.199    |
| MinimumReturn | -20      |
| TotalSamples  | 108095   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.0015815444057807326
Validation loss = 0.0016929672565311193
Validation loss = 0.0015486010815948248
Validation loss = 0.0015946293715387583
Validation loss = 0.0015691129956394434
Validation loss = 0.0017936477670446038
Validation loss = 0.0015047385822981596
Validation loss = 0.0016124264802783728
Validation loss = 0.0016200437676161528
Validation loss = 0.0017414861358702183
Validation loss = 0.0015649156412109733
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 61.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 88.
Path 10 | total_timesteps 97.
Path 11 | total_timesteps 105.
Path 12 | total_timesteps 113.
Path 13 | total_timesteps 121.
Path 14 | total_timesteps 129.
Path 15 | total_timesteps 136.
Path 16 | total_timesteps 146.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 159.
Path 19 | total_timesteps 167.
Path 20 | total_timesteps 173.
Path 21 | total_timesteps 181.
Path 22 | total_timesteps 190.
Path 23 | total_timesteps 196.
Path 24 | total_timesteps 208.
Path 25 | total_timesteps 219.
Path 26 | total_timesteps 229.
Path 27 | total_timesteps 236.
Path 28 | total_timesteps 245.
Path 29 | total_timesteps 253.
Path 30 | total_timesteps 262.
Path 31 | total_timesteps 269.
Path 32 | total_timesteps 278.
Path 33 | total_timesteps 285.
Path 34 | total_timesteps 294.
Path 35 | total_timesteps 304.
Path 36 | total_timesteps 314.
Path 37 | total_timesteps 328.
Path 38 | total_timesteps 336.
Path 39 | total_timesteps 345.
Path 40 | total_timesteps 353.
Path 41 | total_timesteps 360.
Path 42 | total_timesteps 369.
Path 43 | total_timesteps 377.
Path 44 | total_timesteps 385.
Path 45 | total_timesteps 393.
Path 46 | total_timesteps 405.
Path 47 | total_timesteps 414.
Path 48 | total_timesteps 426.
Path 49 | total_timesteps 435.
Path 50 | total_timesteps 447.
Path 51 | total_timesteps 456.
Path 52 | total_timesteps 464.
Path 53 | total_timesteps 474.
Path 54 | total_timesteps 482.
Path 55 | total_timesteps 489.
Path 56 | total_timesteps 498.
Path 57 | total_timesteps 510.
Path 58 | total_timesteps 517.
Path 59 | total_timesteps 527.
Path 60 | total_timesteps 537.
Path 61 | total_timesteps 550.
Path 62 | total_timesteps 559.
Path 63 | total_timesteps 567.
Path 64 | total_timesteps 574.
Path 65 | total_timesteps 582.
Path 66 | total_timesteps 593.
Path 67 | total_timesteps 600.
Path 68 | total_timesteps 607.
Path 69 | total_timesteps 615.
Path 70 | total_timesteps 623.
Path 71 | total_timesteps 631.
Path 72 | total_timesteps 642.
Path 73 | total_timesteps 652.
Path 74 | total_timesteps 659.
Path 75 | total_timesteps 668.
Path 76 | total_timesteps 680.
Path 77 | total_timesteps 687.
Path 78 | total_timesteps 695.
Path 79 | total_timesteps 703.
Path 80 | total_timesteps 712.
Path 81 | total_timesteps 726.
Path 82 | total_timesteps 733.
Path 83 | total_timesteps 741.
Path 84 | total_timesteps 749.
Path 85 | total_timesteps 761.
Path 86 | total_timesteps 771.
Path 87 | total_timesteps 779.
Path 88 | total_timesteps 792.
Path 89 | total_timesteps 800.
Path 90 | total_timesteps 808.
Path 91 | total_timesteps 815.
Path 92 | total_timesteps 822.
Path 93 | total_timesteps 830.
Path 94 | total_timesteps 840.
Path 95 | total_timesteps 848.
Path 96 | total_timesteps 857.
Path 97 | total_timesteps 864.
Path 98 | total_timesteps 873.
Path 99 | total_timesteps 880.
Path 100 | total_timesteps 886.
Path 101 | total_timesteps 896.
Path 102 | total_timesteps 907.
Path 103 | total_timesteps 914.
Path 104 | total_timesteps 921.
Path 105 | total_timesteps 930.
Path 106 | total_timesteps 939.
Path 107 | total_timesteps 945.
Path 108 | total_timesteps 955.
Path 109 | total_timesteps 965.
Path 110 | total_timesteps 972.
Path 111 | total_timesteps 979.
Path 112 | total_timesteps 989.
Path 113 | total_timesteps 1001.
Path 114 | total_timesteps 1009.
Path 115 | total_timesteps 1020.
Path 116 | total_timesteps 1031.
Path 117 | total_timesteps 1039.
Path 118 | total_timesteps 1046.
Path 119 | total_timesteps 1052.
Path 120 | total_timesteps 1059.
Path 121 | total_timesteps 1068.
Path 122 | total_timesteps 1074.
Path 123 | total_timesteps 1080.
Path 124 | total_timesteps 1090.
Path 125 | total_timesteps 1098.
Path 126 | total_timesteps 1105.
Path 127 | total_timesteps 1114.
Path 128 | total_timesteps 1123.
Path 129 | total_timesteps 1129.
Path 130 | total_timesteps 1136.
Path 131 | total_timesteps 1143.
Path 132 | total_timesteps 1151.
Path 133 | total_timesteps 1161.
Path 134 | total_timesteps 1169.
Path 135 | total_timesteps 1180.
Path 136 | total_timesteps 1192.
Path 137 | total_timesteps 1201.
Path 138 | total_timesteps 1208.
Path 139 | total_timesteps 1215.
Path 140 | total_timesteps 1222.
Path 141 | total_timesteps 1232.
Path 142 | total_timesteps 1239.
Path 143 | total_timesteps 1246.
Path 144 | total_timesteps 1256.
Path 145 | total_timesteps 1264.
Path 146 | total_timesteps 1274.
Path 147 | total_timesteps 1282.
Path 148 | total_timesteps 1288.
Path 149 | total_timesteps 1297.
Path 150 | total_timesteps 1305.
Path 151 | total_timesteps 1313.
Path 152 | total_timesteps 1319.
Path 153 | total_timesteps 1327.
Path 154 | total_timesteps 1337.
Path 155 | total_timesteps 1344.
Path 156 | total_timesteps 1353.
Path 157 | total_timesteps 1360.
Path 158 | total_timesteps 1367.
Path 159 | total_timesteps 1378.
Path 160 | total_timesteps 1385.
Path 161 | total_timesteps 1394.
Path 162 | total_timesteps 1401.
Path 163 | total_timesteps 1409.
Path 164 | total_timesteps 1419.
Path 165 | total_timesteps 1433.
Path 166 | total_timesteps 1440.
Path 167 | total_timesteps 1447.
Path 168 | total_timesteps 1454.
Path 169 | total_timesteps 1462.
Path 170 | total_timesteps 1474.
Path 171 | total_timesteps 1481.
Path 172 | total_timesteps 1489.
Path 173 | total_timesteps 1496.
Path 174 | total_timesteps 1506.
Path 175 | total_timesteps 1514.
Path 176 | total_timesteps 1523.
Path 177 | total_timesteps 1532.
Path 178 | total_timesteps 1540.
Path 179 | total_timesteps 1548.
Path 180 | total_timesteps 1560.
Path 181 | total_timesteps 1568.
Path 182 | total_timesteps 1576.
Path 183 | total_timesteps 1584.
Path 184 | total_timesteps 1594.
Path 185 | total_timesteps 1602.
Path 186 | total_timesteps 1610.
Path 187 | total_timesteps 1617.
Path 188 | total_timesteps 1624.
Path 189 | total_timesteps 1632.
Path 190 | total_timesteps 1641.
Path 191 | total_timesteps 1649.
Path 192 | total_timesteps 1655.
Path 193 | total_timesteps 1664.
Path 194 | total_timesteps 1675.
Path 195 | total_timesteps 1684.
Path 196 | total_timesteps 1693.
Path 197 | total_timesteps 1701.
Path 198 | total_timesteps 1707.
Path 199 | total_timesteps 1714.
Path 200 | total_timesteps 1722.
Path 201 | total_timesteps 1728.
Path 202 | total_timesteps 1740.
Path 203 | total_timesteps 1749.
Path 204 | total_timesteps 1759.
Path 205 | total_timesteps 1766.
Path 206 | total_timesteps 1777.
Path 207 | total_timesteps 1786.
Path 208 | total_timesteps 1795.
Path 209 | total_timesteps 1802.
Path 210 | total_timesteps 1810.
Path 211 | total_timesteps 1818.
Path 212 | total_timesteps 1825.
Path 213 | total_timesteps 1834.
Path 214 | total_timesteps 1842.
Path 215 | total_timesteps 1848.
Path 216 | total_timesteps 1855.
Path 217 | total_timesteps 1862.
Path 218 | total_timesteps 1871.
Path 219 | total_timesteps 1883.
Path 220 | total_timesteps 1892.
Path 221 | total_timesteps 1906.
Path 222 | total_timesteps 1914.
Path 223 | total_timesteps 1925.
Path 224 | total_timesteps 1938.
Path 225 | total_timesteps 1945.
Path 226 | total_timesteps 1956.
Path 227 | total_timesteps 1965.
Path 228 | total_timesteps 1971.
Path 229 | total_timesteps 1984.
Path 230 | total_timesteps 1991.
Path 231 | total_timesteps 1997.
Path 232 | total_timesteps 2004.
Path 233 | total_timesteps 2010.
Path 234 | total_timesteps 2017.
Path 235 | total_timesteps 2025.
Path 236 | total_timesteps 2034.
Path 237 | total_timesteps 2042.
Path 238 | total_timesteps 2053.
Path 239 | total_timesteps 2063.
Path 240 | total_timesteps 2073.
Path 241 | total_timesteps 2084.
Path 242 | total_timesteps 2093.
Path 243 | total_timesteps 2108.
Path 244 | total_timesteps 2117.
Path 245 | total_timesteps 2124.
Path 246 | total_timesteps 2131.
Path 247 | total_timesteps 2141.
Path 248 | total_timesteps 2150.
Path 249 | total_timesteps 2156.
Path 250 | total_timesteps 2163.
Path 251 | total_timesteps 2170.
Path 252 | total_timesteps 2177.
Path 253 | total_timesteps 2184.
Path 254 | total_timesteps 2194.
Path 255 | total_timesteps 2205.
Path 256 | total_timesteps 2212.
Path 257 | total_timesteps 2221.
Path 258 | total_timesteps 2235.
Path 259 | total_timesteps 2244.
Path 260 | total_timesteps 2251.
Path 261 | total_timesteps 2257.
Path 262 | total_timesteps 2266.
Path 263 | total_timesteps 2277.
Path 264 | total_timesteps 2289.
Path 265 | total_timesteps 2295.
Path 266 | total_timesteps 2303.
Path 267 | total_timesteps 2312.
Path 268 | total_timesteps 2324.
Path 269 | total_timesteps 2332.
Path 270 | total_timesteps 2343.
Path 271 | total_timesteps 2351.
Path 272 | total_timesteps 2359.
Path 273 | total_timesteps 2368.
Path 274 | total_timesteps 2375.
Path 275 | total_timesteps 2388.
Path 276 | total_timesteps 2395.
Path 277 | total_timesteps 2407.
Path 278 | total_timesteps 2418.
Path 279 | total_timesteps 2427.
Path 280 | total_timesteps 2434.
Path 281 | total_timesteps 2441.
Path 282 | total_timesteps 2450.
Path 283 | total_timesteps 2459.
Path 284 | total_timesteps 2466.
Path 285 | total_timesteps 2476.
Path 286 | total_timesteps 2483.
Path 287 | total_timesteps 2489.
Path 288 | total_timesteps 2498.
Path 289 | total_timesteps 2509.
Path 290 | total_timesteps 2516.
Path 291 | total_timesteps 2525.
Path 292 | total_timesteps 2532.
Path 293 | total_timesteps 2539.
Path 294 | total_timesteps 2547.
Path 295 | total_timesteps 2553.
Path 296 | total_timesteps 2563.
Path 297 | total_timesteps 2571.
Path 298 | total_timesteps 2578.
Path 299 | total_timesteps 2589.
Path 300 | total_timesteps 2602.
Path 301 | total_timesteps 2609.
Path 302 | total_timesteps 2617.
Path 303 | total_timesteps 2627.
Path 304 | total_timesteps 2635.
Path 305 | total_timesteps 2644.
Path 306 | total_timesteps 2650.
Path 307 | total_timesteps 2662.
Path 308 | total_timesteps 2673.
Path 309 | total_timesteps 2680.
Path 310 | total_timesteps 2689.
Path 311 | total_timesteps 2697.
Path 312 | total_timesteps 2704.
Path 313 | total_timesteps 2712.
Path 314 | total_timesteps 2721.
Path 315 | total_timesteps 2728.
Path 316 | total_timesteps 2737.
Path 317 | total_timesteps 2746.
Path 318 | total_timesteps 2752.
Path 319 | total_timesteps 2759.
Path 320 | total_timesteps 2769.
Path 321 | total_timesteps 2782.
Path 322 | total_timesteps 2791.
Path 323 | total_timesteps 2800.
Path 324 | total_timesteps 2806.
Path 325 | total_timesteps 2815.
Path 326 | total_timesteps 2823.
Path 327 | total_timesteps 2833.
Path 328 | total_timesteps 2841.
Path 329 | total_timesteps 2851.
Path 330 | total_timesteps 2859.
Path 331 | total_timesteps 2867.
Path 332 | total_timesteps 2875.
Path 333 | total_timesteps 2884.
Path 334 | total_timesteps 2892.
Path 335 | total_timesteps 2898.
Path 336 | total_timesteps 2907.
Path 337 | total_timesteps 2917.
Path 338 | total_timesteps 2929.
Path 339 | total_timesteps 2935.
Path 340 | total_timesteps 2943.
Path 341 | total_timesteps 2950.
Path 342 | total_timesteps 2957.
Path 343 | total_timesteps 2964.
Path 344 | total_timesteps 2970.
Path 345 | total_timesteps 2978.
Path 346 | total_timesteps 2985.
Path 347 | total_timesteps 2999.
Path 348 | total_timesteps 3007.
Path 349 | total_timesteps 3015.
Path 350 | total_timesteps 3024.
Path 351 | total_timesteps 3032.
Path 352 | total_timesteps 3038.
Path 353 | total_timesteps 3046.
Path 354 | total_timesteps 3054.
Path 355 | total_timesteps 3061.
Path 356 | total_timesteps 3070.
Path 357 | total_timesteps 3079.
Path 358 | total_timesteps 3088.
Path 359 | total_timesteps 3096.
Path 360 | total_timesteps 3104.
Path 361 | total_timesteps 3112.
Path 362 | total_timesteps 3123.
Path 363 | total_timesteps 3131.
Path 364 | total_timesteps 3137.
Path 365 | total_timesteps 3143.
Path 366 | total_timesteps 3151.
Path 367 | total_timesteps 3159.
Path 368 | total_timesteps 3167.
Path 369 | total_timesteps 3182.
Path 370 | total_timesteps 3189.
Path 371 | total_timesteps 3200.
Path 372 | total_timesteps 3209.
Path 373 | total_timesteps 3215.
Path 374 | total_timesteps 3223.
Path 375 | total_timesteps 3233.
Path 376 | total_timesteps 3243.
Path 377 | total_timesteps 3253.
Path 378 | total_timesteps 3260.
Path 379 | total_timesteps 3267.
Path 380 | total_timesteps 3274.
Path 381 | total_timesteps 3282.
Path 382 | total_timesteps 3289.
Path 383 | total_timesteps 3296.
Path 384 | total_timesteps 3303.
Path 385 | total_timesteps 3319.
Path 386 | total_timesteps 3327.
Path 387 | total_timesteps 3338.
Path 388 | total_timesteps 3344.
Path 389 | total_timesteps 3353.
Path 390 | total_timesteps 3361.
Path 391 | total_timesteps 3368.
Path 392 | total_timesteps 3380.
Path 393 | total_timesteps 3389.
Path 394 | total_timesteps 3398.
Path 395 | total_timesteps 3405.
Path 396 | total_timesteps 3416.
Path 397 | total_timesteps 3423.
Path 398 | total_timesteps 3431.
Path 399 | total_timesteps 3442.
Path 400 | total_timesteps 3450.
Path 401 | total_timesteps 3460.
Path 402 | total_timesteps 3468.
Path 403 | total_timesteps 3477.
Path 404 | total_timesteps 3484.
Path 405 | total_timesteps 3492.
Path 406 | total_timesteps 3499.
Path 407 | total_timesteps 3507.
Path 408 | total_timesteps 3519.
Path 409 | total_timesteps 3528.
Path 410 | total_timesteps 3539.
Path 411 | total_timesteps 3548.
Path 412 | total_timesteps 3555.
Path 413 | total_timesteps 3563.
Path 414 | total_timesteps 3570.
Path 415 | total_timesteps 3580.
Path 416 | total_timesteps 3588.
Path 417 | total_timesteps 3599.
Path 418 | total_timesteps 3607.
Path 419 | total_timesteps 3615.
Path 420 | total_timesteps 3621.
Path 421 | total_timesteps 3629.
Path 422 | total_timesteps 3642.
Path 423 | total_timesteps 3649.
Path 424 | total_timesteps 3657.
Path 425 | total_timesteps 3667.
Path 426 | total_timesteps 3674.
Path 427 | total_timesteps 3681.
Path 428 | total_timesteps 3687.
Path 429 | total_timesteps 3697.
Path 430 | total_timesteps 3705.
Path 431 | total_timesteps 3713.
Path 432 | total_timesteps 3720.
Path 433 | total_timesteps 3728.
Path 434 | total_timesteps 3737.
Path 435 | total_timesteps 3746.
Path 436 | total_timesteps 3755.
Path 437 | total_timesteps 3767.
Path 438 | total_timesteps 3774.
Path 439 | total_timesteps 3784.
Path 440 | total_timesteps 3792.
Path 441 | total_timesteps 3798.
Path 442 | total_timesteps 3806.
Path 443 | total_timesteps 3813.
Path 444 | total_timesteps 3820.
Path 445 | total_timesteps 3828.
Path 446 | total_timesteps 3840.
Path 447 | total_timesteps 3847.
Path 448 | total_timesteps 3854.
Path 449 | total_timesteps 3861.
Path 450 | total_timesteps 3868.
Path 451 | total_timesteps 3877.
Path 452 | total_timesteps 3888.
Path 453 | total_timesteps 3897.
Path 454 | total_timesteps 3903.
Path 455 | total_timesteps 3913.
Path 456 | total_timesteps 3923.
Path 457 | total_timesteps 3934.
Path 458 | total_timesteps 3941.
Path 459 | total_timesteps 3949.
Path 460 | total_timesteps 3967.
Path 461 | total_timesteps 3976.
Path 462 | total_timesteps 3985.
Path 463 | total_timesteps 3993.
Path 464 | total_timesteps 4006.
Path 465 | total_timesteps 4013.
Path 466 | total_timesteps 4020.
Path 467 | total_timesteps 4030.
Path 468 | total_timesteps 4043.
Path 469 | total_timesteps 4050.
Path 470 | total_timesteps 4058.
Path 471 | total_timesteps 4068.
Path 472 | total_timesteps 4075.
Path 473 | total_timesteps 4087.
Path 474 | total_timesteps 4096.
Path 475 | total_timesteps 4104.
Path 476 | total_timesteps 4110.
Path 477 | total_timesteps 4121.
Path 478 | total_timesteps 4129.
Path 479 | total_timesteps 4136.
Path 480 | total_timesteps 4143.
Path 481 | total_timesteps 4152.
Path 482 | total_timesteps 4159.
Path 483 | total_timesteps 4168.
Path 484 | total_timesteps 4174.
Path 485 | total_timesteps 4182.
Path 486 | total_timesteps 4190.
Path 487 | total_timesteps 4199.
Path 488 | total_timesteps 4206.
Path 489 | total_timesteps 4213.
Path 490 | total_timesteps 4219.
Path 491 | total_timesteps 4228.
Path 492 | total_timesteps 4239.
Path 493 | total_timesteps 4248.
Path 494 | total_timesteps 4254.
Path 495 | total_timesteps 4262.
Path 496 | total_timesteps 4273.
Path 497 | total_timesteps 4280.
Path 498 | total_timesteps 4286.
Path 499 | total_timesteps 4293.
Path 500 | total_timesteps 4301.
Path 501 | total_timesteps 4309.
Path 502 | total_timesteps 4317.
Path 503 | total_timesteps 4324.
Path 504 | total_timesteps 4332.
Path 505 | total_timesteps 4338.
Path 506 | total_timesteps 4350.
Path 507 | total_timesteps 4358.
Path 508 | total_timesteps 4368.
Path 509 | total_timesteps 4377.
Path 510 | total_timesteps 4385.
Path 511 | total_timesteps 4396.
Path 512 | total_timesteps 4402.
Path 513 | total_timesteps 4410.
Path 514 | total_timesteps 4416.
Path 515 | total_timesteps 4426.
Path 516 | total_timesteps 4434.
Path 517 | total_timesteps 4442.
Path 518 | total_timesteps 4448.
Path 519 | total_timesteps 4459.
Path 520 | total_timesteps 4467.
Path 521 | total_timesteps 4479.
Path 522 | total_timesteps 4486.
Path 523 | total_timesteps 4492.
Path 524 | total_timesteps 4500.
Path 525 | total_timesteps 4511.
Path 526 | total_timesteps 4521.
Path 527 | total_timesteps 4529.
Path 528 | total_timesteps 4539.
Path 529 | total_timesteps 4546.
Path 530 | total_timesteps 4556.
Path 531 | total_timesteps 4563.
Path 532 | total_timesteps 4572.
Path 533 | total_timesteps 4583.
Path 534 | total_timesteps 4590.
Path 535 | total_timesteps 4597.
Path 536 | total_timesteps 4606.
Path 537 | total_timesteps 4615.
Path 538 | total_timesteps 4621.
Path 539 | total_timesteps 4630.
Path 540 | total_timesteps 4637.
Path 541 | total_timesteps 4644.
Path 542 | total_timesteps 4652.
Path 543 | total_timesteps 4663.
Path 544 | total_timesteps 4670.
Path 545 | total_timesteps 4680.
Path 546 | total_timesteps 4689.
Path 547 | total_timesteps 4698.
Path 548 | total_timesteps 4704.
Path 549 | total_timesteps 4712.
Path 550 | total_timesteps 4724.
Path 551 | total_timesteps 4735.
Path 552 | total_timesteps 4743.
Path 553 | total_timesteps 4751.
Path 554 | total_timesteps 4762.
Path 555 | total_timesteps 4768.
Path 556 | total_timesteps 4776.
Path 557 | total_timesteps 4788.
Path 558 | total_timesteps 4795.
Path 559 | total_timesteps 4804.
Path 560 | total_timesteps 4812.
Path 561 | total_timesteps 4820.
Path 562 | total_timesteps 4826.
Path 563 | total_timesteps 4837.
Path 564 | total_timesteps 4846.
Path 565 | total_timesteps 4854.
Path 566 | total_timesteps 4862.
Path 567 | total_timesteps 4876.
Path 568 | total_timesteps 4884.
Path 569 | total_timesteps 4894.
Path 570 | total_timesteps 4902.
Path 571 | total_timesteps 4911.
Path 572 | total_timesteps 4918.
Path 573 | total_timesteps 4925.
Path 574 | total_timesteps 4931.
Path 575 | total_timesteps 4941.
Path 576 | total_timesteps 4948.
Path 577 | total_timesteps 4958.
Path 578 | total_timesteps 4964.
Path 579 | total_timesteps 4974.
Path 580 | total_timesteps 4982.
Path 581 | total_timesteps 4989.
Path 582 | total_timesteps 4998.
Path 583 | total_timesteps 5007.
Path 584 | total_timesteps 5014.
Path 585 | total_timesteps 5022.
Path 586 | total_timesteps 5030.
Path 587 | total_timesteps 5039.
Path 588 | total_timesteps 5049.
Path 589 | total_timesteps 5056.
Path 590 | total_timesteps 5063.
Path 591 | total_timesteps 5072.
Path 592 | total_timesteps 5081.
Path 593 | total_timesteps 5090.
Path 594 | total_timesteps 5098.
Path 595 | total_timesteps 5106.
Path 596 | total_timesteps 5113.
Path 597 | total_timesteps 5120.
Path 598 | total_timesteps 5129.
Path 599 | total_timesteps 5142.
Path 600 | total_timesteps 5150.
Path 601 | total_timesteps 5162.
Path 602 | total_timesteps 5173.
Path 603 | total_timesteps 5181.
Path 604 | total_timesteps 5190.
Path 605 | total_timesteps 5197.
Path 606 | total_timesteps 5206.
Path 607 | total_timesteps 5213.
Path 608 | total_timesteps 5224.
Path 609 | total_timesteps 5232.
Path 610 | total_timesteps 5244.
Path 611 | total_timesteps 5251.
Path 612 | total_timesteps 5259.
Path 613 | total_timesteps 5266.
Path 614 | total_timesteps 5273.
Path 615 | total_timesteps 5282.
Path 616 | total_timesteps 5292.
Path 617 | total_timesteps 5300.
Path 618 | total_timesteps 5314.
Path 619 | total_timesteps 5323.
Path 620 | total_timesteps 5331.
Path 621 | total_timesteps 5342.
Path 622 | total_timesteps 5349.
Path 623 | total_timesteps 5360.
Path 624 | total_timesteps 5369.
Path 625 | total_timesteps 5377.
Path 626 | total_timesteps 5384.
Path 627 | total_timesteps 5392.
Path 628 | total_timesteps 5401.
Path 629 | total_timesteps 5410.
Path 630 | total_timesteps 5419.
Path 631 | total_timesteps 5426.
Path 632 | total_timesteps 5435.
Path 633 | total_timesteps 5442.
Path 634 | total_timesteps 5449.
Path 635 | total_timesteps 5457.
Path 636 | total_timesteps 5468.
Path 637 | total_timesteps 5477.
Path 638 | total_timesteps 5484.
Path 639 | total_timesteps 5493.
Path 640 | total_timesteps 5502.
Path 641 | total_timesteps 5512.
Path 642 | total_timesteps 5521.
Path 643 | total_timesteps 5529.
Path 644 | total_timesteps 5537.
Path 645 | total_timesteps 5550.
Path 646 | total_timesteps 5557.
Path 647 | total_timesteps 5565.
Path 648 | total_timesteps 5571.
Path 649 | total_timesteps 5579.
Path 650 | total_timesteps 5591.
Path 651 | total_timesteps 5602.
Path 652 | total_timesteps 5609.
Path 653 | total_timesteps 5616.
Path 654 | total_timesteps 5625.
Path 655 | total_timesteps 5636.
Path 656 | total_timesteps 5647.
Path 657 | total_timesteps 5657.
Path 658 | total_timesteps 5665.
Path 659 | total_timesteps 5677.
Path 660 | total_timesteps 5687.
Path 661 | total_timesteps 5697.
Path 662 | total_timesteps 5706.
Path 663 | total_timesteps 5713.
Path 664 | total_timesteps 5721.
Path 665 | total_timesteps 5728.
Path 666 | total_timesteps 5735.
Path 667 | total_timesteps 5743.
Path 668 | total_timesteps 5753.
Path 669 | total_timesteps 5759.
Path 670 | total_timesteps 5766.
Path 671 | total_timesteps 5775.
Path 672 | total_timesteps 5783.
Path 673 | total_timesteps 5791.
Path 674 | total_timesteps 5799.
Path 675 | total_timesteps 5808.
Path 676 | total_timesteps 5816.
Path 677 | total_timesteps 5824.
Path 678 | total_timesteps 5831.
Path 679 | total_timesteps 5838.
Path 680 | total_timesteps 5846.
Path 681 | total_timesteps 5853.
Path 682 | total_timesteps 5861.
Path 683 | total_timesteps 5868.
Path 684 | total_timesteps 5876.
Path 685 | total_timesteps 5884.
Path 686 | total_timesteps 5896.
Path 687 | total_timesteps 5904.
Path 688 | total_timesteps 5913.
Path 689 | total_timesteps 5923.
Path 690 | total_timesteps 5930.
Path 691 | total_timesteps 5937.
Path 692 | total_timesteps 5945.
Path 693 | total_timesteps 5952.
Path 694 | total_timesteps 5958.
Path 695 | total_timesteps 5967.
Path 696 | total_timesteps 5975.
Path 697 | total_timesteps 5982.
Path 698 | total_timesteps 5988.
Path 699 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.84    |
| Iteration     | 26       |
| MaximumReturn | -0.949   |
| MinimumReturn | -17.5    |
| TotalSamples  | 112100   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.001714216312393546
Validation loss = 0.001709306612610817
Validation loss = 0.0013978646602481604
Validation loss = 0.0014632036909461021
Validation loss = 0.0015194958541542292
Validation loss = 0.0014733636053279042
Validation loss = 0.00203284015879035
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 43.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 58.
Path 8 | total_timesteps 66.
Path 9 | total_timesteps 77.
Path 10 | total_timesteps 89.
Path 11 | total_timesteps 100.
Path 12 | total_timesteps 107.
Path 13 | total_timesteps 120.
Path 14 | total_timesteps 130.
Path 15 | total_timesteps 137.
Path 16 | total_timesteps 144.
Path 17 | total_timesteps 152.
Path 18 | total_timesteps 160.
Path 19 | total_timesteps 169.
Path 20 | total_timesteps 177.
Path 21 | total_timesteps 190.
Path 22 | total_timesteps 198.
Path 23 | total_timesteps 206.
Path 24 | total_timesteps 217.
Path 25 | total_timesteps 226.
Path 26 | total_timesteps 237.
Path 27 | total_timesteps 244.
Path 28 | total_timesteps 252.
Path 29 | total_timesteps 261.
Path 30 | total_timesteps 273.
Path 31 | total_timesteps 280.
Path 32 | total_timesteps 286.
Path 33 | total_timesteps 293.
Path 34 | total_timesteps 303.
Path 35 | total_timesteps 313.
Path 36 | total_timesteps 324.
Path 37 | total_timesteps 336.
Path 38 | total_timesteps 343.
Path 39 | total_timesteps 350.
Path 40 | total_timesteps 359.
Path 41 | total_timesteps 366.
Path 42 | total_timesteps 375.
Path 43 | total_timesteps 384.
Path 44 | total_timesteps 394.
Path 45 | total_timesteps 401.
Path 46 | total_timesteps 412.
Path 47 | total_timesteps 420.
Path 48 | total_timesteps 433.
Path 49 | total_timesteps 442.
Path 50 | total_timesteps 449.
Path 51 | total_timesteps 458.
Path 52 | total_timesteps 465.
Path 53 | total_timesteps 471.
Path 54 | total_timesteps 479.
Path 55 | total_timesteps 486.
Path 56 | total_timesteps 497.
Path 57 | total_timesteps 507.
Path 58 | total_timesteps 513.
Path 59 | total_timesteps 524.
Path 60 | total_timesteps 532.
Path 61 | total_timesteps 540.
Path 62 | total_timesteps 550.
Path 63 | total_timesteps 561.
Path 64 | total_timesteps 571.
Path 65 | total_timesteps 583.
Path 66 | total_timesteps 590.
Path 67 | total_timesteps 603.
Path 68 | total_timesteps 611.
Path 69 | total_timesteps 617.
Path 70 | total_timesteps 626.
Path 71 | total_timesteps 634.
Path 72 | total_timesteps 642.
Path 73 | total_timesteps 648.
Path 74 | total_timesteps 655.
Path 75 | total_timesteps 665.
Path 76 | total_timesteps 672.
Path 77 | total_timesteps 684.
Path 78 | total_timesteps 690.
Path 79 | total_timesteps 698.
Path 80 | total_timesteps 710.
Path 81 | total_timesteps 719.
Path 82 | total_timesteps 729.
Path 83 | total_timesteps 736.
Path 84 | total_timesteps 744.
Path 85 | total_timesteps 750.
Path 86 | total_timesteps 756.
Path 87 | total_timesteps 762.
Path 88 | total_timesteps 770.
Path 89 | total_timesteps 778.
Path 90 | total_timesteps 789.
Path 91 | total_timesteps 796.
Path 92 | total_timesteps 805.
Path 93 | total_timesteps 813.
Path 94 | total_timesteps 822.
Path 95 | total_timesteps 829.
Path 96 | total_timesteps 839.
Path 97 | total_timesteps 847.
Path 98 | total_timesteps 857.
Path 99 | total_timesteps 866.
Path 100 | total_timesteps 875.
Path 101 | total_timesteps 882.
Path 102 | total_timesteps 889.
Path 103 | total_timesteps 899.
Path 104 | total_timesteps 907.
Path 105 | total_timesteps 915.
Path 106 | total_timesteps 923.
Path 107 | total_timesteps 934.
Path 108 | total_timesteps 941.
Path 109 | total_timesteps 950.
Path 110 | total_timesteps 957.
Path 111 | total_timesteps 963.
Path 112 | total_timesteps 971.
Path 113 | total_timesteps 978.
Path 114 | total_timesteps 987.
Path 115 | total_timesteps 995.
Path 116 | total_timesteps 1004.
Path 117 | total_timesteps 1019.
Path 118 | total_timesteps 1026.
Path 119 | total_timesteps 1034.
Path 120 | total_timesteps 1042.
Path 121 | total_timesteps 1052.
Path 122 | total_timesteps 1059.
Path 123 | total_timesteps 1066.
Path 124 | total_timesteps 1073.
Path 125 | total_timesteps 1083.
Path 126 | total_timesteps 1092.
Path 127 | total_timesteps 1100.
Path 128 | total_timesteps 1109.
Path 129 | total_timesteps 1117.
Path 130 | total_timesteps 1125.
Path 131 | total_timesteps 1142.
Path 132 | total_timesteps 1150.
Path 133 | total_timesteps 1157.
Path 134 | total_timesteps 1163.
Path 135 | total_timesteps 1170.
Path 136 | total_timesteps 1177.
Path 137 | total_timesteps 1185.
Path 138 | total_timesteps 1196.
Path 139 | total_timesteps 1204.
Path 140 | total_timesteps 1212.
Path 141 | total_timesteps 1220.
Path 142 | total_timesteps 1229.
Path 143 | total_timesteps 1237.
Path 144 | total_timesteps 1244.
Path 145 | total_timesteps 1255.
Path 146 | total_timesteps 1264.
Path 147 | total_timesteps 1271.
Path 148 | total_timesteps 1280.
Path 149 | total_timesteps 1289.
Path 150 | total_timesteps 1301.
Path 151 | total_timesteps 1309.
Path 152 | total_timesteps 1316.
Path 153 | total_timesteps 1324.
Path 154 | total_timesteps 1335.
Path 155 | total_timesteps 1347.
Path 156 | total_timesteps 1355.
Path 157 | total_timesteps 1362.
Path 158 | total_timesteps 1368.
Path 159 | total_timesteps 1379.
Path 160 | total_timesteps 1390.
Path 161 | total_timesteps 1397.
Path 162 | total_timesteps 1406.
Path 163 | total_timesteps 1414.
Path 164 | total_timesteps 1422.
Path 165 | total_timesteps 1438.
Path 166 | total_timesteps 1452.
Path 167 | total_timesteps 1463.
Path 168 | total_timesteps 1478.
Path 169 | total_timesteps 1488.
Path 170 | total_timesteps 1496.
Path 171 | total_timesteps 1504.
Path 172 | total_timesteps 1514.
Path 173 | total_timesteps 1525.
Path 174 | total_timesteps 1533.
Path 175 | total_timesteps 1542.
Path 176 | total_timesteps 1549.
Path 177 | total_timesteps 1555.
Path 178 | total_timesteps 1563.
Path 179 | total_timesteps 1570.
Path 180 | total_timesteps 1578.
Path 181 | total_timesteps 1587.
Path 182 | total_timesteps 1598.
Path 183 | total_timesteps 1605.
Path 184 | total_timesteps 1614.
Path 185 | total_timesteps 1623.
Path 186 | total_timesteps 1631.
Path 187 | total_timesteps 1638.
Path 188 | total_timesteps 1647.
Path 189 | total_timesteps 1653.
Path 190 | total_timesteps 1663.
Path 191 | total_timesteps 1672.
Path 192 | total_timesteps 1679.
Path 193 | total_timesteps 1687.
Path 194 | total_timesteps 1700.
Path 195 | total_timesteps 1708.
Path 196 | total_timesteps 1717.
Path 197 | total_timesteps 1729.
Path 198 | total_timesteps 1737.
Path 199 | total_timesteps 1747.
Path 200 | total_timesteps 1756.
Path 201 | total_timesteps 1763.
Path 202 | total_timesteps 1770.
Path 203 | total_timesteps 1776.
Path 204 | total_timesteps 1785.
Path 205 | total_timesteps 1794.
Path 206 | total_timesteps 1804.
Path 207 | total_timesteps 1816.
Path 208 | total_timesteps 1827.
Path 209 | total_timesteps 1834.
Path 210 | total_timesteps 1842.
Path 211 | total_timesteps 1851.
Path 212 | total_timesteps 1862.
Path 213 | total_timesteps 1870.
Path 214 | total_timesteps 1880.
Path 215 | total_timesteps 1887.
Path 216 | total_timesteps 1900.
Path 217 | total_timesteps 1908.
Path 218 | total_timesteps 1915.
Path 219 | total_timesteps 1923.
Path 220 | total_timesteps 1930.
Path 221 | total_timesteps 1937.
Path 222 | total_timesteps 1945.
Path 223 | total_timesteps 1955.
Path 224 | total_timesteps 1963.
Path 225 | total_timesteps 1976.
Path 226 | total_timesteps 1984.
Path 227 | total_timesteps 1996.
Path 228 | total_timesteps 2004.
Path 229 | total_timesteps 2012.
Path 230 | total_timesteps 2019.
Path 231 | total_timesteps 2028.
Path 232 | total_timesteps 2034.
Path 233 | total_timesteps 2043.
Path 234 | total_timesteps 2050.
Path 235 | total_timesteps 2058.
Path 236 | total_timesteps 2067.
Path 237 | total_timesteps 2075.
Path 238 | total_timesteps 2084.
Path 239 | total_timesteps 2091.
Path 240 | total_timesteps 2102.
Path 241 | total_timesteps 2110.
Path 242 | total_timesteps 2118.
Path 243 | total_timesteps 2127.
Path 244 | total_timesteps 2139.
Path 245 | total_timesteps 2147.
Path 246 | total_timesteps 2153.
Path 247 | total_timesteps 2162.
Path 248 | total_timesteps 2172.
Path 249 | total_timesteps 2182.
Path 250 | total_timesteps 2191.
Path 251 | total_timesteps 2201.
Path 252 | total_timesteps 2211.
Path 253 | total_timesteps 2219.
Path 254 | total_timesteps 2226.
Path 255 | total_timesteps 2236.
Path 256 | total_timesteps 2242.
Path 257 | total_timesteps 2256.
Path 258 | total_timesteps 2266.
Path 259 | total_timesteps 2273.
Path 260 | total_timesteps 2281.
Path 261 | total_timesteps 2292.
Path 262 | total_timesteps 2301.
Path 263 | total_timesteps 2310.
Path 264 | total_timesteps 2318.
Path 265 | total_timesteps 2325.
Path 266 | total_timesteps 2333.
Path 267 | total_timesteps 2343.
Path 268 | total_timesteps 2353.
Path 269 | total_timesteps 2359.
Path 270 | total_timesteps 2367.
Path 271 | total_timesteps 2376.
Path 272 | total_timesteps 2385.
Path 273 | total_timesteps 2393.
Path 274 | total_timesteps 2400.
Path 275 | total_timesteps 2407.
Path 276 | total_timesteps 2416.
Path 277 | total_timesteps 2424.
Path 278 | total_timesteps 2432.
Path 279 | total_timesteps 2441.
Path 280 | total_timesteps 2448.
Path 281 | total_timesteps 2456.
Path 282 | total_timesteps 2468.
Path 283 | total_timesteps 2475.
Path 284 | total_timesteps 2483.
Path 285 | total_timesteps 2491.
Path 286 | total_timesteps 2501.
Path 287 | total_timesteps 2509.
Path 288 | total_timesteps 2516.
Path 289 | total_timesteps 2523.
Path 290 | total_timesteps 2531.
Path 291 | total_timesteps 2537.
Path 292 | total_timesteps 2547.
Path 293 | total_timesteps 2554.
Path 294 | total_timesteps 2563.
Path 295 | total_timesteps 2572.
Path 296 | total_timesteps 2583.
Path 297 | total_timesteps 2592.
Path 298 | total_timesteps 2603.
Path 299 | total_timesteps 2611.
Path 300 | total_timesteps 2621.
Path 301 | total_timesteps 2632.
Path 302 | total_timesteps 2639.
Path 303 | total_timesteps 2648.
Path 304 | total_timesteps 2658.
Path 305 | total_timesteps 2666.
Path 306 | total_timesteps 2677.
Path 307 | total_timesteps 2685.
Path 308 | total_timesteps 2695.
Path 309 | total_timesteps 2706.
Path 310 | total_timesteps 2715.
Path 311 | total_timesteps 2725.
Path 312 | total_timesteps 2734.
Path 313 | total_timesteps 2747.
Path 314 | total_timesteps 2754.
Path 315 | total_timesteps 2763.
Path 316 | total_timesteps 2770.
Path 317 | total_timesteps 2781.
Path 318 | total_timesteps 2788.
Path 319 | total_timesteps 2795.
Path 320 | total_timesteps 2802.
Path 321 | total_timesteps 2810.
Path 322 | total_timesteps 2818.
Path 323 | total_timesteps 2826.
Path 324 | total_timesteps 2833.
Path 325 | total_timesteps 2843.
Path 326 | total_timesteps 2850.
Path 327 | total_timesteps 2857.
Path 328 | total_timesteps 2865.
Path 329 | total_timesteps 2875.
Path 330 | total_timesteps 2886.
Path 331 | total_timesteps 2893.
Path 332 | total_timesteps 2900.
Path 333 | total_timesteps 2908.
Path 334 | total_timesteps 2916.
Path 335 | total_timesteps 2923.
Path 336 | total_timesteps 2931.
Path 337 | total_timesteps 2949.
Path 338 | total_timesteps 2958.
Path 339 | total_timesteps 2970.
Path 340 | total_timesteps 2978.
Path 341 | total_timesteps 2989.
Path 342 | total_timesteps 2997.
Path 343 | total_timesteps 3008.
Path 344 | total_timesteps 3016.
Path 345 | total_timesteps 3024.
Path 346 | total_timesteps 3034.
Path 347 | total_timesteps 3043.
Path 348 | total_timesteps 3049.
Path 349 | total_timesteps 3059.
Path 350 | total_timesteps 3066.
Path 351 | total_timesteps 3074.
Path 352 | total_timesteps 3089.
Path 353 | total_timesteps 3096.
Path 354 | total_timesteps 3104.
Path 355 | total_timesteps 3110.
Path 356 | total_timesteps 3120.
Path 357 | total_timesteps 3127.
Path 358 | total_timesteps 3134.
Path 359 | total_timesteps 3142.
Path 360 | total_timesteps 3151.
Path 361 | total_timesteps 3159.
Path 362 | total_timesteps 3169.
Path 363 | total_timesteps 3176.
Path 364 | total_timesteps 3189.
Path 365 | total_timesteps 3198.
Path 366 | total_timesteps 3207.
Path 367 | total_timesteps 3216.
Path 368 | total_timesteps 3222.
Path 369 | total_timesteps 3231.
Path 370 | total_timesteps 3237.
Path 371 | total_timesteps 3252.
Path 372 | total_timesteps 3259.
Path 373 | total_timesteps 3268.
Path 374 | total_timesteps 3281.
Path 375 | total_timesteps 3290.
Path 376 | total_timesteps 3297.
Path 377 | total_timesteps 3307.
Path 378 | total_timesteps 3313.
Path 379 | total_timesteps 3327.
Path 380 | total_timesteps 3335.
Path 381 | total_timesteps 3344.
Path 382 | total_timesteps 3353.
Path 383 | total_timesteps 3367.
Path 384 | total_timesteps 3374.
Path 385 | total_timesteps 3387.
Path 386 | total_timesteps 3396.
Path 387 | total_timesteps 3405.
Path 388 | total_timesteps 3415.
Path 389 | total_timesteps 3422.
Path 390 | total_timesteps 3429.
Path 391 | total_timesteps 3435.
Path 392 | total_timesteps 3442.
Path 393 | total_timesteps 3450.
Path 394 | total_timesteps 3460.
Path 395 | total_timesteps 3470.
Path 396 | total_timesteps 3483.
Path 397 | total_timesteps 3492.
Path 398 | total_timesteps 3499.
Path 399 | total_timesteps 3506.
Path 400 | total_timesteps 3514.
Path 401 | total_timesteps 3524.
Path 402 | total_timesteps 3532.
Path 403 | total_timesteps 3542.
Path 404 | total_timesteps 3551.
Path 405 | total_timesteps 3558.
Path 406 | total_timesteps 3564.
Path 407 | total_timesteps 3576.
Path 408 | total_timesteps 3584.
Path 409 | total_timesteps 3592.
Path 410 | total_timesteps 3600.
Path 411 | total_timesteps 3610.
Path 412 | total_timesteps 3618.
Path 413 | total_timesteps 3630.
Path 414 | total_timesteps 3637.
Path 415 | total_timesteps 3645.
Path 416 | total_timesteps 3653.
Path 417 | total_timesteps 3660.
Path 418 | total_timesteps 3666.
Path 419 | total_timesteps 3673.
Path 420 | total_timesteps 3685.
Path 421 | total_timesteps 3692.
Path 422 | total_timesteps 3703.
Path 423 | total_timesteps 3712.
Path 424 | total_timesteps 3724.
Path 425 | total_timesteps 3733.
Path 426 | total_timesteps 3741.
Path 427 | total_timesteps 3749.
Path 428 | total_timesteps 3760.
Path 429 | total_timesteps 3767.
Path 430 | total_timesteps 3774.
Path 431 | total_timesteps 3783.
Path 432 | total_timesteps 3790.
Path 433 | total_timesteps 3802.
Path 434 | total_timesteps 3812.
Path 435 | total_timesteps 3819.
Path 436 | total_timesteps 3826.
Path 437 | total_timesteps 3837.
Path 438 | total_timesteps 3844.
Path 439 | total_timesteps 3858.
Path 440 | total_timesteps 3867.
Path 441 | total_timesteps 3873.
Path 442 | total_timesteps 3883.
Path 443 | total_timesteps 3893.
Path 444 | total_timesteps 3901.
Path 445 | total_timesteps 3909.
Path 446 | total_timesteps 3916.
Path 447 | total_timesteps 3922.
Path 448 | total_timesteps 3928.
Path 449 | total_timesteps 3939.
Path 450 | total_timesteps 3948.
Path 451 | total_timesteps 3958.
Path 452 | total_timesteps 3967.
Path 453 | total_timesteps 3977.
Path 454 | total_timesteps 3984.
Path 455 | total_timesteps 3993.
Path 456 | total_timesteps 4001.
Path 457 | total_timesteps 4009.
Path 458 | total_timesteps 4019.
Path 459 | total_timesteps 4032.
Path 460 | total_timesteps 4039.
Path 461 | total_timesteps 4050.
Path 462 | total_timesteps 4058.
Path 463 | total_timesteps 4065.
Path 464 | total_timesteps 4075.
Path 465 | total_timesteps 4083.
Path 466 | total_timesteps 4092.
Path 467 | total_timesteps 4103.
Path 468 | total_timesteps 4111.
Path 469 | total_timesteps 4121.
Path 470 | total_timesteps 4129.
Path 471 | total_timesteps 4135.
Path 472 | total_timesteps 4146.
Path 473 | total_timesteps 4155.
Path 474 | total_timesteps 4164.
Path 475 | total_timesteps 4171.
Path 476 | total_timesteps 4182.
Path 477 | total_timesteps 4190.
Path 478 | total_timesteps 4196.
Path 479 | total_timesteps 4205.
Path 480 | total_timesteps 4212.
Path 481 | total_timesteps 4220.
Path 482 | total_timesteps 4228.
Path 483 | total_timesteps 4238.
Path 484 | total_timesteps 4246.
Path 485 | total_timesteps 4254.
Path 486 | total_timesteps 4261.
Path 487 | total_timesteps 4271.
Path 488 | total_timesteps 4278.
Path 489 | total_timesteps 4288.
Path 490 | total_timesteps 4296.
Path 491 | total_timesteps 4306.
Path 492 | total_timesteps 4315.
Path 493 | total_timesteps 4327.
Path 494 | total_timesteps 4334.
Path 495 | total_timesteps 4340.
Path 496 | total_timesteps 4346.
Path 497 | total_timesteps 4354.
Path 498 | total_timesteps 4364.
Path 499 | total_timesteps 4371.
Path 500 | total_timesteps 4379.
Path 501 | total_timesteps 4386.
Path 502 | total_timesteps 4396.
Path 503 | total_timesteps 4402.
Path 504 | total_timesteps 4411.
Path 505 | total_timesteps 4420.
Path 506 | total_timesteps 4428.
Path 507 | total_timesteps 4435.
Path 508 | total_timesteps 4445.
Path 509 | total_timesteps 4452.
Path 510 | total_timesteps 4459.
Path 511 | total_timesteps 4466.
Path 512 | total_timesteps 4475.
Path 513 | total_timesteps 4487.
Path 514 | total_timesteps 4498.
Path 515 | total_timesteps 4506.
Path 516 | total_timesteps 4514.
Path 517 | total_timesteps 4520.
Path 518 | total_timesteps 4529.
Path 519 | total_timesteps 4541.
Path 520 | total_timesteps 4548.
Path 521 | total_timesteps 4556.
Path 522 | total_timesteps 4567.
Path 523 | total_timesteps 4576.
Path 524 | total_timesteps 4587.
Path 525 | total_timesteps 4597.
Path 526 | total_timesteps 4604.
Path 527 | total_timesteps 4614.
Path 528 | total_timesteps 4625.
Path 529 | total_timesteps 4641.
Path 530 | total_timesteps 4653.
Path 531 | total_timesteps 4660.
Path 532 | total_timesteps 4670.
Path 533 | total_timesteps 4680.
Path 534 | total_timesteps 4689.
Path 535 | total_timesteps 4696.
Path 536 | total_timesteps 4703.
Path 537 | total_timesteps 4711.
Path 538 | total_timesteps 4720.
Path 539 | total_timesteps 4727.
Path 540 | total_timesteps 4735.
Path 541 | total_timesteps 4743.
Path 542 | total_timesteps 4754.
Path 543 | total_timesteps 4762.
Path 544 | total_timesteps 4769.
Path 545 | total_timesteps 4776.
Path 546 | total_timesteps 4785.
Path 547 | total_timesteps 4793.
Path 548 | total_timesteps 4803.
Path 549 | total_timesteps 4809.
Path 550 | total_timesteps 4817.
Path 551 | total_timesteps 4826.
Path 552 | total_timesteps 4835.
Path 553 | total_timesteps 4842.
Path 554 | total_timesteps 4853.
Path 555 | total_timesteps 4859.
Path 556 | total_timesteps 4868.
Path 557 | total_timesteps 4876.
Path 558 | total_timesteps 4883.
Path 559 | total_timesteps 4893.
Path 560 | total_timesteps 4910.
Path 561 | total_timesteps 4919.
Path 562 | total_timesteps 4927.
Path 563 | total_timesteps 4938.
Path 564 | total_timesteps 4944.
Path 565 | total_timesteps 4952.
Path 566 | total_timesteps 4964.
Path 567 | total_timesteps 4971.
Path 568 | total_timesteps 4979.
Path 569 | total_timesteps 4985.
Path 570 | total_timesteps 4993.
Path 571 | total_timesteps 5002.
Path 572 | total_timesteps 5010.
Path 573 | total_timesteps 5019.
Path 574 | total_timesteps 5030.
Path 575 | total_timesteps 5037.
Path 576 | total_timesteps 5046.
Path 577 | total_timesteps 5058.
Path 578 | total_timesteps 5067.
Path 579 | total_timesteps 5074.
Path 580 | total_timesteps 5082.
Path 581 | total_timesteps 5094.
Path 582 | total_timesteps 5101.
Path 583 | total_timesteps 5109.
Path 584 | total_timesteps 5117.
Path 585 | total_timesteps 5123.
Path 586 | total_timesteps 5129.
Path 587 | total_timesteps 5136.
Path 588 | total_timesteps 5144.
Path 589 | total_timesteps 5151.
Path 590 | total_timesteps 5159.
Path 591 | total_timesteps 5168.
Path 592 | total_timesteps 5178.
Path 593 | total_timesteps 5188.
Path 594 | total_timesteps 5195.
Path 595 | total_timesteps 5203.
Path 596 | total_timesteps 5212.
Path 597 | total_timesteps 5218.
Path 598 | total_timesteps 5227.
Path 599 | total_timesteps 5233.
Path 600 | total_timesteps 5240.
Path 601 | total_timesteps 5249.
Path 602 | total_timesteps 5256.
Path 603 | total_timesteps 5263.
Path 604 | total_timesteps 5272.
Path 605 | total_timesteps 5278.
Path 606 | total_timesteps 5284.
Path 607 | total_timesteps 5294.
Path 608 | total_timesteps 5304.
Path 609 | total_timesteps 5311.
Path 610 | total_timesteps 5321.
Path 611 | total_timesteps 5329.
Path 612 | total_timesteps 5336.
Path 613 | total_timesteps 5348.
Path 614 | total_timesteps 5355.
Path 615 | total_timesteps 5362.
Path 616 | total_timesteps 5375.
Path 617 | total_timesteps 5382.
Path 618 | total_timesteps 5390.
Path 619 | total_timesteps 5398.
Path 620 | total_timesteps 5406.
Path 621 | total_timesteps 5412.
Path 622 | total_timesteps 5420.
Path 623 | total_timesteps 5434.
Path 624 | total_timesteps 5443.
Path 625 | total_timesteps 5451.
Path 626 | total_timesteps 5458.
Path 627 | total_timesteps 5464.
Path 628 | total_timesteps 5477.
Path 629 | total_timesteps 5490.
Path 630 | total_timesteps 5498.
Path 631 | total_timesteps 5507.
Path 632 | total_timesteps 5518.
Path 633 | total_timesteps 5530.
Path 634 | total_timesteps 5538.
Path 635 | total_timesteps 5545.
Path 636 | total_timesteps 5552.
Path 637 | total_timesteps 5560.
Path 638 | total_timesteps 5568.
Path 639 | total_timesteps 5576.
Path 640 | total_timesteps 5589.
Path 641 | total_timesteps 5598.
Path 642 | total_timesteps 5607.
Path 643 | total_timesteps 5617.
Path 644 | total_timesteps 5626.
Path 645 | total_timesteps 5634.
Path 646 | total_timesteps 5641.
Path 647 | total_timesteps 5648.
Path 648 | total_timesteps 5655.
Path 649 | total_timesteps 5665.
Path 650 | total_timesteps 5672.
Path 651 | total_timesteps 5684.
Path 652 | total_timesteps 5695.
Path 653 | total_timesteps 5701.
Path 654 | total_timesteps 5711.
Path 655 | total_timesteps 5719.
Path 656 | total_timesteps 5726.
Path 657 | total_timesteps 5732.
Path 658 | total_timesteps 5739.
Path 659 | total_timesteps 5748.
Path 660 | total_timesteps 5756.
Path 661 | total_timesteps 5763.
Path 662 | total_timesteps 5776.
Path 663 | total_timesteps 5787.
Path 664 | total_timesteps 5795.
Path 665 | total_timesteps 5805.
Path 666 | total_timesteps 5814.
Path 667 | total_timesteps 5822.
Path 668 | total_timesteps 5834.
Path 669 | total_timesteps 5842.
Path 670 | total_timesteps 5852.
Path 671 | total_timesteps 5861.
Path 672 | total_timesteps 5868.
Path 673 | total_timesteps 5875.
Path 674 | total_timesteps 5881.
Path 675 | total_timesteps 5887.
Path 676 | total_timesteps 5897.
Path 677 | total_timesteps 5905.
Path 678 | total_timesteps 5912.
Path 679 | total_timesteps 5921.
Path 680 | total_timesteps 5932.
Path 681 | total_timesteps 5939.
Path 682 | total_timesteps 5946.
Path 683 | total_timesteps 5955.
Path 684 | total_timesteps 5961.
Path 685 | total_timesteps 5970.
Path 686 | total_timesteps 5979.
Path 687 | total_timesteps 5986.
Path 688 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.7     |
| Iteration     | 27       |
| MaximumReturn | -0.701   |
| MinimumReturn | -17.1    |
| TotalSamples  | 116101   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.0016641662223264575
Validation loss = 0.0013846613001078367
Validation loss = 0.0015067880740389228
Validation loss = 0.0015963660553097725
Validation loss = 0.0015400731936097145
Validation loss = 0.001492067938670516
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 24.
Path 4 | total_timesteps 32.
Path 5 | total_timesteps 40.
Path 6 | total_timesteps 47.
Path 7 | total_timesteps 54.
Path 8 | total_timesteps 61.
Path 9 | total_timesteps 69.
Path 10 | total_timesteps 76.
Path 11 | total_timesteps 90.
Path 12 | total_timesteps 100.
Path 13 | total_timesteps 108.
Path 14 | total_timesteps 115.
Path 15 | total_timesteps 124.
Path 16 | total_timesteps 133.
Path 17 | total_timesteps 140.
Path 18 | total_timesteps 147.
Path 19 | total_timesteps 154.
Path 20 | total_timesteps 162.
Path 21 | total_timesteps 170.
Path 22 | total_timesteps 176.
Path 23 | total_timesteps 184.
Path 24 | total_timesteps 192.
Path 25 | total_timesteps 204.
Path 26 | total_timesteps 213.
Path 27 | total_timesteps 224.
Path 28 | total_timesteps 235.
Path 29 | total_timesteps 242.
Path 30 | total_timesteps 249.
Path 31 | total_timesteps 258.
Path 32 | total_timesteps 265.
Path 33 | total_timesteps 274.
Path 34 | total_timesteps 283.
Path 35 | total_timesteps 292.
Path 36 | total_timesteps 300.
Path 37 | total_timesteps 307.
Path 38 | total_timesteps 314.
Path 39 | total_timesteps 322.
Path 40 | total_timesteps 331.
Path 41 | total_timesteps 341.
Path 42 | total_timesteps 347.
Path 43 | total_timesteps 358.
Path 44 | total_timesteps 365.
Path 45 | total_timesteps 373.
Path 46 | total_timesteps 382.
Path 47 | total_timesteps 392.
Path 48 | total_timesteps 400.
Path 49 | total_timesteps 416.
Path 50 | total_timesteps 423.
Path 51 | total_timesteps 432.
Path 52 | total_timesteps 444.
Path 53 | total_timesteps 455.
Path 54 | total_timesteps 462.
Path 55 | total_timesteps 474.
Path 56 | total_timesteps 482.
Path 57 | total_timesteps 491.
Path 58 | total_timesteps 500.
Path 59 | total_timesteps 514.
Path 60 | total_timesteps 526.
Path 61 | total_timesteps 535.
Path 62 | total_timesteps 541.
Path 63 | total_timesteps 547.
Path 64 | total_timesteps 558.
Path 65 | total_timesteps 568.
Path 66 | total_timesteps 576.
Path 67 | total_timesteps 585.
Path 68 | total_timesteps 591.
Path 69 | total_timesteps 598.
Path 70 | total_timesteps 611.
Path 71 | total_timesteps 618.
Path 72 | total_timesteps 626.
Path 73 | total_timesteps 638.
Path 74 | total_timesteps 645.
Path 75 | total_timesteps 652.
Path 76 | total_timesteps 658.
Path 77 | total_timesteps 667.
Path 78 | total_timesteps 674.
Path 79 | total_timesteps 680.
Path 80 | total_timesteps 691.
Path 81 | total_timesteps 698.
Path 82 | total_timesteps 705.
Path 83 | total_timesteps 713.
Path 84 | total_timesteps 720.
Path 85 | total_timesteps 728.
Path 86 | total_timesteps 737.
Path 87 | total_timesteps 744.
Path 88 | total_timesteps 752.
Path 89 | total_timesteps 759.
Path 90 | total_timesteps 771.
Path 91 | total_timesteps 779.
Path 92 | total_timesteps 787.
Path 93 | total_timesteps 795.
Path 94 | total_timesteps 804.
Path 95 | total_timesteps 812.
Path 96 | total_timesteps 818.
Path 97 | total_timesteps 830.
Path 98 | total_timesteps 838.
Path 99 | total_timesteps 846.
Path 100 | total_timesteps 854.
Path 101 | total_timesteps 862.
Path 102 | total_timesteps 870.
Path 103 | total_timesteps 877.
Path 104 | total_timesteps 886.
Path 105 | total_timesteps 895.
Path 106 | total_timesteps 904.
Path 107 | total_timesteps 912.
Path 108 | total_timesteps 919.
Path 109 | total_timesteps 926.
Path 110 | total_timesteps 936.
Path 111 | total_timesteps 943.
Path 112 | total_timesteps 951.
Path 113 | total_timesteps 960.
Path 114 | total_timesteps 968.
Path 115 | total_timesteps 976.
Path 116 | total_timesteps 987.
Path 117 | total_timesteps 994.
Path 118 | total_timesteps 1000.
Path 119 | total_timesteps 1009.
Path 120 | total_timesteps 1021.
Path 121 | total_timesteps 1033.
Path 122 | total_timesteps 1041.
Path 123 | total_timesteps 1052.
Path 124 | total_timesteps 1061.
Path 125 | total_timesteps 1071.
Path 126 | total_timesteps 1082.
Path 127 | total_timesteps 1090.
Path 128 | total_timesteps 1098.
Path 129 | total_timesteps 1108.
Path 130 | total_timesteps 1117.
Path 131 | total_timesteps 1127.
Path 132 | total_timesteps 1138.
Path 133 | total_timesteps 1146.
Path 134 | total_timesteps 1157.
Path 135 | total_timesteps 1164.
Path 136 | total_timesteps 1172.
Path 137 | total_timesteps 1179.
Path 138 | total_timesteps 1186.
Path 139 | total_timesteps 1195.
Path 140 | total_timesteps 1203.
Path 141 | total_timesteps 1212.
Path 142 | total_timesteps 1219.
Path 143 | total_timesteps 1227.
Path 144 | total_timesteps 1236.
Path 145 | total_timesteps 1243.
Path 146 | total_timesteps 1252.
Path 147 | total_timesteps 1259.
Path 148 | total_timesteps 1266.
Path 149 | total_timesteps 1272.
Path 150 | total_timesteps 1279.
Path 151 | total_timesteps 1288.
Path 152 | total_timesteps 1297.
Path 153 | total_timesteps 1304.
Path 154 | total_timesteps 1312.
Path 155 | total_timesteps 1321.
Path 156 | total_timesteps 1329.
Path 157 | total_timesteps 1337.
Path 158 | total_timesteps 1345.
Path 159 | total_timesteps 1356.
Path 160 | total_timesteps 1363.
Path 161 | total_timesteps 1370.
Path 162 | total_timesteps 1377.
Path 163 | total_timesteps 1386.
Path 164 | total_timesteps 1394.
Path 165 | total_timesteps 1401.
Path 166 | total_timesteps 1411.
Path 167 | total_timesteps 1418.
Path 168 | total_timesteps 1425.
Path 169 | total_timesteps 1434.
Path 170 | total_timesteps 1442.
Path 171 | total_timesteps 1452.
Path 172 | total_timesteps 1460.
Path 173 | total_timesteps 1467.
Path 174 | total_timesteps 1476.
Path 175 | total_timesteps 1484.
Path 176 | total_timesteps 1491.
Path 177 | total_timesteps 1498.
Path 178 | total_timesteps 1506.
Path 179 | total_timesteps 1514.
Path 180 | total_timesteps 1521.
Path 181 | total_timesteps 1528.
Path 182 | total_timesteps 1535.
Path 183 | total_timesteps 1546.
Path 184 | total_timesteps 1554.
Path 185 | total_timesteps 1564.
Path 186 | total_timesteps 1570.
Path 187 | total_timesteps 1577.
Path 188 | total_timesteps 1586.
Path 189 | total_timesteps 1594.
Path 190 | total_timesteps 1603.
Path 191 | total_timesteps 1612.
Path 192 | total_timesteps 1619.
Path 193 | total_timesteps 1628.
Path 194 | total_timesteps 1639.
Path 195 | total_timesteps 1651.
Path 196 | total_timesteps 1658.
Path 197 | total_timesteps 1666.
Path 198 | total_timesteps 1675.
Path 199 | total_timesteps 1682.
Path 200 | total_timesteps 1693.
Path 201 | total_timesteps 1700.
Path 202 | total_timesteps 1714.
Path 203 | total_timesteps 1721.
Path 204 | total_timesteps 1729.
Path 205 | total_timesteps 1737.
Path 206 | total_timesteps 1748.
Path 207 | total_timesteps 1756.
Path 208 | total_timesteps 1766.
Path 209 | total_timesteps 1773.
Path 210 | total_timesteps 1783.
Path 211 | total_timesteps 1790.
Path 212 | total_timesteps 1797.
Path 213 | total_timesteps 1805.
Path 214 | total_timesteps 1811.
Path 215 | total_timesteps 1819.
Path 216 | total_timesteps 1826.
Path 217 | total_timesteps 1833.
Path 218 | total_timesteps 1842.
Path 219 | total_timesteps 1852.
Path 220 | total_timesteps 1859.
Path 221 | total_timesteps 1870.
Path 222 | total_timesteps 1878.
Path 223 | total_timesteps 1888.
Path 224 | total_timesteps 1896.
Path 225 | total_timesteps 1906.
Path 226 | total_timesteps 1920.
Path 227 | total_timesteps 1928.
Path 228 | total_timesteps 1940.
Path 229 | total_timesteps 1949.
Path 230 | total_timesteps 1961.
Path 231 | total_timesteps 1968.
Path 232 | total_timesteps 1976.
Path 233 | total_timesteps 1983.
Path 234 | total_timesteps 1991.
Path 235 | total_timesteps 1999.
Path 236 | total_timesteps 2008.
Path 237 | total_timesteps 2014.
Path 238 | total_timesteps 2024.
Path 239 | total_timesteps 2031.
Path 240 | total_timesteps 2038.
Path 241 | total_timesteps 2048.
Path 242 | total_timesteps 2061.
Path 243 | total_timesteps 2068.
Path 244 | total_timesteps 2076.
Path 245 | total_timesteps 2086.
Path 246 | total_timesteps 2096.
Path 247 | total_timesteps 2103.
Path 248 | total_timesteps 2110.
Path 249 | total_timesteps 2120.
Path 250 | total_timesteps 2127.
Path 251 | total_timesteps 2138.
Path 252 | total_timesteps 2144.
Path 253 | total_timesteps 2154.
Path 254 | total_timesteps 2162.
Path 255 | total_timesteps 2168.
Path 256 | total_timesteps 2175.
Path 257 | total_timesteps 2183.
Path 258 | total_timesteps 2190.
Path 259 | total_timesteps 2199.
Path 260 | total_timesteps 2209.
Path 261 | total_timesteps 2217.
Path 262 | total_timesteps 2224.
Path 263 | total_timesteps 2231.
Path 264 | total_timesteps 2240.
Path 265 | total_timesteps 2248.
Path 266 | total_timesteps 2258.
Path 267 | total_timesteps 2267.
Path 268 | total_timesteps 2276.
Path 269 | total_timesteps 2283.
Path 270 | total_timesteps 2295.
Path 271 | total_timesteps 2305.
Path 272 | total_timesteps 2313.
Path 273 | total_timesteps 2320.
Path 274 | total_timesteps 2328.
Path 275 | total_timesteps 2336.
Path 276 | total_timesteps 2344.
Path 277 | total_timesteps 2351.
Path 278 | total_timesteps 2358.
Path 279 | total_timesteps 2365.
Path 280 | total_timesteps 2374.
Path 281 | total_timesteps 2381.
Path 282 | total_timesteps 2401.
Path 283 | total_timesteps 2408.
Path 284 | total_timesteps 2418.
Path 285 | total_timesteps 2426.
Path 286 | total_timesteps 2433.
Path 287 | total_timesteps 2444.
Path 288 | total_timesteps 2452.
Path 289 | total_timesteps 2462.
Path 290 | total_timesteps 2470.
Path 291 | total_timesteps 2477.
Path 292 | total_timesteps 2485.
Path 293 | total_timesteps 2493.
Path 294 | total_timesteps 2502.
Path 295 | total_timesteps 2516.
Path 296 | total_timesteps 2526.
Path 297 | total_timesteps 2534.
Path 298 | total_timesteps 2546.
Path 299 | total_timesteps 2554.
Path 300 | total_timesteps 2566.
Path 301 | total_timesteps 2575.
Path 302 | total_timesteps 2588.
Path 303 | total_timesteps 2596.
Path 304 | total_timesteps 2605.
Path 305 | total_timesteps 2611.
Path 306 | total_timesteps 2617.
Path 307 | total_timesteps 2627.
Path 308 | total_timesteps 2635.
Path 309 | total_timesteps 2646.
Path 310 | total_timesteps 2661.
Path 311 | total_timesteps 2673.
Path 312 | total_timesteps 2680.
Path 313 | total_timesteps 2689.
Path 314 | total_timesteps 2697.
Path 315 | total_timesteps 2704.
Path 316 | total_timesteps 2714.
Path 317 | total_timesteps 2722.
Path 318 | total_timesteps 2729.
Path 319 | total_timesteps 2736.
Path 320 | total_timesteps 2742.
Path 321 | total_timesteps 2756.
Path 322 | total_timesteps 2767.
Path 323 | total_timesteps 2776.
Path 324 | total_timesteps 2784.
Path 325 | total_timesteps 2790.
Path 326 | total_timesteps 2799.
Path 327 | total_timesteps 2806.
Path 328 | total_timesteps 2817.
Path 329 | total_timesteps 2828.
Path 330 | total_timesteps 2837.
Path 331 | total_timesteps 2846.
Path 332 | total_timesteps 2853.
Path 333 | total_timesteps 2864.
Path 334 | total_timesteps 2871.
Path 335 | total_timesteps 2881.
Path 336 | total_timesteps 2891.
Path 337 | total_timesteps 2900.
Path 338 | total_timesteps 2909.
Path 339 | total_timesteps 2915.
Path 340 | total_timesteps 2922.
Path 341 | total_timesteps 2931.
Path 342 | total_timesteps 2945.
Path 343 | total_timesteps 2952.
Path 344 | total_timesteps 2963.
Path 345 | total_timesteps 2969.
Path 346 | total_timesteps 2976.
Path 347 | total_timesteps 2982.
Path 348 | total_timesteps 2989.
Path 349 | total_timesteps 2998.
Path 350 | total_timesteps 3009.
Path 351 | total_timesteps 3017.
Path 352 | total_timesteps 3026.
Path 353 | total_timesteps 3033.
Path 354 | total_timesteps 3039.
Path 355 | total_timesteps 3047.
Path 356 | total_timesteps 3055.
Path 357 | total_timesteps 3066.
Path 358 | total_timesteps 3076.
Path 359 | total_timesteps 3087.
Path 360 | total_timesteps 3093.
Path 361 | total_timesteps 3101.
Path 362 | total_timesteps 3109.
Path 363 | total_timesteps 3116.
Path 364 | total_timesteps 3124.
Path 365 | total_timesteps 3133.
Path 366 | total_timesteps 3146.
Path 367 | total_timesteps 3154.
Path 368 | total_timesteps 3160.
Path 369 | total_timesteps 3171.
Path 370 | total_timesteps 3180.
Path 371 | total_timesteps 3190.
Path 372 | total_timesteps 3200.
Path 373 | total_timesteps 3206.
Path 374 | total_timesteps 3215.
Path 375 | total_timesteps 3222.
Path 376 | total_timesteps 3230.
Path 377 | total_timesteps 3238.
Path 378 | total_timesteps 3245.
Path 379 | total_timesteps 3253.
Path 380 | total_timesteps 3261.
Path 381 | total_timesteps 3268.
Path 382 | total_timesteps 3275.
Path 383 | total_timesteps 3283.
Path 384 | total_timesteps 3291.
Path 385 | total_timesteps 3298.
Path 386 | total_timesteps 3305.
Path 387 | total_timesteps 3313.
Path 388 | total_timesteps 3321.
Path 389 | total_timesteps 3333.
Path 390 | total_timesteps 3340.
Path 391 | total_timesteps 3349.
Path 392 | total_timesteps 3360.
Path 393 | total_timesteps 3369.
Path 394 | total_timesteps 3381.
Path 395 | total_timesteps 3389.
Path 396 | total_timesteps 3398.
Path 397 | total_timesteps 3406.
Path 398 | total_timesteps 3414.
Path 399 | total_timesteps 3422.
Path 400 | total_timesteps 3429.
Path 401 | total_timesteps 3437.
Path 402 | total_timesteps 3446.
Path 403 | total_timesteps 3452.
Path 404 | total_timesteps 3459.
Path 405 | total_timesteps 3466.
Path 406 | total_timesteps 3474.
Path 407 | total_timesteps 3480.
Path 408 | total_timesteps 3491.
Path 409 | total_timesteps 3498.
Path 410 | total_timesteps 3508.
Path 411 | total_timesteps 3516.
Path 412 | total_timesteps 3524.
Path 413 | total_timesteps 3531.
Path 414 | total_timesteps 3539.
Path 415 | total_timesteps 3545.
Path 416 | total_timesteps 3552.
Path 417 | total_timesteps 3561.
Path 418 | total_timesteps 3569.
Path 419 | total_timesteps 3578.
Path 420 | total_timesteps 3586.
Path 421 | total_timesteps 3596.
Path 422 | total_timesteps 3602.
Path 423 | total_timesteps 3608.
Path 424 | total_timesteps 3620.
Path 425 | total_timesteps 3629.
Path 426 | total_timesteps 3637.
Path 427 | total_timesteps 3649.
Path 428 | total_timesteps 3657.
Path 429 | total_timesteps 3665.
Path 430 | total_timesteps 3671.
Path 431 | total_timesteps 3679.
Path 432 | total_timesteps 3686.
Path 433 | total_timesteps 3693.
Path 434 | total_timesteps 3704.
Path 435 | total_timesteps 3713.
Path 436 | total_timesteps 3722.
Path 437 | total_timesteps 3730.
Path 438 | total_timesteps 3736.
Path 439 | total_timesteps 3744.
Path 440 | total_timesteps 3751.
Path 441 | total_timesteps 3763.
Path 442 | total_timesteps 3769.
Path 443 | total_timesteps 3776.
Path 444 | total_timesteps 3784.
Path 445 | total_timesteps 3798.
Path 446 | total_timesteps 3810.
Path 447 | total_timesteps 3818.
Path 448 | total_timesteps 3827.
Path 449 | total_timesteps 3837.
Path 450 | total_timesteps 3848.
Path 451 | total_timesteps 3855.
Path 452 | total_timesteps 3862.
Path 453 | total_timesteps 3870.
Path 454 | total_timesteps 3877.
Path 455 | total_timesteps 3888.
Path 456 | total_timesteps 3897.
Path 457 | total_timesteps 3906.
Path 458 | total_timesteps 3913.
Path 459 | total_timesteps 3920.
Path 460 | total_timesteps 3929.
Path 461 | total_timesteps 3937.
Path 462 | total_timesteps 3946.
Path 463 | total_timesteps 3956.
Path 464 | total_timesteps 3963.
Path 465 | total_timesteps 3970.
Path 466 | total_timesteps 3978.
Path 467 | total_timesteps 3985.
Path 468 | total_timesteps 3996.
Path 469 | total_timesteps 4004.
Path 470 | total_timesteps 4012.
Path 471 | total_timesteps 4018.
Path 472 | total_timesteps 4024.
Path 473 | total_timesteps 4032.
Path 474 | total_timesteps 4040.
Path 475 | total_timesteps 4052.
Path 476 | total_timesteps 4060.
Path 477 | total_timesteps 4068.
Path 478 | total_timesteps 4075.
Path 479 | total_timesteps 4083.
Path 480 | total_timesteps 4092.
Path 481 | total_timesteps 4100.
Path 482 | total_timesteps 4107.
Path 483 | total_timesteps 4115.
Path 484 | total_timesteps 4124.
Path 485 | total_timesteps 4130.
Path 486 | total_timesteps 4139.
Path 487 | total_timesteps 4148.
Path 488 | total_timesteps 4158.
Path 489 | total_timesteps 4168.
Path 490 | total_timesteps 4177.
Path 491 | total_timesteps 4187.
Path 492 | total_timesteps 4196.
Path 493 | total_timesteps 4205.
Path 494 | total_timesteps 4215.
Path 495 | total_timesteps 4223.
Path 496 | total_timesteps 4231.
Path 497 | total_timesteps 4242.
Path 498 | total_timesteps 4250.
Path 499 | total_timesteps 4261.
Path 500 | total_timesteps 4272.
Path 501 | total_timesteps 4280.
Path 502 | total_timesteps 4287.
Path 503 | total_timesteps 4294.
Path 504 | total_timesteps 4304.
Path 505 | total_timesteps 4311.
Path 506 | total_timesteps 4323.
Path 507 | total_timesteps 4332.
Path 508 | total_timesteps 4340.
Path 509 | total_timesteps 4351.
Path 510 | total_timesteps 4358.
Path 511 | total_timesteps 4366.
Path 512 | total_timesteps 4374.
Path 513 | total_timesteps 4386.
Path 514 | total_timesteps 4394.
Path 515 | total_timesteps 4402.
Path 516 | total_timesteps 4411.
Path 517 | total_timesteps 4419.
Path 518 | total_timesteps 4426.
Path 519 | total_timesteps 4435.
Path 520 | total_timesteps 4442.
Path 521 | total_timesteps 4451.
Path 522 | total_timesteps 4458.
Path 523 | total_timesteps 4465.
Path 524 | total_timesteps 4473.
Path 525 | total_timesteps 4481.
Path 526 | total_timesteps 4490.
Path 527 | total_timesteps 4496.
Path 528 | total_timesteps 4505.
Path 529 | total_timesteps 4514.
Path 530 | total_timesteps 4527.
Path 531 | total_timesteps 4535.
Path 532 | total_timesteps 4542.
Path 533 | total_timesteps 4557.
Path 534 | total_timesteps 4568.
Path 535 | total_timesteps 4575.
Path 536 | total_timesteps 4584.
Path 537 | total_timesteps 4593.
Path 538 | total_timesteps 4603.
Path 539 | total_timesteps 4611.
Path 540 | total_timesteps 4625.
Path 541 | total_timesteps 4633.
Path 542 | total_timesteps 4640.
Path 543 | total_timesteps 4649.
Path 544 | total_timesteps 4659.
Path 545 | total_timesteps 4666.
Path 546 | total_timesteps 4672.
Path 547 | total_timesteps 4681.
Path 548 | total_timesteps 4688.
Path 549 | total_timesteps 4697.
Path 550 | total_timesteps 4703.
Path 551 | total_timesteps 4711.
Path 552 | total_timesteps 4718.
Path 553 | total_timesteps 4726.
Path 554 | total_timesteps 4734.
Path 555 | total_timesteps 4742.
Path 556 | total_timesteps 4749.
Path 557 | total_timesteps 4756.
Path 558 | total_timesteps 4763.
Path 559 | total_timesteps 4772.
Path 560 | total_timesteps 4780.
Path 561 | total_timesteps 4786.
Path 562 | total_timesteps 4799.
Path 563 | total_timesteps 4808.
Path 564 | total_timesteps 4817.
Path 565 | total_timesteps 4824.
Path 566 | total_timesteps 4832.
Path 567 | total_timesteps 4840.
Path 568 | total_timesteps 4848.
Path 569 | total_timesteps 4859.
Path 570 | total_timesteps 4869.
Path 571 | total_timesteps 4880.
Path 572 | total_timesteps 4888.
Path 573 | total_timesteps 4896.
Path 574 | total_timesteps 4903.
Path 575 | total_timesteps 4911.
Path 576 | total_timesteps 4924.
Path 577 | total_timesteps 4932.
Path 578 | total_timesteps 4941.
Path 579 | total_timesteps 4949.
Path 580 | total_timesteps 4956.
Path 581 | total_timesteps 4965.
Path 582 | total_timesteps 4974.
Path 583 | total_timesteps 4983.
Path 584 | total_timesteps 4995.
Path 585 | total_timesteps 5004.
Path 586 | total_timesteps 5010.
Path 587 | total_timesteps 5017.
Path 588 | total_timesteps 5024.
Path 589 | total_timesteps 5033.
Path 590 | total_timesteps 5046.
Path 591 | total_timesteps 5055.
Path 592 | total_timesteps 5062.
Path 593 | total_timesteps 5075.
Path 594 | total_timesteps 5084.
Path 595 | total_timesteps 5090.
Path 596 | total_timesteps 5102.
Path 597 | total_timesteps 5109.
Path 598 | total_timesteps 5117.
Path 599 | total_timesteps 5126.
Path 600 | total_timesteps 5133.
Path 601 | total_timesteps 5141.
Path 602 | total_timesteps 5147.
Path 603 | total_timesteps 5153.
Path 604 | total_timesteps 5162.
Path 605 | total_timesteps 5169.
Path 606 | total_timesteps 5176.
Path 607 | total_timesteps 5185.
Path 608 | total_timesteps 5193.
Path 609 | total_timesteps 5204.
Path 610 | total_timesteps 5212.
Path 611 | total_timesteps 5221.
Path 612 | total_timesteps 5228.
Path 613 | total_timesteps 5235.
Path 614 | total_timesteps 5242.
Path 615 | total_timesteps 5249.
Path 616 | total_timesteps 5258.
Path 617 | total_timesteps 5269.
Path 618 | total_timesteps 5278.
Path 619 | total_timesteps 5290.
Path 620 | total_timesteps 5298.
Path 621 | total_timesteps 5305.
Path 622 | total_timesteps 5312.
Path 623 | total_timesteps 5318.
Path 624 | total_timesteps 5326.
Path 625 | total_timesteps 5335.
Path 626 | total_timesteps 5342.
Path 627 | total_timesteps 5352.
Path 628 | total_timesteps 5360.
Path 629 | total_timesteps 5367.
Path 630 | total_timesteps 5378.
Path 631 | total_timesteps 5388.
Path 632 | total_timesteps 5397.
Path 633 | total_timesteps 5403.
Path 634 | total_timesteps 5410.
Path 635 | total_timesteps 5424.
Path 636 | total_timesteps 5436.
Path 637 | total_timesteps 5444.
Path 638 | total_timesteps 5451.
Path 639 | total_timesteps 5465.
Path 640 | total_timesteps 5473.
Path 641 | total_timesteps 5481.
Path 642 | total_timesteps 5491.
Path 643 | total_timesteps 5497.
Path 644 | total_timesteps 5504.
Path 645 | total_timesteps 5514.
Path 646 | total_timesteps 5525.
Path 647 | total_timesteps 5532.
Path 648 | total_timesteps 5543.
Path 649 | total_timesteps 5554.
Path 650 | total_timesteps 5560.
Path 651 | total_timesteps 5570.
Path 652 | total_timesteps 5579.
Path 653 | total_timesteps 5591.
Path 654 | total_timesteps 5598.
Path 655 | total_timesteps 5606.
Path 656 | total_timesteps 5612.
Path 657 | total_timesteps 5619.
Path 658 | total_timesteps 5625.
Path 659 | total_timesteps 5634.
Path 660 | total_timesteps 5643.
Path 661 | total_timesteps 5650.
Path 662 | total_timesteps 5658.
Path 663 | total_timesteps 5666.
Path 664 | total_timesteps 5672.
Path 665 | total_timesteps 5680.
Path 666 | total_timesteps 5687.
Path 667 | total_timesteps 5695.
Path 668 | total_timesteps 5703.
Path 669 | total_timesteps 5710.
Path 670 | total_timesteps 5718.
Path 671 | total_timesteps 5726.
Path 672 | total_timesteps 5734.
Path 673 | total_timesteps 5742.
Path 674 | total_timesteps 5749.
Path 675 | total_timesteps 5755.
Path 676 | total_timesteps 5764.
Path 677 | total_timesteps 5771.
Path 678 | total_timesteps 5779.
Path 679 | total_timesteps 5797.
Path 680 | total_timesteps 5804.
Path 681 | total_timesteps 5812.
Path 682 | total_timesteps 5821.
Path 683 | total_timesteps 5831.
Path 684 | total_timesteps 5839.
Path 685 | total_timesteps 5850.
Path 686 | total_timesteps 5859.
Path 687 | total_timesteps 5868.
Path 688 | total_timesteps 5876.
Path 689 | total_timesteps 5883.
Path 690 | total_timesteps 5891.
Path 691 | total_timesteps 5902.
Path 692 | total_timesteps 5910.
Path 693 | total_timesteps 5917.
Path 694 | total_timesteps 5927.
Path 695 | total_timesteps 5934.
Path 696 | total_timesteps 5942.
Path 697 | total_timesteps 5948.
Path 698 | total_timesteps 5956.
Path 699 | total_timesteps 5970.
Path 700 | total_timesteps 5977.
Path 701 | total_timesteps 5983.
Path 702 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.86    |
| Iteration     | 28       |
| MaximumReturn | -1.25    |
| MinimumReturn | -14.6    |
| TotalSamples  | 120103   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.0015629003755748272
Validation loss = 0.0015739725204184651
Validation loss = 0.0015856478130444884
Validation loss = 0.001332168816588819
Validation loss = 0.001564766513183713
Validation loss = 0.001418666448444128
Validation loss = 0.0014268351951614022
Validation loss = 0.001538509619422257
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 13.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 62.
Path 8 | total_timesteps 69.
Path 9 | total_timesteps 79.
Path 10 | total_timesteps 87.
Path 11 | total_timesteps 96.
Path 12 | total_timesteps 104.
Path 13 | total_timesteps 111.
Path 14 | total_timesteps 121.
Path 15 | total_timesteps 128.
Path 16 | total_timesteps 139.
Path 17 | total_timesteps 146.
Path 18 | total_timesteps 153.
Path 19 | total_timesteps 162.
Path 20 | total_timesteps 168.
Path 21 | total_timesteps 177.
Path 22 | total_timesteps 184.
Path 23 | total_timesteps 191.
Path 24 | total_timesteps 207.
Path 25 | total_timesteps 215.
Path 26 | total_timesteps 223.
Path 27 | total_timesteps 230.
Path 28 | total_timesteps 237.
Path 29 | total_timesteps 247.
Path 30 | total_timesteps 255.
Path 31 | total_timesteps 263.
Path 32 | total_timesteps 269.
Path 33 | total_timesteps 277.
Path 34 | total_timesteps 284.
Path 35 | total_timesteps 291.
Path 36 | total_timesteps 297.
Path 37 | total_timesteps 303.
Path 38 | total_timesteps 310.
Path 39 | total_timesteps 319.
Path 40 | total_timesteps 326.
Path 41 | total_timesteps 332.
Path 42 | total_timesteps 339.
Path 43 | total_timesteps 345.
Path 44 | total_timesteps 352.
Path 45 | total_timesteps 362.
Path 46 | total_timesteps 370.
Path 47 | total_timesteps 377.
Path 48 | total_timesteps 383.
Path 49 | total_timesteps 391.
Path 50 | total_timesteps 397.
Path 51 | total_timesteps 406.
Path 52 | total_timesteps 417.
Path 53 | total_timesteps 428.
Path 54 | total_timesteps 435.
Path 55 | total_timesteps 443.
Path 56 | total_timesteps 451.
Path 57 | total_timesteps 460.
Path 58 | total_timesteps 467.
Path 59 | total_timesteps 477.
Path 60 | total_timesteps 488.
Path 61 | total_timesteps 495.
Path 62 | total_timesteps 505.
Path 63 | total_timesteps 511.
Path 64 | total_timesteps 521.
Path 65 | total_timesteps 528.
Path 66 | total_timesteps 534.
Path 67 | total_timesteps 542.
Path 68 | total_timesteps 550.
Path 69 | total_timesteps 559.
Path 70 | total_timesteps 570.
Path 71 | total_timesteps 578.
Path 72 | total_timesteps 589.
Path 73 | total_timesteps 595.
Path 74 | total_timesteps 602.
Path 75 | total_timesteps 610.
Path 76 | total_timesteps 618.
Path 77 | total_timesteps 624.
Path 78 | total_timesteps 631.
Path 79 | total_timesteps 637.
Path 80 | total_timesteps 648.
Path 81 | total_timesteps 657.
Path 82 | total_timesteps 665.
Path 83 | total_timesteps 673.
Path 84 | total_timesteps 681.
Path 85 | total_timesteps 691.
Path 86 | total_timesteps 697.
Path 87 | total_timesteps 705.
Path 88 | total_timesteps 714.
Path 89 | total_timesteps 726.
Path 90 | total_timesteps 733.
Path 91 | total_timesteps 741.
Path 92 | total_timesteps 747.
Path 93 | total_timesteps 755.
Path 94 | total_timesteps 765.
Path 95 | total_timesteps 777.
Path 96 | total_timesteps 784.
Path 97 | total_timesteps 794.
Path 98 | total_timesteps 802.
Path 99 | total_timesteps 811.
Path 100 | total_timesteps 822.
Path 101 | total_timesteps 830.
Path 102 | total_timesteps 842.
Path 103 | total_timesteps 851.
Path 104 | total_timesteps 858.
Path 105 | total_timesteps 869.
Path 106 | total_timesteps 881.
Path 107 | total_timesteps 888.
Path 108 | total_timesteps 894.
Path 109 | total_timesteps 906.
Path 110 | total_timesteps 914.
Path 111 | total_timesteps 920.
Path 112 | total_timesteps 932.
Path 113 | total_timesteps 940.
Path 114 | total_timesteps 950.
Path 115 | total_timesteps 960.
Path 116 | total_timesteps 966.
Path 117 | total_timesteps 975.
Path 118 | total_timesteps 982.
Path 119 | total_timesteps 990.
Path 120 | total_timesteps 997.
Path 121 | total_timesteps 1005.
Path 122 | total_timesteps 1014.
Path 123 | total_timesteps 1023.
Path 124 | total_timesteps 1030.
Path 125 | total_timesteps 1040.
Path 126 | total_timesteps 1048.
Path 127 | total_timesteps 1056.
Path 128 | total_timesteps 1066.
Path 129 | total_timesteps 1077.
Path 130 | total_timesteps 1083.
Path 131 | total_timesteps 1091.
Path 132 | total_timesteps 1099.
Path 133 | total_timesteps 1108.
Path 134 | total_timesteps 1116.
Path 135 | total_timesteps 1124.
Path 136 | total_timesteps 1132.
Path 137 | total_timesteps 1145.
Path 138 | total_timesteps 1154.
Path 139 | total_timesteps 1162.
Path 140 | total_timesteps 1170.
Path 141 | total_timesteps 1181.
Path 142 | total_timesteps 1189.
Path 143 | total_timesteps 1197.
Path 144 | total_timesteps 1204.
Path 145 | total_timesteps 1211.
Path 146 | total_timesteps 1219.
Path 147 | total_timesteps 1227.
Path 148 | total_timesteps 1234.
Path 149 | total_timesteps 1245.
Path 150 | total_timesteps 1253.
Path 151 | total_timesteps 1260.
Path 152 | total_timesteps 1268.
Path 153 | total_timesteps 1274.
Path 154 | total_timesteps 1282.
Path 155 | total_timesteps 1289.
Path 156 | total_timesteps 1296.
Path 157 | total_timesteps 1302.
Path 158 | total_timesteps 1310.
Path 159 | total_timesteps 1318.
Path 160 | total_timesteps 1326.
Path 161 | total_timesteps 1334.
Path 162 | total_timesteps 1342.
Path 163 | total_timesteps 1350.
Path 164 | total_timesteps 1358.
Path 165 | total_timesteps 1367.
Path 166 | total_timesteps 1377.
Path 167 | total_timesteps 1386.
Path 168 | total_timesteps 1395.
Path 169 | total_timesteps 1404.
Path 170 | total_timesteps 1412.
Path 171 | total_timesteps 1419.
Path 172 | total_timesteps 1429.
Path 173 | total_timesteps 1436.
Path 174 | total_timesteps 1442.
Path 175 | total_timesteps 1449.
Path 176 | total_timesteps 1459.
Path 177 | total_timesteps 1466.
Path 178 | total_timesteps 1474.
Path 179 | total_timesteps 1480.
Path 180 | total_timesteps 1486.
Path 181 | total_timesteps 1493.
Path 182 | total_timesteps 1500.
Path 183 | total_timesteps 1506.
Path 184 | total_timesteps 1513.
Path 185 | total_timesteps 1519.
Path 186 | total_timesteps 1526.
Path 187 | total_timesteps 1535.
Path 188 | total_timesteps 1542.
Path 189 | total_timesteps 1548.
Path 190 | total_timesteps 1555.
Path 191 | total_timesteps 1562.
Path 192 | total_timesteps 1569.
Path 193 | total_timesteps 1577.
Path 194 | total_timesteps 1584.
Path 195 | total_timesteps 1591.
Path 196 | total_timesteps 1603.
Path 197 | total_timesteps 1613.
Path 198 | total_timesteps 1621.
Path 199 | total_timesteps 1627.
Path 200 | total_timesteps 1633.
Path 201 | total_timesteps 1640.
Path 202 | total_timesteps 1649.
Path 203 | total_timesteps 1655.
Path 204 | total_timesteps 1662.
Path 205 | total_timesteps 1671.
Path 206 | total_timesteps 1680.
Path 207 | total_timesteps 1691.
Path 208 | total_timesteps 1698.
Path 209 | total_timesteps 1706.
Path 210 | total_timesteps 1714.
Path 211 | total_timesteps 1721.
Path 212 | total_timesteps 1731.
Path 213 | total_timesteps 1741.
Path 214 | total_timesteps 1747.
Path 215 | total_timesteps 1760.
Path 216 | total_timesteps 1767.
Path 217 | total_timesteps 1774.
Path 218 | total_timesteps 1780.
Path 219 | total_timesteps 1788.
Path 220 | total_timesteps 1796.
Path 221 | total_timesteps 1804.
Path 222 | total_timesteps 1811.
Path 223 | total_timesteps 1819.
Path 224 | total_timesteps 1829.
Path 225 | total_timesteps 1837.
Path 226 | total_timesteps 1844.
Path 227 | total_timesteps 1851.
Path 228 | total_timesteps 1864.
Path 229 | total_timesteps 1871.
Path 230 | total_timesteps 1879.
Path 231 | total_timesteps 1887.
Path 232 | total_timesteps 1898.
Path 233 | total_timesteps 1908.
Path 234 | total_timesteps 1917.
Path 235 | total_timesteps 1924.
Path 236 | total_timesteps 1930.
Path 237 | total_timesteps 1936.
Path 238 | total_timesteps 1944.
Path 239 | total_timesteps 1951.
Path 240 | total_timesteps 1957.
Path 241 | total_timesteps 1967.
Path 242 | total_timesteps 1976.
Path 243 | total_timesteps 1983.
Path 244 | total_timesteps 1990.
Path 245 | total_timesteps 1998.
Path 246 | total_timesteps 2006.
Path 247 | total_timesteps 2014.
Path 248 | total_timesteps 2021.
Path 249 | total_timesteps 2028.
Path 250 | total_timesteps 2041.
Path 251 | total_timesteps 2050.
Path 252 | total_timesteps 2062.
Path 253 | total_timesteps 2069.
Path 254 | total_timesteps 2076.
Path 255 | total_timesteps 2083.
Path 256 | total_timesteps 2093.
Path 257 | total_timesteps 2102.
Path 258 | total_timesteps 2108.
Path 259 | total_timesteps 2115.
Path 260 | total_timesteps 2121.
Path 261 | total_timesteps 2127.
Path 262 | total_timesteps 2136.
Path 263 | total_timesteps 2145.
Path 264 | total_timesteps 2153.
Path 265 | total_timesteps 2162.
Path 266 | total_timesteps 2169.
Path 267 | total_timesteps 2178.
Path 268 | total_timesteps 2184.
Path 269 | total_timesteps 2192.
Path 270 | total_timesteps 2202.
Path 271 | total_timesteps 2212.
Path 272 | total_timesteps 2223.
Path 273 | total_timesteps 2229.
Path 274 | total_timesteps 2240.
Path 275 | total_timesteps 2249.
Path 276 | total_timesteps 2255.
Path 277 | total_timesteps 2261.
Path 278 | total_timesteps 2270.
Path 279 | total_timesteps 2277.
Path 280 | total_timesteps 2286.
Path 281 | total_timesteps 2297.
Path 282 | total_timesteps 2305.
Path 283 | total_timesteps 2312.
Path 284 | total_timesteps 2320.
Path 285 | total_timesteps 2327.
Path 286 | total_timesteps 2335.
Path 287 | total_timesteps 2344.
Path 288 | total_timesteps 2356.
Path 289 | total_timesteps 2362.
Path 290 | total_timesteps 2368.
Path 291 | total_timesteps 2378.
Path 292 | total_timesteps 2385.
Path 293 | total_timesteps 2391.
Path 294 | total_timesteps 2398.
Path 295 | total_timesteps 2405.
Path 296 | total_timesteps 2414.
Path 297 | total_timesteps 2422.
Path 298 | total_timesteps 2428.
Path 299 | total_timesteps 2435.
Path 300 | total_timesteps 2441.
Path 301 | total_timesteps 2449.
Path 302 | total_timesteps 2456.
Path 303 | total_timesteps 2463.
Path 304 | total_timesteps 2474.
Path 305 | total_timesteps 2480.
Path 306 | total_timesteps 2488.
Path 307 | total_timesteps 2497.
Path 308 | total_timesteps 2504.
Path 309 | total_timesteps 2513.
Path 310 | total_timesteps 2522.
Path 311 | total_timesteps 2529.
Path 312 | total_timesteps 2538.
Path 313 | total_timesteps 2546.
Path 314 | total_timesteps 2553.
Path 315 | total_timesteps 2565.
Path 316 | total_timesteps 2571.
Path 317 | total_timesteps 2577.
Path 318 | total_timesteps 2585.
Path 319 | total_timesteps 2594.
Path 320 | total_timesteps 2601.
Path 321 | total_timesteps 2607.
Path 322 | total_timesteps 2615.
Path 323 | total_timesteps 2621.
Path 324 | total_timesteps 2630.
Path 325 | total_timesteps 2638.
Path 326 | total_timesteps 2647.
Path 327 | total_timesteps 2657.
Path 328 | total_timesteps 2664.
Path 329 | total_timesteps 2670.
Path 330 | total_timesteps 2676.
Path 331 | total_timesteps 2683.
Path 332 | total_timesteps 2690.
Path 333 | total_timesteps 2698.
Path 334 | total_timesteps 2704.
Path 335 | total_timesteps 2711.
Path 336 | total_timesteps 2719.
Path 337 | total_timesteps 2727.
Path 338 | total_timesteps 2734.
Path 339 | total_timesteps 2743.
Path 340 | total_timesteps 2749.
Path 341 | total_timesteps 2760.
Path 342 | total_timesteps 2770.
Path 343 | total_timesteps 2776.
Path 344 | total_timesteps 2785.
Path 345 | total_timesteps 2792.
Path 346 | total_timesteps 2800.
Path 347 | total_timesteps 2807.
Path 348 | total_timesteps 2813.
Path 349 | total_timesteps 2825.
Path 350 | total_timesteps 2834.
Path 351 | total_timesteps 2841.
Path 352 | total_timesteps 2849.
Path 353 | total_timesteps 2856.
Path 354 | total_timesteps 2866.
Path 355 | total_timesteps 2873.
Path 356 | total_timesteps 2881.
Path 357 | total_timesteps 2890.
Path 358 | total_timesteps 2899.
Path 359 | total_timesteps 2907.
Path 360 | total_timesteps 2915.
Path 361 | total_timesteps 2921.
Path 362 | total_timesteps 2929.
Path 363 | total_timesteps 2937.
Path 364 | total_timesteps 2945.
Path 365 | total_timesteps 2957.
Path 366 | total_timesteps 2965.
Path 367 | total_timesteps 2973.
Path 368 | total_timesteps 2981.
Path 369 | total_timesteps 2991.
Path 370 | total_timesteps 2998.
Path 371 | total_timesteps 3006.
Path 372 | total_timesteps 3015.
Path 373 | total_timesteps 3022.
Path 374 | total_timesteps 3029.
Path 375 | total_timesteps 3037.
Path 376 | total_timesteps 3045.
Path 377 | total_timesteps 3056.
Path 378 | total_timesteps 3064.
Path 379 | total_timesteps 3071.
Path 380 | total_timesteps 3079.
Path 381 | total_timesteps 3088.
Path 382 | total_timesteps 3096.
Path 383 | total_timesteps 3104.
Path 384 | total_timesteps 3111.
Path 385 | total_timesteps 3119.
Path 386 | total_timesteps 3128.
Path 387 | total_timesteps 3138.
Path 388 | total_timesteps 3145.
Path 389 | total_timesteps 3152.
Path 390 | total_timesteps 3159.
Path 391 | total_timesteps 3167.
Path 392 | total_timesteps 3176.
Path 393 | total_timesteps 3183.
Path 394 | total_timesteps 3190.
Path 395 | total_timesteps 3201.
Path 396 | total_timesteps 3209.
Path 397 | total_timesteps 3216.
Path 398 | total_timesteps 3224.
Path 399 | total_timesteps 3232.
Path 400 | total_timesteps 3239.
Path 401 | total_timesteps 3246.
Path 402 | total_timesteps 3252.
Path 403 | total_timesteps 3261.
Path 404 | total_timesteps 3268.
Path 405 | total_timesteps 3275.
Path 406 | total_timesteps 3282.
Path 407 | total_timesteps 3288.
Path 408 | total_timesteps 3296.
Path 409 | total_timesteps 3304.
Path 410 | total_timesteps 3310.
Path 411 | total_timesteps 3320.
Path 412 | total_timesteps 3327.
Path 413 | total_timesteps 3336.
Path 414 | total_timesteps 3346.
Path 415 | total_timesteps 3353.
Path 416 | total_timesteps 3359.
Path 417 | total_timesteps 3368.
Path 418 | total_timesteps 3376.
Path 419 | total_timesteps 3384.
Path 420 | total_timesteps 3391.
Path 421 | total_timesteps 3398.
Path 422 | total_timesteps 3406.
Path 423 | total_timesteps 3418.
Path 424 | total_timesteps 3425.
Path 425 | total_timesteps 3434.
Path 426 | total_timesteps 3443.
Path 427 | total_timesteps 3450.
Path 428 | total_timesteps 3457.
Path 429 | total_timesteps 3464.
Path 430 | total_timesteps 3472.
Path 431 | total_timesteps 3478.
Path 432 | total_timesteps 3486.
Path 433 | total_timesteps 3494.
Path 434 | total_timesteps 3500.
Path 435 | total_timesteps 3509.
Path 436 | total_timesteps 3518.
Path 437 | total_timesteps 3525.
Path 438 | total_timesteps 3534.
Path 439 | total_timesteps 3543.
Path 440 | total_timesteps 3551.
Path 441 | total_timesteps 3558.
Path 442 | total_timesteps 3567.
Path 443 | total_timesteps 3575.
Path 444 | total_timesteps 3583.
Path 445 | total_timesteps 3593.
Path 446 | total_timesteps 3600.
Path 447 | total_timesteps 3607.
Path 448 | total_timesteps 3615.
Path 449 | total_timesteps 3623.
Path 450 | total_timesteps 3636.
Path 451 | total_timesteps 3645.
Path 452 | total_timesteps 3654.
Path 453 | total_timesteps 3663.
Path 454 | total_timesteps 3671.
Path 455 | total_timesteps 3678.
Path 456 | total_timesteps 3685.
Path 457 | total_timesteps 3694.
Path 458 | total_timesteps 3701.
Path 459 | total_timesteps 3708.
Path 460 | total_timesteps 3719.
Path 461 | total_timesteps 3729.
Path 462 | total_timesteps 3738.
Path 463 | total_timesteps 3746.
Path 464 | total_timesteps 3754.
Path 465 | total_timesteps 3766.
Path 466 | total_timesteps 3774.
Path 467 | total_timesteps 3781.
Path 468 | total_timesteps 3788.
Path 469 | total_timesteps 3796.
Path 470 | total_timesteps 3804.
Path 471 | total_timesteps 3814.
Path 472 | total_timesteps 3823.
Path 473 | total_timesteps 3830.
Path 474 | total_timesteps 3836.
Path 475 | total_timesteps 3843.
Path 476 | total_timesteps 3851.
Path 477 | total_timesteps 3857.
Path 478 | total_timesteps 3863.
Path 479 | total_timesteps 3870.
Path 480 | total_timesteps 3878.
Path 481 | total_timesteps 3888.
Path 482 | total_timesteps 3896.
Path 483 | total_timesteps 3905.
Path 484 | total_timesteps 3915.
Path 485 | total_timesteps 3927.
Path 486 | total_timesteps 3934.
Path 487 | total_timesteps 3943.
Path 488 | total_timesteps 3949.
Path 489 | total_timesteps 3959.
Path 490 | total_timesteps 3966.
Path 491 | total_timesteps 3976.
Path 492 | total_timesteps 3985.
Path 493 | total_timesteps 3993.
Path 494 | total_timesteps 3999.
Path 495 | total_timesteps 4009.
Path 496 | total_timesteps 4017.
Path 497 | total_timesteps 4023.
Path 498 | total_timesteps 4032.
Path 499 | total_timesteps 4043.
Path 500 | total_timesteps 4051.
Path 501 | total_timesteps 4062.
Path 502 | total_timesteps 4069.
Path 503 | total_timesteps 4082.
Path 504 | total_timesteps 4090.
Path 505 | total_timesteps 4097.
Path 506 | total_timesteps 4107.
Path 507 | total_timesteps 4113.
Path 508 | total_timesteps 4120.
Path 509 | total_timesteps 4127.
Path 510 | total_timesteps 4133.
Path 511 | total_timesteps 4140.
Path 512 | total_timesteps 4150.
Path 513 | total_timesteps 4157.
Path 514 | total_timesteps 4170.
Path 515 | total_timesteps 4177.
Path 516 | total_timesteps 4183.
Path 517 | total_timesteps 4191.
Path 518 | total_timesteps 4203.
Path 519 | total_timesteps 4212.
Path 520 | total_timesteps 4218.
Path 521 | total_timesteps 4226.
Path 522 | total_timesteps 4234.
Path 523 | total_timesteps 4244.
Path 524 | total_timesteps 4250.
Path 525 | total_timesteps 4257.
Path 526 | total_timesteps 4268.
Path 527 | total_timesteps 4275.
Path 528 | total_timesteps 4282.
Path 529 | total_timesteps 4288.
Path 530 | total_timesteps 4295.
Path 531 | total_timesteps 4302.
Path 532 | total_timesteps 4309.
Path 533 | total_timesteps 4319.
Path 534 | total_timesteps 4328.
Path 535 | total_timesteps 4335.
Path 536 | total_timesteps 4343.
Path 537 | total_timesteps 4351.
Path 538 | total_timesteps 4358.
Path 539 | total_timesteps 4367.
Path 540 | total_timesteps 4374.
Path 541 | total_timesteps 4380.
Path 542 | total_timesteps 4387.
Path 543 | total_timesteps 4394.
Path 544 | total_timesteps 4404.
Path 545 | total_timesteps 4411.
Path 546 | total_timesteps 4419.
Path 547 | total_timesteps 4426.
Path 548 | total_timesteps 4436.
Path 549 | total_timesteps 4443.
Path 550 | total_timesteps 4449.
Path 551 | total_timesteps 4458.
Path 552 | total_timesteps 4465.
Path 553 | total_timesteps 4473.
Path 554 | total_timesteps 4487.
Path 555 | total_timesteps 4494.
Path 556 | total_timesteps 4504.
Path 557 | total_timesteps 4511.
Path 558 | total_timesteps 4521.
Path 559 | total_timesteps 4531.
Path 560 | total_timesteps 4539.
Path 561 | total_timesteps 4550.
Path 562 | total_timesteps 4557.
Path 563 | total_timesteps 4564.
Path 564 | total_timesteps 4571.
Path 565 | total_timesteps 4580.
Path 566 | total_timesteps 4587.
Path 567 | total_timesteps 4596.
Path 568 | total_timesteps 4603.
Path 569 | total_timesteps 4615.
Path 570 | total_timesteps 4623.
Path 571 | total_timesteps 4630.
Path 572 | total_timesteps 4641.
Path 573 | total_timesteps 4651.
Path 574 | total_timesteps 4659.
Path 575 | total_timesteps 4670.
Path 576 | total_timesteps 4676.
Path 577 | total_timesteps 4683.
Path 578 | total_timesteps 4689.
Path 579 | total_timesteps 4695.
Path 580 | total_timesteps 4704.
Path 581 | total_timesteps 4711.
Path 582 | total_timesteps 4717.
Path 583 | total_timesteps 4727.
Path 584 | total_timesteps 4738.
Path 585 | total_timesteps 4745.
Path 586 | total_timesteps 4751.
Path 587 | total_timesteps 4759.
Path 588 | total_timesteps 4766.
Path 589 | total_timesteps 4781.
Path 590 | total_timesteps 4790.
Path 591 | total_timesteps 4799.
Path 592 | total_timesteps 4807.
Path 593 | total_timesteps 4814.
Path 594 | total_timesteps 4822.
Path 595 | total_timesteps 4830.
Path 596 | total_timesteps 4838.
Path 597 | total_timesteps 4845.
Path 598 | total_timesteps 4851.
Path 599 | total_timesteps 4859.
Path 600 | total_timesteps 4865.
Path 601 | total_timesteps 4875.
Path 602 | total_timesteps 4883.
Path 603 | total_timesteps 4892.
Path 604 | total_timesteps 4898.
Path 605 | total_timesteps 4907.
Path 606 | total_timesteps 4916.
Path 607 | total_timesteps 4922.
Path 608 | total_timesteps 4929.
Path 609 | total_timesteps 4939.
Path 610 | total_timesteps 4951.
Path 611 | total_timesteps 4959.
Path 612 | total_timesteps 4966.
Path 613 | total_timesteps 4975.
Path 614 | total_timesteps 4983.
Path 615 | total_timesteps 4993.
Path 616 | total_timesteps 5001.
Path 617 | total_timesteps 5008.
Path 618 | total_timesteps 5016.
Path 619 | total_timesteps 5023.
Path 620 | total_timesteps 5031.
Path 621 | total_timesteps 5037.
Path 622 | total_timesteps 5045.
Path 623 | total_timesteps 5053.
Path 624 | total_timesteps 5061.
Path 625 | total_timesteps 5068.
Path 626 | total_timesteps 5075.
Path 627 | total_timesteps 5084.
Path 628 | total_timesteps 5092.
Path 629 | total_timesteps 5101.
Path 630 | total_timesteps 5108.
Path 631 | total_timesteps 5114.
Path 632 | total_timesteps 5122.
Path 633 | total_timesteps 5130.
Path 634 | total_timesteps 5136.
Path 635 | total_timesteps 5144.
Path 636 | total_timesteps 5154.
Path 637 | total_timesteps 5163.
Path 638 | total_timesteps 5170.
Path 639 | total_timesteps 5178.
Path 640 | total_timesteps 5187.
Path 641 | total_timesteps 5193.
Path 642 | total_timesteps 5199.
Path 643 | total_timesteps 5206.
Path 644 | total_timesteps 5213.
Path 645 | total_timesteps 5220.
Path 646 | total_timesteps 5227.
Path 647 | total_timesteps 5233.
Path 648 | total_timesteps 5241.
Path 649 | total_timesteps 5248.
Path 650 | total_timesteps 5256.
Path 651 | total_timesteps 5264.
Path 652 | total_timesteps 5272.
Path 653 | total_timesteps 5279.
Path 654 | total_timesteps 5286.
Path 655 | total_timesteps 5293.
Path 656 | total_timesteps 5300.
Path 657 | total_timesteps 5309.
Path 658 | total_timesteps 5318.
Path 659 | total_timesteps 5326.
Path 660 | total_timesteps 5334.
Path 661 | total_timesteps 5342.
Path 662 | total_timesteps 5349.
Path 663 | total_timesteps 5359.
Path 664 | total_timesteps 5365.
Path 665 | total_timesteps 5377.
Path 666 | total_timesteps 5384.
Path 667 | total_timesteps 5392.
Path 668 | total_timesteps 5400.
Path 669 | total_timesteps 5410.
Path 670 | total_timesteps 5419.
Path 671 | total_timesteps 5426.
Path 672 | total_timesteps 5434.
Path 673 | total_timesteps 5443.
Path 674 | total_timesteps 5452.
Path 675 | total_timesteps 5459.
Path 676 | total_timesteps 5465.
Path 677 | total_timesteps 5471.
Path 678 | total_timesteps 5477.
Path 679 | total_timesteps 5486.
Path 680 | total_timesteps 5497.
Path 681 | total_timesteps 5504.
Path 682 | total_timesteps 5512.
Path 683 | total_timesteps 5519.
Path 684 | total_timesteps 5526.
Path 685 | total_timesteps 5533.
Path 686 | total_timesteps 5540.
Path 687 | total_timesteps 5549.
Path 688 | total_timesteps 5555.
Path 689 | total_timesteps 5561.
Path 690 | total_timesteps 5569.
Path 691 | total_timesteps 5575.
Path 692 | total_timesteps 5582.
Path 693 | total_timesteps 5589.
Path 694 | total_timesteps 5599.
Path 695 | total_timesteps 5608.
Path 696 | total_timesteps 5615.
Path 697 | total_timesteps 5621.
Path 698 | total_timesteps 5627.
Path 699 | total_timesteps 5636.
Path 700 | total_timesteps 5643.
Path 701 | total_timesteps 5649.
Path 702 | total_timesteps 5658.
Path 703 | total_timesteps 5666.
Path 704 | total_timesteps 5677.
Path 705 | total_timesteps 5686.
Path 706 | total_timesteps 5692.
Path 707 | total_timesteps 5701.
Path 708 | total_timesteps 5708.
Path 709 | total_timesteps 5715.
Path 710 | total_timesteps 5725.
Path 711 | total_timesteps 5736.
Path 712 | total_timesteps 5745.
Path 713 | total_timesteps 5753.
Path 714 | total_timesteps 5762.
Path 715 | total_timesteps 5774.
Path 716 | total_timesteps 5783.
Path 717 | total_timesteps 5795.
Path 718 | total_timesteps 5803.
Path 719 | total_timesteps 5815.
Path 720 | total_timesteps 5823.
Path 721 | total_timesteps 5832.
Path 722 | total_timesteps 5839.
Path 723 | total_timesteps 5847.
Path 724 | total_timesteps 5854.
Path 725 | total_timesteps 5860.
Path 726 | total_timesteps 5868.
Path 727 | total_timesteps 5878.
Path 728 | total_timesteps 5885.
Path 729 | total_timesteps 5893.
Path 730 | total_timesteps 5902.
Path 731 | total_timesteps 5910.
Path 732 | total_timesteps 5921.
Path 733 | total_timesteps 5928.
Path 734 | total_timesteps 5939.
Path 735 | total_timesteps 5949.
Path 736 | total_timesteps 5959.
Path 737 | total_timesteps 5967.
Path 738 | total_timesteps 5975.
Path 739 | total_timesteps 5982.
Path 740 | total_timesteps 5992.
Path 741 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.78    |
| Iteration     | 29       |
| MaximumReturn | -2.07    |
| MinimumReturn | -13.4    |
| TotalSamples  | 124107   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.0013715841341763735
Validation loss = 0.001408121082931757
Validation loss = 0.001388457603752613
Validation loss = 0.0013484803494066
Validation loss = 0.0014117032987996936
Validation loss = 0.0017797981854528189
Validation loss = 0.0014998289989307523
Validation loss = 0.001309216720983386
Validation loss = 0.0012945325579494238
Validation loss = 0.0014058963861316442
Validation loss = 0.0013252065982669592
Validation loss = 0.0013842439511790872
Validation loss = 0.0014622349990531802
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 40.
Path 5 | total_timesteps 47.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 62.
Path 8 | total_timesteps 71.
Path 9 | total_timesteps 82.
Path 10 | total_timesteps 96.
Path 11 | total_timesteps 103.
Path 12 | total_timesteps 110.
Path 13 | total_timesteps 116.
Path 14 | total_timesteps 125.
Path 15 | total_timesteps 131.
Path 16 | total_timesteps 138.
Path 17 | total_timesteps 147.
Path 18 | total_timesteps 154.
Path 19 | total_timesteps 161.
Path 20 | total_timesteps 169.
Path 21 | total_timesteps 180.
Path 22 | total_timesteps 189.
Path 23 | total_timesteps 198.
Path 24 | total_timesteps 205.
Path 25 | total_timesteps 213.
Path 26 | total_timesteps 226.
Path 27 | total_timesteps 236.
Path 28 | total_timesteps 242.
Path 29 | total_timesteps 252.
Path 30 | total_timesteps 260.
Path 31 | total_timesteps 271.
Path 32 | total_timesteps 282.
Path 33 | total_timesteps 289.
Path 34 | total_timesteps 304.
Path 35 | total_timesteps 312.
Path 36 | total_timesteps 322.
Path 37 | total_timesteps 331.
Path 38 | total_timesteps 339.
Path 39 | total_timesteps 351.
Path 40 | total_timesteps 364.
Path 41 | total_timesteps 372.
Path 42 | total_timesteps 378.
Path 43 | total_timesteps 385.
Path 44 | total_timesteps 393.
Path 45 | total_timesteps 401.
Path 46 | total_timesteps 408.
Path 47 | total_timesteps 419.
Path 48 | total_timesteps 427.
Path 49 | total_timesteps 435.
Path 50 | total_timesteps 442.
Path 51 | total_timesteps 449.
Path 52 | total_timesteps 456.
Path 53 | total_timesteps 466.
Path 54 | total_timesteps 472.
Path 55 | total_timesteps 481.
Path 56 | total_timesteps 488.
Path 57 | total_timesteps 500.
Path 58 | total_timesteps 508.
Path 59 | total_timesteps 516.
Path 60 | total_timesteps 523.
Path 61 | total_timesteps 530.
Path 62 | total_timesteps 538.
Path 63 | total_timesteps 545.
Path 64 | total_timesteps 559.
Path 65 | total_timesteps 566.
Path 66 | total_timesteps 573.
Path 67 | total_timesteps 585.
Path 68 | total_timesteps 592.
Path 69 | total_timesteps 601.
Path 70 | total_timesteps 608.
Path 71 | total_timesteps 618.
Path 72 | total_timesteps 627.
Path 73 | total_timesteps 636.
Path 74 | total_timesteps 644.
Path 75 | total_timesteps 654.
Path 76 | total_timesteps 661.
Path 77 | total_timesteps 668.
Path 78 | total_timesteps 674.
Path 79 | total_timesteps 684.
Path 80 | total_timesteps 690.
Path 81 | total_timesteps 697.
Path 82 | total_timesteps 703.
Path 83 | total_timesteps 709.
Path 84 | total_timesteps 716.
Path 85 | total_timesteps 724.
Path 86 | total_timesteps 733.
Path 87 | total_timesteps 743.
Path 88 | total_timesteps 754.
Path 89 | total_timesteps 761.
Path 90 | total_timesteps 770.
Path 91 | total_timesteps 777.
Path 92 | total_timesteps 784.
Path 93 | total_timesteps 790.
Path 94 | total_timesteps 798.
Path 95 | total_timesteps 804.
Path 96 | total_timesteps 811.
Path 97 | total_timesteps 818.
Path 98 | total_timesteps 825.
Path 99 | total_timesteps 832.
Path 100 | total_timesteps 838.
Path 101 | total_timesteps 845.
Path 102 | total_timesteps 853.
Path 103 | total_timesteps 861.
Path 104 | total_timesteps 874.
Path 105 | total_timesteps 882.
Path 106 | total_timesteps 891.
Path 107 | total_timesteps 898.
Path 108 | total_timesteps 906.
Path 109 | total_timesteps 914.
Path 110 | total_timesteps 921.
Path 111 | total_timesteps 933.
Path 112 | total_timesteps 940.
Path 113 | total_timesteps 947.
Path 114 | total_timesteps 957.
Path 115 | total_timesteps 963.
Path 116 | total_timesteps 975.
Path 117 | total_timesteps 983.
Path 118 | total_timesteps 991.
Path 119 | total_timesteps 999.
Path 120 | total_timesteps 1007.
Path 121 | total_timesteps 1015.
Path 122 | total_timesteps 1021.
Path 123 | total_timesteps 1029.
Path 124 | total_timesteps 1038.
Path 125 | total_timesteps 1044.
Path 126 | total_timesteps 1051.
Path 127 | total_timesteps 1058.
Path 128 | total_timesteps 1065.
Path 129 | total_timesteps 1073.
Path 130 | total_timesteps 1080.
Path 131 | total_timesteps 1087.
Path 132 | total_timesteps 1094.
Path 133 | total_timesteps 1103.
Path 134 | total_timesteps 1112.
Path 135 | total_timesteps 1118.
Path 136 | total_timesteps 1126.
Path 137 | total_timesteps 1133.
Path 138 | total_timesteps 1140.
Path 139 | total_timesteps 1147.
Path 140 | total_timesteps 1154.
Path 141 | total_timesteps 1164.
Path 142 | total_timesteps 1173.
Path 143 | total_timesteps 1181.
Path 144 | total_timesteps 1189.
Path 145 | total_timesteps 1198.
Path 146 | total_timesteps 1204.
Path 147 | total_timesteps 1210.
Path 148 | total_timesteps 1220.
Path 149 | total_timesteps 1228.
Path 150 | total_timesteps 1238.
Path 151 | total_timesteps 1245.
Path 152 | total_timesteps 1254.
Path 153 | total_timesteps 1261.
Path 154 | total_timesteps 1272.
Path 155 | total_timesteps 1279.
Path 156 | total_timesteps 1287.
Path 157 | total_timesteps 1299.
Path 158 | total_timesteps 1310.
Path 159 | total_timesteps 1317.
Path 160 | total_timesteps 1325.
Path 161 | total_timesteps 1333.
Path 162 | total_timesteps 1342.
Path 163 | total_timesteps 1354.
Path 164 | total_timesteps 1364.
Path 165 | total_timesteps 1371.
Path 166 | total_timesteps 1379.
Path 167 | total_timesteps 1388.
Path 168 | total_timesteps 1396.
Path 169 | total_timesteps 1404.
Path 170 | total_timesteps 1413.
Path 171 | total_timesteps 1420.
Path 172 | total_timesteps 1428.
Path 173 | total_timesteps 1439.
Path 174 | total_timesteps 1449.
Path 175 | total_timesteps 1459.
Path 176 | total_timesteps 1468.
Path 177 | total_timesteps 1477.
Path 178 | total_timesteps 1485.
Path 179 | total_timesteps 1492.
Path 180 | total_timesteps 1502.
Path 181 | total_timesteps 1512.
Path 182 | total_timesteps 1521.
Path 183 | total_timesteps 1532.
Path 184 | total_timesteps 1539.
Path 185 | total_timesteps 1548.
Path 186 | total_timesteps 1556.
Path 187 | total_timesteps 1567.
Path 188 | total_timesteps 1573.
Path 189 | total_timesteps 1583.
Path 190 | total_timesteps 1591.
Path 191 | total_timesteps 1598.
Path 192 | total_timesteps 1604.
Path 193 | total_timesteps 1617.
Path 194 | total_timesteps 1627.
Path 195 | total_timesteps 1637.
Path 196 | total_timesteps 1644.
Path 197 | total_timesteps 1651.
Path 198 | total_timesteps 1659.
Path 199 | total_timesteps 1666.
Path 200 | total_timesteps 1676.
Path 201 | total_timesteps 1683.
Path 202 | total_timesteps 1691.
Path 203 | total_timesteps 1699.
Path 204 | total_timesteps 1706.
Path 205 | total_timesteps 1713.
Path 206 | total_timesteps 1720.
Path 207 | total_timesteps 1727.
Path 208 | total_timesteps 1733.
Path 209 | total_timesteps 1739.
Path 210 | total_timesteps 1747.
Path 211 | total_timesteps 1754.
Path 212 | total_timesteps 1761.
Path 213 | total_timesteps 1769.
Path 214 | total_timesteps 1777.
Path 215 | total_timesteps 1784.
Path 216 | total_timesteps 1792.
Path 217 | total_timesteps 1800.
Path 218 | total_timesteps 1807.
Path 219 | total_timesteps 1815.
Path 220 | total_timesteps 1822.
Path 221 | total_timesteps 1828.
Path 222 | total_timesteps 1835.
Path 223 | total_timesteps 1842.
Path 224 | total_timesteps 1849.
Path 225 | total_timesteps 1856.
Path 226 | total_timesteps 1863.
Path 227 | total_timesteps 1870.
Path 228 | total_timesteps 1876.
Path 229 | total_timesteps 1882.
Path 230 | total_timesteps 1891.
Path 231 | total_timesteps 1900.
Path 232 | total_timesteps 1907.
Path 233 | total_timesteps 1914.
Path 234 | total_timesteps 1922.
Path 235 | total_timesteps 1931.
Path 236 | total_timesteps 1941.
Path 237 | total_timesteps 1952.
Path 238 | total_timesteps 1958.
Path 239 | total_timesteps 1965.
Path 240 | total_timesteps 1976.
Path 241 | total_timesteps 1983.
Path 242 | total_timesteps 1991.
Path 243 | total_timesteps 2003.
Path 244 | total_timesteps 2009.
Path 245 | total_timesteps 2019.
Path 246 | total_timesteps 2030.
Path 247 | total_timesteps 2037.
Path 248 | total_timesteps 2045.
Path 249 | total_timesteps 2052.
Path 250 | total_timesteps 2064.
Path 251 | total_timesteps 2072.
Path 252 | total_timesteps 2080.
Path 253 | total_timesteps 2092.
Path 254 | total_timesteps 2100.
Path 255 | total_timesteps 2109.
Path 256 | total_timesteps 2118.
Path 257 | total_timesteps 2127.
Path 258 | total_timesteps 2134.
Path 259 | total_timesteps 2142.
Path 260 | total_timesteps 2149.
Path 261 | total_timesteps 2155.
Path 262 | total_timesteps 2163.
Path 263 | total_timesteps 2170.
Path 264 | total_timesteps 2176.
Path 265 | total_timesteps 2183.
Path 266 | total_timesteps 2192.
Path 267 | total_timesteps 2201.
Path 268 | total_timesteps 2208.
Path 269 | total_timesteps 2217.
Path 270 | total_timesteps 2225.
Path 271 | total_timesteps 2234.
Path 272 | total_timesteps 2243.
Path 273 | total_timesteps 2250.
Path 274 | total_timesteps 2257.
Path 275 | total_timesteps 2264.
Path 276 | total_timesteps 2271.
Path 277 | total_timesteps 2277.
Path 278 | total_timesteps 2285.
Path 279 | total_timesteps 2295.
Path 280 | total_timesteps 2303.
Path 281 | total_timesteps 2311.
Path 282 | total_timesteps 2322.
Path 283 | total_timesteps 2331.
Path 284 | total_timesteps 2341.
Path 285 | total_timesteps 2348.
Path 286 | total_timesteps 2360.
Path 287 | total_timesteps 2369.
Path 288 | total_timesteps 2376.
Path 289 | total_timesteps 2387.
Path 290 | total_timesteps 2396.
Path 291 | total_timesteps 2403.
Path 292 | total_timesteps 2412.
Path 293 | total_timesteps 2421.
Path 294 | total_timesteps 2431.
Path 295 | total_timesteps 2437.
Path 296 | total_timesteps 2443.
Path 297 | total_timesteps 2453.
Path 298 | total_timesteps 2459.
Path 299 | total_timesteps 2472.
Path 300 | total_timesteps 2478.
Path 301 | total_timesteps 2485.
Path 302 | total_timesteps 2496.
Path 303 | total_timesteps 2507.
Path 304 | total_timesteps 2514.
Path 305 | total_timesteps 2521.
Path 306 | total_timesteps 2528.
Path 307 | total_timesteps 2535.
Path 308 | total_timesteps 2542.
Path 309 | total_timesteps 2549.
Path 310 | total_timesteps 2557.
Path 311 | total_timesteps 2566.
Path 312 | total_timesteps 2573.
Path 313 | total_timesteps 2582.
Path 314 | total_timesteps 2590.
Path 315 | total_timesteps 2598.
Path 316 | total_timesteps 2608.
Path 317 | total_timesteps 2615.
Path 318 | total_timesteps 2624.
Path 319 | total_timesteps 2633.
Path 320 | total_timesteps 2642.
Path 321 | total_timesteps 2648.
Path 322 | total_timesteps 2660.
Path 323 | total_timesteps 2668.
Path 324 | total_timesteps 2676.
Path 325 | total_timesteps 2685.
Path 326 | total_timesteps 2693.
Path 327 | total_timesteps 2700.
Path 328 | total_timesteps 2707.
Path 329 | total_timesteps 2716.
Path 330 | total_timesteps 2722.
Path 331 | total_timesteps 2730.
Path 332 | total_timesteps 2737.
Path 333 | total_timesteps 2746.
Path 334 | total_timesteps 2753.
Path 335 | total_timesteps 2763.
Path 336 | total_timesteps 2770.
Path 337 | total_timesteps 2778.
Path 338 | total_timesteps 2788.
Path 339 | total_timesteps 2799.
Path 340 | total_timesteps 2810.
Path 341 | total_timesteps 2817.
Path 342 | total_timesteps 2823.
Path 343 | total_timesteps 2832.
Path 344 | total_timesteps 2840.
Path 345 | total_timesteps 2848.
Path 346 | total_timesteps 2856.
Path 347 | total_timesteps 2865.
Path 348 | total_timesteps 2872.
Path 349 | total_timesteps 2878.
Path 350 | total_timesteps 2884.
Path 351 | total_timesteps 2891.
Path 352 | total_timesteps 2897.
Path 353 | total_timesteps 2905.
Path 354 | total_timesteps 2912.
Path 355 | total_timesteps 2919.
Path 356 | total_timesteps 2927.
Path 357 | total_timesteps 2934.
Path 358 | total_timesteps 2943.
Path 359 | total_timesteps 2953.
Path 360 | total_timesteps 2961.
Path 361 | total_timesteps 2967.
Path 362 | total_timesteps 2974.
Path 363 | total_timesteps 2981.
Path 364 | total_timesteps 2988.
Path 365 | total_timesteps 2996.
Path 366 | total_timesteps 3003.
Path 367 | total_timesteps 3017.
Path 368 | total_timesteps 3024.
Path 369 | total_timesteps 3032.
Path 370 | total_timesteps 3040.
Path 371 | total_timesteps 3048.
Path 372 | total_timesteps 3055.
Path 373 | total_timesteps 3066.
Path 374 | total_timesteps 3073.
Path 375 | total_timesteps 3081.
Path 376 | total_timesteps 3092.
Path 377 | total_timesteps 3098.
Path 378 | total_timesteps 3106.
Path 379 | total_timesteps 3114.
Path 380 | total_timesteps 3121.
Path 381 | total_timesteps 3127.
Path 382 | total_timesteps 3134.
Path 383 | total_timesteps 3142.
Path 384 | total_timesteps 3149.
Path 385 | total_timesteps 3156.
Path 386 | total_timesteps 3162.
Path 387 | total_timesteps 3171.
Path 388 | total_timesteps 3179.
Path 389 | total_timesteps 3186.
Path 390 | total_timesteps 3198.
Path 391 | total_timesteps 3206.
Path 392 | total_timesteps 3214.
Path 393 | total_timesteps 3220.
Path 394 | total_timesteps 3229.
Path 395 | total_timesteps 3235.
Path 396 | total_timesteps 3243.
Path 397 | total_timesteps 3253.
Path 398 | total_timesteps 3260.
Path 399 | total_timesteps 3273.
Path 400 | total_timesteps 3280.
Path 401 | total_timesteps 3287.
Path 402 | total_timesteps 3296.
Path 403 | total_timesteps 3302.
Path 404 | total_timesteps 3309.
Path 405 | total_timesteps 3316.
Path 406 | total_timesteps 3325.
Path 407 | total_timesteps 3334.
Path 408 | total_timesteps 3346.
Path 409 | total_timesteps 3353.
Path 410 | total_timesteps 3360.
Path 411 | total_timesteps 3372.
Path 412 | total_timesteps 3380.
Path 413 | total_timesteps 3390.
Path 414 | total_timesteps 3399.
Path 415 | total_timesteps 3407.
Path 416 | total_timesteps 3415.
Path 417 | total_timesteps 3425.
Path 418 | total_timesteps 3433.
Path 419 | total_timesteps 3439.
Path 420 | total_timesteps 3447.
Path 421 | total_timesteps 3457.
Path 422 | total_timesteps 3466.
Path 423 | total_timesteps 3474.
Path 424 | total_timesteps 3484.
Path 425 | total_timesteps 3491.
Path 426 | total_timesteps 3501.
Path 427 | total_timesteps 3508.
Path 428 | total_timesteps 3515.
Path 429 | total_timesteps 3521.
Path 430 | total_timesteps 3529.
Path 431 | total_timesteps 3539.
Path 432 | total_timesteps 3546.
Path 433 | total_timesteps 3556.
Path 434 | total_timesteps 3564.
Path 435 | total_timesteps 3573.
Path 436 | total_timesteps 3580.
Path 437 | total_timesteps 3586.
Path 438 | total_timesteps 3594.
Path 439 | total_timesteps 3601.
Path 440 | total_timesteps 3608.
Path 441 | total_timesteps 3616.
Path 442 | total_timesteps 3625.
Path 443 | total_timesteps 3635.
Path 444 | total_timesteps 3641.
Path 445 | total_timesteps 3648.
Path 446 | total_timesteps 3656.
Path 447 | total_timesteps 3667.
Path 448 | total_timesteps 3674.
Path 449 | total_timesteps 3681.
Path 450 | total_timesteps 3687.
Path 451 | total_timesteps 3694.
Path 452 | total_timesteps 3701.
Path 453 | total_timesteps 3717.
Path 454 | total_timesteps 3726.
Path 455 | total_timesteps 3738.
Path 456 | total_timesteps 3748.
Path 457 | total_timesteps 3755.
Path 458 | total_timesteps 3762.
Path 459 | total_timesteps 3773.
Path 460 | total_timesteps 3781.
Path 461 | total_timesteps 3788.
Path 462 | total_timesteps 3798.
Path 463 | total_timesteps 3805.
Path 464 | total_timesteps 3811.
Path 465 | total_timesteps 3818.
Path 466 | total_timesteps 3825.
Path 467 | total_timesteps 3831.
Path 468 | total_timesteps 3839.
Path 469 | total_timesteps 3847.
Path 470 | total_timesteps 3855.
Path 471 | total_timesteps 3864.
Path 472 | total_timesteps 3872.
Path 473 | total_timesteps 3879.
Path 474 | total_timesteps 3887.
Path 475 | total_timesteps 3895.
Path 476 | total_timesteps 3902.
Path 477 | total_timesteps 3915.
Path 478 | total_timesteps 3923.
Path 479 | total_timesteps 3933.
Path 480 | total_timesteps 3942.
Path 481 | total_timesteps 3950.
Path 482 | total_timesteps 3959.
Path 483 | total_timesteps 3965.
Path 484 | total_timesteps 3976.
Path 485 | total_timesteps 3982.
Path 486 | total_timesteps 3989.
Path 487 | total_timesteps 3998.
Path 488 | total_timesteps 4014.
Path 489 | total_timesteps 4024.
Path 490 | total_timesteps 4034.
Path 491 | total_timesteps 4042.
Path 492 | total_timesteps 4049.
Path 493 | total_timesteps 4061.
Path 494 | total_timesteps 4068.
Path 495 | total_timesteps 4076.
Path 496 | total_timesteps 4084.
Path 497 | total_timesteps 4092.
Path 498 | total_timesteps 4107.
Path 499 | total_timesteps 4120.
Path 500 | total_timesteps 4128.
Path 501 | total_timesteps 4135.
Path 502 | total_timesteps 4143.
Path 503 | total_timesteps 4149.
Path 504 | total_timesteps 4157.
Path 505 | total_timesteps 4168.
Path 506 | total_timesteps 4174.
Path 507 | total_timesteps 4181.
Path 508 | total_timesteps 4189.
Path 509 | total_timesteps 4195.
Path 510 | total_timesteps 4201.
Path 511 | total_timesteps 4209.
Path 512 | total_timesteps 4218.
Path 513 | total_timesteps 4224.
Path 514 | total_timesteps 4233.
Path 515 | total_timesteps 4242.
Path 516 | total_timesteps 4249.
Path 517 | total_timesteps 4259.
Path 518 | total_timesteps 4272.
Path 519 | total_timesteps 4280.
Path 520 | total_timesteps 4288.
Path 521 | total_timesteps 4296.
Path 522 | total_timesteps 4306.
Path 523 | total_timesteps 4318.
Path 524 | total_timesteps 4324.
Path 525 | total_timesteps 4333.
Path 526 | total_timesteps 4340.
Path 527 | total_timesteps 4347.
Path 528 | total_timesteps 4354.
Path 529 | total_timesteps 4361.
Path 530 | total_timesteps 4371.
Path 531 | total_timesteps 4381.
Path 532 | total_timesteps 4389.
Path 533 | total_timesteps 4396.
Path 534 | total_timesteps 4403.
Path 535 | total_timesteps 4411.
Path 536 | total_timesteps 4421.
Path 537 | total_timesteps 4435.
Path 538 | total_timesteps 4442.
Path 539 | total_timesteps 4449.
Path 540 | total_timesteps 4456.
Path 541 | total_timesteps 4465.
Path 542 | total_timesteps 4474.
Path 543 | total_timesteps 4481.
Path 544 | total_timesteps 4489.
Path 545 | total_timesteps 4496.
Path 546 | total_timesteps 4509.
Path 547 | total_timesteps 4519.
Path 548 | total_timesteps 4530.
Path 549 | total_timesteps 4536.
Path 550 | total_timesteps 4547.
Path 551 | total_timesteps 4557.
Path 552 | total_timesteps 4568.
Path 553 | total_timesteps 4575.
Path 554 | total_timesteps 4586.
Path 555 | total_timesteps 4593.
Path 556 | total_timesteps 4601.
Path 557 | total_timesteps 4610.
Path 558 | total_timesteps 4617.
Path 559 | total_timesteps 4627.
Path 560 | total_timesteps 4635.
Path 561 | total_timesteps 4646.
Path 562 | total_timesteps 4653.
Path 563 | total_timesteps 4660.
Path 564 | total_timesteps 4671.
Path 565 | total_timesteps 4677.
Path 566 | total_timesteps 4684.
Path 567 | total_timesteps 4692.
Path 568 | total_timesteps 4700.
Path 569 | total_timesteps 4706.
Path 570 | total_timesteps 4713.
Path 571 | total_timesteps 4731.
Path 572 | total_timesteps 4741.
Path 573 | total_timesteps 4748.
Path 574 | total_timesteps 4756.
Path 575 | total_timesteps 4766.
Path 576 | total_timesteps 4776.
Path 577 | total_timesteps 4782.
Path 578 | total_timesteps 4795.
Path 579 | total_timesteps 4803.
Path 580 | total_timesteps 4811.
Path 581 | total_timesteps 4820.
Path 582 | total_timesteps 4828.
Path 583 | total_timesteps 4840.
Path 584 | total_timesteps 4850.
Path 585 | total_timesteps 4860.
Path 586 | total_timesteps 4867.
Path 587 | total_timesteps 4874.
Path 588 | total_timesteps 4884.
Path 589 | total_timesteps 4892.
Path 590 | total_timesteps 4902.
Path 591 | total_timesteps 4912.
Path 592 | total_timesteps 4921.
Path 593 | total_timesteps 4929.
Path 594 | total_timesteps 4935.
Path 595 | total_timesteps 4947.
Path 596 | total_timesteps 4955.
Path 597 | total_timesteps 4965.
Path 598 | total_timesteps 4976.
Path 599 | total_timesteps 4985.
Path 600 | total_timesteps 4992.
Path 601 | total_timesteps 5000.
Path 602 | total_timesteps 5007.
Path 603 | total_timesteps 5016.
Path 604 | total_timesteps 5022.
Path 605 | total_timesteps 5029.
Path 606 | total_timesteps 5039.
Path 607 | total_timesteps 5047.
Path 608 | total_timesteps 5063.
Path 609 | total_timesteps 5070.
Path 610 | total_timesteps 5076.
Path 611 | total_timesteps 5083.
Path 612 | total_timesteps 5090.
Path 613 | total_timesteps 5096.
Path 614 | total_timesteps 5103.
Path 615 | total_timesteps 5110.
Path 616 | total_timesteps 5121.
Path 617 | total_timesteps 5136.
Path 618 | total_timesteps 5143.
Path 619 | total_timesteps 5149.
Path 620 | total_timesteps 5159.
Path 621 | total_timesteps 5166.
Path 622 | total_timesteps 5174.
Path 623 | total_timesteps 5183.
Path 624 | total_timesteps 5197.
Path 625 | total_timesteps 5203.
Path 626 | total_timesteps 5211.
Path 627 | total_timesteps 5218.
Path 628 | total_timesteps 5226.
Path 629 | total_timesteps 5234.
Path 630 | total_timesteps 5242.
Path 631 | total_timesteps 5250.
Path 632 | total_timesteps 5259.
Path 633 | total_timesteps 5268.
Path 634 | total_timesteps 5276.
Path 635 | total_timesteps 5288.
Path 636 | total_timesteps 5294.
Path 637 | total_timesteps 5300.
Path 638 | total_timesteps 5306.
Path 639 | total_timesteps 5314.
Path 640 | total_timesteps 5321.
Path 641 | total_timesteps 5329.
Path 642 | total_timesteps 5338.
Path 643 | total_timesteps 5347.
Path 644 | total_timesteps 5354.
Path 645 | total_timesteps 5363.
Path 646 | total_timesteps 5369.
Path 647 | total_timesteps 5378.
Path 648 | total_timesteps 5386.
Path 649 | total_timesteps 5395.
Path 650 | total_timesteps 5403.
Path 651 | total_timesteps 5412.
Path 652 | total_timesteps 5422.
Path 653 | total_timesteps 5430.
Path 654 | total_timesteps 5436.
Path 655 | total_timesteps 5445.
Path 656 | total_timesteps 5454.
Path 657 | total_timesteps 5465.
Path 658 | total_timesteps 5474.
Path 659 | total_timesteps 5483.
Path 660 | total_timesteps 5490.
Path 661 | total_timesteps 5497.
Path 662 | total_timesteps 5503.
Path 663 | total_timesteps 5511.
Path 664 | total_timesteps 5524.
Path 665 | total_timesteps 5530.
Path 666 | total_timesteps 5538.
Path 667 | total_timesteps 5545.
Path 668 | total_timesteps 5553.
Path 669 | total_timesteps 5562.
Path 670 | total_timesteps 5571.
Path 671 | total_timesteps 5582.
Path 672 | total_timesteps 5591.
Path 673 | total_timesteps 5599.
Path 674 | total_timesteps 5607.
Path 675 | total_timesteps 5618.
Path 676 | total_timesteps 5626.
Path 677 | total_timesteps 5633.
Path 678 | total_timesteps 5640.
Path 679 | total_timesteps 5648.
Path 680 | total_timesteps 5655.
Path 681 | total_timesteps 5663.
Path 682 | total_timesteps 5674.
Path 683 | total_timesteps 5686.
Path 684 | total_timesteps 5693.
Path 685 | total_timesteps 5700.
Path 686 | total_timesteps 5711.
Path 687 | total_timesteps 5718.
Path 688 | total_timesteps 5724.
Path 689 | total_timesteps 5732.
Path 690 | total_timesteps 5741.
Path 691 | total_timesteps 5748.
Path 692 | total_timesteps 5754.
Path 693 | total_timesteps 5762.
Path 694 | total_timesteps 5771.
Path 695 | total_timesteps 5778.
Path 696 | total_timesteps 5787.
Path 697 | total_timesteps 5794.
Path 698 | total_timesteps 5800.
Path 699 | total_timesteps 5810.
Path 700 | total_timesteps 5820.
Path 701 | total_timesteps 5832.
Path 702 | total_timesteps 5839.
Path 703 | total_timesteps 5848.
Path 704 | total_timesteps 5855.
Path 705 | total_timesteps 5868.
Path 706 | total_timesteps 5877.
Path 707 | total_timesteps 5885.
Path 708 | total_timesteps 5891.
Path 709 | total_timesteps 5899.
Path 710 | total_timesteps 5907.
Path 711 | total_timesteps 5915.
Path 712 | total_timesteps 5924.
Path 713 | total_timesteps 5931.
Path 714 | total_timesteps 5938.
Path 715 | total_timesteps 5948.
Path 716 | total_timesteps 5955.
Path 717 | total_timesteps 5963.
Path 718 | total_timesteps 5973.
Path 719 | total_timesteps 5980.
Path 720 | total_timesteps 5989.
Path 721 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.8     |
| Iteration     | 30       |
| MaximumReturn | -0.662   |
| MinimumReturn | -17.9    |
| TotalSamples  | 128111   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.0012806979939341545
Validation loss = 0.001312923850491643
Validation loss = 0.0013617699733003974
Validation loss = 0.0012445863103494048
Validation loss = 0.0013150281738489866
Validation loss = 0.0016019250033423305
Validation loss = 0.0017211189260706306
Validation loss = 0.0012989750830456614
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 23.
Path 4 | total_timesteps 31.
Path 5 | total_timesteps 37.
Path 6 | total_timesteps 43.
Path 7 | total_timesteps 53.
Path 8 | total_timesteps 60.
Path 9 | total_timesteps 67.
Path 10 | total_timesteps 74.
Path 11 | total_timesteps 84.
Path 12 | total_timesteps 92.
Path 13 | total_timesteps 101.
Path 14 | total_timesteps 109.
Path 15 | total_timesteps 116.
Path 16 | total_timesteps 124.
Path 17 | total_timesteps 131.
Path 18 | total_timesteps 141.
Path 19 | total_timesteps 148.
Path 20 | total_timesteps 157.
Path 21 | total_timesteps 165.
Path 22 | total_timesteps 171.
Path 23 | total_timesteps 179.
Path 24 | total_timesteps 185.
Path 25 | total_timesteps 193.
Path 26 | total_timesteps 201.
Path 27 | total_timesteps 208.
Path 28 | total_timesteps 216.
Path 29 | total_timesteps 223.
Path 30 | total_timesteps 231.
Path 31 | total_timesteps 239.
Path 32 | total_timesteps 246.
Path 33 | total_timesteps 253.
Path 34 | total_timesteps 263.
Path 35 | total_timesteps 272.
Path 36 | total_timesteps 279.
Path 37 | total_timesteps 285.
Path 38 | total_timesteps 294.
Path 39 | total_timesteps 303.
Path 40 | total_timesteps 311.
Path 41 | total_timesteps 317.
Path 42 | total_timesteps 325.
Path 43 | total_timesteps 332.
Path 44 | total_timesteps 340.
Path 45 | total_timesteps 347.
Path 46 | total_timesteps 358.
Path 47 | total_timesteps 366.
Path 48 | total_timesteps 374.
Path 49 | total_timesteps 380.
Path 50 | total_timesteps 388.
Path 51 | total_timesteps 396.
Path 52 | total_timesteps 403.
Path 53 | total_timesteps 411.
Path 54 | total_timesteps 418.
Path 55 | total_timesteps 428.
Path 56 | total_timesteps 435.
Path 57 | total_timesteps 445.
Path 58 | total_timesteps 452.
Path 59 | total_timesteps 459.
Path 60 | total_timesteps 467.
Path 61 | total_timesteps 473.
Path 62 | total_timesteps 480.
Path 63 | total_timesteps 489.
Path 64 | total_timesteps 498.
Path 65 | total_timesteps 506.
Path 66 | total_timesteps 514.
Path 67 | total_timesteps 521.
Path 68 | total_timesteps 530.
Path 69 | total_timesteps 539.
Path 70 | total_timesteps 546.
Path 71 | total_timesteps 553.
Path 72 | total_timesteps 559.
Path 73 | total_timesteps 568.
Path 74 | total_timesteps 578.
Path 75 | total_timesteps 589.
Path 76 | total_timesteps 597.
Path 77 | total_timesteps 605.
Path 78 | total_timesteps 613.
Path 79 | total_timesteps 624.
Path 80 | total_timesteps 631.
Path 81 | total_timesteps 638.
Path 82 | total_timesteps 644.
Path 83 | total_timesteps 652.
Path 84 | total_timesteps 658.
Path 85 | total_timesteps 666.
Path 86 | total_timesteps 674.
Path 87 | total_timesteps 682.
Path 88 | total_timesteps 690.
Path 89 | total_timesteps 696.
Path 90 | total_timesteps 705.
Path 91 | total_timesteps 711.
Path 92 | total_timesteps 717.
Path 93 | total_timesteps 726.
Path 94 | total_timesteps 734.
Path 95 | total_timesteps 740.
Path 96 | total_timesteps 747.
Path 97 | total_timesteps 756.
Path 98 | total_timesteps 763.
Path 99 | total_timesteps 770.
Path 100 | total_timesteps 780.
Path 101 | total_timesteps 787.
Path 102 | total_timesteps 795.
Path 103 | total_timesteps 805.
Path 104 | total_timesteps 812.
Path 105 | total_timesteps 820.
Path 106 | total_timesteps 826.
Path 107 | total_timesteps 834.
Path 108 | total_timesteps 840.
Path 109 | total_timesteps 848.
Path 110 | total_timesteps 855.
Path 111 | total_timesteps 861.
Path 112 | total_timesteps 867.
Path 113 | total_timesteps 874.
Path 114 | total_timesteps 882.
Path 115 | total_timesteps 891.
Path 116 | total_timesteps 900.
Path 117 | total_timesteps 909.
Path 118 | total_timesteps 917.
Path 119 | total_timesteps 924.
Path 120 | total_timesteps 934.
Path 121 | total_timesteps 941.
Path 122 | total_timesteps 947.
Path 123 | total_timesteps 958.
Path 124 | total_timesteps 966.
Path 125 | total_timesteps 973.
Path 126 | total_timesteps 980.
Path 127 | total_timesteps 989.
Path 128 | total_timesteps 997.
Path 129 | total_timesteps 1005.
Path 130 | total_timesteps 1017.
Path 131 | total_timesteps 1025.
Path 132 | total_timesteps 1036.
Path 133 | total_timesteps 1046.
Path 134 | total_timesteps 1053.
Path 135 | total_timesteps 1059.
Path 136 | total_timesteps 1069.
Path 137 | total_timesteps 1078.
Path 138 | total_timesteps 1087.
Path 139 | total_timesteps 1093.
Path 140 | total_timesteps 1102.
Path 141 | total_timesteps 1110.
Path 142 | total_timesteps 1118.
Path 143 | total_timesteps 1128.
Path 144 | total_timesteps 1135.
Path 145 | total_timesteps 1146.
Path 146 | total_timesteps 1160.
Path 147 | total_timesteps 1167.
Path 148 | total_timesteps 1175.
Path 149 | total_timesteps 1184.
Path 150 | total_timesteps 1192.
Path 151 | total_timesteps 1198.
Path 152 | total_timesteps 1207.
Path 153 | total_timesteps 1216.
Path 154 | total_timesteps 1224.
Path 155 | total_timesteps 1231.
Path 156 | total_timesteps 1239.
Path 157 | total_timesteps 1247.
Path 158 | total_timesteps 1255.
Path 159 | total_timesteps 1266.
Path 160 | total_timesteps 1272.
Path 161 | total_timesteps 1281.
Path 162 | total_timesteps 1288.
Path 163 | total_timesteps 1294.
Path 164 | total_timesteps 1303.
Path 165 | total_timesteps 1312.
Path 166 | total_timesteps 1319.
Path 167 | total_timesteps 1327.
Path 168 | total_timesteps 1335.
Path 169 | total_timesteps 1344.
Path 170 | total_timesteps 1353.
Path 171 | total_timesteps 1361.
Path 172 | total_timesteps 1369.
Path 173 | total_timesteps 1377.
Path 174 | total_timesteps 1385.
Path 175 | total_timesteps 1393.
Path 176 | total_timesteps 1401.
Path 177 | total_timesteps 1408.
Path 178 | total_timesteps 1415.
Path 179 | total_timesteps 1422.
Path 180 | total_timesteps 1428.
Path 181 | total_timesteps 1435.
Path 182 | total_timesteps 1447.
Path 183 | total_timesteps 1456.
Path 184 | total_timesteps 1463.
Path 185 | total_timesteps 1471.
Path 186 | total_timesteps 1477.
Path 187 | total_timesteps 1484.
Path 188 | total_timesteps 1493.
Path 189 | total_timesteps 1501.
Path 190 | total_timesteps 1509.
Path 191 | total_timesteps 1517.
Path 192 | total_timesteps 1524.
Path 193 | total_timesteps 1530.
Path 194 | total_timesteps 1537.
Path 195 | total_timesteps 1546.
Path 196 | total_timesteps 1556.
Path 197 | total_timesteps 1564.
Path 198 | total_timesteps 1570.
Path 199 | total_timesteps 1578.
Path 200 | total_timesteps 1585.
Path 201 | total_timesteps 1593.
Path 202 | total_timesteps 1602.
Path 203 | total_timesteps 1610.
Path 204 | total_timesteps 1617.
Path 205 | total_timesteps 1623.
Path 206 | total_timesteps 1630.
Path 207 | total_timesteps 1638.
Path 208 | total_timesteps 1646.
Path 209 | total_timesteps 1658.
Path 210 | total_timesteps 1665.
Path 211 | total_timesteps 1672.
Path 212 | total_timesteps 1681.
Path 213 | total_timesteps 1689.
Path 214 | total_timesteps 1699.
Path 215 | total_timesteps 1707.
Path 216 | total_timesteps 1719.
Path 217 | total_timesteps 1726.
Path 218 | total_timesteps 1732.
Path 219 | total_timesteps 1742.
Path 220 | total_timesteps 1748.
Path 221 | total_timesteps 1754.
Path 222 | total_timesteps 1762.
Path 223 | total_timesteps 1769.
Path 224 | total_timesteps 1779.
Path 225 | total_timesteps 1785.
Path 226 | total_timesteps 1794.
Path 227 | total_timesteps 1803.
Path 228 | total_timesteps 1810.
Path 229 | total_timesteps 1819.
Path 230 | total_timesteps 1826.
Path 231 | total_timesteps 1833.
Path 232 | total_timesteps 1841.
Path 233 | total_timesteps 1849.
Path 234 | total_timesteps 1859.
Path 235 | total_timesteps 1869.
Path 236 | total_timesteps 1875.
Path 237 | total_timesteps 1884.
Path 238 | total_timesteps 1891.
Path 239 | total_timesteps 1899.
Path 240 | total_timesteps 1915.
Path 241 | total_timesteps 1926.
Path 242 | total_timesteps 1932.
Path 243 | total_timesteps 1940.
Path 244 | total_timesteps 1948.
Path 245 | total_timesteps 1956.
Path 246 | total_timesteps 1964.
Path 247 | total_timesteps 1970.
Path 248 | total_timesteps 1978.
Path 249 | total_timesteps 1991.
Path 250 | total_timesteps 2001.
Path 251 | total_timesteps 2011.
Path 252 | total_timesteps 2019.
Path 253 | total_timesteps 2030.
Path 254 | total_timesteps 2038.
Path 255 | total_timesteps 2046.
Path 256 | total_timesteps 2057.
Path 257 | total_timesteps 2065.
Path 258 | total_timesteps 2072.
Path 259 | total_timesteps 2078.
Path 260 | total_timesteps 2087.
Path 261 | total_timesteps 2095.
Path 262 | total_timesteps 2102.
Path 263 | total_timesteps 2110.
Path 264 | total_timesteps 2117.
Path 265 | total_timesteps 2123.
Path 266 | total_timesteps 2131.
Path 267 | total_timesteps 2139.
Path 268 | total_timesteps 2146.
Path 269 | total_timesteps 2153.
Path 270 | total_timesteps 2160.
Path 271 | total_timesteps 2172.
Path 272 | total_timesteps 2180.
Path 273 | total_timesteps 2187.
Path 274 | total_timesteps 2194.
Path 275 | total_timesteps 2202.
Path 276 | total_timesteps 2210.
Path 277 | total_timesteps 2217.
Path 278 | total_timesteps 2223.
Path 279 | total_timesteps 2232.
Path 280 | total_timesteps 2240.
Path 281 | total_timesteps 2247.
Path 282 | total_timesteps 2255.
Path 283 | total_timesteps 2261.
Path 284 | total_timesteps 2269.
Path 285 | total_timesteps 2277.
Path 286 | total_timesteps 2284.
Path 287 | total_timesteps 2291.
Path 288 | total_timesteps 2298.
Path 289 | total_timesteps 2305.
Path 290 | total_timesteps 2318.
Path 291 | total_timesteps 2324.
Path 292 | total_timesteps 2335.
Path 293 | total_timesteps 2342.
Path 294 | total_timesteps 2349.
Path 295 | total_timesteps 2360.
Path 296 | total_timesteps 2368.
Path 297 | total_timesteps 2379.
Path 298 | total_timesteps 2386.
Path 299 | total_timesteps 2392.
Path 300 | total_timesteps 2399.
Path 301 | total_timesteps 2406.
Path 302 | total_timesteps 2413.
Path 303 | total_timesteps 2420.
Path 304 | total_timesteps 2431.
Path 305 | total_timesteps 2440.
Path 306 | total_timesteps 2446.
Path 307 | total_timesteps 2455.
Path 308 | total_timesteps 2466.
Path 309 | total_timesteps 2475.
Path 310 | total_timesteps 2482.
Path 311 | total_timesteps 2490.
Path 312 | total_timesteps 2497.
Path 313 | total_timesteps 2503.
Path 314 | total_timesteps 2515.
Path 315 | total_timesteps 2530.
Path 316 | total_timesteps 2541.
Path 317 | total_timesteps 2548.
Path 318 | total_timesteps 2557.
Path 319 | total_timesteps 2566.
Path 320 | total_timesteps 2573.
Path 321 | total_timesteps 2583.
Path 322 | total_timesteps 2592.
Path 323 | total_timesteps 2602.
Path 324 | total_timesteps 2609.
Path 325 | total_timesteps 2618.
Path 326 | total_timesteps 2626.
Path 327 | total_timesteps 2632.
Path 328 | total_timesteps 2639.
Path 329 | total_timesteps 2646.
Path 330 | total_timesteps 2652.
Path 331 | total_timesteps 2660.
Path 332 | total_timesteps 2670.
Path 333 | total_timesteps 2679.
Path 334 | total_timesteps 2686.
Path 335 | total_timesteps 2701.
Path 336 | total_timesteps 2708.
Path 337 | total_timesteps 2717.
Path 338 | total_timesteps 2726.
Path 339 | total_timesteps 2733.
Path 340 | total_timesteps 2742.
Path 341 | total_timesteps 2751.
Path 342 | total_timesteps 2758.
Path 343 | total_timesteps 2766.
Path 344 | total_timesteps 2774.
Path 345 | total_timesteps 2781.
Path 346 | total_timesteps 2795.
Path 347 | total_timesteps 2805.
Path 348 | total_timesteps 2813.
Path 349 | total_timesteps 2820.
Path 350 | total_timesteps 2828.
Path 351 | total_timesteps 2841.
Path 352 | total_timesteps 2849.
Path 353 | total_timesteps 2857.
Path 354 | total_timesteps 2864.
Path 355 | total_timesteps 2870.
Path 356 | total_timesteps 2876.
Path 357 | total_timesteps 2884.
Path 358 | total_timesteps 2893.
Path 359 | total_timesteps 2900.
Path 360 | total_timesteps 2906.
Path 361 | total_timesteps 2913.
Path 362 | total_timesteps 2921.
Path 363 | total_timesteps 2928.
Path 364 | total_timesteps 2935.
Path 365 | total_timesteps 2944.
Path 366 | total_timesteps 2952.
Path 367 | total_timesteps 2959.
Path 368 | total_timesteps 2966.
Path 369 | total_timesteps 2980.
Path 370 | total_timesteps 2986.
Path 371 | total_timesteps 2992.
Path 372 | total_timesteps 2998.
Path 373 | total_timesteps 3007.
Path 374 | total_timesteps 3014.
Path 375 | total_timesteps 3023.
Path 376 | total_timesteps 3030.
Path 377 | total_timesteps 3037.
Path 378 | total_timesteps 3047.
Path 379 | total_timesteps 3054.
Path 380 | total_timesteps 3065.
Path 381 | total_timesteps 3072.
Path 382 | total_timesteps 3078.
Path 383 | total_timesteps 3085.
Path 384 | total_timesteps 3093.
Path 385 | total_timesteps 3100.
Path 386 | total_timesteps 3108.
Path 387 | total_timesteps 3116.
Path 388 | total_timesteps 3123.
Path 389 | total_timesteps 3134.
Path 390 | total_timesteps 3144.
Path 391 | total_timesteps 3151.
Path 392 | total_timesteps 3159.
Path 393 | total_timesteps 3165.
Path 394 | total_timesteps 3173.
Path 395 | total_timesteps 3183.
Path 396 | total_timesteps 3193.
Path 397 | total_timesteps 3199.
Path 398 | total_timesteps 3206.
Path 399 | total_timesteps 3212.
Path 400 | total_timesteps 3219.
Path 401 | total_timesteps 3225.
Path 402 | total_timesteps 3233.
Path 403 | total_timesteps 3240.
Path 404 | total_timesteps 3249.
Path 405 | total_timesteps 3260.
Path 406 | total_timesteps 3267.
Path 407 | total_timesteps 3276.
Path 408 | total_timesteps 3283.
Path 409 | total_timesteps 3292.
Path 410 | total_timesteps 3305.
Path 411 | total_timesteps 3314.
Path 412 | total_timesteps 3322.
Path 413 | total_timesteps 3330.
Path 414 | total_timesteps 3341.
Path 415 | total_timesteps 3350.
Path 416 | total_timesteps 3357.
Path 417 | total_timesteps 3364.
Path 418 | total_timesteps 3371.
Path 419 | total_timesteps 3377.
Path 420 | total_timesteps 3383.
Path 421 | total_timesteps 3398.
Path 422 | total_timesteps 3404.
Path 423 | total_timesteps 3416.
Path 424 | total_timesteps 3424.
Path 425 | total_timesteps 3433.
Path 426 | total_timesteps 3442.
Path 427 | total_timesteps 3449.
Path 428 | total_timesteps 3456.
Path 429 | total_timesteps 3463.
Path 430 | total_timesteps 3471.
Path 431 | total_timesteps 3484.
Path 432 | total_timesteps 3490.
Path 433 | total_timesteps 3496.
Path 434 | total_timesteps 3503.
Path 435 | total_timesteps 3510.
Path 436 | total_timesteps 3517.
Path 437 | total_timesteps 3526.
Path 438 | total_timesteps 3535.
Path 439 | total_timesteps 3545.
Path 440 | total_timesteps 3555.
Path 441 | total_timesteps 3564.
Path 442 | total_timesteps 3571.
Path 443 | total_timesteps 3578.
Path 444 | total_timesteps 3586.
Path 445 | total_timesteps 3594.
Path 446 | total_timesteps 3602.
Path 447 | total_timesteps 3610.
Path 448 | total_timesteps 3617.
Path 449 | total_timesteps 3628.
Path 450 | total_timesteps 3635.
Path 451 | total_timesteps 3642.
Path 452 | total_timesteps 3654.
Path 453 | total_timesteps 3662.
Path 454 | total_timesteps 3672.
Path 455 | total_timesteps 3678.
Path 456 | total_timesteps 3684.
Path 457 | total_timesteps 3691.
Path 458 | total_timesteps 3699.
Path 459 | total_timesteps 3707.
Path 460 | total_timesteps 3718.
Path 461 | total_timesteps 3724.
Path 462 | total_timesteps 3733.
Path 463 | total_timesteps 3742.
Path 464 | total_timesteps 3752.
Path 465 | total_timesteps 3760.
Path 466 | total_timesteps 3768.
Path 467 | total_timesteps 3774.
Path 468 | total_timesteps 3782.
Path 469 | total_timesteps 3793.
Path 470 | total_timesteps 3802.
Path 471 | total_timesteps 3810.
Path 472 | total_timesteps 3819.
Path 473 | total_timesteps 3827.
Path 474 | total_timesteps 3835.
Path 475 | total_timesteps 3844.
Path 476 | total_timesteps 3852.
Path 477 | total_timesteps 3858.
Path 478 | total_timesteps 3866.
Path 479 | total_timesteps 3873.
Path 480 | total_timesteps 3885.
Path 481 | total_timesteps 3891.
Path 482 | total_timesteps 3898.
Path 483 | total_timesteps 3904.
Path 484 | total_timesteps 3913.
Path 485 | total_timesteps 3925.
Path 486 | total_timesteps 3933.
Path 487 | total_timesteps 3944.
Path 488 | total_timesteps 3951.
Path 489 | total_timesteps 3957.
Path 490 | total_timesteps 3964.
Path 491 | total_timesteps 3973.
Path 492 | total_timesteps 3981.
Path 493 | total_timesteps 3989.
Path 494 | total_timesteps 3998.
Path 495 | total_timesteps 4004.
Path 496 | total_timesteps 4010.
Path 497 | total_timesteps 4022.
Path 498 | total_timesteps 4031.
Path 499 | total_timesteps 4039.
Path 500 | total_timesteps 4046.
Path 501 | total_timesteps 4056.
Path 502 | total_timesteps 4063.
Path 503 | total_timesteps 4073.
Path 504 | total_timesteps 4081.
Path 505 | total_timesteps 4088.
Path 506 | total_timesteps 4097.
Path 507 | total_timesteps 4106.
Path 508 | total_timesteps 4113.
Path 509 | total_timesteps 4121.
Path 510 | total_timesteps 4129.
Path 511 | total_timesteps 4136.
Path 512 | total_timesteps 4143.
Path 513 | total_timesteps 4150.
Path 514 | total_timesteps 4156.
Path 515 | total_timesteps 4165.
Path 516 | total_timesteps 4175.
Path 517 | total_timesteps 4183.
Path 518 | total_timesteps 4190.
Path 519 | total_timesteps 4197.
Path 520 | total_timesteps 4206.
Path 521 | total_timesteps 4212.
Path 522 | total_timesteps 4220.
Path 523 | total_timesteps 4228.
Path 524 | total_timesteps 4236.
Path 525 | total_timesteps 4244.
Path 526 | total_timesteps 4251.
Path 527 | total_timesteps 4260.
Path 528 | total_timesteps 4269.
Path 529 | total_timesteps 4278.
Path 530 | total_timesteps 4285.
Path 531 | total_timesteps 4294.
Path 532 | total_timesteps 4303.
Path 533 | total_timesteps 4312.
Path 534 | total_timesteps 4320.
Path 535 | total_timesteps 4327.
Path 536 | total_timesteps 4334.
Path 537 | total_timesteps 4346.
Path 538 | total_timesteps 4354.
Path 539 | total_timesteps 4360.
Path 540 | total_timesteps 4366.
Path 541 | total_timesteps 4374.
Path 542 | total_timesteps 4382.
Path 543 | total_timesteps 4392.
Path 544 | total_timesteps 4401.
Path 545 | total_timesteps 4413.
Path 546 | total_timesteps 4422.
Path 547 | total_timesteps 4431.
Path 548 | total_timesteps 4438.
Path 549 | total_timesteps 4446.
Path 550 | total_timesteps 4454.
Path 551 | total_timesteps 4461.
Path 552 | total_timesteps 4469.
Path 553 | total_timesteps 4477.
Path 554 | total_timesteps 4484.
Path 555 | total_timesteps 4491.
Path 556 | total_timesteps 4500.
Path 557 | total_timesteps 4506.
Path 558 | total_timesteps 4516.
Path 559 | total_timesteps 4522.
Path 560 | total_timesteps 4532.
Path 561 | total_timesteps 4539.
Path 562 | total_timesteps 4550.
Path 563 | total_timesteps 4559.
Path 564 | total_timesteps 4569.
Path 565 | total_timesteps 4577.
Path 566 | total_timesteps 4585.
Path 567 | total_timesteps 4591.
Path 568 | total_timesteps 4599.
Path 569 | total_timesteps 4607.
Path 570 | total_timesteps 4614.
Path 571 | total_timesteps 4623.
Path 572 | total_timesteps 4632.
Path 573 | total_timesteps 4639.
Path 574 | total_timesteps 4646.
Path 575 | total_timesteps 4653.
Path 576 | total_timesteps 4661.
Path 577 | total_timesteps 4667.
Path 578 | total_timesteps 4678.
Path 579 | total_timesteps 4685.
Path 580 | total_timesteps 4692.
Path 581 | total_timesteps 4700.
Path 582 | total_timesteps 4706.
Path 583 | total_timesteps 4714.
Path 584 | total_timesteps 4723.
Path 585 | total_timesteps 4731.
Path 586 | total_timesteps 4739.
Path 587 | total_timesteps 4750.
Path 588 | total_timesteps 4759.
Path 589 | total_timesteps 4767.
Path 590 | total_timesteps 4774.
Path 591 | total_timesteps 4783.
Path 592 | total_timesteps 4789.
Path 593 | total_timesteps 4798.
Path 594 | total_timesteps 4805.
Path 595 | total_timesteps 4813.
Path 596 | total_timesteps 4820.
Path 597 | total_timesteps 4827.
Path 598 | total_timesteps 4835.
Path 599 | total_timesteps 4841.
Path 600 | total_timesteps 4847.
Path 601 | total_timesteps 4854.
Path 602 | total_timesteps 4861.
Path 603 | total_timesteps 4870.
Path 604 | total_timesteps 4877.
Path 605 | total_timesteps 4885.
Path 606 | total_timesteps 4892.
Path 607 | total_timesteps 4899.
Path 608 | total_timesteps 4909.
Path 609 | total_timesteps 4916.
Path 610 | total_timesteps 4923.
Path 611 | total_timesteps 4931.
Path 612 | total_timesteps 4937.
Path 613 | total_timesteps 4948.
Path 614 | total_timesteps 4959.
Path 615 | total_timesteps 4966.
Path 616 | total_timesteps 4972.
Path 617 | total_timesteps 4979.
Path 618 | total_timesteps 4986.
Path 619 | total_timesteps 4992.
Path 620 | total_timesteps 5000.
Path 621 | total_timesteps 5008.
Path 622 | total_timesteps 5016.
Path 623 | total_timesteps 5023.
Path 624 | total_timesteps 5033.
Path 625 | total_timesteps 5044.
Path 626 | total_timesteps 5053.
Path 627 | total_timesteps 5061.
Path 628 | total_timesteps 5069.
Path 629 | total_timesteps 5076.
Path 630 | total_timesteps 5084.
Path 631 | total_timesteps 5090.
Path 632 | total_timesteps 5100.
Path 633 | total_timesteps 5107.
Path 634 | total_timesteps 5115.
Path 635 | total_timesteps 5125.
Path 636 | total_timesteps 5135.
Path 637 | total_timesteps 5142.
Path 638 | total_timesteps 5150.
Path 639 | total_timesteps 5157.
Path 640 | total_timesteps 5165.
Path 641 | total_timesteps 5173.
Path 642 | total_timesteps 5182.
Path 643 | total_timesteps 5188.
Path 644 | total_timesteps 5194.
Path 645 | total_timesteps 5201.
Path 646 | total_timesteps 5211.
Path 647 | total_timesteps 5218.
Path 648 | total_timesteps 5227.
Path 649 | total_timesteps 5236.
Path 650 | total_timesteps 5243.
Path 651 | total_timesteps 5252.
Path 652 | total_timesteps 5259.
Path 653 | total_timesteps 5271.
Path 654 | total_timesteps 5279.
Path 655 | total_timesteps 5286.
Path 656 | total_timesteps 5298.
Path 657 | total_timesteps 5308.
Path 658 | total_timesteps 5316.
Path 659 | total_timesteps 5323.
Path 660 | total_timesteps 5331.
Path 661 | total_timesteps 5342.
Path 662 | total_timesteps 5349.
Path 663 | total_timesteps 5356.
Path 664 | total_timesteps 5364.
Path 665 | total_timesteps 5371.
Path 666 | total_timesteps 5383.
Path 667 | total_timesteps 5390.
Path 668 | total_timesteps 5396.
Path 669 | total_timesteps 5405.
Path 670 | total_timesteps 5413.
Path 671 | total_timesteps 5423.
Path 672 | total_timesteps 5430.
Path 673 | total_timesteps 5438.
Path 674 | total_timesteps 5449.
Path 675 | total_timesteps 5457.
Path 676 | total_timesteps 5465.
Path 677 | total_timesteps 5474.
Path 678 | total_timesteps 5481.
Path 679 | total_timesteps 5488.
Path 680 | total_timesteps 5494.
Path 681 | total_timesteps 5502.
Path 682 | total_timesteps 5508.
Path 683 | total_timesteps 5515.
Path 684 | total_timesteps 5526.
Path 685 | total_timesteps 5533.
Path 686 | total_timesteps 5545.
Path 687 | total_timesteps 5557.
Path 688 | total_timesteps 5565.
Path 689 | total_timesteps 5573.
Path 690 | total_timesteps 5582.
Path 691 | total_timesteps 5590.
Path 692 | total_timesteps 5597.
Path 693 | total_timesteps 5604.
Path 694 | total_timesteps 5610.
Path 695 | total_timesteps 5616.
Path 696 | total_timesteps 5626.
Path 697 | total_timesteps 5635.
Path 698 | total_timesteps 5644.
Path 699 | total_timesteps 5651.
Path 700 | total_timesteps 5664.
Path 701 | total_timesteps 5672.
Path 702 | total_timesteps 5679.
Path 703 | total_timesteps 5688.
Path 704 | total_timesteps 5697.
Path 705 | total_timesteps 5704.
Path 706 | total_timesteps 5711.
Path 707 | total_timesteps 5720.
Path 708 | total_timesteps 5727.
Path 709 | total_timesteps 5737.
Path 710 | total_timesteps 5744.
Path 711 | total_timesteps 5750.
Path 712 | total_timesteps 5762.
Path 713 | total_timesteps 5769.
Path 714 | total_timesteps 5776.
Path 715 | total_timesteps 5782.
Path 716 | total_timesteps 5790.
Path 717 | total_timesteps 5797.
Path 718 | total_timesteps 5805.
Path 719 | total_timesteps 5813.
Path 720 | total_timesteps 5820.
Path 721 | total_timesteps 5827.
Path 722 | total_timesteps 5836.
Path 723 | total_timesteps 5844.
Path 724 | total_timesteps 5852.
Path 725 | total_timesteps 5860.
Path 726 | total_timesteps 5870.
Path 727 | total_timesteps 5878.
Path 728 | total_timesteps 5884.
Path 729 | total_timesteps 5894.
Path 730 | total_timesteps 5902.
Path 731 | total_timesteps 5912.
Path 732 | total_timesteps 5921.
Path 733 | total_timesteps 5928.
Path 734 | total_timesteps 5938.
Path 735 | total_timesteps 5945.
Path 736 | total_timesteps 5954.
Path 737 | total_timesteps 5963.
Path 738 | total_timesteps 5970.
Path 739 | total_timesteps 5979.
Path 740 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.86    |
| Iteration     | 31       |
| MaximumReturn | -2.3     |
| MinimumReturn | -16      |
| TotalSamples  | 132111   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.0014069222379475832
Validation loss = 0.0014439766528084874
Validation loss = 0.0013063413789495826
Validation loss = 0.0012745633721351624
Validation loss = 0.0012886979384347796
Validation loss = 0.00152431963942945
Validation loss = 0.001804209197871387
Validation loss = 0.0014809775166213512
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 25.
Path 4 | total_timesteps 31.
Path 5 | total_timesteps 41.
Path 6 | total_timesteps 49.
Path 7 | total_timesteps 55.
Path 8 | total_timesteps 63.
Path 9 | total_timesteps 70.
Path 10 | total_timesteps 77.
Path 11 | total_timesteps 85.
Path 12 | total_timesteps 91.
Path 13 | total_timesteps 98.
Path 14 | total_timesteps 104.
Path 15 | total_timesteps 111.
Path 16 | total_timesteps 118.
Path 17 | total_timesteps 124.
Path 18 | total_timesteps 131.
Path 19 | total_timesteps 138.
Path 20 | total_timesteps 145.
Path 21 | total_timesteps 152.
Path 22 | total_timesteps 158.
Path 23 | total_timesteps 165.
Path 24 | total_timesteps 174.
Path 25 | total_timesteps 184.
Path 26 | total_timesteps 193.
Path 27 | total_timesteps 200.
Path 28 | total_timesteps 207.
Path 29 | total_timesteps 214.
Path 30 | total_timesteps 224.
Path 31 | total_timesteps 232.
Path 32 | total_timesteps 239.
Path 33 | total_timesteps 247.
Path 34 | total_timesteps 260.
Path 35 | total_timesteps 270.
Path 36 | total_timesteps 279.
Path 37 | total_timesteps 287.
Path 38 | total_timesteps 295.
Path 39 | total_timesteps 302.
Path 40 | total_timesteps 312.
Path 41 | total_timesteps 319.
Path 42 | total_timesteps 327.
Path 43 | total_timesteps 335.
Path 44 | total_timesteps 342.
Path 45 | total_timesteps 349.
Path 46 | total_timesteps 358.
Path 47 | total_timesteps 367.
Path 48 | total_timesteps 373.
Path 49 | total_timesteps 382.
Path 50 | total_timesteps 393.
Path 51 | total_timesteps 401.
Path 52 | total_timesteps 408.
Path 53 | total_timesteps 415.
Path 54 | total_timesteps 422.
Path 55 | total_timesteps 429.
Path 56 | total_timesteps 438.
Path 57 | total_timesteps 445.
Path 58 | total_timesteps 454.
Path 59 | total_timesteps 466.
Path 60 | total_timesteps 476.
Path 61 | total_timesteps 485.
Path 62 | total_timesteps 492.
Path 63 | total_timesteps 499.
Path 64 | total_timesteps 508.
Path 65 | total_timesteps 515.
Path 66 | total_timesteps 521.
Path 67 | total_timesteps 530.
Path 68 | total_timesteps 537.
Path 69 | total_timesteps 544.
Path 70 | total_timesteps 553.
Path 71 | total_timesteps 559.
Path 72 | total_timesteps 569.
Path 73 | total_timesteps 575.
Path 74 | total_timesteps 582.
Path 75 | total_timesteps 589.
Path 76 | total_timesteps 596.
Path 77 | total_timesteps 604.
Path 78 | total_timesteps 613.
Path 79 | total_timesteps 620.
Path 80 | total_timesteps 628.
Path 81 | total_timesteps 641.
Path 82 | total_timesteps 649.
Path 83 | total_timesteps 656.
Path 84 | total_timesteps 664.
Path 85 | total_timesteps 672.
Path 86 | total_timesteps 680.
Path 87 | total_timesteps 689.
Path 88 | total_timesteps 696.
Path 89 | total_timesteps 705.
Path 90 | total_timesteps 712.
Path 91 | total_timesteps 718.
Path 92 | total_timesteps 727.
Path 93 | total_timesteps 734.
Path 94 | total_timesteps 743.
Path 95 | total_timesteps 754.
Path 96 | total_timesteps 763.
Path 97 | total_timesteps 777.
Path 98 | total_timesteps 784.
Path 99 | total_timesteps 790.
Path 100 | total_timesteps 797.
Path 101 | total_timesteps 806.
Path 102 | total_timesteps 812.
Path 103 | total_timesteps 820.
Path 104 | total_timesteps 828.
Path 105 | total_timesteps 836.
Path 106 | total_timesteps 843.
Path 107 | total_timesteps 850.
Path 108 | total_timesteps 859.
Path 109 | total_timesteps 869.
Path 110 | total_timesteps 877.
Path 111 | total_timesteps 884.
Path 112 | total_timesteps 891.
Path 113 | total_timesteps 899.
Path 114 | total_timesteps 908.
Path 115 | total_timesteps 919.
Path 116 | total_timesteps 925.
Path 117 | total_timesteps 932.
Path 118 | total_timesteps 942.
Path 119 | total_timesteps 949.
Path 120 | total_timesteps 958.
Path 121 | total_timesteps 965.
Path 122 | total_timesteps 973.
Path 123 | total_timesteps 980.
Path 124 | total_timesteps 988.
Path 125 | total_timesteps 996.
Path 126 | total_timesteps 1004.
Path 127 | total_timesteps 1012.
Path 128 | total_timesteps 1021.
Path 129 | total_timesteps 1030.
Path 130 | total_timesteps 1037.
Path 131 | total_timesteps 1044.
Path 132 | total_timesteps 1052.
Path 133 | total_timesteps 1058.
Path 134 | total_timesteps 1066.
Path 135 | total_timesteps 1075.
Path 136 | total_timesteps 1081.
Path 137 | total_timesteps 1089.
Path 138 | total_timesteps 1095.
Path 139 | total_timesteps 1103.
Path 140 | total_timesteps 1109.
Path 141 | total_timesteps 1117.
Path 142 | total_timesteps 1123.
Path 143 | total_timesteps 1131.
Path 144 | total_timesteps 1141.
Path 145 | total_timesteps 1149.
Path 146 | total_timesteps 1162.
Path 147 | total_timesteps 1170.
Path 148 | total_timesteps 1178.
Path 149 | total_timesteps 1185.
Path 150 | total_timesteps 1191.
Path 151 | total_timesteps 1198.
Path 152 | total_timesteps 1207.
Path 153 | total_timesteps 1215.
Path 154 | total_timesteps 1224.
Path 155 | total_timesteps 1232.
Path 156 | total_timesteps 1238.
Path 157 | total_timesteps 1244.
Path 158 | total_timesteps 1251.
Path 159 | total_timesteps 1265.
Path 160 | total_timesteps 1271.
Path 161 | total_timesteps 1279.
Path 162 | total_timesteps 1287.
Path 163 | total_timesteps 1294.
Path 164 | total_timesteps 1303.
Path 165 | total_timesteps 1312.
Path 166 | total_timesteps 1318.
Path 167 | total_timesteps 1326.
Path 168 | total_timesteps 1335.
Path 169 | total_timesteps 1341.
Path 170 | total_timesteps 1349.
Path 171 | total_timesteps 1358.
Path 172 | total_timesteps 1366.
Path 173 | total_timesteps 1374.
Path 174 | total_timesteps 1380.
Path 175 | total_timesteps 1386.
Path 176 | total_timesteps 1394.
Path 177 | total_timesteps 1400.
Path 178 | total_timesteps 1408.
Path 179 | total_timesteps 1414.
Path 180 | total_timesteps 1420.
Path 181 | total_timesteps 1430.
Path 182 | total_timesteps 1438.
Path 183 | total_timesteps 1450.
Path 184 | total_timesteps 1457.
Path 185 | total_timesteps 1467.
Path 186 | total_timesteps 1477.
Path 187 | total_timesteps 1483.
Path 188 | total_timesteps 1491.
Path 189 | total_timesteps 1498.
Path 190 | total_timesteps 1504.
Path 191 | total_timesteps 1514.
Path 192 | total_timesteps 1524.
Path 193 | total_timesteps 1532.
Path 194 | total_timesteps 1542.
Path 195 | total_timesteps 1551.
Path 196 | total_timesteps 1560.
Path 197 | total_timesteps 1567.
Path 198 | total_timesteps 1573.
Path 199 | total_timesteps 1581.
Path 200 | total_timesteps 1590.
Path 201 | total_timesteps 1601.
Path 202 | total_timesteps 1609.
Path 203 | total_timesteps 1617.
Path 204 | total_timesteps 1628.
Path 205 | total_timesteps 1638.
Path 206 | total_timesteps 1644.
Path 207 | total_timesteps 1650.
Path 208 | total_timesteps 1661.
Path 209 | total_timesteps 1668.
Path 210 | total_timesteps 1676.
Path 211 | total_timesteps 1682.
Path 212 | total_timesteps 1692.
Path 213 | total_timesteps 1701.
Path 214 | total_timesteps 1709.
Path 215 | total_timesteps 1719.
Path 216 | total_timesteps 1726.
Path 217 | total_timesteps 1733.
Path 218 | total_timesteps 1740.
Path 219 | total_timesteps 1748.
Path 220 | total_timesteps 1754.
Path 221 | total_timesteps 1762.
Path 222 | total_timesteps 1770.
Path 223 | total_timesteps 1778.
Path 224 | total_timesteps 1786.
Path 225 | total_timesteps 1796.
Path 226 | total_timesteps 1804.
Path 227 | total_timesteps 1810.
Path 228 | total_timesteps 1818.
Path 229 | total_timesteps 1828.
Path 230 | total_timesteps 1837.
Path 231 | total_timesteps 1845.
Path 232 | total_timesteps 1852.
Path 233 | total_timesteps 1859.
Path 234 | total_timesteps 1882.
Path 235 | total_timesteps 1892.
Path 236 | total_timesteps 1901.
Path 237 | total_timesteps 1908.
Path 238 | total_timesteps 1914.
Path 239 | total_timesteps 1922.
Path 240 | total_timesteps 1936.
Path 241 | total_timesteps 1945.
Path 242 | total_timesteps 1951.
Path 243 | total_timesteps 1958.
Path 244 | total_timesteps 1965.
Path 245 | total_timesteps 1973.
Path 246 | total_timesteps 1980.
Path 247 | total_timesteps 1989.
Path 248 | total_timesteps 1997.
Path 249 | total_timesteps 2003.
Path 250 | total_timesteps 2011.
Path 251 | total_timesteps 2017.
Path 252 | total_timesteps 2024.
Path 253 | total_timesteps 2040.
Path 254 | total_timesteps 2047.
Path 255 | total_timesteps 2056.
Path 256 | total_timesteps 2063.
Path 257 | total_timesteps 2070.
Path 258 | total_timesteps 2077.
Path 259 | total_timesteps 2088.
Path 260 | total_timesteps 2094.
Path 261 | total_timesteps 2101.
Path 262 | total_timesteps 2109.
Path 263 | total_timesteps 2118.
Path 264 | total_timesteps 2126.
Path 265 | total_timesteps 2133.
Path 266 | total_timesteps 2140.
Path 267 | total_timesteps 2147.
Path 268 | total_timesteps 2155.
Path 269 | total_timesteps 2164.
Path 270 | total_timesteps 2171.
Path 271 | total_timesteps 2177.
Path 272 | total_timesteps 2183.
Path 273 | total_timesteps 2189.
Path 274 | total_timesteps 2198.
Path 275 | total_timesteps 2207.
Path 276 | total_timesteps 2216.
Path 277 | total_timesteps 2223.
Path 278 | total_timesteps 2232.
Path 279 | total_timesteps 2239.
Path 280 | total_timesteps 2246.
Path 281 | total_timesteps 2253.
Path 282 | total_timesteps 2261.
Path 283 | total_timesteps 2267.
Path 284 | total_timesteps 2275.
Path 285 | total_timesteps 2285.
Path 286 | total_timesteps 2295.
Path 287 | total_timesteps 2302.
Path 288 | total_timesteps 2314.
Path 289 | total_timesteps 2320.
Path 290 | total_timesteps 2327.
Path 291 | total_timesteps 2337.
Path 292 | total_timesteps 2347.
Path 293 | total_timesteps 2355.
Path 294 | total_timesteps 2363.
Path 295 | total_timesteps 2373.
Path 296 | total_timesteps 2379.
Path 297 | total_timesteps 2387.
Path 298 | total_timesteps 2395.
Path 299 | total_timesteps 2404.
Path 300 | total_timesteps 2412.
Path 301 | total_timesteps 2419.
Path 302 | total_timesteps 2426.
Path 303 | total_timesteps 2438.
Path 304 | total_timesteps 2447.
Path 305 | total_timesteps 2462.
Path 306 | total_timesteps 2468.
Path 307 | total_timesteps 2477.
Path 308 | total_timesteps 2483.
Path 309 | total_timesteps 2490.
Path 310 | total_timesteps 2501.
Path 311 | total_timesteps 2513.
Path 312 | total_timesteps 2520.
Path 313 | total_timesteps 2529.
Path 314 | total_timesteps 2535.
Path 315 | total_timesteps 2542.
Path 316 | total_timesteps 2548.
Path 317 | total_timesteps 2555.
Path 318 | total_timesteps 2562.
Path 319 | total_timesteps 2571.
Path 320 | total_timesteps 2577.
Path 321 | total_timesteps 2584.
Path 322 | total_timesteps 2592.
Path 323 | total_timesteps 2601.
Path 324 | total_timesteps 2608.
Path 325 | total_timesteps 2615.
Path 326 | total_timesteps 2623.
Path 327 | total_timesteps 2631.
Path 328 | total_timesteps 2639.
Path 329 | total_timesteps 2645.
Path 330 | total_timesteps 2653.
Path 331 | total_timesteps 2660.
Path 332 | total_timesteps 2667.
Path 333 | total_timesteps 2675.
Path 334 | total_timesteps 2689.
Path 335 | total_timesteps 2698.
Path 336 | total_timesteps 2707.
Path 337 | total_timesteps 2717.
Path 338 | total_timesteps 2723.
Path 339 | total_timesteps 2730.
Path 340 | total_timesteps 2737.
Path 341 | total_timesteps 2746.
Path 342 | total_timesteps 2755.
Path 343 | total_timesteps 2761.
Path 344 | total_timesteps 2767.
Path 345 | total_timesteps 2776.
Path 346 | total_timesteps 2787.
Path 347 | total_timesteps 2794.
Path 348 | total_timesteps 2802.
Path 349 | total_timesteps 2810.
Path 350 | total_timesteps 2817.
Path 351 | total_timesteps 2824.
Path 352 | total_timesteps 2831.
Path 353 | total_timesteps 2840.
Path 354 | total_timesteps 2848.
Path 355 | total_timesteps 2854.
Path 356 | total_timesteps 2862.
Path 357 | total_timesteps 2870.
Path 358 | total_timesteps 2877.
Path 359 | total_timesteps 2884.
Path 360 | total_timesteps 2891.
Path 361 | total_timesteps 2898.
Path 362 | total_timesteps 2904.
Path 363 | total_timesteps 2910.
Path 364 | total_timesteps 2918.
Path 365 | total_timesteps 2926.
Path 366 | total_timesteps 2934.
Path 367 | total_timesteps 2942.
Path 368 | total_timesteps 2950.
Path 369 | total_timesteps 2958.
Path 370 | total_timesteps 2964.
Path 371 | total_timesteps 2972.
Path 372 | total_timesteps 2979.
Path 373 | total_timesteps 2987.
Path 374 | total_timesteps 2994.
Path 375 | total_timesteps 3001.
Path 376 | total_timesteps 3008.
Path 377 | total_timesteps 3018.
Path 378 | total_timesteps 3024.
Path 379 | total_timesteps 3033.
Path 380 | total_timesteps 3041.
Path 381 | total_timesteps 3047.
Path 382 | total_timesteps 3055.
Path 383 | total_timesteps 3063.
Path 384 | total_timesteps 3070.
Path 385 | total_timesteps 3078.
Path 386 | total_timesteps 3085.
Path 387 | total_timesteps 3095.
Path 388 | total_timesteps 3103.
Path 389 | total_timesteps 3110.
Path 390 | total_timesteps 3118.
Path 391 | total_timesteps 3126.
Path 392 | total_timesteps 3133.
Path 393 | total_timesteps 3139.
Path 394 | total_timesteps 3147.
Path 395 | total_timesteps 3156.
Path 396 | total_timesteps 3164.
Path 397 | total_timesteps 3175.
Path 398 | total_timesteps 3183.
Path 399 | total_timesteps 3193.
Path 400 | total_timesteps 3201.
Path 401 | total_timesteps 3209.
Path 402 | total_timesteps 3215.
Path 403 | total_timesteps 3223.
Path 404 | total_timesteps 3232.
Path 405 | total_timesteps 3240.
Path 406 | total_timesteps 3253.
Path 407 | total_timesteps 3262.
Path 408 | total_timesteps 3270.
Path 409 | total_timesteps 3277.
Path 410 | total_timesteps 3286.
Path 411 | total_timesteps 3296.
Path 412 | total_timesteps 3304.
Path 413 | total_timesteps 3314.
Path 414 | total_timesteps 3324.
Path 415 | total_timesteps 3331.
Path 416 | total_timesteps 3339.
Path 417 | total_timesteps 3346.
Path 418 | total_timesteps 3353.
Path 419 | total_timesteps 3361.
Path 420 | total_timesteps 3368.
Path 421 | total_timesteps 3374.
Path 422 | total_timesteps 3383.
Path 423 | total_timesteps 3390.
Path 424 | total_timesteps 3397.
Path 425 | total_timesteps 3405.
Path 426 | total_timesteps 3414.
Path 427 | total_timesteps 3421.
Path 428 | total_timesteps 3431.
Path 429 | total_timesteps 3439.
Path 430 | total_timesteps 3446.
Path 431 | total_timesteps 3455.
Path 432 | total_timesteps 3461.
Path 433 | total_timesteps 3468.
Path 434 | total_timesteps 3476.
Path 435 | total_timesteps 3484.
Path 436 | total_timesteps 3491.
Path 437 | total_timesteps 3499.
Path 438 | total_timesteps 3508.
Path 439 | total_timesteps 3515.
Path 440 | total_timesteps 3522.
Path 441 | total_timesteps 3531.
Path 442 | total_timesteps 3539.
Path 443 | total_timesteps 3549.
Path 444 | total_timesteps 3556.
Path 445 | total_timesteps 3563.
Path 446 | total_timesteps 3570.
Path 447 | total_timesteps 3576.
Path 448 | total_timesteps 3585.
Path 449 | total_timesteps 3591.
Path 450 | total_timesteps 3597.
Path 451 | total_timesteps 3608.
Path 452 | total_timesteps 3616.
Path 453 | total_timesteps 3623.
Path 454 | total_timesteps 3629.
Path 455 | total_timesteps 3636.
Path 456 | total_timesteps 3645.
Path 457 | total_timesteps 3652.
Path 458 | total_timesteps 3660.
Path 459 | total_timesteps 3670.
Path 460 | total_timesteps 3677.
Path 461 | total_timesteps 3683.
Path 462 | total_timesteps 3691.
Path 463 | total_timesteps 3697.
Path 464 | total_timesteps 3708.
Path 465 | total_timesteps 3716.
Path 466 | total_timesteps 3726.
Path 467 | total_timesteps 3735.
Path 468 | total_timesteps 3743.
Path 469 | total_timesteps 3750.
Path 470 | total_timesteps 3757.
Path 471 | total_timesteps 3764.
Path 472 | total_timesteps 3772.
Path 473 | total_timesteps 3780.
Path 474 | total_timesteps 3787.
Path 475 | total_timesteps 3794.
Path 476 | total_timesteps 3801.
Path 477 | total_timesteps 3807.
Path 478 | total_timesteps 3813.
Path 479 | total_timesteps 3821.
Path 480 | total_timesteps 3828.
Path 481 | total_timesteps 3837.
Path 482 | total_timesteps 3844.
Path 483 | total_timesteps 3852.
Path 484 | total_timesteps 3860.
Path 485 | total_timesteps 3868.
Path 486 | total_timesteps 3875.
Path 487 | total_timesteps 3883.
Path 488 | total_timesteps 3890.
Path 489 | total_timesteps 3898.
Path 490 | total_timesteps 3907.
Path 491 | total_timesteps 3916.
Path 492 | total_timesteps 3930.
Path 493 | total_timesteps 3937.
Path 494 | total_timesteps 3944.
Path 495 | total_timesteps 3956.
Path 496 | total_timesteps 3964.
Path 497 | total_timesteps 3970.
Path 498 | total_timesteps 3976.
Path 499 | total_timesteps 3983.
Path 500 | total_timesteps 3992.
Path 501 | total_timesteps 4000.
Path 502 | total_timesteps 4006.
Path 503 | total_timesteps 4012.
Path 504 | total_timesteps 4022.
Path 505 | total_timesteps 4031.
Path 506 | total_timesteps 4038.
Path 507 | total_timesteps 4047.
Path 508 | total_timesteps 4055.
Path 509 | total_timesteps 4065.
Path 510 | total_timesteps 4073.
Path 511 | total_timesteps 4081.
Path 512 | total_timesteps 4089.
Path 513 | total_timesteps 4095.
Path 514 | total_timesteps 4103.
Path 515 | total_timesteps 4113.
Path 516 | total_timesteps 4119.
Path 517 | total_timesteps 4127.
Path 518 | total_timesteps 4137.
Path 519 | total_timesteps 4144.
Path 520 | total_timesteps 4152.
Path 521 | total_timesteps 4158.
Path 522 | total_timesteps 4165.
Path 523 | total_timesteps 4176.
Path 524 | total_timesteps 4184.
Path 525 | total_timesteps 4194.
Path 526 | total_timesteps 4202.
Path 527 | total_timesteps 4209.
Path 528 | total_timesteps 4218.
Path 529 | total_timesteps 4229.
Path 530 | total_timesteps 4239.
Path 531 | total_timesteps 4247.
Path 532 | total_timesteps 4256.
Path 533 | total_timesteps 4263.
Path 534 | total_timesteps 4271.
Path 535 | total_timesteps 4280.
Path 536 | total_timesteps 4287.
Path 537 | total_timesteps 4294.
Path 538 | total_timesteps 4304.
Path 539 | total_timesteps 4315.
Path 540 | total_timesteps 4322.
Path 541 | total_timesteps 4330.
Path 542 | total_timesteps 4338.
Path 543 | total_timesteps 4348.
Path 544 | total_timesteps 4357.
Path 545 | total_timesteps 4364.
Path 546 | total_timesteps 4371.
Path 547 | total_timesteps 4383.
Path 548 | total_timesteps 4392.
Path 549 | total_timesteps 4398.
Path 550 | total_timesteps 4406.
Path 551 | total_timesteps 4415.
Path 552 | total_timesteps 4421.
Path 553 | total_timesteps 4427.
Path 554 | total_timesteps 4433.
Path 555 | total_timesteps 4440.
Path 556 | total_timesteps 4449.
Path 557 | total_timesteps 4459.
Path 558 | total_timesteps 4465.
Path 559 | total_timesteps 4475.
Path 560 | total_timesteps 4488.
Path 561 | total_timesteps 4495.
Path 562 | total_timesteps 4505.
Path 563 | total_timesteps 4512.
Path 564 | total_timesteps 4520.
Path 565 | total_timesteps 4528.
Path 566 | total_timesteps 4535.
Path 567 | total_timesteps 4542.
Path 568 | total_timesteps 4549.
Path 569 | total_timesteps 4557.
Path 570 | total_timesteps 4564.
Path 571 | total_timesteps 4572.
Path 572 | total_timesteps 4580.
Path 573 | total_timesteps 4590.
Path 574 | total_timesteps 4597.
Path 575 | total_timesteps 4605.
Path 576 | total_timesteps 4615.
Path 577 | total_timesteps 4624.
Path 578 | total_timesteps 4633.
Path 579 | total_timesteps 4644.
Path 580 | total_timesteps 4651.
Path 581 | total_timesteps 4661.
Path 582 | total_timesteps 4668.
Path 583 | total_timesteps 4676.
Path 584 | total_timesteps 4685.
Path 585 | total_timesteps 4694.
Path 586 | total_timesteps 4700.
Path 587 | total_timesteps 4711.
Path 588 | total_timesteps 4719.
Path 589 | total_timesteps 4727.
Path 590 | total_timesteps 4735.
Path 591 | total_timesteps 4742.
Path 592 | total_timesteps 4749.
Path 593 | total_timesteps 4761.
Path 594 | total_timesteps 4767.
Path 595 | total_timesteps 4773.
Path 596 | total_timesteps 4780.
Path 597 | total_timesteps 4788.
Path 598 | total_timesteps 4796.
Path 599 | total_timesteps 4805.
Path 600 | total_timesteps 4813.
Path 601 | total_timesteps 4823.
Path 602 | total_timesteps 4835.
Path 603 | total_timesteps 4845.
Path 604 | total_timesteps 4853.
Path 605 | total_timesteps 4863.
Path 606 | total_timesteps 4869.
Path 607 | total_timesteps 4878.
Path 608 | total_timesteps 4886.
Path 609 | total_timesteps 4897.
Path 610 | total_timesteps 4904.
Path 611 | total_timesteps 4912.
Path 612 | total_timesteps 4928.
Path 613 | total_timesteps 4935.
Path 614 | total_timesteps 4943.
Path 615 | total_timesteps 4950.
Path 616 | total_timesteps 4957.
Path 617 | total_timesteps 4970.
Path 618 | total_timesteps 4978.
Path 619 | total_timesteps 4985.
Path 620 | total_timesteps 4995.
Path 621 | total_timesteps 5008.
Path 622 | total_timesteps 5019.
Path 623 | total_timesteps 5029.
Path 624 | total_timesteps 5037.
Path 625 | total_timesteps 5045.
Path 626 | total_timesteps 5052.
Path 627 | total_timesteps 5060.
Path 628 | total_timesteps 5067.
Path 629 | total_timesteps 5077.
Path 630 | total_timesteps 5089.
Path 631 | total_timesteps 5096.
Path 632 | total_timesteps 5102.
Path 633 | total_timesteps 5109.
Path 634 | total_timesteps 5118.
Path 635 | total_timesteps 5130.
Path 636 | total_timesteps 5137.
Path 637 | total_timesteps 5143.
Path 638 | total_timesteps 5155.
Path 639 | total_timesteps 5161.
Path 640 | total_timesteps 5169.
Path 641 | total_timesteps 5178.
Path 642 | total_timesteps 5186.
Path 643 | total_timesteps 5196.
Path 644 | total_timesteps 5203.
Path 645 | total_timesteps 5214.
Path 646 | total_timesteps 5222.
Path 647 | total_timesteps 5228.
Path 648 | total_timesteps 5238.
Path 649 | total_timesteps 5247.
Path 650 | total_timesteps 5254.
Path 651 | total_timesteps 5266.
Path 652 | total_timesteps 5277.
Path 653 | total_timesteps 5284.
Path 654 | total_timesteps 5291.
Path 655 | total_timesteps 5299.
Path 656 | total_timesteps 5307.
Path 657 | total_timesteps 5314.
Path 658 | total_timesteps 5321.
Path 659 | total_timesteps 5328.
Path 660 | total_timesteps 5336.
Path 661 | total_timesteps 5346.
Path 662 | total_timesteps 5356.
Path 663 | total_timesteps 5363.
Path 664 | total_timesteps 5379.
Path 665 | total_timesteps 5388.
Path 666 | total_timesteps 5397.
Path 667 | total_timesteps 5405.
Path 668 | total_timesteps 5414.
Path 669 | total_timesteps 5424.
Path 670 | total_timesteps 5430.
Path 671 | total_timesteps 5439.
Path 672 | total_timesteps 5447.
Path 673 | total_timesteps 5454.
Path 674 | total_timesteps 5461.
Path 675 | total_timesteps 5469.
Path 676 | total_timesteps 5477.
Path 677 | total_timesteps 5485.
Path 678 | total_timesteps 5491.
Path 679 | total_timesteps 5499.
Path 680 | total_timesteps 5506.
Path 681 | total_timesteps 5513.
Path 682 | total_timesteps 5523.
Path 683 | total_timesteps 5530.
Path 684 | total_timesteps 5539.
Path 685 | total_timesteps 5546.
Path 686 | total_timesteps 5552.
Path 687 | total_timesteps 5559.
Path 688 | total_timesteps 5565.
Path 689 | total_timesteps 5574.
Path 690 | total_timesteps 5583.
Path 691 | total_timesteps 5589.
Path 692 | total_timesteps 5596.
Path 693 | total_timesteps 5604.
Path 694 | total_timesteps 5614.
Path 695 | total_timesteps 5623.
Path 696 | total_timesteps 5633.
Path 697 | total_timesteps 5640.
Path 698 | total_timesteps 5650.
Path 699 | total_timesteps 5658.
Path 700 | total_timesteps 5666.
Path 701 | total_timesteps 5675.
Path 702 | total_timesteps 5684.
Path 703 | total_timesteps 5692.
Path 704 | total_timesteps 5698.
Path 705 | total_timesteps 5706.
Path 706 | total_timesteps 5713.
Path 707 | total_timesteps 5719.
Path 708 | total_timesteps 5732.
Path 709 | total_timesteps 5738.
Path 710 | total_timesteps 5745.
Path 711 | total_timesteps 5752.
Path 712 | total_timesteps 5758.
Path 713 | total_timesteps 5764.
Path 714 | total_timesteps 5772.
Path 715 | total_timesteps 5780.
Path 716 | total_timesteps 5787.
Path 717 | total_timesteps 5796.
Path 718 | total_timesteps 5803.
Path 719 | total_timesteps 5810.
Path 720 | total_timesteps 5821.
Path 721 | total_timesteps 5832.
Path 722 | total_timesteps 5838.
Path 723 | total_timesteps 5850.
Path 724 | total_timesteps 5859.
Path 725 | total_timesteps 5870.
Path 726 | total_timesteps 5879.
Path 727 | total_timesteps 5885.
Path 728 | total_timesteps 5892.
Path 729 | total_timesteps 5900.
Path 730 | total_timesteps 5906.
Path 731 | total_timesteps 5913.
Path 732 | total_timesteps 5921.
Path 733 | total_timesteps 5928.
Path 734 | total_timesteps 5937.
Path 735 | total_timesteps 5947.
Path 736 | total_timesteps 5957.
Path 737 | total_timesteps 5966.
Path 738 | total_timesteps 5974.
Path 739 | total_timesteps 5982.
Path 740 | total_timesteps 5989.
Path 741 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.85    |
| Iteration     | 32       |
| MaximumReturn | -2.07    |
| MinimumReturn | -19.3    |
| TotalSamples  | 136112   |
----------------------------
