Logging to experiments/gym_fwalker2d/Wo01/Mon-07-Nov-2022-10-30-38-AM-CST_gym_fwalker2d_trpo_iteration_20_seed2231
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 50.
Path 4 | total_timesteps 69.
Path 5 | total_timesteps 91.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 130.
Path 8 | total_timesteps 140.
Path 9 | total_timesteps 168.
Path 10 | total_timesteps 187.
Path 11 | total_timesteps 200.
Path 12 | total_timesteps 214.
Path 13 | total_timesteps 239.
Path 14 | total_timesteps 265.
Path 15 | total_timesteps 291.
Path 16 | total_timesteps 315.
Path 17 | total_timesteps 329.
Path 18 | total_timesteps 350.
Path 19 | total_timesteps 392.
Path 20 | total_timesteps 416.
Path 21 | total_timesteps 425.
Path 22 | total_timesteps 447.
Path 23 | total_timesteps 481.
Path 24 | total_timesteps 496.
Path 25 | total_timesteps 526.
Path 26 | total_timesteps 563.
Path 27 | total_timesteps 586.
Path 28 | total_timesteps 604.
Path 29 | total_timesteps 618.
Path 30 | total_timesteps 631.
Path 31 | total_timesteps 665.
Path 32 | total_timesteps 701.
Path 33 | total_timesteps 723.
Path 34 | total_timesteps 745.
Path 35 | total_timesteps 778.
Path 36 | total_timesteps 800.
Path 37 | total_timesteps 821.
Path 38 | total_timesteps 835.
Path 39 | total_timesteps 853.
Path 40 | total_timesteps 872.
Path 41 | total_timesteps 889.
Path 42 | total_timesteps 918.
Path 43 | total_timesteps 932.
Path 44 | total_timesteps 945.
Path 45 | total_timesteps 956.
Path 46 | total_timesteps 979.
Path 47 | total_timesteps 1005.
Path 48 | total_timesteps 1025.
Path 49 | total_timesteps 1042.
Path 50 | total_timesteps 1066.
Path 51 | total_timesteps 1080.
Path 52 | total_timesteps 1091.
Path 53 | total_timesteps 1122.
Path 54 | total_timesteps 1135.
Path 55 | total_timesteps 1161.
Path 56 | total_timesteps 1186.
Path 57 | total_timesteps 1226.
Path 58 | total_timesteps 1248.
Path 59 | total_timesteps 1265.
Path 60 | total_timesteps 1299.
Path 61 | total_timesteps 1313.
Path 62 | total_timesteps 1328.
Path 63 | total_timesteps 1345.
Path 64 | total_timesteps 1382.
Path 65 | total_timesteps 1400.
Path 66 | total_timesteps 1412.
Path 67 | total_timesteps 1426.
Path 68 | total_timesteps 1441.
Path 69 | total_timesteps 1464.
Path 70 | total_timesteps 1520.
Path 71 | total_timesteps 1542.
Path 72 | total_timesteps 1551.
Path 73 | total_timesteps 1563.
Path 74 | total_timesteps 1620.
Path 75 | total_timesteps 1641.
Path 76 | total_timesteps 1666.
Path 77 | total_timesteps 1683.
Path 78 | total_timesteps 1693.
Path 79 | total_timesteps 1718.
Path 80 | total_timesteps 1731.
Path 81 | total_timesteps 1760.
Path 82 | total_timesteps 1785.
Path 83 | total_timesteps 1799.
Path 84 | total_timesteps 1821.
Path 85 | total_timesteps 1851.
Path 86 | total_timesteps 1865.
Path 87 | total_timesteps 1878.
Path 88 | total_timesteps 1898.
Path 89 | total_timesteps 1921.
Path 90 | total_timesteps 1932.
Path 91 | total_timesteps 1951.
Path 92 | total_timesteps 1971.
Path 93 | total_timesteps 1995.
Path 94 | total_timesteps 2025.
Path 95 | total_timesteps 2050.
Path 96 | total_timesteps 2074.
Path 97 | total_timesteps 2092.
Path 98 | total_timesteps 2118.
Path 99 | total_timesteps 2127.
Path 100 | total_timesteps 2143.
Path 101 | total_timesteps 2153.
Path 102 | total_timesteps 2163.
Path 103 | total_timesteps 2187.
Path 104 | total_timesteps 2213.
Path 105 | total_timesteps 2237.
Path 106 | total_timesteps 2253.
Path 107 | total_timesteps 2271.
Path 108 | total_timesteps 2310.
Path 109 | total_timesteps 2337.
Path 110 | total_timesteps 2356.
Path 111 | total_timesteps 2400.
Path 112 | total_timesteps 2417.
Path 113 | total_timesteps 2451.
Path 114 | total_timesteps 2469.
Path 115 | total_timesteps 2487.
Path 116 | total_timesteps 2503.
Path 117 | total_timesteps 2541.
Path 118 | total_timesteps 2552.
Path 119 | total_timesteps 2568.
Path 120 | total_timesteps 2584.
Path 121 | total_timesteps 2613.
Path 122 | total_timesteps 2627.
Path 123 | total_timesteps 2642.
Path 124 | total_timesteps 2664.
Path 125 | total_timesteps 2680.
Path 126 | total_timesteps 2691.
Path 127 | total_timesteps 2714.
Path 128 | total_timesteps 2741.
Path 129 | total_timesteps 2763.
Path 130 | total_timesteps 2780.
Path 131 | total_timesteps 2791.
Path 132 | total_timesteps 2815.
Path 133 | total_timesteps 2825.
Path 134 | total_timesteps 2842.
Path 135 | total_timesteps 2853.
Path 136 | total_timesteps 2864.
Path 137 | total_timesteps 2915.
Path 138 | total_timesteps 2935.
Path 139 | total_timesteps 2981.
Path 140 | total_timesteps 2993.
Path 141 | total_timesteps 3013.
Path 142 | total_timesteps 3022.
Path 143 | total_timesteps 3031.
Path 144 | total_timesteps 3052.
Path 145 | total_timesteps 3119.
Path 146 | total_timesteps 3143.
Path 147 | total_timesteps 3160.
Path 148 | total_timesteps 3184.
Path 149 | total_timesteps 3201.
Path 150 | total_timesteps 3213.
Path 151 | total_timesteps 3242.
Path 152 | total_timesteps 3256.
Path 153 | total_timesteps 3286.
Path 154 | total_timesteps 3308.
Path 155 | total_timesteps 3331.
Path 156 | total_timesteps 3348.
Path 157 | total_timesteps 3381.
Path 158 | total_timesteps 3421.
Path 159 | total_timesteps 3440.
Path 160 | total_timesteps 3465.
Path 161 | total_timesteps 3484.
Path 162 | total_timesteps 3498.
Path 163 | total_timesteps 3510.
Path 164 | total_timesteps 3524.
Path 165 | total_timesteps 3549.
Path 166 | total_timesteps 3565.
Path 167 | total_timesteps 3585.
Path 168 | total_timesteps 3611.
Path 169 | total_timesteps 3629.
Path 170 | total_timesteps 3656.
Path 171 | total_timesteps 3668.
Path 172 | total_timesteps 3699.
Path 173 | total_timesteps 3721.
Path 174 | total_timesteps 3738.
Path 175 | total_timesteps 3765.
Path 176 | total_timesteps 3797.
Path 177 | total_timesteps 3820.
Path 178 | total_timesteps 3837.
Path 179 | total_timesteps 3846.
Path 180 | total_timesteps 3878.
Path 181 | total_timesteps 3891.
Path 182 | total_timesteps 3911.
Path 183 | total_timesteps 3929.
Path 184 | total_timesteps 3956.
Path 185 | total_timesteps 3971.
Path 186 | total_timesteps 3983.
Path 187 | total_timesteps 4013.
Path 188 | total_timesteps 4048.
Path 189 | total_timesteps 4072.
Path 190 | total_timesteps 4086.
Path 191 | total_timesteps 4110.
Path 192 | total_timesteps 4123.
Path 193 | total_timesteps 4151.
Path 194 | total_timesteps 4178.
Path 195 | total_timesteps 4191.
Path 196 | total_timesteps 4212.
Path 197 | total_timesteps 4233.
Path 198 | total_timesteps 4255.
Path 199 | total_timesteps 4280.
Path 200 | total_timesteps 4312.
Path 201 | total_timesteps 4327.
Path 202 | total_timesteps 4347.
Path 203 | total_timesteps 4355.
Path 204 | total_timesteps 4380.
Path 205 | total_timesteps 4390.
Path 206 | total_timesteps 4412.
Path 207 | total_timesteps 4446.
Path 208 | total_timesteps 4461.
Path 209 | total_timesteps 4487.
Path 210 | total_timesteps 4507.
Path 211 | total_timesteps 4529.
Path 212 | total_timesteps 4547.
Path 213 | total_timesteps 4566.
Path 214 | total_timesteps 4593.
Path 215 | total_timesteps 4617.
Path 216 | total_timesteps 4631.
Path 217 | total_timesteps 4653.
Path 218 | total_timesteps 4664.
Path 219 | total_timesteps 4677.
Path 220 | total_timesteps 4715.
Path 221 | total_timesteps 4727.
Path 222 | total_timesteps 4759.
Path 223 | total_timesteps 4782.
Path 224 | total_timesteps 4817.
Path 225 | total_timesteps 4837.
Path 226 | total_timesteps 4874.
Path 227 | total_timesteps 4888.
Path 228 | total_timesteps 4911.
Path 229 | total_timesteps 4939.
Path 230 | total_timesteps 4967.
Path 231 | total_timesteps 4985.
Path 232 | total_timesteps 5015.
Path 233 | total_timesteps 5034.
Path 234 | total_timesteps 5050.
Path 235 | total_timesteps 5081.
Path 236 | total_timesteps 5099.
Path 237 | total_timesteps 5128.
Path 238 | total_timesteps 5145.
Path 239 | total_timesteps 5184.
Path 240 | total_timesteps 5207.
Path 241 | total_timesteps 5230.
Path 242 | total_timesteps 5265.
Path 243 | total_timesteps 5286.
Path 244 | total_timesteps 5302.
Path 245 | total_timesteps 5326.
Path 246 | total_timesteps 5338.
Path 247 | total_timesteps 5355.
Path 248 | total_timesteps 5370.
Path 249 | total_timesteps 5390.
Path 250 | total_timesteps 5419.
Path 251 | total_timesteps 5434.
Path 252 | total_timesteps 5447.
Path 253 | total_timesteps 5466.
Path 254 | total_timesteps 5495.
Path 255 | total_timesteps 5510.
Path 256 | total_timesteps 5538.
Path 257 | total_timesteps 5554.
Path 258 | total_timesteps 5573.
Path 259 | total_timesteps 5596.
Path 260 | total_timesteps 5631.
Path 261 | total_timesteps 5642.
Path 262 | total_timesteps 5662.
Path 263 | total_timesteps 5681.
Path 264 | total_timesteps 5712.
Path 265 | total_timesteps 5719.
Path 266 | total_timesteps 5739.
Path 267 | total_timesteps 5778.
Path 268 | total_timesteps 5793.
Path 269 | total_timesteps 5806.
Path 270 | total_timesteps 5828.
Path 271 | total_timesteps 5846.
Path 272 | total_timesteps 5861.
Path 273 | total_timesteps 5889.
Path 274 | total_timesteps 5900.
Path 275 | total_timesteps 5913.
Path 276 | total_timesteps 5933.
Path 277 | total_timesteps 5955.
Path 278 | total_timesteps 5979.
Path 279 | total_timesteps 5991.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.7578297257423401
Validation loss = 0.3654283285140991
Validation loss = 0.3393453359603882
Validation loss = 0.32255738973617554
Validation loss = 0.3162755072116852
Validation loss = 0.3233274221420288
Validation loss = 0.31474894285202026
Validation loss = 0.31922680139541626
Validation loss = 0.33636170625686646
Validation loss = 0.3623148798942566
Validation loss = 0.33170846104621887
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 25.
Path 2 | total_timesteps 39.
Path 3 | total_timesteps 48.
Path 4 | total_timesteps 64.
Path 5 | total_timesteps 75.
Path 6 | total_timesteps 88.
Path 7 | total_timesteps 107.
Path 8 | total_timesteps 116.
Path 9 | total_timesteps 135.
Path 10 | total_timesteps 157.
Path 11 | total_timesteps 170.
Path 12 | total_timesteps 177.
Path 13 | total_timesteps 198.
Path 14 | total_timesteps 213.
Path 15 | total_timesteps 229.
Path 16 | total_timesteps 250.
Path 17 | total_timesteps 260.
Path 18 | total_timesteps 270.
Path 19 | total_timesteps 282.
Path 20 | total_timesteps 292.
Path 21 | total_timesteps 312.
Path 22 | total_timesteps 320.
Path 23 | total_timesteps 333.
Path 24 | total_timesteps 341.
Path 25 | total_timesteps 356.
Path 26 | total_timesteps 365.
Path 27 | total_timesteps 379.
Path 28 | total_timesteps 393.
Path 29 | total_timesteps 400.
Path 30 | total_timesteps 414.
Path 31 | total_timesteps 428.
Path 32 | total_timesteps 441.
Path 33 | total_timesteps 459.
Path 34 | total_timesteps 479.
Path 35 | total_timesteps 488.
Path 36 | total_timesteps 497.
Path 37 | total_timesteps 509.
Path 38 | total_timesteps 525.
Path 39 | total_timesteps 535.
Path 40 | total_timesteps 543.
Path 41 | total_timesteps 558.
Path 42 | total_timesteps 574.
Path 43 | total_timesteps 581.
Path 44 | total_timesteps 589.
Path 45 | total_timesteps 601.
Path 46 | total_timesteps 612.
Path 47 | total_timesteps 624.
Path 48 | total_timesteps 640.
Path 49 | total_timesteps 666.
Path 50 | total_timesteps 673.
Path 51 | total_timesteps 687.
Path 52 | total_timesteps 697.
Path 53 | total_timesteps 708.
Path 54 | total_timesteps 715.
Path 55 | total_timesteps 726.
Path 56 | total_timesteps 734.
Path 57 | total_timesteps 760.
Path 58 | total_timesteps 774.
Path 59 | total_timesteps 789.
Path 60 | total_timesteps 800.
Path 61 | total_timesteps 811.
Path 62 | total_timesteps 818.
Path 63 | total_timesteps 828.
Path 64 | total_timesteps 836.
Path 65 | total_timesteps 843.
Path 66 | total_timesteps 860.
Path 67 | total_timesteps 873.
Path 68 | total_timesteps 887.
Path 69 | total_timesteps 899.
Path 70 | total_timesteps 907.
Path 71 | total_timesteps 918.
Path 72 | total_timesteps 928.
Path 73 | total_timesteps 950.
Path 74 | total_timesteps 959.
Path 75 | total_timesteps 970.
Path 76 | total_timesteps 980.
Path 77 | total_timesteps 992.
Path 78 | total_timesteps 1000.
Path 79 | total_timesteps 1012.
Path 80 | total_timesteps 1020.
Path 81 | total_timesteps 1031.
Path 82 | total_timesteps 1040.
Path 83 | total_timesteps 1062.
Path 84 | total_timesteps 1071.
Path 85 | total_timesteps 1085.
Path 86 | total_timesteps 1094.
Path 87 | total_timesteps 1104.
Path 88 | total_timesteps 1116.
Path 89 | total_timesteps 1126.
Path 90 | total_timesteps 1145.
Path 91 | total_timesteps 1158.
Path 92 | total_timesteps 1166.
Path 93 | total_timesteps 1182.
Path 94 | total_timesteps 1195.
Path 95 | total_timesteps 1211.
Path 96 | total_timesteps 1224.
Path 97 | total_timesteps 1235.
Path 98 | total_timesteps 1244.
Path 99 | total_timesteps 1254.
Path 100 | total_timesteps 1273.
Path 101 | total_timesteps 1283.
Path 102 | total_timesteps 1302.
Path 103 | total_timesteps 1309.
Path 104 | total_timesteps 1322.
Path 105 | total_timesteps 1330.
Path 106 | total_timesteps 1346.
Path 107 | total_timesteps 1357.
Path 108 | total_timesteps 1370.
Path 109 | total_timesteps 1382.
Path 110 | total_timesteps 1397.
Path 111 | total_timesteps 1415.
Path 112 | total_timesteps 1425.
Path 113 | total_timesteps 1436.
Path 114 | total_timesteps 1451.
Path 115 | total_timesteps 1470.
Path 116 | total_timesteps 1479.
Path 117 | total_timesteps 1495.
Path 118 | total_timesteps 1506.
Path 119 | total_timesteps 1518.
Path 120 | total_timesteps 1526.
Path 121 | total_timesteps 1535.
Path 122 | total_timesteps 1543.
Path 123 | total_timesteps 1555.
Path 124 | total_timesteps 1580.
Path 125 | total_timesteps 1587.
Path 126 | total_timesteps 1596.
Path 127 | total_timesteps 1605.
Path 128 | total_timesteps 1616.
Path 129 | total_timesteps 1626.
Path 130 | total_timesteps 1636.
Path 131 | total_timesteps 1647.
Path 132 | total_timesteps 1664.
Path 133 | total_timesteps 1682.
Path 134 | total_timesteps 1699.
Path 135 | total_timesteps 1708.
Path 136 | total_timesteps 1717.
Path 137 | total_timesteps 1730.
Path 138 | total_timesteps 1743.
Path 139 | total_timesteps 1767.
Path 140 | total_timesteps 1786.
Path 141 | total_timesteps 1794.
Path 142 | total_timesteps 1804.
Path 143 | total_timesteps 1821.
Path 144 | total_timesteps 1827.
Path 145 | total_timesteps 1838.
Path 146 | total_timesteps 1849.
Path 147 | total_timesteps 1860.
Path 148 | total_timesteps 1875.
Path 149 | total_timesteps 1887.
Path 150 | total_timesteps 1917.
Path 151 | total_timesteps 1925.
Path 152 | total_timesteps 1947.
Path 153 | total_timesteps 1961.
Path 154 | total_timesteps 1969.
Path 155 | total_timesteps 1979.
Path 156 | total_timesteps 1987.
Path 157 | total_timesteps 1999.
Path 158 | total_timesteps 2013.
Path 159 | total_timesteps 2028.
Path 160 | total_timesteps 2040.
Path 161 | total_timesteps 2057.
Path 162 | total_timesteps 2075.
Path 163 | total_timesteps 2083.
Path 164 | total_timesteps 2091.
Path 165 | total_timesteps 2102.
Path 166 | total_timesteps 2111.
Path 167 | total_timesteps 2118.
Path 168 | total_timesteps 2127.
Path 169 | total_timesteps 2140.
Path 170 | total_timesteps 2147.
Path 171 | total_timesteps 2156.
Path 172 | total_timesteps 2170.
Path 173 | total_timesteps 2181.
Path 174 | total_timesteps 2190.
Path 175 | total_timesteps 2204.
Path 176 | total_timesteps 2217.
Path 177 | total_timesteps 2230.
Path 178 | total_timesteps 2238.
Path 179 | total_timesteps 2246.
Path 180 | total_timesteps 2267.
Path 181 | total_timesteps 2276.
Path 182 | total_timesteps 2287.
Path 183 | total_timesteps 2295.
Path 184 | total_timesteps 2315.
Path 185 | total_timesteps 2325.
Path 186 | total_timesteps 2335.
Path 187 | total_timesteps 2347.
Path 188 | total_timesteps 2359.
Path 189 | total_timesteps 2366.
Path 190 | total_timesteps 2377.
Path 191 | total_timesteps 2400.
Path 192 | total_timesteps 2410.
Path 193 | total_timesteps 2425.
Path 194 | total_timesteps 2435.
Path 195 | total_timesteps 2453.
Path 196 | total_timesteps 2471.
Path 197 | total_timesteps 2483.
Path 198 | total_timesteps 2498.
Path 199 | total_timesteps 2525.
Path 200 | total_timesteps 2542.
Path 201 | total_timesteps 2558.
Path 202 | total_timesteps 2570.
Path 203 | total_timesteps 2580.
Path 204 | total_timesteps 2588.
Path 205 | total_timesteps 2598.
Path 206 | total_timesteps 2619.
Path 207 | total_timesteps 2648.
Path 208 | total_timesteps 2662.
Path 209 | total_timesteps 2676.
Path 210 | total_timesteps 2687.
Path 211 | total_timesteps 2709.
Path 212 | total_timesteps 2720.
Path 213 | total_timesteps 2728.
Path 214 | total_timesteps 2736.
Path 215 | total_timesteps 2745.
Path 216 | total_timesteps 2758.
Path 217 | total_timesteps 2770.
Path 218 | total_timesteps 2777.
Path 219 | total_timesteps 2789.
Path 220 | total_timesteps 2799.
Path 221 | total_timesteps 2814.
Path 222 | total_timesteps 2823.
Path 223 | total_timesteps 2838.
Path 224 | total_timesteps 2853.
Path 225 | total_timesteps 2868.
Path 226 | total_timesteps 2883.
Path 227 | total_timesteps 2890.
Path 228 | total_timesteps 2915.
Path 229 | total_timesteps 2926.
Path 230 | total_timesteps 2939.
Path 231 | total_timesteps 2948.
Path 232 | total_timesteps 2963.
Path 233 | total_timesteps 2974.
Path 234 | total_timesteps 2985.
Path 235 | total_timesteps 3001.
Path 236 | total_timesteps 3013.
Path 237 | total_timesteps 3022.
Path 238 | total_timesteps 3032.
Path 239 | total_timesteps 3045.
Path 240 | total_timesteps 3060.
Path 241 | total_timesteps 3069.
Path 242 | total_timesteps 3080.
Path 243 | total_timesteps 3095.
Path 244 | total_timesteps 3104.
Path 245 | total_timesteps 3113.
Path 246 | total_timesteps 3126.
Path 247 | total_timesteps 3135.
Path 248 | total_timesteps 3150.
Path 249 | total_timesteps 3160.
Path 250 | total_timesteps 3170.
Path 251 | total_timesteps 3177.
Path 252 | total_timesteps 3184.
Path 253 | total_timesteps 3195.
Path 254 | total_timesteps 3203.
Path 255 | total_timesteps 3219.
Path 256 | total_timesteps 3234.
Path 257 | total_timesteps 3249.
Path 258 | total_timesteps 3265.
Path 259 | total_timesteps 3285.
Path 260 | total_timesteps 3293.
Path 261 | total_timesteps 3305.
Path 262 | total_timesteps 3321.
Path 263 | total_timesteps 3337.
Path 264 | total_timesteps 3355.
Path 265 | total_timesteps 3363.
Path 266 | total_timesteps 3374.
Path 267 | total_timesteps 3383.
Path 268 | total_timesteps 3403.
Path 269 | total_timesteps 3413.
Path 270 | total_timesteps 3434.
Path 271 | total_timesteps 3446.
Path 272 | total_timesteps 3458.
Path 273 | total_timesteps 3476.
Path 274 | total_timesteps 3490.
Path 275 | total_timesteps 3504.
Path 276 | total_timesteps 3517.
Path 277 | total_timesteps 3530.
Path 278 | total_timesteps 3538.
Path 279 | total_timesteps 3545.
Path 280 | total_timesteps 3556.
Path 281 | total_timesteps 3565.
Path 282 | total_timesteps 3577.
Path 283 | total_timesteps 3585.
Path 284 | total_timesteps 3597.
Path 285 | total_timesteps 3607.
Path 286 | total_timesteps 3623.
Path 287 | total_timesteps 3633.
Path 288 | total_timesteps 3645.
Path 289 | total_timesteps 3654.
Path 290 | total_timesteps 3661.
Path 291 | total_timesteps 3678.
Path 292 | total_timesteps 3697.
Path 293 | total_timesteps 3707.
Path 294 | total_timesteps 3724.
Path 295 | total_timesteps 3739.
Path 296 | total_timesteps 3753.
Path 297 | total_timesteps 3767.
Path 298 | total_timesteps 3777.
Path 299 | total_timesteps 3789.
Path 300 | total_timesteps 3798.
Path 301 | total_timesteps 3816.
Path 302 | total_timesteps 3825.
Path 303 | total_timesteps 3833.
Path 304 | total_timesteps 3844.
Path 305 | total_timesteps 3858.
Path 306 | total_timesteps 3873.
Path 307 | total_timesteps 3885.
Path 308 | total_timesteps 3892.
Path 309 | total_timesteps 3909.
Path 310 | total_timesteps 3920.
Path 311 | total_timesteps 3934.
Path 312 | total_timesteps 3943.
Path 313 | total_timesteps 3954.
Path 314 | total_timesteps 3962.
Path 315 | total_timesteps 3974.
Path 316 | total_timesteps 3984.
Path 317 | total_timesteps 3995.
Path 318 | total_timesteps 4004.
Path 319 | total_timesteps 4018.
Path 320 | total_timesteps 4027.
Path 321 | total_timesteps 4036.
Path 322 | total_timesteps 4069.
Path 323 | total_timesteps 4086.
Path 324 | total_timesteps 4094.
Path 325 | total_timesteps 4103.
Path 326 | total_timesteps 4110.
Path 327 | total_timesteps 4123.
Path 328 | total_timesteps 4133.
Path 329 | total_timesteps 4142.
Path 330 | total_timesteps 4151.
Path 331 | total_timesteps 4160.
Path 332 | total_timesteps 4173.
Path 333 | total_timesteps 4186.
Path 334 | total_timesteps 4197.
Path 335 | total_timesteps 4207.
Path 336 | total_timesteps 4221.
Path 337 | total_timesteps 4229.
Path 338 | total_timesteps 4240.
Path 339 | total_timesteps 4250.
Path 340 | total_timesteps 4257.
Path 341 | total_timesteps 4266.
Path 342 | total_timesteps 4274.
Path 343 | total_timesteps 4312.
Path 344 | total_timesteps 4322.
Path 345 | total_timesteps 4335.
Path 346 | total_timesteps 4342.
Path 347 | total_timesteps 4354.
Path 348 | total_timesteps 4376.
Path 349 | total_timesteps 4387.
Path 350 | total_timesteps 4401.
Path 351 | total_timesteps 4410.
Path 352 | total_timesteps 4422.
Path 353 | total_timesteps 4440.
Path 354 | total_timesteps 4448.
Path 355 | total_timesteps 4456.
Path 356 | total_timesteps 4474.
Path 357 | total_timesteps 4484.
Path 358 | total_timesteps 4493.
Path 359 | total_timesteps 4508.
Path 360 | total_timesteps 4518.
Path 361 | total_timesteps 4531.
Path 362 | total_timesteps 4542.
Path 363 | total_timesteps 4549.
Path 364 | total_timesteps 4558.
Path 365 | total_timesteps 4566.
Path 366 | total_timesteps 4580.
Path 367 | total_timesteps 4600.
Path 368 | total_timesteps 4611.
Path 369 | total_timesteps 4622.
Path 370 | total_timesteps 4631.
Path 371 | total_timesteps 4656.
Path 372 | total_timesteps 4664.
Path 373 | total_timesteps 4676.
Path 374 | total_timesteps 4694.
Path 375 | total_timesteps 4705.
Path 376 | total_timesteps 4722.
Path 377 | total_timesteps 4739.
Path 378 | total_timesteps 4747.
Path 379 | total_timesteps 4757.
Path 380 | total_timesteps 4770.
Path 381 | total_timesteps 4784.
Path 382 | total_timesteps 4800.
Path 383 | total_timesteps 4831.
Path 384 | total_timesteps 4842.
Path 385 | total_timesteps 4853.
Path 386 | total_timesteps 4861.
Path 387 | total_timesteps 4873.
Path 388 | total_timesteps 4883.
Path 389 | total_timesteps 4896.
Path 390 | total_timesteps 4907.
Path 391 | total_timesteps 4916.
Path 392 | total_timesteps 4928.
Path 393 | total_timesteps 4943.
Path 394 | total_timesteps 4954.
Path 395 | total_timesteps 4963.
Path 396 | total_timesteps 4975.
Path 397 | total_timesteps 4996.
Path 398 | total_timesteps 5011.
Path 399 | total_timesteps 5018.
Path 400 | total_timesteps 5027.
Path 401 | total_timesteps 5036.
Path 402 | total_timesteps 5046.
Path 403 | total_timesteps 5054.
Path 404 | total_timesteps 5075.
Path 405 | total_timesteps 5093.
Path 406 | total_timesteps 5109.
Path 407 | total_timesteps 5120.
Path 408 | total_timesteps 5130.
Path 409 | total_timesteps 5144.
Path 410 | total_timesteps 5158.
Path 411 | total_timesteps 5168.
Path 412 | total_timesteps 5177.
Path 413 | total_timesteps 5197.
Path 414 | total_timesteps 5207.
Path 415 | total_timesteps 5219.
Path 416 | total_timesteps 5230.
Path 417 | total_timesteps 5250.
Path 418 | total_timesteps 5260.
Path 419 | total_timesteps 5268.
Path 420 | total_timesteps 5277.
Path 421 | total_timesteps 5289.
Path 422 | total_timesteps 5301.
Path 423 | total_timesteps 5321.
Path 424 | total_timesteps 5357.
Path 425 | total_timesteps 5371.
Path 426 | total_timesteps 5381.
Path 427 | total_timesteps 5402.
Path 428 | total_timesteps 5419.
Path 429 | total_timesteps 5430.
Path 430 | total_timesteps 5438.
Path 431 | total_timesteps 5446.
Path 432 | total_timesteps 5461.
Path 433 | total_timesteps 5469.
Path 434 | total_timesteps 5480.
Path 435 | total_timesteps 5488.
Path 436 | total_timesteps 5496.
Path 437 | total_timesteps 5509.
Path 438 | total_timesteps 5522.
Path 439 | total_timesteps 5530.
Path 440 | total_timesteps 5544.
Path 441 | total_timesteps 5569.
Path 442 | total_timesteps 5577.
Path 443 | total_timesteps 5591.
Path 444 | total_timesteps 5602.
Path 445 | total_timesteps 5617.
Path 446 | total_timesteps 5626.
Path 447 | total_timesteps 5644.
Path 448 | total_timesteps 5661.
Path 449 | total_timesteps 5673.
Path 450 | total_timesteps 5686.
Path 451 | total_timesteps 5705.
Path 452 | total_timesteps 5731.
Path 453 | total_timesteps 5748.
Path 454 | total_timesteps 5761.
Path 455 | total_timesteps 5771.
Path 456 | total_timesteps 5793.
Path 457 | total_timesteps 5820.
Path 458 | total_timesteps 5842.
Path 459 | total_timesteps 5854.
Path 460 | total_timesteps 5864.
Path 461 | total_timesteps 5881.
Path 462 | total_timesteps 5891.
Path 463 | total_timesteps 5899.
Path 464 | total_timesteps 5908.
Path 465 | total_timesteps 5923.
Path 466 | total_timesteps 5938.
Path 467 | total_timesteps 5948.
Path 468 | total_timesteps 5957.
Path 469 | total_timesteps 5965.
Path 470 | total_timesteps 5973.
Path 471 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.07    |
| Iteration     | 0        |
| MaximumReturn | 6.78     |
| MinimumReturn | -20.9    |
| TotalSamples  | 8023     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.33086317777633667
Validation loss = 0.30562806129455566
Validation loss = 0.30051735043525696
Validation loss = 0.30876147747039795
Validation loss = 0.3055386245250702
Validation loss = 0.3168272376060486
Validation loss = 0.32398301362991333
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 30.
Path 3 | total_timesteps 42.
Path 4 | total_timesteps 53.
Path 5 | total_timesteps 69.
Path 6 | total_timesteps 85.
Path 7 | total_timesteps 97.
Path 8 | total_timesteps 108.
Path 9 | total_timesteps 118.
Path 10 | total_timesteps 142.
Path 11 | total_timesteps 152.
Path 12 | total_timesteps 167.
Path 13 | total_timesteps 198.
Path 14 | total_timesteps 209.
Path 15 | total_timesteps 219.
Path 16 | total_timesteps 239.
Path 17 | total_timesteps 251.
Path 18 | total_timesteps 262.
Path 19 | total_timesteps 271.
Path 20 | total_timesteps 293.
Path 21 | total_timesteps 301.
Path 22 | total_timesteps 311.
Path 23 | total_timesteps 325.
Path 24 | total_timesteps 341.
Path 25 | total_timesteps 355.
Path 26 | total_timesteps 370.
Path 27 | total_timesteps 378.
Path 28 | total_timesteps 392.
Path 29 | total_timesteps 412.
Path 30 | total_timesteps 422.
Path 31 | total_timesteps 449.
Path 32 | total_timesteps 458.
Path 33 | total_timesteps 466.
Path 34 | total_timesteps 482.
Path 35 | total_timesteps 491.
Path 36 | total_timesteps 507.
Path 37 | total_timesteps 517.
Path 38 | total_timesteps 528.
Path 39 | total_timesteps 541.
Path 40 | total_timesteps 555.
Path 41 | total_timesteps 563.
Path 42 | total_timesteps 573.
Path 43 | total_timesteps 583.
Path 44 | total_timesteps 593.
Path 45 | total_timesteps 601.
Path 46 | total_timesteps 611.
Path 47 | total_timesteps 628.
Path 48 | total_timesteps 645.
Path 49 | total_timesteps 656.
Path 50 | total_timesteps 666.
Path 51 | total_timesteps 676.
Path 52 | total_timesteps 684.
Path 53 | total_timesteps 693.
Path 54 | total_timesteps 701.
Path 55 | total_timesteps 712.
Path 56 | total_timesteps 724.
Path 57 | total_timesteps 737.
Path 58 | total_timesteps 746.
Path 59 | total_timesteps 753.
Path 60 | total_timesteps 763.
Path 61 | total_timesteps 782.
Path 62 | total_timesteps 789.
Path 63 | total_timesteps 803.
Path 64 | total_timesteps 817.
Path 65 | total_timesteps 829.
Path 66 | total_timesteps 838.
Path 67 | total_timesteps 862.
Path 68 | total_timesteps 887.
Path 69 | total_timesteps 900.
Path 70 | total_timesteps 919.
Path 71 | total_timesteps 931.
Path 72 | total_timesteps 940.
Path 73 | total_timesteps 948.
Path 74 | total_timesteps 958.
Path 75 | total_timesteps 974.
Path 76 | total_timesteps 984.
Path 77 | total_timesteps 995.
Path 78 | total_timesteps 1002.
Path 79 | total_timesteps 1014.
Path 80 | total_timesteps 1022.
Path 81 | total_timesteps 1032.
Path 82 | total_timesteps 1048.
Path 83 | total_timesteps 1059.
Path 84 | total_timesteps 1069.
Path 85 | total_timesteps 1076.
Path 86 | total_timesteps 1086.
Path 87 | total_timesteps 1094.
Path 88 | total_timesteps 1103.
Path 89 | total_timesteps 1114.
Path 90 | total_timesteps 1129.
Path 91 | total_timesteps 1139.
Path 92 | total_timesteps 1154.
Path 93 | total_timesteps 1166.
Path 94 | total_timesteps 1175.
Path 95 | total_timesteps 1183.
Path 96 | total_timesteps 1201.
Path 97 | total_timesteps 1211.
Path 98 | total_timesteps 1221.
Path 99 | total_timesteps 1232.
Path 100 | total_timesteps 1244.
Path 101 | total_timesteps 1259.
Path 102 | total_timesteps 1281.
Path 103 | total_timesteps 1292.
Path 104 | total_timesteps 1308.
Path 105 | total_timesteps 1323.
Path 106 | total_timesteps 1337.
Path 107 | total_timesteps 1347.
Path 108 | total_timesteps 1365.
Path 109 | total_timesteps 1378.
Path 110 | total_timesteps 1387.
Path 111 | total_timesteps 1412.
Path 112 | total_timesteps 1423.
Path 113 | total_timesteps 1433.
Path 114 | total_timesteps 1441.
Path 115 | total_timesteps 1458.
Path 116 | total_timesteps 1470.
Path 117 | total_timesteps 1487.
Path 118 | total_timesteps 1496.
Path 119 | total_timesteps 1505.
Path 120 | total_timesteps 1516.
Path 121 | total_timesteps 1531.
Path 122 | total_timesteps 1541.
Path 123 | total_timesteps 1557.
Path 124 | total_timesteps 1567.
Path 125 | total_timesteps 1575.
Path 126 | total_timesteps 1586.
Path 127 | total_timesteps 1594.
Path 128 | total_timesteps 1607.
Path 129 | total_timesteps 1617.
Path 130 | total_timesteps 1632.
Path 131 | total_timesteps 1640.
Path 132 | total_timesteps 1652.
Path 133 | total_timesteps 1677.
Path 134 | total_timesteps 1689.
Path 135 | total_timesteps 1699.
Path 136 | total_timesteps 1707.
Path 137 | total_timesteps 1717.
Path 138 | total_timesteps 1730.
Path 139 | total_timesteps 1739.
Path 140 | total_timesteps 1749.
Path 141 | total_timesteps 1764.
Path 142 | total_timesteps 1777.
Path 143 | total_timesteps 1811.
Path 144 | total_timesteps 1821.
Path 145 | total_timesteps 1842.
Path 146 | total_timesteps 1850.
Path 147 | total_timesteps 1868.
Path 148 | total_timesteps 1878.
Path 149 | total_timesteps 1891.
Path 150 | total_timesteps 1899.
Path 151 | total_timesteps 1911.
Path 152 | total_timesteps 1922.
Path 153 | total_timesteps 1936.
Path 154 | total_timesteps 1946.
Path 155 | total_timesteps 1958.
Path 156 | total_timesteps 1965.
Path 157 | total_timesteps 1977.
Path 158 | total_timesteps 1987.
Path 159 | total_timesteps 2000.
Path 160 | total_timesteps 2008.
Path 161 | total_timesteps 2016.
Path 162 | total_timesteps 2029.
Path 163 | total_timesteps 2051.
Path 164 | total_timesteps 2062.
Path 165 | total_timesteps 2075.
Path 166 | total_timesteps 2085.
Path 167 | total_timesteps 2093.
Path 168 | total_timesteps 2105.
Path 169 | total_timesteps 2115.
Path 170 | total_timesteps 2122.
Path 171 | total_timesteps 2131.
Path 172 | total_timesteps 2143.
Path 173 | total_timesteps 2155.
Path 174 | total_timesteps 2164.
Path 175 | total_timesteps 2173.
Path 176 | total_timesteps 2185.
Path 177 | total_timesteps 2193.
Path 178 | total_timesteps 2207.
Path 179 | total_timesteps 2219.
Path 180 | total_timesteps 2227.
Path 181 | total_timesteps 2242.
Path 182 | total_timesteps 2251.
Path 183 | total_timesteps 2261.
Path 184 | total_timesteps 2270.
Path 185 | total_timesteps 2281.
Path 186 | total_timesteps 2295.
Path 187 | total_timesteps 2310.
Path 188 | total_timesteps 2320.
Path 189 | total_timesteps 2338.
Path 190 | total_timesteps 2346.
Path 191 | total_timesteps 2355.
Path 192 | total_timesteps 2364.
Path 193 | total_timesteps 2372.
Path 194 | total_timesteps 2384.
Path 195 | total_timesteps 2401.
Path 196 | total_timesteps 2412.
Path 197 | total_timesteps 2439.
Path 198 | total_timesteps 2452.
Path 199 | total_timesteps 2461.
Path 200 | total_timesteps 2476.
Path 201 | total_timesteps 2486.
Path 202 | total_timesteps 2499.
Path 203 | total_timesteps 2510.
Path 204 | total_timesteps 2520.
Path 205 | total_timesteps 2530.
Path 206 | total_timesteps 2541.
Path 207 | total_timesteps 2550.
Path 208 | total_timesteps 2560.
Path 209 | total_timesteps 2571.
Path 210 | total_timesteps 2587.
Path 211 | total_timesteps 2598.
Path 212 | total_timesteps 2607.
Path 213 | total_timesteps 2619.
Path 214 | total_timesteps 2630.
Path 215 | total_timesteps 2640.
Path 216 | total_timesteps 2655.
Path 217 | total_timesteps 2669.
Path 218 | total_timesteps 2691.
Path 219 | total_timesteps 2720.
Path 220 | total_timesteps 2728.
Path 221 | total_timesteps 2737.
Path 222 | total_timesteps 2751.
Path 223 | total_timesteps 2767.
Path 224 | total_timesteps 2779.
Path 225 | total_timesteps 2789.
Path 226 | total_timesteps 2796.
Path 227 | total_timesteps 2812.
Path 228 | total_timesteps 2823.
Path 229 | total_timesteps 2836.
Path 230 | total_timesteps 2844.
Path 231 | total_timesteps 2857.
Path 232 | total_timesteps 2868.
Path 233 | total_timesteps 2878.
Path 234 | total_timesteps 2891.
Path 235 | total_timesteps 2905.
Path 236 | total_timesteps 2915.
Path 237 | total_timesteps 2927.
Path 238 | total_timesteps 2944.
Path 239 | total_timesteps 2952.
Path 240 | total_timesteps 2964.
Path 241 | total_timesteps 2973.
Path 242 | total_timesteps 2987.
Path 243 | total_timesteps 2995.
Path 244 | total_timesteps 3002.
Path 245 | total_timesteps 3011.
Path 246 | total_timesteps 3022.
Path 247 | total_timesteps 3031.
Path 248 | total_timesteps 3041.
Path 249 | total_timesteps 3072.
Path 250 | total_timesteps 3080.
Path 251 | total_timesteps 3090.
Path 252 | total_timesteps 3097.
Path 253 | total_timesteps 3104.
Path 254 | total_timesteps 3115.
Path 255 | total_timesteps 3122.
Path 256 | total_timesteps 3130.
Path 257 | total_timesteps 3143.
Path 258 | total_timesteps 3152.
Path 259 | total_timesteps 3161.
Path 260 | total_timesteps 3182.
Path 261 | total_timesteps 3190.
Path 262 | total_timesteps 3200.
Path 263 | total_timesteps 3210.
Path 264 | total_timesteps 3219.
Path 265 | total_timesteps 3232.
Path 266 | total_timesteps 3238.
Path 267 | total_timesteps 3248.
Path 268 | total_timesteps 3256.
Path 269 | total_timesteps 3276.
Path 270 | total_timesteps 3290.
Path 271 | total_timesteps 3297.
Path 272 | total_timesteps 3336.
Path 273 | total_timesteps 3347.
Path 274 | total_timesteps 3360.
Path 275 | total_timesteps 3371.
Path 276 | total_timesteps 3379.
Path 277 | total_timesteps 3387.
Path 278 | total_timesteps 3394.
Path 279 | total_timesteps 3404.
Path 280 | total_timesteps 3419.
Path 281 | total_timesteps 3434.
Path 282 | total_timesteps 3442.
Path 283 | total_timesteps 3453.
Path 284 | total_timesteps 3468.
Path 285 | total_timesteps 3479.
Path 286 | total_timesteps 3496.
Path 287 | total_timesteps 3509.
Path 288 | total_timesteps 3517.
Path 289 | total_timesteps 3529.
Path 290 | total_timesteps 3539.
Path 291 | total_timesteps 3548.
Path 292 | total_timesteps 3557.
Path 293 | total_timesteps 3575.
Path 294 | total_timesteps 3585.
Path 295 | total_timesteps 3598.
Path 296 | total_timesteps 3605.
Path 297 | total_timesteps 3612.
Path 298 | total_timesteps 3625.
Path 299 | total_timesteps 3635.
Path 300 | total_timesteps 3651.
Path 301 | total_timesteps 3661.
Path 302 | total_timesteps 3669.
Path 303 | total_timesteps 3678.
Path 304 | total_timesteps 3687.
Path 305 | total_timesteps 3703.
Path 306 | total_timesteps 3720.
Path 307 | total_timesteps 3738.
Path 308 | total_timesteps 3754.
Path 309 | total_timesteps 3762.
Path 310 | total_timesteps 3771.
Path 311 | total_timesteps 3781.
Path 312 | total_timesteps 3792.
Path 313 | total_timesteps 3798.
Path 314 | total_timesteps 3816.
Path 315 | total_timesteps 3835.
Path 316 | total_timesteps 3847.
Path 317 | total_timesteps 3861.
Path 318 | total_timesteps 3871.
Path 319 | total_timesteps 3880.
Path 320 | total_timesteps 3894.
Path 321 | total_timesteps 3904.
Path 322 | total_timesteps 3916.
Path 323 | total_timesteps 3946.
Path 324 | total_timesteps 3959.
Path 325 | total_timesteps 3971.
Path 326 | total_timesteps 3991.
Path 327 | total_timesteps 4007.
Path 328 | total_timesteps 4014.
Path 329 | total_timesteps 4027.
Path 330 | total_timesteps 4038.
Path 331 | total_timesteps 4048.
Path 332 | total_timesteps 4061.
Path 333 | total_timesteps 4071.
Path 334 | total_timesteps 4083.
Path 335 | total_timesteps 4093.
Path 336 | total_timesteps 4105.
Path 337 | total_timesteps 4116.
Path 338 | total_timesteps 4122.
Path 339 | total_timesteps 4132.
Path 340 | total_timesteps 4145.
Path 341 | total_timesteps 4163.
Path 342 | total_timesteps 4182.
Path 343 | total_timesteps 4195.
Path 344 | total_timesteps 4205.
Path 345 | total_timesteps 4216.
Path 346 | total_timesteps 4233.
Path 347 | total_timesteps 4249.
Path 348 | total_timesteps 4261.
Path 349 | total_timesteps 4272.
Path 350 | total_timesteps 4282.
Path 351 | total_timesteps 4300.
Path 352 | total_timesteps 4310.
Path 353 | total_timesteps 4319.
Path 354 | total_timesteps 4330.
Path 355 | total_timesteps 4340.
Path 356 | total_timesteps 4350.
Path 357 | total_timesteps 4364.
Path 358 | total_timesteps 4372.
Path 359 | total_timesteps 4388.
Path 360 | total_timesteps 4398.
Path 361 | total_timesteps 4406.
Path 362 | total_timesteps 4422.
Path 363 | total_timesteps 4430.
Path 364 | total_timesteps 4440.
Path 365 | total_timesteps 4452.
Path 366 | total_timesteps 4464.
Path 367 | total_timesteps 4473.
Path 368 | total_timesteps 4490.
Path 369 | total_timesteps 4499.
Path 370 | total_timesteps 4508.
Path 371 | total_timesteps 4515.
Path 372 | total_timesteps 4526.
Path 373 | total_timesteps 4536.
Path 374 | total_timesteps 4548.
Path 375 | total_timesteps 4562.
Path 376 | total_timesteps 4571.
Path 377 | total_timesteps 4581.
Path 378 | total_timesteps 4591.
Path 379 | total_timesteps 4601.
Path 380 | total_timesteps 4621.
Path 381 | total_timesteps 4629.
Path 382 | total_timesteps 4641.
Path 383 | total_timesteps 4657.
Path 384 | total_timesteps 4667.
Path 385 | total_timesteps 4678.
Path 386 | total_timesteps 4694.
Path 387 | total_timesteps 4703.
Path 388 | total_timesteps 4712.
Path 389 | total_timesteps 4725.
Path 390 | total_timesteps 4733.
Path 391 | total_timesteps 4746.
Path 392 | total_timesteps 4755.
Path 393 | total_timesteps 4765.
Path 394 | total_timesteps 4776.
Path 395 | total_timesteps 4785.
Path 396 | total_timesteps 4801.
Path 397 | total_timesteps 4810.
Path 398 | total_timesteps 4817.
Path 399 | total_timesteps 4829.
Path 400 | total_timesteps 4840.
Path 401 | total_timesteps 4849.
Path 402 | total_timesteps 4864.
Path 403 | total_timesteps 4875.
Path 404 | total_timesteps 4882.
Path 405 | total_timesteps 4890.
Path 406 | total_timesteps 4900.
Path 407 | total_timesteps 4910.
Path 408 | total_timesteps 4918.
Path 409 | total_timesteps 4925.
Path 410 | total_timesteps 4938.
Path 411 | total_timesteps 4951.
Path 412 | total_timesteps 4959.
Path 413 | total_timesteps 4975.
Path 414 | total_timesteps 4984.
Path 415 | total_timesteps 4994.
Path 416 | total_timesteps 5008.
Path 417 | total_timesteps 5025.
Path 418 | total_timesteps 5034.
Path 419 | total_timesteps 5041.
Path 420 | total_timesteps 5053.
Path 421 | total_timesteps 5064.
Path 422 | total_timesteps 5074.
Path 423 | total_timesteps 5082.
Path 424 | total_timesteps 5091.
Path 425 | total_timesteps 5099.
Path 426 | total_timesteps 5110.
Path 427 | total_timesteps 5122.
Path 428 | total_timesteps 5129.
Path 429 | total_timesteps 5147.
Path 430 | total_timesteps 5157.
Path 431 | total_timesteps 5167.
Path 432 | total_timesteps 5175.
Path 433 | total_timesteps 5184.
Path 434 | total_timesteps 5193.
Path 435 | total_timesteps 5202.
Path 436 | total_timesteps 5212.
Path 437 | total_timesteps 5250.
Path 438 | total_timesteps 5267.
Path 439 | total_timesteps 5279.
Path 440 | total_timesteps 5286.
Path 441 | total_timesteps 5297.
Path 442 | total_timesteps 5310.
Path 443 | total_timesteps 5319.
Path 444 | total_timesteps 5328.
Path 445 | total_timesteps 5336.
Path 446 | total_timesteps 5348.
Path 447 | total_timesteps 5361.
Path 448 | total_timesteps 5368.
Path 449 | total_timesteps 5375.
Path 450 | total_timesteps 5389.
Path 451 | total_timesteps 5402.
Path 452 | total_timesteps 5415.
Path 453 | total_timesteps 5426.
Path 454 | total_timesteps 5434.
Path 455 | total_timesteps 5444.
Path 456 | total_timesteps 5452.
Path 457 | total_timesteps 5461.
Path 458 | total_timesteps 5473.
Path 459 | total_timesteps 5484.
Path 460 | total_timesteps 5495.
Path 461 | total_timesteps 5503.
Path 462 | total_timesteps 5526.
Path 463 | total_timesteps 5535.
Path 464 | total_timesteps 5544.
Path 465 | total_timesteps 5557.
Path 466 | total_timesteps 5577.
Path 467 | total_timesteps 5589.
Path 468 | total_timesteps 5604.
Path 469 | total_timesteps 5618.
Path 470 | total_timesteps 5632.
Path 471 | total_timesteps 5643.
Path 472 | total_timesteps 5651.
Path 473 | total_timesteps 5661.
Path 474 | total_timesteps 5668.
Path 475 | total_timesteps 5676.
Path 476 | total_timesteps 5686.
Path 477 | total_timesteps 5699.
Path 478 | total_timesteps 5707.
Path 479 | total_timesteps 5727.
Path 480 | total_timesteps 5739.
Path 481 | total_timesteps 5755.
Path 482 | total_timesteps 5765.
Path 483 | total_timesteps 5774.
Path 484 | total_timesteps 5789.
Path 485 | total_timesteps 5802.
Path 486 | total_timesteps 5814.
Path 487 | total_timesteps 5823.
Path 488 | total_timesteps 5832.
Path 489 | total_timesteps 5845.
Path 490 | total_timesteps 5856.
Path 491 | total_timesteps 5867.
Path 492 | total_timesteps 5880.
Path 493 | total_timesteps 5890.
Path 494 | total_timesteps 5900.
Path 495 | total_timesteps 5908.
Path 496 | total_timesteps 5925.
Path 497 | total_timesteps 5934.
Path 498 | total_timesteps 5954.
Path 499 | total_timesteps 5964.
Path 500 | total_timesteps 5981.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.56    |
| Iteration     | 1        |
| MaximumReturn | 14.7     |
| MinimumReturn | -19.9    |
| TotalSamples  | 12023    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.29571735858917236
Validation loss = 0.28891459107398987
Validation loss = 0.293576717376709
Validation loss = 0.30034223198890686
Validation loss = 0.3082023561000824
Validation loss = 0.31115397810935974
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 6.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 36.
Path 5 | total_timesteps 46.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 72.
Path 8 | total_timesteps 87.
Path 9 | total_timesteps 102.
Path 10 | total_timesteps 114.
Path 11 | total_timesteps 127.
Path 12 | total_timesteps 141.
Path 13 | total_timesteps 151.
Path 14 | total_timesteps 162.
Path 15 | total_timesteps 175.
Path 16 | total_timesteps 190.
Path 17 | total_timesteps 197.
Path 18 | total_timesteps 211.
Path 19 | total_timesteps 223.
Path 20 | total_timesteps 234.
Path 21 | total_timesteps 250.
Path 22 | total_timesteps 258.
Path 23 | total_timesteps 267.
Path 24 | total_timesteps 279.
Path 25 | total_timesteps 288.
Path 26 | total_timesteps 308.
Path 27 | total_timesteps 322.
Path 28 | total_timesteps 339.
Path 29 | total_timesteps 358.
Path 30 | total_timesteps 370.
Path 31 | total_timesteps 387.
Path 32 | total_timesteps 400.
Path 33 | total_timesteps 417.
Path 34 | total_timesteps 428.
Path 35 | total_timesteps 445.
Path 36 | total_timesteps 454.
Path 37 | total_timesteps 466.
Path 38 | total_timesteps 486.
Path 39 | total_timesteps 495.
Path 40 | total_timesteps 509.
Path 41 | total_timesteps 523.
Path 42 | total_timesteps 535.
Path 43 | total_timesteps 548.
Path 44 | total_timesteps 556.
Path 45 | total_timesteps 565.
Path 46 | total_timesteps 575.
Path 47 | total_timesteps 584.
Path 48 | total_timesteps 596.
Path 49 | total_timesteps 605.
Path 50 | total_timesteps 618.
Path 51 | total_timesteps 634.
Path 52 | total_timesteps 643.
Path 53 | total_timesteps 652.
Path 54 | total_timesteps 662.
Path 55 | total_timesteps 669.
Path 56 | total_timesteps 679.
Path 57 | total_timesteps 692.
Path 58 | total_timesteps 702.
Path 59 | total_timesteps 710.
Path 60 | total_timesteps 718.
Path 61 | total_timesteps 736.
Path 62 | total_timesteps 746.
Path 63 | total_timesteps 763.
Path 64 | total_timesteps 774.
Path 65 | total_timesteps 782.
Path 66 | total_timesteps 798.
Path 67 | total_timesteps 815.
Path 68 | total_timesteps 823.
Path 69 | total_timesteps 833.
Path 70 | total_timesteps 843.
Path 71 | total_timesteps 851.
Path 72 | total_timesteps 857.
Path 73 | total_timesteps 870.
Path 74 | total_timesteps 879.
Path 75 | total_timesteps 886.
Path 76 | total_timesteps 893.
Path 77 | total_timesteps 904.
Path 78 | total_timesteps 914.
Path 79 | total_timesteps 925.
Path 80 | total_timesteps 938.
Path 81 | total_timesteps 948.
Path 82 | total_timesteps 963.
Path 83 | total_timesteps 974.
Path 84 | total_timesteps 998.
Path 85 | total_timesteps 1009.
Path 86 | total_timesteps 1023.
Path 87 | total_timesteps 1032.
Path 88 | total_timesteps 1040.
Path 89 | total_timesteps 1053.
Path 90 | total_timesteps 1062.
Path 91 | total_timesteps 1071.
Path 92 | total_timesteps 1080.
Path 93 | total_timesteps 1094.
Path 94 | total_timesteps 1110.
Path 95 | total_timesteps 1125.
Path 96 | total_timesteps 1140.
Path 97 | total_timesteps 1151.
Path 98 | total_timesteps 1165.
Path 99 | total_timesteps 1176.
Path 100 | total_timesteps 1189.
Path 101 | total_timesteps 1202.
Path 102 | total_timesteps 1212.
Path 103 | total_timesteps 1223.
Path 104 | total_timesteps 1229.
Path 105 | total_timesteps 1253.
Path 106 | total_timesteps 1267.
Path 107 | total_timesteps 1277.
Path 108 | total_timesteps 1285.
Path 109 | total_timesteps 1293.
Path 110 | total_timesteps 1302.
Path 111 | total_timesteps 1310.
Path 112 | total_timesteps 1323.
Path 113 | total_timesteps 1340.
Path 114 | total_timesteps 1351.
Path 115 | total_timesteps 1362.
Path 116 | total_timesteps 1377.
Path 117 | total_timesteps 1384.
Path 118 | total_timesteps 1390.
Path 119 | total_timesteps 1398.
Path 120 | total_timesteps 1406.
Path 121 | total_timesteps 1419.
Path 122 | total_timesteps 1431.
Path 123 | total_timesteps 1446.
Path 124 | total_timesteps 1460.
Path 125 | total_timesteps 1467.
Path 126 | total_timesteps 1474.
Path 127 | total_timesteps 1485.
Path 128 | total_timesteps 1494.
Path 129 | total_timesteps 1504.
Path 130 | total_timesteps 1518.
Path 131 | total_timesteps 1529.
Path 132 | total_timesteps 1544.
Path 133 | total_timesteps 1553.
Path 134 | total_timesteps 1569.
Path 135 | total_timesteps 1578.
Path 136 | total_timesteps 1599.
Path 137 | total_timesteps 1614.
Path 138 | total_timesteps 1624.
Path 139 | total_timesteps 1633.
Path 140 | total_timesteps 1642.
Path 141 | total_timesteps 1655.
Path 142 | total_timesteps 1663.
Path 143 | total_timesteps 1671.
Path 144 | total_timesteps 1680.
Path 145 | total_timesteps 1689.
Path 146 | total_timesteps 1700.
Path 147 | total_timesteps 1710.
Path 148 | total_timesteps 1722.
Path 149 | total_timesteps 1732.
Path 150 | total_timesteps 1746.
Path 151 | total_timesteps 1754.
Path 152 | total_timesteps 1768.
Path 153 | total_timesteps 1775.
Path 154 | total_timesteps 1782.
Path 155 | total_timesteps 1798.
Path 156 | total_timesteps 1815.
Path 157 | total_timesteps 1830.
Path 158 | total_timesteps 1839.
Path 159 | total_timesteps 1848.
Path 160 | total_timesteps 1857.
Path 161 | total_timesteps 1875.
Path 162 | total_timesteps 1888.
Path 163 | total_timesteps 1903.
Path 164 | total_timesteps 1911.
Path 165 | total_timesteps 1923.
Path 166 | total_timesteps 1932.
Path 167 | total_timesteps 1942.
Path 168 | total_timesteps 1953.
Path 169 | total_timesteps 1964.
Path 170 | total_timesteps 1973.
Path 171 | total_timesteps 1984.
Path 172 | total_timesteps 1995.
Path 173 | total_timesteps 2003.
Path 174 | total_timesteps 2017.
Path 175 | total_timesteps 2025.
Path 176 | total_timesteps 2051.
Path 177 | total_timesteps 2069.
Path 178 | total_timesteps 2077.
Path 179 | total_timesteps 2091.
Path 180 | total_timesteps 2106.
Path 181 | total_timesteps 2115.
Path 182 | total_timesteps 2127.
Path 183 | total_timesteps 2142.
Path 184 | total_timesteps 2153.
Path 185 | total_timesteps 2163.
Path 186 | total_timesteps 2182.
Path 187 | total_timesteps 2190.
Path 188 | total_timesteps 2200.
Path 189 | total_timesteps 2209.
Path 190 | total_timesteps 2222.
Path 191 | total_timesteps 2233.
Path 192 | total_timesteps 2241.
Path 193 | total_timesteps 2250.
Path 194 | total_timesteps 2257.
Path 195 | total_timesteps 2270.
Path 196 | total_timesteps 2281.
Path 197 | total_timesteps 2297.
Path 198 | total_timesteps 2310.
Path 199 | total_timesteps 2321.
Path 200 | total_timesteps 2328.
Path 201 | total_timesteps 2339.
Path 202 | total_timesteps 2353.
Path 203 | total_timesteps 2366.
Path 204 | total_timesteps 2387.
Path 205 | total_timesteps 2399.
Path 206 | total_timesteps 2411.
Path 207 | total_timesteps 2421.
Path 208 | total_timesteps 2432.
Path 209 | total_timesteps 2442.
Path 210 | total_timesteps 2455.
Path 211 | total_timesteps 2467.
Path 212 | total_timesteps 2480.
Path 213 | total_timesteps 2491.
Path 214 | total_timesteps 2505.
Path 215 | total_timesteps 2515.
Path 216 | total_timesteps 2529.
Path 217 | total_timesteps 2536.
Path 218 | total_timesteps 2550.
Path 219 | total_timesteps 2566.
Path 220 | total_timesteps 2577.
Path 221 | total_timesteps 2588.
Path 222 | total_timesteps 2603.
Path 223 | total_timesteps 2612.
Path 224 | total_timesteps 2621.
Path 225 | total_timesteps 2634.
Path 226 | total_timesteps 2656.
Path 227 | total_timesteps 2664.
Path 228 | total_timesteps 2681.
Path 229 | total_timesteps 2693.
Path 230 | total_timesteps 2702.
Path 231 | total_timesteps 2710.
Path 232 | total_timesteps 2721.
Path 233 | total_timesteps 2730.
Path 234 | total_timesteps 2747.
Path 235 | total_timesteps 2757.
Path 236 | total_timesteps 2767.
Path 237 | total_timesteps 2779.
Path 238 | total_timesteps 2790.
Path 239 | total_timesteps 2802.
Path 240 | total_timesteps 2814.
Path 241 | total_timesteps 2822.
Path 242 | total_timesteps 2839.
Path 243 | total_timesteps 2848.
Path 244 | total_timesteps 2857.
Path 245 | total_timesteps 2865.
Path 246 | total_timesteps 2880.
Path 247 | total_timesteps 2890.
Path 248 | total_timesteps 2898.
Path 249 | total_timesteps 2909.
Path 250 | total_timesteps 2919.
Path 251 | total_timesteps 2932.
Path 252 | total_timesteps 2943.
Path 253 | total_timesteps 2955.
Path 254 | total_timesteps 2968.
Path 255 | total_timesteps 2977.
Path 256 | total_timesteps 2986.
Path 257 | total_timesteps 3000.
Path 258 | total_timesteps 3011.
Path 259 | total_timesteps 3021.
Path 260 | total_timesteps 3030.
Path 261 | total_timesteps 3049.
Path 262 | total_timesteps 3057.
Path 263 | total_timesteps 3074.
Path 264 | total_timesteps 3091.
Path 265 | total_timesteps 3102.
Path 266 | total_timesteps 3111.
Path 267 | total_timesteps 3121.
Path 268 | total_timesteps 3134.
Path 269 | total_timesteps 3142.
Path 270 | total_timesteps 3158.
Path 271 | total_timesteps 3169.
Path 272 | total_timesteps 3177.
Path 273 | total_timesteps 3187.
Path 274 | total_timesteps 3207.
Path 275 | total_timesteps 3216.
Path 276 | total_timesteps 3225.
Path 277 | total_timesteps 3238.
Path 278 | total_timesteps 3250.
Path 279 | total_timesteps 3260.
Path 280 | total_timesteps 3268.
Path 281 | total_timesteps 3277.
Path 282 | total_timesteps 3290.
Path 283 | total_timesteps 3301.
Path 284 | total_timesteps 3315.
Path 285 | total_timesteps 3325.
Path 286 | total_timesteps 3334.
Path 287 | total_timesteps 3345.
Path 288 | total_timesteps 3358.
Path 289 | total_timesteps 3372.
Path 290 | total_timesteps 3384.
Path 291 | total_timesteps 3392.
Path 292 | total_timesteps 3406.
Path 293 | total_timesteps 3414.
Path 294 | total_timesteps 3429.
Path 295 | total_timesteps 3439.
Path 296 | total_timesteps 3453.
Path 297 | total_timesteps 3466.
Path 298 | total_timesteps 3475.
Path 299 | total_timesteps 3485.
Path 300 | total_timesteps 3497.
Path 301 | total_timesteps 3506.
Path 302 | total_timesteps 3519.
Path 303 | total_timesteps 3531.
Path 304 | total_timesteps 3543.
Path 305 | total_timesteps 3558.
Path 306 | total_timesteps 3579.
Path 307 | total_timesteps 3586.
Path 308 | total_timesteps 3595.
Path 309 | total_timesteps 3610.
Path 310 | total_timesteps 3619.
Path 311 | total_timesteps 3632.
Path 312 | total_timesteps 3644.
Path 313 | total_timesteps 3652.
Path 314 | total_timesteps 3663.
Path 315 | total_timesteps 3674.
Path 316 | total_timesteps 3686.
Path 317 | total_timesteps 3701.
Path 318 | total_timesteps 3713.
Path 319 | total_timesteps 3724.
Path 320 | total_timesteps 3733.
Path 321 | total_timesteps 3744.
Path 322 | total_timesteps 3754.
Path 323 | total_timesteps 3766.
Path 324 | total_timesteps 3782.
Path 325 | total_timesteps 3797.
Path 326 | total_timesteps 3807.
Path 327 | total_timesteps 3821.
Path 328 | total_timesteps 3829.
Path 329 | total_timesteps 3839.
Path 330 | total_timesteps 3857.
Path 331 | total_timesteps 3865.
Path 332 | total_timesteps 3879.
Path 333 | total_timesteps 3889.
Path 334 | total_timesteps 3902.
Path 335 | total_timesteps 3913.
Path 336 | total_timesteps 3925.
Path 337 | total_timesteps 3936.
Path 338 | total_timesteps 3944.
Path 339 | total_timesteps 3958.
Path 340 | total_timesteps 3965.
Path 341 | total_timesteps 3978.
Path 342 | total_timesteps 3988.
Path 343 | total_timesteps 3995.
Path 344 | total_timesteps 4005.
Path 345 | total_timesteps 4016.
Path 346 | total_timesteps 4025.
Path 347 | total_timesteps 4037.
Path 348 | total_timesteps 4049.
Path 349 | total_timesteps 4062.
Path 350 | total_timesteps 4071.
Path 351 | total_timesteps 4079.
Path 352 | total_timesteps 4090.
Path 353 | total_timesteps 4101.
Path 354 | total_timesteps 4114.
Path 355 | total_timesteps 4124.
Path 356 | total_timesteps 4132.
Path 357 | total_timesteps 4146.
Path 358 | total_timesteps 4159.
Path 359 | total_timesteps 4177.
Path 360 | total_timesteps 4191.
Path 361 | total_timesteps 4203.
Path 362 | total_timesteps 4212.
Path 363 | total_timesteps 4222.
Path 364 | total_timesteps 4229.
Path 365 | total_timesteps 4239.
Path 366 | total_timesteps 4249.
Path 367 | total_timesteps 4270.
Path 368 | total_timesteps 4286.
Path 369 | total_timesteps 4295.
Path 370 | total_timesteps 4308.
Path 371 | total_timesteps 4322.
Path 372 | total_timesteps 4332.
Path 373 | total_timesteps 4348.
Path 374 | total_timesteps 4355.
Path 375 | total_timesteps 4368.
Path 376 | total_timesteps 4380.
Path 377 | total_timesteps 4388.
Path 378 | total_timesteps 4397.
Path 379 | total_timesteps 4407.
Path 380 | total_timesteps 4414.
Path 381 | total_timesteps 4427.
Path 382 | total_timesteps 4434.
Path 383 | total_timesteps 4447.
Path 384 | total_timesteps 4458.
Path 385 | total_timesteps 4472.
Path 386 | total_timesteps 4482.
Path 387 | total_timesteps 4498.
Path 388 | total_timesteps 4508.
Path 389 | total_timesteps 4521.
Path 390 | total_timesteps 4539.
Path 391 | total_timesteps 4551.
Path 392 | total_timesteps 4571.
Path 393 | total_timesteps 4579.
Path 394 | total_timesteps 4589.
Path 395 | total_timesteps 4595.
Path 396 | total_timesteps 4604.
Path 397 | total_timesteps 4615.
Path 398 | total_timesteps 4626.
Path 399 | total_timesteps 4633.
Path 400 | total_timesteps 4647.
Path 401 | total_timesteps 4657.
Path 402 | total_timesteps 4672.
Path 403 | total_timesteps 4688.
Path 404 | total_timesteps 4697.
Path 405 | total_timesteps 4711.
Path 406 | total_timesteps 4718.
Path 407 | total_timesteps 4741.
Path 408 | total_timesteps 4774.
Path 409 | total_timesteps 4792.
Path 410 | total_timesteps 4803.
Path 411 | total_timesteps 4811.
Path 412 | total_timesteps 4821.
Path 413 | total_timesteps 4835.
Path 414 | total_timesteps 4849.
Path 415 | total_timesteps 4859.
Path 416 | total_timesteps 4870.
Path 417 | total_timesteps 4889.
Path 418 | total_timesteps 4908.
Path 419 | total_timesteps 4919.
Path 420 | total_timesteps 4928.
Path 421 | total_timesteps 4952.
Path 422 | total_timesteps 4965.
Path 423 | total_timesteps 4979.
Path 424 | total_timesteps 4988.
Path 425 | total_timesteps 4997.
Path 426 | total_timesteps 5007.
Path 427 | total_timesteps 5021.
Path 428 | total_timesteps 5034.
Path 429 | total_timesteps 5046.
Path 430 | total_timesteps 5053.
Path 431 | total_timesteps 5064.
Path 432 | total_timesteps 5075.
Path 433 | total_timesteps 5084.
Path 434 | total_timesteps 5097.
Path 435 | total_timesteps 5111.
Path 436 | total_timesteps 5123.
Path 437 | total_timesteps 5133.
Path 438 | total_timesteps 5144.
Path 439 | total_timesteps 5153.
Path 440 | total_timesteps 5181.
Path 441 | total_timesteps 5190.
Path 442 | total_timesteps 5199.
Path 443 | total_timesteps 5210.
Path 444 | total_timesteps 5223.
Path 445 | total_timesteps 5233.
Path 446 | total_timesteps 5247.
Path 447 | total_timesteps 5258.
Path 448 | total_timesteps 5272.
Path 449 | total_timesteps 5282.
Path 450 | total_timesteps 5299.
Path 451 | total_timesteps 5309.
Path 452 | total_timesteps 5323.
Path 453 | total_timesteps 5337.
Path 454 | total_timesteps 5365.
Path 455 | total_timesteps 5374.
Path 456 | total_timesteps 5383.
Path 457 | total_timesteps 5391.
Path 458 | total_timesteps 5399.
Path 459 | total_timesteps 5409.
Path 460 | total_timesteps 5416.
Path 461 | total_timesteps 5428.
Path 462 | total_timesteps 5444.
Path 463 | total_timesteps 5458.
Path 464 | total_timesteps 5467.
Path 465 | total_timesteps 5473.
Path 466 | total_timesteps 5494.
Path 467 | total_timesteps 5515.
Path 468 | total_timesteps 5525.
Path 469 | total_timesteps 5536.
Path 470 | total_timesteps 5556.
Path 471 | total_timesteps 5574.
Path 472 | total_timesteps 5582.
Path 473 | total_timesteps 5597.
Path 474 | total_timesteps 5611.
Path 475 | total_timesteps 5622.
Path 476 | total_timesteps 5632.
Path 477 | total_timesteps 5644.
Path 478 | total_timesteps 5668.
Path 479 | total_timesteps 5683.
Path 480 | total_timesteps 5697.
Path 481 | total_timesteps 5707.
Path 482 | total_timesteps 5716.
Path 483 | total_timesteps 5731.
Path 484 | total_timesteps 5748.
Path 485 | total_timesteps 5759.
Path 486 | total_timesteps 5770.
Path 487 | total_timesteps 5788.
Path 488 | total_timesteps 5812.
Path 489 | total_timesteps 5826.
Path 490 | total_timesteps 5836.
Path 491 | total_timesteps 5846.
Path 492 | total_timesteps 5857.
Path 493 | total_timesteps 5869.
Path 494 | total_timesteps 5875.
Path 495 | total_timesteps 5888.
Path 496 | total_timesteps 5895.
Path 497 | total_timesteps 5906.
Path 498 | total_timesteps 5915.
Path 499 | total_timesteps 5924.
Path 500 | total_timesteps 5938.
Path 501 | total_timesteps 5952.
Path 502 | total_timesteps 5965.
Path 503 | total_timesteps 5975.
Path 504 | total_timesteps 5982.
Path 505 | total_timesteps 5994.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.71    |
| Iteration     | 2        |
| MaximumReturn | 2.07     |
| MinimumReturn | -20.3    |
| TotalSamples  | 16025    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.29086023569107056
Validation loss = 0.29756268858909607
Validation loss = 0.3257213532924652
Validation loss = 0.30053770542144775
Validation loss = 0.3272818922996521
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 43.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 60.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 84.
Path 9 | total_timesteps 92.
Path 10 | total_timesteps 100.
Path 11 | total_timesteps 112.
Path 12 | total_timesteps 120.
Path 13 | total_timesteps 126.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 148.
Path 16 | total_timesteps 156.
Path 17 | total_timesteps 163.
Path 18 | total_timesteps 177.
Path 19 | total_timesteps 186.
Path 20 | total_timesteps 195.
Path 21 | total_timesteps 204.
Path 22 | total_timesteps 221.
Path 23 | total_timesteps 230.
Path 24 | total_timesteps 237.
Path 25 | total_timesteps 251.
Path 26 | total_timesteps 260.
Path 27 | total_timesteps 270.
Path 28 | total_timesteps 279.
Path 29 | total_timesteps 292.
Path 30 | total_timesteps 302.
Path 31 | total_timesteps 313.
Path 32 | total_timesteps 324.
Path 33 | total_timesteps 333.
Path 34 | total_timesteps 343.
Path 35 | total_timesteps 356.
Path 36 | total_timesteps 364.
Path 37 | total_timesteps 375.
Path 38 | total_timesteps 383.
Path 39 | total_timesteps 392.
Path 40 | total_timesteps 402.
Path 41 | total_timesteps 414.
Path 42 | total_timesteps 423.
Path 43 | total_timesteps 440.
Path 44 | total_timesteps 448.
Path 45 | total_timesteps 459.
Path 46 | total_timesteps 468.
Path 47 | total_timesteps 477.
Path 48 | total_timesteps 484.
Path 49 | total_timesteps 492.
Path 50 | total_timesteps 501.
Path 51 | total_timesteps 508.
Path 52 | total_timesteps 517.
Path 53 | total_timesteps 529.
Path 54 | total_timesteps 541.
Path 55 | total_timesteps 553.
Path 56 | total_timesteps 562.
Path 57 | total_timesteps 569.
Path 58 | total_timesteps 577.
Path 59 | total_timesteps 591.
Path 60 | total_timesteps 603.
Path 61 | total_timesteps 610.
Path 62 | total_timesteps 620.
Path 63 | total_timesteps 634.
Path 64 | total_timesteps 646.
Path 65 | total_timesteps 655.
Path 66 | total_timesteps 671.
Path 67 | total_timesteps 693.
Path 68 | total_timesteps 701.
Path 69 | total_timesteps 708.
Path 70 | total_timesteps 715.
Path 71 | total_timesteps 729.
Path 72 | total_timesteps 741.
Path 73 | total_timesteps 752.
Path 74 | total_timesteps 764.
Path 75 | total_timesteps 772.
Path 76 | total_timesteps 786.
Path 77 | total_timesteps 804.
Path 78 | total_timesteps 814.
Path 79 | total_timesteps 821.
Path 80 | total_timesteps 831.
Path 81 | total_timesteps 842.
Path 82 | total_timesteps 852.
Path 83 | total_timesteps 860.
Path 84 | total_timesteps 870.
Path 85 | total_timesteps 886.
Path 86 | total_timesteps 897.
Path 87 | total_timesteps 908.
Path 88 | total_timesteps 917.
Path 89 | total_timesteps 925.
Path 90 | total_timesteps 934.
Path 91 | total_timesteps 944.
Path 92 | total_timesteps 956.
Path 93 | total_timesteps 966.
Path 94 | total_timesteps 974.
Path 95 | total_timesteps 985.
Path 96 | total_timesteps 995.
Path 97 | total_timesteps 1004.
Path 98 | total_timesteps 1017.
Path 99 | total_timesteps 1026.
Path 100 | total_timesteps 1035.
Path 101 | total_timesteps 1045.
Path 102 | total_timesteps 1055.
Path 103 | total_timesteps 1063.
Path 104 | total_timesteps 1072.
Path 105 | total_timesteps 1083.
Path 106 | total_timesteps 1098.
Path 107 | total_timesteps 1107.
Path 108 | total_timesteps 1120.
Path 109 | total_timesteps 1128.
Path 110 | total_timesteps 1139.
Path 111 | total_timesteps 1153.
Path 112 | total_timesteps 1161.
Path 113 | total_timesteps 1168.
Path 114 | total_timesteps 1180.
Path 115 | total_timesteps 1192.
Path 116 | total_timesteps 1202.
Path 117 | total_timesteps 1214.
Path 118 | total_timesteps 1225.
Path 119 | total_timesteps 1238.
Path 120 | total_timesteps 1247.
Path 121 | total_timesteps 1256.
Path 122 | total_timesteps 1265.
Path 123 | total_timesteps 1274.
Path 124 | total_timesteps 1288.
Path 125 | total_timesteps 1295.
Path 126 | total_timesteps 1310.
Path 127 | total_timesteps 1318.
Path 128 | total_timesteps 1325.
Path 129 | total_timesteps 1333.
Path 130 | total_timesteps 1342.
Path 131 | total_timesteps 1350.
Path 132 | total_timesteps 1363.
Path 133 | total_timesteps 1373.
Path 134 | total_timesteps 1383.
Path 135 | total_timesteps 1393.
Path 136 | total_timesteps 1409.
Path 137 | total_timesteps 1419.
Path 138 | total_timesteps 1428.
Path 139 | total_timesteps 1435.
Path 140 | total_timesteps 1446.
Path 141 | total_timesteps 1457.
Path 142 | total_timesteps 1464.
Path 143 | total_timesteps 1477.
Path 144 | total_timesteps 1483.
Path 145 | total_timesteps 1492.
Path 146 | total_timesteps 1500.
Path 147 | total_timesteps 1511.
Path 148 | total_timesteps 1524.
Path 149 | total_timesteps 1542.
Path 150 | total_timesteps 1551.
Path 151 | total_timesteps 1559.
Path 152 | total_timesteps 1568.
Path 153 | total_timesteps 1578.
Path 154 | total_timesteps 1588.
Path 155 | total_timesteps 1599.
Path 156 | total_timesteps 1608.
Path 157 | total_timesteps 1618.
Path 158 | total_timesteps 1625.
Path 159 | total_timesteps 1639.
Path 160 | total_timesteps 1657.
Path 161 | total_timesteps 1665.
Path 162 | total_timesteps 1674.
Path 163 | total_timesteps 1681.
Path 164 | total_timesteps 1692.
Path 165 | total_timesteps 1707.
Path 166 | total_timesteps 1716.
Path 167 | total_timesteps 1724.
Path 168 | total_timesteps 1732.
Path 169 | total_timesteps 1744.
Path 170 | total_timesteps 1757.
Path 171 | total_timesteps 1766.
Path 172 | total_timesteps 1784.
Path 173 | total_timesteps 1796.
Path 174 | total_timesteps 1804.
Path 175 | total_timesteps 1813.
Path 176 | total_timesteps 1824.
Path 177 | total_timesteps 1831.
Path 178 | total_timesteps 1840.
Path 179 | total_timesteps 1856.
Path 180 | total_timesteps 1864.
Path 181 | total_timesteps 1872.
Path 182 | total_timesteps 1892.
Path 183 | total_timesteps 1899.
Path 184 | total_timesteps 1910.
Path 185 | total_timesteps 1918.
Path 186 | total_timesteps 1926.
Path 187 | total_timesteps 1936.
Path 188 | total_timesteps 1948.
Path 189 | total_timesteps 1963.
Path 190 | total_timesteps 1970.
Path 191 | total_timesteps 1980.
Path 192 | total_timesteps 1996.
Path 193 | total_timesteps 2007.
Path 194 | total_timesteps 2014.
Path 195 | total_timesteps 2022.
Path 196 | total_timesteps 2030.
Path 197 | total_timesteps 2040.
Path 198 | total_timesteps 2054.
Path 199 | total_timesteps 2066.
Path 200 | total_timesteps 2085.
Path 201 | total_timesteps 2103.
Path 202 | total_timesteps 2111.
Path 203 | total_timesteps 2123.
Path 204 | total_timesteps 2133.
Path 205 | total_timesteps 2145.
Path 206 | total_timesteps 2156.
Path 207 | total_timesteps 2167.
Path 208 | total_timesteps 2173.
Path 209 | total_timesteps 2186.
Path 210 | total_timesteps 2202.
Path 211 | total_timesteps 2208.
Path 212 | total_timesteps 2220.
Path 213 | total_timesteps 2232.
Path 214 | total_timesteps 2240.
Path 215 | total_timesteps 2253.
Path 216 | total_timesteps 2261.
Path 217 | total_timesteps 2274.
Path 218 | total_timesteps 2283.
Path 219 | total_timesteps 2300.
Path 220 | total_timesteps 2307.
Path 221 | total_timesteps 2315.
Path 222 | total_timesteps 2325.
Path 223 | total_timesteps 2332.
Path 224 | total_timesteps 2338.
Path 225 | total_timesteps 2353.
Path 226 | total_timesteps 2371.
Path 227 | total_timesteps 2381.
Path 228 | total_timesteps 2393.
Path 229 | total_timesteps 2400.
Path 230 | total_timesteps 2407.
Path 231 | total_timesteps 2415.
Path 232 | total_timesteps 2426.
Path 233 | total_timesteps 2437.
Path 234 | total_timesteps 2448.
Path 235 | total_timesteps 2462.
Path 236 | total_timesteps 2471.
Path 237 | total_timesteps 2486.
Path 238 | total_timesteps 2499.
Path 239 | total_timesteps 2511.
Path 240 | total_timesteps 2520.
Path 241 | total_timesteps 2533.
Path 242 | total_timesteps 2551.
Path 243 | total_timesteps 2559.
Path 244 | total_timesteps 2567.
Path 245 | total_timesteps 2588.
Path 246 | total_timesteps 2595.
Path 247 | total_timesteps 2606.
Path 248 | total_timesteps 2621.
Path 249 | total_timesteps 2631.
Path 250 | total_timesteps 2643.
Path 251 | total_timesteps 2652.
Path 252 | total_timesteps 2663.
Path 253 | total_timesteps 2675.
Path 254 | total_timesteps 2687.
Path 255 | total_timesteps 2704.
Path 256 | total_timesteps 2712.
Path 257 | total_timesteps 2720.
Path 258 | total_timesteps 2731.
Path 259 | total_timesteps 2740.
Path 260 | total_timesteps 2747.
Path 261 | total_timesteps 2760.
Path 262 | total_timesteps 2770.
Path 263 | total_timesteps 2788.
Path 264 | total_timesteps 2797.
Path 265 | total_timesteps 2809.
Path 266 | total_timesteps 2825.
Path 267 | total_timesteps 2834.
Path 268 | total_timesteps 2844.
Path 269 | total_timesteps 2852.
Path 270 | total_timesteps 2859.
Path 271 | total_timesteps 2868.
Path 272 | total_timesteps 2876.
Path 273 | total_timesteps 2886.
Path 274 | total_timesteps 2893.
Path 275 | total_timesteps 2900.
Path 276 | total_timesteps 2907.
Path 277 | total_timesteps 2918.
Path 278 | total_timesteps 2925.
Path 279 | total_timesteps 2945.
Path 280 | total_timesteps 2951.
Path 281 | total_timesteps 2961.
Path 282 | total_timesteps 2970.
Path 283 | total_timesteps 2982.
Path 284 | total_timesteps 2992.
Path 285 | total_timesteps 3004.
Path 286 | total_timesteps 3011.
Path 287 | total_timesteps 3020.
Path 288 | total_timesteps 3026.
Path 289 | total_timesteps 3034.
Path 290 | total_timesteps 3040.
Path 291 | total_timesteps 3047.
Path 292 | total_timesteps 3055.
Path 293 | total_timesteps 3065.
Path 294 | total_timesteps 3075.
Path 295 | total_timesteps 3084.
Path 296 | total_timesteps 3094.
Path 297 | total_timesteps 3106.
Path 298 | total_timesteps 3117.
Path 299 | total_timesteps 3128.
Path 300 | total_timesteps 3139.
Path 301 | total_timesteps 3151.
Path 302 | total_timesteps 3162.
Path 303 | total_timesteps 3173.
Path 304 | total_timesteps 3182.
Path 305 | total_timesteps 3193.
Path 306 | total_timesteps 3201.
Path 307 | total_timesteps 3210.
Path 308 | total_timesteps 3221.
Path 309 | total_timesteps 3235.
Path 310 | total_timesteps 3247.
Path 311 | total_timesteps 3257.
Path 312 | total_timesteps 3267.
Path 313 | total_timesteps 3276.
Path 314 | total_timesteps 3287.
Path 315 | total_timesteps 3309.
Path 316 | total_timesteps 3322.
Path 317 | total_timesteps 3340.
Path 318 | total_timesteps 3351.
Path 319 | total_timesteps 3360.
Path 320 | total_timesteps 3366.
Path 321 | total_timesteps 3374.
Path 322 | total_timesteps 3387.
Path 323 | total_timesteps 3400.
Path 324 | total_timesteps 3409.
Path 325 | total_timesteps 3420.
Path 326 | total_timesteps 3433.
Path 327 | total_timesteps 3441.
Path 328 | total_timesteps 3450.
Path 329 | total_timesteps 3459.
Path 330 | total_timesteps 3469.
Path 331 | total_timesteps 3480.
Path 332 | total_timesteps 3487.
Path 333 | total_timesteps 3499.
Path 334 | total_timesteps 3508.
Path 335 | total_timesteps 3523.
Path 336 | total_timesteps 3536.
Path 337 | total_timesteps 3545.
Path 338 | total_timesteps 3554.
Path 339 | total_timesteps 3569.
Path 340 | total_timesteps 3580.
Path 341 | total_timesteps 3587.
Path 342 | total_timesteps 3609.
Path 343 | total_timesteps 3624.
Path 344 | total_timesteps 3633.
Path 345 | total_timesteps 3645.
Path 346 | total_timesteps 3665.
Path 347 | total_timesteps 3676.
Path 348 | total_timesteps 3684.
Path 349 | total_timesteps 3692.
Path 350 | total_timesteps 3701.
Path 351 | total_timesteps 3707.
Path 352 | total_timesteps 3714.
Path 353 | total_timesteps 3725.
Path 354 | total_timesteps 3738.
Path 355 | total_timesteps 3752.
Path 356 | total_timesteps 3768.
Path 357 | total_timesteps 3777.
Path 358 | total_timesteps 3784.
Path 359 | total_timesteps 3797.
Path 360 | total_timesteps 3810.
Path 361 | total_timesteps 3825.
Path 362 | total_timesteps 3834.
Path 363 | total_timesteps 3849.
Path 364 | total_timesteps 3857.
Path 365 | total_timesteps 3865.
Path 366 | total_timesteps 3875.
Path 367 | total_timesteps 3889.
Path 368 | total_timesteps 3898.
Path 369 | total_timesteps 3908.
Path 370 | total_timesteps 3917.
Path 371 | total_timesteps 3925.
Path 372 | total_timesteps 3934.
Path 373 | total_timesteps 3941.
Path 374 | total_timesteps 3950.
Path 375 | total_timesteps 3960.
Path 376 | total_timesteps 3968.
Path 377 | total_timesteps 3985.
Path 378 | total_timesteps 3995.
Path 379 | total_timesteps 4006.
Path 380 | total_timesteps 4016.
Path 381 | total_timesteps 4025.
Path 382 | total_timesteps 4033.
Path 383 | total_timesteps 4045.
Path 384 | total_timesteps 4060.
Path 385 | total_timesteps 4070.
Path 386 | total_timesteps 4080.
Path 387 | total_timesteps 4087.
Path 388 | total_timesteps 4098.
Path 389 | total_timesteps 4108.
Path 390 | total_timesteps 4117.
Path 391 | total_timesteps 4124.
Path 392 | total_timesteps 4132.
Path 393 | total_timesteps 4142.
Path 394 | total_timesteps 4150.
Path 395 | total_timesteps 4163.
Path 396 | total_timesteps 4176.
Path 397 | total_timesteps 4185.
Path 398 | total_timesteps 4195.
Path 399 | total_timesteps 4205.
Path 400 | total_timesteps 4212.
Path 401 | total_timesteps 4224.
Path 402 | total_timesteps 4232.
Path 403 | total_timesteps 4243.
Path 404 | total_timesteps 4253.
Path 405 | total_timesteps 4260.
Path 406 | total_timesteps 4273.
Path 407 | total_timesteps 4283.
Path 408 | total_timesteps 4294.
Path 409 | total_timesteps 4302.
Path 410 | total_timesteps 4311.
Path 411 | total_timesteps 4321.
Path 412 | total_timesteps 4329.
Path 413 | total_timesteps 4340.
Path 414 | total_timesteps 4351.
Path 415 | total_timesteps 4357.
Path 416 | total_timesteps 4363.
Path 417 | total_timesteps 4371.
Path 418 | total_timesteps 4379.
Path 419 | total_timesteps 4388.
Path 420 | total_timesteps 4399.
Path 421 | total_timesteps 4412.
Path 422 | total_timesteps 4423.
Path 423 | total_timesteps 4440.
Path 424 | total_timesteps 4449.
Path 425 | total_timesteps 4461.
Path 426 | total_timesteps 4473.
Path 427 | total_timesteps 4483.
Path 428 | total_timesteps 4495.
Path 429 | total_timesteps 4502.
Path 430 | total_timesteps 4511.
Path 431 | total_timesteps 4526.
Path 432 | total_timesteps 4543.
Path 433 | total_timesteps 4554.
Path 434 | total_timesteps 4564.
Path 435 | total_timesteps 4577.
Path 436 | total_timesteps 4586.
Path 437 | total_timesteps 4596.
Path 438 | total_timesteps 4612.
Path 439 | total_timesteps 4620.
Path 440 | total_timesteps 4627.
Path 441 | total_timesteps 4641.
Path 442 | total_timesteps 4650.
Path 443 | total_timesteps 4659.
Path 444 | total_timesteps 4665.
Path 445 | total_timesteps 4677.
Path 446 | total_timesteps 4688.
Path 447 | total_timesteps 4698.
Path 448 | total_timesteps 4708.
Path 449 | total_timesteps 4718.
Path 450 | total_timesteps 4729.
Path 451 | total_timesteps 4739.
Path 452 | total_timesteps 4752.
Path 453 | total_timesteps 4770.
Path 454 | total_timesteps 4791.
Path 455 | total_timesteps 4798.
Path 456 | total_timesteps 4807.
Path 457 | total_timesteps 4818.
Path 458 | total_timesteps 4826.
Path 459 | total_timesteps 4843.
Path 460 | total_timesteps 4852.
Path 461 | total_timesteps 4863.
Path 462 | total_timesteps 4876.
Path 463 | total_timesteps 4885.
Path 464 | total_timesteps 4894.
Path 465 | total_timesteps 4903.
Path 466 | total_timesteps 4916.
Path 467 | total_timesteps 4925.
Path 468 | total_timesteps 4940.
Path 469 | total_timesteps 4952.
Path 470 | total_timesteps 4961.
Path 471 | total_timesteps 4973.
Path 472 | total_timesteps 4990.
Path 473 | total_timesteps 5000.
Path 474 | total_timesteps 5009.
Path 475 | total_timesteps 5017.
Path 476 | total_timesteps 5027.
Path 477 | total_timesteps 5036.
Path 478 | total_timesteps 5046.
Path 479 | total_timesteps 5053.
Path 480 | total_timesteps 5063.
Path 481 | total_timesteps 5071.
Path 482 | total_timesteps 5078.
Path 483 | total_timesteps 5089.
Path 484 | total_timesteps 5098.
Path 485 | total_timesteps 5109.
Path 486 | total_timesteps 5120.
Path 487 | total_timesteps 5139.
Path 488 | total_timesteps 5147.
Path 489 | total_timesteps 5156.
Path 490 | total_timesteps 5165.
Path 491 | total_timesteps 5175.
Path 492 | total_timesteps 5185.
Path 493 | total_timesteps 5195.
Path 494 | total_timesteps 5203.
Path 495 | total_timesteps 5214.
Path 496 | total_timesteps 5229.
Path 497 | total_timesteps 5238.
Path 498 | total_timesteps 5246.
Path 499 | total_timesteps 5258.
Path 500 | total_timesteps 5268.
Path 501 | total_timesteps 5283.
Path 502 | total_timesteps 5300.
Path 503 | total_timesteps 5309.
Path 504 | total_timesteps 5319.
Path 505 | total_timesteps 5331.
Path 506 | total_timesteps 5346.
Path 507 | total_timesteps 5356.
Path 508 | total_timesteps 5368.
Path 509 | total_timesteps 5379.
Path 510 | total_timesteps 5391.
Path 511 | total_timesteps 5398.
Path 512 | total_timesteps 5408.
Path 513 | total_timesteps 5418.
Path 514 | total_timesteps 5427.
Path 515 | total_timesteps 5436.
Path 516 | total_timesteps 5450.
Path 517 | total_timesteps 5460.
Path 518 | total_timesteps 5473.
Path 519 | total_timesteps 5482.
Path 520 | total_timesteps 5492.
Path 521 | total_timesteps 5499.
Path 522 | total_timesteps 5517.
Path 523 | total_timesteps 5530.
Path 524 | total_timesteps 5541.
Path 525 | total_timesteps 5548.
Path 526 | total_timesteps 5564.
Path 527 | total_timesteps 5571.
Path 528 | total_timesteps 5587.
Path 529 | total_timesteps 5600.
Path 530 | total_timesteps 5613.
Path 531 | total_timesteps 5623.
Path 532 | total_timesteps 5636.
Path 533 | total_timesteps 5652.
Path 534 | total_timesteps 5666.
Path 535 | total_timesteps 5679.
Path 536 | total_timesteps 5690.
Path 537 | total_timesteps 5700.
Path 538 | total_timesteps 5710.
Path 539 | total_timesteps 5721.
Path 540 | total_timesteps 5730.
Path 541 | total_timesteps 5746.
Path 542 | total_timesteps 5754.
Path 543 | total_timesteps 5767.
Path 544 | total_timesteps 5776.
Path 545 | total_timesteps 5784.
Path 546 | total_timesteps 5792.
Path 547 | total_timesteps 5801.
Path 548 | total_timesteps 5812.
Path 549 | total_timesteps 5822.
Path 550 | total_timesteps 5833.
Path 551 | total_timesteps 5846.
Path 552 | total_timesteps 5854.
Path 553 | total_timesteps 5861.
Path 554 | total_timesteps 5869.
Path 555 | total_timesteps 5875.
Path 556 | total_timesteps 5886.
Path 557 | total_timesteps 5894.
Path 558 | total_timesteps 5904.
Path 559 | total_timesteps 5910.
Path 560 | total_timesteps 5919.
Path 561 | total_timesteps 5929.
Path 562 | total_timesteps 5937.
Path 563 | total_timesteps 5946.
Path 564 | total_timesteps 5953.
Path 565 | total_timesteps 5966.
Path 566 | total_timesteps 5979.
Path 567 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.4     |
| Iteration     | 3        |
| MaximumReturn | 2.82     |
| MinimumReturn | -19.6    |
| TotalSamples  | 20029    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.28958749771118164
Validation loss = 0.2941211760044098
Validation loss = 0.2912575900554657
Validation loss = 0.2969430387020111
Validation loss = 0.3072252869606018
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 30.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 60.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 87.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 113.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 130.
Path 14 | total_timesteps 146.
Path 15 | total_timesteps 170.
Path 16 | total_timesteps 179.
Path 17 | total_timesteps 196.
Path 18 | total_timesteps 202.
Path 19 | total_timesteps 209.
Path 20 | total_timesteps 218.
Path 21 | total_timesteps 228.
Path 22 | total_timesteps 241.
Path 23 | total_timesteps 248.
Path 24 | total_timesteps 262.
Path 25 | total_timesteps 275.
Path 26 | total_timesteps 290.
Path 27 | total_timesteps 305.
Path 28 | total_timesteps 312.
Path 29 | total_timesteps 321.
Path 30 | total_timesteps 330.
Path 31 | total_timesteps 338.
Path 32 | total_timesteps 350.
Path 33 | total_timesteps 363.
Path 34 | total_timesteps 373.
Path 35 | total_timesteps 386.
Path 36 | total_timesteps 394.
Path 37 | total_timesteps 410.
Path 38 | total_timesteps 424.
Path 39 | total_timesteps 435.
Path 40 | total_timesteps 443.
Path 41 | total_timesteps 453.
Path 42 | total_timesteps 473.
Path 43 | total_timesteps 483.
Path 44 | total_timesteps 490.
Path 45 | total_timesteps 499.
Path 46 | total_timesteps 512.
Path 47 | total_timesteps 522.
Path 48 | total_timesteps 531.
Path 49 | total_timesteps 538.
Path 50 | total_timesteps 549.
Path 51 | total_timesteps 558.
Path 52 | total_timesteps 569.
Path 53 | total_timesteps 578.
Path 54 | total_timesteps 586.
Path 55 | total_timesteps 601.
Path 56 | total_timesteps 611.
Path 57 | total_timesteps 626.
Path 58 | total_timesteps 636.
Path 59 | total_timesteps 646.
Path 60 | total_timesteps 657.
Path 61 | total_timesteps 665.
Path 62 | total_timesteps 675.
Path 63 | total_timesteps 683.
Path 64 | total_timesteps 691.
Path 65 | total_timesteps 705.
Path 66 | total_timesteps 713.
Path 67 | total_timesteps 724.
Path 68 | total_timesteps 736.
Path 69 | total_timesteps 746.
Path 70 | total_timesteps 752.
Path 71 | total_timesteps 762.
Path 72 | total_timesteps 770.
Path 73 | total_timesteps 778.
Path 74 | total_timesteps 788.
Path 75 | total_timesteps 797.
Path 76 | total_timesteps 808.
Path 77 | total_timesteps 820.
Path 78 | total_timesteps 835.
Path 79 | total_timesteps 845.
Path 80 | total_timesteps 854.
Path 81 | total_timesteps 863.
Path 82 | total_timesteps 871.
Path 83 | total_timesteps 885.
Path 84 | total_timesteps 896.
Path 85 | total_timesteps 909.
Path 86 | total_timesteps 916.
Path 87 | total_timesteps 925.
Path 88 | total_timesteps 937.
Path 89 | total_timesteps 949.
Path 90 | total_timesteps 959.
Path 91 | total_timesteps 967.
Path 92 | total_timesteps 979.
Path 93 | total_timesteps 990.
Path 94 | total_timesteps 999.
Path 95 | total_timesteps 1007.
Path 96 | total_timesteps 1015.
Path 97 | total_timesteps 1024.
Path 98 | total_timesteps 1037.
Path 99 | total_timesteps 1054.
Path 100 | total_timesteps 1065.
Path 101 | total_timesteps 1073.
Path 102 | total_timesteps 1082.
Path 103 | total_timesteps 1088.
Path 104 | total_timesteps 1098.
Path 105 | total_timesteps 1110.
Path 106 | total_timesteps 1120.
Path 107 | total_timesteps 1128.
Path 108 | total_timesteps 1144.
Path 109 | total_timesteps 1154.
Path 110 | total_timesteps 1161.
Path 111 | total_timesteps 1170.
Path 112 | total_timesteps 1183.
Path 113 | total_timesteps 1190.
Path 114 | total_timesteps 1200.
Path 115 | total_timesteps 1214.
Path 116 | total_timesteps 1225.
Path 117 | total_timesteps 1235.
Path 118 | total_timesteps 1247.
Path 119 | total_timesteps 1258.
Path 120 | total_timesteps 1270.
Path 121 | total_timesteps 1278.
Path 122 | total_timesteps 1293.
Path 123 | total_timesteps 1300.
Path 124 | total_timesteps 1308.
Path 125 | total_timesteps 1324.
Path 126 | total_timesteps 1330.
Path 127 | total_timesteps 1339.
Path 128 | total_timesteps 1347.
Path 129 | total_timesteps 1356.
Path 130 | total_timesteps 1368.
Path 131 | total_timesteps 1376.
Path 132 | total_timesteps 1384.
Path 133 | total_timesteps 1396.
Path 134 | total_timesteps 1404.
Path 135 | total_timesteps 1413.
Path 136 | total_timesteps 1429.
Path 137 | total_timesteps 1440.
Path 138 | total_timesteps 1449.
Path 139 | total_timesteps 1467.
Path 140 | total_timesteps 1474.
Path 141 | total_timesteps 1484.
Path 142 | total_timesteps 1491.
Path 143 | total_timesteps 1499.
Path 144 | total_timesteps 1509.
Path 145 | total_timesteps 1521.
Path 146 | total_timesteps 1529.
Path 147 | total_timesteps 1538.
Path 148 | total_timesteps 1552.
Path 149 | total_timesteps 1562.
Path 150 | total_timesteps 1572.
Path 151 | total_timesteps 1583.
Path 152 | total_timesteps 1594.
Path 153 | total_timesteps 1603.
Path 154 | total_timesteps 1612.
Path 155 | total_timesteps 1622.
Path 156 | total_timesteps 1630.
Path 157 | total_timesteps 1643.
Path 158 | total_timesteps 1658.
Path 159 | total_timesteps 1665.
Path 160 | total_timesteps 1674.
Path 161 | total_timesteps 1682.
Path 162 | total_timesteps 1690.
Path 163 | total_timesteps 1699.
Path 164 | total_timesteps 1709.
Path 165 | total_timesteps 1718.
Path 166 | total_timesteps 1734.
Path 167 | total_timesteps 1744.
Path 168 | total_timesteps 1754.
Path 169 | total_timesteps 1774.
Path 170 | total_timesteps 1783.
Path 171 | total_timesteps 1798.
Path 172 | total_timesteps 1813.
Path 173 | total_timesteps 1832.
Path 174 | total_timesteps 1839.
Path 175 | total_timesteps 1852.
Path 176 | total_timesteps 1862.
Path 177 | total_timesteps 1871.
Path 178 | total_timesteps 1882.
Path 179 | total_timesteps 1905.
Path 180 | total_timesteps 1914.
Path 181 | total_timesteps 1921.
Path 182 | total_timesteps 1935.
Path 183 | total_timesteps 1945.
Path 184 | total_timesteps 1962.
Path 185 | total_timesteps 1975.
Path 186 | total_timesteps 1991.
Path 187 | total_timesteps 2005.
Path 188 | total_timesteps 2014.
Path 189 | total_timesteps 2026.
Path 190 | total_timesteps 2033.
Path 191 | total_timesteps 2041.
Path 192 | total_timesteps 2050.
Path 193 | total_timesteps 2059.
Path 194 | total_timesteps 2068.
Path 195 | total_timesteps 2085.
Path 196 | total_timesteps 2095.
Path 197 | total_timesteps 2111.
Path 198 | total_timesteps 2117.
Path 199 | total_timesteps 2131.
Path 200 | total_timesteps 2143.
Path 201 | total_timesteps 2161.
Path 202 | total_timesteps 2169.
Path 203 | total_timesteps 2181.
Path 204 | total_timesteps 2189.
Path 205 | total_timesteps 2207.
Path 206 | total_timesteps 2214.
Path 207 | total_timesteps 2224.
Path 208 | total_timesteps 2233.
Path 209 | total_timesteps 2242.
Path 210 | total_timesteps 2250.
Path 211 | total_timesteps 2264.
Path 212 | total_timesteps 2276.
Path 213 | total_timesteps 2285.
Path 214 | total_timesteps 2293.
Path 215 | total_timesteps 2309.
Path 216 | total_timesteps 2316.
Path 217 | total_timesteps 2325.
Path 218 | total_timesteps 2348.
Path 219 | total_timesteps 2356.
Path 220 | total_timesteps 2364.
Path 221 | total_timesteps 2375.
Path 222 | total_timesteps 2382.
Path 223 | total_timesteps 2390.
Path 224 | total_timesteps 2407.
Path 225 | total_timesteps 2416.
Path 226 | total_timesteps 2427.
Path 227 | total_timesteps 2438.
Path 228 | total_timesteps 2448.
Path 229 | total_timesteps 2460.
Path 230 | total_timesteps 2467.
Path 231 | total_timesteps 2475.
Path 232 | total_timesteps 2486.
Path 233 | total_timesteps 2496.
Path 234 | total_timesteps 2504.
Path 235 | total_timesteps 2512.
Path 236 | total_timesteps 2525.
Path 237 | total_timesteps 2532.
Path 238 | total_timesteps 2540.
Path 239 | total_timesteps 2548.
Path 240 | total_timesteps 2556.
Path 241 | total_timesteps 2565.
Path 242 | total_timesteps 2573.
Path 243 | total_timesteps 2580.
Path 244 | total_timesteps 2591.
Path 245 | total_timesteps 2601.
Path 246 | total_timesteps 2610.
Path 247 | total_timesteps 2618.
Path 248 | total_timesteps 2633.
Path 249 | total_timesteps 2639.
Path 250 | total_timesteps 2649.
Path 251 | total_timesteps 2658.
Path 252 | total_timesteps 2669.
Path 253 | total_timesteps 2680.
Path 254 | total_timesteps 2695.
Path 255 | total_timesteps 2717.
Path 256 | total_timesteps 2724.
Path 257 | total_timesteps 2736.
Path 258 | total_timesteps 2744.
Path 259 | total_timesteps 2751.
Path 260 | total_timesteps 2758.
Path 261 | total_timesteps 2769.
Path 262 | total_timesteps 2782.
Path 263 | total_timesteps 2790.
Path 264 | total_timesteps 2802.
Path 265 | total_timesteps 2816.
Path 266 | total_timesteps 2824.
Path 267 | total_timesteps 2838.
Path 268 | total_timesteps 2845.
Path 269 | total_timesteps 2852.
Path 270 | total_timesteps 2862.
Path 271 | total_timesteps 2874.
Path 272 | total_timesteps 2881.
Path 273 | total_timesteps 2897.
Path 274 | total_timesteps 2906.
Path 275 | total_timesteps 2919.
Path 276 | total_timesteps 2928.
Path 277 | total_timesteps 2935.
Path 278 | total_timesteps 2948.
Path 279 | total_timesteps 2959.
Path 280 | total_timesteps 2970.
Path 281 | total_timesteps 2981.
Path 282 | total_timesteps 2993.
Path 283 | total_timesteps 3005.
Path 284 | total_timesteps 3016.
Path 285 | total_timesteps 3028.
Path 286 | total_timesteps 3037.
Path 287 | total_timesteps 3046.
Path 288 | total_timesteps 3053.
Path 289 | total_timesteps 3064.
Path 290 | total_timesteps 3077.
Path 291 | total_timesteps 3085.
Path 292 | total_timesteps 3096.
Path 293 | total_timesteps 3104.
Path 294 | total_timesteps 3119.
Path 295 | total_timesteps 3126.
Path 296 | total_timesteps 3134.
Path 297 | total_timesteps 3152.
Path 298 | total_timesteps 3163.
Path 299 | total_timesteps 3170.
Path 300 | total_timesteps 3185.
Path 301 | total_timesteps 3196.
Path 302 | total_timesteps 3209.
Path 303 | total_timesteps 3217.
Path 304 | total_timesteps 3227.
Path 305 | total_timesteps 3239.
Path 306 | total_timesteps 3248.
Path 307 | total_timesteps 3260.
Path 308 | total_timesteps 3268.
Path 309 | total_timesteps 3278.
Path 310 | total_timesteps 3285.
Path 311 | total_timesteps 3292.
Path 312 | total_timesteps 3300.
Path 313 | total_timesteps 3310.
Path 314 | total_timesteps 3321.
Path 315 | total_timesteps 3332.
Path 316 | total_timesteps 3345.
Path 317 | total_timesteps 3356.
Path 318 | total_timesteps 3364.
Path 319 | total_timesteps 3372.
Path 320 | total_timesteps 3383.
Path 321 | total_timesteps 3402.
Path 322 | total_timesteps 3411.
Path 323 | total_timesteps 3421.
Path 324 | total_timesteps 3430.
Path 325 | total_timesteps 3437.
Path 326 | total_timesteps 3453.
Path 327 | total_timesteps 3461.
Path 328 | total_timesteps 3473.
Path 329 | total_timesteps 3483.
Path 330 | total_timesteps 3492.
Path 331 | total_timesteps 3506.
Path 332 | total_timesteps 3516.
Path 333 | total_timesteps 3524.
Path 334 | total_timesteps 3534.
Path 335 | total_timesteps 3544.
Path 336 | total_timesteps 3551.
Path 337 | total_timesteps 3559.
Path 338 | total_timesteps 3571.
Path 339 | total_timesteps 3580.
Path 340 | total_timesteps 3587.
Path 341 | total_timesteps 3597.
Path 342 | total_timesteps 3615.
Path 343 | total_timesteps 3630.
Path 344 | total_timesteps 3638.
Path 345 | total_timesteps 3650.
Path 346 | total_timesteps 3664.
Path 347 | total_timesteps 3670.
Path 348 | total_timesteps 3678.
Path 349 | total_timesteps 3689.
Path 350 | total_timesteps 3696.
Path 351 | total_timesteps 3707.
Path 352 | total_timesteps 3717.
Path 353 | total_timesteps 3728.
Path 354 | total_timesteps 3735.
Path 355 | total_timesteps 3751.
Path 356 | total_timesteps 3760.
Path 357 | total_timesteps 3777.
Path 358 | total_timesteps 3784.
Path 359 | total_timesteps 3796.
Path 360 | total_timesteps 3804.
Path 361 | total_timesteps 3813.
Path 362 | total_timesteps 3821.
Path 363 | total_timesteps 3829.
Path 364 | total_timesteps 3835.
Path 365 | total_timesteps 3845.
Path 366 | total_timesteps 3856.
Path 367 | total_timesteps 3867.
Path 368 | total_timesteps 3875.
Path 369 | total_timesteps 3882.
Path 370 | total_timesteps 3899.
Path 371 | total_timesteps 3907.
Path 372 | total_timesteps 3917.
Path 373 | total_timesteps 3924.
Path 374 | total_timesteps 3939.
Path 375 | total_timesteps 3948.
Path 376 | total_timesteps 3964.
Path 377 | total_timesteps 3972.
Path 378 | total_timesteps 3981.
Path 379 | total_timesteps 3990.
Path 380 | total_timesteps 4000.
Path 381 | total_timesteps 4007.
Path 382 | total_timesteps 4017.
Path 383 | total_timesteps 4026.
Path 384 | total_timesteps 4035.
Path 385 | total_timesteps 4052.
Path 386 | total_timesteps 4061.
Path 387 | total_timesteps 4078.
Path 388 | total_timesteps 4086.
Path 389 | total_timesteps 4096.
Path 390 | total_timesteps 4105.
Path 391 | total_timesteps 4114.
Path 392 | total_timesteps 4123.
Path 393 | total_timesteps 4132.
Path 394 | total_timesteps 4139.
Path 395 | total_timesteps 4147.
Path 396 | total_timesteps 4160.
Path 397 | total_timesteps 4168.
Path 398 | total_timesteps 4182.
Path 399 | total_timesteps 4195.
Path 400 | total_timesteps 4207.
Path 401 | total_timesteps 4218.
Path 402 | total_timesteps 4229.
Path 403 | total_timesteps 4237.
Path 404 | total_timesteps 4248.
Path 405 | total_timesteps 4258.
Path 406 | total_timesteps 4270.
Path 407 | total_timesteps 4281.
Path 408 | total_timesteps 4292.
Path 409 | total_timesteps 4302.
Path 410 | total_timesteps 4312.
Path 411 | total_timesteps 4320.
Path 412 | total_timesteps 4329.
Path 413 | total_timesteps 4343.
Path 414 | total_timesteps 4350.
Path 415 | total_timesteps 4366.
Path 416 | total_timesteps 4375.
Path 417 | total_timesteps 4383.
Path 418 | total_timesteps 4392.
Path 419 | total_timesteps 4399.
Path 420 | total_timesteps 4410.
Path 421 | total_timesteps 4423.
Path 422 | total_timesteps 4433.
Path 423 | total_timesteps 4443.
Path 424 | total_timesteps 4451.
Path 425 | total_timesteps 4466.
Path 426 | total_timesteps 4477.
Path 427 | total_timesteps 4490.
Path 428 | total_timesteps 4500.
Path 429 | total_timesteps 4509.
Path 430 | total_timesteps 4516.
Path 431 | total_timesteps 4523.
Path 432 | total_timesteps 4542.
Path 433 | total_timesteps 4555.
Path 434 | total_timesteps 4567.
Path 435 | total_timesteps 4582.
Path 436 | total_timesteps 4592.
Path 437 | total_timesteps 4599.
Path 438 | total_timesteps 4605.
Path 439 | total_timesteps 4613.
Path 440 | total_timesteps 4624.
Path 441 | total_timesteps 4631.
Path 442 | total_timesteps 4645.
Path 443 | total_timesteps 4658.
Path 444 | total_timesteps 4666.
Path 445 | total_timesteps 4674.
Path 446 | total_timesteps 4682.
Path 447 | total_timesteps 4694.
Path 448 | total_timesteps 4707.
Path 449 | total_timesteps 4715.
Path 450 | total_timesteps 4723.
Path 451 | total_timesteps 4734.
Path 452 | total_timesteps 4743.
Path 453 | total_timesteps 4754.
Path 454 | total_timesteps 4767.
Path 455 | total_timesteps 4775.
Path 456 | total_timesteps 4788.
Path 457 | total_timesteps 4801.
Path 458 | total_timesteps 4816.
Path 459 | total_timesteps 4824.
Path 460 | total_timesteps 4836.
Path 461 | total_timesteps 4842.
Path 462 | total_timesteps 4855.
Path 463 | total_timesteps 4863.
Path 464 | total_timesteps 4879.
Path 465 | total_timesteps 4890.
Path 466 | total_timesteps 4896.
Path 467 | total_timesteps 4905.
Path 468 | total_timesteps 4916.
Path 469 | total_timesteps 4925.
Path 470 | total_timesteps 4933.
Path 471 | total_timesteps 4944.
Path 472 | total_timesteps 4955.
Path 473 | total_timesteps 4964.
Path 474 | total_timesteps 4976.
Path 475 | total_timesteps 4987.
Path 476 | total_timesteps 4996.
Path 477 | total_timesteps 5005.
Path 478 | total_timesteps 5014.
Path 479 | total_timesteps 5022.
Path 480 | total_timesteps 5031.
Path 481 | total_timesteps 5040.
Path 482 | total_timesteps 5051.
Path 483 | total_timesteps 5071.
Path 484 | total_timesteps 5082.
Path 485 | total_timesteps 5092.
Path 486 | total_timesteps 5101.
Path 487 | total_timesteps 5114.
Path 488 | total_timesteps 5129.
Path 489 | total_timesteps 5145.
Path 490 | total_timesteps 5156.
Path 491 | total_timesteps 5164.
Path 492 | total_timesteps 5174.
Path 493 | total_timesteps 5182.
Path 494 | total_timesteps 5190.
Path 495 | total_timesteps 5201.
Path 496 | total_timesteps 5209.
Path 497 | total_timesteps 5220.
Path 498 | total_timesteps 5234.
Path 499 | total_timesteps 5246.
Path 500 | total_timesteps 5264.
Path 501 | total_timesteps 5273.
Path 502 | total_timesteps 5281.
Path 503 | total_timesteps 5291.
Path 504 | total_timesteps 5302.
Path 505 | total_timesteps 5315.
Path 506 | total_timesteps 5329.
Path 507 | total_timesteps 5340.
Path 508 | total_timesteps 5352.
Path 509 | total_timesteps 5360.
Path 510 | total_timesteps 5370.
Path 511 | total_timesteps 5379.
Path 512 | total_timesteps 5393.
Path 513 | total_timesteps 5405.
Path 514 | total_timesteps 5418.
Path 515 | total_timesteps 5427.
Path 516 | total_timesteps 5436.
Path 517 | total_timesteps 5447.
Path 518 | total_timesteps 5456.
Path 519 | total_timesteps 5467.
Path 520 | total_timesteps 5476.
Path 521 | total_timesteps 5486.
Path 522 | total_timesteps 5497.
Path 523 | total_timesteps 5504.
Path 524 | total_timesteps 5513.
Path 525 | total_timesteps 5521.
Path 526 | total_timesteps 5530.
Path 527 | total_timesteps 5538.
Path 528 | total_timesteps 5546.
Path 529 | total_timesteps 5563.
Path 530 | total_timesteps 5573.
Path 531 | total_timesteps 5580.
Path 532 | total_timesteps 5587.
Path 533 | total_timesteps 5598.
Path 534 | total_timesteps 5607.
Path 535 | total_timesteps 5616.
Path 536 | total_timesteps 5627.
Path 537 | total_timesteps 5636.
Path 538 | total_timesteps 5647.
Path 539 | total_timesteps 5657.
Path 540 | total_timesteps 5666.
Path 541 | total_timesteps 5675.
Path 542 | total_timesteps 5683.
Path 543 | total_timesteps 5697.
Path 544 | total_timesteps 5704.
Path 545 | total_timesteps 5717.
Path 546 | total_timesteps 5726.
Path 547 | total_timesteps 5735.
Path 548 | total_timesteps 5746.
Path 549 | total_timesteps 5756.
Path 550 | total_timesteps 5768.
Path 551 | total_timesteps 5778.
Path 552 | total_timesteps 5791.
Path 553 | total_timesteps 5800.
Path 554 | total_timesteps 5807.
Path 555 | total_timesteps 5816.
Path 556 | total_timesteps 5825.
Path 557 | total_timesteps 5834.
Path 558 | total_timesteps 5842.
Path 559 | total_timesteps 5853.
Path 560 | total_timesteps 5862.
Path 561 | total_timesteps 5871.
Path 562 | total_timesteps 5881.
Path 563 | total_timesteps 5891.
Path 564 | total_timesteps 5901.
Path 565 | total_timesteps 5909.
Path 566 | total_timesteps 5917.
Path 567 | total_timesteps 5925.
Path 568 | total_timesteps 5934.
Path 569 | total_timesteps 5943.
Path 570 | total_timesteps 5955.
Path 571 | total_timesteps 5973.
Path 572 | total_timesteps 5983.
Path 573 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.61    |
| Iteration     | 4        |
| MaximumReturn | 4.45     |
| MinimumReturn | -20.3    |
| TotalSamples  | 24031    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.292449951171875
Validation loss = 0.2974434792995453
Validation loss = 0.29989388585090637
Validation loss = 0.31687623262405396
Validation loss = 0.3096359670162201
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 55.
Path 7 | total_timesteps 65.
Path 8 | total_timesteps 73.
Path 9 | total_timesteps 81.
Path 10 | total_timesteps 99.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 132.
Path 14 | total_timesteps 144.
Path 15 | total_timesteps 151.
Path 16 | total_timesteps 159.
Path 17 | total_timesteps 176.
Path 18 | total_timesteps 188.
Path 19 | total_timesteps 195.
Path 20 | total_timesteps 205.
Path 21 | total_timesteps 218.
Path 22 | total_timesteps 227.
Path 23 | total_timesteps 234.
Path 24 | total_timesteps 241.
Path 25 | total_timesteps 249.
Path 26 | total_timesteps 259.
Path 27 | total_timesteps 271.
Path 28 | total_timesteps 279.
Path 29 | total_timesteps 297.
Path 30 | total_timesteps 304.
Path 31 | total_timesteps 319.
Path 32 | total_timesteps 329.
Path 33 | total_timesteps 340.
Path 34 | total_timesteps 350.
Path 35 | total_timesteps 364.
Path 36 | total_timesteps 372.
Path 37 | total_timesteps 379.
Path 38 | total_timesteps 392.
Path 39 | total_timesteps 398.
Path 40 | total_timesteps 408.
Path 41 | total_timesteps 417.
Path 42 | total_timesteps 425.
Path 43 | total_timesteps 433.
Path 44 | total_timesteps 442.
Path 45 | total_timesteps 451.
Path 46 | total_timesteps 462.
Path 47 | total_timesteps 475.
Path 48 | total_timesteps 489.
Path 49 | total_timesteps 503.
Path 50 | total_timesteps 513.
Path 51 | total_timesteps 521.
Path 52 | total_timesteps 532.
Path 53 | total_timesteps 539.
Path 54 | total_timesteps 549.
Path 55 | total_timesteps 562.
Path 56 | total_timesteps 577.
Path 57 | total_timesteps 588.
Path 58 | total_timesteps 601.
Path 59 | total_timesteps 610.
Path 60 | total_timesteps 619.
Path 61 | total_timesteps 627.
Path 62 | total_timesteps 633.
Path 63 | total_timesteps 640.
Path 64 | total_timesteps 649.
Path 65 | total_timesteps 656.
Path 66 | total_timesteps 665.
Path 67 | total_timesteps 673.
Path 68 | total_timesteps 682.
Path 69 | total_timesteps 691.
Path 70 | total_timesteps 703.
Path 71 | total_timesteps 711.
Path 72 | total_timesteps 718.
Path 73 | total_timesteps 730.
Path 74 | total_timesteps 738.
Path 75 | total_timesteps 750.
Path 76 | total_timesteps 758.
Path 77 | total_timesteps 768.
Path 78 | total_timesteps 776.
Path 79 | total_timesteps 787.
Path 80 | total_timesteps 794.
Path 81 | total_timesteps 804.
Path 82 | total_timesteps 815.
Path 83 | total_timesteps 822.
Path 84 | total_timesteps 837.
Path 85 | total_timesteps 847.
Path 86 | total_timesteps 866.
Path 87 | total_timesteps 875.
Path 88 | total_timesteps 882.
Path 89 | total_timesteps 895.
Path 90 | total_timesteps 902.
Path 91 | total_timesteps 909.
Path 92 | total_timesteps 918.
Path 93 | total_timesteps 930.
Path 94 | total_timesteps 940.
Path 95 | total_timesteps 950.
Path 96 | total_timesteps 958.
Path 97 | total_timesteps 965.
Path 98 | total_timesteps 979.
Path 99 | total_timesteps 991.
Path 100 | total_timesteps 1008.
Path 101 | total_timesteps 1016.
Path 102 | total_timesteps 1030.
Path 103 | total_timesteps 1042.
Path 104 | total_timesteps 1050.
Path 105 | total_timesteps 1060.
Path 106 | total_timesteps 1067.
Path 107 | total_timesteps 1074.
Path 108 | total_timesteps 1084.
Path 109 | total_timesteps 1097.
Path 110 | total_timesteps 1110.
Path 111 | total_timesteps 1119.
Path 112 | total_timesteps 1135.
Path 113 | total_timesteps 1144.
Path 114 | total_timesteps 1153.
Path 115 | total_timesteps 1163.
Path 116 | total_timesteps 1178.
Path 117 | total_timesteps 1185.
Path 118 | total_timesteps 1197.
Path 119 | total_timesteps 1206.
Path 120 | total_timesteps 1215.
Path 121 | total_timesteps 1224.
Path 122 | total_timesteps 1233.
Path 123 | total_timesteps 1240.
Path 124 | total_timesteps 1247.
Path 125 | total_timesteps 1255.
Path 126 | total_timesteps 1263.
Path 127 | total_timesteps 1272.
Path 128 | total_timesteps 1279.
Path 129 | total_timesteps 1290.
Path 130 | total_timesteps 1305.
Path 131 | total_timesteps 1316.
Path 132 | total_timesteps 1326.
Path 133 | total_timesteps 1334.
Path 134 | total_timesteps 1345.
Path 135 | total_timesteps 1356.
Path 136 | total_timesteps 1366.
Path 137 | total_timesteps 1373.
Path 138 | total_timesteps 1380.
Path 139 | total_timesteps 1388.
Path 140 | total_timesteps 1397.
Path 141 | total_timesteps 1406.
Path 142 | total_timesteps 1422.
Path 143 | total_timesteps 1428.
Path 144 | total_timesteps 1438.
Path 145 | total_timesteps 1454.
Path 146 | total_timesteps 1464.
Path 147 | total_timesteps 1471.
Path 148 | total_timesteps 1488.
Path 149 | total_timesteps 1495.
Path 150 | total_timesteps 1507.
Path 151 | total_timesteps 1519.
Path 152 | total_timesteps 1532.
Path 153 | total_timesteps 1544.
Path 154 | total_timesteps 1552.
Path 155 | total_timesteps 1559.
Path 156 | total_timesteps 1567.
Path 157 | total_timesteps 1577.
Path 158 | total_timesteps 1587.
Path 159 | total_timesteps 1602.
Path 160 | total_timesteps 1612.
Path 161 | total_timesteps 1624.
Path 162 | total_timesteps 1634.
Path 163 | total_timesteps 1645.
Path 164 | total_timesteps 1656.
Path 165 | total_timesteps 1664.
Path 166 | total_timesteps 1672.
Path 167 | total_timesteps 1682.
Path 168 | total_timesteps 1692.
Path 169 | total_timesteps 1700.
Path 170 | total_timesteps 1707.
Path 171 | total_timesteps 1721.
Path 172 | total_timesteps 1731.
Path 173 | total_timesteps 1739.
Path 174 | total_timesteps 1747.
Path 175 | total_timesteps 1756.
Path 176 | total_timesteps 1775.
Path 177 | total_timesteps 1785.
Path 178 | total_timesteps 1793.
Path 179 | total_timesteps 1809.
Path 180 | total_timesteps 1826.
Path 181 | total_timesteps 1832.
Path 182 | total_timesteps 1840.
Path 183 | total_timesteps 1846.
Path 184 | total_timesteps 1853.
Path 185 | total_timesteps 1869.
Path 186 | total_timesteps 1879.
Path 187 | total_timesteps 1891.
Path 188 | total_timesteps 1909.
Path 189 | total_timesteps 1919.
Path 190 | total_timesteps 1929.
Path 191 | total_timesteps 1936.
Path 192 | total_timesteps 1946.
Path 193 | total_timesteps 1957.
Path 194 | total_timesteps 1968.
Path 195 | total_timesteps 1985.
Path 196 | total_timesteps 1992.
Path 197 | total_timesteps 2001.
Path 198 | total_timesteps 2009.
Path 199 | total_timesteps 2015.
Path 200 | total_timesteps 2023.
Path 201 | total_timesteps 2030.
Path 202 | total_timesteps 2044.
Path 203 | total_timesteps 2053.
Path 204 | total_timesteps 2067.
Path 205 | total_timesteps 2074.
Path 206 | total_timesteps 2083.
Path 207 | total_timesteps 2090.
Path 208 | total_timesteps 2104.
Path 209 | total_timesteps 2112.
Path 210 | total_timesteps 2120.
Path 211 | total_timesteps 2131.
Path 212 | total_timesteps 2140.
Path 213 | total_timesteps 2150.
Path 214 | total_timesteps 2165.
Path 215 | total_timesteps 2176.
Path 216 | total_timesteps 2186.
Path 217 | total_timesteps 2195.
Path 218 | total_timesteps 2201.
Path 219 | total_timesteps 2211.
Path 220 | total_timesteps 2221.
Path 221 | total_timesteps 2230.
Path 222 | total_timesteps 2239.
Path 223 | total_timesteps 2252.
Path 224 | total_timesteps 2259.
Path 225 | total_timesteps 2269.
Path 226 | total_timesteps 2284.
Path 227 | total_timesteps 2297.
Path 228 | total_timesteps 2307.
Path 229 | total_timesteps 2315.
Path 230 | total_timesteps 2328.
Path 231 | total_timesteps 2342.
Path 232 | total_timesteps 2349.
Path 233 | total_timesteps 2358.
Path 234 | total_timesteps 2369.
Path 235 | total_timesteps 2386.
Path 236 | total_timesteps 2398.
Path 237 | total_timesteps 2405.
Path 238 | total_timesteps 2422.
Path 239 | total_timesteps 2430.
Path 240 | total_timesteps 2443.
Path 241 | total_timesteps 2454.
Path 242 | total_timesteps 2462.
Path 243 | total_timesteps 2469.
Path 244 | total_timesteps 2477.
Path 245 | total_timesteps 2490.
Path 246 | total_timesteps 2498.
Path 247 | total_timesteps 2507.
Path 248 | total_timesteps 2517.
Path 249 | total_timesteps 2526.
Path 250 | total_timesteps 2533.
Path 251 | total_timesteps 2540.
Path 252 | total_timesteps 2549.
Path 253 | total_timesteps 2558.
Path 254 | total_timesteps 2571.
Path 255 | total_timesteps 2580.
Path 256 | total_timesteps 2588.
Path 257 | total_timesteps 2598.
Path 258 | total_timesteps 2606.
Path 259 | total_timesteps 2616.
Path 260 | total_timesteps 2628.
Path 261 | total_timesteps 2637.
Path 262 | total_timesteps 2646.
Path 263 | total_timesteps 2653.
Path 264 | total_timesteps 2662.
Path 265 | total_timesteps 2668.
Path 266 | total_timesteps 2689.
Path 267 | total_timesteps 2702.
Path 268 | total_timesteps 2710.
Path 269 | total_timesteps 2720.
Path 270 | total_timesteps 2729.
Path 271 | total_timesteps 2739.
Path 272 | total_timesteps 2750.
Path 273 | total_timesteps 2761.
Path 274 | total_timesteps 2771.
Path 275 | total_timesteps 2780.
Path 276 | total_timesteps 2789.
Path 277 | total_timesteps 2812.
Path 278 | total_timesteps 2821.
Path 279 | total_timesteps 2829.
Path 280 | total_timesteps 2840.
Path 281 | total_timesteps 2847.
Path 282 | total_timesteps 2856.
Path 283 | total_timesteps 2866.
Path 284 | total_timesteps 2879.
Path 285 | total_timesteps 2886.
Path 286 | total_timesteps 2899.
Path 287 | total_timesteps 2905.
Path 288 | total_timesteps 2912.
Path 289 | total_timesteps 2927.
Path 290 | total_timesteps 2943.
Path 291 | total_timesteps 2957.
Path 292 | total_timesteps 2964.
Path 293 | total_timesteps 2974.
Path 294 | total_timesteps 2986.
Path 295 | total_timesteps 2995.
Path 296 | total_timesteps 3008.
Path 297 | total_timesteps 3014.
Path 298 | total_timesteps 3021.
Path 299 | total_timesteps 3031.
Path 300 | total_timesteps 3043.
Path 301 | total_timesteps 3052.
Path 302 | total_timesteps 3066.
Path 303 | total_timesteps 3076.
Path 304 | total_timesteps 3085.
Path 305 | total_timesteps 3093.
Path 306 | total_timesteps 3105.
Path 307 | total_timesteps 3113.
Path 308 | total_timesteps 3127.
Path 309 | total_timesteps 3136.
Path 310 | total_timesteps 3144.
Path 311 | total_timesteps 3152.
Path 312 | total_timesteps 3164.
Path 313 | total_timesteps 3174.
Path 314 | total_timesteps 3183.
Path 315 | total_timesteps 3194.
Path 316 | total_timesteps 3203.
Path 317 | total_timesteps 3215.
Path 318 | total_timesteps 3222.
Path 319 | total_timesteps 3233.
Path 320 | total_timesteps 3242.
Path 321 | total_timesteps 3250.
Path 322 | total_timesteps 3258.
Path 323 | total_timesteps 3270.
Path 324 | total_timesteps 3276.
Path 325 | total_timesteps 3287.
Path 326 | total_timesteps 3300.
Path 327 | total_timesteps 3308.
Path 328 | total_timesteps 3328.
Path 329 | total_timesteps 3337.
Path 330 | total_timesteps 3344.
Path 331 | total_timesteps 3361.
Path 332 | total_timesteps 3375.
Path 333 | total_timesteps 3384.
Path 334 | total_timesteps 3393.
Path 335 | total_timesteps 3401.
Path 336 | total_timesteps 3408.
Path 337 | total_timesteps 3418.
Path 338 | total_timesteps 3437.
Path 339 | total_timesteps 3450.
Path 340 | total_timesteps 3464.
Path 341 | total_timesteps 3471.
Path 342 | total_timesteps 3483.
Path 343 | total_timesteps 3500.
Path 344 | total_timesteps 3510.
Path 345 | total_timesteps 3518.
Path 346 | total_timesteps 3525.
Path 347 | total_timesteps 3538.
Path 348 | total_timesteps 3548.
Path 349 | total_timesteps 3558.
Path 350 | total_timesteps 3569.
Path 351 | total_timesteps 3581.
Path 352 | total_timesteps 3593.
Path 353 | total_timesteps 3602.
Path 354 | total_timesteps 3613.
Path 355 | total_timesteps 3621.
Path 356 | total_timesteps 3627.
Path 357 | total_timesteps 3635.
Path 358 | total_timesteps 3652.
Path 359 | total_timesteps 3663.
Path 360 | total_timesteps 3671.
Path 361 | total_timesteps 3680.
Path 362 | total_timesteps 3697.
Path 363 | total_timesteps 3707.
Path 364 | total_timesteps 3716.
Path 365 | total_timesteps 3724.
Path 366 | total_timesteps 3733.
Path 367 | total_timesteps 3752.
Path 368 | total_timesteps 3759.
Path 369 | total_timesteps 3772.
Path 370 | total_timesteps 3785.
Path 371 | total_timesteps 3796.
Path 372 | total_timesteps 3808.
Path 373 | total_timesteps 3822.
Path 374 | total_timesteps 3829.
Path 375 | total_timesteps 3836.
Path 376 | total_timesteps 3845.
Path 377 | total_timesteps 3854.
Path 378 | total_timesteps 3863.
Path 379 | total_timesteps 3874.
Path 380 | total_timesteps 3882.
Path 381 | total_timesteps 3896.
Path 382 | total_timesteps 3905.
Path 383 | total_timesteps 3912.
Path 384 | total_timesteps 3929.
Path 385 | total_timesteps 3939.
Path 386 | total_timesteps 3950.
Path 387 | total_timesteps 3962.
Path 388 | total_timesteps 3970.
Path 389 | total_timesteps 3978.
Path 390 | total_timesteps 3985.
Path 391 | total_timesteps 3994.
Path 392 | total_timesteps 4004.
Path 393 | total_timesteps 4013.
Path 394 | total_timesteps 4024.
Path 395 | total_timesteps 4031.
Path 396 | total_timesteps 4043.
Path 397 | total_timesteps 4053.
Path 398 | total_timesteps 4062.
Path 399 | total_timesteps 4071.
Path 400 | total_timesteps 4080.
Path 401 | total_timesteps 4097.
Path 402 | total_timesteps 4105.
Path 403 | total_timesteps 4114.
Path 404 | total_timesteps 4127.
Path 405 | total_timesteps 4138.
Path 406 | total_timesteps 4146.
Path 407 | total_timesteps 4155.
Path 408 | total_timesteps 4163.
Path 409 | total_timesteps 4174.
Path 410 | total_timesteps 4183.
Path 411 | total_timesteps 4191.
Path 412 | total_timesteps 4202.
Path 413 | total_timesteps 4212.
Path 414 | total_timesteps 4221.
Path 415 | total_timesteps 4232.
Path 416 | total_timesteps 4241.
Path 417 | total_timesteps 4248.
Path 418 | total_timesteps 4258.
Path 419 | total_timesteps 4269.
Path 420 | total_timesteps 4278.
Path 421 | total_timesteps 4287.
Path 422 | total_timesteps 4303.
Path 423 | total_timesteps 4312.
Path 424 | total_timesteps 4327.
Path 425 | total_timesteps 4335.
Path 426 | total_timesteps 4344.
Path 427 | total_timesteps 4351.
Path 428 | total_timesteps 4358.
Path 429 | total_timesteps 4368.
Path 430 | total_timesteps 4376.
Path 431 | total_timesteps 4388.
Path 432 | total_timesteps 4396.
Path 433 | total_timesteps 4405.
Path 434 | total_timesteps 4416.
Path 435 | total_timesteps 4427.
Path 436 | total_timesteps 4437.
Path 437 | total_timesteps 4448.
Path 438 | total_timesteps 4457.
Path 439 | total_timesteps 4465.
Path 440 | total_timesteps 4477.
Path 441 | total_timesteps 4487.
Path 442 | total_timesteps 4498.
Path 443 | total_timesteps 4512.
Path 444 | total_timesteps 4525.
Path 445 | total_timesteps 4534.
Path 446 | total_timesteps 4543.
Path 447 | total_timesteps 4550.
Path 448 | total_timesteps 4560.
Path 449 | total_timesteps 4572.
Path 450 | total_timesteps 4581.
Path 451 | total_timesteps 4596.
Path 452 | total_timesteps 4604.
Path 453 | total_timesteps 4613.
Path 454 | total_timesteps 4621.
Path 455 | total_timesteps 4630.
Path 456 | total_timesteps 4638.
Path 457 | total_timesteps 4644.
Path 458 | total_timesteps 4651.
Path 459 | total_timesteps 4661.
Path 460 | total_timesteps 4667.
Path 461 | total_timesteps 4676.
Path 462 | total_timesteps 4685.
Path 463 | total_timesteps 4693.
Path 464 | total_timesteps 4707.
Path 465 | total_timesteps 4717.
Path 466 | total_timesteps 4728.
Path 467 | total_timesteps 4741.
Path 468 | total_timesteps 4750.
Path 469 | total_timesteps 4758.
Path 470 | total_timesteps 4773.
Path 471 | total_timesteps 4783.
Path 472 | total_timesteps 4791.
Path 473 | total_timesteps 4801.
Path 474 | total_timesteps 4809.
Path 475 | total_timesteps 4820.
Path 476 | total_timesteps 4831.
Path 477 | total_timesteps 4842.
Path 478 | total_timesteps 4850.
Path 479 | total_timesteps 4862.
Path 480 | total_timesteps 4871.
Path 481 | total_timesteps 4878.
Path 482 | total_timesteps 4889.
Path 483 | total_timesteps 4903.
Path 484 | total_timesteps 4913.
Path 485 | total_timesteps 4920.
Path 486 | total_timesteps 4931.
Path 487 | total_timesteps 4944.
Path 488 | total_timesteps 4951.
Path 489 | total_timesteps 4964.
Path 490 | total_timesteps 4976.
Path 491 | total_timesteps 4984.
Path 492 | total_timesteps 4995.
Path 493 | total_timesteps 5004.
Path 494 | total_timesteps 5014.
Path 495 | total_timesteps 5022.
Path 496 | total_timesteps 5032.
Path 497 | total_timesteps 5039.
Path 498 | total_timesteps 5046.
Path 499 | total_timesteps 5052.
Path 500 | total_timesteps 5058.
Path 501 | total_timesteps 5068.
Path 502 | total_timesteps 5075.
Path 503 | total_timesteps 5084.
Path 504 | total_timesteps 5095.
Path 505 | total_timesteps 5104.
Path 506 | total_timesteps 5114.
Path 507 | total_timesteps 5130.
Path 508 | total_timesteps 5139.
Path 509 | total_timesteps 5147.
Path 510 | total_timesteps 5159.
Path 511 | total_timesteps 5166.
Path 512 | total_timesteps 5181.
Path 513 | total_timesteps 5189.
Path 514 | total_timesteps 5196.
Path 515 | total_timesteps 5205.
Path 516 | total_timesteps 5216.
Path 517 | total_timesteps 5229.
Path 518 | total_timesteps 5244.
Path 519 | total_timesteps 5251.
Path 520 | total_timesteps 5263.
Path 521 | total_timesteps 5272.
Path 522 | total_timesteps 5285.
Path 523 | total_timesteps 5297.
Path 524 | total_timesteps 5309.
Path 525 | total_timesteps 5320.
Path 526 | total_timesteps 5328.
Path 527 | total_timesteps 5338.
Path 528 | total_timesteps 5353.
Path 529 | total_timesteps 5363.
Path 530 | total_timesteps 5372.
Path 531 | total_timesteps 5378.
Path 532 | total_timesteps 5390.
Path 533 | total_timesteps 5401.
Path 534 | total_timesteps 5409.
Path 535 | total_timesteps 5419.
Path 536 | total_timesteps 5426.
Path 537 | total_timesteps 5438.
Path 538 | total_timesteps 5447.
Path 539 | total_timesteps 5464.
Path 540 | total_timesteps 5471.
Path 541 | total_timesteps 5485.
Path 542 | total_timesteps 5492.
Path 543 | total_timesteps 5500.
Path 544 | total_timesteps 5509.
Path 545 | total_timesteps 5522.
Path 546 | total_timesteps 5535.
Path 547 | total_timesteps 5544.
Path 548 | total_timesteps 5556.
Path 549 | total_timesteps 5565.
Path 550 | total_timesteps 5574.
Path 551 | total_timesteps 5587.
Path 552 | total_timesteps 5600.
Path 553 | total_timesteps 5609.
Path 554 | total_timesteps 5619.
Path 555 | total_timesteps 5628.
Path 556 | total_timesteps 5640.
Path 557 | total_timesteps 5649.
Path 558 | total_timesteps 5661.
Path 559 | total_timesteps 5668.
Path 560 | total_timesteps 5676.
Path 561 | total_timesteps 5687.
Path 562 | total_timesteps 5698.
Path 563 | total_timesteps 5707.
Path 564 | total_timesteps 5732.
Path 565 | total_timesteps 5744.
Path 566 | total_timesteps 5751.
Path 567 | total_timesteps 5762.
Path 568 | total_timesteps 5774.
Path 569 | total_timesteps 5788.
Path 570 | total_timesteps 5799.
Path 571 | total_timesteps 5810.
Path 572 | total_timesteps 5820.
Path 573 | total_timesteps 5830.
Path 574 | total_timesteps 5838.
Path 575 | total_timesteps 5846.
Path 576 | total_timesteps 5854.
Path 577 | total_timesteps 5867.
Path 578 | total_timesteps 5878.
Path 579 | total_timesteps 5888.
Path 580 | total_timesteps 5896.
Path 581 | total_timesteps 5903.
Path 582 | total_timesteps 5913.
Path 583 | total_timesteps 5921.
Path 584 | total_timesteps 5930.
Path 585 | total_timesteps 5940.
Path 586 | total_timesteps 5950.
Path 587 | total_timesteps 5957.
Path 588 | total_timesteps 5970.
Path 589 | total_timesteps 5984.
Path 590 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.65    |
| Iteration     | 5        |
| MaximumReturn | 1.4      |
| MinimumReturn | -17.8    |
| TotalSamples  | 28031    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.29691463708877563
Validation loss = 0.301207572221756
Validation loss = 0.30230197310447693
Validation loss = 0.3109114170074463
Validation loss = 0.31108999252319336
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 10.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 65.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 85.
Path 8 | total_timesteps 94.
Path 9 | total_timesteps 104.
Path 10 | total_timesteps 112.
Path 11 | total_timesteps 129.
Path 12 | total_timesteps 138.
Path 13 | total_timesteps 146.
Path 14 | total_timesteps 158.
Path 15 | total_timesteps 169.
Path 16 | total_timesteps 181.
Path 17 | total_timesteps 190.
Path 18 | total_timesteps 199.
Path 19 | total_timesteps 213.
Path 20 | total_timesteps 221.
Path 21 | total_timesteps 228.
Path 22 | total_timesteps 244.
Path 23 | total_timesteps 253.
Path 24 | total_timesteps 262.
Path 25 | total_timesteps 271.
Path 26 | total_timesteps 278.
Path 27 | total_timesteps 289.
Path 28 | total_timesteps 297.
Path 29 | total_timesteps 311.
Path 30 | total_timesteps 320.
Path 31 | total_timesteps 329.
Path 32 | total_timesteps 343.
Path 33 | total_timesteps 356.
Path 34 | total_timesteps 369.
Path 35 | total_timesteps 378.
Path 36 | total_timesteps 386.
Path 37 | total_timesteps 395.
Path 38 | total_timesteps 403.
Path 39 | total_timesteps 414.
Path 40 | total_timesteps 422.
Path 41 | total_timesteps 434.
Path 42 | total_timesteps 442.
Path 43 | total_timesteps 454.
Path 44 | total_timesteps 461.
Path 45 | total_timesteps 469.
Path 46 | total_timesteps 478.
Path 47 | total_timesteps 487.
Path 48 | total_timesteps 499.
Path 49 | total_timesteps 507.
Path 50 | total_timesteps 519.
Path 51 | total_timesteps 530.
Path 52 | total_timesteps 546.
Path 53 | total_timesteps 554.
Path 54 | total_timesteps 563.
Path 55 | total_timesteps 576.
Path 56 | total_timesteps 585.
Path 57 | total_timesteps 593.
Path 58 | total_timesteps 604.
Path 59 | total_timesteps 618.
Path 60 | total_timesteps 631.
Path 61 | total_timesteps 640.
Path 62 | total_timesteps 649.
Path 63 | total_timesteps 661.
Path 64 | total_timesteps 672.
Path 65 | total_timesteps 685.
Path 66 | total_timesteps 693.
Path 67 | total_timesteps 706.
Path 68 | total_timesteps 714.
Path 69 | total_timesteps 724.
Path 70 | total_timesteps 735.
Path 71 | total_timesteps 745.
Path 72 | total_timesteps 755.
Path 73 | total_timesteps 771.
Path 74 | total_timesteps 779.
Path 75 | total_timesteps 787.
Path 76 | total_timesteps 801.
Path 77 | total_timesteps 817.
Path 78 | total_timesteps 827.
Path 79 | total_timesteps 835.
Path 80 | total_timesteps 846.
Path 81 | total_timesteps 854.
Path 82 | total_timesteps 864.
Path 83 | total_timesteps 876.
Path 84 | total_timesteps 890.
Path 85 | total_timesteps 911.
Path 86 | total_timesteps 919.
Path 87 | total_timesteps 929.
Path 88 | total_timesteps 937.
Path 89 | total_timesteps 945.
Path 90 | total_timesteps 953.
Path 91 | total_timesteps 961.
Path 92 | total_timesteps 973.
Path 93 | total_timesteps 981.
Path 94 | total_timesteps 989.
Path 95 | total_timesteps 998.
Path 96 | total_timesteps 1008.
Path 97 | total_timesteps 1016.
Path 98 | total_timesteps 1026.
Path 99 | total_timesteps 1033.
Path 100 | total_timesteps 1042.
Path 101 | total_timesteps 1053.
Path 102 | total_timesteps 1062.
Path 103 | total_timesteps 1068.
Path 104 | total_timesteps 1081.
Path 105 | total_timesteps 1091.
Path 106 | total_timesteps 1100.
Path 107 | total_timesteps 1109.
Path 108 | total_timesteps 1120.
Path 109 | total_timesteps 1129.
Path 110 | total_timesteps 1138.
Path 111 | total_timesteps 1145.
Path 112 | total_timesteps 1156.
Path 113 | total_timesteps 1166.
Path 114 | total_timesteps 1174.
Path 115 | total_timesteps 1182.
Path 116 | total_timesteps 1197.
Path 117 | total_timesteps 1208.
Path 118 | total_timesteps 1218.
Path 119 | total_timesteps 1230.
Path 120 | total_timesteps 1238.
Path 121 | total_timesteps 1246.
Path 122 | total_timesteps 1255.
Path 123 | total_timesteps 1266.
Path 124 | total_timesteps 1275.
Path 125 | total_timesteps 1285.
Path 126 | total_timesteps 1293.
Path 127 | total_timesteps 1301.
Path 128 | total_timesteps 1314.
Path 129 | total_timesteps 1323.
Path 130 | total_timesteps 1332.
Path 131 | total_timesteps 1339.
Path 132 | total_timesteps 1350.
Path 133 | total_timesteps 1359.
Path 134 | total_timesteps 1378.
Path 135 | total_timesteps 1386.
Path 136 | total_timesteps 1401.
Path 137 | total_timesteps 1414.
Path 138 | total_timesteps 1427.
Path 139 | total_timesteps 1439.
Path 140 | total_timesteps 1451.
Path 141 | total_timesteps 1460.
Path 142 | total_timesteps 1479.
Path 143 | total_timesteps 1485.
Path 144 | total_timesteps 1493.
Path 145 | total_timesteps 1502.
Path 146 | total_timesteps 1516.
Path 147 | total_timesteps 1525.
Path 148 | total_timesteps 1536.
Path 149 | total_timesteps 1543.
Path 150 | total_timesteps 1553.
Path 151 | total_timesteps 1563.
Path 152 | total_timesteps 1571.
Path 153 | total_timesteps 1580.
Path 154 | total_timesteps 1594.
Path 155 | total_timesteps 1605.
Path 156 | total_timesteps 1619.
Path 157 | total_timesteps 1636.
Path 158 | total_timesteps 1644.
Path 159 | total_timesteps 1654.
Path 160 | total_timesteps 1669.
Path 161 | total_timesteps 1679.
Path 162 | total_timesteps 1686.
Path 163 | total_timesteps 1693.
Path 164 | total_timesteps 1704.
Path 165 | total_timesteps 1711.
Path 166 | total_timesteps 1722.
Path 167 | total_timesteps 1730.
Path 168 | total_timesteps 1740.
Path 169 | total_timesteps 1746.
Path 170 | total_timesteps 1752.
Path 171 | total_timesteps 1760.
Path 172 | total_timesteps 1770.
Path 173 | total_timesteps 1790.
Path 174 | total_timesteps 1800.
Path 175 | total_timesteps 1811.
Path 176 | total_timesteps 1819.
Path 177 | total_timesteps 1826.
Path 178 | total_timesteps 1838.
Path 179 | total_timesteps 1847.
Path 180 | total_timesteps 1857.
Path 181 | total_timesteps 1864.
Path 182 | total_timesteps 1871.
Path 183 | total_timesteps 1880.
Path 184 | total_timesteps 1891.
Path 185 | total_timesteps 1898.
Path 186 | total_timesteps 1905.
Path 187 | total_timesteps 1917.
Path 188 | total_timesteps 1934.
Path 189 | total_timesteps 1943.
Path 190 | total_timesteps 1952.
Path 191 | total_timesteps 1960.
Path 192 | total_timesteps 1973.
Path 193 | total_timesteps 1982.
Path 194 | total_timesteps 1990.
Path 195 | total_timesteps 2000.
Path 196 | total_timesteps 2018.
Path 197 | total_timesteps 2028.
Path 198 | total_timesteps 2037.
Path 199 | total_timesteps 2043.
Path 200 | total_timesteps 2051.
Path 201 | total_timesteps 2062.
Path 202 | total_timesteps 2078.
Path 203 | total_timesteps 2089.
Path 204 | total_timesteps 2096.
Path 205 | total_timesteps 2103.
Path 206 | total_timesteps 2110.
Path 207 | total_timesteps 2119.
Path 208 | total_timesteps 2127.
Path 209 | total_timesteps 2137.
Path 210 | total_timesteps 2148.
Path 211 | total_timesteps 2158.
Path 212 | total_timesteps 2170.
Path 213 | total_timesteps 2180.
Path 214 | total_timesteps 2188.
Path 215 | total_timesteps 2197.
Path 216 | total_timesteps 2206.
Path 217 | total_timesteps 2219.
Path 218 | total_timesteps 2229.
Path 219 | total_timesteps 2239.
Path 220 | total_timesteps 2246.
Path 221 | total_timesteps 2261.
Path 222 | total_timesteps 2268.
Path 223 | total_timesteps 2281.
Path 224 | total_timesteps 2292.
Path 225 | total_timesteps 2300.
Path 226 | total_timesteps 2307.
Path 227 | total_timesteps 2319.
Path 228 | total_timesteps 2331.
Path 229 | total_timesteps 2343.
Path 230 | total_timesteps 2354.
Path 231 | total_timesteps 2361.
Path 232 | total_timesteps 2370.
Path 233 | total_timesteps 2379.
Path 234 | total_timesteps 2389.
Path 235 | total_timesteps 2396.
Path 236 | total_timesteps 2409.
Path 237 | total_timesteps 2421.
Path 238 | total_timesteps 2429.
Path 239 | total_timesteps 2435.
Path 240 | total_timesteps 2444.
Path 241 | total_timesteps 2455.
Path 242 | total_timesteps 2464.
Path 243 | total_timesteps 2470.
Path 244 | total_timesteps 2485.
Path 245 | total_timesteps 2492.
Path 246 | total_timesteps 2498.
Path 247 | total_timesteps 2508.
Path 248 | total_timesteps 2516.
Path 249 | total_timesteps 2526.
Path 250 | total_timesteps 2542.
Path 251 | total_timesteps 2553.
Path 252 | total_timesteps 2565.
Path 253 | total_timesteps 2572.
Path 254 | total_timesteps 2580.
Path 255 | total_timesteps 2592.
Path 256 | total_timesteps 2600.
Path 257 | total_timesteps 2608.
Path 258 | total_timesteps 2617.
Path 259 | total_timesteps 2627.
Path 260 | total_timesteps 2637.
Path 261 | total_timesteps 2645.
Path 262 | total_timesteps 2659.
Path 263 | total_timesteps 2666.
Path 264 | total_timesteps 2676.
Path 265 | total_timesteps 2684.
Path 266 | total_timesteps 2697.
Path 267 | total_timesteps 2709.
Path 268 | total_timesteps 2717.
Path 269 | total_timesteps 2726.
Path 270 | total_timesteps 2737.
Path 271 | total_timesteps 2749.
Path 272 | total_timesteps 2759.
Path 273 | total_timesteps 2769.
Path 274 | total_timesteps 2781.
Path 275 | total_timesteps 2794.
Path 276 | total_timesteps 2802.
Path 277 | total_timesteps 2813.
Path 278 | total_timesteps 2823.
Path 279 | total_timesteps 2837.
Path 280 | total_timesteps 2848.
Path 281 | total_timesteps 2859.
Path 282 | total_timesteps 2870.
Path 283 | total_timesteps 2878.
Path 284 | total_timesteps 2890.
Path 285 | total_timesteps 2899.
Path 286 | total_timesteps 2906.
Path 287 | total_timesteps 2914.
Path 288 | total_timesteps 2921.
Path 289 | total_timesteps 2930.
Path 290 | total_timesteps 2937.
Path 291 | total_timesteps 2949.
Path 292 | total_timesteps 2958.
Path 293 | total_timesteps 2969.
Path 294 | total_timesteps 2976.
Path 295 | total_timesteps 2990.
Path 296 | total_timesteps 2999.
Path 297 | total_timesteps 3009.
Path 298 | total_timesteps 3018.
Path 299 | total_timesteps 3029.
Path 300 | total_timesteps 3040.
Path 301 | total_timesteps 3049.
Path 302 | total_timesteps 3057.
Path 303 | total_timesteps 3067.
Path 304 | total_timesteps 3075.
Path 305 | total_timesteps 3086.
Path 306 | total_timesteps 3095.
Path 307 | total_timesteps 3106.
Path 308 | total_timesteps 3125.
Path 309 | total_timesteps 3135.
Path 310 | total_timesteps 3142.
Path 311 | total_timesteps 3152.
Path 312 | total_timesteps 3162.
Path 313 | total_timesteps 3169.
Path 314 | total_timesteps 3176.
Path 315 | total_timesteps 3185.
Path 316 | total_timesteps 3191.
Path 317 | total_timesteps 3203.
Path 318 | total_timesteps 3211.
Path 319 | total_timesteps 3219.
Path 320 | total_timesteps 3228.
Path 321 | total_timesteps 3237.
Path 322 | total_timesteps 3246.
Path 323 | total_timesteps 3255.
Path 324 | total_timesteps 3264.
Path 325 | total_timesteps 3271.
Path 326 | total_timesteps 3280.
Path 327 | total_timesteps 3289.
Path 328 | total_timesteps 3298.
Path 329 | total_timesteps 3305.
Path 330 | total_timesteps 3314.
Path 331 | total_timesteps 3322.
Path 332 | total_timesteps 3329.
Path 333 | total_timesteps 3344.
Path 334 | total_timesteps 3352.
Path 335 | total_timesteps 3361.
Path 336 | total_timesteps 3369.
Path 337 | total_timesteps 3384.
Path 338 | total_timesteps 3393.
Path 339 | total_timesteps 3404.
Path 340 | total_timesteps 3413.
Path 341 | total_timesteps 3421.
Path 342 | total_timesteps 3436.
Path 343 | total_timesteps 3446.
Path 344 | total_timesteps 3453.
Path 345 | total_timesteps 3466.
Path 346 | total_timesteps 3475.
Path 347 | total_timesteps 3484.
Path 348 | total_timesteps 3490.
Path 349 | total_timesteps 3501.
Path 350 | total_timesteps 3509.
Path 351 | total_timesteps 3518.
Path 352 | total_timesteps 3531.
Path 353 | total_timesteps 3543.
Path 354 | total_timesteps 3552.
Path 355 | total_timesteps 3560.
Path 356 | total_timesteps 3568.
Path 357 | total_timesteps 3582.
Path 358 | total_timesteps 3593.
Path 359 | total_timesteps 3608.
Path 360 | total_timesteps 3614.
Path 361 | total_timesteps 3620.
Path 362 | total_timesteps 3632.
Path 363 | total_timesteps 3645.
Path 364 | total_timesteps 3658.
Path 365 | total_timesteps 3668.
Path 366 | total_timesteps 3677.
Path 367 | total_timesteps 3687.
Path 368 | total_timesteps 3701.
Path 369 | total_timesteps 3714.
Path 370 | total_timesteps 3724.
Path 371 | total_timesteps 3733.
Path 372 | total_timesteps 3740.
Path 373 | total_timesteps 3750.
Path 374 | total_timesteps 3767.
Path 375 | total_timesteps 3778.
Path 376 | total_timesteps 3793.
Path 377 | total_timesteps 3801.
Path 378 | total_timesteps 3817.
Path 379 | total_timesteps 3827.
Path 380 | total_timesteps 3838.
Path 381 | total_timesteps 3845.
Path 382 | total_timesteps 3855.
Path 383 | total_timesteps 3862.
Path 384 | total_timesteps 3877.
Path 385 | total_timesteps 3884.
Path 386 | total_timesteps 3892.
Path 387 | total_timesteps 3904.
Path 388 | total_timesteps 3910.
Path 389 | total_timesteps 3920.
Path 390 | total_timesteps 3927.
Path 391 | total_timesteps 3940.
Path 392 | total_timesteps 3948.
Path 393 | total_timesteps 3956.
Path 394 | total_timesteps 3965.
Path 395 | total_timesteps 3980.
Path 396 | total_timesteps 3989.
Path 397 | total_timesteps 4000.
Path 398 | total_timesteps 4010.
Path 399 | total_timesteps 4017.
Path 400 | total_timesteps 4026.
Path 401 | total_timesteps 4035.
Path 402 | total_timesteps 4045.
Path 403 | total_timesteps 4053.
Path 404 | total_timesteps 4061.
Path 405 | total_timesteps 4075.
Path 406 | total_timesteps 4083.
Path 407 | total_timesteps 4092.
Path 408 | total_timesteps 4102.
Path 409 | total_timesteps 4110.
Path 410 | total_timesteps 4123.
Path 411 | total_timesteps 4131.
Path 412 | total_timesteps 4140.
Path 413 | total_timesteps 4150.
Path 414 | total_timesteps 4165.
Path 415 | total_timesteps 4174.
Path 416 | total_timesteps 4182.
Path 417 | total_timesteps 4189.
Path 418 | total_timesteps 4201.
Path 419 | total_timesteps 4211.
Path 420 | total_timesteps 4218.
Path 421 | total_timesteps 4226.
Path 422 | total_timesteps 4234.
Path 423 | total_timesteps 4246.
Path 424 | total_timesteps 4257.
Path 425 | total_timesteps 4270.
Path 426 | total_timesteps 4279.
Path 427 | total_timesteps 4293.
Path 428 | total_timesteps 4300.
Path 429 | total_timesteps 4311.
Path 430 | total_timesteps 4319.
Path 431 | total_timesteps 4326.
Path 432 | total_timesteps 4339.
Path 433 | total_timesteps 4350.
Path 434 | total_timesteps 4360.
Path 435 | total_timesteps 4367.
Path 436 | total_timesteps 4375.
Path 437 | total_timesteps 4384.
Path 438 | total_timesteps 4395.
Path 439 | total_timesteps 4404.
Path 440 | total_timesteps 4411.
Path 441 | total_timesteps 4420.
Path 442 | total_timesteps 4431.
Path 443 | total_timesteps 4440.
Path 444 | total_timesteps 4449.
Path 445 | total_timesteps 4462.
Path 446 | total_timesteps 4473.
Path 447 | total_timesteps 4482.
Path 448 | total_timesteps 4494.
Path 449 | total_timesteps 4501.
Path 450 | total_timesteps 4511.
Path 451 | total_timesteps 4519.
Path 452 | total_timesteps 4529.
Path 453 | total_timesteps 4538.
Path 454 | total_timesteps 4549.
Path 455 | total_timesteps 4562.
Path 456 | total_timesteps 4570.
Path 457 | total_timesteps 4579.
Path 458 | total_timesteps 4588.
Path 459 | total_timesteps 4601.
Path 460 | total_timesteps 4607.
Path 461 | total_timesteps 4615.
Path 462 | total_timesteps 4624.
Path 463 | total_timesteps 4631.
Path 464 | total_timesteps 4640.
Path 465 | total_timesteps 4647.
Path 466 | total_timesteps 4655.
Path 467 | total_timesteps 4663.
Path 468 | total_timesteps 4672.
Path 469 | total_timesteps 4683.
Path 470 | total_timesteps 4690.
Path 471 | total_timesteps 4702.
Path 472 | total_timesteps 4710.
Path 473 | total_timesteps 4721.
Path 474 | total_timesteps 4730.
Path 475 | total_timesteps 4744.
Path 476 | total_timesteps 4755.
Path 477 | total_timesteps 4763.
Path 478 | total_timesteps 4772.
Path 479 | total_timesteps 4781.
Path 480 | total_timesteps 4789.
Path 481 | total_timesteps 4803.
Path 482 | total_timesteps 4814.
Path 483 | total_timesteps 4826.
Path 484 | total_timesteps 4836.
Path 485 | total_timesteps 4846.
Path 486 | total_timesteps 4854.
Path 487 | total_timesteps 4866.
Path 488 | total_timesteps 4875.
Path 489 | total_timesteps 4890.
Path 490 | total_timesteps 4900.
Path 491 | total_timesteps 4909.
Path 492 | total_timesteps 4918.
Path 493 | total_timesteps 4924.
Path 494 | total_timesteps 4935.
Path 495 | total_timesteps 4945.
Path 496 | total_timesteps 4954.
Path 497 | total_timesteps 4965.
Path 498 | total_timesteps 4972.
Path 499 | total_timesteps 4982.
Path 500 | total_timesteps 4992.
Path 501 | total_timesteps 5000.
Path 502 | total_timesteps 5008.
Path 503 | total_timesteps 5019.
Path 504 | total_timesteps 5029.
Path 505 | total_timesteps 5038.
Path 506 | total_timesteps 5052.
Path 507 | total_timesteps 5060.
Path 508 | total_timesteps 5067.
Path 509 | total_timesteps 5076.
Path 510 | total_timesteps 5087.
Path 511 | total_timesteps 5095.
Path 512 | total_timesteps 5106.
Path 513 | total_timesteps 5114.
Path 514 | total_timesteps 5122.
Path 515 | total_timesteps 5139.
Path 516 | total_timesteps 5148.
Path 517 | total_timesteps 5155.
Path 518 | total_timesteps 5164.
Path 519 | total_timesteps 5180.
Path 520 | total_timesteps 5187.
Path 521 | total_timesteps 5200.
Path 522 | total_timesteps 5207.
Path 523 | total_timesteps 5215.
Path 524 | total_timesteps 5222.
Path 525 | total_timesteps 5229.
Path 526 | total_timesteps 5236.
Path 527 | total_timesteps 5245.
Path 528 | total_timesteps 5253.
Path 529 | total_timesteps 5263.
Path 530 | total_timesteps 5273.
Path 531 | total_timesteps 5287.
Path 532 | total_timesteps 5300.
Path 533 | total_timesteps 5310.
Path 534 | total_timesteps 5320.
Path 535 | total_timesteps 5332.
Path 536 | total_timesteps 5348.
Path 537 | total_timesteps 5360.
Path 538 | total_timesteps 5367.
Path 539 | total_timesteps 5377.
Path 540 | total_timesteps 5393.
Path 541 | total_timesteps 5406.
Path 542 | total_timesteps 5416.
Path 543 | total_timesteps 5424.
Path 544 | total_timesteps 5440.
Path 545 | total_timesteps 5448.
Path 546 | total_timesteps 5460.
Path 547 | total_timesteps 5469.
Path 548 | total_timesteps 5479.
Path 549 | total_timesteps 5488.
Path 550 | total_timesteps 5496.
Path 551 | total_timesteps 5511.
Path 552 | total_timesteps 5521.
Path 553 | total_timesteps 5534.
Path 554 | total_timesteps 5543.
Path 555 | total_timesteps 5555.
Path 556 | total_timesteps 5565.
Path 557 | total_timesteps 5571.
Path 558 | total_timesteps 5578.
Path 559 | total_timesteps 5590.
Path 560 | total_timesteps 5601.
Path 561 | total_timesteps 5613.
Path 562 | total_timesteps 5624.
Path 563 | total_timesteps 5632.
Path 564 | total_timesteps 5646.
Path 565 | total_timesteps 5662.
Path 566 | total_timesteps 5679.
Path 567 | total_timesteps 5687.
Path 568 | total_timesteps 5699.
Path 569 | total_timesteps 5707.
Path 570 | total_timesteps 5716.
Path 571 | total_timesteps 5725.
Path 572 | total_timesteps 5735.
Path 573 | total_timesteps 5744.
Path 574 | total_timesteps 5757.
Path 575 | total_timesteps 5765.
Path 576 | total_timesteps 5773.
Path 577 | total_timesteps 5780.
Path 578 | total_timesteps 5794.
Path 579 | total_timesteps 5803.
Path 580 | total_timesteps 5810.
Path 581 | total_timesteps 5819.
Path 582 | total_timesteps 5831.
Path 583 | total_timesteps 5840.
Path 584 | total_timesteps 5849.
Path 585 | total_timesteps 5859.
Path 586 | total_timesteps 5880.
Path 587 | total_timesteps 5890.
Path 588 | total_timesteps 5898.
Path 589 | total_timesteps 5907.
Path 590 | total_timesteps 5919.
Path 591 | total_timesteps 5931.
Path 592 | total_timesteps 5948.
Path 593 | total_timesteps 5954.
Path 594 | total_timesteps 5967.
Path 595 | total_timesteps 5984.
Path 596 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.86    |
| Iteration     | 6        |
| MaximumReturn | 2.92     |
| MinimumReturn | -18.4    |
| TotalSamples  | 32031    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.2981738746166229
Validation loss = 0.29821282625198364
Validation loss = 0.30438339710235596
Validation loss = 0.31251394748687744
Validation loss = 0.3118784427642822
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 28.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 42.
Path 6 | total_timesteps 54.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 72.
Path 9 | total_timesteps 81.
Path 10 | total_timesteps 96.
Path 11 | total_timesteps 105.
Path 12 | total_timesteps 114.
Path 13 | total_timesteps 125.
Path 14 | total_timesteps 134.
Path 15 | total_timesteps 143.
Path 16 | total_timesteps 152.
Path 17 | total_timesteps 165.
Path 18 | total_timesteps 174.
Path 19 | total_timesteps 182.
Path 20 | total_timesteps 196.
Path 21 | total_timesteps 205.
Path 22 | total_timesteps 212.
Path 23 | total_timesteps 219.
Path 24 | total_timesteps 229.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 246.
Path 27 | total_timesteps 257.
Path 28 | total_timesteps 267.
Path 29 | total_timesteps 275.
Path 30 | total_timesteps 285.
Path 31 | total_timesteps 297.
Path 32 | total_timesteps 307.
Path 33 | total_timesteps 314.
Path 34 | total_timesteps 320.
Path 35 | total_timesteps 328.
Path 36 | total_timesteps 337.
Path 37 | total_timesteps 351.
Path 38 | total_timesteps 359.
Path 39 | total_timesteps 373.
Path 40 | total_timesteps 379.
Path 41 | total_timesteps 387.
Path 42 | total_timesteps 394.
Path 43 | total_timesteps 409.
Path 44 | total_timesteps 418.
Path 45 | total_timesteps 429.
Path 46 | total_timesteps 438.
Path 47 | total_timesteps 445.
Path 48 | total_timesteps 453.
Path 49 | total_timesteps 461.
Path 50 | total_timesteps 472.
Path 51 | total_timesteps 480.
Path 52 | total_timesteps 489.
Path 53 | total_timesteps 501.
Path 54 | total_timesteps 508.
Path 55 | total_timesteps 517.
Path 56 | total_timesteps 526.
Path 57 | total_timesteps 532.
Path 58 | total_timesteps 541.
Path 59 | total_timesteps 553.
Path 60 | total_timesteps 570.
Path 61 | total_timesteps 584.
Path 62 | total_timesteps 591.
Path 63 | total_timesteps 601.
Path 64 | total_timesteps 613.
Path 65 | total_timesteps 621.
Path 66 | total_timesteps 632.
Path 67 | total_timesteps 643.
Path 68 | total_timesteps 654.
Path 69 | total_timesteps 663.
Path 70 | total_timesteps 673.
Path 71 | total_timesteps 683.
Path 72 | total_timesteps 695.
Path 73 | total_timesteps 709.
Path 74 | total_timesteps 719.
Path 75 | total_timesteps 727.
Path 76 | total_timesteps 750.
Path 77 | total_timesteps 757.
Path 78 | total_timesteps 764.
Path 79 | total_timesteps 773.
Path 80 | total_timesteps 781.
Path 81 | total_timesteps 793.
Path 82 | total_timesteps 802.
Path 83 | total_timesteps 815.
Path 84 | total_timesteps 822.
Path 85 | total_timesteps 845.
Path 86 | total_timesteps 860.
Path 87 | total_timesteps 868.
Path 88 | total_timesteps 878.
Path 89 | total_timesteps 891.
Path 90 | total_timesteps 907.
Path 91 | total_timesteps 916.
Path 92 | total_timesteps 925.
Path 93 | total_timesteps 938.
Path 94 | total_timesteps 945.
Path 95 | total_timesteps 956.
Path 96 | total_timesteps 966.
Path 97 | total_timesteps 976.
Path 98 | total_timesteps 987.
Path 99 | total_timesteps 996.
Path 100 | total_timesteps 1003.
Path 101 | total_timesteps 1016.
Path 102 | total_timesteps 1023.
Path 103 | total_timesteps 1035.
Path 104 | total_timesteps 1044.
Path 105 | total_timesteps 1054.
Path 106 | total_timesteps 1071.
Path 107 | total_timesteps 1080.
Path 108 | total_timesteps 1089.
Path 109 | total_timesteps 1105.
Path 110 | total_timesteps 1117.
Path 111 | total_timesteps 1128.
Path 112 | total_timesteps 1138.
Path 113 | total_timesteps 1149.
Path 114 | total_timesteps 1157.
Path 115 | total_timesteps 1165.
Path 116 | total_timesteps 1173.
Path 117 | total_timesteps 1184.
Path 118 | total_timesteps 1194.
Path 119 | total_timesteps 1201.
Path 120 | total_timesteps 1209.
Path 121 | total_timesteps 1224.
Path 122 | total_timesteps 1238.
Path 123 | total_timesteps 1247.
Path 124 | total_timesteps 1254.
Path 125 | total_timesteps 1262.
Path 126 | total_timesteps 1271.
Path 127 | total_timesteps 1285.
Path 128 | total_timesteps 1293.
Path 129 | total_timesteps 1300.
Path 130 | total_timesteps 1309.
Path 131 | total_timesteps 1317.
Path 132 | total_timesteps 1331.
Path 133 | total_timesteps 1340.
Path 134 | total_timesteps 1348.
Path 135 | total_timesteps 1356.
Path 136 | total_timesteps 1364.
Path 137 | total_timesteps 1374.
Path 138 | total_timesteps 1387.
Path 139 | total_timesteps 1394.
Path 140 | total_timesteps 1406.
Path 141 | total_timesteps 1416.
Path 142 | total_timesteps 1432.
Path 143 | total_timesteps 1441.
Path 144 | total_timesteps 1450.
Path 145 | total_timesteps 1457.
Path 146 | total_timesteps 1464.
Path 147 | total_timesteps 1477.
Path 148 | total_timesteps 1486.
Path 149 | total_timesteps 1495.
Path 150 | total_timesteps 1503.
Path 151 | total_timesteps 1513.
Path 152 | total_timesteps 1522.
Path 153 | total_timesteps 1530.
Path 154 | total_timesteps 1544.
Path 155 | total_timesteps 1554.
Path 156 | total_timesteps 1573.
Path 157 | total_timesteps 1587.
Path 158 | total_timesteps 1595.
Path 159 | total_timesteps 1608.
Path 160 | total_timesteps 1622.
Path 161 | total_timesteps 1633.
Path 162 | total_timesteps 1647.
Path 163 | total_timesteps 1657.
Path 164 | total_timesteps 1669.
Path 165 | total_timesteps 1676.
Path 166 | total_timesteps 1684.
Path 167 | total_timesteps 1700.
Path 168 | total_timesteps 1714.
Path 169 | total_timesteps 1728.
Path 170 | total_timesteps 1736.
Path 171 | total_timesteps 1746.
Path 172 | total_timesteps 1754.
Path 173 | total_timesteps 1761.
Path 174 | total_timesteps 1770.
Path 175 | total_timesteps 1783.
Path 176 | total_timesteps 1795.
Path 177 | total_timesteps 1811.
Path 178 | total_timesteps 1820.
Path 179 | total_timesteps 1830.
Path 180 | total_timesteps 1839.
Path 181 | total_timesteps 1848.
Path 182 | total_timesteps 1858.
Path 183 | total_timesteps 1866.
Path 184 | total_timesteps 1879.
Path 185 | total_timesteps 1888.
Path 186 | total_timesteps 1899.
Path 187 | total_timesteps 1908.
Path 188 | total_timesteps 1917.
Path 189 | total_timesteps 1926.
Path 190 | total_timesteps 1934.
Path 191 | total_timesteps 1943.
Path 192 | total_timesteps 1961.
Path 193 | total_timesteps 1974.
Path 194 | total_timesteps 1988.
Path 195 | total_timesteps 1995.
Path 196 | total_timesteps 2003.
Path 197 | total_timesteps 2013.
Path 198 | total_timesteps 2021.
Path 199 | total_timesteps 2036.
Path 200 | total_timesteps 2045.
Path 201 | total_timesteps 2053.
Path 202 | total_timesteps 2082.
Path 203 | total_timesteps 2092.
Path 204 | total_timesteps 2101.
Path 205 | total_timesteps 2116.
Path 206 | total_timesteps 2125.
Path 207 | total_timesteps 2134.
Path 208 | total_timesteps 2142.
Path 209 | total_timesteps 2153.
Path 210 | total_timesteps 2162.
Path 211 | total_timesteps 2168.
Path 212 | total_timesteps 2175.
Path 213 | total_timesteps 2184.
Path 214 | total_timesteps 2193.
Path 215 | total_timesteps 2202.
Path 216 | total_timesteps 2212.
Path 217 | total_timesteps 2220.
Path 218 | total_timesteps 2229.
Path 219 | total_timesteps 2247.
Path 220 | total_timesteps 2256.
Path 221 | total_timesteps 2267.
Path 222 | total_timesteps 2287.
Path 223 | total_timesteps 2296.
Path 224 | total_timesteps 2307.
Path 225 | total_timesteps 2317.
Path 226 | total_timesteps 2330.
Path 227 | total_timesteps 2339.
Path 228 | total_timesteps 2348.
Path 229 | total_timesteps 2356.
Path 230 | total_timesteps 2365.
Path 231 | total_timesteps 2373.
Path 232 | total_timesteps 2385.
Path 233 | total_timesteps 2395.
Path 234 | total_timesteps 2408.
Path 235 | total_timesteps 2418.
Path 236 | total_timesteps 2430.
Path 237 | total_timesteps 2437.
Path 238 | total_timesteps 2449.
Path 239 | total_timesteps 2460.
Path 240 | total_timesteps 2468.
Path 241 | total_timesteps 2475.
Path 242 | total_timesteps 2482.
Path 243 | total_timesteps 2495.
Path 244 | total_timesteps 2507.
Path 245 | total_timesteps 2516.
Path 246 | total_timesteps 2526.
Path 247 | total_timesteps 2534.
Path 248 | total_timesteps 2547.
Path 249 | total_timesteps 2562.
Path 250 | total_timesteps 2572.
Path 251 | total_timesteps 2584.
Path 252 | total_timesteps 2591.
Path 253 | total_timesteps 2600.
Path 254 | total_timesteps 2611.
Path 255 | total_timesteps 2623.
Path 256 | total_timesteps 2630.
Path 257 | total_timesteps 2639.
Path 258 | total_timesteps 2647.
Path 259 | total_timesteps 2658.
Path 260 | total_timesteps 2667.
Path 261 | total_timesteps 2675.
Path 262 | total_timesteps 2686.
Path 263 | total_timesteps 2693.
Path 264 | total_timesteps 2704.
Path 265 | total_timesteps 2716.
Path 266 | total_timesteps 2734.
Path 267 | total_timesteps 2743.
Path 268 | total_timesteps 2751.
Path 269 | total_timesteps 2760.
Path 270 | total_timesteps 2768.
Path 271 | total_timesteps 2776.
Path 272 | total_timesteps 2783.
Path 273 | total_timesteps 2797.
Path 274 | total_timesteps 2806.
Path 275 | total_timesteps 2814.
Path 276 | total_timesteps 2826.
Path 277 | total_timesteps 2837.
Path 278 | total_timesteps 2847.
Path 279 | total_timesteps 2854.
Path 280 | total_timesteps 2866.
Path 281 | total_timesteps 2880.
Path 282 | total_timesteps 2890.
Path 283 | total_timesteps 2899.
Path 284 | total_timesteps 2905.
Path 285 | total_timesteps 2913.
Path 286 | total_timesteps 2925.
Path 287 | total_timesteps 2934.
Path 288 | total_timesteps 2949.
Path 289 | total_timesteps 2960.
Path 290 | total_timesteps 2968.
Path 291 | total_timesteps 2976.
Path 292 | total_timesteps 2987.
Path 293 | total_timesteps 2997.
Path 294 | total_timesteps 3008.
Path 295 | total_timesteps 3017.
Path 296 | total_timesteps 3026.
Path 297 | total_timesteps 3032.
Path 298 | total_timesteps 3039.
Path 299 | total_timesteps 3046.
Path 300 | total_timesteps 3053.
Path 301 | total_timesteps 3062.
Path 302 | total_timesteps 3073.
Path 303 | total_timesteps 3093.
Path 304 | total_timesteps 3100.
Path 305 | total_timesteps 3109.
Path 306 | total_timesteps 3118.
Path 307 | total_timesteps 3135.
Path 308 | total_timesteps 3148.
Path 309 | total_timesteps 3168.
Path 310 | total_timesteps 3179.
Path 311 | total_timesteps 3188.
Path 312 | total_timesteps 3207.
Path 313 | total_timesteps 3215.
Path 314 | total_timesteps 3223.
Path 315 | total_timesteps 3233.
Path 316 | total_timesteps 3246.
Path 317 | total_timesteps 3256.
Path 318 | total_timesteps 3267.
Path 319 | total_timesteps 3275.
Path 320 | total_timesteps 3286.
Path 321 | total_timesteps 3294.
Path 322 | total_timesteps 3302.
Path 323 | total_timesteps 3314.
Path 324 | total_timesteps 3326.
Path 325 | total_timesteps 3341.
Path 326 | total_timesteps 3351.
Path 327 | total_timesteps 3361.
Path 328 | total_timesteps 3369.
Path 329 | total_timesteps 3381.
Path 330 | total_timesteps 3393.
Path 331 | total_timesteps 3401.
Path 332 | total_timesteps 3409.
Path 333 | total_timesteps 3418.
Path 334 | total_timesteps 3427.
Path 335 | total_timesteps 3438.
Path 336 | total_timesteps 3448.
Path 337 | total_timesteps 3459.
Path 338 | total_timesteps 3469.
Path 339 | total_timesteps 3482.
Path 340 | total_timesteps 3492.
Path 341 | total_timesteps 3501.
Path 342 | total_timesteps 3509.
Path 343 | total_timesteps 3519.
Path 344 | total_timesteps 3531.
Path 345 | total_timesteps 3541.
Path 346 | total_timesteps 3550.
Path 347 | total_timesteps 3558.
Path 348 | total_timesteps 3564.
Path 349 | total_timesteps 3573.
Path 350 | total_timesteps 3589.
Path 351 | total_timesteps 3601.
Path 352 | total_timesteps 3616.
Path 353 | total_timesteps 3623.
Path 354 | total_timesteps 3630.
Path 355 | total_timesteps 3638.
Path 356 | total_timesteps 3645.
Path 357 | total_timesteps 3653.
Path 358 | total_timesteps 3659.
Path 359 | total_timesteps 3667.
Path 360 | total_timesteps 3675.
Path 361 | total_timesteps 3685.
Path 362 | total_timesteps 3694.
Path 363 | total_timesteps 3701.
Path 364 | total_timesteps 3711.
Path 365 | total_timesteps 3722.
Path 366 | total_timesteps 3729.
Path 367 | total_timesteps 3738.
Path 368 | total_timesteps 3745.
Path 369 | total_timesteps 3755.
Path 370 | total_timesteps 3766.
Path 371 | total_timesteps 3774.
Path 372 | total_timesteps 3786.
Path 373 | total_timesteps 3802.
Path 374 | total_timesteps 3812.
Path 375 | total_timesteps 3819.
Path 376 | total_timesteps 3831.
Path 377 | total_timesteps 3845.
Path 378 | total_timesteps 3856.
Path 379 | total_timesteps 3864.
Path 380 | total_timesteps 3874.
Path 381 | total_timesteps 3884.
Path 382 | total_timesteps 3894.
Path 383 | total_timesteps 3903.
Path 384 | total_timesteps 3916.
Path 385 | total_timesteps 3923.
Path 386 | total_timesteps 3932.
Path 387 | total_timesteps 3942.
Path 388 | total_timesteps 3949.
Path 389 | total_timesteps 3956.
Path 390 | total_timesteps 3963.
Path 391 | total_timesteps 3972.
Path 392 | total_timesteps 3984.
Path 393 | total_timesteps 3992.
Path 394 | total_timesteps 3999.
Path 395 | total_timesteps 4022.
Path 396 | total_timesteps 4030.
Path 397 | total_timesteps 4039.
Path 398 | total_timesteps 4046.
Path 399 | total_timesteps 4054.
Path 400 | total_timesteps 4064.
Path 401 | total_timesteps 4072.
Path 402 | total_timesteps 4081.
Path 403 | total_timesteps 4095.
Path 404 | total_timesteps 4103.
Path 405 | total_timesteps 4110.
Path 406 | total_timesteps 4119.
Path 407 | total_timesteps 4127.
Path 408 | total_timesteps 4135.
Path 409 | total_timesteps 4142.
Path 410 | total_timesteps 4150.
Path 411 | total_timesteps 4159.
Path 412 | total_timesteps 4171.
Path 413 | total_timesteps 4179.
Path 414 | total_timesteps 4191.
Path 415 | total_timesteps 4200.
Path 416 | total_timesteps 4209.
Path 417 | total_timesteps 4216.
Path 418 | total_timesteps 4226.
Path 419 | total_timesteps 4235.
Path 420 | total_timesteps 4243.
Path 421 | total_timesteps 4259.
Path 422 | total_timesteps 4269.
Path 423 | total_timesteps 4279.
Path 424 | total_timesteps 4287.
Path 425 | total_timesteps 4296.
Path 426 | total_timesteps 4305.
Path 427 | total_timesteps 4315.
Path 428 | total_timesteps 4325.
Path 429 | total_timesteps 4332.
Path 430 | total_timesteps 4344.
Path 431 | total_timesteps 4359.
Path 432 | total_timesteps 4371.
Path 433 | total_timesteps 4380.
Path 434 | total_timesteps 4390.
Path 435 | total_timesteps 4398.
Path 436 | total_timesteps 4405.
Path 437 | total_timesteps 4417.
Path 438 | total_timesteps 4426.
Path 439 | total_timesteps 4436.
Path 440 | total_timesteps 4444.
Path 441 | total_timesteps 4453.
Path 442 | total_timesteps 4462.
Path 443 | total_timesteps 4473.
Path 444 | total_timesteps 4482.
Path 445 | total_timesteps 4506.
Path 446 | total_timesteps 4517.
Path 447 | total_timesteps 4529.
Path 448 | total_timesteps 4547.
Path 449 | total_timesteps 4557.
Path 450 | total_timesteps 4565.
Path 451 | total_timesteps 4573.
Path 452 | total_timesteps 4581.
Path 453 | total_timesteps 4592.
Path 454 | total_timesteps 4602.
Path 455 | total_timesteps 4609.
Path 456 | total_timesteps 4619.
Path 457 | total_timesteps 4633.
Path 458 | total_timesteps 4643.
Path 459 | total_timesteps 4656.
Path 460 | total_timesteps 4667.
Path 461 | total_timesteps 4678.
Path 462 | total_timesteps 4688.
Path 463 | total_timesteps 4702.
Path 464 | total_timesteps 4713.
Path 465 | total_timesteps 4720.
Path 466 | total_timesteps 4730.
Path 467 | total_timesteps 4737.
Path 468 | total_timesteps 4748.
Path 469 | total_timesteps 4760.
Path 470 | total_timesteps 4767.
Path 471 | total_timesteps 4776.
Path 472 | total_timesteps 4786.
Path 473 | total_timesteps 4798.
Path 474 | total_timesteps 4809.
Path 475 | total_timesteps 4821.
Path 476 | total_timesteps 4833.
Path 477 | total_timesteps 4844.
Path 478 | total_timesteps 4856.
Path 479 | total_timesteps 4871.
Path 480 | total_timesteps 4882.
Path 481 | total_timesteps 4892.
Path 482 | total_timesteps 4905.
Path 483 | total_timesteps 4914.
Path 484 | total_timesteps 4922.
Path 485 | total_timesteps 4939.
Path 486 | total_timesteps 4951.
Path 487 | total_timesteps 4959.
Path 488 | total_timesteps 4969.
Path 489 | total_timesteps 4978.
Path 490 | total_timesteps 4996.
Path 491 | total_timesteps 5005.
Path 492 | total_timesteps 5023.
Path 493 | total_timesteps 5030.
Path 494 | total_timesteps 5037.
Path 495 | total_timesteps 5046.
Path 496 | total_timesteps 5059.
Path 497 | total_timesteps 5069.
Path 498 | total_timesteps 5077.
Path 499 | total_timesteps 5086.
Path 500 | total_timesteps 5097.
Path 501 | total_timesteps 5107.
Path 502 | total_timesteps 5127.
Path 503 | total_timesteps 5138.
Path 504 | total_timesteps 5149.
Path 505 | total_timesteps 5156.
Path 506 | total_timesteps 5166.
Path 507 | total_timesteps 5173.
Path 508 | total_timesteps 5182.
Path 509 | total_timesteps 5193.
Path 510 | total_timesteps 5201.
Path 511 | total_timesteps 5208.
Path 512 | total_timesteps 5215.
Path 513 | total_timesteps 5223.
Path 514 | total_timesteps 5232.
Path 515 | total_timesteps 5240.
Path 516 | total_timesteps 5263.
Path 517 | total_timesteps 5273.
Path 518 | total_timesteps 5283.
Path 519 | total_timesteps 5297.
Path 520 | total_timesteps 5305.
Path 521 | total_timesteps 5312.
Path 522 | total_timesteps 5320.
Path 523 | total_timesteps 5331.
Path 524 | total_timesteps 5340.
Path 525 | total_timesteps 5350.
Path 526 | total_timesteps 5358.
Path 527 | total_timesteps 5367.
Path 528 | total_timesteps 5378.
Path 529 | total_timesteps 5389.
Path 530 | total_timesteps 5396.
Path 531 | total_timesteps 5408.
Path 532 | total_timesteps 5418.
Path 533 | total_timesteps 5433.
Path 534 | total_timesteps 5445.
Path 535 | total_timesteps 5456.
Path 536 | total_timesteps 5470.
Path 537 | total_timesteps 5482.
Path 538 | total_timesteps 5493.
Path 539 | total_timesteps 5500.
Path 540 | total_timesteps 5510.
Path 541 | total_timesteps 5518.
Path 542 | total_timesteps 5529.
Path 543 | total_timesteps 5545.
Path 544 | total_timesteps 5557.
Path 545 | total_timesteps 5569.
Path 546 | total_timesteps 5578.
Path 547 | total_timesteps 5587.
Path 548 | total_timesteps 5596.
Path 549 | total_timesteps 5607.
Path 550 | total_timesteps 5616.
Path 551 | total_timesteps 5628.
Path 552 | total_timesteps 5638.
Path 553 | total_timesteps 5648.
Path 554 | total_timesteps 5660.
Path 555 | total_timesteps 5669.
Path 556 | total_timesteps 5676.
Path 557 | total_timesteps 5685.
Path 558 | total_timesteps 5696.
Path 559 | total_timesteps 5705.
Path 560 | total_timesteps 5712.
Path 561 | total_timesteps 5723.
Path 562 | total_timesteps 5730.
Path 563 | total_timesteps 5739.
Path 564 | total_timesteps 5748.
Path 565 | total_timesteps 5758.
Path 566 | total_timesteps 5768.
Path 567 | total_timesteps 5778.
Path 568 | total_timesteps 5785.
Path 569 | total_timesteps 5796.
Path 570 | total_timesteps 5812.
Path 571 | total_timesteps 5821.
Path 572 | total_timesteps 5836.
Path 573 | total_timesteps 5848.
Path 574 | total_timesteps 5856.
Path 575 | total_timesteps 5872.
Path 576 | total_timesteps 5880.
Path 577 | total_timesteps 5889.
Path 578 | total_timesteps 5904.
Path 579 | total_timesteps 5915.
Path 580 | total_timesteps 5926.
Path 581 | total_timesteps 5936.
Path 582 | total_timesteps 5946.
Path 583 | total_timesteps 5957.
Path 584 | total_timesteps 5975.
Path 585 | total_timesteps 5985.
Path 586 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.7     |
| Iteration     | 7        |
| MaximumReturn | -1.01    |
| MinimumReturn | -17.9    |
| TotalSamples  | 36034    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.3002457618713379
Validation loss = 0.32241642475128174
Validation loss = 0.31143712997436523
Validation loss = 0.32048359513282776
Validation loss = 0.3201744854450226
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 35.
Path 5 | total_timesteps 49.
Path 6 | total_timesteps 58.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 93.
Path 10 | total_timesteps 105.
Path 11 | total_timesteps 113.
Path 12 | total_timesteps 121.
Path 13 | total_timesteps 128.
Path 14 | total_timesteps 143.
Path 15 | total_timesteps 153.
Path 16 | total_timesteps 160.
Path 17 | total_timesteps 168.
Path 18 | total_timesteps 179.
Path 19 | total_timesteps 192.
Path 20 | total_timesteps 202.
Path 21 | total_timesteps 209.
Path 22 | total_timesteps 218.
Path 23 | total_timesteps 226.
Path 24 | total_timesteps 238.
Path 25 | total_timesteps 246.
Path 26 | total_timesteps 255.
Path 27 | total_timesteps 266.
Path 28 | total_timesteps 274.
Path 29 | total_timesteps 282.
Path 30 | total_timesteps 297.
Path 31 | total_timesteps 305.
Path 32 | total_timesteps 314.
Path 33 | total_timesteps 323.
Path 34 | total_timesteps 332.
Path 35 | total_timesteps 340.
Path 36 | total_timesteps 350.
Path 37 | total_timesteps 358.
Path 38 | total_timesteps 366.
Path 39 | total_timesteps 375.
Path 40 | total_timesteps 387.
Path 41 | total_timesteps 394.
Path 42 | total_timesteps 409.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 438.
Path 45 | total_timesteps 446.
Path 46 | total_timesteps 456.
Path 47 | total_timesteps 466.
Path 48 | total_timesteps 475.
Path 49 | total_timesteps 485.
Path 50 | total_timesteps 494.
Path 51 | total_timesteps 505.
Path 52 | total_timesteps 513.
Path 53 | total_timesteps 523.
Path 54 | total_timesteps 533.
Path 55 | total_timesteps 545.
Path 56 | total_timesteps 559.
Path 57 | total_timesteps 569.
Path 58 | total_timesteps 578.
Path 59 | total_timesteps 588.
Path 60 | total_timesteps 596.
Path 61 | total_timesteps 603.
Path 62 | total_timesteps 615.
Path 63 | total_timesteps 624.
Path 64 | total_timesteps 633.
Path 65 | total_timesteps 646.
Path 66 | total_timesteps 657.
Path 67 | total_timesteps 669.
Path 68 | total_timesteps 679.
Path 69 | total_timesteps 687.
Path 70 | total_timesteps 698.
Path 71 | total_timesteps 711.
Path 72 | total_timesteps 721.
Path 73 | total_timesteps 734.
Path 74 | total_timesteps 743.
Path 75 | total_timesteps 752.
Path 76 | total_timesteps 764.
Path 77 | total_timesteps 774.
Path 78 | total_timesteps 781.
Path 79 | total_timesteps 793.
Path 80 | total_timesteps 800.
Path 81 | total_timesteps 807.
Path 82 | total_timesteps 815.
Path 83 | total_timesteps 825.
Path 84 | total_timesteps 837.
Path 85 | total_timesteps 844.
Path 86 | total_timesteps 856.
Path 87 | total_timesteps 864.
Path 88 | total_timesteps 877.
Path 89 | total_timesteps 887.
Path 90 | total_timesteps 899.
Path 91 | total_timesteps 909.
Path 92 | total_timesteps 922.
Path 93 | total_timesteps 934.
Path 94 | total_timesteps 944.
Path 95 | total_timesteps 954.
Path 96 | total_timesteps 965.
Path 97 | total_timesteps 978.
Path 98 | total_timesteps 990.
Path 99 | total_timesteps 998.
Path 100 | total_timesteps 1005.
Path 101 | total_timesteps 1018.
Path 102 | total_timesteps 1031.
Path 103 | total_timesteps 1040.
Path 104 | total_timesteps 1049.
Path 105 | total_timesteps 1058.
Path 106 | total_timesteps 1064.
Path 107 | total_timesteps 1076.
Path 108 | total_timesteps 1082.
Path 109 | total_timesteps 1092.
Path 110 | total_timesteps 1102.
Path 111 | total_timesteps 1118.
Path 112 | total_timesteps 1129.
Path 113 | total_timesteps 1141.
Path 114 | total_timesteps 1151.
Path 115 | total_timesteps 1161.
Path 116 | total_timesteps 1170.
Path 117 | total_timesteps 1181.
Path 118 | total_timesteps 1192.
Path 119 | total_timesteps 1202.
Path 120 | total_timesteps 1216.
Path 121 | total_timesteps 1225.
Path 122 | total_timesteps 1243.
Path 123 | total_timesteps 1253.
Path 124 | total_timesteps 1264.
Path 125 | total_timesteps 1272.
Path 126 | total_timesteps 1283.
Path 127 | total_timesteps 1292.
Path 128 | total_timesteps 1300.
Path 129 | total_timesteps 1308.
Path 130 | total_timesteps 1322.
Path 131 | total_timesteps 1343.
Path 132 | total_timesteps 1351.
Path 133 | total_timesteps 1358.
Path 134 | total_timesteps 1367.
Path 135 | total_timesteps 1381.
Path 136 | total_timesteps 1392.
Path 137 | total_timesteps 1401.
Path 138 | total_timesteps 1413.
Path 139 | total_timesteps 1421.
Path 140 | total_timesteps 1433.
Path 141 | total_timesteps 1446.
Path 142 | total_timesteps 1452.
Path 143 | total_timesteps 1466.
Path 144 | total_timesteps 1474.
Path 145 | total_timesteps 1482.
Path 146 | total_timesteps 1492.
Path 147 | total_timesteps 1499.
Path 148 | total_timesteps 1508.
Path 149 | total_timesteps 1526.
Path 150 | total_timesteps 1533.
Path 151 | total_timesteps 1547.
Path 152 | total_timesteps 1558.
Path 153 | total_timesteps 1572.
Path 154 | total_timesteps 1587.
Path 155 | total_timesteps 1599.
Path 156 | total_timesteps 1609.
Path 157 | total_timesteps 1618.
Path 158 | total_timesteps 1625.
Path 159 | total_timesteps 1635.
Path 160 | total_timesteps 1643.
Path 161 | total_timesteps 1653.
Path 162 | total_timesteps 1660.
Path 163 | total_timesteps 1667.
Path 164 | total_timesteps 1676.
Path 165 | total_timesteps 1682.
Path 166 | total_timesteps 1690.
Path 167 | total_timesteps 1705.
Path 168 | total_timesteps 1713.
Path 169 | total_timesteps 1723.
Path 170 | total_timesteps 1733.
Path 171 | total_timesteps 1739.
Path 172 | total_timesteps 1746.
Path 173 | total_timesteps 1756.
Path 174 | total_timesteps 1764.
Path 175 | total_timesteps 1774.
Path 176 | total_timesteps 1781.
Path 177 | total_timesteps 1788.
Path 178 | total_timesteps 1795.
Path 179 | total_timesteps 1806.
Path 180 | total_timesteps 1813.
Path 181 | total_timesteps 1823.
Path 182 | total_timesteps 1830.
Path 183 | total_timesteps 1843.
Path 184 | total_timesteps 1856.
Path 185 | total_timesteps 1874.
Path 186 | total_timesteps 1886.
Path 187 | total_timesteps 1898.
Path 188 | total_timesteps 1920.
Path 189 | total_timesteps 1927.
Path 190 | total_timesteps 1934.
Path 191 | total_timesteps 1943.
Path 192 | total_timesteps 1951.
Path 193 | total_timesteps 1963.
Path 194 | total_timesteps 1977.
Path 195 | total_timesteps 1986.
Path 196 | total_timesteps 1993.
Path 197 | total_timesteps 2002.
Path 198 | total_timesteps 2010.
Path 199 | total_timesteps 2028.
Path 200 | total_timesteps 2036.
Path 201 | total_timesteps 2043.
Path 202 | total_timesteps 2052.
Path 203 | total_timesteps 2059.
Path 204 | total_timesteps 2069.
Path 205 | total_timesteps 2077.
Path 206 | total_timesteps 2092.
Path 207 | total_timesteps 2099.
Path 208 | total_timesteps 2108.
Path 209 | total_timesteps 2123.
Path 210 | total_timesteps 2131.
Path 211 | total_timesteps 2139.
Path 212 | total_timesteps 2147.
Path 213 | total_timesteps 2154.
Path 214 | total_timesteps 2163.
Path 215 | total_timesteps 2171.
Path 216 | total_timesteps 2179.
Path 217 | total_timesteps 2192.
Path 218 | total_timesteps 2201.
Path 219 | total_timesteps 2209.
Path 220 | total_timesteps 2217.
Path 221 | total_timesteps 2229.
Path 222 | total_timesteps 2238.
Path 223 | total_timesteps 2249.
Path 224 | total_timesteps 2259.
Path 225 | total_timesteps 2273.
Path 226 | total_timesteps 2281.
Path 227 | total_timesteps 2291.
Path 228 | total_timesteps 2299.
Path 229 | total_timesteps 2311.
Path 230 | total_timesteps 2319.
Path 231 | total_timesteps 2325.
Path 232 | total_timesteps 2332.
Path 233 | total_timesteps 2345.
Path 234 | total_timesteps 2357.
Path 235 | total_timesteps 2368.
Path 236 | total_timesteps 2375.
Path 237 | total_timesteps 2384.
Path 238 | total_timesteps 2403.
Path 239 | total_timesteps 2413.
Path 240 | total_timesteps 2426.
Path 241 | total_timesteps 2435.
Path 242 | total_timesteps 2442.
Path 243 | total_timesteps 2452.
Path 244 | total_timesteps 2463.
Path 245 | total_timesteps 2476.
Path 246 | total_timesteps 2485.
Path 247 | total_timesteps 2494.
Path 248 | total_timesteps 2502.
Path 249 | total_timesteps 2512.
Path 250 | total_timesteps 2521.
Path 251 | total_timesteps 2533.
Path 252 | total_timesteps 2546.
Path 253 | total_timesteps 2556.
Path 254 | total_timesteps 2564.
Path 255 | total_timesteps 2574.
Path 256 | total_timesteps 2582.
Path 257 | total_timesteps 2589.
Path 258 | total_timesteps 2597.
Path 259 | total_timesteps 2605.
Path 260 | total_timesteps 2614.
Path 261 | total_timesteps 2626.
Path 262 | total_timesteps 2636.
Path 263 | total_timesteps 2644.
Path 264 | total_timesteps 2656.
Path 265 | total_timesteps 2668.
Path 266 | total_timesteps 2679.
Path 267 | total_timesteps 2691.
Path 268 | total_timesteps 2701.
Path 269 | total_timesteps 2711.
Path 270 | total_timesteps 2725.
Path 271 | total_timesteps 2733.
Path 272 | total_timesteps 2741.
Path 273 | total_timesteps 2749.
Path 274 | total_timesteps 2763.
Path 275 | total_timesteps 2774.
Path 276 | total_timesteps 2788.
Path 277 | total_timesteps 2797.
Path 278 | total_timesteps 2808.
Path 279 | total_timesteps 2817.
Path 280 | total_timesteps 2826.
Path 281 | total_timesteps 2843.
Path 282 | total_timesteps 2851.
Path 283 | total_timesteps 2859.
Path 284 | total_timesteps 2867.
Path 285 | total_timesteps 2878.
Path 286 | total_timesteps 2889.
Path 287 | total_timesteps 2900.
Path 288 | total_timesteps 2913.
Path 289 | total_timesteps 2923.
Path 290 | total_timesteps 2932.
Path 291 | total_timesteps 2941.
Path 292 | total_timesteps 2953.
Path 293 | total_timesteps 2964.
Path 294 | total_timesteps 2976.
Path 295 | total_timesteps 2992.
Path 296 | total_timesteps 3002.
Path 297 | total_timesteps 3015.
Path 298 | total_timesteps 3025.
Path 299 | total_timesteps 3034.
Path 300 | total_timesteps 3043.
Path 301 | total_timesteps 3051.
Path 302 | total_timesteps 3067.
Path 303 | total_timesteps 3075.
Path 304 | total_timesteps 3084.
Path 305 | total_timesteps 3094.
Path 306 | total_timesteps 3103.
Path 307 | total_timesteps 3115.
Path 308 | total_timesteps 3128.
Path 309 | total_timesteps 3137.
Path 310 | total_timesteps 3145.
Path 311 | total_timesteps 3152.
Path 312 | total_timesteps 3167.
Path 313 | total_timesteps 3180.
Path 314 | total_timesteps 3189.
Path 315 | total_timesteps 3198.
Path 316 | total_timesteps 3210.
Path 317 | total_timesteps 3220.
Path 318 | total_timesteps 3231.
Path 319 | total_timesteps 3252.
Path 320 | total_timesteps 3262.
Path 321 | total_timesteps 3274.
Path 322 | total_timesteps 3281.
Path 323 | total_timesteps 3288.
Path 324 | total_timesteps 3297.
Path 325 | total_timesteps 3307.
Path 326 | total_timesteps 3315.
Path 327 | total_timesteps 3326.
Path 328 | total_timesteps 3336.
Path 329 | total_timesteps 3345.
Path 330 | total_timesteps 3364.
Path 331 | total_timesteps 3372.
Path 332 | total_timesteps 3381.
Path 333 | total_timesteps 3389.
Path 334 | total_timesteps 3397.
Path 335 | total_timesteps 3404.
Path 336 | total_timesteps 3417.
Path 337 | total_timesteps 3428.
Path 338 | total_timesteps 3438.
Path 339 | total_timesteps 3444.
Path 340 | total_timesteps 3452.
Path 341 | total_timesteps 3462.
Path 342 | total_timesteps 3475.
Path 343 | total_timesteps 3484.
Path 344 | total_timesteps 3495.
Path 345 | total_timesteps 3507.
Path 346 | total_timesteps 3517.
Path 347 | total_timesteps 3528.
Path 348 | total_timesteps 3539.
Path 349 | total_timesteps 3545.
Path 350 | total_timesteps 3555.
Path 351 | total_timesteps 3571.
Path 352 | total_timesteps 3579.
Path 353 | total_timesteps 3587.
Path 354 | total_timesteps 3596.
Path 355 | total_timesteps 3607.
Path 356 | total_timesteps 3619.
Path 357 | total_timesteps 3626.
Path 358 | total_timesteps 3633.
Path 359 | total_timesteps 3642.
Path 360 | total_timesteps 3648.
Path 361 | total_timesteps 3660.
Path 362 | total_timesteps 3673.
Path 363 | total_timesteps 3681.
Path 364 | total_timesteps 3687.
Path 365 | total_timesteps 3696.
Path 366 | total_timesteps 3709.
Path 367 | total_timesteps 3723.
Path 368 | total_timesteps 3735.
Path 369 | total_timesteps 3742.
Path 370 | total_timesteps 3748.
Path 371 | total_timesteps 3759.
Path 372 | total_timesteps 3772.
Path 373 | total_timesteps 3786.
Path 374 | total_timesteps 3800.
Path 375 | total_timesteps 3813.
Path 376 | total_timesteps 3835.
Path 377 | total_timesteps 3852.
Path 378 | total_timesteps 3860.
Path 379 | total_timesteps 3867.
Path 380 | total_timesteps 3874.
Path 381 | total_timesteps 3886.
Path 382 | total_timesteps 3897.
Path 383 | total_timesteps 3905.
Path 384 | total_timesteps 3915.
Path 385 | total_timesteps 3925.
Path 386 | total_timesteps 3933.
Path 387 | total_timesteps 3941.
Path 388 | total_timesteps 3953.
Path 389 | total_timesteps 3960.
Path 390 | total_timesteps 3967.
Path 391 | total_timesteps 3978.
Path 392 | total_timesteps 3987.
Path 393 | total_timesteps 3995.
Path 394 | total_timesteps 4005.
Path 395 | total_timesteps 4014.
Path 396 | total_timesteps 4021.
Path 397 | total_timesteps 4032.
Path 398 | total_timesteps 4040.
Path 399 | total_timesteps 4050.
Path 400 | total_timesteps 4059.
Path 401 | total_timesteps 4067.
Path 402 | total_timesteps 4076.
Path 403 | total_timesteps 4083.
Path 404 | total_timesteps 4094.
Path 405 | total_timesteps 4106.
Path 406 | total_timesteps 4122.
Path 407 | total_timesteps 4135.
Path 408 | total_timesteps 4142.
Path 409 | total_timesteps 4150.
Path 410 | total_timesteps 4161.
Path 411 | total_timesteps 4174.
Path 412 | total_timesteps 4182.
Path 413 | total_timesteps 4198.
Path 414 | total_timesteps 4208.
Path 415 | total_timesteps 4216.
Path 416 | total_timesteps 4232.
Path 417 | total_timesteps 4240.
Path 418 | total_timesteps 4248.
Path 419 | total_timesteps 4256.
Path 420 | total_timesteps 4263.
Path 421 | total_timesteps 4272.
Path 422 | total_timesteps 4278.
Path 423 | total_timesteps 4292.
Path 424 | total_timesteps 4302.
Path 425 | total_timesteps 4315.
Path 426 | total_timesteps 4324.
Path 427 | total_timesteps 4335.
Path 428 | total_timesteps 4343.
Path 429 | total_timesteps 4353.
Path 430 | total_timesteps 4366.
Path 431 | total_timesteps 4377.
Path 432 | total_timesteps 4386.
Path 433 | total_timesteps 4394.
Path 434 | total_timesteps 4406.
Path 435 | total_timesteps 4414.
Path 436 | total_timesteps 4426.
Path 437 | total_timesteps 4434.
Path 438 | total_timesteps 4447.
Path 439 | total_timesteps 4454.
Path 440 | total_timesteps 4467.
Path 441 | total_timesteps 4476.
Path 442 | total_timesteps 4483.
Path 443 | total_timesteps 4490.
Path 444 | total_timesteps 4499.
Path 445 | total_timesteps 4507.
Path 446 | total_timesteps 4515.
Path 447 | total_timesteps 4526.
Path 448 | total_timesteps 4537.
Path 449 | total_timesteps 4551.
Path 450 | total_timesteps 4561.
Path 451 | total_timesteps 4571.
Path 452 | total_timesteps 4585.
Path 453 | total_timesteps 4593.
Path 454 | total_timesteps 4605.
Path 455 | total_timesteps 4620.
Path 456 | total_timesteps 4626.
Path 457 | total_timesteps 4636.
Path 458 | total_timesteps 4644.
Path 459 | total_timesteps 4658.
Path 460 | total_timesteps 4670.
Path 461 | total_timesteps 4680.
Path 462 | total_timesteps 4693.
Path 463 | total_timesteps 4702.
Path 464 | total_timesteps 4715.
Path 465 | total_timesteps 4722.
Path 466 | total_timesteps 4731.
Path 467 | total_timesteps 4739.
Path 468 | total_timesteps 4746.
Path 469 | total_timesteps 4755.
Path 470 | total_timesteps 4762.
Path 471 | total_timesteps 4770.
Path 472 | total_timesteps 4777.
Path 473 | total_timesteps 4790.
Path 474 | total_timesteps 4801.
Path 475 | total_timesteps 4814.
Path 476 | total_timesteps 4822.
Path 477 | total_timesteps 4832.
Path 478 | total_timesteps 4847.
Path 479 | total_timesteps 4857.
Path 480 | total_timesteps 4866.
Path 481 | total_timesteps 4875.
Path 482 | total_timesteps 4887.
Path 483 | total_timesteps 4894.
Path 484 | total_timesteps 4906.
Path 485 | total_timesteps 4914.
Path 486 | total_timesteps 4925.
Path 487 | total_timesteps 4938.
Path 488 | total_timesteps 4947.
Path 489 | total_timesteps 4965.
Path 490 | total_timesteps 4975.
Path 491 | total_timesteps 4985.
Path 492 | total_timesteps 4992.
Path 493 | total_timesteps 4998.
Path 494 | total_timesteps 5012.
Path 495 | total_timesteps 5022.
Path 496 | total_timesteps 5039.
Path 497 | total_timesteps 5045.
Path 498 | total_timesteps 5056.
Path 499 | total_timesteps 5065.
Path 500 | total_timesteps 5073.
Path 501 | total_timesteps 5086.
Path 502 | total_timesteps 5094.
Path 503 | total_timesteps 5105.
Path 504 | total_timesteps 5112.
Path 505 | total_timesteps 5125.
Path 506 | total_timesteps 5146.
Path 507 | total_timesteps 5155.
Path 508 | total_timesteps 5165.
Path 509 | total_timesteps 5175.
Path 510 | total_timesteps 5182.
Path 511 | total_timesteps 5192.
Path 512 | total_timesteps 5204.
Path 513 | total_timesteps 5214.
Path 514 | total_timesteps 5223.
Path 515 | total_timesteps 5232.
Path 516 | total_timesteps 5239.
Path 517 | total_timesteps 5247.
Path 518 | total_timesteps 5258.
Path 519 | total_timesteps 5266.
Path 520 | total_timesteps 5280.
Path 521 | total_timesteps 5289.
Path 522 | total_timesteps 5296.
Path 523 | total_timesteps 5305.
Path 524 | total_timesteps 5314.
Path 525 | total_timesteps 5322.
Path 526 | total_timesteps 5329.
Path 527 | total_timesteps 5336.
Path 528 | total_timesteps 5347.
Path 529 | total_timesteps 5355.
Path 530 | total_timesteps 5365.
Path 531 | total_timesteps 5376.
Path 532 | total_timesteps 5385.
Path 533 | total_timesteps 5395.
Path 534 | total_timesteps 5410.
Path 535 | total_timesteps 5419.
Path 536 | total_timesteps 5427.
Path 537 | total_timesteps 5434.
Path 538 | total_timesteps 5449.
Path 539 | total_timesteps 5461.
Path 540 | total_timesteps 5471.
Path 541 | total_timesteps 5484.
Path 542 | total_timesteps 5493.
Path 543 | total_timesteps 5502.
Path 544 | total_timesteps 5510.
Path 545 | total_timesteps 5521.
Path 546 | total_timesteps 5529.
Path 547 | total_timesteps 5537.
Path 548 | total_timesteps 5547.
Path 549 | total_timesteps 5555.
Path 550 | total_timesteps 5567.
Path 551 | total_timesteps 5575.
Path 552 | total_timesteps 5586.
Path 553 | total_timesteps 5597.
Path 554 | total_timesteps 5609.
Path 555 | total_timesteps 5618.
Path 556 | total_timesteps 5627.
Path 557 | total_timesteps 5639.
Path 558 | total_timesteps 5651.
Path 559 | total_timesteps 5669.
Path 560 | total_timesteps 5692.
Path 561 | total_timesteps 5704.
Path 562 | total_timesteps 5715.
Path 563 | total_timesteps 5721.
Path 564 | total_timesteps 5729.
Path 565 | total_timesteps 5737.
Path 566 | total_timesteps 5744.
Path 567 | total_timesteps 5756.
Path 568 | total_timesteps 5770.
Path 569 | total_timesteps 5782.
Path 570 | total_timesteps 5791.
Path 571 | total_timesteps 5799.
Path 572 | total_timesteps 5806.
Path 573 | total_timesteps 5816.
Path 574 | total_timesteps 5824.
Path 575 | total_timesteps 5834.
Path 576 | total_timesteps 5841.
Path 577 | total_timesteps 5850.
Path 578 | total_timesteps 5866.
Path 579 | total_timesteps 5877.
Path 580 | total_timesteps 5886.
Path 581 | total_timesteps 5896.
Path 582 | total_timesteps 5906.
Path 583 | total_timesteps 5913.
Path 584 | total_timesteps 5924.
Path 585 | total_timesteps 5933.
Path 586 | total_timesteps 5946.
Path 587 | total_timesteps 5954.
Path 588 | total_timesteps 5970.
Path 589 | total_timesteps 5985.
Path 590 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.69    |
| Iteration     | 8        |
| MaximumReturn | 0.565    |
| MinimumReturn | -17.7    |
| TotalSamples  | 40036    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.3023492693901062
Validation loss = 0.3123077154159546
Validation loss = 0.30885663628578186
Validation loss = 0.31147903203964233
Validation loss = 0.3161734640598297
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 19.
Path 3 | total_timesteps 27.
Path 4 | total_timesteps 38.
Path 5 | total_timesteps 50.
Path 6 | total_timesteps 59.
Path 7 | total_timesteps 67.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 86.
Path 10 | total_timesteps 95.
Path 11 | total_timesteps 108.
Path 12 | total_timesteps 119.
Path 13 | total_timesteps 126.
Path 14 | total_timesteps 138.
Path 15 | total_timesteps 147.
Path 16 | total_timesteps 162.
Path 17 | total_timesteps 174.
Path 18 | total_timesteps 186.
Path 19 | total_timesteps 196.
Path 20 | total_timesteps 203.
Path 21 | total_timesteps 211.
Path 22 | total_timesteps 221.
Path 23 | total_timesteps 230.
Path 24 | total_timesteps 243.
Path 25 | total_timesteps 256.
Path 26 | total_timesteps 264.
Path 27 | total_timesteps 274.
Path 28 | total_timesteps 292.
Path 29 | total_timesteps 303.
Path 30 | total_timesteps 314.
Path 31 | total_timesteps 326.
Path 32 | total_timesteps 336.
Path 33 | total_timesteps 349.
Path 34 | total_timesteps 355.
Path 35 | total_timesteps 366.
Path 36 | total_timesteps 375.
Path 37 | total_timesteps 387.
Path 38 | total_timesteps 395.
Path 39 | total_timesteps 403.
Path 40 | total_timesteps 410.
Path 41 | total_timesteps 419.
Path 42 | total_timesteps 426.
Path 43 | total_timesteps 441.
Path 44 | total_timesteps 451.
Path 45 | total_timesteps 462.
Path 46 | total_timesteps 474.
Path 47 | total_timesteps 484.
Path 48 | total_timesteps 491.
Path 49 | total_timesteps 500.
Path 50 | total_timesteps 508.
Path 51 | total_timesteps 523.
Path 52 | total_timesteps 531.
Path 53 | total_timesteps 540.
Path 54 | total_timesteps 551.
Path 55 | total_timesteps 561.
Path 56 | total_timesteps 571.
Path 57 | total_timesteps 578.
Path 58 | total_timesteps 587.
Path 59 | total_timesteps 595.
Path 60 | total_timesteps 604.
Path 61 | total_timesteps 614.
Path 62 | total_timesteps 625.
Path 63 | total_timesteps 638.
Path 64 | total_timesteps 651.
Path 65 | total_timesteps 668.
Path 66 | total_timesteps 675.
Path 67 | total_timesteps 683.
Path 68 | total_timesteps 696.
Path 69 | total_timesteps 704.
Path 70 | total_timesteps 712.
Path 71 | total_timesteps 729.
Path 72 | total_timesteps 741.
Path 73 | total_timesteps 748.
Path 74 | total_timesteps 760.
Path 75 | total_timesteps 770.
Path 76 | total_timesteps 778.
Path 77 | total_timesteps 789.
Path 78 | total_timesteps 799.
Path 79 | total_timesteps 808.
Path 80 | total_timesteps 819.
Path 81 | total_timesteps 828.
Path 82 | total_timesteps 838.
Path 83 | total_timesteps 849.
Path 84 | total_timesteps 855.
Path 85 | total_timesteps 862.
Path 86 | total_timesteps 870.
Path 87 | total_timesteps 881.
Path 88 | total_timesteps 891.
Path 89 | total_timesteps 899.
Path 90 | total_timesteps 906.
Path 91 | total_timesteps 916.
Path 92 | total_timesteps 924.
Path 93 | total_timesteps 932.
Path 94 | total_timesteps 940.
Path 95 | total_timesteps 953.
Path 96 | total_timesteps 960.
Path 97 | total_timesteps 969.
Path 98 | total_timesteps 981.
Path 99 | total_timesteps 993.
Path 100 | total_timesteps 1006.
Path 101 | total_timesteps 1018.
Path 102 | total_timesteps 1029.
Path 103 | total_timesteps 1040.
Path 104 | total_timesteps 1048.
Path 105 | total_timesteps 1061.
Path 106 | total_timesteps 1070.
Path 107 | total_timesteps 1078.
Path 108 | total_timesteps 1086.
Path 109 | total_timesteps 1092.
Path 110 | total_timesteps 1102.
Path 111 | total_timesteps 1109.
Path 112 | total_timesteps 1117.
Path 113 | total_timesteps 1126.
Path 114 | total_timesteps 1135.
Path 115 | total_timesteps 1144.
Path 116 | total_timesteps 1152.
Path 117 | total_timesteps 1163.
Path 118 | total_timesteps 1171.
Path 119 | total_timesteps 1180.
Path 120 | total_timesteps 1189.
Path 121 | total_timesteps 1199.
Path 122 | total_timesteps 1213.
Path 123 | total_timesteps 1225.
Path 124 | total_timesteps 1234.
Path 125 | total_timesteps 1245.
Path 126 | total_timesteps 1262.
Path 127 | total_timesteps 1273.
Path 128 | total_timesteps 1280.
Path 129 | total_timesteps 1291.
Path 130 | total_timesteps 1300.
Path 131 | total_timesteps 1310.
Path 132 | total_timesteps 1322.
Path 133 | total_timesteps 1330.
Path 134 | total_timesteps 1339.
Path 135 | total_timesteps 1351.
Path 136 | total_timesteps 1363.
Path 137 | total_timesteps 1374.
Path 138 | total_timesteps 1382.
Path 139 | total_timesteps 1391.
Path 140 | total_timesteps 1398.
Path 141 | total_timesteps 1406.
Path 142 | total_timesteps 1420.
Path 143 | total_timesteps 1430.
Path 144 | total_timesteps 1437.
Path 145 | total_timesteps 1447.
Path 146 | total_timesteps 1455.
Path 147 | total_timesteps 1463.
Path 148 | total_timesteps 1476.
Path 149 | total_timesteps 1488.
Path 150 | total_timesteps 1498.
Path 151 | total_timesteps 1509.
Path 152 | total_timesteps 1523.
Path 153 | total_timesteps 1534.
Path 154 | total_timesteps 1549.
Path 155 | total_timesteps 1558.
Path 156 | total_timesteps 1568.
Path 157 | total_timesteps 1576.
Path 158 | total_timesteps 1585.
Path 159 | total_timesteps 1597.
Path 160 | total_timesteps 1610.
Path 161 | total_timesteps 1618.
Path 162 | total_timesteps 1628.
Path 163 | total_timesteps 1643.
Path 164 | total_timesteps 1651.
Path 165 | total_timesteps 1658.
Path 166 | total_timesteps 1666.
Path 167 | total_timesteps 1674.
Path 168 | total_timesteps 1683.
Path 169 | total_timesteps 1691.
Path 170 | total_timesteps 1701.
Path 171 | total_timesteps 1711.
Path 172 | total_timesteps 1719.
Path 173 | total_timesteps 1733.
Path 174 | total_timesteps 1740.
Path 175 | total_timesteps 1748.
Path 176 | total_timesteps 1758.
Path 177 | total_timesteps 1766.
Path 178 | total_timesteps 1778.
Path 179 | total_timesteps 1789.
Path 180 | total_timesteps 1801.
Path 181 | total_timesteps 1811.
Path 182 | total_timesteps 1818.
Path 183 | total_timesteps 1827.
Path 184 | total_timesteps 1842.
Path 185 | total_timesteps 1856.
Path 186 | total_timesteps 1866.
Path 187 | total_timesteps 1877.
Path 188 | total_timesteps 1887.
Path 189 | total_timesteps 1901.
Path 190 | total_timesteps 1916.
Path 191 | total_timesteps 1926.
Path 192 | total_timesteps 1937.
Path 193 | total_timesteps 1946.
Path 194 | total_timesteps 1955.
Path 195 | total_timesteps 1963.
Path 196 | total_timesteps 1970.
Path 197 | total_timesteps 1981.
Path 198 | total_timesteps 1992.
Path 199 | total_timesteps 2005.
Path 200 | total_timesteps 2014.
Path 201 | total_timesteps 2022.
Path 202 | total_timesteps 2031.
Path 203 | total_timesteps 2044.
Path 204 | total_timesteps 2054.
Path 205 | total_timesteps 2062.
Path 206 | total_timesteps 2078.
Path 207 | total_timesteps 2092.
Path 208 | total_timesteps 2099.
Path 209 | total_timesteps 2108.
Path 210 | total_timesteps 2118.
Path 211 | total_timesteps 2129.
Path 212 | total_timesteps 2138.
Path 213 | total_timesteps 2149.
Path 214 | total_timesteps 2157.
Path 215 | total_timesteps 2166.
Path 216 | total_timesteps 2174.
Path 217 | total_timesteps 2181.
Path 218 | total_timesteps 2191.
Path 219 | total_timesteps 2199.
Path 220 | total_timesteps 2209.
Path 221 | total_timesteps 2218.
Path 222 | total_timesteps 2228.
Path 223 | total_timesteps 2235.
Path 224 | total_timesteps 2247.
Path 225 | total_timesteps 2256.
Path 226 | total_timesteps 2267.
Path 227 | total_timesteps 2279.
Path 228 | total_timesteps 2298.
Path 229 | total_timesteps 2306.
Path 230 | total_timesteps 2321.
Path 231 | total_timesteps 2329.
Path 232 | total_timesteps 2339.
Path 233 | total_timesteps 2348.
Path 234 | total_timesteps 2363.
Path 235 | total_timesteps 2373.
Path 236 | total_timesteps 2385.
Path 237 | total_timesteps 2392.
Path 238 | total_timesteps 2410.
Path 239 | total_timesteps 2419.
Path 240 | total_timesteps 2427.
Path 241 | total_timesteps 2435.
Path 242 | total_timesteps 2442.
Path 243 | total_timesteps 2456.
Path 244 | total_timesteps 2468.
Path 245 | total_timesteps 2476.
Path 246 | total_timesteps 2485.
Path 247 | total_timesteps 2493.
Path 248 | total_timesteps 2505.
Path 249 | total_timesteps 2513.
Path 250 | total_timesteps 2521.
Path 251 | total_timesteps 2531.
Path 252 | total_timesteps 2545.
Path 253 | total_timesteps 2554.
Path 254 | total_timesteps 2561.
Path 255 | total_timesteps 2568.
Path 256 | total_timesteps 2583.
Path 257 | total_timesteps 2595.
Path 258 | total_timesteps 2604.
Path 259 | total_timesteps 2611.
Path 260 | total_timesteps 2622.
Path 261 | total_timesteps 2632.
Path 262 | total_timesteps 2642.
Path 263 | total_timesteps 2654.
Path 264 | total_timesteps 2666.
Path 265 | total_timesteps 2673.
Path 266 | total_timesteps 2686.
Path 267 | total_timesteps 2693.
Path 268 | total_timesteps 2702.
Path 269 | total_timesteps 2718.
Path 270 | total_timesteps 2727.
Path 271 | total_timesteps 2735.
Path 272 | total_timesteps 2747.
Path 273 | total_timesteps 2757.
Path 274 | total_timesteps 2764.
Path 275 | total_timesteps 2773.
Path 276 | total_timesteps 2785.
Path 277 | total_timesteps 2794.
Path 278 | total_timesteps 2804.
Path 279 | total_timesteps 2820.
Path 280 | total_timesteps 2826.
Path 281 | total_timesteps 2838.
Path 282 | total_timesteps 2848.
Path 283 | total_timesteps 2855.
Path 284 | total_timesteps 2866.
Path 285 | total_timesteps 2878.
Path 286 | total_timesteps 2894.
Path 287 | total_timesteps 2904.
Path 288 | total_timesteps 2911.
Path 289 | total_timesteps 2921.
Path 290 | total_timesteps 2935.
Path 291 | total_timesteps 2947.
Path 292 | total_timesteps 2959.
Path 293 | total_timesteps 2969.
Path 294 | total_timesteps 2978.
Path 295 | total_timesteps 2987.
Path 296 | total_timesteps 3001.
Path 297 | total_timesteps 3008.
Path 298 | total_timesteps 3023.
Path 299 | total_timesteps 3033.
Path 300 | total_timesteps 3046.
Path 301 | total_timesteps 3056.
Path 302 | total_timesteps 3063.
Path 303 | total_timesteps 3074.
Path 304 | total_timesteps 3085.
Path 305 | total_timesteps 3096.
Path 306 | total_timesteps 3105.
Path 307 | total_timesteps 3120.
Path 308 | total_timesteps 3130.
Path 309 | total_timesteps 3138.
Path 310 | total_timesteps 3148.
Path 311 | total_timesteps 3171.
Path 312 | total_timesteps 3178.
Path 313 | total_timesteps 3186.
Path 314 | total_timesteps 3196.
Path 315 | total_timesteps 3216.
Path 316 | total_timesteps 3224.
Path 317 | total_timesteps 3234.
Path 318 | total_timesteps 3248.
Path 319 | total_timesteps 3255.
Path 320 | total_timesteps 3269.
Path 321 | total_timesteps 3277.
Path 322 | total_timesteps 3289.
Path 323 | total_timesteps 3300.
Path 324 | total_timesteps 3308.
Path 325 | total_timesteps 3318.
Path 326 | total_timesteps 3326.
Path 327 | total_timesteps 3335.
Path 328 | total_timesteps 3345.
Path 329 | total_timesteps 3352.
Path 330 | total_timesteps 3360.
Path 331 | total_timesteps 3372.
Path 332 | total_timesteps 3381.
Path 333 | total_timesteps 3389.
Path 334 | total_timesteps 3397.
Path 335 | total_timesteps 3405.
Path 336 | total_timesteps 3415.
Path 337 | total_timesteps 3426.
Path 338 | total_timesteps 3443.
Path 339 | total_timesteps 3452.
Path 340 | total_timesteps 3463.
Path 341 | total_timesteps 3474.
Path 342 | total_timesteps 3482.
Path 343 | total_timesteps 3491.
Path 344 | total_timesteps 3499.
Path 345 | total_timesteps 3510.
Path 346 | total_timesteps 3519.
Path 347 | total_timesteps 3526.
Path 348 | total_timesteps 3542.
Path 349 | total_timesteps 3550.
Path 350 | total_timesteps 3558.
Path 351 | total_timesteps 3578.
Path 352 | total_timesteps 3590.
Path 353 | total_timesteps 3596.
Path 354 | total_timesteps 3612.
Path 355 | total_timesteps 3624.
Path 356 | total_timesteps 3634.
Path 357 | total_timesteps 3645.
Path 358 | total_timesteps 3659.
Path 359 | total_timesteps 3668.
Path 360 | total_timesteps 3678.
Path 361 | total_timesteps 3685.
Path 362 | total_timesteps 3697.
Path 363 | total_timesteps 3704.
Path 364 | total_timesteps 3713.
Path 365 | total_timesteps 3720.
Path 366 | total_timesteps 3728.
Path 367 | total_timesteps 3737.
Path 368 | total_timesteps 3748.
Path 369 | total_timesteps 3758.
Path 370 | total_timesteps 3766.
Path 371 | total_timesteps 3772.
Path 372 | total_timesteps 3784.
Path 373 | total_timesteps 3794.
Path 374 | total_timesteps 3801.
Path 375 | total_timesteps 3810.
Path 376 | total_timesteps 3820.
Path 377 | total_timesteps 3828.
Path 378 | total_timesteps 3838.
Path 379 | total_timesteps 3848.
Path 380 | total_timesteps 3855.
Path 381 | total_timesteps 3870.
Path 382 | total_timesteps 3876.
Path 383 | total_timesteps 3886.
Path 384 | total_timesteps 3896.
Path 385 | total_timesteps 3906.
Path 386 | total_timesteps 3913.
Path 387 | total_timesteps 3921.
Path 388 | total_timesteps 3935.
Path 389 | total_timesteps 3944.
Path 390 | total_timesteps 3952.
Path 391 | total_timesteps 3960.
Path 392 | total_timesteps 3967.
Path 393 | total_timesteps 3976.
Path 394 | total_timesteps 3985.
Path 395 | total_timesteps 3993.
Path 396 | total_timesteps 4002.
Path 397 | total_timesteps 4010.
Path 398 | total_timesteps 4027.
Path 399 | total_timesteps 4039.
Path 400 | total_timesteps 4050.
Path 401 | total_timesteps 4064.
Path 402 | total_timesteps 4071.
Path 403 | total_timesteps 4083.
Path 404 | total_timesteps 4094.
Path 405 | total_timesteps 4107.
Path 406 | total_timesteps 4115.
Path 407 | total_timesteps 4131.
Path 408 | total_timesteps 4144.
Path 409 | total_timesteps 4154.
Path 410 | total_timesteps 4164.
Path 411 | total_timesteps 4171.
Path 412 | total_timesteps 4184.
Path 413 | total_timesteps 4194.
Path 414 | total_timesteps 4202.
Path 415 | total_timesteps 4215.
Path 416 | total_timesteps 4222.
Path 417 | total_timesteps 4231.
Path 418 | total_timesteps 4242.
Path 419 | total_timesteps 4250.
Path 420 | total_timesteps 4262.
Path 421 | total_timesteps 4274.
Path 422 | total_timesteps 4283.
Path 423 | total_timesteps 4294.
Path 424 | total_timesteps 4301.
Path 425 | total_timesteps 4323.
Path 426 | total_timesteps 4333.
Path 427 | total_timesteps 4341.
Path 428 | total_timesteps 4353.
Path 429 | total_timesteps 4362.
Path 430 | total_timesteps 4371.
Path 431 | total_timesteps 4379.
Path 432 | total_timesteps 4387.
Path 433 | total_timesteps 4398.
Path 434 | total_timesteps 4407.
Path 435 | total_timesteps 4418.
Path 436 | total_timesteps 4426.
Path 437 | total_timesteps 4444.
Path 438 | total_timesteps 4452.
Path 439 | total_timesteps 4461.
Path 440 | total_timesteps 4468.
Path 441 | total_timesteps 4476.
Path 442 | total_timesteps 4483.
Path 443 | total_timesteps 4489.
Path 444 | total_timesteps 4497.
Path 445 | total_timesteps 4509.
Path 446 | total_timesteps 4520.
Path 447 | total_timesteps 4533.
Path 448 | total_timesteps 4547.
Path 449 | total_timesteps 4557.
Path 450 | total_timesteps 4566.
Path 451 | total_timesteps 4574.
Path 452 | total_timesteps 4587.
Path 453 | total_timesteps 4599.
Path 454 | total_timesteps 4606.
Path 455 | total_timesteps 4612.
Path 456 | total_timesteps 4620.
Path 457 | total_timesteps 4627.
Path 458 | total_timesteps 4641.
Path 459 | total_timesteps 4651.
Path 460 | total_timesteps 4657.
Path 461 | total_timesteps 4669.
Path 462 | total_timesteps 4678.
Path 463 | total_timesteps 4685.
Path 464 | total_timesteps 4696.
Path 465 | total_timesteps 4705.
Path 466 | total_timesteps 4713.
Path 467 | total_timesteps 4722.
Path 468 | total_timesteps 4729.
Path 469 | total_timesteps 4738.
Path 470 | total_timesteps 4747.
Path 471 | total_timesteps 4754.
Path 472 | total_timesteps 4762.
Path 473 | total_timesteps 4770.
Path 474 | total_timesteps 4782.
Path 475 | total_timesteps 4794.
Path 476 | total_timesteps 4804.
Path 477 | total_timesteps 4815.
Path 478 | total_timesteps 4822.
Path 479 | total_timesteps 4832.
Path 480 | total_timesteps 4839.
Path 481 | total_timesteps 4848.
Path 482 | total_timesteps 4857.
Path 483 | total_timesteps 4865.
Path 484 | total_timesteps 4877.
Path 485 | total_timesteps 4884.
Path 486 | total_timesteps 4896.
Path 487 | total_timesteps 4902.
Path 488 | total_timesteps 4915.
Path 489 | total_timesteps 4928.
Path 490 | total_timesteps 4936.
Path 491 | total_timesteps 4947.
Path 492 | total_timesteps 4956.
Path 493 | total_timesteps 4963.
Path 494 | total_timesteps 4974.
Path 495 | total_timesteps 4983.
Path 496 | total_timesteps 4991.
Path 497 | total_timesteps 5004.
Path 498 | total_timesteps 5015.
Path 499 | total_timesteps 5031.
Path 500 | total_timesteps 5040.
Path 501 | total_timesteps 5049.
Path 502 | total_timesteps 5060.
Path 503 | total_timesteps 5067.
Path 504 | total_timesteps 5074.
Path 505 | total_timesteps 5085.
Path 506 | total_timesteps 5095.
Path 507 | total_timesteps 5104.
Path 508 | total_timesteps 5112.
Path 509 | total_timesteps 5119.
Path 510 | total_timesteps 5129.
Path 511 | total_timesteps 5139.
Path 512 | total_timesteps 5154.
Path 513 | total_timesteps 5166.
Path 514 | total_timesteps 5178.
Path 515 | total_timesteps 5189.
Path 516 | total_timesteps 5204.
Path 517 | total_timesteps 5215.
Path 518 | total_timesteps 5224.
Path 519 | total_timesteps 5231.
Path 520 | total_timesteps 5241.
Path 521 | total_timesteps 5248.
Path 522 | total_timesteps 5257.
Path 523 | total_timesteps 5269.
Path 524 | total_timesteps 5275.
Path 525 | total_timesteps 5283.
Path 526 | total_timesteps 5293.
Path 527 | total_timesteps 5300.
Path 528 | total_timesteps 5307.
Path 529 | total_timesteps 5313.
Path 530 | total_timesteps 5326.
Path 531 | total_timesteps 5340.
Path 532 | total_timesteps 5348.
Path 533 | total_timesteps 5359.
Path 534 | total_timesteps 5373.
Path 535 | total_timesteps 5383.
Path 536 | total_timesteps 5398.
Path 537 | total_timesteps 5413.
Path 538 | total_timesteps 5424.
Path 539 | total_timesteps 5437.
Path 540 | total_timesteps 5447.
Path 541 | total_timesteps 5462.
Path 542 | total_timesteps 5469.
Path 543 | total_timesteps 5481.
Path 544 | total_timesteps 5493.
Path 545 | total_timesteps 5505.
Path 546 | total_timesteps 5512.
Path 547 | total_timesteps 5533.
Path 548 | total_timesteps 5543.
Path 549 | total_timesteps 5550.
Path 550 | total_timesteps 5558.
Path 551 | total_timesteps 5568.
Path 552 | total_timesteps 5578.
Path 553 | total_timesteps 5591.
Path 554 | total_timesteps 5599.
Path 555 | total_timesteps 5611.
Path 556 | total_timesteps 5628.
Path 557 | total_timesteps 5645.
Path 558 | total_timesteps 5653.
Path 559 | total_timesteps 5666.
Path 560 | total_timesteps 5687.
Path 561 | total_timesteps 5701.
Path 562 | total_timesteps 5708.
Path 563 | total_timesteps 5719.
Path 564 | total_timesteps 5726.
Path 565 | total_timesteps 5735.
Path 566 | total_timesteps 5745.
Path 567 | total_timesteps 5756.
Path 568 | total_timesteps 5766.
Path 569 | total_timesteps 5777.
Path 570 | total_timesteps 5784.
Path 571 | total_timesteps 5792.
Path 572 | total_timesteps 5801.
Path 573 | total_timesteps 5816.
Path 574 | total_timesteps 5835.
Path 575 | total_timesteps 5844.
Path 576 | total_timesteps 5851.
Path 577 | total_timesteps 5881.
Path 578 | total_timesteps 5893.
Path 579 | total_timesteps 5907.
Path 580 | total_timesteps 5915.
Path 581 | total_timesteps 5925.
Path 582 | total_timesteps 5935.
Path 583 | total_timesteps 5944.
Path 584 | total_timesteps 5956.
Path 585 | total_timesteps 5968.
Path 586 | total_timesteps 5981.
Path 587 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.53    |
| Iteration     | 9        |
| MaximumReturn | 0.0693   |
| MinimumReturn | -19      |
| TotalSamples  | 44038    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.3067024052143097
Validation loss = 0.31392067670822144
Validation loss = 0.3144800364971161
Validation loss = 0.315183162689209
Validation loss = 0.3182113468647003
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 32.
Path 4 | total_timesteps 39.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 56.
Path 7 | total_timesteps 66.
Path 8 | total_timesteps 73.
Path 9 | total_timesteps 85.
Path 10 | total_timesteps 93.
Path 11 | total_timesteps 104.
Path 12 | total_timesteps 116.
Path 13 | total_timesteps 125.
Path 14 | total_timesteps 140.
Path 15 | total_timesteps 147.
Path 16 | total_timesteps 153.
Path 17 | total_timesteps 165.
Path 18 | total_timesteps 172.
Path 19 | total_timesteps 187.
Path 20 | total_timesteps 193.
Path 21 | total_timesteps 199.
Path 22 | total_timesteps 206.
Path 23 | total_timesteps 215.
Path 24 | total_timesteps 230.
Path 25 | total_timesteps 236.
Path 26 | total_timesteps 244.
Path 27 | total_timesteps 253.
Path 28 | total_timesteps 261.
Path 29 | total_timesteps 267.
Path 30 | total_timesteps 277.
Path 31 | total_timesteps 287.
Path 32 | total_timesteps 294.
Path 33 | total_timesteps 302.
Path 34 | total_timesteps 310.
Path 35 | total_timesteps 319.
Path 36 | total_timesteps 331.
Path 37 | total_timesteps 346.
Path 38 | total_timesteps 359.
Path 39 | total_timesteps 370.
Path 40 | total_timesteps 379.
Path 41 | total_timesteps 389.
Path 42 | total_timesteps 396.
Path 43 | total_timesteps 407.
Path 44 | total_timesteps 416.
Path 45 | total_timesteps 433.
Path 46 | total_timesteps 441.
Path 47 | total_timesteps 450.
Path 48 | total_timesteps 464.
Path 49 | total_timesteps 473.
Path 50 | total_timesteps 481.
Path 51 | total_timesteps 488.
Path 52 | total_timesteps 497.
Path 53 | total_timesteps 507.
Path 54 | total_timesteps 523.
Path 55 | total_timesteps 532.
Path 56 | total_timesteps 540.
Path 57 | total_timesteps 552.
Path 58 | total_timesteps 564.
Path 59 | total_timesteps 572.
Path 60 | total_timesteps 579.
Path 61 | total_timesteps 587.
Path 62 | total_timesteps 597.
Path 63 | total_timesteps 608.
Path 64 | total_timesteps 619.
Path 65 | total_timesteps 626.
Path 66 | total_timesteps 636.
Path 67 | total_timesteps 643.
Path 68 | total_timesteps 650.
Path 69 | total_timesteps 661.
Path 70 | total_timesteps 673.
Path 71 | total_timesteps 681.
Path 72 | total_timesteps 689.
Path 73 | total_timesteps 703.
Path 74 | total_timesteps 712.
Path 75 | total_timesteps 727.
Path 76 | total_timesteps 737.
Path 77 | total_timesteps 752.
Path 78 | total_timesteps 762.
Path 79 | total_timesteps 777.
Path 80 | total_timesteps 784.
Path 81 | total_timesteps 795.
Path 82 | total_timesteps 807.
Path 83 | total_timesteps 815.
Path 84 | total_timesteps 826.
Path 85 | total_timesteps 836.
Path 86 | total_timesteps 843.
Path 87 | total_timesteps 852.
Path 88 | total_timesteps 865.
Path 89 | total_timesteps 874.
Path 90 | total_timesteps 882.
Path 91 | total_timesteps 889.
Path 92 | total_timesteps 901.
Path 93 | total_timesteps 912.
Path 94 | total_timesteps 923.
Path 95 | total_timesteps 932.
Path 96 | total_timesteps 944.
Path 97 | total_timesteps 958.
Path 98 | total_timesteps 972.
Path 99 | total_timesteps 978.
Path 100 | total_timesteps 988.
Path 101 | total_timesteps 995.
Path 102 | total_timesteps 1004.
Path 103 | total_timesteps 1011.
Path 104 | total_timesteps 1018.
Path 105 | total_timesteps 1026.
Path 106 | total_timesteps 1035.
Path 107 | total_timesteps 1044.
Path 108 | total_timesteps 1054.
Path 109 | total_timesteps 1061.
Path 110 | total_timesteps 1074.
Path 111 | total_timesteps 1082.
Path 112 | total_timesteps 1091.
Path 113 | total_timesteps 1097.
Path 114 | total_timesteps 1107.
Path 115 | total_timesteps 1122.
Path 116 | total_timesteps 1131.
Path 117 | total_timesteps 1138.
Path 118 | total_timesteps 1152.
Path 119 | total_timesteps 1164.
Path 120 | total_timesteps 1173.
Path 121 | total_timesteps 1181.
Path 122 | total_timesteps 1192.
Path 123 | total_timesteps 1200.
Path 124 | total_timesteps 1208.
Path 125 | total_timesteps 1216.
Path 126 | total_timesteps 1224.
Path 127 | total_timesteps 1236.
Path 128 | total_timesteps 1247.
Path 129 | total_timesteps 1256.
Path 130 | total_timesteps 1268.
Path 131 | total_timesteps 1278.
Path 132 | total_timesteps 1286.
Path 133 | total_timesteps 1294.
Path 134 | total_timesteps 1301.
Path 135 | total_timesteps 1310.
Path 136 | total_timesteps 1318.
Path 137 | total_timesteps 1333.
Path 138 | total_timesteps 1346.
Path 139 | total_timesteps 1355.
Path 140 | total_timesteps 1362.
Path 141 | total_timesteps 1369.
Path 142 | total_timesteps 1376.
Path 143 | total_timesteps 1391.
Path 144 | total_timesteps 1401.
Path 145 | total_timesteps 1409.
Path 146 | total_timesteps 1421.
Path 147 | total_timesteps 1433.
Path 148 | total_timesteps 1441.
Path 149 | total_timesteps 1454.
Path 150 | total_timesteps 1463.
Path 151 | total_timesteps 1470.
Path 152 | total_timesteps 1481.
Path 153 | total_timesteps 1490.
Path 154 | total_timesteps 1505.
Path 155 | total_timesteps 1514.
Path 156 | total_timesteps 1523.
Path 157 | total_timesteps 1531.
Path 158 | total_timesteps 1540.
Path 159 | total_timesteps 1548.
Path 160 | total_timesteps 1555.
Path 161 | total_timesteps 1563.
Path 162 | total_timesteps 1575.
Path 163 | total_timesteps 1590.
Path 164 | total_timesteps 1597.
Path 165 | total_timesteps 1606.
Path 166 | total_timesteps 1614.
Path 167 | total_timesteps 1624.
Path 168 | total_timesteps 1631.
Path 169 | total_timesteps 1640.
Path 170 | total_timesteps 1651.
Path 171 | total_timesteps 1661.
Path 172 | total_timesteps 1672.
Path 173 | total_timesteps 1684.
Path 174 | total_timesteps 1691.
Path 175 | total_timesteps 1699.
Path 176 | total_timesteps 1706.
Path 177 | total_timesteps 1718.
Path 178 | total_timesteps 1726.
Path 179 | total_timesteps 1737.
Path 180 | total_timesteps 1747.
Path 181 | total_timesteps 1757.
Path 182 | total_timesteps 1767.
Path 183 | total_timesteps 1775.
Path 184 | total_timesteps 1783.
Path 185 | total_timesteps 1792.
Path 186 | total_timesteps 1800.
Path 187 | total_timesteps 1810.
Path 188 | total_timesteps 1819.
Path 189 | total_timesteps 1827.
Path 190 | total_timesteps 1837.
Path 191 | total_timesteps 1848.
Path 192 | total_timesteps 1858.
Path 193 | total_timesteps 1867.
Path 194 | total_timesteps 1875.
Path 195 | total_timesteps 1887.
Path 196 | total_timesteps 1896.
Path 197 | total_timesteps 1906.
Path 198 | total_timesteps 1916.
Path 199 | total_timesteps 1923.
Path 200 | total_timesteps 1930.
Path 201 | total_timesteps 1938.
Path 202 | total_timesteps 1952.
Path 203 | total_timesteps 1959.
Path 204 | total_timesteps 1968.
Path 205 | total_timesteps 1977.
Path 206 | total_timesteps 1989.
Path 207 | total_timesteps 1996.
Path 208 | total_timesteps 2007.
Path 209 | total_timesteps 2017.
Path 210 | total_timesteps 2030.
Path 211 | total_timesteps 2037.
Path 212 | total_timesteps 2048.
Path 213 | total_timesteps 2056.
Path 214 | total_timesteps 2068.
Path 215 | total_timesteps 2079.
Path 216 | total_timesteps 2089.
Path 217 | total_timesteps 2098.
Path 218 | total_timesteps 2106.
Path 219 | total_timesteps 2118.
Path 220 | total_timesteps 2126.
Path 221 | total_timesteps 2138.
Path 222 | total_timesteps 2145.
Path 223 | total_timesteps 2157.
Path 224 | total_timesteps 2163.
Path 225 | total_timesteps 2175.
Path 226 | total_timesteps 2182.
Path 227 | total_timesteps 2195.
Path 228 | total_timesteps 2202.
Path 229 | total_timesteps 2213.
Path 230 | total_timesteps 2231.
Path 231 | total_timesteps 2238.
Path 232 | total_timesteps 2246.
Path 233 | total_timesteps 2253.
Path 234 | total_timesteps 2261.
Path 235 | total_timesteps 2273.
Path 236 | total_timesteps 2283.
Path 237 | total_timesteps 2291.
Path 238 | total_timesteps 2300.
Path 239 | total_timesteps 2306.
Path 240 | total_timesteps 2316.
Path 241 | total_timesteps 2333.
Path 242 | total_timesteps 2343.
Path 243 | total_timesteps 2353.
Path 244 | total_timesteps 2367.
Path 245 | total_timesteps 2380.
Path 246 | total_timesteps 2391.
Path 247 | total_timesteps 2401.
Path 248 | total_timesteps 2415.
Path 249 | total_timesteps 2427.
Path 250 | total_timesteps 2436.
Path 251 | total_timesteps 2444.
Path 252 | total_timesteps 2452.
Path 253 | total_timesteps 2461.
Path 254 | total_timesteps 2472.
Path 255 | total_timesteps 2482.
Path 256 | total_timesteps 2493.
Path 257 | total_timesteps 2507.
Path 258 | total_timesteps 2517.
Path 259 | total_timesteps 2529.
Path 260 | total_timesteps 2542.
Path 261 | total_timesteps 2550.
Path 262 | total_timesteps 2558.
Path 263 | total_timesteps 2565.
Path 264 | total_timesteps 2576.
Path 265 | total_timesteps 2590.
Path 266 | total_timesteps 2597.
Path 267 | total_timesteps 2605.
Path 268 | total_timesteps 2617.
Path 269 | total_timesteps 2627.
Path 270 | total_timesteps 2635.
Path 271 | total_timesteps 2641.
Path 272 | total_timesteps 2648.
Path 273 | total_timesteps 2658.
Path 274 | total_timesteps 2665.
Path 275 | total_timesteps 2673.
Path 276 | total_timesteps 2686.
Path 277 | total_timesteps 2697.
Path 278 | total_timesteps 2703.
Path 279 | total_timesteps 2711.
Path 280 | total_timesteps 2720.
Path 281 | total_timesteps 2727.
Path 282 | total_timesteps 2739.
Path 283 | total_timesteps 2748.
Path 284 | total_timesteps 2759.
Path 285 | total_timesteps 2766.
Path 286 | total_timesteps 2774.
Path 287 | total_timesteps 2786.
Path 288 | total_timesteps 2793.
Path 289 | total_timesteps 2800.
Path 290 | total_timesteps 2807.
Path 291 | total_timesteps 2814.
Path 292 | total_timesteps 2822.
Path 293 | total_timesteps 2831.
Path 294 | total_timesteps 2840.
Path 295 | total_timesteps 2848.
Path 296 | total_timesteps 2856.
Path 297 | total_timesteps 2863.
Path 298 | total_timesteps 2870.
Path 299 | total_timesteps 2879.
Path 300 | total_timesteps 2886.
Path 301 | total_timesteps 2893.
Path 302 | total_timesteps 2902.
Path 303 | total_timesteps 2912.
Path 304 | total_timesteps 2921.
Path 305 | total_timesteps 2930.
Path 306 | total_timesteps 2939.
Path 307 | total_timesteps 2948.
Path 308 | total_timesteps 2961.
Path 309 | total_timesteps 2969.
Path 310 | total_timesteps 2978.
Path 311 | total_timesteps 2984.
Path 312 | total_timesteps 2994.
Path 313 | total_timesteps 3003.
Path 314 | total_timesteps 3011.
Path 315 | total_timesteps 3025.
Path 316 | total_timesteps 3035.
Path 317 | total_timesteps 3042.
Path 318 | total_timesteps 3050.
Path 319 | total_timesteps 3057.
Path 320 | total_timesteps 3065.
Path 321 | total_timesteps 3078.
Path 322 | total_timesteps 3086.
Path 323 | total_timesteps 3093.
Path 324 | total_timesteps 3100.
Path 325 | total_timesteps 3106.
Path 326 | total_timesteps 3118.
Path 327 | total_timesteps 3127.
Path 328 | total_timesteps 3136.
Path 329 | total_timesteps 3151.
Path 330 | total_timesteps 3163.
Path 331 | total_timesteps 3169.
Path 332 | total_timesteps 3180.
Path 333 | total_timesteps 3189.
Path 334 | total_timesteps 3196.
Path 335 | total_timesteps 3203.
Path 336 | total_timesteps 3222.
Path 337 | total_timesteps 3230.
Path 338 | total_timesteps 3239.
Path 339 | total_timesteps 3247.
Path 340 | total_timesteps 3255.
Path 341 | total_timesteps 3264.
Path 342 | total_timesteps 3274.
Path 343 | total_timesteps 3289.
Path 344 | total_timesteps 3300.
Path 345 | total_timesteps 3309.
Path 346 | total_timesteps 3320.
Path 347 | total_timesteps 3331.
Path 348 | total_timesteps 3339.
Path 349 | total_timesteps 3352.
Path 350 | total_timesteps 3359.
Path 351 | total_timesteps 3366.
Path 352 | total_timesteps 3372.
Path 353 | total_timesteps 3384.
Path 354 | total_timesteps 3392.
Path 355 | total_timesteps 3399.
Path 356 | total_timesteps 3407.
Path 357 | total_timesteps 3414.
Path 358 | total_timesteps 3427.
Path 359 | total_timesteps 3435.
Path 360 | total_timesteps 3446.
Path 361 | total_timesteps 3455.
Path 362 | total_timesteps 3471.
Path 363 | total_timesteps 3479.
Path 364 | total_timesteps 3493.
Path 365 | total_timesteps 3502.
Path 366 | total_timesteps 3512.
Path 367 | total_timesteps 3521.
Path 368 | total_timesteps 3540.
Path 369 | total_timesteps 3547.
Path 370 | total_timesteps 3554.
Path 371 | total_timesteps 3561.
Path 372 | total_timesteps 3568.
Path 373 | total_timesteps 3584.
Path 374 | total_timesteps 3591.
Path 375 | total_timesteps 3605.
Path 376 | total_timesteps 3615.
Path 377 | total_timesteps 3625.
Path 378 | total_timesteps 3635.
Path 379 | total_timesteps 3642.
Path 380 | total_timesteps 3654.
Path 381 | total_timesteps 3663.
Path 382 | total_timesteps 3673.
Path 383 | total_timesteps 3682.
Path 384 | total_timesteps 3692.
Path 385 | total_timesteps 3706.
Path 386 | total_timesteps 3714.
Path 387 | total_timesteps 3729.
Path 388 | total_timesteps 3741.
Path 389 | total_timesteps 3755.
Path 390 | total_timesteps 3761.
Path 391 | total_timesteps 3768.
Path 392 | total_timesteps 3775.
Path 393 | total_timesteps 3791.
Path 394 | total_timesteps 3798.
Path 395 | total_timesteps 3806.
Path 396 | total_timesteps 3817.
Path 397 | total_timesteps 3824.
Path 398 | total_timesteps 3830.
Path 399 | total_timesteps 3838.
Path 400 | total_timesteps 3846.
Path 401 | total_timesteps 3860.
Path 402 | total_timesteps 3869.
Path 403 | total_timesteps 3878.
Path 404 | total_timesteps 3886.
Path 405 | total_timesteps 3895.
Path 406 | total_timesteps 3908.
Path 407 | total_timesteps 3917.
Path 408 | total_timesteps 3930.
Path 409 | total_timesteps 3938.
Path 410 | total_timesteps 3945.
Path 411 | total_timesteps 3952.
Path 412 | total_timesteps 3962.
Path 413 | total_timesteps 3972.
Path 414 | total_timesteps 3979.
Path 415 | total_timesteps 3985.
Path 416 | total_timesteps 3995.
Path 417 | total_timesteps 4004.
Path 418 | total_timesteps 4013.
Path 419 | total_timesteps 4022.
Path 420 | total_timesteps 4031.
Path 421 | total_timesteps 4038.
Path 422 | total_timesteps 4050.
Path 423 | total_timesteps 4059.
Path 424 | total_timesteps 4069.
Path 425 | total_timesteps 4078.
Path 426 | total_timesteps 4089.
Path 427 | total_timesteps 4097.
Path 428 | total_timesteps 4104.
Path 429 | total_timesteps 4116.
Path 430 | total_timesteps 4123.
Path 431 | total_timesteps 4133.
Path 432 | total_timesteps 4142.
Path 433 | total_timesteps 4152.
Path 434 | total_timesteps 4162.
Path 435 | total_timesteps 4169.
Path 436 | total_timesteps 4183.
Path 437 | total_timesteps 4194.
Path 438 | total_timesteps 4201.
Path 439 | total_timesteps 4209.
Path 440 | total_timesteps 4217.
Path 441 | total_timesteps 4229.
Path 442 | total_timesteps 4238.
Path 443 | total_timesteps 4246.
Path 444 | total_timesteps 4253.
Path 445 | total_timesteps 4263.
Path 446 | total_timesteps 4278.
Path 447 | total_timesteps 4286.
Path 448 | total_timesteps 4298.
Path 449 | total_timesteps 4307.
Path 450 | total_timesteps 4323.
Path 451 | total_timesteps 4334.
Path 452 | total_timesteps 4341.
Path 453 | total_timesteps 4350.
Path 454 | total_timesteps 4358.
Path 455 | total_timesteps 4367.
Path 456 | total_timesteps 4375.
Path 457 | total_timesteps 4387.
Path 458 | total_timesteps 4394.
Path 459 | total_timesteps 4411.
Path 460 | total_timesteps 4418.
Path 461 | total_timesteps 4425.
Path 462 | total_timesteps 4433.
Path 463 | total_timesteps 4442.
Path 464 | total_timesteps 4450.
Path 465 | total_timesteps 4457.
Path 466 | total_timesteps 4469.
Path 467 | total_timesteps 4478.
Path 468 | total_timesteps 4489.
Path 469 | total_timesteps 4503.
Path 470 | total_timesteps 4511.
Path 471 | total_timesteps 4519.
Path 472 | total_timesteps 4532.
Path 473 | total_timesteps 4545.
Path 474 | total_timesteps 4553.
Path 475 | total_timesteps 4560.
Path 476 | total_timesteps 4571.
Path 477 | total_timesteps 4580.
Path 478 | total_timesteps 4587.
Path 479 | total_timesteps 4594.
Path 480 | total_timesteps 4602.
Path 481 | total_timesteps 4617.
Path 482 | total_timesteps 4626.
Path 483 | total_timesteps 4636.
Path 484 | total_timesteps 4642.
Path 485 | total_timesteps 4650.
Path 486 | total_timesteps 4657.
Path 487 | total_timesteps 4670.
Path 488 | total_timesteps 4678.
Path 489 | total_timesteps 4693.
Path 490 | total_timesteps 4705.
Path 491 | total_timesteps 4712.
Path 492 | total_timesteps 4725.
Path 493 | total_timesteps 4735.
Path 494 | total_timesteps 4749.
Path 495 | total_timesteps 4766.
Path 496 | total_timesteps 4774.
Path 497 | total_timesteps 4783.
Path 498 | total_timesteps 4790.
Path 499 | total_timesteps 4799.
Path 500 | total_timesteps 4807.
Path 501 | total_timesteps 4820.
Path 502 | total_timesteps 4832.
Path 503 | total_timesteps 4845.
Path 504 | total_timesteps 4860.
Path 505 | total_timesteps 4868.
Path 506 | total_timesteps 4877.
Path 507 | total_timesteps 4886.
Path 508 | total_timesteps 4893.
Path 509 | total_timesteps 4901.
Path 510 | total_timesteps 4910.
Path 511 | total_timesteps 4925.
Path 512 | total_timesteps 4935.
Path 513 | total_timesteps 4947.
Path 514 | total_timesteps 4956.
Path 515 | total_timesteps 4964.
Path 516 | total_timesteps 4971.
Path 517 | total_timesteps 4982.
Path 518 | total_timesteps 4991.
Path 519 | total_timesteps 4998.
Path 520 | total_timesteps 5006.
Path 521 | total_timesteps 5012.
Path 522 | total_timesteps 5019.
Path 523 | total_timesteps 5028.
Path 524 | total_timesteps 5041.
Path 525 | total_timesteps 5051.
Path 526 | total_timesteps 5059.
Path 527 | total_timesteps 5073.
Path 528 | total_timesteps 5085.
Path 529 | total_timesteps 5092.
Path 530 | total_timesteps 5098.
Path 531 | total_timesteps 5109.
Path 532 | total_timesteps 5116.
Path 533 | total_timesteps 5123.
Path 534 | total_timesteps 5133.
Path 535 | total_timesteps 5143.
Path 536 | total_timesteps 5151.
Path 537 | total_timesteps 5159.
Path 538 | total_timesteps 5177.
Path 539 | total_timesteps 5186.
Path 540 | total_timesteps 5195.
Path 541 | total_timesteps 5204.
Path 542 | total_timesteps 5212.
Path 543 | total_timesteps 5218.
Path 544 | total_timesteps 5226.
Path 545 | total_timesteps 5238.
Path 546 | total_timesteps 5252.
Path 547 | total_timesteps 5263.
Path 548 | total_timesteps 5273.
Path 549 | total_timesteps 5284.
Path 550 | total_timesteps 5291.
Path 551 | total_timesteps 5300.
Path 552 | total_timesteps 5311.
Path 553 | total_timesteps 5319.
Path 554 | total_timesteps 5332.
Path 555 | total_timesteps 5340.
Path 556 | total_timesteps 5350.
Path 557 | total_timesteps 5356.
Path 558 | total_timesteps 5366.
Path 559 | total_timesteps 5373.
Path 560 | total_timesteps 5381.
Path 561 | total_timesteps 5395.
Path 562 | total_timesteps 5402.
Path 563 | total_timesteps 5411.
Path 564 | total_timesteps 5422.
Path 565 | total_timesteps 5430.
Path 566 | total_timesteps 5438.
Path 567 | total_timesteps 5450.
Path 568 | total_timesteps 5458.
Path 569 | total_timesteps 5467.
Path 570 | total_timesteps 5479.
Path 571 | total_timesteps 5487.
Path 572 | total_timesteps 5498.
Path 573 | total_timesteps 5515.
Path 574 | total_timesteps 5522.
Path 575 | total_timesteps 5530.
Path 576 | total_timesteps 5539.
Path 577 | total_timesteps 5547.
Path 578 | total_timesteps 5558.
Path 579 | total_timesteps 5569.
Path 580 | total_timesteps 5579.
Path 581 | total_timesteps 5590.
Path 582 | total_timesteps 5601.
Path 583 | total_timesteps 5610.
Path 584 | total_timesteps 5622.
Path 585 | total_timesteps 5629.
Path 586 | total_timesteps 5642.
Path 587 | total_timesteps 5648.
Path 588 | total_timesteps 5657.
Path 589 | total_timesteps 5677.
Path 590 | total_timesteps 5686.
Path 591 | total_timesteps 5703.
Path 592 | total_timesteps 5710.
Path 593 | total_timesteps 5717.
Path 594 | total_timesteps 5725.
Path 595 | total_timesteps 5735.
Path 596 | total_timesteps 5743.
Path 597 | total_timesteps 5752.
Path 598 | total_timesteps 5765.
Path 599 | total_timesteps 5771.
Path 600 | total_timesteps 5779.
Path 601 | total_timesteps 5791.
Path 602 | total_timesteps 5805.
Path 603 | total_timesteps 5817.
Path 604 | total_timesteps 5828.
Path 605 | total_timesteps 5837.
Path 606 | total_timesteps 5847.
Path 607 | total_timesteps 5856.
Path 608 | total_timesteps 5871.
Path 609 | total_timesteps 5881.
Path 610 | total_timesteps 5890.
Path 611 | total_timesteps 5900.
Path 612 | total_timesteps 5906.
Path 613 | total_timesteps 5918.
Path 614 | total_timesteps 5927.
Path 615 | total_timesteps 5935.
Path 616 | total_timesteps 5948.
Path 617 | total_timesteps 5959.
Path 618 | total_timesteps 5972.
Path 619 | total_timesteps 5979.
Path 620 | total_timesteps 5987.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.61    |
| Iteration     | 10       |
| MaximumReturn | 0.942    |
| MinimumReturn | -17.7    |
| TotalSamples  | 48038    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.30785107612609863
Validation loss = 0.31464600563049316
Validation loss = 0.31468477845191956
Validation loss = 0.32198378443717957
Validation loss = 0.32872244715690613
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 62.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 79.
Path 9 | total_timesteps 87.
Path 10 | total_timesteps 98.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 120.
Path 13 | total_timesteps 133.
Path 14 | total_timesteps 143.
Path 15 | total_timesteps 159.
Path 16 | total_timesteps 168.
Path 17 | total_timesteps 178.
Path 18 | total_timesteps 190.
Path 19 | total_timesteps 205.
Path 20 | total_timesteps 213.
Path 21 | total_timesteps 229.
Path 22 | total_timesteps 238.
Path 23 | total_timesteps 247.
Path 24 | total_timesteps 262.
Path 25 | total_timesteps 271.
Path 26 | total_timesteps 282.
Path 27 | total_timesteps 294.
Path 28 | total_timesteps 307.
Path 29 | total_timesteps 319.
Path 30 | total_timesteps 329.
Path 31 | total_timesteps 343.
Path 32 | total_timesteps 353.
Path 33 | total_timesteps 367.
Path 34 | total_timesteps 376.
Path 35 | total_timesteps 388.
Path 36 | total_timesteps 396.
Path 37 | total_timesteps 403.
Path 38 | total_timesteps 410.
Path 39 | total_timesteps 418.
Path 40 | total_timesteps 435.
Path 41 | total_timesteps 447.
Path 42 | total_timesteps 454.
Path 43 | total_timesteps 465.
Path 44 | total_timesteps 476.
Path 45 | total_timesteps 489.
Path 46 | total_timesteps 502.
Path 47 | total_timesteps 510.
Path 48 | total_timesteps 522.
Path 49 | total_timesteps 533.
Path 50 | total_timesteps 545.
Path 51 | total_timesteps 555.
Path 52 | total_timesteps 568.
Path 53 | total_timesteps 577.
Path 54 | total_timesteps 588.
Path 55 | total_timesteps 596.
Path 56 | total_timesteps 611.
Path 57 | total_timesteps 623.
Path 58 | total_timesteps 637.
Path 59 | total_timesteps 647.
Path 60 | total_timesteps 660.
Path 61 | total_timesteps 670.
Path 62 | total_timesteps 678.
Path 63 | total_timesteps 688.
Path 64 | total_timesteps 696.
Path 65 | total_timesteps 705.
Path 66 | total_timesteps 715.
Path 67 | total_timesteps 722.
Path 68 | total_timesteps 733.
Path 69 | total_timesteps 742.
Path 70 | total_timesteps 752.
Path 71 | total_timesteps 761.
Path 72 | total_timesteps 769.
Path 73 | total_timesteps 776.
Path 74 | total_timesteps 787.
Path 75 | total_timesteps 797.
Path 76 | total_timesteps 805.
Path 77 | total_timesteps 814.
Path 78 | total_timesteps 831.
Path 79 | total_timesteps 841.
Path 80 | total_timesteps 851.
Path 81 | total_timesteps 861.
Path 82 | total_timesteps 871.
Path 83 | total_timesteps 883.
Path 84 | total_timesteps 898.
Path 85 | total_timesteps 906.
Path 86 | total_timesteps 913.
Path 87 | total_timesteps 922.
Path 88 | total_timesteps 931.
Path 89 | total_timesteps 947.
Path 90 | total_timesteps 957.
Path 91 | total_timesteps 967.
Path 92 | total_timesteps 974.
Path 93 | total_timesteps 983.
Path 94 | total_timesteps 991.
Path 95 | total_timesteps 1001.
Path 96 | total_timesteps 1018.
Path 97 | total_timesteps 1028.
Path 98 | total_timesteps 1037.
Path 99 | total_timesteps 1046.
Path 100 | total_timesteps 1054.
Path 101 | total_timesteps 1070.
Path 102 | total_timesteps 1079.
Path 103 | total_timesteps 1088.
Path 104 | total_timesteps 1097.
Path 105 | total_timesteps 1104.
Path 106 | total_timesteps 1114.
Path 107 | total_timesteps 1122.
Path 108 | total_timesteps 1134.
Path 109 | total_timesteps 1146.
Path 110 | total_timesteps 1155.
Path 111 | total_timesteps 1169.
Path 112 | total_timesteps 1181.
Path 113 | total_timesteps 1194.
Path 114 | total_timesteps 1204.
Path 115 | total_timesteps 1213.
Path 116 | total_timesteps 1226.
Path 117 | total_timesteps 1235.
Path 118 | total_timesteps 1245.
Path 119 | total_timesteps 1255.
Path 120 | total_timesteps 1264.
Path 121 | total_timesteps 1282.
Path 122 | total_timesteps 1290.
Path 123 | total_timesteps 1300.
Path 124 | total_timesteps 1310.
Path 125 | total_timesteps 1319.
Path 126 | total_timesteps 1329.
Path 127 | total_timesteps 1338.
Path 128 | total_timesteps 1345.
Path 129 | total_timesteps 1356.
Path 130 | total_timesteps 1373.
Path 131 | total_timesteps 1380.
Path 132 | total_timesteps 1389.
Path 133 | total_timesteps 1398.
Path 134 | total_timesteps 1406.
Path 135 | total_timesteps 1419.
Path 136 | total_timesteps 1432.
Path 137 | total_timesteps 1442.
Path 138 | total_timesteps 1451.
Path 139 | total_timesteps 1458.
Path 140 | total_timesteps 1466.
Path 141 | total_timesteps 1477.
Path 142 | total_timesteps 1488.
Path 143 | total_timesteps 1500.
Path 144 | total_timesteps 1507.
Path 145 | total_timesteps 1515.
Path 146 | total_timesteps 1524.
Path 147 | total_timesteps 1533.
Path 148 | total_timesteps 1541.
Path 149 | total_timesteps 1554.
Path 150 | total_timesteps 1566.
Path 151 | total_timesteps 1572.
Path 152 | total_timesteps 1581.
Path 153 | total_timesteps 1596.
Path 154 | total_timesteps 1605.
Path 155 | total_timesteps 1614.
Path 156 | total_timesteps 1624.
Path 157 | total_timesteps 1642.
Path 158 | total_timesteps 1651.
Path 159 | total_timesteps 1660.
Path 160 | total_timesteps 1671.
Path 161 | total_timesteps 1681.
Path 162 | total_timesteps 1690.
Path 163 | total_timesteps 1704.
Path 164 | total_timesteps 1712.
Path 165 | total_timesteps 1722.
Path 166 | total_timesteps 1737.
Path 167 | total_timesteps 1743.
Path 168 | total_timesteps 1749.
Path 169 | total_timesteps 1759.
Path 170 | total_timesteps 1769.
Path 171 | total_timesteps 1781.
Path 172 | total_timesteps 1791.
Path 173 | total_timesteps 1801.
Path 174 | total_timesteps 1811.
Path 175 | total_timesteps 1820.
Path 176 | total_timesteps 1831.
Path 177 | total_timesteps 1849.
Path 178 | total_timesteps 1864.
Path 179 | total_timesteps 1877.
Path 180 | total_timesteps 1887.
Path 181 | total_timesteps 1904.
Path 182 | total_timesteps 1916.
Path 183 | total_timesteps 1924.
Path 184 | total_timesteps 1932.
Path 185 | total_timesteps 1950.
Path 186 | total_timesteps 1960.
Path 187 | total_timesteps 1969.
Path 188 | total_timesteps 1981.
Path 189 | total_timesteps 1989.
Path 190 | total_timesteps 2005.
Path 191 | total_timesteps 2012.
Path 192 | total_timesteps 2021.
Path 193 | total_timesteps 2036.
Path 194 | total_timesteps 2047.
Path 195 | total_timesteps 2056.
Path 196 | total_timesteps 2065.
Path 197 | total_timesteps 2085.
Path 198 | total_timesteps 2096.
Path 199 | total_timesteps 2107.
Path 200 | total_timesteps 2116.
Path 201 | total_timesteps 2123.
Path 202 | total_timesteps 2131.
Path 203 | total_timesteps 2139.
Path 204 | total_timesteps 2152.
Path 205 | total_timesteps 2161.
Path 206 | total_timesteps 2169.
Path 207 | total_timesteps 2182.
Path 208 | total_timesteps 2192.
Path 209 | total_timesteps 2200.
Path 210 | total_timesteps 2213.
Path 211 | total_timesteps 2226.
Path 212 | total_timesteps 2239.
Path 213 | total_timesteps 2248.
Path 214 | total_timesteps 2262.
Path 215 | total_timesteps 2271.
Path 216 | total_timesteps 2278.
Path 217 | total_timesteps 2287.
Path 218 | total_timesteps 2301.
Path 219 | total_timesteps 2311.
Path 220 | total_timesteps 2320.
Path 221 | total_timesteps 2334.
Path 222 | total_timesteps 2350.
Path 223 | total_timesteps 2359.
Path 224 | total_timesteps 2367.
Path 225 | total_timesteps 2376.
Path 226 | total_timesteps 2385.
Path 227 | total_timesteps 2396.
Path 228 | total_timesteps 2407.
Path 229 | total_timesteps 2419.
Path 230 | total_timesteps 2434.
Path 231 | total_timesteps 2443.
Path 232 | total_timesteps 2457.
Path 233 | total_timesteps 2467.
Path 234 | total_timesteps 2477.
Path 235 | total_timesteps 2488.
Path 236 | total_timesteps 2498.
Path 237 | total_timesteps 2511.
Path 238 | total_timesteps 2522.
Path 239 | total_timesteps 2530.
Path 240 | total_timesteps 2541.
Path 241 | total_timesteps 2551.
Path 242 | total_timesteps 2560.
Path 243 | total_timesteps 2569.
Path 244 | total_timesteps 2585.
Path 245 | total_timesteps 2597.
Path 246 | total_timesteps 2605.
Path 247 | total_timesteps 2615.
Path 248 | total_timesteps 2624.
Path 249 | total_timesteps 2634.
Path 250 | total_timesteps 2643.
Path 251 | total_timesteps 2651.
Path 252 | total_timesteps 2662.
Path 253 | total_timesteps 2672.
Path 254 | total_timesteps 2682.
Path 255 | total_timesteps 2692.
Path 256 | total_timesteps 2707.
Path 257 | total_timesteps 2719.
Path 258 | total_timesteps 2727.
Path 259 | total_timesteps 2738.
Path 260 | total_timesteps 2747.
Path 261 | total_timesteps 2759.
Path 262 | total_timesteps 2770.
Path 263 | total_timesteps 2783.
Path 264 | total_timesteps 2796.
Path 265 | total_timesteps 2809.
Path 266 | total_timesteps 2819.
Path 267 | total_timesteps 2829.
Path 268 | total_timesteps 2836.
Path 269 | total_timesteps 2843.
Path 270 | total_timesteps 2852.
Path 271 | total_timesteps 2866.
Path 272 | total_timesteps 2875.
Path 273 | total_timesteps 2883.
Path 274 | total_timesteps 2896.
Path 275 | total_timesteps 2906.
Path 276 | total_timesteps 2918.
Path 277 | total_timesteps 2929.
Path 278 | total_timesteps 2943.
Path 279 | total_timesteps 2950.
Path 280 | total_timesteps 2959.
Path 281 | total_timesteps 2969.
Path 282 | total_timesteps 2978.
Path 283 | total_timesteps 2988.
Path 284 | total_timesteps 2997.
Path 285 | total_timesteps 3013.
Path 286 | total_timesteps 3026.
Path 287 | total_timesteps 3033.
Path 288 | total_timesteps 3047.
Path 289 | total_timesteps 3057.
Path 290 | total_timesteps 3067.
Path 291 | total_timesteps 3076.
Path 292 | total_timesteps 3084.
Path 293 | total_timesteps 3093.
Path 294 | total_timesteps 3101.
Path 295 | total_timesteps 3115.
Path 296 | total_timesteps 3124.
Path 297 | total_timesteps 3133.
Path 298 | total_timesteps 3141.
Path 299 | total_timesteps 3148.
Path 300 | total_timesteps 3156.
Path 301 | total_timesteps 3166.
Path 302 | total_timesteps 3180.
Path 303 | total_timesteps 3191.
Path 304 | total_timesteps 3201.
Path 305 | total_timesteps 3210.
Path 306 | total_timesteps 3224.
Path 307 | total_timesteps 3234.
Path 308 | total_timesteps 3242.
Path 309 | total_timesteps 3264.
Path 310 | total_timesteps 3272.
Path 311 | total_timesteps 3281.
Path 312 | total_timesteps 3289.
Path 313 | total_timesteps 3300.
Path 314 | total_timesteps 3311.
Path 315 | total_timesteps 3320.
Path 316 | total_timesteps 3328.
Path 317 | total_timesteps 3337.
Path 318 | total_timesteps 3345.
Path 319 | total_timesteps 3357.
Path 320 | total_timesteps 3366.
Path 321 | total_timesteps 3375.
Path 322 | total_timesteps 3390.
Path 323 | total_timesteps 3400.
Path 324 | total_timesteps 3409.
Path 325 | total_timesteps 3419.
Path 326 | total_timesteps 3429.
Path 327 | total_timesteps 3440.
Path 328 | total_timesteps 3448.
Path 329 | total_timesteps 3465.
Path 330 | total_timesteps 3478.
Path 331 | total_timesteps 3487.
Path 332 | total_timesteps 3497.
Path 333 | total_timesteps 3507.
Path 334 | total_timesteps 3514.
Path 335 | total_timesteps 3524.
Path 336 | total_timesteps 3536.
Path 337 | total_timesteps 3546.
Path 338 | total_timesteps 3556.
Path 339 | total_timesteps 3564.
Path 340 | total_timesteps 3578.
Path 341 | total_timesteps 3587.
Path 342 | total_timesteps 3599.
Path 343 | total_timesteps 3610.
Path 344 | total_timesteps 3631.
Path 345 | total_timesteps 3640.
Path 346 | total_timesteps 3648.
Path 347 | total_timesteps 3657.
Path 348 | total_timesteps 3666.
Path 349 | total_timesteps 3673.
Path 350 | total_timesteps 3684.
Path 351 | total_timesteps 3694.
Path 352 | total_timesteps 3705.
Path 353 | total_timesteps 3717.
Path 354 | total_timesteps 3730.
Path 355 | total_timesteps 3741.
Path 356 | total_timesteps 3756.
Path 357 | total_timesteps 3769.
Path 358 | total_timesteps 3784.
Path 359 | total_timesteps 3795.
Path 360 | total_timesteps 3807.
Path 361 | total_timesteps 3815.
Path 362 | total_timesteps 3824.
Path 363 | total_timesteps 3836.
Path 364 | total_timesteps 3846.
Path 365 | total_timesteps 3856.
Path 366 | total_timesteps 3864.
Path 367 | total_timesteps 3877.
Path 368 | total_timesteps 3891.
Path 369 | total_timesteps 3901.
Path 370 | total_timesteps 3907.
Path 371 | total_timesteps 3915.
Path 372 | total_timesteps 3922.
Path 373 | total_timesteps 3936.
Path 374 | total_timesteps 3948.
Path 375 | total_timesteps 3959.
Path 376 | total_timesteps 3966.
Path 377 | total_timesteps 3974.
Path 378 | total_timesteps 3988.
Path 379 | total_timesteps 3997.
Path 380 | total_timesteps 4010.
Path 381 | total_timesteps 4018.
Path 382 | total_timesteps 4027.
Path 383 | total_timesteps 4036.
Path 384 | total_timesteps 4046.
Path 385 | total_timesteps 4059.
Path 386 | total_timesteps 4070.
Path 387 | total_timesteps 4082.
Path 388 | total_timesteps 4093.
Path 389 | total_timesteps 4101.
Path 390 | total_timesteps 4114.
Path 391 | total_timesteps 4123.
Path 392 | total_timesteps 4135.
Path 393 | total_timesteps 4148.
Path 394 | total_timesteps 4178.
Path 395 | total_timesteps 4192.
Path 396 | total_timesteps 4201.
Path 397 | total_timesteps 4211.
Path 398 | total_timesteps 4220.
Path 399 | total_timesteps 4232.
Path 400 | total_timesteps 4241.
Path 401 | total_timesteps 4249.
Path 402 | total_timesteps 4255.
Path 403 | total_timesteps 4271.
Path 404 | total_timesteps 4280.
Path 405 | total_timesteps 4287.
Path 406 | total_timesteps 4296.
Path 407 | total_timesteps 4304.
Path 408 | total_timesteps 4314.
Path 409 | total_timesteps 4325.
Path 410 | total_timesteps 4333.
Path 411 | total_timesteps 4347.
Path 412 | total_timesteps 4354.
Path 413 | total_timesteps 4366.
Path 414 | total_timesteps 4375.
Path 415 | total_timesteps 4383.
Path 416 | total_timesteps 4390.
Path 417 | total_timesteps 4397.
Path 418 | total_timesteps 4409.
Path 419 | total_timesteps 4419.
Path 420 | total_timesteps 4427.
Path 421 | total_timesteps 4438.
Path 422 | total_timesteps 4445.
Path 423 | total_timesteps 4454.
Path 424 | total_timesteps 4461.
Path 425 | total_timesteps 4478.
Path 426 | total_timesteps 4490.
Path 427 | total_timesteps 4501.
Path 428 | total_timesteps 4513.
Path 429 | total_timesteps 4521.
Path 430 | total_timesteps 4531.
Path 431 | total_timesteps 4540.
Path 432 | total_timesteps 4547.
Path 433 | total_timesteps 4558.
Path 434 | total_timesteps 4569.
Path 435 | total_timesteps 4578.
Path 436 | total_timesteps 4594.
Path 437 | total_timesteps 4616.
Path 438 | total_timesteps 4627.
Path 439 | total_timesteps 4636.
Path 440 | total_timesteps 4645.
Path 441 | total_timesteps 4655.
Path 442 | total_timesteps 4666.
Path 443 | total_timesteps 4676.
Path 444 | total_timesteps 4683.
Path 445 | total_timesteps 4695.
Path 446 | total_timesteps 4709.
Path 447 | total_timesteps 4718.
Path 448 | total_timesteps 4729.
Path 449 | total_timesteps 4741.
Path 450 | total_timesteps 4753.
Path 451 | total_timesteps 4762.
Path 452 | total_timesteps 4772.
Path 453 | total_timesteps 4784.
Path 454 | total_timesteps 4794.
Path 455 | total_timesteps 4810.
Path 456 | total_timesteps 4819.
Path 457 | total_timesteps 4827.
Path 458 | total_timesteps 4834.
Path 459 | total_timesteps 4846.
Path 460 | total_timesteps 4856.
Path 461 | total_timesteps 4864.
Path 462 | total_timesteps 4871.
Path 463 | total_timesteps 4879.
Path 464 | total_timesteps 4886.
Path 465 | total_timesteps 4896.
Path 466 | total_timesteps 4905.
Path 467 | total_timesteps 4914.
Path 468 | total_timesteps 4931.
Path 469 | total_timesteps 4937.
Path 470 | total_timesteps 4945.
Path 471 | total_timesteps 4955.
Path 472 | total_timesteps 4965.
Path 473 | total_timesteps 4981.
Path 474 | total_timesteps 4992.
Path 475 | total_timesteps 5006.
Path 476 | total_timesteps 5015.
Path 477 | total_timesteps 5025.
Path 478 | total_timesteps 5034.
Path 479 | total_timesteps 5047.
Path 480 | total_timesteps 5057.
Path 481 | total_timesteps 5066.
Path 482 | total_timesteps 5077.
Path 483 | total_timesteps 5088.
Path 484 | total_timesteps 5097.
Path 485 | total_timesteps 5105.
Path 486 | total_timesteps 5113.
Path 487 | total_timesteps 5123.
Path 488 | total_timesteps 5130.
Path 489 | total_timesteps 5141.
Path 490 | total_timesteps 5169.
Path 491 | total_timesteps 5178.
Path 492 | total_timesteps 5193.
Path 493 | total_timesteps 5206.
Path 494 | total_timesteps 5214.
Path 495 | total_timesteps 5227.
Path 496 | total_timesteps 5235.
Path 497 | total_timesteps 5244.
Path 498 | total_timesteps 5254.
Path 499 | total_timesteps 5270.
Path 500 | total_timesteps 5278.
Path 501 | total_timesteps 5290.
Path 502 | total_timesteps 5304.
Path 503 | total_timesteps 5312.
Path 504 | total_timesteps 5323.
Path 505 | total_timesteps 5331.
Path 506 | total_timesteps 5344.
Path 507 | total_timesteps 5354.
Path 508 | total_timesteps 5363.
Path 509 | total_timesteps 5370.
Path 510 | total_timesteps 5379.
Path 511 | total_timesteps 5388.
Path 512 | total_timesteps 5396.
Path 513 | total_timesteps 5410.
Path 514 | total_timesteps 5424.
Path 515 | total_timesteps 5435.
Path 516 | total_timesteps 5448.
Path 517 | total_timesteps 5457.
Path 518 | total_timesteps 5467.
Path 519 | total_timesteps 5477.
Path 520 | total_timesteps 5489.
Path 521 | total_timesteps 5508.
Path 522 | total_timesteps 5516.
Path 523 | total_timesteps 5529.
Path 524 | total_timesteps 5538.
Path 525 | total_timesteps 5547.
Path 526 | total_timesteps 5562.
Path 527 | total_timesteps 5571.
Path 528 | total_timesteps 5581.
Path 529 | total_timesteps 5596.
Path 530 | total_timesteps 5605.
Path 531 | total_timesteps 5617.
Path 532 | total_timesteps 5626.
Path 533 | total_timesteps 5634.
Path 534 | total_timesteps 5653.
Path 535 | total_timesteps 5667.
Path 536 | total_timesteps 5677.
Path 537 | total_timesteps 5686.
Path 538 | total_timesteps 5694.
Path 539 | total_timesteps 5713.
Path 540 | total_timesteps 5723.
Path 541 | total_timesteps 5738.
Path 542 | total_timesteps 5746.
Path 543 | total_timesteps 5759.
Path 544 | total_timesteps 5769.
Path 545 | total_timesteps 5781.
Path 546 | total_timesteps 5798.
Path 547 | total_timesteps 5810.
Path 548 | total_timesteps 5828.
Path 549 | total_timesteps 5837.
Path 550 | total_timesteps 5847.
Path 551 | total_timesteps 5855.
Path 552 | total_timesteps 5869.
Path 553 | total_timesteps 5879.
Path 554 | total_timesteps 5889.
Path 555 | total_timesteps 5900.
Path 556 | total_timesteps 5911.
Path 557 | total_timesteps 5920.
Path 558 | total_timesteps 5930.
Path 559 | total_timesteps 5940.
Path 560 | total_timesteps 5963.
Path 561 | total_timesteps 5978.
Path 562 | total_timesteps 5986.
Path 563 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.52    |
| Iteration     | 11       |
| MaximumReturn | 1.09     |
| MinimumReturn | -21.8    |
| TotalSamples  | 52042    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.3107544779777527
Validation loss = 0.31657880544662476
Validation loss = 0.31878888607025146
Validation loss = 0.3157835304737091
Validation loss = 0.3284752070903778
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 13.
Path 2 | total_timesteps 23.
Path 3 | total_timesteps 39.
Path 4 | total_timesteps 49.
Path 5 | total_timesteps 60.
Path 6 | total_timesteps 78.
Path 7 | total_timesteps 97.
Path 8 | total_timesteps 110.
Path 9 | total_timesteps 132.
Path 10 | total_timesteps 142.
Path 11 | total_timesteps 153.
Path 12 | total_timesteps 161.
Path 13 | total_timesteps 171.
Path 14 | total_timesteps 196.
Path 15 | total_timesteps 206.
Path 16 | total_timesteps 217.
Path 17 | total_timesteps 235.
Path 18 | total_timesteps 244.
Path 19 | total_timesteps 254.
Path 20 | total_timesteps 262.
Path 21 | total_timesteps 271.
Path 22 | total_timesteps 283.
Path 23 | total_timesteps 294.
Path 24 | total_timesteps 309.
Path 25 | total_timesteps 319.
Path 26 | total_timesteps 340.
Path 27 | total_timesteps 350.
Path 28 | total_timesteps 361.
Path 29 | total_timesteps 371.
Path 30 | total_timesteps 387.
Path 31 | total_timesteps 396.
Path 32 | total_timesteps 406.
Path 33 | total_timesteps 420.
Path 34 | total_timesteps 430.
Path 35 | total_timesteps 442.
Path 36 | total_timesteps 450.
Path 37 | total_timesteps 460.
Path 38 | total_timesteps 474.
Path 39 | total_timesteps 487.
Path 40 | total_timesteps 506.
Path 41 | total_timesteps 520.
Path 42 | total_timesteps 535.
Path 43 | total_timesteps 549.
Path 44 | total_timesteps 557.
Path 45 | total_timesteps 574.
Path 46 | total_timesteps 586.
Path 47 | total_timesteps 593.
Path 48 | total_timesteps 607.
Path 49 | total_timesteps 620.
Path 50 | total_timesteps 630.
Path 51 | total_timesteps 640.
Path 52 | total_timesteps 657.
Path 53 | total_timesteps 669.
Path 54 | total_timesteps 679.
Path 55 | total_timesteps 691.
Path 56 | total_timesteps 707.
Path 57 | total_timesteps 723.
Path 58 | total_timesteps 734.
Path 59 | total_timesteps 760.
Path 60 | total_timesteps 772.
Path 61 | total_timesteps 789.
Path 62 | total_timesteps 798.
Path 63 | total_timesteps 814.
Path 64 | total_timesteps 824.
Path 65 | total_timesteps 832.
Path 66 | total_timesteps 841.
Path 67 | total_timesteps 853.
Path 68 | total_timesteps 862.
Path 69 | total_timesteps 895.
Path 70 | total_timesteps 907.
Path 71 | total_timesteps 923.
Path 72 | total_timesteps 938.
Path 73 | total_timesteps 951.
Path 74 | total_timesteps 959.
Path 75 | total_timesteps 967.
Path 76 | total_timesteps 977.
Path 77 | total_timesteps 990.
Path 78 | total_timesteps 1001.
Path 79 | total_timesteps 1010.
Path 80 | total_timesteps 1017.
Path 81 | total_timesteps 1028.
Path 82 | total_timesteps 1037.
Path 83 | total_timesteps 1046.
Path 84 | total_timesteps 1056.
Path 85 | total_timesteps 1069.
Path 86 | total_timesteps 1079.
Path 87 | total_timesteps 1087.
Path 88 | total_timesteps 1093.
Path 89 | total_timesteps 1113.
Path 90 | total_timesteps 1127.
Path 91 | total_timesteps 1139.
Path 92 | total_timesteps 1149.
Path 93 | total_timesteps 1163.
Path 94 | total_timesteps 1176.
Path 95 | total_timesteps 1189.
Path 96 | total_timesteps 1198.
Path 97 | total_timesteps 1214.
Path 98 | total_timesteps 1221.
Path 99 | total_timesteps 1234.
Path 100 | total_timesteps 1245.
Path 101 | total_timesteps 1258.
Path 102 | total_timesteps 1272.
Path 103 | total_timesteps 1285.
Path 104 | total_timesteps 1297.
Path 105 | total_timesteps 1310.
Path 106 | total_timesteps 1320.
Path 107 | total_timesteps 1332.
Path 108 | total_timesteps 1346.
Path 109 | total_timesteps 1366.
Path 110 | total_timesteps 1376.
Path 111 | total_timesteps 1389.
Path 112 | total_timesteps 1401.
Path 113 | total_timesteps 1415.
Path 114 | total_timesteps 1428.
Path 115 | total_timesteps 1437.
Path 116 | total_timesteps 1448.
Path 117 | total_timesteps 1468.
Path 118 | total_timesteps 1481.
Path 119 | total_timesteps 1510.
Path 120 | total_timesteps 1530.
Path 121 | total_timesteps 1545.
Path 122 | total_timesteps 1552.
Path 123 | total_timesteps 1563.
Path 124 | total_timesteps 1570.
Path 125 | total_timesteps 1581.
Path 126 | total_timesteps 1587.
Path 127 | total_timesteps 1597.
Path 128 | total_timesteps 1608.
Path 129 | total_timesteps 1618.
Path 130 | total_timesteps 1629.
Path 131 | total_timesteps 1637.
Path 132 | total_timesteps 1648.
Path 133 | total_timesteps 1657.
Path 134 | total_timesteps 1670.
Path 135 | total_timesteps 1678.
Path 136 | total_timesteps 1692.
Path 137 | total_timesteps 1707.
Path 138 | total_timesteps 1719.
Path 139 | total_timesteps 1728.
Path 140 | total_timesteps 1744.
Path 141 | total_timesteps 1754.
Path 142 | total_timesteps 1763.
Path 143 | total_timesteps 1776.
Path 144 | total_timesteps 1789.
Path 145 | total_timesteps 1807.
Path 146 | total_timesteps 1818.
Path 147 | total_timesteps 1831.
Path 148 | total_timesteps 1840.
Path 149 | total_timesteps 1851.
Path 150 | total_timesteps 1859.
Path 151 | total_timesteps 1867.
Path 152 | total_timesteps 1874.
Path 153 | total_timesteps 1883.
Path 154 | total_timesteps 1898.
Path 155 | total_timesteps 1916.
Path 156 | total_timesteps 1925.
Path 157 | total_timesteps 1938.
Path 158 | total_timesteps 1947.
Path 159 | total_timesteps 1956.
Path 160 | total_timesteps 1965.
Path 161 | total_timesteps 1979.
Path 162 | total_timesteps 2002.
Path 163 | total_timesteps 2011.
Path 164 | total_timesteps 2018.
Path 165 | total_timesteps 2034.
Path 166 | total_timesteps 2046.
Path 167 | total_timesteps 2055.
Path 168 | total_timesteps 2063.
Path 169 | total_timesteps 2069.
Path 170 | total_timesteps 2094.
Path 171 | total_timesteps 2109.
Path 172 | total_timesteps 2128.
Path 173 | total_timesteps 2147.
Path 174 | total_timesteps 2159.
Path 175 | total_timesteps 2172.
Path 176 | total_timesteps 2193.
Path 177 | total_timesteps 2205.
Path 178 | total_timesteps 2220.
Path 179 | total_timesteps 2229.
Path 180 | total_timesteps 2247.
Path 181 | total_timesteps 2266.
Path 182 | total_timesteps 2277.
Path 183 | total_timesteps 2292.
Path 184 | total_timesteps 2318.
Path 185 | total_timesteps 2325.
Path 186 | total_timesteps 2349.
Path 187 | total_timesteps 2358.
Path 188 | total_timesteps 2365.
Path 189 | total_timesteps 2379.
Path 190 | total_timesteps 2392.
Path 191 | total_timesteps 2413.
Path 192 | total_timesteps 2439.
Path 193 | total_timesteps 2449.
Path 194 | total_timesteps 2468.
Path 195 | total_timesteps 2484.
Path 196 | total_timesteps 2498.
Path 197 | total_timesteps 2513.
Path 198 | total_timesteps 2540.
Path 199 | total_timesteps 2553.
Path 200 | total_timesteps 2566.
Path 201 | total_timesteps 2578.
Path 202 | total_timesteps 2595.
Path 203 | total_timesteps 2605.
Path 204 | total_timesteps 2613.
Path 205 | total_timesteps 2627.
Path 206 | total_timesteps 2639.
Path 207 | total_timesteps 2647.
Path 208 | total_timesteps 2660.
Path 209 | total_timesteps 2675.
Path 210 | total_timesteps 2690.
Path 211 | total_timesteps 2705.
Path 212 | total_timesteps 2729.
Path 213 | total_timesteps 2737.
Path 214 | total_timesteps 2755.
Path 215 | total_timesteps 2764.
Path 216 | total_timesteps 2773.
Path 217 | total_timesteps 2785.
Path 218 | total_timesteps 2800.
Path 219 | total_timesteps 2812.
Path 220 | total_timesteps 2834.
Path 221 | total_timesteps 2846.
Path 222 | total_timesteps 2855.
Path 223 | total_timesteps 2863.
Path 224 | total_timesteps 2876.
Path 225 | total_timesteps 2890.
Path 226 | total_timesteps 2899.
Path 227 | total_timesteps 2908.
Path 228 | total_timesteps 2917.
Path 229 | total_timesteps 2926.
Path 230 | total_timesteps 2938.
Path 231 | total_timesteps 2949.
Path 232 | total_timesteps 2964.
Path 233 | total_timesteps 2978.
Path 234 | total_timesteps 2996.
Path 235 | total_timesteps 3006.
Path 236 | total_timesteps 3016.
Path 237 | total_timesteps 3027.
Path 238 | total_timesteps 3036.
Path 239 | total_timesteps 3044.
Path 240 | total_timesteps 3052.
Path 241 | total_timesteps 3063.
Path 242 | total_timesteps 3072.
Path 243 | total_timesteps 3093.
Path 244 | total_timesteps 3106.
Path 245 | total_timesteps 3120.
Path 246 | total_timesteps 3135.
Path 247 | total_timesteps 3146.
Path 248 | total_timesteps 3155.
Path 249 | total_timesteps 3174.
Path 250 | total_timesteps 3182.
Path 251 | total_timesteps 3190.
Path 252 | total_timesteps 3219.
Path 253 | total_timesteps 3231.
Path 254 | total_timesteps 3241.
Path 255 | total_timesteps 3269.
Path 256 | total_timesteps 3281.
Path 257 | total_timesteps 3294.
Path 258 | total_timesteps 3327.
Path 259 | total_timesteps 3338.
Path 260 | total_timesteps 3352.
Path 261 | total_timesteps 3364.
Path 262 | total_timesteps 3376.
Path 263 | total_timesteps 3384.
Path 264 | total_timesteps 3396.
Path 265 | total_timesteps 3431.
Path 266 | total_timesteps 3440.
Path 267 | total_timesteps 3450.
Path 268 | total_timesteps 3460.
Path 269 | total_timesteps 3468.
Path 270 | total_timesteps 3479.
Path 271 | total_timesteps 3490.
Path 272 | total_timesteps 3499.
Path 273 | total_timesteps 3507.
Path 274 | total_timesteps 3537.
Path 275 | total_timesteps 3554.
Path 276 | total_timesteps 3566.
Path 277 | total_timesteps 3581.
Path 278 | total_timesteps 3589.
Path 279 | total_timesteps 3601.
Path 280 | total_timesteps 3613.
Path 281 | total_timesteps 3623.
Path 282 | total_timesteps 3634.
Path 283 | total_timesteps 3645.
Path 284 | total_timesteps 3653.
Path 285 | total_timesteps 3662.
Path 286 | total_timesteps 3692.
Path 287 | total_timesteps 3701.
Path 288 | total_timesteps 3718.
Path 289 | total_timesteps 3731.
Path 290 | total_timesteps 3742.
Path 291 | total_timesteps 3750.
Path 292 | total_timesteps 3763.
Path 293 | total_timesteps 3779.
Path 294 | total_timesteps 3788.
Path 295 | total_timesteps 3796.
Path 296 | total_timesteps 3807.
Path 297 | total_timesteps 3822.
Path 298 | total_timesteps 3837.
Path 299 | total_timesteps 3851.
Path 300 | total_timesteps 3862.
Path 301 | total_timesteps 3878.
Path 302 | total_timesteps 3888.
Path 303 | total_timesteps 3898.
Path 304 | total_timesteps 3909.
Path 305 | total_timesteps 3923.
Path 306 | total_timesteps 3934.
Path 307 | total_timesteps 3950.
Path 308 | total_timesteps 3959.
Path 309 | total_timesteps 3974.
Path 310 | total_timesteps 3982.
Path 311 | total_timesteps 3991.
Path 312 | total_timesteps 4003.
Path 313 | total_timesteps 4014.
Path 314 | total_timesteps 4033.
Path 315 | total_timesteps 4056.
Path 316 | total_timesteps 4065.
Path 317 | total_timesteps 4074.
Path 318 | total_timesteps 4098.
Path 319 | total_timesteps 4114.
Path 320 | total_timesteps 4122.
Path 321 | total_timesteps 4135.
Path 322 | total_timesteps 4148.
Path 323 | total_timesteps 4158.
Path 324 | total_timesteps 4176.
Path 325 | total_timesteps 4184.
Path 326 | total_timesteps 4194.
Path 327 | total_timesteps 4216.
Path 328 | total_timesteps 4229.
Path 329 | total_timesteps 4238.
Path 330 | total_timesteps 4248.
Path 331 | total_timesteps 4258.
Path 332 | total_timesteps 4270.
Path 333 | total_timesteps 4279.
Path 334 | total_timesteps 4298.
Path 335 | total_timesteps 4312.
Path 336 | total_timesteps 4323.
Path 337 | total_timesteps 4336.
Path 338 | total_timesteps 4348.
Path 339 | total_timesteps 4357.
Path 340 | total_timesteps 4367.
Path 341 | total_timesteps 4376.
Path 342 | total_timesteps 4392.
Path 343 | total_timesteps 4404.
Path 344 | total_timesteps 4413.
Path 345 | total_timesteps 4422.
Path 346 | total_timesteps 4431.
Path 347 | total_timesteps 4449.
Path 348 | total_timesteps 4459.
Path 349 | total_timesteps 4468.
Path 350 | total_timesteps 4476.
Path 351 | total_timesteps 4490.
Path 352 | total_timesteps 4501.
Path 353 | total_timesteps 4509.
Path 354 | total_timesteps 4532.
Path 355 | total_timesteps 4544.
Path 356 | total_timesteps 4559.
Path 357 | total_timesteps 4571.
Path 358 | total_timesteps 4579.
Path 359 | total_timesteps 4588.
Path 360 | total_timesteps 4598.
Path 361 | total_timesteps 4610.
Path 362 | total_timesteps 4620.
Path 363 | total_timesteps 4635.
Path 364 | total_timesteps 4658.
Path 365 | total_timesteps 4666.
Path 366 | total_timesteps 4685.
Path 367 | total_timesteps 4694.
Path 368 | total_timesteps 4731.
Path 369 | total_timesteps 4743.
Path 370 | total_timesteps 4753.
Path 371 | total_timesteps 4760.
Path 372 | total_timesteps 4773.
Path 373 | total_timesteps 4783.
Path 374 | total_timesteps 4793.
Path 375 | total_timesteps 4804.
Path 376 | total_timesteps 4815.
Path 377 | total_timesteps 4827.
Path 378 | total_timesteps 4839.
Path 379 | total_timesteps 4857.
Path 380 | total_timesteps 4870.
Path 381 | total_timesteps 4879.
Path 382 | total_timesteps 4890.
Path 383 | total_timesteps 4900.
Path 384 | total_timesteps 4911.
Path 385 | total_timesteps 4920.
Path 386 | total_timesteps 4930.
Path 387 | total_timesteps 4941.
Path 388 | total_timesteps 4952.
Path 389 | total_timesteps 4960.
Path 390 | total_timesteps 4970.
Path 391 | total_timesteps 4983.
Path 392 | total_timesteps 4991.
Path 393 | total_timesteps 5001.
Path 394 | total_timesteps 5012.
Path 395 | total_timesteps 5023.
Path 396 | total_timesteps 5030.
Path 397 | total_timesteps 5045.
Path 398 | total_timesteps 5054.
Path 399 | total_timesteps 5065.
Path 400 | total_timesteps 5073.
Path 401 | total_timesteps 5086.
Path 402 | total_timesteps 5093.
Path 403 | total_timesteps 5102.
Path 404 | total_timesteps 5112.
Path 405 | total_timesteps 5121.
Path 406 | total_timesteps 5133.
Path 407 | total_timesteps 5145.
Path 408 | total_timesteps 5168.
Path 409 | total_timesteps 5177.
Path 410 | total_timesteps 5185.
Path 411 | total_timesteps 5197.
Path 412 | total_timesteps 5221.
Path 413 | total_timesteps 5239.
Path 414 | total_timesteps 5249.
Path 415 | total_timesteps 5258.
Path 416 | total_timesteps 5268.
Path 417 | total_timesteps 5281.
Path 418 | total_timesteps 5291.
Path 419 | total_timesteps 5302.
Path 420 | total_timesteps 5310.
Path 421 | total_timesteps 5318.
Path 422 | total_timesteps 5336.
Path 423 | total_timesteps 5348.
Path 424 | total_timesteps 5360.
Path 425 | total_timesteps 5369.
Path 426 | total_timesteps 5377.
Path 427 | total_timesteps 5388.
Path 428 | total_timesteps 5396.
Path 429 | total_timesteps 5404.
Path 430 | total_timesteps 5414.
Path 431 | total_timesteps 5422.
Path 432 | total_timesteps 5437.
Path 433 | total_timesteps 5457.
Path 434 | total_timesteps 5474.
Path 435 | total_timesteps 5493.
Path 436 | total_timesteps 5507.
Path 437 | total_timesteps 5516.
Path 438 | total_timesteps 5530.
Path 439 | total_timesteps 5548.
Path 440 | total_timesteps 5567.
Path 441 | total_timesteps 5587.
Path 442 | total_timesteps 5596.
Path 443 | total_timesteps 5611.
Path 444 | total_timesteps 5620.
Path 445 | total_timesteps 5632.
Path 446 | total_timesteps 5639.
Path 447 | total_timesteps 5659.
Path 448 | total_timesteps 5668.
Path 449 | total_timesteps 5677.
Path 450 | total_timesteps 5688.
Path 451 | total_timesteps 5699.
Path 452 | total_timesteps 5711.
Path 453 | total_timesteps 5727.
Path 454 | total_timesteps 5739.
Path 455 | total_timesteps 5748.
Path 456 | total_timesteps 5758.
Path 457 | total_timesteps 5768.
Path 458 | total_timesteps 5783.
Path 459 | total_timesteps 5792.
Path 460 | total_timesteps 5802.
Path 461 | total_timesteps 5809.
Path 462 | total_timesteps 5822.
Path 463 | total_timesteps 5834.
Path 464 | total_timesteps 5859.
Path 465 | total_timesteps 5889.
Path 466 | total_timesteps 5896.
Path 467 | total_timesteps 5914.
Path 468 | total_timesteps 5930.
Path 469 | total_timesteps 5938.
Path 470 | total_timesteps 5948.
Path 471 | total_timesteps 5963.
Path 472 | total_timesteps 5973.
Path 473 | total_timesteps 5986.
Path 474 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.21    |
| Iteration     | 12       |
| MaximumReturn | 2.61     |
| MinimumReturn | -20.9    |
| TotalSamples  | 56050    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.30733922123908997
Validation loss = 0.31283825635910034
Validation loss = 0.3178248107433319
Validation loss = 0.32864704728126526
Validation loss = 0.3195907175540924
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 38.
Path 4 | total_timesteps 49.
Path 5 | total_timesteps 64.
Path 6 | total_timesteps 73.
Path 7 | total_timesteps 94.
Path 8 | total_timesteps 115.
Path 9 | total_timesteps 125.
Path 10 | total_timesteps 135.
Path 11 | total_timesteps 143.
Path 12 | total_timesteps 153.
Path 13 | total_timesteps 166.
Path 14 | total_timesteps 176.
Path 15 | total_timesteps 185.
Path 16 | total_timesteps 199.
Path 17 | total_timesteps 216.
Path 18 | total_timesteps 230.
Path 19 | total_timesteps 238.
Path 20 | total_timesteps 252.
Path 21 | total_timesteps 262.
Path 22 | total_timesteps 279.
Path 23 | total_timesteps 295.
Path 24 | total_timesteps 306.
Path 25 | total_timesteps 330.
Path 26 | total_timesteps 339.
Path 27 | total_timesteps 352.
Path 28 | total_timesteps 360.
Path 29 | total_timesteps 379.
Path 30 | total_timesteps 388.
Path 31 | total_timesteps 396.
Path 32 | total_timesteps 408.
Path 33 | total_timesteps 420.
Path 34 | total_timesteps 433.
Path 35 | total_timesteps 441.
Path 36 | total_timesteps 451.
Path 37 | total_timesteps 464.
Path 38 | total_timesteps 476.
Path 39 | total_timesteps 489.
Path 40 | total_timesteps 501.
Path 41 | total_timesteps 514.
Path 42 | total_timesteps 533.
Path 43 | total_timesteps 546.
Path 44 | total_timesteps 554.
Path 45 | total_timesteps 567.
Path 46 | total_timesteps 579.
Path 47 | total_timesteps 602.
Path 48 | total_timesteps 611.
Path 49 | total_timesteps 625.
Path 50 | total_timesteps 633.
Path 51 | total_timesteps 647.
Path 52 | total_timesteps 663.
Path 53 | total_timesteps 675.
Path 54 | total_timesteps 685.
Path 55 | total_timesteps 704.
Path 56 | total_timesteps 715.
Path 57 | total_timesteps 730.
Path 58 | total_timesteps 739.
Path 59 | total_timesteps 750.
Path 60 | total_timesteps 760.
Path 61 | total_timesteps 770.
Path 62 | total_timesteps 777.
Path 63 | total_timesteps 787.
Path 64 | total_timesteps 798.
Path 65 | total_timesteps 810.
Path 66 | total_timesteps 817.
Path 67 | total_timesteps 829.
Path 68 | total_timesteps 848.
Path 69 | total_timesteps 855.
Path 70 | total_timesteps 864.
Path 71 | total_timesteps 873.
Path 72 | total_timesteps 884.
Path 73 | total_timesteps 897.
Path 74 | total_timesteps 906.
Path 75 | total_timesteps 914.
Path 76 | total_timesteps 924.
Path 77 | total_timesteps 939.
Path 78 | total_timesteps 951.
Path 79 | total_timesteps 967.
Path 80 | total_timesteps 979.
Path 81 | total_timesteps 987.
Path 82 | total_timesteps 998.
Path 83 | total_timesteps 1005.
Path 84 | total_timesteps 1013.
Path 85 | total_timesteps 1030.
Path 86 | total_timesteps 1040.
Path 87 | total_timesteps 1051.
Path 88 | total_timesteps 1064.
Path 89 | total_timesteps 1080.
Path 90 | total_timesteps 1095.
Path 91 | total_timesteps 1110.
Path 92 | total_timesteps 1118.
Path 93 | total_timesteps 1130.
Path 94 | total_timesteps 1145.
Path 95 | total_timesteps 1159.
Path 96 | total_timesteps 1169.
Path 97 | total_timesteps 1178.
Path 98 | total_timesteps 1189.
Path 99 | total_timesteps 1200.
Path 100 | total_timesteps 1211.
Path 101 | total_timesteps 1222.
Path 102 | total_timesteps 1230.
Path 103 | total_timesteps 1240.
Path 104 | total_timesteps 1256.
Path 105 | total_timesteps 1266.
Path 106 | total_timesteps 1273.
Path 107 | total_timesteps 1292.
Path 108 | total_timesteps 1301.
Path 109 | total_timesteps 1312.
Path 110 | total_timesteps 1320.
Path 111 | total_timesteps 1339.
Path 112 | total_timesteps 1352.
Path 113 | total_timesteps 1380.
Path 114 | total_timesteps 1393.
Path 115 | total_timesteps 1415.
Path 116 | total_timesteps 1425.
Path 117 | total_timesteps 1442.
Path 118 | total_timesteps 1470.
Path 119 | total_timesteps 1497.
Path 120 | total_timesteps 1515.
Path 121 | total_timesteps 1527.
Path 122 | total_timesteps 1539.
Path 123 | total_timesteps 1550.
Path 124 | total_timesteps 1563.
Path 125 | total_timesteps 1578.
Path 126 | total_timesteps 1588.
Path 127 | total_timesteps 1596.
Path 128 | total_timesteps 1607.
Path 129 | total_timesteps 1633.
Path 130 | total_timesteps 1643.
Path 131 | total_timesteps 1656.
Path 132 | total_timesteps 1664.
Path 133 | total_timesteps 1678.
Path 134 | total_timesteps 1688.
Path 135 | total_timesteps 1703.
Path 136 | total_timesteps 1711.
Path 137 | total_timesteps 1720.
Path 138 | total_timesteps 1728.
Path 139 | total_timesteps 1736.
Path 140 | total_timesteps 1748.
Path 141 | total_timesteps 1761.
Path 142 | total_timesteps 1775.
Path 143 | total_timesteps 1788.
Path 144 | total_timesteps 1803.
Path 145 | total_timesteps 1816.
Path 146 | total_timesteps 1835.
Path 147 | total_timesteps 1851.
Path 148 | total_timesteps 1870.
Path 149 | total_timesteps 1884.
Path 150 | total_timesteps 1900.
Path 151 | total_timesteps 1913.
Path 152 | total_timesteps 1920.
Path 153 | total_timesteps 1942.
Path 154 | total_timesteps 1952.
Path 155 | total_timesteps 1964.
Path 156 | total_timesteps 1976.
Path 157 | total_timesteps 1993.
Path 158 | total_timesteps 2004.
Path 159 | total_timesteps 2020.
Path 160 | total_timesteps 2040.
Path 161 | total_timesteps 2047.
Path 162 | total_timesteps 2058.
Path 163 | total_timesteps 2067.
Path 164 | total_timesteps 2079.
Path 165 | total_timesteps 2092.
Path 166 | total_timesteps 2105.
Path 167 | total_timesteps 2115.
Path 168 | total_timesteps 2126.
Path 169 | total_timesteps 2134.
Path 170 | total_timesteps 2143.
Path 171 | total_timesteps 2155.
Path 172 | total_timesteps 2163.
Path 173 | total_timesteps 2172.
Path 174 | total_timesteps 2187.
Path 175 | total_timesteps 2201.
Path 176 | total_timesteps 2215.
Path 177 | total_timesteps 2222.
Path 178 | total_timesteps 2241.
Path 179 | total_timesteps 2251.
Path 180 | total_timesteps 2262.
Path 181 | total_timesteps 2271.
Path 182 | total_timesteps 2283.
Path 183 | total_timesteps 2293.
Path 184 | total_timesteps 2307.
Path 185 | total_timesteps 2317.
Path 186 | total_timesteps 2331.
Path 187 | total_timesteps 2339.
Path 188 | total_timesteps 2354.
Path 189 | total_timesteps 2365.
Path 190 | total_timesteps 2378.
Path 191 | total_timesteps 2393.
Path 192 | total_timesteps 2404.
Path 193 | total_timesteps 2420.
Path 194 | total_timesteps 2431.
Path 195 | total_timesteps 2442.
Path 196 | total_timesteps 2455.
Path 197 | total_timesteps 2466.
Path 198 | total_timesteps 2478.
Path 199 | total_timesteps 2491.
Path 200 | total_timesteps 2502.
Path 201 | total_timesteps 2517.
Path 202 | total_timesteps 2528.
Path 203 | total_timesteps 2539.
Path 204 | total_timesteps 2555.
Path 205 | total_timesteps 2565.
Path 206 | total_timesteps 2593.
Path 207 | total_timesteps 2601.
Path 208 | total_timesteps 2617.
Path 209 | total_timesteps 2627.
Path 210 | total_timesteps 2635.
Path 211 | total_timesteps 2642.
Path 212 | total_timesteps 2653.
Path 213 | total_timesteps 2669.
Path 214 | total_timesteps 2678.
Path 215 | total_timesteps 2695.
Path 216 | total_timesteps 2707.
Path 217 | total_timesteps 2720.
Path 218 | total_timesteps 2730.
Path 219 | total_timesteps 2739.
Path 220 | total_timesteps 2761.
Path 221 | total_timesteps 2771.
Path 222 | total_timesteps 2786.
Path 223 | total_timesteps 2798.
Path 224 | total_timesteps 2805.
Path 225 | total_timesteps 2814.
Path 226 | total_timesteps 2829.
Path 227 | total_timesteps 2844.
Path 228 | total_timesteps 2862.
Path 229 | total_timesteps 2883.
Path 230 | total_timesteps 2892.
Path 231 | total_timesteps 2917.
Path 232 | total_timesteps 2928.
Path 233 | total_timesteps 2939.
Path 234 | total_timesteps 2952.
Path 235 | total_timesteps 2969.
Path 236 | total_timesteps 2985.
Path 237 | total_timesteps 2997.
Path 238 | total_timesteps 3014.
Path 239 | total_timesteps 3022.
Path 240 | total_timesteps 3039.
Path 241 | total_timesteps 3050.
Path 242 | total_timesteps 3061.
Path 243 | total_timesteps 3076.
Path 244 | total_timesteps 3102.
Path 245 | total_timesteps 3113.
Path 246 | total_timesteps 3122.
Path 247 | total_timesteps 3135.
Path 248 | total_timesteps 3143.
Path 249 | total_timesteps 3156.
Path 250 | total_timesteps 3164.
Path 251 | total_timesteps 3175.
Path 252 | total_timesteps 3188.
Path 253 | total_timesteps 3199.
Path 254 | total_timesteps 3217.
Path 255 | total_timesteps 3228.
Path 256 | total_timesteps 3240.
Path 257 | total_timesteps 3255.
Path 258 | total_timesteps 3269.
Path 259 | total_timesteps 3280.
Path 260 | total_timesteps 3300.
Path 261 | total_timesteps 3319.
Path 262 | total_timesteps 3330.
Path 263 | total_timesteps 3346.
Path 264 | total_timesteps 3358.
Path 265 | total_timesteps 3370.
Path 266 | total_timesteps 3379.
Path 267 | total_timesteps 3387.
Path 268 | total_timesteps 3407.
Path 269 | total_timesteps 3429.
Path 270 | total_timesteps 3444.
Path 271 | total_timesteps 3458.
Path 272 | total_timesteps 3467.
Path 273 | total_timesteps 3475.
Path 274 | total_timesteps 3487.
Path 275 | total_timesteps 3500.
Path 276 | total_timesteps 3510.
Path 277 | total_timesteps 3532.
Path 278 | total_timesteps 3548.
Path 279 | total_timesteps 3566.
Path 280 | total_timesteps 3581.
Path 281 | total_timesteps 3598.
Path 282 | total_timesteps 3608.
Path 283 | total_timesteps 3620.
Path 284 | total_timesteps 3634.
Path 285 | total_timesteps 3652.
Path 286 | total_timesteps 3662.
Path 287 | total_timesteps 3680.
Path 288 | total_timesteps 3693.
Path 289 | total_timesteps 3702.
Path 290 | total_timesteps 3712.
Path 291 | total_timesteps 3727.
Path 292 | total_timesteps 3740.
Path 293 | total_timesteps 3753.
Path 294 | total_timesteps 3764.
Path 295 | total_timesteps 3784.
Path 296 | total_timesteps 3793.
Path 297 | total_timesteps 3801.
Path 298 | total_timesteps 3822.
Path 299 | total_timesteps 3832.
Path 300 | total_timesteps 3856.
Path 301 | total_timesteps 3864.
Path 302 | total_timesteps 3872.
Path 303 | total_timesteps 3892.
Path 304 | total_timesteps 3910.
Path 305 | total_timesteps 3923.
Path 306 | total_timesteps 3936.
Path 307 | total_timesteps 3947.
Path 308 | total_timesteps 3955.
Path 309 | total_timesteps 3968.
Path 310 | total_timesteps 3979.
Path 311 | total_timesteps 3996.
Path 312 | total_timesteps 4006.
Path 313 | total_timesteps 4015.
Path 314 | total_timesteps 4031.
Path 315 | total_timesteps 4044.
Path 316 | total_timesteps 4058.
Path 317 | total_timesteps 4070.
Path 318 | total_timesteps 4078.
Path 319 | total_timesteps 4090.
Path 320 | total_timesteps 4102.
Path 321 | total_timesteps 4113.
Path 322 | total_timesteps 4126.
Path 323 | total_timesteps 4141.
Path 324 | total_timesteps 4155.
Path 325 | total_timesteps 4163.
Path 326 | total_timesteps 4175.
Path 327 | total_timesteps 4182.
Path 328 | total_timesteps 4196.
Path 329 | total_timesteps 4206.
Path 330 | total_timesteps 4216.
Path 331 | total_timesteps 4228.
Path 332 | total_timesteps 4243.
Path 333 | total_timesteps 4251.
Path 334 | total_timesteps 4262.
Path 335 | total_timesteps 4273.
Path 336 | total_timesteps 4280.
Path 337 | total_timesteps 4289.
Path 338 | total_timesteps 4300.
Path 339 | total_timesteps 4319.
Path 340 | total_timesteps 4328.
Path 341 | total_timesteps 4336.
Path 342 | total_timesteps 4347.
Path 343 | total_timesteps 4358.
Path 344 | total_timesteps 4371.
Path 345 | total_timesteps 4382.
Path 346 | total_timesteps 4392.
Path 347 | total_timesteps 4400.
Path 348 | total_timesteps 4419.
Path 349 | total_timesteps 4431.
Path 350 | total_timesteps 4441.
Path 351 | total_timesteps 4464.
Path 352 | total_timesteps 4484.
Path 353 | total_timesteps 4497.
Path 354 | total_timesteps 4512.
Path 355 | total_timesteps 4534.
Path 356 | total_timesteps 4545.
Path 357 | total_timesteps 4557.
Path 358 | total_timesteps 4568.
Path 359 | total_timesteps 4576.
Path 360 | total_timesteps 4586.
Path 361 | total_timesteps 4597.
Path 362 | total_timesteps 4608.
Path 363 | total_timesteps 4636.
Path 364 | total_timesteps 4647.
Path 365 | total_timesteps 4656.
Path 366 | total_timesteps 4673.
Path 367 | total_timesteps 4680.
Path 368 | total_timesteps 4691.
Path 369 | total_timesteps 4705.
Path 370 | total_timesteps 4714.
Path 371 | total_timesteps 4725.
Path 372 | total_timesteps 4738.
Path 373 | total_timesteps 4747.
Path 374 | total_timesteps 4761.
Path 375 | total_timesteps 4768.
Path 376 | total_timesteps 4784.
Path 377 | total_timesteps 4795.
Path 378 | total_timesteps 4804.
Path 379 | total_timesteps 4816.
Path 380 | total_timesteps 4825.
Path 381 | total_timesteps 4839.
Path 382 | total_timesteps 4855.
Path 383 | total_timesteps 4864.
Path 384 | total_timesteps 4871.
Path 385 | total_timesteps 4878.
Path 386 | total_timesteps 4888.
Path 387 | total_timesteps 4908.
Path 388 | total_timesteps 4922.
Path 389 | total_timesteps 4937.
Path 390 | total_timesteps 4944.
Path 391 | total_timesteps 4959.
Path 392 | total_timesteps 4973.
Path 393 | total_timesteps 4983.
Path 394 | total_timesteps 5005.
Path 395 | total_timesteps 5019.
Path 396 | total_timesteps 5030.
Path 397 | total_timesteps 5038.
Path 398 | total_timesteps 5050.
Path 399 | total_timesteps 5068.
Path 400 | total_timesteps 5081.
Path 401 | total_timesteps 5103.
Path 402 | total_timesteps 5116.
Path 403 | total_timesteps 5125.
Path 404 | total_timesteps 5135.
Path 405 | total_timesteps 5143.
Path 406 | total_timesteps 5157.
Path 407 | total_timesteps 5166.
Path 408 | total_timesteps 5183.
Path 409 | total_timesteps 5197.
Path 410 | total_timesteps 5204.
Path 411 | total_timesteps 5214.
Path 412 | total_timesteps 5225.
Path 413 | total_timesteps 5234.
Path 414 | total_timesteps 5244.
Path 415 | total_timesteps 5263.
Path 416 | total_timesteps 5274.
Path 417 | total_timesteps 5284.
Path 418 | total_timesteps 5296.
Path 419 | total_timesteps 5310.
Path 420 | total_timesteps 5319.
Path 421 | total_timesteps 5331.
Path 422 | total_timesteps 5348.
Path 423 | total_timesteps 5356.
Path 424 | total_timesteps 5368.
Path 425 | total_timesteps 5392.
Path 426 | total_timesteps 5403.
Path 427 | total_timesteps 5417.
Path 428 | total_timesteps 5438.
Path 429 | total_timesteps 5451.
Path 430 | total_timesteps 5463.
Path 431 | total_timesteps 5482.
Path 432 | total_timesteps 5497.
Path 433 | total_timesteps 5507.
Path 434 | total_timesteps 5531.
Path 435 | total_timesteps 5544.
Path 436 | total_timesteps 5557.
Path 437 | total_timesteps 5565.
Path 438 | total_timesteps 5591.
Path 439 | total_timesteps 5601.
Path 440 | total_timesteps 5612.
Path 441 | total_timesteps 5623.
Path 442 | total_timesteps 5636.
Path 443 | total_timesteps 5648.
Path 444 | total_timesteps 5659.
Path 445 | total_timesteps 5670.
Path 446 | total_timesteps 5678.
Path 447 | total_timesteps 5697.
Path 448 | total_timesteps 5715.
Path 449 | total_timesteps 5722.
Path 450 | total_timesteps 5739.
Path 451 | total_timesteps 5753.
Path 452 | total_timesteps 5770.
Path 453 | total_timesteps 5777.
Path 454 | total_timesteps 5791.
Path 455 | total_timesteps 5806.
Path 456 | total_timesteps 5815.
Path 457 | total_timesteps 5822.
Path 458 | total_timesteps 5832.
Path 459 | total_timesteps 5849.
Path 460 | total_timesteps 5862.
Path 461 | total_timesteps 5876.
Path 462 | total_timesteps 5886.
Path 463 | total_timesteps 5897.
Path 464 | total_timesteps 5906.
Path 465 | total_timesteps 5922.
Path 466 | total_timesteps 5934.
Path 467 | total_timesteps 5941.
Path 468 | total_timesteps 5961.
Path 469 | total_timesteps 5980.
Path 470 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.29    |
| Iteration     | 13       |
| MaximumReturn | 1.71     |
| MinimumReturn | -23.4    |
| TotalSamples  | 60058    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.3113296926021576
Validation loss = 0.31855854392051697
Validation loss = 0.321641743183136
Validation loss = 0.3237757086753845
Validation loss = 0.31978675723075867
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 162.
Path 2 | total_timesteps 256.
Path 3 | total_timesteps 302.
Path 4 | total_timesteps 397.
Path 5 | total_timesteps 451.
Path 6 | total_timesteps 543.
Path 7 | total_timesteps 658.
Path 8 | total_timesteps 718.
Path 9 | total_timesteps 824.
Path 10 | total_timesteps 882.
Path 11 | total_timesteps 957.
Path 12 | total_timesteps 1032.
Path 13 | total_timesteps 1111.
Path 14 | total_timesteps 1286.
Path 15 | total_timesteps 1383.
Path 16 | total_timesteps 1454.
Path 17 | total_timesteps 1527.
Path 18 | total_timesteps 1611.
Path 19 | total_timesteps 1680.
Path 20 | total_timesteps 1754.
Path 21 | total_timesteps 1862.
Path 22 | total_timesteps 1925.
Path 23 | total_timesteps 2022.
Path 24 | total_timesteps 2107.
Path 25 | total_timesteps 2161.
Path 26 | total_timesteps 2203.
Path 27 | total_timesteps 2297.
Path 28 | total_timesteps 2381.
Path 29 | total_timesteps 2458.
Path 30 | total_timesteps 2486.
Path 31 | total_timesteps 2580.
Path 32 | total_timesteps 2638.
Path 33 | total_timesteps 2731.
Path 34 | total_timesteps 2803.
Path 35 | total_timesteps 2951.
Path 36 | total_timesteps 3036.
Path 37 | total_timesteps 3133.
Path 38 | total_timesteps 3212.
Path 39 | total_timesteps 3280.
Path 40 | total_timesteps 3344.
Path 41 | total_timesteps 3418.
Path 42 | total_timesteps 3528.
Path 43 | total_timesteps 3634.
Path 44 | total_timesteps 3688.
Path 45 | total_timesteps 3761.
Path 46 | total_timesteps 3833.
Path 47 | total_timesteps 3892.
Path 48 | total_timesteps 3982.
Path 49 | total_timesteps 4052.
Path 50 | total_timesteps 4109.
Path 51 | total_timesteps 4177.
Path 52 | total_timesteps 4216.
Path 53 | total_timesteps 4253.
Path 54 | total_timesteps 4330.
Path 55 | total_timesteps 4413.
Path 56 | total_timesteps 4504.
Path 57 | total_timesteps 4565.
Path 58 | total_timesteps 4657.
Path 59 | total_timesteps 4780.
Path 60 | total_timesteps 4838.
Path 61 | total_timesteps 4919.
Path 62 | total_timesteps 4969.
Path 63 | total_timesteps 5054.
Path 64 | total_timesteps 5152.
Path 65 | total_timesteps 5269.
Path 66 | total_timesteps 5359.
Path 67 | total_timesteps 5452.
Path 68 | total_timesteps 5508.
Path 69 | total_timesteps 5590.
Path 70 | total_timesteps 5681.
Path 71 | total_timesteps 5765.
Path 72 | total_timesteps 5874.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -47.3    |
| Iteration     | 14       |
| MaximumReturn | 9.43     |
| MinimumReturn | -96.7    |
| TotalSamples  | 64093    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.31948357820510864
Validation loss = 0.3245639204978943
Validation loss = 0.32607972621917725
Validation loss = 0.32930099964141846
Validation loss = 0.3312026858329773
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 79.
Path 2 | total_timesteps 137.
Path 3 | total_timesteps 208.
Path 4 | total_timesteps 254.
Path 5 | total_timesteps 335.
Path 6 | total_timesteps 406.
Path 7 | total_timesteps 525.
Path 8 | total_timesteps 611.
Path 9 | total_timesteps 677.
Path 10 | total_timesteps 783.
Path 11 | total_timesteps 829.
Path 12 | total_timesteps 901.
Path 13 | total_timesteps 991.
Path 14 | total_timesteps 1091.
Path 15 | total_timesteps 1154.
Path 16 | total_timesteps 1236.
Path 17 | total_timesteps 1330.
Path 18 | total_timesteps 1420.
Path 19 | total_timesteps 1521.
Path 20 | total_timesteps 1607.
Path 21 | total_timesteps 1680.
Path 22 | total_timesteps 1854.
Path 23 | total_timesteps 1964.
Path 24 | total_timesteps 2074.
Path 25 | total_timesteps 2132.
Path 26 | total_timesteps 2193.
Path 27 | total_timesteps 2272.
Path 28 | total_timesteps 2356.
Path 29 | total_timesteps 2417.
Path 30 | total_timesteps 2477.
Path 31 | total_timesteps 2574.
Path 32 | total_timesteps 2617.
Path 33 | total_timesteps 2663.
Path 34 | total_timesteps 2727.
Path 35 | total_timesteps 2800.
Path 36 | total_timesteps 2875.
Path 37 | total_timesteps 2931.
Path 38 | total_timesteps 3010.
Path 39 | total_timesteps 3067.
Path 40 | total_timesteps 3119.
Path 41 | total_timesteps 3165.
Path 42 | total_timesteps 3249.
Path 43 | total_timesteps 3354.
Path 44 | total_timesteps 3439.
Path 45 | total_timesteps 3509.
Path 46 | total_timesteps 3565.
Path 47 | total_timesteps 3717.
Path 48 | total_timesteps 3794.
Path 49 | total_timesteps 3890.
Path 50 | total_timesteps 3963.
Path 51 | total_timesteps 4053.
Path 52 | total_timesteps 4136.
Path 53 | total_timesteps 4175.
Path 54 | total_timesteps 4228.
Path 55 | total_timesteps 4318.
Path 56 | total_timesteps 4394.
Path 57 | total_timesteps 4500.
Path 58 | total_timesteps 4557.
Path 59 | total_timesteps 4627.
Path 60 | total_timesteps 4688.
Path 61 | total_timesteps 4775.
Path 62 | total_timesteps 4862.
Path 63 | total_timesteps 4915.
Path 64 | total_timesteps 4989.
Path 65 | total_timesteps 5042.
Path 66 | total_timesteps 5119.
Path 67 | total_timesteps 5184.
Path 68 | total_timesteps 5213.
Path 69 | total_timesteps 5284.
Path 70 | total_timesteps 5396.
Path 71 | total_timesteps 5435.
Path 72 | total_timesteps 5536.
Path 73 | total_timesteps 5635.
Path 74 | total_timesteps 5737.
Path 75 | total_timesteps 5799.
Path 76 | total_timesteps 5866.
Path 77 | total_timesteps 5984.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -43.4    |
| Iteration     | 15       |
| MaximumReturn | 159      |
| MinimumReturn | -127     |
| TotalSamples  | 68177    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.32140347361564636
Validation loss = 0.3274167776107788
Validation loss = 0.3276132643222809
Validation loss = 0.33294036984443665
Validation loss = 0.3348267078399658
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 66.
Path 2 | total_timesteps 107.
Path 3 | total_timesteps 193.
Path 4 | total_timesteps 272.
Path 5 | total_timesteps 336.
Path 6 | total_timesteps 400.
Path 7 | total_timesteps 459.
Path 8 | total_timesteps 545.
Path 9 | total_timesteps 653.
Path 10 | total_timesteps 745.
Path 11 | total_timesteps 773.
Path 12 | total_timesteps 841.
Path 13 | total_timesteps 926.
Path 14 | total_timesteps 1023.
Path 15 | total_timesteps 1080.
Path 16 | total_timesteps 1158.
Path 17 | total_timesteps 1225.
Path 18 | total_timesteps 1309.
Path 19 | total_timesteps 1374.
Path 20 | total_timesteps 1490.
Path 21 | total_timesteps 1570.
Path 22 | total_timesteps 1644.
Path 23 | total_timesteps 1737.
Path 24 | total_timesteps 1798.
Path 25 | total_timesteps 1868.
Path 26 | total_timesteps 1956.
Path 27 | total_timesteps 2022.
Path 28 | total_timesteps 2102.
Path 29 | total_timesteps 2145.
Path 30 | total_timesteps 2214.
Path 31 | total_timesteps 2284.
Path 32 | total_timesteps 2383.
Path 33 | total_timesteps 2462.
Path 34 | total_timesteps 2525.
Path 35 | total_timesteps 2589.
Path 36 | total_timesteps 2673.
Path 37 | total_timesteps 2786.
Path 38 | total_timesteps 2873.
Path 39 | total_timesteps 2902.
Path 40 | total_timesteps 3033.
Path 41 | total_timesteps 3112.
Path 42 | total_timesteps 3184.
Path 43 | total_timesteps 3273.
Path 44 | total_timesteps 3329.
Path 45 | total_timesteps 3419.
Path 46 | total_timesteps 3457.
Path 47 | total_timesteps 3537.
Path 48 | total_timesteps 3614.
Path 49 | total_timesteps 3732.
Path 50 | total_timesteps 3820.
Path 51 | total_timesteps 3889.
Path 52 | total_timesteps 3980.
Path 53 | total_timesteps 4061.
Path 54 | total_timesteps 4169.
Path 55 | total_timesteps 4226.
Path 56 | total_timesteps 4329.
Path 57 | total_timesteps 4383.
Path 58 | total_timesteps 4452.
Path 59 | total_timesteps 4512.
Path 60 | total_timesteps 4600.
Path 61 | total_timesteps 4717.
Path 62 | total_timesteps 4794.
Path 63 | total_timesteps 4834.
Path 64 | total_timesteps 4917.
Path 65 | total_timesteps 4965.
Path 66 | total_timesteps 5034.
Path 67 | total_timesteps 5070.
Path 68 | total_timesteps 5148.
Path 69 | total_timesteps 5239.
Path 70 | total_timesteps 5314.
Path 71 | total_timesteps 5415.
Path 72 | total_timesteps 5486.
Path 73 | total_timesteps 5566.
Path 74 | total_timesteps 5645.
Path 75 | total_timesteps 5750.
Path 76 | total_timesteps 5830.
Path 77 | total_timesteps 5928.
Path 78 | total_timesteps 5971.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -46.8    |
| Iteration     | 16       |
| MaximumReturn | 0.0423   |
| MinimumReturn | -111     |
| TotalSamples  | 72203    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.32807502150535583
Validation loss = 0.33675384521484375
Validation loss = 0.33538082242012024
Validation loss = 0.33539536595344543
Validation loss = 0.3407459855079651
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 68.
Path 2 | total_timesteps 172.
Path 3 | total_timesteps 230.
Path 4 | total_timesteps 299.
Path 5 | total_timesteps 357.
Path 6 | total_timesteps 413.
Path 7 | total_timesteps 474.
Path 8 | total_timesteps 709.
Path 9 | total_timesteps 837.
Path 10 | total_timesteps 912.
Path 11 | total_timesteps 981.
Path 12 | total_timesteps 1049.
Path 13 | total_timesteps 1108.
Path 14 | total_timesteps 1198.
Path 15 | total_timesteps 1238.
Path 16 | total_timesteps 1329.
Path 17 | total_timesteps 1388.
Path 18 | total_timesteps 1468.
Path 19 | total_timesteps 1541.
Path 20 | total_timesteps 1606.
Path 21 | total_timesteps 1686.
Path 22 | total_timesteps 1776.
Path 23 | total_timesteps 1930.
Path 24 | total_timesteps 2009.
Path 25 | total_timesteps 2131.
Path 26 | total_timesteps 2173.
Path 27 | total_timesteps 2238.
Path 28 | total_timesteps 2318.
Path 29 | total_timesteps 2347.
Path 30 | total_timesteps 2449.
Path 31 | total_timesteps 2477.
Path 32 | total_timesteps 2537.
Path 33 | total_timesteps 2622.
Path 34 | total_timesteps 2677.
Path 35 | total_timesteps 2762.
Path 36 | total_timesteps 2899.
Path 37 | total_timesteps 3042.
Path 38 | total_timesteps 3111.
Path 39 | total_timesteps 3202.
Path 40 | total_timesteps 3283.
Path 41 | total_timesteps 3372.
Path 42 | total_timesteps 3455.
Path 43 | total_timesteps 3562.
Path 44 | total_timesteps 3646.
Path 45 | total_timesteps 3736.
Path 46 | total_timesteps 3829.
Path 47 | total_timesteps 3963.
Path 48 | total_timesteps 4052.
Path 49 | total_timesteps 4092.
Path 50 | total_timesteps 4177.
Path 51 | total_timesteps 4279.
Path 52 | total_timesteps 4353.
Path 53 | total_timesteps 4403.
Path 54 | total_timesteps 4488.
Path 55 | total_timesteps 4567.
Path 56 | total_timesteps 4637.
Path 57 | total_timesteps 4693.
Path 58 | total_timesteps 4828.
Path 59 | total_timesteps 4913.
Path 60 | total_timesteps 4976.
Path 61 | total_timesteps 5073.
Path 62 | total_timesteps 5132.
Path 63 | total_timesteps 5176.
Path 64 | total_timesteps 5268.
Path 65 | total_timesteps 5361.
Path 66 | total_timesteps 5443.
Path 67 | total_timesteps 5507.
Path 68 | total_timesteps 5544.
Path 69 | total_timesteps 5598.
Path 70 | total_timesteps 5669.
Path 71 | total_timesteps 5738.
Path 72 | total_timesteps 5814.
Path 73 | total_timesteps 5909.
Path 74 | total_timesteps 5962.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -42.3    |
| Iteration     | 17       |
| MaximumReturn | -2.08    |
| MinimumReturn | -94.8    |
| TotalSamples  | 76242    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.3335449695587158
Validation loss = 0.33812984824180603
Validation loss = 0.3428990840911865
Validation loss = 0.34333065152168274
Validation loss = 0.3450136184692383
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 93.
Path 2 | total_timesteps 191.
Path 3 | total_timesteps 265.
Path 4 | total_timesteps 303.
Path 5 | total_timesteps 389.
Path 6 | total_timesteps 487.
Path 7 | total_timesteps 718.
Path 8 | total_timesteps 792.
Path 9 | total_timesteps 882.
Path 10 | total_timesteps 944.
Path 11 | total_timesteps 1006.
Path 12 | total_timesteps 1089.
Path 13 | total_timesteps 1164.
Path 14 | total_timesteps 1236.
Path 15 | total_timesteps 1256.
Path 16 | total_timesteps 1350.
Path 17 | total_timesteps 1416.
Path 18 | total_timesteps 1485.
Path 19 | total_timesteps 1545.
Path 20 | total_timesteps 1600.
Path 21 | total_timesteps 1698.
Path 22 | total_timesteps 1762.
Path 23 | total_timesteps 1836.
Path 24 | total_timesteps 1917.
Path 25 | total_timesteps 1991.
Path 26 | total_timesteps 2073.
Path 27 | total_timesteps 2144.
Path 28 | total_timesteps 2219.
Path 29 | total_timesteps 2295.
Path 30 | total_timesteps 2347.
Path 31 | total_timesteps 2405.
Path 32 | total_timesteps 2493.
Path 33 | total_timesteps 2567.
Path 34 | total_timesteps 2622.
Path 35 | total_timesteps 2672.
Path 36 | total_timesteps 2726.
Path 37 | total_timesteps 2807.
Path 38 | total_timesteps 2904.
Path 39 | total_timesteps 2984.
Path 40 | total_timesteps 3066.
Path 41 | total_timesteps 3134.
Path 42 | total_timesteps 3204.
Path 43 | total_timesteps 3269.
Path 44 | total_timesteps 3333.
Path 45 | total_timesteps 3388.
Path 46 | total_timesteps 3455.
Path 47 | total_timesteps 3519.
Path 48 | total_timesteps 3553.
Path 49 | total_timesteps 3635.
Path 50 | total_timesteps 3707.
Path 51 | total_timesteps 3778.
Path 52 | total_timesteps 3845.
Path 53 | total_timesteps 3934.
Path 54 | total_timesteps 4032.
Path 55 | total_timesteps 4124.
Path 56 | total_timesteps 4215.
Path 57 | total_timesteps 4240.
Path 58 | total_timesteps 4407.
Path 59 | total_timesteps 4496.
Path 60 | total_timesteps 4566.
Path 61 | total_timesteps 4659.
Path 62 | total_timesteps 4763.
Path 63 | total_timesteps 4866.
Path 64 | total_timesteps 4949.
Path 65 | total_timesteps 5001.
Path 66 | total_timesteps 5065.
Path 67 | total_timesteps 5152.
Path 68 | total_timesteps 5214.
Path 69 | total_timesteps 5281.
Path 70 | total_timesteps 5365.
Path 71 | total_timesteps 5422.
Path 72 | total_timesteps 5481.
Path 73 | total_timesteps 5593.
Path 74 | total_timesteps 5643.
Path 75 | total_timesteps 5670.
Path 76 | total_timesteps 5753.
Path 77 | total_timesteps 5827.
Path 78 | total_timesteps 5893.
Path 79 | total_timesteps 5977.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -43.4    |
| Iteration     | 18       |
| MaximumReturn | -1.92    |
| MinimumReturn | -97      |
| TotalSamples  | 80266    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.33812254667282104
Validation loss = 0.3432859778404236
Validation loss = 0.3444298207759857
Validation loss = 0.3467123508453369
Validation loss = 0.34534120559692383
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 127.
Path 2 | total_timesteps 224.
Path 3 | total_timesteps 315.
Path 4 | total_timesteps 396.
Path 5 | total_timesteps 477.
Path 6 | total_timesteps 577.
Path 7 | total_timesteps 672.
Path 8 | total_timesteps 757.
Path 9 | total_timesteps 834.
Path 10 | total_timesteps 932.
Path 11 | total_timesteps 1024.
Path 12 | total_timesteps 1116.
Path 13 | total_timesteps 1214.
Path 14 | total_timesteps 1327.
Path 15 | total_timesteps 1455.
Path 16 | total_timesteps 1546.
Path 17 | total_timesteps 1630.
Path 18 | total_timesteps 1710.
Path 19 | total_timesteps 1804.
Path 20 | total_timesteps 1928.
Path 21 | total_timesteps 2027.
Path 22 | total_timesteps 2156.
Path 23 | total_timesteps 2254.
Path 24 | total_timesteps 2364.
Path 25 | total_timesteps 2429.
Path 26 | total_timesteps 2520.
Path 27 | total_timesteps 2616.
Path 28 | total_timesteps 2685.
Path 29 | total_timesteps 2759.
Path 30 | total_timesteps 2844.
Path 31 | total_timesteps 2948.
Path 32 | total_timesteps 3045.
Path 33 | total_timesteps 3122.
Path 34 | total_timesteps 3221.
Path 35 | total_timesteps 3294.
Path 36 | total_timesteps 3385.
Path 37 | total_timesteps 3452.
Path 38 | total_timesteps 3559.
Path 39 | total_timesteps 3691.
Path 40 | total_timesteps 3776.
Path 41 | total_timesteps 3843.
Path 42 | total_timesteps 3941.
Path 43 | total_timesteps 4016.
Path 44 | total_timesteps 4093.
Path 45 | total_timesteps 4188.
Path 46 | total_timesteps 4290.
Path 47 | total_timesteps 4376.
Path 48 | total_timesteps 4460.
Path 49 | total_timesteps 4568.
Path 50 | total_timesteps 4663.
Path 51 | total_timesteps 4762.
Path 52 | total_timesteps 4852.
Path 53 | total_timesteps 4946.
Path 54 | total_timesteps 5033.
Path 55 | total_timesteps 5122.
Path 56 | total_timesteps 5223.
Path 57 | total_timesteps 5353.
Path 58 | total_timesteps 5445.
Path 59 | total_timesteps 5561.
Path 60 | total_timesteps 5672.
Path 61 | total_timesteps 5794.
Path 62 | total_timesteps 5885.
Path 63 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -76.1    |
| Iteration     | 19       |
| MaximumReturn | -30.5    |
| MinimumReturn | -123     |
| TotalSamples  | 84315    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.34462133049964905
Validation loss = 0.3472751975059509
Validation loss = 0.3497920632362366
Validation loss = 0.34788480401039124
Validation loss = 0.3545689582824707
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 167.
Path 2 | total_timesteps 286.
Path 3 | total_timesteps 394.
Path 4 | total_timesteps 507.
Path 5 | total_timesteps 596.
Path 6 | total_timesteps 675.
Path 7 | total_timesteps 750.
Path 8 | total_timesteps 869.
Path 9 | total_timesteps 965.
Path 10 | total_timesteps 1062.
Path 11 | total_timesteps 1182.
Path 12 | total_timesteps 1261.
Path 13 | total_timesteps 1361.
Path 14 | total_timesteps 1462.
Path 15 | total_timesteps 1556.
Path 16 | total_timesteps 1653.
Path 17 | total_timesteps 1742.
Path 18 | total_timesteps 1835.
Path 19 | total_timesteps 1925.
Path 20 | total_timesteps 1976.
Path 21 | total_timesteps 2061.
Path 22 | total_timesteps 2144.
Path 23 | total_timesteps 2214.
Path 24 | total_timesteps 2308.
Path 25 | total_timesteps 2440.
Path 26 | total_timesteps 2535.
Path 27 | total_timesteps 2636.
Path 28 | total_timesteps 2733.
Path 29 | total_timesteps 2797.
Path 30 | total_timesteps 2899.
Path 31 | total_timesteps 2994.
Path 32 | total_timesteps 3083.
Path 33 | total_timesteps 3147.
Path 34 | total_timesteps 3228.
Path 35 | total_timesteps 3306.
Path 36 | total_timesteps 3401.
Path 37 | total_timesteps 3507.
Path 38 | total_timesteps 3598.
Path 39 | total_timesteps 3683.
Path 40 | total_timesteps 3802.
Path 41 | total_timesteps 3895.
Path 42 | total_timesteps 3983.
Path 43 | total_timesteps 4078.
Path 44 | total_timesteps 4157.
Path 45 | total_timesteps 4281.
Path 46 | total_timesteps 4374.
Path 47 | total_timesteps 4470.
Path 48 | total_timesteps 4570.
Path 49 | total_timesteps 4668.
Path 50 | total_timesteps 4857.
Path 51 | total_timesteps 4944.
Path 52 | total_timesteps 5031.
Path 53 | total_timesteps 5129.
Path 54 | total_timesteps 5198.
Path 55 | total_timesteps 5283.
Path 56 | total_timesteps 5380.
Path 57 | total_timesteps 5470.
Path 58 | total_timesteps 5569.
Path 59 | total_timesteps 5673.
Path 60 | total_timesteps 5749.
Path 61 | total_timesteps 5859.
Path 62 | total_timesteps 5952.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -70.6    |
| Iteration     | 20       |
| MaximumReturn | 10       |
| MinimumReturn | -132     |
| TotalSamples  | 88342    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.34662318229675293
Validation loss = 0.3496055006980896
Validation loss = 0.3505490720272064
Validation loss = 0.356138676404953
Validation loss = 0.35396620631217957
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 80.
Path 2 | total_timesteps 176.
Path 3 | total_timesteps 278.
Path 4 | total_timesteps 408.
Path 5 | total_timesteps 496.
Path 6 | total_timesteps 606.
Path 7 | total_timesteps 691.
Path 8 | total_timesteps 790.
Path 9 | total_timesteps 875.
Path 10 | total_timesteps 964.
Path 11 | total_timesteps 1050.
Path 12 | total_timesteps 1150.
Path 13 | total_timesteps 1233.
Path 14 | total_timesteps 1324.
Path 15 | total_timesteps 1418.
Path 16 | total_timesteps 1516.
Path 17 | total_timesteps 1634.
Path 18 | total_timesteps 1734.
Path 19 | total_timesteps 1826.
Path 20 | total_timesteps 1909.
Path 21 | total_timesteps 2019.
Path 22 | total_timesteps 2114.
Path 23 | total_timesteps 2213.
Path 24 | total_timesteps 2299.
Path 25 | total_timesteps 2429.
Path 26 | total_timesteps 2516.
Path 27 | total_timesteps 2609.
Path 28 | total_timesteps 2699.
Path 29 | total_timesteps 2785.
Path 30 | total_timesteps 2875.
Path 31 | total_timesteps 2975.
Path 32 | total_timesteps 3074.
Path 33 | total_timesteps 3201.
Path 34 | total_timesteps 3282.
Path 35 | total_timesteps 3376.
Path 36 | total_timesteps 3469.
Path 37 | total_timesteps 3564.
Path 38 | total_timesteps 3649.
Path 39 | total_timesteps 3789.
Path 40 | total_timesteps 3875.
Path 41 | total_timesteps 4017.
Path 42 | total_timesteps 4097.
Path 43 | total_timesteps 4235.
Path 44 | total_timesteps 4339.
Path 45 | total_timesteps 4439.
Path 46 | total_timesteps 4527.
Path 47 | total_timesteps 4641.
Path 48 | total_timesteps 4739.
Path 49 | total_timesteps 4830.
Path 50 | total_timesteps 4928.
Path 51 | total_timesteps 5030.
Path 52 | total_timesteps 5140.
Path 53 | total_timesteps 5234.
Path 54 | total_timesteps 5334.
Path 55 | total_timesteps 5446.
Path 56 | total_timesteps 5528.
Path 57 | total_timesteps 5617.
Path 58 | total_timesteps 5737.
Path 59 | total_timesteps 5817.
Path 60 | total_timesteps 5912.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -74.9    |
| Iteration     | 21       |
| MaximumReturn | -22.1    |
| MinimumReturn | -110     |
| TotalSamples  | 92352    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.3485192358493805
Validation loss = 0.3537699580192566
Validation loss = 0.35625723004341125
Validation loss = 0.35880184173583984
Validation loss = 0.3589099049568176
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 87.
Path 2 | total_timesteps 209.
Path 3 | total_timesteps 306.
Path 4 | total_timesteps 415.
Path 5 | total_timesteps 504.
Path 6 | total_timesteps 588.
Path 7 | total_timesteps 669.
Path 8 | total_timesteps 835.
Path 9 | total_timesteps 942.
Path 10 | total_timesteps 1037.
Path 11 | total_timesteps 1122.
Path 12 | total_timesteps 1250.
Path 13 | total_timesteps 1330.
Path 14 | total_timesteps 1414.
Path 15 | total_timesteps 1500.
Path 16 | total_timesteps 1589.
Path 17 | total_timesteps 1683.
Path 18 | total_timesteps 1782.
Path 19 | total_timesteps 1873.
Path 20 | total_timesteps 1966.
Path 21 | total_timesteps 2055.
Path 22 | total_timesteps 2151.
Path 23 | total_timesteps 2212.
Path 24 | total_timesteps 2307.
Path 25 | total_timesteps 2414.
Path 26 | total_timesteps 2502.
Path 27 | total_timesteps 2602.
Path 28 | total_timesteps 2678.
Path 29 | total_timesteps 2785.
Path 30 | total_timesteps 2893.
Path 31 | total_timesteps 2978.
Path 32 | total_timesteps 3073.
Path 33 | total_timesteps 3242.
Path 34 | total_timesteps 3355.
Path 35 | total_timesteps 3468.
Path 36 | total_timesteps 3553.
Path 37 | total_timesteps 3632.
Path 38 | total_timesteps 3721.
Path 39 | total_timesteps 3813.
Path 40 | total_timesteps 3901.
Path 41 | total_timesteps 4001.
Path 42 | total_timesteps 4085.
Path 43 | total_timesteps 4185.
Path 44 | total_timesteps 4263.
Path 45 | total_timesteps 4358.
Path 46 | total_timesteps 4449.
Path 47 | total_timesteps 4543.
Path 48 | total_timesteps 4648.
Path 49 | total_timesteps 4733.
Path 50 | total_timesteps 4812.
Path 51 | total_timesteps 4907.
Path 52 | total_timesteps 5009.
Path 53 | total_timesteps 5105.
Path 54 | total_timesteps 5204.
Path 55 | total_timesteps 5291.
Path 56 | total_timesteps 5383.
Path 57 | total_timesteps 5473.
Path 58 | total_timesteps 5549.
Path 59 | total_timesteps 5690.
Path 60 | total_timesteps 5780.
Path 61 | total_timesteps 5880.
Path 62 | total_timesteps 5971.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -74.3    |
| Iteration     | 22       |
| MaximumReturn | -29.9    |
| MinimumReturn | -103     |
| TotalSamples  | 96411    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.35445818305015564
Validation loss = 0.3573159873485565
Validation loss = 0.3612558841705322
Validation loss = 0.36120495200157166
Validation loss = 0.3626374304294586
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 104.
Path 2 | total_timesteps 214.
Path 3 | total_timesteps 315.
Path 4 | total_timesteps 410.
Path 5 | total_timesteps 511.
Path 6 | total_timesteps 592.
Path 7 | total_timesteps 722.
Path 8 | total_timesteps 839.
Path 9 | total_timesteps 941.
Path 10 | total_timesteps 1025.
Path 11 | total_timesteps 1127.
Path 12 | total_timesteps 1223.
Path 13 | total_timesteps 1303.
Path 14 | total_timesteps 1375.
Path 15 | total_timesteps 1482.
Path 16 | total_timesteps 1578.
Path 17 | total_timesteps 1680.
Path 18 | total_timesteps 1771.
Path 19 | total_timesteps 1878.
Path 20 | total_timesteps 1977.
Path 21 | total_timesteps 2056.
Path 22 | total_timesteps 2147.
Path 23 | total_timesteps 2231.
Path 24 | total_timesteps 2394.
Path 25 | total_timesteps 2490.
Path 26 | total_timesteps 2594.
Path 27 | total_timesteps 2686.
Path 28 | total_timesteps 2783.
Path 29 | total_timesteps 2866.
Path 30 | total_timesteps 2953.
Path 31 | total_timesteps 3067.
Path 32 | total_timesteps 3172.
Path 33 | total_timesteps 3265.
Path 34 | total_timesteps 3371.
Path 35 | total_timesteps 3462.
Path 36 | total_timesteps 3551.
Path 37 | total_timesteps 3633.
Path 38 | total_timesteps 3730.
Path 39 | total_timesteps 3830.
Path 40 | total_timesteps 3907.
Path 41 | total_timesteps 4103.
Path 42 | total_timesteps 4191.
Path 43 | total_timesteps 4285.
Path 44 | total_timesteps 4390.
Path 45 | total_timesteps 4471.
Path 46 | total_timesteps 4561.
Path 47 | total_timesteps 4743.
Path 48 | total_timesteps 4826.
Path 49 | total_timesteps 4918.
Path 50 | total_timesteps 5032.
Path 51 | total_timesteps 5124.
Path 52 | total_timesteps 5210.
Path 53 | total_timesteps 5307.
Path 54 | total_timesteps 5411.
Path 55 | total_timesteps 5497.
Path 56 | total_timesteps 5597.
Path 57 | total_timesteps 5691.
Path 58 | total_timesteps 5825.
Path 59 | total_timesteps 5910.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -77.7    |
| Iteration     | 23       |
| MaximumReturn | -35      |
| MinimumReturn | -111     |
| TotalSamples  | 100428   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.36231642961502075
Validation loss = 0.3607439398765564
Validation loss = 0.3631893992424011
Validation loss = 0.3637794852256775
Validation loss = 0.36519256234169006
Validation loss = 0.3683033883571625
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 82.
Path 2 | total_timesteps 179.
Path 3 | total_timesteps 271.
Path 4 | total_timesteps 383.
Path 5 | total_timesteps 466.
Path 6 | total_timesteps 566.
Path 7 | total_timesteps 639.
Path 8 | total_timesteps 742.
Path 9 | total_timesteps 844.
Path 10 | total_timesteps 927.
Path 11 | total_timesteps 1004.
Path 12 | total_timesteps 1096.
Path 13 | total_timesteps 1200.
Path 14 | total_timesteps 1293.
Path 15 | total_timesteps 1432.
Path 16 | total_timesteps 1531.
Path 17 | total_timesteps 1622.
Path 18 | total_timesteps 1712.
Path 19 | total_timesteps 1827.
Path 20 | total_timesteps 1931.
Path 21 | total_timesteps 2023.
Path 22 | total_timesteps 2113.
Path 23 | total_timesteps 2216.
Path 24 | total_timesteps 2309.
Path 25 | total_timesteps 2399.
Path 26 | total_timesteps 2492.
Path 27 | total_timesteps 2566.
Path 28 | total_timesteps 2649.
Path 29 | total_timesteps 2745.
Path 30 | total_timesteps 2858.
Path 31 | total_timesteps 2955.
Path 32 | total_timesteps 3043.
Path 33 | total_timesteps 3201.
Path 34 | total_timesteps 3299.
Path 35 | total_timesteps 3382.
Path 36 | total_timesteps 3472.
Path 37 | total_timesteps 3555.
Path 38 | total_timesteps 3677.
Path 39 | total_timesteps 3771.
Path 40 | total_timesteps 3868.
Path 41 | total_timesteps 4025.
Path 42 | total_timesteps 4113.
Path 43 | total_timesteps 4194.
Path 44 | total_timesteps 4289.
Path 45 | total_timesteps 4369.
Path 46 | total_timesteps 4458.
Path 47 | total_timesteps 4555.
Path 48 | total_timesteps 4644.
Path 49 | total_timesteps 4763.
Path 50 | total_timesteps 4875.
Path 51 | total_timesteps 4979.
Path 52 | total_timesteps 5073.
Path 53 | total_timesteps 5156.
Path 54 | total_timesteps 5242.
Path 55 | total_timesteps 5325.
Path 56 | total_timesteps 5408.
Path 57 | total_timesteps 5500.
Path 58 | total_timesteps 5584.
Path 59 | total_timesteps 5673.
Path 60 | total_timesteps 5791.
Path 61 | total_timesteps 5866.
Path 62 | total_timesteps 5960.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -74.1    |
| Iteration     | 24       |
| MaximumReturn | -35.2    |
| MinimumReturn | -105     |
| TotalSamples  | 104450   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.36092662811279297
Validation loss = 0.3663035035133362
Validation loss = 0.36777129769325256
Validation loss = 0.3673437833786011
Validation loss = 0.3686431646347046
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 91.
Path 2 | total_timesteps 191.
Path 3 | total_timesteps 297.
Path 4 | total_timesteps 390.
Path 5 | total_timesteps 481.
Path 6 | total_timesteps 579.
Path 7 | total_timesteps 665.
Path 8 | total_timesteps 756.
Path 9 | total_timesteps 856.
Path 10 | total_timesteps 943.
Path 11 | total_timesteps 1015.
Path 12 | total_timesteps 1114.
Path 13 | total_timesteps 1210.
Path 14 | total_timesteps 1343.
Path 15 | total_timesteps 1444.
Path 16 | total_timesteps 1538.
Path 17 | total_timesteps 1625.
Path 18 | total_timesteps 1722.
Path 19 | total_timesteps 1872.
Path 20 | total_timesteps 1958.
Path 21 | total_timesteps 2059.
Path 22 | total_timesteps 2136.
Path 23 | total_timesteps 2219.
Path 24 | total_timesteps 2306.
Path 25 | total_timesteps 2391.
Path 26 | total_timesteps 2482.
Path 27 | total_timesteps 2573.
Path 28 | total_timesteps 2654.
Path 29 | total_timesteps 2749.
Path 30 | total_timesteps 2868.
Path 31 | total_timesteps 2970.
Path 32 | total_timesteps 3063.
Path 33 | total_timesteps 3143.
Path 34 | total_timesteps 3293.
Path 35 | total_timesteps 3394.
Path 36 | total_timesteps 3488.
Path 37 | total_timesteps 3589.
Path 38 | total_timesteps 3672.
Path 39 | total_timesteps 3793.
Path 40 | total_timesteps 3888.
Path 41 | total_timesteps 4009.
Path 42 | total_timesteps 4100.
Path 43 | total_timesteps 4193.
Path 44 | total_timesteps 4291.
Path 45 | total_timesteps 4378.
Path 46 | total_timesteps 4469.
Path 47 | total_timesteps 4552.
Path 48 | total_timesteps 4648.
Path 49 | total_timesteps 4747.
Path 50 | total_timesteps 4831.
Path 51 | total_timesteps 4918.
Path 52 | total_timesteps 5009.
Path 53 | total_timesteps 5125.
Path 54 | total_timesteps 5208.
Path 55 | total_timesteps 5370.
Path 56 | total_timesteps 5465.
Path 57 | total_timesteps 5559.
Path 58 | total_timesteps 5656.
Path 59 | total_timesteps 5753.
Path 60 | total_timesteps 5860.
Path 61 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -75.6    |
| Iteration     | 25       |
| MaximumReturn | -29.3    |
| MinimumReturn | -124     |
| TotalSamples  | 108506   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.3641265332698822
Validation loss = 0.368324339389801
Validation loss = 0.37124061584472656
Validation loss = 0.3712283670902252
Validation loss = 0.3712846338748932
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 83.
Path 2 | total_timesteps 182.
Path 3 | total_timesteps 282.
Path 4 | total_timesteps 384.
Path 5 | total_timesteps 464.
Path 6 | total_timesteps 585.
Path 7 | total_timesteps 711.
Path 8 | total_timesteps 817.
Path 9 | total_timesteps 912.
Path 10 | total_timesteps 1022.
Path 11 | total_timesteps 1122.
Path 12 | total_timesteps 1225.
Path 13 | total_timesteps 1370.
Path 14 | total_timesteps 1490.
Path 15 | total_timesteps 1570.
Path 16 | total_timesteps 1664.
Path 17 | total_timesteps 1751.
Path 18 | total_timesteps 1837.
Path 19 | total_timesteps 1956.
Path 20 | total_timesteps 2055.
Path 21 | total_timesteps 2160.
Path 22 | total_timesteps 2239.
Path 23 | total_timesteps 2356.
Path 24 | total_timesteps 2437.
Path 25 | total_timesteps 2531.
Path 26 | total_timesteps 2634.
Path 27 | total_timesteps 2721.
Path 28 | total_timesteps 2814.
Path 29 | total_timesteps 2903.
Path 30 | total_timesteps 3009.
Path 31 | total_timesteps 3118.
Path 32 | total_timesteps 3223.
Path 33 | total_timesteps 3332.
Path 34 | total_timesteps 3415.
Path 35 | total_timesteps 3518.
Path 36 | total_timesteps 3616.
Path 37 | total_timesteps 3712.
Path 38 | total_timesteps 3804.
Path 39 | total_timesteps 3924.
Path 40 | total_timesteps 4004.
Path 41 | total_timesteps 4098.
Path 42 | total_timesteps 4209.
Path 43 | total_timesteps 4318.
Path 44 | total_timesteps 4406.
Path 45 | total_timesteps 4509.
Path 46 | total_timesteps 4599.
Path 47 | total_timesteps 4689.
Path 48 | total_timesteps 4774.
Path 49 | total_timesteps 4859.
Path 50 | total_timesteps 4950.
Path 51 | total_timesteps 5038.
Path 52 | total_timesteps 5147.
Path 53 | total_timesteps 5251.
Path 54 | total_timesteps 5371.
Path 55 | total_timesteps 5453.
Path 56 | total_timesteps 5612.
Path 57 | total_timesteps 5726.
Path 58 | total_timesteps 5819.
Path 59 | total_timesteps 5895.
Path 60 | total_timesteps 5975.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -75.8    |
| Iteration     | 26       |
| MaximumReturn | -45.8    |
| MinimumReturn | -125     |
| TotalSamples  | 112557   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.3681773543357849
Validation loss = 0.3718048632144928
Validation loss = 0.3721669614315033
Validation loss = 0.3731895387172699
Validation loss = 0.3726237714290619
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 133.
Path 2 | total_timesteps 238.
Path 3 | total_timesteps 321.
Path 4 | total_timesteps 415.
Path 5 | total_timesteps 511.
Path 6 | total_timesteps 616.
Path 7 | total_timesteps 723.
Path 8 | total_timesteps 810.
Path 9 | total_timesteps 913.
Path 10 | total_timesteps 1009.
Path 11 | total_timesteps 1120.
Path 12 | total_timesteps 1208.
Path 13 | total_timesteps 1298.
Path 14 | total_timesteps 1397.
Path 15 | total_timesteps 1491.
Path 16 | total_timesteps 1607.
Path 17 | total_timesteps 1714.
Path 18 | total_timesteps 1820.
Path 19 | total_timesteps 1906.
Path 20 | total_timesteps 2005.
Path 21 | total_timesteps 2102.
Path 22 | total_timesteps 2213.
Path 23 | total_timesteps 2312.
Path 24 | total_timesteps 2409.
Path 25 | total_timesteps 2505.
Path 26 | total_timesteps 2626.
Path 27 | total_timesteps 2731.
Path 28 | total_timesteps 2817.
Path 29 | total_timesteps 2906.
Path 30 | total_timesteps 3004.
Path 31 | total_timesteps 3115.
Path 32 | total_timesteps 3220.
Path 33 | total_timesteps 3306.
Path 34 | total_timesteps 3401.
Path 35 | total_timesteps 3491.
Path 36 | total_timesteps 3595.
Path 37 | total_timesteps 3728.
Path 38 | total_timesteps 3851.
Path 39 | total_timesteps 3953.
Path 40 | total_timesteps 4062.
Path 41 | total_timesteps 4157.
Path 42 | total_timesteps 4245.
Path 43 | total_timesteps 4341.
Path 44 | total_timesteps 4438.
Path 45 | total_timesteps 4530.
Path 46 | total_timesteps 4617.
Path 47 | total_timesteps 4728.
Path 48 | total_timesteps 4832.
Path 49 | total_timesteps 4951.
Path 50 | total_timesteps 5042.
Path 51 | total_timesteps 5149.
Path 52 | total_timesteps 5237.
Path 53 | total_timesteps 5333.
Path 54 | total_timesteps 5416.
Path 55 | total_timesteps 5543.
Path 56 | total_timesteps 5639.
Path 57 | total_timesteps 5757.
Path 58 | total_timesteps 5847.
Path 59 | total_timesteps 5960.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -78.1    |
| Iteration     | 27       |
| MaximumReturn | -34.7    |
| MinimumReturn | -105     |
| TotalSamples  | 116588   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.36946678161621094
Validation loss = 0.37323740124702454
Validation loss = 0.37555450201034546
Validation loss = 0.3780931830406189
Validation loss = 0.37673419713974
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 105.
Path 2 | total_timesteps 219.
Path 3 | total_timesteps 305.
Path 4 | total_timesteps 394.
Path 5 | total_timesteps 508.
Path 6 | total_timesteps 613.
Path 7 | total_timesteps 708.
Path 8 | total_timesteps 806.
Path 9 | total_timesteps 905.
Path 10 | total_timesteps 985.
Path 11 | total_timesteps 1075.
Path 12 | total_timesteps 1175.
Path 13 | total_timesteps 1289.
Path 14 | total_timesteps 1390.
Path 15 | total_timesteps 1491.
Path 16 | total_timesteps 1583.
Path 17 | total_timesteps 1696.
Path 18 | total_timesteps 1794.
Path 19 | total_timesteps 1889.
Path 20 | total_timesteps 1988.
Path 21 | total_timesteps 2068.
Path 22 | total_timesteps 2163.
Path 23 | total_timesteps 2264.
Path 24 | total_timesteps 2385.
Path 25 | total_timesteps 2496.
Path 26 | total_timesteps 2584.
Path 27 | total_timesteps 2678.
Path 28 | total_timesteps 2821.
Path 29 | total_timesteps 2930.
Path 30 | total_timesteps 3016.
Path 31 | total_timesteps 3106.
Path 32 | total_timesteps 3192.
Path 33 | total_timesteps 3299.
Path 34 | total_timesteps 3456.
Path 35 | total_timesteps 3551.
Path 36 | total_timesteps 3661.
Path 37 | total_timesteps 3794.
Path 38 | total_timesteps 3876.
Path 39 | total_timesteps 3977.
Path 40 | total_timesteps 4130.
Path 41 | total_timesteps 4231.
Path 42 | total_timesteps 4350.
Path 43 | total_timesteps 4440.
Path 44 | total_timesteps 4535.
Path 45 | total_timesteps 4601.
Path 46 | total_timesteps 4745.
Path 47 | total_timesteps 4835.
Path 48 | total_timesteps 4936.
Path 49 | total_timesteps 5041.
Path 50 | total_timesteps 5139.
Path 51 | total_timesteps 5215.
Path 52 | total_timesteps 5317.
Path 53 | total_timesteps 5414.
Path 54 | total_timesteps 5506.
Path 55 | total_timesteps 5623.
Path 56 | total_timesteps 5712.
Path 57 | total_timesteps 5812.
Path 58 | total_timesteps 5892.
Path 59 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -78.1    |
| Iteration     | 28       |
| MaximumReturn | -48.4    |
| MinimumReturn | -112     |
| TotalSamples  | 120655   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.3729110062122345
Validation loss = 0.3765859603881836
Validation loss = 0.3785911798477173
Validation loss = 0.37837931513786316
Validation loss = 0.38042911887168884
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 74.
Path 2 | total_timesteps 155.
Path 3 | total_timesteps 254.
Path 4 | total_timesteps 361.
Path 5 | total_timesteps 476.
Path 6 | total_timesteps 572.
Path 7 | total_timesteps 670.
Path 8 | total_timesteps 759.
Path 9 | total_timesteps 870.
Path 10 | total_timesteps 958.
Path 11 | total_timesteps 1050.
Path 12 | total_timesteps 1143.
Path 13 | total_timesteps 1233.
Path 14 | total_timesteps 1334.
Path 15 | total_timesteps 1438.
Path 16 | total_timesteps 1534.
Path 17 | total_timesteps 1624.
Path 18 | total_timesteps 1719.
Path 19 | total_timesteps 1804.
Path 20 | total_timesteps 1911.
Path 21 | total_timesteps 2007.
Path 22 | total_timesteps 2102.
Path 23 | total_timesteps 2186.
Path 24 | total_timesteps 2290.
Path 25 | total_timesteps 2394.
Path 26 | total_timesteps 2528.
Path 27 | total_timesteps 2608.
Path 28 | total_timesteps 2711.
Path 29 | total_timesteps 2902.
Path 30 | total_timesteps 2983.
Path 31 | total_timesteps 3075.
Path 32 | total_timesteps 3164.
Path 33 | total_timesteps 3251.
Path 34 | total_timesteps 3384.
Path 35 | total_timesteps 3492.
Path 36 | total_timesteps 3598.
Path 37 | total_timesteps 3676.
Path 38 | total_timesteps 3751.
Path 39 | total_timesteps 3927.
Path 40 | total_timesteps 4024.
Path 41 | total_timesteps 4106.
Path 42 | total_timesteps 4217.
Path 43 | total_timesteps 4306.
Path 44 | total_timesteps 4398.
Path 45 | total_timesteps 4496.
Path 46 | total_timesteps 4596.
Path 47 | total_timesteps 4679.
Path 48 | total_timesteps 4760.
Path 49 | total_timesteps 4851.
Path 50 | total_timesteps 4941.
Path 51 | total_timesteps 5024.
Path 52 | total_timesteps 5107.
Path 53 | total_timesteps 5204.
Path 54 | total_timesteps 5317.
Path 55 | total_timesteps 5427.
Path 56 | total_timesteps 5530.
Path 57 | total_timesteps 5606.
Path 58 | total_timesteps 5710.
Path 59 | total_timesteps 5828.
Path 60 | total_timesteps 5917.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -77.6    |
| Iteration     | 29       |
| MaximumReturn | -54.1    |
| MinimumReturn | -106     |
| TotalSamples  | 124680   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.37908998131752014
Validation loss = 0.380977988243103
Validation loss = 0.3812275230884552
Validation loss = 0.3827403485774994
Validation loss = 0.38232800364494324
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 92.
Path 2 | total_timesteps 182.
Path 3 | total_timesteps 264.
Path 4 | total_timesteps 379.
Path 5 | total_timesteps 510.
Path 6 | total_timesteps 608.
Path 7 | total_timesteps 695.
Path 8 | total_timesteps 787.
Path 9 | total_timesteps 900.
Path 10 | total_timesteps 980.
Path 11 | total_timesteps 1079.
Path 12 | total_timesteps 1192.
Path 13 | total_timesteps 1276.
Path 14 | total_timesteps 1373.
Path 15 | total_timesteps 1515.
Path 16 | total_timesteps 1616.
Path 17 | total_timesteps 1700.
Path 18 | total_timesteps 1803.
Path 19 | total_timesteps 1894.
Path 20 | total_timesteps 1997.
Path 21 | total_timesteps 2115.
Path 22 | total_timesteps 2221.
Path 23 | total_timesteps 2311.
Path 24 | total_timesteps 2469.
Path 25 | total_timesteps 2567.
Path 26 | total_timesteps 2663.
Path 27 | total_timesteps 2761.
Path 28 | total_timesteps 2866.
Path 29 | total_timesteps 2967.
Path 30 | total_timesteps 3066.
Path 31 | total_timesteps 3157.
Path 32 | total_timesteps 3250.
Path 33 | total_timesteps 3322.
Path 34 | total_timesteps 3411.
Path 35 | total_timesteps 3724.
Path 36 | total_timesteps 3816.
Path 37 | total_timesteps 3915.
Path 38 | total_timesteps 4023.
Path 39 | total_timesteps 4145.
Path 40 | total_timesteps 4248.
Path 41 | total_timesteps 4358.
Path 42 | total_timesteps 4452.
Path 43 | total_timesteps 4537.
Path 44 | total_timesteps 4662.
Path 45 | total_timesteps 4765.
Path 46 | total_timesteps 4839.
Path 47 | total_timesteps 4926.
Path 48 | total_timesteps 5035.
Path 49 | total_timesteps 5108.
Path 50 | total_timesteps 5208.
Path 51 | total_timesteps 5301.
Path 52 | total_timesteps 5397.
Path 53 | total_timesteps 5487.
Path 54 | total_timesteps 5592.
Path 55 | total_timesteps 5709.
Path 56 | total_timesteps 5789.
Path 57 | total_timesteps 5889.
Path 58 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -76.1    |
| Iteration     | 30       |
| MaximumReturn | 11.6     |
| MinimumReturn | -103     |
| TotalSamples  | 128723   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.37817758321762085
Validation loss = 0.3838779926300049
Validation loss = 0.3853784203529358
Validation loss = 0.38504862785339355
Validation loss = 0.3855366110801697
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 102.
Path 2 | total_timesteps 193.
Path 3 | total_timesteps 289.
Path 4 | total_timesteps 398.
Path 5 | total_timesteps 494.
Path 6 | total_timesteps 611.
Path 7 | total_timesteps 709.
Path 8 | total_timesteps 788.
Path 9 | total_timesteps 875.
Path 10 | total_timesteps 979.
Path 11 | total_timesteps 1071.
Path 12 | total_timesteps 1155.
Path 13 | total_timesteps 1282.
Path 14 | total_timesteps 1374.
Path 15 | total_timesteps 1465.
Path 16 | total_timesteps 1550.
Path 17 | total_timesteps 1644.
Path 18 | total_timesteps 1743.
Path 19 | total_timesteps 1868.
Path 20 | total_timesteps 1957.
Path 21 | total_timesteps 2037.
Path 22 | total_timesteps 2171.
Path 23 | total_timesteps 2281.
Path 24 | total_timesteps 2372.
Path 25 | total_timesteps 2466.
Path 26 | total_timesteps 2588.
Path 27 | total_timesteps 2682.
Path 28 | total_timesteps 2778.
Path 29 | total_timesteps 2868.
Path 30 | total_timesteps 2980.
Path 31 | total_timesteps 3093.
Path 32 | total_timesteps 3196.
Path 33 | total_timesteps 3283.
Path 34 | total_timesteps 3374.
Path 35 | total_timesteps 3473.
Path 36 | total_timesteps 3564.
Path 37 | total_timesteps 3659.
Path 38 | total_timesteps 3755.
Path 39 | total_timesteps 3860.
Path 40 | total_timesteps 3950.
Path 41 | total_timesteps 4047.
Path 42 | total_timesteps 4155.
Path 43 | total_timesteps 4251.
Path 44 | total_timesteps 4345.
Path 45 | total_timesteps 4433.
Path 46 | total_timesteps 4527.
Path 47 | total_timesteps 4629.
Path 48 | total_timesteps 4740.
Path 49 | total_timesteps 4854.
Path 50 | total_timesteps 4942.
Path 51 | total_timesteps 5021.
Path 52 | total_timesteps 5115.
Path 53 | total_timesteps 5208.
Path 54 | total_timesteps 5297.
Path 55 | total_timesteps 5425.
Path 56 | total_timesteps 5535.
Path 57 | total_timesteps 5638.
Path 58 | total_timesteps 5726.
Path 59 | total_timesteps 5811.
Path 60 | total_timesteps 5913.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -77.3    |
| Iteration     | 31       |
| MaximumReturn | -43.2    |
| MinimumReturn | -113     |
| TotalSamples  | 132724   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.38325580954551697
Validation loss = 0.38467633724212646
Validation loss = 0.3895981013774872
Validation loss = 0.38805532455444336
Validation loss = 0.3894555866718292
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 94.
Path 2 | total_timesteps 252.
Path 3 | total_timesteps 357.
Path 4 | total_timesteps 456.
Path 5 | total_timesteps 574.
Path 6 | total_timesteps 670.
Path 7 | total_timesteps 768.
Path 8 | total_timesteps 874.
Path 9 | total_timesteps 963.
Path 10 | total_timesteps 1081.
Path 11 | total_timesteps 1176.
Path 12 | total_timesteps 1264.
Path 13 | total_timesteps 1357.
Path 14 | total_timesteps 1466.
Path 15 | total_timesteps 1551.
Path 16 | total_timesteps 1653.
Path 17 | total_timesteps 1755.
Path 18 | total_timesteps 1850.
Path 19 | total_timesteps 1961.
Path 20 | total_timesteps 2055.
Path 21 | total_timesteps 2165.
Path 22 | total_timesteps 2272.
Path 23 | total_timesteps 2363.
Path 24 | total_timesteps 2441.
Path 25 | total_timesteps 2530.
Path 26 | total_timesteps 2616.
Path 27 | total_timesteps 2712.
Path 28 | total_timesteps 2817.
Path 29 | total_timesteps 2917.
Path 30 | total_timesteps 3007.
Path 31 | total_timesteps 3106.
Path 32 | total_timesteps 3205.
Path 33 | total_timesteps 3285.
Path 34 | total_timesteps 3391.
Path 35 | total_timesteps 3491.
Path 36 | total_timesteps 3584.
Path 37 | total_timesteps 3692.
Path 38 | total_timesteps 3792.
Path 39 | total_timesteps 3879.
Path 40 | total_timesteps 3980.
Path 41 | total_timesteps 4076.
Path 42 | total_timesteps 4193.
Path 43 | total_timesteps 4309.
Path 44 | total_timesteps 4400.
Path 45 | total_timesteps 4513.
Path 46 | total_timesteps 4606.
Path 47 | total_timesteps 4700.
Path 48 | total_timesteps 4800.
Path 49 | total_timesteps 4890.
Path 50 | total_timesteps 4997.
Path 51 | total_timesteps 5100.
Path 52 | total_timesteps 5212.
Path 53 | total_timesteps 5295.
Path 54 | total_timesteps 5389.
Path 55 | total_timesteps 5486.
Path 56 | total_timesteps 5600.
Path 57 | total_timesteps 5691.
Path 58 | total_timesteps 5781.
Path 59 | total_timesteps 5887.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -79.8    |
| Iteration     | 32       |
| MaximumReturn | -43.3    |
| MinimumReturn | -121     |
| TotalSamples  | 136737   |
----------------------------
