Logging to experiments/gym_fwalker2d/W/Mon-07-Nov-2022-10-28-41-AM-CST_gym_fwalker2d_trpo_iteration_20_seed3214
Print configuration .....
{'env_name': 'gym_fwalker2d', 'random_seeds': [3214, 2431, 2531, 2231], 'save_variables': False, 'model_save_dir': '/tmp/gym_fwalker2d_models/', 'restore_variables': False, 'start_onpol_iter': 0, 'onpol_iters': 33, 'num_path_random': 6, 'num_path_onpol': 6, 'env_horizon': 1000, 'max_train_data': 200000, 'max_val_data': 100000, 'discard_ratio': 0.0, 'dynamics': {'pre_training': {'mode': 'intrinsic_reward', 'itr': 0, 'policy_itr': 20}, 'model': 'nn', 'ensemble': False, 'ensemble_model_count': 5, 'enable_particle_ensemble': True, 'particles': 5, 'obs_var': 1.0, 'intrinsic_reward_coeff': 1.0, 'ita': 1.0, 'mode': 'random', 'val': True, 'n_layers': 4, 'hidden_size': 1000, 'activation': 'relu', 'batch_size': 1000, 'learning_rate': 0.001, 'reg_coeff': 0.0, 'epochs': 200, 'kfac_params': {'learning_rate': 0.1, 'damping': 0.001, 'momentum': 0.9, 'kl_clip': 0.0001, 'cov_ema_decay': 0.99}}, 'policy': {'network_shape': [64, 64], 'init_logstd': 0.0, 'activation': 'tanh', 'reinitialize_every_itr': False}, 'trpo': {'horizon': 1000, 'gamma': 0.99, 'step_size': 0.01, 'iterations': 20, 'batch_size': 50000, 'gae': 0.95, 'visualization': False, 'visualize_iterations': [0]}, 'algo': 'trpo'}
Generating random rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 40.
Path 3 | total_timesteps 55.
Path 4 | total_timesteps 65.
Path 5 | total_timesteps 76.
Path 6 | total_timesteps 91.
Path 7 | total_timesteps 110.
Path 8 | total_timesteps 124.
Path 9 | total_timesteps 149.
Path 10 | total_timesteps 165.
Path 11 | total_timesteps 178.
Path 12 | total_timesteps 198.
Path 13 | total_timesteps 219.
Path 14 | total_timesteps 258.
Path 15 | total_timesteps 271.
Path 16 | total_timesteps 293.
Path 17 | total_timesteps 320.
Path 18 | total_timesteps 336.
Path 19 | total_timesteps 354.
Path 20 | total_timesteps 393.
Path 21 | total_timesteps 419.
Path 22 | total_timesteps 436.
Path 23 | total_timesteps 457.
Path 24 | total_timesteps 466.
Path 25 | total_timesteps 499.
Path 26 | total_timesteps 523.
Path 27 | total_timesteps 542.
Path 28 | total_timesteps 564.
Path 29 | total_timesteps 580.
Path 30 | total_timesteps 600.
Path 31 | total_timesteps 624.
Path 32 | total_timesteps 646.
Path 33 | total_timesteps 660.
Path 34 | total_timesteps 670.
Path 35 | total_timesteps 689.
Path 36 | total_timesteps 715.
Path 37 | total_timesteps 734.
Path 38 | total_timesteps 768.
Path 39 | total_timesteps 786.
Path 40 | total_timesteps 799.
Path 41 | total_timesteps 810.
Path 42 | total_timesteps 830.
Path 43 | total_timesteps 845.
Path 44 | total_timesteps 859.
Path 45 | total_timesteps 869.
Path 46 | total_timesteps 894.
Path 47 | total_timesteps 906.
Path 48 | total_timesteps 923.
Path 49 | total_timesteps 940.
Path 50 | total_timesteps 958.
Path 51 | total_timesteps 977.
Path 52 | total_timesteps 992.
Path 53 | total_timesteps 1016.
Path 54 | total_timesteps 1030.
Path 55 | total_timesteps 1053.
Path 56 | total_timesteps 1071.
Path 57 | total_timesteps 1088.
Path 58 | total_timesteps 1110.
Path 59 | total_timesteps 1131.
Path 60 | total_timesteps 1145.
Path 61 | total_timesteps 1163.
Path 62 | total_timesteps 1188.
Path 63 | total_timesteps 1201.
Path 64 | total_timesteps 1220.
Path 65 | total_timesteps 1254.
Path 66 | total_timesteps 1268.
Path 67 | total_timesteps 1292.
Path 68 | total_timesteps 1314.
Path 69 | total_timesteps 1325.
Path 70 | total_timesteps 1341.
Path 71 | total_timesteps 1354.
Path 72 | total_timesteps 1374.
Path 73 | total_timesteps 1383.
Path 74 | total_timesteps 1394.
Path 75 | total_timesteps 1417.
Path 76 | total_timesteps 1427.
Path 77 | total_timesteps 1460.
Path 78 | total_timesteps 1483.
Path 79 | total_timesteps 1504.
Path 80 | total_timesteps 1521.
Path 81 | total_timesteps 1537.
Path 82 | total_timesteps 1545.
Path 83 | total_timesteps 1570.
Path 84 | total_timesteps 1595.
Path 85 | total_timesteps 1610.
Path 86 | total_timesteps 1664.
Path 87 | total_timesteps 1688.
Path 88 | total_timesteps 1709.
Path 89 | total_timesteps 1729.
Path 90 | total_timesteps 1756.
Path 91 | total_timesteps 1778.
Path 92 | total_timesteps 1794.
Path 93 | total_timesteps 1834.
Path 94 | total_timesteps 1852.
Path 95 | total_timesteps 1876.
Path 96 | total_timesteps 1895.
Path 97 | total_timesteps 1915.
Path 98 | total_timesteps 1940.
Path 99 | total_timesteps 1963.
Path 100 | total_timesteps 1985.
Path 101 | total_timesteps 2011.
Path 102 | total_timesteps 2026.
Path 103 | total_timesteps 2040.
Path 104 | total_timesteps 2056.
Path 105 | total_timesteps 2068.
Path 106 | total_timesteps 2085.
Path 107 | total_timesteps 2095.
Path 108 | total_timesteps 2109.
Path 109 | total_timesteps 2120.
Path 110 | total_timesteps 2171.
Path 111 | total_timesteps 2192.
Path 112 | total_timesteps 2209.
Path 113 | total_timesteps 2236.
Path 114 | total_timesteps 2257.
Path 115 | total_timesteps 2280.
Path 116 | total_timesteps 2313.
Path 117 | total_timesteps 2331.
Path 118 | total_timesteps 2343.
Path 119 | total_timesteps 2380.
Path 120 | total_timesteps 2400.
Path 121 | total_timesteps 2413.
Path 122 | total_timesteps 2436.
Path 123 | total_timesteps 2448.
Path 124 | total_timesteps 2466.
Path 125 | total_timesteps 2489.
Path 126 | total_timesteps 2509.
Path 127 | total_timesteps 2540.
Path 128 | total_timesteps 2556.
Path 129 | total_timesteps 2572.
Path 130 | total_timesteps 2587.
Path 131 | total_timesteps 2607.
Path 132 | total_timesteps 2625.
Path 133 | total_timesteps 2638.
Path 134 | total_timesteps 2655.
Path 135 | total_timesteps 2678.
Path 136 | total_timesteps 2702.
Path 137 | total_timesteps 2724.
Path 138 | total_timesteps 2766.
Path 139 | total_timesteps 2775.
Path 140 | total_timesteps 2794.
Path 141 | total_timesteps 2815.
Path 142 | total_timesteps 2827.
Path 143 | total_timesteps 2848.
Path 144 | total_timesteps 2869.
Path 145 | total_timesteps 2893.
Path 146 | total_timesteps 2922.
Path 147 | total_timesteps 2935.
Path 148 | total_timesteps 2952.
Path 149 | total_timesteps 2982.
Path 150 | total_timesteps 3009.
Path 151 | total_timesteps 3029.
Path 152 | total_timesteps 3057.
Path 153 | total_timesteps 3071.
Path 154 | total_timesteps 3082.
Path 155 | total_timesteps 3092.
Path 156 | total_timesteps 3111.
Path 157 | total_timesteps 3130.
Path 158 | total_timesteps 3147.
Path 159 | total_timesteps 3167.
Path 160 | total_timesteps 3196.
Path 161 | total_timesteps 3213.
Path 162 | total_timesteps 3233.
Path 163 | total_timesteps 3247.
Path 164 | total_timesteps 3265.
Path 165 | total_timesteps 3276.
Path 166 | total_timesteps 3297.
Path 167 | total_timesteps 3320.
Path 168 | total_timesteps 3334.
Path 169 | total_timesteps 3345.
Path 170 | total_timesteps 3373.
Path 171 | total_timesteps 3387.
Path 172 | total_timesteps 3404.
Path 173 | total_timesteps 3418.
Path 174 | total_timesteps 3434.
Path 175 | total_timesteps 3451.
Path 176 | total_timesteps 3465.
Path 177 | total_timesteps 3491.
Path 178 | total_timesteps 3519.
Path 179 | total_timesteps 3548.
Path 180 | total_timesteps 3567.
Path 181 | total_timesteps 3586.
Path 182 | total_timesteps 3607.
Path 183 | total_timesteps 3629.
Path 184 | total_timesteps 3643.
Path 185 | total_timesteps 3654.
Path 186 | total_timesteps 3677.
Path 187 | total_timesteps 3702.
Path 188 | total_timesteps 3727.
Path 189 | total_timesteps 3743.
Path 190 | total_timesteps 3762.
Path 191 | total_timesteps 3775.
Path 192 | total_timesteps 3797.
Path 193 | total_timesteps 3818.
Path 194 | total_timesteps 3840.
Path 195 | total_timesteps 3865.
Path 196 | total_timesteps 3882.
Path 197 | total_timesteps 3895.
Path 198 | total_timesteps 3907.
Path 199 | total_timesteps 3923.
Path 200 | total_timesteps 3949.
Path 201 | total_timesteps 3956.
Path 202 | total_timesteps 3968.
Path 203 | total_timesteps 3979.
Path 204 | total_timesteps 3996.
Path 205 | total_timesteps 4022.
Path 206 | total_timesteps 4046.
Path 207 | total_timesteps 4060.
Path 208 | total_timesteps 4077.
Path 209 | total_timesteps 4096.
Path 210 | total_timesteps 4121.
Path 211 | total_timesteps 4151.
Path 212 | total_timesteps 4163.
Path 213 | total_timesteps 4188.
Path 214 | total_timesteps 4195.
Path 215 | total_timesteps 4216.
Path 216 | total_timesteps 4235.
Path 217 | total_timesteps 4255.
Path 218 | total_timesteps 4273.
Path 219 | total_timesteps 4299.
Path 220 | total_timesteps 4314.
Path 221 | total_timesteps 4330.
Path 222 | total_timesteps 4370.
Path 223 | total_timesteps 4397.
Path 224 | total_timesteps 4420.
Path 225 | total_timesteps 4430.
Path 226 | total_timesteps 4451.
Path 227 | total_timesteps 4465.
Path 228 | total_timesteps 4501.
Path 229 | total_timesteps 4531.
Path 230 | total_timesteps 4561.
Path 231 | total_timesteps 4582.
Path 232 | total_timesteps 4603.
Path 233 | total_timesteps 4623.
Path 234 | total_timesteps 4664.
Path 235 | total_timesteps 4681.
Path 236 | total_timesteps 4694.
Path 237 | total_timesteps 4707.
Path 238 | total_timesteps 4749.
Path 239 | total_timesteps 4774.
Path 240 | total_timesteps 4796.
Path 241 | total_timesteps 4828.
Path 242 | total_timesteps 4839.
Path 243 | total_timesteps 4853.
Path 244 | total_timesteps 4873.
Path 245 | total_timesteps 4898.
Path 246 | total_timesteps 4923.
Path 247 | total_timesteps 4942.
Path 248 | total_timesteps 4959.
Path 249 | total_timesteps 4993.
Path 250 | total_timesteps 5017.
Path 251 | total_timesteps 5029.
Path 252 | total_timesteps 5045.
Path 253 | total_timesteps 5055.
Path 254 | total_timesteps 5069.
Path 255 | total_timesteps 5082.
Path 256 | total_timesteps 5105.
Path 257 | total_timesteps 5115.
Path 258 | total_timesteps 5138.
Path 259 | total_timesteps 5155.
Path 260 | total_timesteps 5167.
Path 261 | total_timesteps 5193.
Path 262 | total_timesteps 5220.
Path 263 | total_timesteps 5233.
Path 264 | total_timesteps 5260.
Path 265 | total_timesteps 5286.
Path 266 | total_timesteps 5307.
Path 267 | total_timesteps 5319.
Path 268 | total_timesteps 5349.
Path 269 | total_timesteps 5369.
Path 270 | total_timesteps 5392.
Path 271 | total_timesteps 5427.
Path 272 | total_timesteps 5440.
Path 273 | total_timesteps 5458.
Path 274 | total_timesteps 5477.
Path 275 | total_timesteps 5493.
Path 276 | total_timesteps 5510.
Path 277 | total_timesteps 5532.
Path 278 | total_timesteps 5560.
Path 279 | total_timesteps 5580.
Path 280 | total_timesteps 5589.
Path 281 | total_timesteps 5604.
Path 282 | total_timesteps 5632.
Path 283 | total_timesteps 5652.
Path 284 | total_timesteps 5667.
Path 285 | total_timesteps 5704.
Path 286 | total_timesteps 5729.
Path 287 | total_timesteps 5749.
Path 288 | total_timesteps 5771.
Path 289 | total_timesteps 5794.
Path 290 | total_timesteps 5821.
Path 291 | total_timesteps 5848.
Path 292 | total_timesteps 5875.
Path 293 | total_timesteps 5889.
Path 294 | total_timesteps 5917.
Path 295 | total_timesteps 5929.
Path 296 | total_timesteps 5945.
Path 297 | total_timesteps 5962.
Path 298 | total_timesteps 5978.
Path 299 | total_timesteps 5995.
Done generating random rollouts.
Creating normalization for training data.
Done creating normalization for training data.
Train dynamics model with intrinsic reward only? False
Pre-training enabled. Using only intrinsic reward.
Pre-training dynamics model for 0 iterations...
Done pre-training dynamics model.
Using external reward only.
itr #0 | 
Fitting dynamics.
Validation loss = 0.5096908211708069
Validation loss = 0.12155133485794067
Validation loss = 0.09314990043640137
Validation loss = 0.08557695895433426
Validation loss = 0.0759066790342331
Validation loss = 0.0698205828666687
Validation loss = 0.06545142829418182
Validation loss = 0.06336560845375061
Validation loss = 0.057106487452983856
Validation loss = 0.05453263223171234
Validation loss = 0.05410988628864288
Validation loss = 0.0696793794631958
Validation loss = 0.0496666356921196
Validation loss = 0.048590198159217834
Validation loss = 0.049679048359394073
Validation loss = 0.04876793920993805
Validation loss = 0.0526120588183403
Validation loss = 0.04589543864130974
Validation loss = 0.04638070613145828
Validation loss = 0.04877597093582153
Validation loss = 0.04694363474845886
Validation loss = 0.051613472402095795
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 68.
Path 5 | total_timesteps 94.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 127.
Path 8 | total_timesteps 139.
Path 9 | total_timesteps 155.
Path 10 | total_timesteps 182.
Path 11 | total_timesteps 198.
Path 12 | total_timesteps 207.
Path 13 | total_timesteps 216.
Path 14 | total_timesteps 247.
Path 15 | total_timesteps 259.
Path 16 | total_timesteps 272.
Path 17 | total_timesteps 287.
Path 18 | total_timesteps 302.
Path 19 | total_timesteps 318.
Path 20 | total_timesteps 327.
Path 21 | total_timesteps 349.
Path 22 | total_timesteps 362.
Path 23 | total_timesteps 370.
Path 24 | total_timesteps 382.
Path 25 | total_timesteps 408.
Path 26 | total_timesteps 435.
Path 27 | total_timesteps 449.
Path 28 | total_timesteps 468.
Path 29 | total_timesteps 479.
Path 30 | total_timesteps 499.
Path 31 | total_timesteps 514.
Path 32 | total_timesteps 535.
Path 33 | total_timesteps 550.
Path 34 | total_timesteps 568.
Path 35 | total_timesteps 582.
Path 36 | total_timesteps 593.
Path 37 | total_timesteps 615.
Path 38 | total_timesteps 635.
Path 39 | total_timesteps 647.
Path 40 | total_timesteps 661.
Path 41 | total_timesteps 680.
Path 42 | total_timesteps 698.
Path 43 | total_timesteps 724.
Path 44 | total_timesteps 741.
Path 45 | total_timesteps 756.
Path 46 | total_timesteps 764.
Path 47 | total_timesteps 771.
Path 48 | total_timesteps 783.
Path 49 | total_timesteps 797.
Path 50 | total_timesteps 814.
Path 51 | total_timesteps 835.
Path 52 | total_timesteps 856.
Path 53 | total_timesteps 866.
Path 54 | total_timesteps 886.
Path 55 | total_timesteps 895.
Path 56 | total_timesteps 910.
Path 57 | total_timesteps 927.
Path 58 | total_timesteps 939.
Path 59 | total_timesteps 953.
Path 60 | total_timesteps 966.
Path 61 | total_timesteps 996.
Path 62 | total_timesteps 1005.
Path 63 | total_timesteps 1020.
Path 64 | total_timesteps 1037.
Path 65 | total_timesteps 1050.
Path 66 | total_timesteps 1068.
Path 67 | total_timesteps 1082.
Path 68 | total_timesteps 1104.
Path 69 | total_timesteps 1113.
Path 70 | total_timesteps 1130.
Path 71 | total_timesteps 1139.
Path 72 | total_timesteps 1157.
Path 73 | total_timesteps 1174.
Path 74 | total_timesteps 1200.
Path 75 | total_timesteps 1212.
Path 76 | total_timesteps 1226.
Path 77 | total_timesteps 1246.
Path 78 | total_timesteps 1275.
Path 79 | total_timesteps 1296.
Path 80 | total_timesteps 1312.
Path 81 | total_timesteps 1326.
Path 82 | total_timesteps 1352.
Path 83 | total_timesteps 1361.
Path 84 | total_timesteps 1376.
Path 85 | total_timesteps 1394.
Path 86 | total_timesteps 1406.
Path 87 | total_timesteps 1423.
Path 88 | total_timesteps 1433.
Path 89 | total_timesteps 1457.
Path 90 | total_timesteps 1471.
Path 91 | total_timesteps 1483.
Path 92 | total_timesteps 1506.
Path 93 | total_timesteps 1531.
Path 94 | total_timesteps 1546.
Path 95 | total_timesteps 1565.
Path 96 | total_timesteps 1584.
Path 97 | total_timesteps 1594.
Path 98 | total_timesteps 1614.
Path 99 | total_timesteps 1634.
Path 100 | total_timesteps 1655.
Path 101 | total_timesteps 1671.
Path 102 | total_timesteps 1699.
Path 103 | total_timesteps 1709.
Path 104 | total_timesteps 1726.
Path 105 | total_timesteps 1749.
Path 106 | total_timesteps 1767.
Path 107 | total_timesteps 1788.
Path 108 | total_timesteps 1804.
Path 109 | total_timesteps 1811.
Path 110 | total_timesteps 1824.
Path 111 | total_timesteps 1833.
Path 112 | total_timesteps 1851.
Path 113 | total_timesteps 1868.
Path 114 | total_timesteps 1878.
Path 115 | total_timesteps 1898.
Path 116 | total_timesteps 1912.
Path 117 | total_timesteps 1924.
Path 118 | total_timesteps 1945.
Path 119 | total_timesteps 1957.
Path 120 | total_timesteps 1976.
Path 121 | total_timesteps 1991.
Path 122 | total_timesteps 2007.
Path 123 | total_timesteps 2015.
Path 124 | total_timesteps 2042.
Path 125 | total_timesteps 2051.
Path 126 | total_timesteps 2063.
Path 127 | total_timesteps 2084.
Path 128 | total_timesteps 2103.
Path 129 | total_timesteps 2125.
Path 130 | total_timesteps 2138.
Path 131 | total_timesteps 2155.
Path 132 | total_timesteps 2171.
Path 133 | total_timesteps 2195.
Path 134 | total_timesteps 2206.
Path 135 | total_timesteps 2231.
Path 136 | total_timesteps 2252.
Path 137 | total_timesteps 2268.
Path 138 | total_timesteps 2278.
Path 139 | total_timesteps 2293.
Path 140 | total_timesteps 2306.
Path 141 | total_timesteps 2326.
Path 142 | total_timesteps 2361.
Path 143 | total_timesteps 2378.
Path 144 | total_timesteps 2387.
Path 145 | total_timesteps 2403.
Path 146 | total_timesteps 2419.
Path 147 | total_timesteps 2432.
Path 148 | total_timesteps 2445.
Path 149 | total_timesteps 2455.
Path 150 | total_timesteps 2473.
Path 151 | total_timesteps 2487.
Path 152 | total_timesteps 2513.
Path 153 | total_timesteps 2525.
Path 154 | total_timesteps 2547.
Path 155 | total_timesteps 2564.
Path 156 | total_timesteps 2578.
Path 157 | total_timesteps 2601.
Path 158 | total_timesteps 2617.
Path 159 | total_timesteps 2642.
Path 160 | total_timesteps 2657.
Path 161 | total_timesteps 2672.
Path 162 | total_timesteps 2691.
Path 163 | total_timesteps 2707.
Path 164 | total_timesteps 2715.
Path 165 | total_timesteps 2749.
Path 166 | total_timesteps 2763.
Path 167 | total_timesteps 2774.
Path 168 | total_timesteps 2794.
Path 169 | total_timesteps 2807.
Path 170 | total_timesteps 2825.
Path 171 | total_timesteps 2841.
Path 172 | total_timesteps 2851.
Path 173 | total_timesteps 2867.
Path 174 | total_timesteps 2877.
Path 175 | total_timesteps 2900.
Path 176 | total_timesteps 2909.
Path 177 | total_timesteps 2924.
Path 178 | total_timesteps 2950.
Path 179 | total_timesteps 2962.
Path 180 | total_timesteps 2980.
Path 181 | total_timesteps 3000.
Path 182 | total_timesteps 3011.
Path 183 | total_timesteps 3043.
Path 184 | total_timesteps 3058.
Path 185 | total_timesteps 3092.
Path 186 | total_timesteps 3103.
Path 187 | total_timesteps 3121.
Path 188 | total_timesteps 3136.
Path 189 | total_timesteps 3153.
Path 190 | total_timesteps 3164.
Path 191 | total_timesteps 3178.
Path 192 | total_timesteps 3195.
Path 193 | total_timesteps 3205.
Path 194 | total_timesteps 3220.
Path 195 | total_timesteps 3240.
Path 196 | total_timesteps 3248.
Path 197 | total_timesteps 3273.
Path 198 | total_timesteps 3300.
Path 199 | total_timesteps 3325.
Path 200 | total_timesteps 3336.
Path 201 | total_timesteps 3350.
Path 202 | total_timesteps 3363.
Path 203 | total_timesteps 3379.
Path 204 | total_timesteps 3392.
Path 205 | total_timesteps 3417.
Path 206 | total_timesteps 3436.
Path 207 | total_timesteps 3446.
Path 208 | total_timesteps 3472.
Path 209 | total_timesteps 3496.
Path 210 | total_timesteps 3509.
Path 211 | total_timesteps 3518.
Path 212 | total_timesteps 3555.
Path 213 | total_timesteps 3576.
Path 214 | total_timesteps 3594.
Path 215 | total_timesteps 3607.
Path 216 | total_timesteps 3622.
Path 217 | total_timesteps 3632.
Path 218 | total_timesteps 3656.
Path 219 | total_timesteps 3685.
Path 220 | total_timesteps 3728.
Path 221 | total_timesteps 3740.
Path 222 | total_timesteps 3758.
Path 223 | total_timesteps 3790.
Path 224 | total_timesteps 3815.
Path 225 | total_timesteps 3836.
Path 226 | total_timesteps 3854.
Path 227 | total_timesteps 3870.
Path 228 | total_timesteps 3887.
Path 229 | total_timesteps 3907.
Path 230 | total_timesteps 3930.
Path 231 | total_timesteps 3941.
Path 232 | total_timesteps 3950.
Path 233 | total_timesteps 3971.
Path 234 | total_timesteps 3997.
Path 235 | total_timesteps 4015.
Path 236 | total_timesteps 4031.
Path 237 | total_timesteps 4044.
Path 238 | total_timesteps 4065.
Path 239 | total_timesteps 4075.
Path 240 | total_timesteps 4108.
Path 241 | total_timesteps 4126.
Path 242 | total_timesteps 4144.
Path 243 | total_timesteps 4160.
Path 244 | total_timesteps 4174.
Path 245 | total_timesteps 4188.
Path 246 | total_timesteps 4206.
Path 247 | total_timesteps 4221.
Path 248 | total_timesteps 4240.
Path 249 | total_timesteps 4248.
Path 250 | total_timesteps 4256.
Path 251 | total_timesteps 4269.
Path 252 | total_timesteps 4293.
Path 253 | total_timesteps 4323.
Path 254 | total_timesteps 4337.
Path 255 | total_timesteps 4351.
Path 256 | total_timesteps 4371.
Path 257 | total_timesteps 4392.
Path 258 | total_timesteps 4406.
Path 259 | total_timesteps 4415.
Path 260 | total_timesteps 4424.
Path 261 | total_timesteps 4437.
Path 262 | total_timesteps 4454.
Path 263 | total_timesteps 4475.
Path 264 | total_timesteps 4491.
Path 265 | total_timesteps 4505.
Path 266 | total_timesteps 4536.
Path 267 | total_timesteps 4553.
Path 268 | total_timesteps 4575.
Path 269 | total_timesteps 4602.
Path 270 | total_timesteps 4614.
Path 271 | total_timesteps 4629.
Path 272 | total_timesteps 4648.
Path 273 | total_timesteps 4660.
Path 274 | total_timesteps 4675.
Path 275 | total_timesteps 4691.
Path 276 | total_timesteps 4703.
Path 277 | total_timesteps 4716.
Path 278 | total_timesteps 4744.
Path 279 | total_timesteps 4753.
Path 280 | total_timesteps 4765.
Path 281 | total_timesteps 4779.
Path 282 | total_timesteps 4791.
Path 283 | total_timesteps 4804.
Path 284 | total_timesteps 4821.
Path 285 | total_timesteps 4839.
Path 286 | total_timesteps 4863.
Path 287 | total_timesteps 4888.
Path 288 | total_timesteps 4895.
Path 289 | total_timesteps 4923.
Path 290 | total_timesteps 4939.
Path 291 | total_timesteps 4948.
Path 292 | total_timesteps 4963.
Path 293 | total_timesteps 4982.
Path 294 | total_timesteps 4990.
Path 295 | total_timesteps 5014.
Path 296 | total_timesteps 5022.
Path 297 | total_timesteps 5035.
Path 298 | total_timesteps 5056.
Path 299 | total_timesteps 5069.
Path 300 | total_timesteps 5078.
Path 301 | total_timesteps 5100.
Path 302 | total_timesteps 5115.
Path 303 | total_timesteps 5129.
Path 304 | total_timesteps 5138.
Path 305 | total_timesteps 5175.
Path 306 | total_timesteps 5202.
Path 307 | total_timesteps 5212.
Path 308 | total_timesteps 5231.
Path 309 | total_timesteps 5246.
Path 310 | total_timesteps 5272.
Path 311 | total_timesteps 5285.
Path 312 | total_timesteps 5298.
Path 313 | total_timesteps 5316.
Path 314 | total_timesteps 5344.
Path 315 | total_timesteps 5359.
Path 316 | total_timesteps 5384.
Path 317 | total_timesteps 5398.
Path 318 | total_timesteps 5413.
Path 319 | total_timesteps 5425.
Path 320 | total_timesteps 5436.
Path 321 | total_timesteps 5449.
Path 322 | total_timesteps 5459.
Path 323 | total_timesteps 5474.
Path 324 | total_timesteps 5489.
Path 325 | total_timesteps 5518.
Path 326 | total_timesteps 5537.
Path 327 | total_timesteps 5548.
Path 328 | total_timesteps 5559.
Path 329 | total_timesteps 5568.
Path 330 | total_timesteps 5587.
Path 331 | total_timesteps 5601.
Path 332 | total_timesteps 5617.
Path 333 | total_timesteps 5640.
Path 334 | total_timesteps 5648.
Path 335 | total_timesteps 5659.
Path 336 | total_timesteps 5687.
Path 337 | total_timesteps 5709.
Path 338 | total_timesteps 5724.
Path 339 | total_timesteps 5753.
Path 340 | total_timesteps 5777.
Path 341 | total_timesteps 5796.
Path 342 | total_timesteps 5811.
Path 343 | total_timesteps 5828.
Path 344 | total_timesteps 5839.
Path 345 | total_timesteps 5851.
Path 346 | total_timesteps 5867.
Path 347 | total_timesteps 5895.
Path 348 | total_timesteps 5904.
Path 349 | total_timesteps 5918.
Path 350 | total_timesteps 5969.
Path 351 | total_timesteps 5980.
Path 352 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.19    |
| Iteration     | 0        |
| MaximumReturn | 5.59     |
| MinimumReturn | -20.5    |
| TotalSamples  | 8035     |
----------------------------
itr #1 | 
Fitting dynamics.
Validation loss = 0.08287151157855988
Validation loss = 0.06836672127246857
Validation loss = 0.050871722400188446
Validation loss = 0.04252996668219566
Validation loss = 0.04165756329894066
Validation loss = 0.03802827000617981
Validation loss = 0.037393029779195786
Validation loss = 0.034445229917764664
Validation loss = 0.035038355737924576
Validation loss = 0.03546346724033356
Validation loss = 0.03368210047483444
Validation loss = 0.032508932054042816
Validation loss = 0.056678395718336105
Validation loss = 0.03432216867804527
Validation loss = 0.030970130115747452
Validation loss = 0.03105352818965912
Validation loss = 0.03316826745867729
Validation loss = 0.033810872584581375
Validation loss = 0.03321851044893265
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 11.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 52.
Path 6 | total_timesteps 67.
Path 7 | total_timesteps 78.
Path 8 | total_timesteps 86.
Path 9 | total_timesteps 102.
Path 10 | total_timesteps 113.
Path 11 | total_timesteps 123.
Path 12 | total_timesteps 141.
Path 13 | total_timesteps 167.
Path 14 | total_timesteps 180.
Path 15 | total_timesteps 211.
Path 16 | total_timesteps 224.
Path 17 | total_timesteps 242.
Path 18 | total_timesteps 257.
Path 19 | total_timesteps 270.
Path 20 | total_timesteps 280.
Path 21 | total_timesteps 292.
Path 22 | total_timesteps 302.
Path 23 | total_timesteps 313.
Path 24 | total_timesteps 324.
Path 25 | total_timesteps 339.
Path 26 | total_timesteps 359.
Path 27 | total_timesteps 370.
Path 28 | total_timesteps 386.
Path 29 | total_timesteps 401.
Path 30 | total_timesteps 415.
Path 31 | total_timesteps 435.
Path 32 | total_timesteps 451.
Path 33 | total_timesteps 461.
Path 34 | total_timesteps 478.
Path 35 | total_timesteps 495.
Path 36 | total_timesteps 509.
Path 37 | total_timesteps 519.
Path 38 | total_timesteps 535.
Path 39 | total_timesteps 555.
Path 40 | total_timesteps 576.
Path 41 | total_timesteps 598.
Path 42 | total_timesteps 611.
Path 43 | total_timesteps 622.
Path 44 | total_timesteps 642.
Path 45 | total_timesteps 652.
Path 46 | total_timesteps 663.
Path 47 | total_timesteps 675.
Path 48 | total_timesteps 685.
Path 49 | total_timesteps 693.
Path 50 | total_timesteps 705.
Path 51 | total_timesteps 731.
Path 52 | total_timesteps 750.
Path 53 | total_timesteps 757.
Path 54 | total_timesteps 772.
Path 55 | total_timesteps 781.
Path 56 | total_timesteps 792.
Path 57 | total_timesteps 815.
Path 58 | total_timesteps 824.
Path 59 | total_timesteps 836.
Path 60 | total_timesteps 853.
Path 61 | total_timesteps 870.
Path 62 | total_timesteps 879.
Path 63 | total_timesteps 891.
Path 64 | total_timesteps 921.
Path 65 | total_timesteps 930.
Path 66 | total_timesteps 941.
Path 67 | total_timesteps 950.
Path 68 | total_timesteps 962.
Path 69 | total_timesteps 972.
Path 70 | total_timesteps 992.
Path 71 | total_timesteps 1015.
Path 72 | total_timesteps 1030.
Path 73 | total_timesteps 1045.
Path 74 | total_timesteps 1063.
Path 75 | total_timesteps 1077.
Path 76 | total_timesteps 1090.
Path 77 | total_timesteps 1104.
Path 78 | total_timesteps 1125.
Path 79 | total_timesteps 1146.
Path 80 | total_timesteps 1167.
Path 81 | total_timesteps 1190.
Path 82 | total_timesteps 1212.
Path 83 | total_timesteps 1228.
Path 84 | total_timesteps 1241.
Path 85 | total_timesteps 1257.
Path 86 | total_timesteps 1283.
Path 87 | total_timesteps 1299.
Path 88 | total_timesteps 1318.
Path 89 | total_timesteps 1328.
Path 90 | total_timesteps 1338.
Path 91 | total_timesteps 1349.
Path 92 | total_timesteps 1364.
Path 93 | total_timesteps 1391.
Path 94 | total_timesteps 1401.
Path 95 | total_timesteps 1413.
Path 96 | total_timesteps 1428.
Path 97 | total_timesteps 1444.
Path 98 | total_timesteps 1478.
Path 99 | total_timesteps 1492.
Path 100 | total_timesteps 1507.
Path 101 | total_timesteps 1514.
Path 102 | total_timesteps 1525.
Path 103 | total_timesteps 1535.
Path 104 | total_timesteps 1543.
Path 105 | total_timesteps 1560.
Path 106 | total_timesteps 1589.
Path 107 | total_timesteps 1598.
Path 108 | total_timesteps 1617.
Path 109 | total_timesteps 1637.
Path 110 | total_timesteps 1645.
Path 111 | total_timesteps 1655.
Path 112 | total_timesteps 1673.
Path 113 | total_timesteps 1686.
Path 114 | total_timesteps 1700.
Path 115 | total_timesteps 1707.
Path 116 | total_timesteps 1727.
Path 117 | total_timesteps 1742.
Path 118 | total_timesteps 1754.
Path 119 | total_timesteps 1781.
Path 120 | total_timesteps 1792.
Path 121 | total_timesteps 1802.
Path 122 | total_timesteps 1812.
Path 123 | total_timesteps 1824.
Path 124 | total_timesteps 1854.
Path 125 | total_timesteps 1864.
Path 126 | total_timesteps 1876.
Path 127 | total_timesteps 1886.
Path 128 | total_timesteps 1908.
Path 129 | total_timesteps 1929.
Path 130 | total_timesteps 1964.
Path 131 | total_timesteps 1977.
Path 132 | total_timesteps 1987.
Path 133 | total_timesteps 2006.
Path 134 | total_timesteps 2025.
Path 135 | total_timesteps 2034.
Path 136 | total_timesteps 2052.
Path 137 | total_timesteps 2068.
Path 138 | total_timesteps 2078.
Path 139 | total_timesteps 2092.
Path 140 | total_timesteps 2104.
Path 141 | total_timesteps 2130.
Path 142 | total_timesteps 2141.
Path 143 | total_timesteps 2156.
Path 144 | total_timesteps 2169.
Path 145 | total_timesteps 2179.
Path 146 | total_timesteps 2203.
Path 147 | total_timesteps 2213.
Path 148 | total_timesteps 2223.
Path 149 | total_timesteps 2239.
Path 150 | total_timesteps 2254.
Path 151 | total_timesteps 2282.
Path 152 | total_timesteps 2297.
Path 153 | total_timesteps 2333.
Path 154 | total_timesteps 2358.
Path 155 | total_timesteps 2370.
Path 156 | total_timesteps 2378.
Path 157 | total_timesteps 2395.
Path 158 | total_timesteps 2410.
Path 159 | total_timesteps 2418.
Path 160 | total_timesteps 2431.
Path 161 | total_timesteps 2444.
Path 162 | total_timesteps 2459.
Path 163 | total_timesteps 2469.
Path 164 | total_timesteps 2483.
Path 165 | total_timesteps 2493.
Path 166 | total_timesteps 2520.
Path 167 | total_timesteps 2538.
Path 168 | total_timesteps 2549.
Path 169 | total_timesteps 2565.
Path 170 | total_timesteps 2578.
Path 171 | total_timesteps 2587.
Path 172 | total_timesteps 2620.
Path 173 | total_timesteps 2641.
Path 174 | total_timesteps 2651.
Path 175 | total_timesteps 2660.
Path 176 | total_timesteps 2671.
Path 177 | total_timesteps 2682.
Path 178 | total_timesteps 2695.
Path 179 | total_timesteps 2707.
Path 180 | total_timesteps 2718.
Path 181 | total_timesteps 2748.
Path 182 | total_timesteps 2760.
Path 183 | total_timesteps 2779.
Path 184 | total_timesteps 2816.
Path 185 | total_timesteps 2826.
Path 186 | total_timesteps 2836.
Path 187 | total_timesteps 2848.
Path 188 | total_timesteps 2866.
Path 189 | total_timesteps 2876.
Path 190 | total_timesteps 2891.
Path 191 | total_timesteps 2903.
Path 192 | total_timesteps 2915.
Path 193 | total_timesteps 2938.
Path 194 | total_timesteps 2955.
Path 195 | total_timesteps 2972.
Path 196 | total_timesteps 2980.
Path 197 | total_timesteps 2994.
Path 198 | total_timesteps 3010.
Path 199 | total_timesteps 3019.
Path 200 | total_timesteps 3035.
Path 201 | total_timesteps 3050.
Path 202 | total_timesteps 3064.
Path 203 | total_timesteps 3075.
Path 204 | total_timesteps 3096.
Path 205 | total_timesteps 3108.
Path 206 | total_timesteps 3131.
Path 207 | total_timesteps 3160.
Path 208 | total_timesteps 3174.
Path 209 | total_timesteps 3205.
Path 210 | total_timesteps 3221.
Path 211 | total_timesteps 3235.
Path 212 | total_timesteps 3264.
Path 213 | total_timesteps 3283.
Path 214 | total_timesteps 3292.
Path 215 | total_timesteps 3312.
Path 216 | total_timesteps 3339.
Path 217 | total_timesteps 3350.
Path 218 | total_timesteps 3367.
Path 219 | total_timesteps 3387.
Path 220 | total_timesteps 3396.
Path 221 | total_timesteps 3407.
Path 222 | total_timesteps 3422.
Path 223 | total_timesteps 3434.
Path 224 | total_timesteps 3455.
Path 225 | total_timesteps 3467.
Path 226 | total_timesteps 3479.
Path 227 | total_timesteps 3490.
Path 228 | total_timesteps 3518.
Path 229 | total_timesteps 3531.
Path 230 | total_timesteps 3553.
Path 231 | total_timesteps 3566.
Path 232 | total_timesteps 3581.
Path 233 | total_timesteps 3593.
Path 234 | total_timesteps 3607.
Path 235 | total_timesteps 3628.
Path 236 | total_timesteps 3648.
Path 237 | total_timesteps 3669.
Path 238 | total_timesteps 3678.
Path 239 | total_timesteps 3690.
Path 240 | total_timesteps 3699.
Path 241 | total_timesteps 3709.
Path 242 | total_timesteps 3725.
Path 243 | total_timesteps 3749.
Path 244 | total_timesteps 3763.
Path 245 | total_timesteps 3775.
Path 246 | total_timesteps 3790.
Path 247 | total_timesteps 3811.
Path 248 | total_timesteps 3819.
Path 249 | total_timesteps 3834.
Path 250 | total_timesteps 3844.
Path 251 | total_timesteps 3854.
Path 252 | total_timesteps 3879.
Path 253 | total_timesteps 3896.
Path 254 | total_timesteps 3910.
Path 255 | total_timesteps 3927.
Path 256 | total_timesteps 3941.
Path 257 | total_timesteps 3966.
Path 258 | total_timesteps 3986.
Path 259 | total_timesteps 3995.
Path 260 | total_timesteps 4014.
Path 261 | total_timesteps 4032.
Path 262 | total_timesteps 4051.
Path 263 | total_timesteps 4069.
Path 264 | total_timesteps 4084.
Path 265 | total_timesteps 4110.
Path 266 | total_timesteps 4139.
Path 267 | total_timesteps 4152.
Path 268 | total_timesteps 4163.
Path 269 | total_timesteps 4185.
Path 270 | total_timesteps 4209.
Path 271 | total_timesteps 4220.
Path 272 | total_timesteps 4253.
Path 273 | total_timesteps 4267.
Path 274 | total_timesteps 4284.
Path 275 | total_timesteps 4295.
Path 276 | total_timesteps 4314.
Path 277 | total_timesteps 4331.
Path 278 | total_timesteps 4347.
Path 279 | total_timesteps 4373.
Path 280 | total_timesteps 4391.
Path 281 | total_timesteps 4412.
Path 282 | total_timesteps 4428.
Path 283 | total_timesteps 4441.
Path 284 | total_timesteps 4456.
Path 285 | total_timesteps 4475.
Path 286 | total_timesteps 4489.
Path 287 | total_timesteps 4507.
Path 288 | total_timesteps 4538.
Path 289 | total_timesteps 4559.
Path 290 | total_timesteps 4570.
Path 291 | total_timesteps 4585.
Path 292 | total_timesteps 4601.
Path 293 | total_timesteps 4610.
Path 294 | total_timesteps 4632.
Path 295 | total_timesteps 4645.
Path 296 | total_timesteps 4660.
Path 297 | total_timesteps 4667.
Path 298 | total_timesteps 4675.
Path 299 | total_timesteps 4691.
Path 300 | total_timesteps 4701.
Path 301 | total_timesteps 4716.
Path 302 | total_timesteps 4736.
Path 303 | total_timesteps 4756.
Path 304 | total_timesteps 4776.
Path 305 | total_timesteps 4791.
Path 306 | total_timesteps 4806.
Path 307 | total_timesteps 4821.
Path 308 | total_timesteps 4831.
Path 309 | total_timesteps 4847.
Path 310 | total_timesteps 4862.
Path 311 | total_timesteps 4885.
Path 312 | total_timesteps 4902.
Path 313 | total_timesteps 4920.
Path 314 | total_timesteps 4945.
Path 315 | total_timesteps 4968.
Path 316 | total_timesteps 4991.
Path 317 | total_timesteps 5005.
Path 318 | total_timesteps 5017.
Path 319 | total_timesteps 5032.
Path 320 | total_timesteps 5043.
Path 321 | total_timesteps 5057.
Path 322 | total_timesteps 5076.
Path 323 | total_timesteps 5086.
Path 324 | total_timesteps 5108.
Path 325 | total_timesteps 5129.
Path 326 | total_timesteps 5139.
Path 327 | total_timesteps 5165.
Path 328 | total_timesteps 5186.
Path 329 | total_timesteps 5200.
Path 330 | total_timesteps 5207.
Path 331 | total_timesteps 5222.
Path 332 | total_timesteps 5231.
Path 333 | total_timesteps 5252.
Path 334 | total_timesteps 5263.
Path 335 | total_timesteps 5286.
Path 336 | total_timesteps 5303.
Path 337 | total_timesteps 5315.
Path 338 | total_timesteps 5339.
Path 339 | total_timesteps 5347.
Path 340 | total_timesteps 5366.
Path 341 | total_timesteps 5383.
Path 342 | total_timesteps 5398.
Path 343 | total_timesteps 5420.
Path 344 | total_timesteps 5432.
Path 345 | total_timesteps 5450.
Path 346 | total_timesteps 5462.
Path 347 | total_timesteps 5479.
Path 348 | total_timesteps 5517.
Path 349 | total_timesteps 5531.
Path 350 | total_timesteps 5542.
Path 351 | total_timesteps 5554.
Path 352 | total_timesteps 5562.
Path 353 | total_timesteps 5579.
Path 354 | total_timesteps 5587.
Path 355 | total_timesteps 5595.
Path 356 | total_timesteps 5609.
Path 357 | total_timesteps 5618.
Path 358 | total_timesteps 5636.
Path 359 | total_timesteps 5652.
Path 360 | total_timesteps 5666.
Path 361 | total_timesteps 5693.
Path 362 | total_timesteps 5705.
Path 363 | total_timesteps 5720.
Path 364 | total_timesteps 5758.
Path 365 | total_timesteps 5777.
Path 366 | total_timesteps 5796.
Path 367 | total_timesteps 5814.
Path 368 | total_timesteps 5828.
Path 369 | total_timesteps 5841.
Path 370 | total_timesteps 5855.
Path 371 | total_timesteps 5873.
Path 372 | total_timesteps 5886.
Path 373 | total_timesteps 5894.
Path 374 | total_timesteps 5920.
Path 375 | total_timesteps 5935.
Path 376 | total_timesteps 5953.
Path 377 | total_timesteps 5964.
Path 378 | total_timesteps 5971.
Path 379 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.15    |
| Iteration     | 1        |
| MaximumReturn | 5        |
| MinimumReturn | -20.3    |
| TotalSamples  | 12042    |
----------------------------
itr #2 | 
Fitting dynamics.
Validation loss = 0.03485555574297905
Validation loss = 0.026926210150122643
Validation loss = 0.026444189250469208
Validation loss = 0.025563059374690056
Validation loss = 0.029732443392276764
Validation loss = 0.02432531863451004
Validation loss = 0.025506742298603058
Validation loss = 0.02580723911523819
Validation loss = 0.022537140175700188
Validation loss = 0.02333110384643078
Validation loss = 0.02487972378730774
Validation loss = 0.022198686376214027
Validation loss = 0.021308334544301033
Validation loss = 0.022328177466988564
Validation loss = 0.022805744782090187
Validation loss = 0.022511102259159088
Validation loss = 0.020400209352374077
Validation loss = 0.022615810856223106
Validation loss = 0.02113323099911213
Validation loss = 0.01986747793853283
Validation loss = 0.0221976637840271
Validation loss = 0.01957075670361519
Validation loss = 0.02241371013224125
Validation loss = 0.019124066457152367
Validation loss = 0.01903369277715683
Validation loss = 0.019990738481283188
Validation loss = 0.03501322492957115
Validation loss = 0.019372517243027687
Validation loss = 0.02278103493154049
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 25.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 57.
Path 4 | total_timesteps 76.
Path 5 | total_timesteps 92.
Path 6 | total_timesteps 105.
Path 7 | total_timesteps 122.
Path 8 | total_timesteps 146.
Path 9 | total_timesteps 154.
Path 10 | total_timesteps 182.
Path 11 | total_timesteps 191.
Path 12 | total_timesteps 200.
Path 13 | total_timesteps 218.
Path 14 | total_timesteps 231.
Path 15 | total_timesteps 248.
Path 16 | total_timesteps 259.
Path 17 | total_timesteps 297.
Path 18 | total_timesteps 315.
Path 19 | total_timesteps 338.
Path 20 | total_timesteps 351.
Path 21 | total_timesteps 363.
Path 22 | total_timesteps 386.
Path 23 | total_timesteps 400.
Path 24 | total_timesteps 416.
Path 25 | total_timesteps 433.
Path 26 | total_timesteps 450.
Path 27 | total_timesteps 472.
Path 28 | total_timesteps 481.
Path 29 | total_timesteps 489.
Path 30 | total_timesteps 497.
Path 31 | total_timesteps 513.
Path 32 | total_timesteps 532.
Path 33 | total_timesteps 550.
Path 34 | total_timesteps 562.
Path 35 | total_timesteps 575.
Path 36 | total_timesteps 593.
Path 37 | total_timesteps 605.
Path 38 | total_timesteps 613.
Path 39 | total_timesteps 630.
Path 40 | total_timesteps 644.
Path 41 | total_timesteps 702.
Path 42 | total_timesteps 719.
Path 43 | total_timesteps 741.
Path 44 | total_timesteps 751.
Path 45 | total_timesteps 760.
Path 46 | total_timesteps 776.
Path 47 | total_timesteps 792.
Path 48 | total_timesteps 811.
Path 49 | total_timesteps 843.
Path 50 | total_timesteps 869.
Path 51 | total_timesteps 877.
Path 52 | total_timesteps 896.
Path 53 | total_timesteps 907.
Path 54 | total_timesteps 930.
Path 55 | total_timesteps 952.
Path 56 | total_timesteps 963.
Path 57 | total_timesteps 974.
Path 58 | total_timesteps 993.
Path 59 | total_timesteps 1012.
Path 60 | total_timesteps 1021.
Path 61 | total_timesteps 1037.
Path 62 | total_timesteps 1047.
Path 63 | total_timesteps 1066.
Path 64 | total_timesteps 1079.
Path 65 | total_timesteps 1087.
Path 66 | total_timesteps 1096.
Path 67 | total_timesteps 1115.
Path 68 | total_timesteps 1137.
Path 69 | total_timesteps 1151.
Path 70 | total_timesteps 1159.
Path 71 | total_timesteps 1190.
Path 72 | total_timesteps 1203.
Path 73 | total_timesteps 1214.
Path 74 | total_timesteps 1222.
Path 75 | total_timesteps 1241.
Path 76 | total_timesteps 1251.
Path 77 | total_timesteps 1271.
Path 78 | total_timesteps 1283.
Path 79 | total_timesteps 1299.
Path 80 | total_timesteps 1326.
Path 81 | total_timesteps 1338.
Path 82 | total_timesteps 1351.
Path 83 | total_timesteps 1376.
Path 84 | total_timesteps 1401.
Path 85 | total_timesteps 1416.
Path 86 | total_timesteps 1451.
Path 87 | total_timesteps 1472.
Path 88 | total_timesteps 1498.
Path 89 | total_timesteps 1512.
Path 90 | total_timesteps 1535.
Path 91 | total_timesteps 1553.
Path 92 | total_timesteps 1565.
Path 93 | total_timesteps 1578.
Path 94 | total_timesteps 1585.
Path 95 | total_timesteps 1606.
Path 96 | total_timesteps 1635.
Path 97 | total_timesteps 1644.
Path 98 | total_timesteps 1673.
Path 99 | total_timesteps 1689.
Path 100 | total_timesteps 1703.
Path 101 | total_timesteps 1715.
Path 102 | total_timesteps 1732.
Path 103 | total_timesteps 1747.
Path 104 | total_timesteps 1767.
Path 105 | total_timesteps 1786.
Path 106 | total_timesteps 1799.
Path 107 | total_timesteps 1811.
Path 108 | total_timesteps 1833.
Path 109 | total_timesteps 1852.
Path 110 | total_timesteps 1875.
Path 111 | total_timesteps 1908.
Path 112 | total_timesteps 1931.
Path 113 | total_timesteps 1947.
Path 114 | total_timesteps 1967.
Path 115 | total_timesteps 1983.
Path 116 | total_timesteps 1996.
Path 117 | total_timesteps 2010.
Path 118 | total_timesteps 2024.
Path 119 | total_timesteps 2031.
Path 120 | total_timesteps 2046.
Path 121 | total_timesteps 2066.
Path 122 | total_timesteps 2084.
Path 123 | total_timesteps 2099.
Path 124 | total_timesteps 2131.
Path 125 | total_timesteps 2155.
Path 126 | total_timesteps 2171.
Path 127 | total_timesteps 2180.
Path 128 | total_timesteps 2211.
Path 129 | total_timesteps 2221.
Path 130 | total_timesteps 2236.
Path 131 | total_timesteps 2247.
Path 132 | total_timesteps 2256.
Path 133 | total_timesteps 2274.
Path 134 | total_timesteps 2315.
Path 135 | total_timesteps 2331.
Path 136 | total_timesteps 2366.
Path 137 | total_timesteps 2378.
Path 138 | total_timesteps 2398.
Path 139 | total_timesteps 2412.
Path 140 | total_timesteps 2426.
Path 141 | total_timesteps 2439.
Path 142 | total_timesteps 2449.
Path 143 | total_timesteps 2463.
Path 144 | total_timesteps 2476.
Path 145 | total_timesteps 2489.
Path 146 | total_timesteps 2499.
Path 147 | total_timesteps 2517.
Path 148 | total_timesteps 2532.
Path 149 | total_timesteps 2546.
Path 150 | total_timesteps 2562.
Path 151 | total_timesteps 2578.
Path 152 | total_timesteps 2586.
Path 153 | total_timesteps 2598.
Path 154 | total_timesteps 2623.
Path 155 | total_timesteps 2636.
Path 156 | total_timesteps 2651.
Path 157 | total_timesteps 2666.
Path 158 | total_timesteps 2677.
Path 159 | total_timesteps 2689.
Path 160 | total_timesteps 2705.
Path 161 | total_timesteps 2733.
Path 162 | total_timesteps 2746.
Path 163 | total_timesteps 2759.
Path 164 | total_timesteps 2778.
Path 165 | total_timesteps 2790.
Path 166 | total_timesteps 2801.
Path 167 | total_timesteps 2840.
Path 168 | total_timesteps 2861.
Path 169 | total_timesteps 2875.
Path 170 | total_timesteps 2886.
Path 171 | total_timesteps 2900.
Path 172 | total_timesteps 2934.
Path 173 | total_timesteps 2940.
Path 174 | total_timesteps 2950.
Path 175 | total_timesteps 2968.
Path 176 | total_timesteps 2991.
Path 177 | total_timesteps 3014.
Path 178 | total_timesteps 3045.
Path 179 | total_timesteps 3075.
Path 180 | total_timesteps 3083.
Path 181 | total_timesteps 3097.
Path 182 | total_timesteps 3121.
Path 183 | total_timesteps 3147.
Path 184 | total_timesteps 3159.
Path 185 | total_timesteps 3175.
Path 186 | total_timesteps 3190.
Path 187 | total_timesteps 3232.
Path 188 | total_timesteps 3243.
Path 189 | total_timesteps 3253.
Path 190 | total_timesteps 3270.
Path 191 | total_timesteps 3291.
Path 192 | total_timesteps 3313.
Path 193 | total_timesteps 3340.
Path 194 | total_timesteps 3352.
Path 195 | total_timesteps 3366.
Path 196 | total_timesteps 3386.
Path 197 | total_timesteps 3397.
Path 198 | total_timesteps 3408.
Path 199 | total_timesteps 3418.
Path 200 | total_timesteps 3439.
Path 201 | total_timesteps 3460.
Path 202 | total_timesteps 3478.
Path 203 | total_timesteps 3492.
Path 204 | total_timesteps 3502.
Path 205 | total_timesteps 3510.
Path 206 | total_timesteps 3521.
Path 207 | total_timesteps 3539.
Path 208 | total_timesteps 3554.
Path 209 | total_timesteps 3565.
Path 210 | total_timesteps 3582.
Path 211 | total_timesteps 3591.
Path 212 | total_timesteps 3612.
Path 213 | total_timesteps 3633.
Path 214 | total_timesteps 3644.
Path 215 | total_timesteps 3665.
Path 216 | total_timesteps 3680.
Path 217 | total_timesteps 3692.
Path 218 | total_timesteps 3711.
Path 219 | total_timesteps 3721.
Path 220 | total_timesteps 3731.
Path 221 | total_timesteps 3752.
Path 222 | total_timesteps 3760.
Path 223 | total_timesteps 3776.
Path 224 | total_timesteps 3796.
Path 225 | total_timesteps 3818.
Path 226 | total_timesteps 3831.
Path 227 | total_timesteps 3843.
Path 228 | total_timesteps 3851.
Path 229 | total_timesteps 3871.
Path 230 | total_timesteps 3891.
Path 231 | total_timesteps 3903.
Path 232 | total_timesteps 3919.
Path 233 | total_timesteps 3929.
Path 234 | total_timesteps 3945.
Path 235 | total_timesteps 3965.
Path 236 | total_timesteps 3988.
Path 237 | total_timesteps 3999.
Path 238 | total_timesteps 4046.
Path 239 | total_timesteps 4083.
Path 240 | total_timesteps 4096.
Path 241 | total_timesteps 4137.
Path 242 | total_timesteps 4154.
Path 243 | total_timesteps 4167.
Path 244 | total_timesteps 4181.
Path 245 | total_timesteps 4190.
Path 246 | total_timesteps 4209.
Path 247 | total_timesteps 4229.
Path 248 | total_timesteps 4235.
Path 249 | total_timesteps 4243.
Path 250 | total_timesteps 4260.
Path 251 | total_timesteps 4272.
Path 252 | total_timesteps 4304.
Path 253 | total_timesteps 4317.
Path 254 | total_timesteps 4337.
Path 255 | total_timesteps 4355.
Path 256 | total_timesteps 4387.
Path 257 | total_timesteps 4403.
Path 258 | total_timesteps 4415.
Path 259 | total_timesteps 4440.
Path 260 | total_timesteps 4454.
Path 261 | total_timesteps 4471.
Path 262 | total_timesteps 4494.
Path 263 | total_timesteps 4512.
Path 264 | total_timesteps 4522.
Path 265 | total_timesteps 4547.
Path 266 | total_timesteps 4557.
Path 267 | total_timesteps 4579.
Path 268 | total_timesteps 4605.
Path 269 | total_timesteps 4619.
Path 270 | total_timesteps 4646.
Path 271 | total_timesteps 4663.
Path 272 | total_timesteps 4676.
Path 273 | total_timesteps 4688.
Path 274 | total_timesteps 4716.
Path 275 | total_timesteps 4738.
Path 276 | total_timesteps 4755.
Path 277 | total_timesteps 4776.
Path 278 | total_timesteps 4787.
Path 279 | total_timesteps 4804.
Path 280 | total_timesteps 4823.
Path 281 | total_timesteps 4838.
Path 282 | total_timesteps 4846.
Path 283 | total_timesteps 4863.
Path 284 | total_timesteps 4876.
Path 285 | total_timesteps 4884.
Path 286 | total_timesteps 4898.
Path 287 | total_timesteps 4925.
Path 288 | total_timesteps 4939.
Path 289 | total_timesteps 4952.
Path 290 | total_timesteps 4965.
Path 291 | total_timesteps 4992.
Path 292 | total_timesteps 5007.
Path 293 | total_timesteps 5018.
Path 294 | total_timesteps 5035.
Path 295 | total_timesteps 5058.
Path 296 | total_timesteps 5073.
Path 297 | total_timesteps 5087.
Path 298 | total_timesteps 5115.
Path 299 | total_timesteps 5131.
Path 300 | total_timesteps 5151.
Path 301 | total_timesteps 5167.
Path 302 | total_timesteps 5180.
Path 303 | total_timesteps 5214.
Path 304 | total_timesteps 5228.
Path 305 | total_timesteps 5240.
Path 306 | total_timesteps 5251.
Path 307 | total_timesteps 5280.
Path 308 | total_timesteps 5303.
Path 309 | total_timesteps 5323.
Path 310 | total_timesteps 5334.
Path 311 | total_timesteps 5350.
Path 312 | total_timesteps 5363.
Path 313 | total_timesteps 5372.
Path 314 | total_timesteps 5381.
Path 315 | total_timesteps 5397.
Path 316 | total_timesteps 5416.
Path 317 | total_timesteps 5433.
Path 318 | total_timesteps 5452.
Path 319 | total_timesteps 5476.
Path 320 | total_timesteps 5489.
Path 321 | total_timesteps 5497.
Path 322 | total_timesteps 5520.
Path 323 | total_timesteps 5538.
Path 324 | total_timesteps 5570.
Path 325 | total_timesteps 5584.
Path 326 | total_timesteps 5599.
Path 327 | total_timesteps 5625.
Path 328 | total_timesteps 5634.
Path 329 | total_timesteps 5648.
Path 330 | total_timesteps 5668.
Path 331 | total_timesteps 5681.
Path 332 | total_timesteps 5695.
Path 333 | total_timesteps 5712.
Path 334 | total_timesteps 5746.
Path 335 | total_timesteps 5763.
Path 336 | total_timesteps 5773.
Path 337 | total_timesteps 5799.
Path 338 | total_timesteps 5818.
Path 339 | total_timesteps 5832.
Path 340 | total_timesteps 5840.
Path 341 | total_timesteps 5865.
Path 342 | total_timesteps 5893.
Path 343 | total_timesteps 5913.
Path 344 | total_timesteps 5932.
Path 345 | total_timesteps 5943.
Path 346 | total_timesteps 5959.
Path 347 | total_timesteps 5974.
Path 348 | total_timesteps 5995.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.45    |
| Iteration     | 2        |
| MaximumReturn | 14.2     |
| MinimumReturn | -22.6    |
| TotalSamples  | 16046    |
----------------------------
itr #3 | 
Fitting dynamics.
Validation loss = 0.02317952737212181
Validation loss = 0.01888250559568405
Validation loss = 0.01903633400797844
Validation loss = 0.01679893024265766
Validation loss = 0.016722142696380615
Validation loss = 0.0189701858907938
Validation loss = 0.0172380730509758
Validation loss = 0.016186734661459923
Validation loss = 0.019436277449131012
Validation loss = 0.01870601996779442
Validation loss = 0.017484158277511597
Validation loss = 0.01623041182756424
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 65.
Path 5 | total_timesteps 88.
Path 6 | total_timesteps 109.
Path 7 | total_timesteps 124.
Path 8 | total_timesteps 137.
Path 9 | total_timesteps 174.
Path 10 | total_timesteps 195.
Path 11 | total_timesteps 206.
Path 12 | total_timesteps 228.
Path 13 | total_timesteps 247.
Path 14 | total_timesteps 275.
Path 15 | total_timesteps 299.
Path 16 | total_timesteps 333.
Path 17 | total_timesteps 343.
Path 18 | total_timesteps 364.
Path 19 | total_timesteps 379.
Path 20 | total_timesteps 397.
Path 21 | total_timesteps 412.
Path 22 | total_timesteps 428.
Path 23 | total_timesteps 449.
Path 24 | total_timesteps 474.
Path 25 | total_timesteps 491.
Path 26 | total_timesteps 507.
Path 27 | total_timesteps 527.
Path 28 | total_timesteps 541.
Path 29 | total_timesteps 557.
Path 30 | total_timesteps 581.
Path 31 | total_timesteps 602.
Path 32 | total_timesteps 621.
Path 33 | total_timesteps 636.
Path 34 | total_timesteps 654.
Path 35 | total_timesteps 677.
Path 36 | total_timesteps 705.
Path 37 | total_timesteps 723.
Path 38 | total_timesteps 749.
Path 39 | total_timesteps 763.
Path 40 | total_timesteps 772.
Path 41 | total_timesteps 809.
Path 42 | total_timesteps 836.
Path 43 | total_timesteps 857.
Path 44 | total_timesteps 875.
Path 45 | total_timesteps 892.
Path 46 | total_timesteps 925.
Path 47 | total_timesteps 951.
Path 48 | total_timesteps 969.
Path 49 | total_timesteps 980.
Path 50 | total_timesteps 996.
Path 51 | total_timesteps 1025.
Path 52 | total_timesteps 1047.
Path 53 | total_timesteps 1056.
Path 54 | total_timesteps 1075.
Path 55 | total_timesteps 1115.
Path 56 | total_timesteps 1135.
Path 57 | total_timesteps 1154.
Path 58 | total_timesteps 1161.
Path 59 | total_timesteps 1172.
Path 60 | total_timesteps 1208.
Path 61 | total_timesteps 1236.
Path 62 | total_timesteps 1260.
Path 63 | total_timesteps 1274.
Path 64 | total_timesteps 1292.
Path 65 | total_timesteps 1311.
Path 66 | total_timesteps 1330.
Path 67 | total_timesteps 1368.
Path 68 | total_timesteps 1384.
Path 69 | total_timesteps 1403.
Path 70 | total_timesteps 1422.
Path 71 | total_timesteps 1437.
Path 72 | total_timesteps 1476.
Path 73 | total_timesteps 1494.
Path 74 | total_timesteps 1504.
Path 75 | total_timesteps 1517.
Path 76 | total_timesteps 1537.
Path 77 | total_timesteps 1557.
Path 78 | total_timesteps 1569.
Path 79 | total_timesteps 1594.
Path 80 | total_timesteps 1615.
Path 81 | total_timesteps 1635.
Path 82 | total_timesteps 1647.
Path 83 | total_timesteps 1670.
Path 84 | total_timesteps 1694.
Path 85 | total_timesteps 1709.
Path 86 | total_timesteps 1721.
Path 87 | total_timesteps 1741.
Path 88 | total_timesteps 1755.
Path 89 | total_timesteps 1785.
Path 90 | total_timesteps 1796.
Path 91 | total_timesteps 1814.
Path 92 | total_timesteps 1845.
Path 93 | total_timesteps 1862.
Path 94 | total_timesteps 1876.
Path 95 | total_timesteps 1895.
Path 96 | total_timesteps 1922.
Path 97 | total_timesteps 1942.
Path 98 | total_timesteps 1969.
Path 99 | total_timesteps 1992.
Path 100 | total_timesteps 2012.
Path 101 | total_timesteps 2025.
Path 102 | total_timesteps 2048.
Path 103 | total_timesteps 2057.
Path 104 | total_timesteps 2067.
Path 105 | total_timesteps 2092.
Path 106 | total_timesteps 2114.
Path 107 | total_timesteps 2126.
Path 108 | total_timesteps 2151.
Path 109 | total_timesteps 2177.
Path 110 | total_timesteps 2203.
Path 111 | total_timesteps 2233.
Path 112 | total_timesteps 2246.
Path 113 | total_timesteps 2261.
Path 114 | total_timesteps 2282.
Path 115 | total_timesteps 2295.
Path 116 | total_timesteps 2318.
Path 117 | total_timesteps 2332.
Path 118 | total_timesteps 2369.
Path 119 | total_timesteps 2385.
Path 120 | total_timesteps 2403.
Path 121 | total_timesteps 2416.
Path 122 | total_timesteps 2433.
Path 123 | total_timesteps 2443.
Path 124 | total_timesteps 2459.
Path 125 | total_timesteps 2480.
Path 126 | total_timesteps 2499.
Path 127 | total_timesteps 2536.
Path 128 | total_timesteps 2570.
Path 129 | total_timesteps 2592.
Path 130 | total_timesteps 2605.
Path 131 | total_timesteps 2629.
Path 132 | total_timesteps 2650.
Path 133 | total_timesteps 2664.
Path 134 | total_timesteps 2689.
Path 135 | total_timesteps 2712.
Path 136 | total_timesteps 2723.
Path 137 | total_timesteps 2742.
Path 138 | total_timesteps 2761.
Path 139 | total_timesteps 2779.
Path 140 | total_timesteps 2798.
Path 141 | total_timesteps 2811.
Path 142 | total_timesteps 2825.
Path 143 | total_timesteps 2850.
Path 144 | total_timesteps 2868.
Path 145 | total_timesteps 2884.
Path 146 | total_timesteps 2905.
Path 147 | total_timesteps 2927.
Path 148 | total_timesteps 2939.
Path 149 | total_timesteps 2955.
Path 150 | total_timesteps 2983.
Path 151 | total_timesteps 2999.
Path 152 | total_timesteps 3014.
Path 153 | total_timesteps 3021.
Path 154 | total_timesteps 3046.
Path 155 | total_timesteps 3066.
Path 156 | total_timesteps 3081.
Path 157 | total_timesteps 3097.
Path 158 | total_timesteps 3121.
Path 159 | total_timesteps 3144.
Path 160 | total_timesteps 3159.
Path 161 | total_timesteps 3169.
Path 162 | total_timesteps 3181.
Path 163 | total_timesteps 3199.
Path 164 | total_timesteps 3212.
Path 165 | total_timesteps 3228.
Path 166 | total_timesteps 3247.
Path 167 | total_timesteps 3270.
Path 168 | total_timesteps 3290.
Path 169 | total_timesteps 3303.
Path 170 | total_timesteps 3330.
Path 171 | total_timesteps 3340.
Path 172 | total_timesteps 3354.
Path 173 | total_timesteps 3371.
Path 174 | total_timesteps 3386.
Path 175 | total_timesteps 3397.
Path 176 | total_timesteps 3413.
Path 177 | total_timesteps 3425.
Path 178 | total_timesteps 3440.
Path 179 | total_timesteps 3452.
Path 180 | total_timesteps 3470.
Path 181 | total_timesteps 3495.
Path 182 | total_timesteps 3521.
Path 183 | total_timesteps 3535.
Path 184 | total_timesteps 3553.
Path 185 | total_timesteps 3573.
Path 186 | total_timesteps 3581.
Path 187 | total_timesteps 3596.
Path 188 | total_timesteps 3607.
Path 189 | total_timesteps 3633.
Path 190 | total_timesteps 3644.
Path 191 | total_timesteps 3673.
Path 192 | total_timesteps 3689.
Path 193 | total_timesteps 3712.
Path 194 | total_timesteps 3744.
Path 195 | total_timesteps 3759.
Path 196 | total_timesteps 3782.
Path 197 | total_timesteps 3804.
Path 198 | total_timesteps 3824.
Path 199 | total_timesteps 3833.
Path 200 | total_timesteps 3853.
Path 201 | total_timesteps 3872.
Path 202 | total_timesteps 3888.
Path 203 | total_timesteps 3909.
Path 204 | total_timesteps 3945.
Path 205 | total_timesteps 3966.
Path 206 | total_timesteps 3989.
Path 207 | total_timesteps 4013.
Path 208 | total_timesteps 4025.
Path 209 | total_timesteps 4044.
Path 210 | total_timesteps 4056.
Path 211 | total_timesteps 4080.
Path 212 | total_timesteps 4099.
Path 213 | total_timesteps 4120.
Path 214 | total_timesteps 4131.
Path 215 | total_timesteps 4140.
Path 216 | total_timesteps 4156.
Path 217 | total_timesteps 4178.
Path 218 | total_timesteps 4189.
Path 219 | total_timesteps 4201.
Path 220 | total_timesteps 4216.
Path 221 | total_timesteps 4223.
Path 222 | total_timesteps 4238.
Path 223 | total_timesteps 4251.
Path 224 | total_timesteps 4265.
Path 225 | total_timesteps 4282.
Path 226 | total_timesteps 4298.
Path 227 | total_timesteps 4324.
Path 228 | total_timesteps 4338.
Path 229 | total_timesteps 4361.
Path 230 | total_timesteps 4386.
Path 231 | total_timesteps 4409.
Path 232 | total_timesteps 4423.
Path 233 | total_timesteps 4448.
Path 234 | total_timesteps 4460.
Path 235 | total_timesteps 4480.
Path 236 | total_timesteps 4496.
Path 237 | total_timesteps 4513.
Path 238 | total_timesteps 4544.
Path 239 | total_timesteps 4561.
Path 240 | total_timesteps 4584.
Path 241 | total_timesteps 4594.
Path 242 | total_timesteps 4613.
Path 243 | total_timesteps 4624.
Path 244 | total_timesteps 4639.
Path 245 | total_timesteps 4668.
Path 246 | total_timesteps 4681.
Path 247 | total_timesteps 4695.
Path 248 | total_timesteps 4712.
Path 249 | total_timesteps 4722.
Path 250 | total_timesteps 4735.
Path 251 | total_timesteps 4751.
Path 252 | total_timesteps 4773.
Path 253 | total_timesteps 4790.
Path 254 | total_timesteps 4800.
Path 255 | total_timesteps 4820.
Path 256 | total_timesteps 4836.
Path 257 | total_timesteps 4867.
Path 258 | total_timesteps 4912.
Path 259 | total_timesteps 4930.
Path 260 | total_timesteps 4943.
Path 261 | total_timesteps 4961.
Path 262 | total_timesteps 4973.
Path 263 | total_timesteps 4988.
Path 264 | total_timesteps 5007.
Path 265 | total_timesteps 5024.
Path 266 | total_timesteps 5044.
Path 267 | total_timesteps 5055.
Path 268 | total_timesteps 5074.
Path 269 | total_timesteps 5098.
Path 270 | total_timesteps 5108.
Path 271 | total_timesteps 5133.
Path 272 | total_timesteps 5153.
Path 273 | total_timesteps 5163.
Path 274 | total_timesteps 5196.
Path 275 | total_timesteps 5213.
Path 276 | total_timesteps 5230.
Path 277 | total_timesteps 5241.
Path 278 | total_timesteps 5267.
Path 279 | total_timesteps 5286.
Path 280 | total_timesteps 5305.
Path 281 | total_timesteps 5324.
Path 282 | total_timesteps 5342.
Path 283 | total_timesteps 5349.
Path 284 | total_timesteps 5370.
Path 285 | total_timesteps 5388.
Path 286 | total_timesteps 5407.
Path 287 | total_timesteps 5430.
Path 288 | total_timesteps 5456.
Path 289 | total_timesteps 5481.
Path 290 | total_timesteps 5508.
Path 291 | total_timesteps 5521.
Path 292 | total_timesteps 5546.
Path 293 | total_timesteps 5559.
Path 294 | total_timesteps 5573.
Path 295 | total_timesteps 5589.
Path 296 | total_timesteps 5613.
Path 297 | total_timesteps 5634.
Path 298 | total_timesteps 5651.
Path 299 | total_timesteps 5667.
Path 300 | total_timesteps 5685.
Path 301 | total_timesteps 5712.
Path 302 | total_timesteps 5730.
Path 303 | total_timesteps 5753.
Path 304 | total_timesteps 5776.
Path 305 | total_timesteps 5795.
Path 306 | total_timesteps 5813.
Path 307 | total_timesteps 5825.
Path 308 | total_timesteps 5834.
Path 309 | total_timesteps 5859.
Path 310 | total_timesteps 5903.
Path 311 | total_timesteps 5914.
Path 312 | total_timesteps 5942.
Path 313 | total_timesteps 5963.
Path 314 | total_timesteps 5979.
Path 315 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.76    |
| Iteration     | 3        |
| MaximumReturn | 2.64     |
| MinimumReturn | -23.6    |
| TotalSamples  | 20063    |
----------------------------
itr #4 | 
Fitting dynamics.
Validation loss = 0.01740000769495964
Validation loss = 0.015044057741761208
Validation loss = 0.015377169474959373
Validation loss = 0.014781003817915916
Validation loss = 0.01424498576670885
Validation loss = 0.015606053173542023
Validation loss = 0.014096039347350597
Validation loss = 0.016141891479492188
Validation loss = 0.014230032451450825
Validation loss = 0.013247670605778694
Validation loss = 0.013751871883869171
Validation loss = 0.01412123441696167
Validation loss = 0.012984816916286945
Validation loss = 0.01518992055207491
Validation loss = 0.01370634138584137
Validation loss = 0.013416212983429432
Validation loss = 0.016331572085618973
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 47.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 73.
Path 6 | total_timesteps 91.
Path 7 | total_timesteps 107.
Path 8 | total_timesteps 121.
Path 9 | total_timesteps 162.
Path 10 | total_timesteps 171.
Path 11 | total_timesteps 192.
Path 12 | total_timesteps 205.
Path 13 | total_timesteps 223.
Path 14 | total_timesteps 257.
Path 15 | total_timesteps 285.
Path 16 | total_timesteps 306.
Path 17 | total_timesteps 331.
Path 18 | total_timesteps 349.
Path 19 | total_timesteps 360.
Path 20 | total_timesteps 376.
Path 21 | total_timesteps 397.
Path 22 | total_timesteps 414.
Path 23 | total_timesteps 432.
Path 24 | total_timesteps 448.
Path 25 | total_timesteps 477.
Path 26 | total_timesteps 490.
Path 27 | total_timesteps 517.
Path 28 | total_timesteps 524.
Path 29 | total_timesteps 547.
Path 30 | total_timesteps 572.
Path 31 | total_timesteps 598.
Path 32 | total_timesteps 614.
Path 33 | total_timesteps 640.
Path 34 | total_timesteps 658.
Path 35 | total_timesteps 675.
Path 36 | total_timesteps 689.
Path 37 | total_timesteps 729.
Path 38 | total_timesteps 764.
Path 39 | total_timesteps 783.
Path 40 | total_timesteps 806.
Path 41 | total_timesteps 820.
Path 42 | total_timesteps 842.
Path 43 | total_timesteps 857.
Path 44 | total_timesteps 870.
Path 45 | total_timesteps 901.
Path 46 | total_timesteps 915.
Path 47 | total_timesteps 936.
Path 48 | total_timesteps 964.
Path 49 | total_timesteps 977.
Path 50 | total_timesteps 997.
Path 51 | total_timesteps 1033.
Path 52 | total_timesteps 1058.
Path 53 | total_timesteps 1088.
Path 54 | total_timesteps 1107.
Path 55 | total_timesteps 1124.
Path 56 | total_timesteps 1140.
Path 57 | total_timesteps 1152.
Path 58 | total_timesteps 1179.
Path 59 | total_timesteps 1200.
Path 60 | total_timesteps 1213.
Path 61 | total_timesteps 1227.
Path 62 | total_timesteps 1243.
Path 63 | total_timesteps 1257.
Path 64 | total_timesteps 1289.
Path 65 | total_timesteps 1301.
Path 66 | total_timesteps 1313.
Path 67 | total_timesteps 1331.
Path 68 | total_timesteps 1356.
Path 69 | total_timesteps 1381.
Path 70 | total_timesteps 1391.
Path 71 | total_timesteps 1402.
Path 72 | total_timesteps 1411.
Path 73 | total_timesteps 1446.
Path 74 | total_timesteps 1460.
Path 75 | total_timesteps 1491.
Path 76 | total_timesteps 1506.
Path 77 | total_timesteps 1523.
Path 78 | total_timesteps 1538.
Path 79 | total_timesteps 1549.
Path 80 | total_timesteps 1568.
Path 81 | total_timesteps 1578.
Path 82 | total_timesteps 1604.
Path 83 | total_timesteps 1635.
Path 84 | total_timesteps 1655.
Path 85 | total_timesteps 1685.
Path 86 | total_timesteps 1701.
Path 87 | total_timesteps 1714.
Path 88 | total_timesteps 1740.
Path 89 | total_timesteps 1757.
Path 90 | total_timesteps 1777.
Path 91 | total_timesteps 1791.
Path 92 | total_timesteps 1804.
Path 93 | total_timesteps 1824.
Path 94 | total_timesteps 1840.
Path 95 | total_timesteps 1861.
Path 96 | total_timesteps 1893.
Path 97 | total_timesteps 1905.
Path 98 | total_timesteps 1930.
Path 99 | total_timesteps 1943.
Path 100 | total_timesteps 1964.
Path 101 | total_timesteps 1986.
Path 102 | total_timesteps 1999.
Path 103 | total_timesteps 2011.
Path 104 | total_timesteps 2047.
Path 105 | total_timesteps 2060.
Path 106 | total_timesteps 2078.
Path 107 | total_timesteps 2108.
Path 108 | total_timesteps 2127.
Path 109 | total_timesteps 2140.
Path 110 | total_timesteps 2161.
Path 111 | total_timesteps 2173.
Path 112 | total_timesteps 2201.
Path 113 | total_timesteps 2217.
Path 114 | total_timesteps 2241.
Path 115 | total_timesteps 2254.
Path 116 | total_timesteps 2297.
Path 117 | total_timesteps 2318.
Path 118 | total_timesteps 2333.
Path 119 | total_timesteps 2357.
Path 120 | total_timesteps 2370.
Path 121 | total_timesteps 2391.
Path 122 | total_timesteps 2421.
Path 123 | total_timesteps 2439.
Path 124 | total_timesteps 2454.
Path 125 | total_timesteps 2485.
Path 126 | total_timesteps 2504.
Path 127 | total_timesteps 2526.
Path 128 | total_timesteps 2554.
Path 129 | total_timesteps 2573.
Path 130 | total_timesteps 2587.
Path 131 | total_timesteps 2615.
Path 132 | total_timesteps 2643.
Path 133 | total_timesteps 2663.
Path 134 | total_timesteps 2684.
Path 135 | total_timesteps 2695.
Path 136 | total_timesteps 2715.
Path 137 | total_timesteps 2738.
Path 138 | total_timesteps 2756.
Path 139 | total_timesteps 2771.
Path 140 | total_timesteps 2790.
Path 141 | total_timesteps 2804.
Path 142 | total_timesteps 2817.
Path 143 | total_timesteps 2843.
Path 144 | total_timesteps 2861.
Path 145 | total_timesteps 2886.
Path 146 | total_timesteps 2916.
Path 147 | total_timesteps 2933.
Path 148 | total_timesteps 2960.
Path 149 | total_timesteps 2984.
Path 150 | total_timesteps 2996.
Path 151 | total_timesteps 3027.
Path 152 | total_timesteps 3042.
Path 153 | total_timesteps 3058.
Path 154 | total_timesteps 3077.
Path 155 | total_timesteps 3098.
Path 156 | total_timesteps 3120.
Path 157 | total_timesteps 3141.
Path 158 | total_timesteps 3154.
Path 159 | total_timesteps 3174.
Path 160 | total_timesteps 3195.
Path 161 | total_timesteps 3202.
Path 162 | total_timesteps 3224.
Path 163 | total_timesteps 3238.
Path 164 | total_timesteps 3259.
Path 165 | total_timesteps 3277.
Path 166 | total_timesteps 3298.
Path 167 | total_timesteps 3323.
Path 168 | total_timesteps 3348.
Path 169 | total_timesteps 3369.
Path 170 | total_timesteps 3391.
Path 171 | total_timesteps 3404.
Path 172 | total_timesteps 3421.
Path 173 | total_timesteps 3438.
Path 174 | total_timesteps 3451.
Path 175 | total_timesteps 3463.
Path 176 | total_timesteps 3477.
Path 177 | total_timesteps 3487.
Path 178 | total_timesteps 3508.
Path 179 | total_timesteps 3531.
Path 180 | total_timesteps 3569.
Path 181 | total_timesteps 3590.
Path 182 | total_timesteps 3609.
Path 183 | total_timesteps 3625.
Path 184 | total_timesteps 3639.
Path 185 | total_timesteps 3654.
Path 186 | total_timesteps 3666.
Path 187 | total_timesteps 3690.
Path 188 | total_timesteps 3714.
Path 189 | total_timesteps 3736.
Path 190 | total_timesteps 3769.
Path 191 | total_timesteps 3788.
Path 192 | total_timesteps 3811.
Path 193 | total_timesteps 3832.
Path 194 | total_timesteps 3850.
Path 195 | total_timesteps 3874.
Path 196 | total_timesteps 3896.
Path 197 | total_timesteps 3911.
Path 198 | total_timesteps 3936.
Path 199 | total_timesteps 3950.
Path 200 | total_timesteps 3969.
Path 201 | total_timesteps 3994.
Path 202 | total_timesteps 4013.
Path 203 | total_timesteps 4031.
Path 204 | total_timesteps 4053.
Path 205 | total_timesteps 4071.
Path 206 | total_timesteps 4092.
Path 207 | total_timesteps 4108.
Path 208 | total_timesteps 4126.
Path 209 | total_timesteps 4143.
Path 210 | total_timesteps 4157.
Path 211 | total_timesteps 4178.
Path 212 | total_timesteps 4190.
Path 213 | total_timesteps 4203.
Path 214 | total_timesteps 4227.
Path 215 | total_timesteps 4238.
Path 216 | total_timesteps 4251.
Path 217 | total_timesteps 4274.
Path 218 | total_timesteps 4311.
Path 219 | total_timesteps 4328.
Path 220 | total_timesteps 4339.
Path 221 | total_timesteps 4360.
Path 222 | total_timesteps 4379.
Path 223 | total_timesteps 4395.
Path 224 | total_timesteps 4414.
Path 225 | total_timesteps 4422.
Path 226 | total_timesteps 4445.
Path 227 | total_timesteps 4470.
Path 228 | total_timesteps 4483.
Path 229 | total_timesteps 4505.
Path 230 | total_timesteps 4518.
Path 231 | total_timesteps 4534.
Path 232 | total_timesteps 4557.
Path 233 | total_timesteps 4577.
Path 234 | total_timesteps 4593.
Path 235 | total_timesteps 4615.
Path 236 | total_timesteps 4641.
Path 237 | total_timesteps 4652.
Path 238 | total_timesteps 4672.
Path 239 | total_timesteps 4694.
Path 240 | total_timesteps 4714.
Path 241 | total_timesteps 4745.
Path 242 | total_timesteps 4781.
Path 243 | total_timesteps 4801.
Path 244 | total_timesteps 4831.
Path 245 | total_timesteps 4867.
Path 246 | total_timesteps 4879.
Path 247 | total_timesteps 4897.
Path 248 | total_timesteps 4925.
Path 249 | total_timesteps 4954.
Path 250 | total_timesteps 4980.
Path 251 | total_timesteps 5004.
Path 252 | total_timesteps 5017.
Path 253 | total_timesteps 5029.
Path 254 | total_timesteps 5045.
Path 255 | total_timesteps 5064.
Path 256 | total_timesteps 5079.
Path 257 | total_timesteps 5091.
Path 258 | total_timesteps 5116.
Path 259 | total_timesteps 5133.
Path 260 | total_timesteps 5152.
Path 261 | total_timesteps 5170.
Path 262 | total_timesteps 5190.
Path 263 | total_timesteps 5200.
Path 264 | total_timesteps 5210.
Path 265 | total_timesteps 5228.
Path 266 | total_timesteps 5250.
Path 267 | total_timesteps 5270.
Path 268 | total_timesteps 5285.
Path 269 | total_timesteps 5295.
Path 270 | total_timesteps 5317.
Path 271 | total_timesteps 5342.
Path 272 | total_timesteps 5351.
Path 273 | total_timesteps 5366.
Path 274 | total_timesteps 5381.
Path 275 | total_timesteps 5408.
Path 276 | total_timesteps 5429.
Path 277 | total_timesteps 5450.
Path 278 | total_timesteps 5464.
Path 279 | total_timesteps 5486.
Path 280 | total_timesteps 5501.
Path 281 | total_timesteps 5528.
Path 282 | total_timesteps 5547.
Path 283 | total_timesteps 5563.
Path 284 | total_timesteps 5585.
Path 285 | total_timesteps 5603.
Path 286 | total_timesteps 5623.
Path 287 | total_timesteps 5644.
Path 288 | total_timesteps 5663.
Path 289 | total_timesteps 5685.
Path 290 | total_timesteps 5702.
Path 291 | total_timesteps 5728.
Path 292 | total_timesteps 5748.
Path 293 | total_timesteps 5782.
Path 294 | total_timesteps 5809.
Path 295 | total_timesteps 5828.
Path 296 | total_timesteps 5856.
Path 297 | total_timesteps 5875.
Path 298 | total_timesteps 5891.
Path 299 | total_timesteps 5907.
Path 300 | total_timesteps 5926.
Path 301 | total_timesteps 5938.
Path 302 | total_timesteps 5964.
Path 303 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.4    |
| Iteration     | 4        |
| MaximumReturn | 3.23     |
| MinimumReturn | -25.1    |
| TotalSamples  | 24068    |
----------------------------
itr #5 | 
Fitting dynamics.
Validation loss = 0.014944496564567089
Validation loss = 0.012360424734652042
Validation loss = 0.011709672398865223
Validation loss = 0.011618044227361679
Validation loss = 0.014067091047763824
Validation loss = 0.010924487374722958
Validation loss = 0.01190484594553709
Validation loss = 0.013995826244354248
Validation loss = 0.0119546540081501
Validation loss = 0.011057275347411633
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 41.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 75.
Path 5 | total_timesteps 95.
Path 6 | total_timesteps 115.
Path 7 | total_timesteps 147.
Path 8 | total_timesteps 177.
Path 9 | total_timesteps 198.
Path 10 | total_timesteps 218.
Path 11 | total_timesteps 240.
Path 12 | total_timesteps 250.
Path 13 | total_timesteps 263.
Path 14 | total_timesteps 281.
Path 15 | total_timesteps 305.
Path 16 | total_timesteps 331.
Path 17 | total_timesteps 355.
Path 18 | total_timesteps 376.
Path 19 | total_timesteps 402.
Path 20 | total_timesteps 426.
Path 21 | total_timesteps 461.
Path 22 | total_timesteps 480.
Path 23 | total_timesteps 510.
Path 24 | total_timesteps 526.
Path 25 | total_timesteps 550.
Path 26 | total_timesteps 572.
Path 27 | total_timesteps 586.
Path 28 | total_timesteps 605.
Path 29 | total_timesteps 627.
Path 30 | total_timesteps 646.
Path 31 | total_timesteps 675.
Path 32 | total_timesteps 700.
Path 33 | total_timesteps 729.
Path 34 | total_timesteps 742.
Path 35 | total_timesteps 781.
Path 36 | total_timesteps 795.
Path 37 | total_timesteps 818.
Path 38 | total_timesteps 840.
Path 39 | total_timesteps 859.
Path 40 | total_timesteps 887.
Path 41 | total_timesteps 903.
Path 42 | total_timesteps 935.
Path 43 | total_timesteps 958.
Path 44 | total_timesteps 982.
Path 45 | total_timesteps 1006.
Path 46 | total_timesteps 1031.
Path 47 | total_timesteps 1047.
Path 48 | total_timesteps 1062.
Path 49 | total_timesteps 1080.
Path 50 | total_timesteps 1102.
Path 51 | total_timesteps 1122.
Path 52 | total_timesteps 1139.
Path 53 | total_timesteps 1176.
Path 54 | total_timesteps 1207.
Path 55 | total_timesteps 1223.
Path 56 | total_timesteps 1243.
Path 57 | total_timesteps 1269.
Path 58 | total_timesteps 1296.
Path 59 | total_timesteps 1331.
Path 60 | total_timesteps 1349.
Path 61 | total_timesteps 1369.
Path 62 | total_timesteps 1405.
Path 63 | total_timesteps 1415.
Path 64 | total_timesteps 1437.
Path 65 | total_timesteps 1449.
Path 66 | total_timesteps 1468.
Path 67 | total_timesteps 1496.
Path 68 | total_timesteps 1514.
Path 69 | total_timesteps 1523.
Path 70 | total_timesteps 1540.
Path 71 | total_timesteps 1553.
Path 72 | total_timesteps 1567.
Path 73 | total_timesteps 1582.
Path 74 | total_timesteps 1598.
Path 75 | total_timesteps 1621.
Path 76 | total_timesteps 1643.
Path 77 | total_timesteps 1660.
Path 78 | total_timesteps 1680.
Path 79 | total_timesteps 1698.
Path 80 | total_timesteps 1722.
Path 81 | total_timesteps 1753.
Path 82 | total_timesteps 1785.
Path 83 | total_timesteps 1805.
Path 84 | total_timesteps 1822.
Path 85 | total_timesteps 1834.
Path 86 | total_timesteps 1853.
Path 87 | total_timesteps 1879.
Path 88 | total_timesteps 1894.
Path 89 | total_timesteps 1908.
Path 90 | total_timesteps 1924.
Path 91 | total_timesteps 1942.
Path 92 | total_timesteps 1954.
Path 93 | total_timesteps 1965.
Path 94 | total_timesteps 1986.
Path 95 | total_timesteps 2004.
Path 96 | total_timesteps 2027.
Path 97 | total_timesteps 2045.
Path 98 | total_timesteps 2074.
Path 99 | total_timesteps 2096.
Path 100 | total_timesteps 2117.
Path 101 | total_timesteps 2131.
Path 102 | total_timesteps 2149.
Path 103 | total_timesteps 2178.
Path 104 | total_timesteps 2193.
Path 105 | total_timesteps 2224.
Path 106 | total_timesteps 2251.
Path 107 | total_timesteps 2270.
Path 108 | total_timesteps 2292.
Path 109 | total_timesteps 2311.
Path 110 | total_timesteps 2326.
Path 111 | total_timesteps 2348.
Path 112 | total_timesteps 2371.
Path 113 | total_timesteps 2392.
Path 114 | total_timesteps 2414.
Path 115 | total_timesteps 2434.
Path 116 | total_timesteps 2465.
Path 117 | total_timesteps 2480.
Path 118 | total_timesteps 2502.
Path 119 | total_timesteps 2524.
Path 120 | total_timesteps 2547.
Path 121 | total_timesteps 2570.
Path 122 | total_timesteps 2607.
Path 123 | total_timesteps 2627.
Path 124 | total_timesteps 2649.
Path 125 | total_timesteps 2665.
Path 126 | total_timesteps 2689.
Path 127 | total_timesteps 2721.
Path 128 | total_timesteps 2736.
Path 129 | total_timesteps 2760.
Path 130 | total_timesteps 2768.
Path 131 | total_timesteps 2782.
Path 132 | total_timesteps 2804.
Path 133 | total_timesteps 2832.
Path 134 | total_timesteps 2851.
Path 135 | total_timesteps 2870.
Path 136 | total_timesteps 2888.
Path 137 | total_timesteps 2907.
Path 138 | total_timesteps 2929.
Path 139 | total_timesteps 2941.
Path 140 | total_timesteps 2957.
Path 141 | total_timesteps 2979.
Path 142 | total_timesteps 2997.
Path 143 | total_timesteps 3019.
Path 144 | total_timesteps 3039.
Path 145 | total_timesteps 3067.
Path 146 | total_timesteps 3077.
Path 147 | total_timesteps 3096.
Path 148 | total_timesteps 3119.
Path 149 | total_timesteps 3167.
Path 150 | total_timesteps 3192.
Path 151 | total_timesteps 3217.
Path 152 | total_timesteps 3229.
Path 153 | total_timesteps 3253.
Path 154 | total_timesteps 3274.
Path 155 | total_timesteps 3290.
Path 156 | total_timesteps 3313.
Path 157 | total_timesteps 3330.
Path 158 | total_timesteps 3343.
Path 159 | total_timesteps 3362.
Path 160 | total_timesteps 3382.
Path 161 | total_timesteps 3400.
Path 162 | total_timesteps 3430.
Path 163 | total_timesteps 3456.
Path 164 | total_timesteps 3478.
Path 165 | total_timesteps 3494.
Path 166 | total_timesteps 3512.
Path 167 | total_timesteps 3530.
Path 168 | total_timesteps 3557.
Path 169 | total_timesteps 3592.
Path 170 | total_timesteps 3614.
Path 171 | total_timesteps 3626.
Path 172 | total_timesteps 3652.
Path 173 | total_timesteps 3665.
Path 174 | total_timesteps 3688.
Path 175 | total_timesteps 3706.
Path 176 | total_timesteps 3724.
Path 177 | total_timesteps 3742.
Path 178 | total_timesteps 3768.
Path 179 | total_timesteps 3786.
Path 180 | total_timesteps 3806.
Path 181 | total_timesteps 3818.
Path 182 | total_timesteps 3831.
Path 183 | total_timesteps 3845.
Path 184 | total_timesteps 3875.
Path 185 | total_timesteps 3900.
Path 186 | total_timesteps 3917.
Path 187 | total_timesteps 3938.
Path 188 | total_timesteps 3957.
Path 189 | total_timesteps 3975.
Path 190 | total_timesteps 3999.
Path 191 | total_timesteps 4016.
Path 192 | total_timesteps 4041.
Path 193 | total_timesteps 4057.
Path 194 | total_timesteps 4074.
Path 195 | total_timesteps 4087.
Path 196 | total_timesteps 4105.
Path 197 | total_timesteps 4134.
Path 198 | total_timesteps 4159.
Path 199 | total_timesteps 4175.
Path 200 | total_timesteps 4198.
Path 201 | total_timesteps 4237.
Path 202 | total_timesteps 4253.
Path 203 | total_timesteps 4267.
Path 204 | total_timesteps 4291.
Path 205 | total_timesteps 4318.
Path 206 | total_timesteps 4344.
Path 207 | total_timesteps 4360.
Path 208 | total_timesteps 4376.
Path 209 | total_timesteps 4391.
Path 210 | total_timesteps 4407.
Path 211 | total_timesteps 4428.
Path 212 | total_timesteps 4445.
Path 213 | total_timesteps 4462.
Path 214 | total_timesteps 4477.
Path 215 | total_timesteps 4503.
Path 216 | total_timesteps 4521.
Path 217 | total_timesteps 4546.
Path 218 | total_timesteps 4553.
Path 219 | total_timesteps 4567.
Path 220 | total_timesteps 4582.
Path 221 | total_timesteps 4606.
Path 222 | total_timesteps 4625.
Path 223 | total_timesteps 4643.
Path 224 | total_timesteps 4668.
Path 225 | total_timesteps 4703.
Path 226 | total_timesteps 4742.
Path 227 | total_timesteps 4759.
Path 228 | total_timesteps 4781.
Path 229 | total_timesteps 4799.
Path 230 | total_timesteps 4821.
Path 231 | total_timesteps 4845.
Path 232 | total_timesteps 4896.
Path 233 | total_timesteps 4911.
Path 234 | total_timesteps 4954.
Path 235 | total_timesteps 4970.
Path 236 | total_timesteps 4980.
Path 237 | total_timesteps 4996.
Path 238 | total_timesteps 5011.
Path 239 | total_timesteps 5029.
Path 240 | total_timesteps 5053.
Path 241 | total_timesteps 5079.
Path 242 | total_timesteps 5094.
Path 243 | total_timesteps 5115.
Path 244 | total_timesteps 5136.
Path 245 | total_timesteps 5162.
Path 246 | total_timesteps 5179.
Path 247 | total_timesteps 5192.
Path 248 | total_timesteps 5223.
Path 249 | total_timesteps 5252.
Path 250 | total_timesteps 5260.
Path 251 | total_timesteps 5283.
Path 252 | total_timesteps 5301.
Path 253 | total_timesteps 5322.
Path 254 | total_timesteps 5343.
Path 255 | total_timesteps 5368.
Path 256 | total_timesteps 5389.
Path 257 | total_timesteps 5408.
Path 258 | total_timesteps 5443.
Path 259 | total_timesteps 5454.
Path 260 | total_timesteps 5474.
Path 261 | total_timesteps 5489.
Path 262 | total_timesteps 5517.
Path 263 | total_timesteps 5540.
Path 264 | total_timesteps 5558.
Path 265 | total_timesteps 5574.
Path 266 | total_timesteps 5597.
Path 267 | total_timesteps 5630.
Path 268 | total_timesteps 5641.
Path 269 | total_timesteps 5665.
Path 270 | total_timesteps 5690.
Path 271 | total_timesteps 5708.
Path 272 | total_timesteps 5730.
Path 273 | total_timesteps 5748.
Path 274 | total_timesteps 5780.
Path 275 | total_timesteps 5813.
Path 276 | total_timesteps 5832.
Path 277 | total_timesteps 5846.
Path 278 | total_timesteps 5880.
Path 279 | total_timesteps 5902.
Path 280 | total_timesteps 5912.
Path 281 | total_timesteps 5930.
Path 282 | total_timesteps 5952.
Path 283 | total_timesteps 5965.
Path 284 | total_timesteps 5985.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.7    |
| Iteration     | 5        |
| MaximumReturn | 4.03     |
| MinimumReturn | -25.2    |
| TotalSamples  | 28082    |
----------------------------
itr #6 | 
Fitting dynamics.
Validation loss = 0.014707791619002819
Validation loss = 0.0108469408005476
Validation loss = 0.010321290232241154
Validation loss = 0.012064251117408276
Validation loss = 0.013829709030687809
Validation loss = 0.010625715367496014
Validation loss = 0.010698227211833
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 26.
Path 2 | total_timesteps 45.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 84.
Path 5 | total_timesteps 106.
Path 6 | total_timesteps 142.
Path 7 | total_timesteps 163.
Path 8 | total_timesteps 187.
Path 9 | total_timesteps 204.
Path 10 | total_timesteps 222.
Path 11 | total_timesteps 246.
Path 12 | total_timesteps 264.
Path 13 | total_timesteps 288.
Path 14 | total_timesteps 305.
Path 15 | total_timesteps 340.
Path 16 | total_timesteps 368.
Path 17 | total_timesteps 390.
Path 18 | total_timesteps 418.
Path 19 | total_timesteps 448.
Path 20 | total_timesteps 470.
Path 21 | total_timesteps 488.
Path 22 | total_timesteps 501.
Path 23 | total_timesteps 527.
Path 24 | total_timesteps 553.
Path 25 | total_timesteps 599.
Path 26 | total_timesteps 620.
Path 27 | total_timesteps 638.
Path 28 | total_timesteps 662.
Path 29 | total_timesteps 678.
Path 30 | total_timesteps 691.
Path 31 | total_timesteps 710.
Path 32 | total_timesteps 729.
Path 33 | total_timesteps 740.
Path 34 | total_timesteps 765.
Path 35 | total_timesteps 783.
Path 36 | total_timesteps 817.
Path 37 | total_timesteps 847.
Path 38 | total_timesteps 869.
Path 39 | total_timesteps 896.
Path 40 | total_timesteps 913.
Path 41 | total_timesteps 943.
Path 42 | total_timesteps 960.
Path 43 | total_timesteps 976.
Path 44 | total_timesteps 992.
Path 45 | total_timesteps 1012.
Path 46 | total_timesteps 1031.
Path 47 | total_timesteps 1056.
Path 48 | total_timesteps 1072.
Path 49 | total_timesteps 1105.
Path 50 | total_timesteps 1120.
Path 51 | total_timesteps 1134.
Path 52 | total_timesteps 1165.
Path 53 | total_timesteps 1184.
Path 54 | total_timesteps 1209.
Path 55 | total_timesteps 1228.
Path 56 | total_timesteps 1242.
Path 57 | total_timesteps 1260.
Path 58 | total_timesteps 1272.
Path 59 | total_timesteps 1321.
Path 60 | total_timesteps 1351.
Path 61 | total_timesteps 1386.
Path 62 | total_timesteps 1408.
Path 63 | total_timesteps 1419.
Path 64 | total_timesteps 1441.
Path 65 | total_timesteps 1458.
Path 66 | total_timesteps 1508.
Path 67 | total_timesteps 1522.
Path 68 | total_timesteps 1545.
Path 69 | total_timesteps 1561.
Path 70 | total_timesteps 1594.
Path 71 | total_timesteps 1623.
Path 72 | total_timesteps 1645.
Path 73 | total_timesteps 1668.
Path 74 | total_timesteps 1692.
Path 75 | total_timesteps 1708.
Path 76 | total_timesteps 1720.
Path 77 | total_timesteps 1739.
Path 78 | total_timesteps 1753.
Path 79 | total_timesteps 1777.
Path 80 | total_timesteps 1798.
Path 81 | total_timesteps 1821.
Path 82 | total_timesteps 1842.
Path 83 | total_timesteps 1861.
Path 84 | total_timesteps 1882.
Path 85 | total_timesteps 1922.
Path 86 | total_timesteps 1933.
Path 87 | total_timesteps 1991.
Path 88 | total_timesteps 2019.
Path 89 | total_timesteps 2047.
Path 90 | total_timesteps 2063.
Path 91 | total_timesteps 2078.
Path 92 | total_timesteps 2091.
Path 93 | total_timesteps 2118.
Path 94 | total_timesteps 2129.
Path 95 | total_timesteps 2144.
Path 96 | total_timesteps 2161.
Path 97 | total_timesteps 2181.
Path 98 | total_timesteps 2198.
Path 99 | total_timesteps 2226.
Path 100 | total_timesteps 2237.
Path 101 | total_timesteps 2251.
Path 102 | total_timesteps 2270.
Path 103 | total_timesteps 2291.
Path 104 | total_timesteps 2313.
Path 105 | total_timesteps 2370.
Path 106 | total_timesteps 2390.
Path 107 | total_timesteps 2407.
Path 108 | total_timesteps 2427.
Path 109 | total_timesteps 2452.
Path 110 | total_timesteps 2486.
Path 111 | total_timesteps 2525.
Path 112 | total_timesteps 2547.
Path 113 | total_timesteps 2563.
Path 114 | total_timesteps 2586.
Path 115 | total_timesteps 2603.
Path 116 | total_timesteps 2629.
Path 117 | total_timesteps 2656.
Path 118 | total_timesteps 2683.
Path 119 | total_timesteps 2731.
Path 120 | total_timesteps 2754.
Path 121 | total_timesteps 2773.
Path 122 | total_timesteps 2788.
Path 123 | total_timesteps 2804.
Path 124 | total_timesteps 2822.
Path 125 | total_timesteps 2844.
Path 126 | total_timesteps 2879.
Path 127 | total_timesteps 2907.
Path 128 | total_timesteps 2921.
Path 129 | total_timesteps 2941.
Path 130 | total_timesteps 2956.
Path 131 | total_timesteps 2964.
Path 132 | total_timesteps 2981.
Path 133 | total_timesteps 2995.
Path 134 | total_timesteps 3010.
Path 135 | total_timesteps 3043.
Path 136 | total_timesteps 3055.
Path 137 | total_timesteps 3078.
Path 138 | total_timesteps 3095.
Path 139 | total_timesteps 3105.
Path 140 | total_timesteps 3123.
Path 141 | total_timesteps 3152.
Path 142 | total_timesteps 3169.
Path 143 | total_timesteps 3185.
Path 144 | total_timesteps 3209.
Path 145 | total_timesteps 3233.
Path 146 | total_timesteps 3255.
Path 147 | total_timesteps 3303.
Path 148 | total_timesteps 3331.
Path 149 | total_timesteps 3353.
Path 150 | total_timesteps 3373.
Path 151 | total_timesteps 3400.
Path 152 | total_timesteps 3420.
Path 153 | total_timesteps 3445.
Path 154 | total_timesteps 3469.
Path 155 | total_timesteps 3481.
Path 156 | total_timesteps 3510.
Path 157 | total_timesteps 3532.
Path 158 | total_timesteps 3555.
Path 159 | total_timesteps 3579.
Path 160 | total_timesteps 3597.
Path 161 | total_timesteps 3621.
Path 162 | total_timesteps 3640.
Path 163 | total_timesteps 3660.
Path 164 | total_timesteps 3679.
Path 165 | total_timesteps 3701.
Path 166 | total_timesteps 3717.
Path 167 | total_timesteps 3737.
Path 168 | total_timesteps 3751.
Path 169 | total_timesteps 3766.
Path 170 | total_timesteps 3783.
Path 171 | total_timesteps 3807.
Path 172 | total_timesteps 3827.
Path 173 | total_timesteps 3857.
Path 174 | total_timesteps 3876.
Path 175 | total_timesteps 3888.
Path 176 | total_timesteps 3901.
Path 177 | total_timesteps 3932.
Path 178 | total_timesteps 3946.
Path 179 | total_timesteps 3965.
Path 180 | total_timesteps 3984.
Path 181 | total_timesteps 4008.
Path 182 | total_timesteps 4024.
Path 183 | total_timesteps 4040.
Path 184 | total_timesteps 4060.
Path 185 | total_timesteps 4078.
Path 186 | total_timesteps 4102.
Path 187 | total_timesteps 4132.
Path 188 | total_timesteps 4153.
Path 189 | total_timesteps 4172.
Path 190 | total_timesteps 4196.
Path 191 | total_timesteps 4223.
Path 192 | total_timesteps 4240.
Path 193 | total_timesteps 4258.
Path 194 | total_timesteps 4272.
Path 195 | total_timesteps 4310.
Path 196 | total_timesteps 4326.
Path 197 | total_timesteps 4340.
Path 198 | total_timesteps 4363.
Path 199 | total_timesteps 4378.
Path 200 | total_timesteps 4405.
Path 201 | total_timesteps 4430.
Path 202 | total_timesteps 4453.
Path 203 | total_timesteps 4474.
Path 204 | total_timesteps 4502.
Path 205 | total_timesteps 4516.
Path 206 | total_timesteps 4532.
Path 207 | total_timesteps 4558.
Path 208 | total_timesteps 4592.
Path 209 | total_timesteps 4606.
Path 210 | total_timesteps 4620.
Path 211 | total_timesteps 4641.
Path 212 | total_timesteps 4665.
Path 213 | total_timesteps 4683.
Path 214 | total_timesteps 4709.
Path 215 | total_timesteps 4721.
Path 216 | total_timesteps 4751.
Path 217 | total_timesteps 4772.
Path 218 | total_timesteps 4793.
Path 219 | total_timesteps 4812.
Path 220 | total_timesteps 4831.
Path 221 | total_timesteps 4866.
Path 222 | total_timesteps 4887.
Path 223 | total_timesteps 4910.
Path 224 | total_timesteps 4933.
Path 225 | total_timesteps 4953.
Path 226 | total_timesteps 4967.
Path 227 | total_timesteps 4990.
Path 228 | total_timesteps 5011.
Path 229 | total_timesteps 5022.
Path 230 | total_timesteps 5048.
Path 231 | total_timesteps 5080.
Path 232 | total_timesteps 5094.
Path 233 | total_timesteps 5109.
Path 234 | total_timesteps 5149.
Path 235 | total_timesteps 5168.
Path 236 | total_timesteps 5184.
Path 237 | total_timesteps 5208.
Path 238 | total_timesteps 5230.
Path 239 | total_timesteps 5265.
Path 240 | total_timesteps 5286.
Path 241 | total_timesteps 5306.
Path 242 | total_timesteps 5319.
Path 243 | total_timesteps 5351.
Path 244 | total_timesteps 5362.
Path 245 | total_timesteps 5374.
Path 246 | total_timesteps 5411.
Path 247 | total_timesteps 5438.
Path 248 | total_timesteps 5468.
Path 249 | total_timesteps 5500.
Path 250 | total_timesteps 5512.
Path 251 | total_timesteps 5533.
Path 252 | total_timesteps 5568.
Path 253 | total_timesteps 5596.
Path 254 | total_timesteps 5640.
Path 255 | total_timesteps 5653.
Path 256 | total_timesteps 5669.
Path 257 | total_timesteps 5690.
Path 258 | total_timesteps 5704.
Path 259 | total_timesteps 5727.
Path 260 | total_timesteps 5740.
Path 261 | total_timesteps 5760.
Path 262 | total_timesteps 5781.
Path 263 | total_timesteps 5807.
Path 264 | total_timesteps 5830.
Path 265 | total_timesteps 5857.
Path 266 | total_timesteps 5874.
Path 267 | total_timesteps 5896.
Path 268 | total_timesteps 5924.
Path 269 | total_timesteps 5939.
Path 270 | total_timesteps 5961.
Path 271 | total_timesteps 5974.
Path 272 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -12.2    |
| Iteration     | 6        |
| MaximumReturn | 3.53     |
| MinimumReturn | -27.2    |
| TotalSamples  | 32088    |
----------------------------
itr #7 | 
Fitting dynamics.
Validation loss = 0.013690972700715065
Validation loss = 0.010931722819805145
Validation loss = 0.013833477161824703
Validation loss = 0.009632878936827183
Validation loss = 0.011389360763132572
Validation loss = 0.009579294361174107
Validation loss = 0.009738274849951267
Validation loss = 0.009015667252242565
Validation loss = 0.009921366348862648
Validation loss = 0.012425746768712997
Validation loss = 0.009476525709033012
Validation loss = 0.008684173226356506
Validation loss = 0.010494723916053772
Validation loss = 0.01036752201616764
Validation loss = 0.00863780826330185
Validation loss = 0.00876800436526537
Validation loss = 0.008752981200814247
Validation loss = 0.009259374812245369
Validation loss = 0.00958600640296936
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 41.
Path 3 | total_timesteps 61.
Path 4 | total_timesteps 87.
Path 5 | total_timesteps 114.
Path 6 | total_timesteps 143.
Path 7 | total_timesteps 183.
Path 8 | total_timesteps 203.
Path 9 | total_timesteps 231.
Path 10 | total_timesteps 251.
Path 11 | total_timesteps 279.
Path 12 | total_timesteps 298.
Path 13 | total_timesteps 324.
Path 14 | total_timesteps 341.
Path 15 | total_timesteps 362.
Path 16 | total_timesteps 375.
Path 17 | total_timesteps 412.
Path 18 | total_timesteps 423.
Path 19 | total_timesteps 449.
Path 20 | total_timesteps 483.
Path 21 | total_timesteps 523.
Path 22 | total_timesteps 542.
Path 23 | total_timesteps 564.
Path 24 | total_timesteps 588.
Path 25 | total_timesteps 621.
Path 26 | total_timesteps 641.
Path 27 | total_timesteps 666.
Path 28 | total_timesteps 684.
Path 29 | total_timesteps 699.
Path 30 | total_timesteps 725.
Path 31 | total_timesteps 758.
Path 32 | total_timesteps 802.
Path 33 | total_timesteps 834.
Path 34 | total_timesteps 853.
Path 35 | total_timesteps 875.
Path 36 | total_timesteps 905.
Path 37 | total_timesteps 930.
Path 38 | total_timesteps 948.
Path 39 | total_timesteps 959.
Path 40 | total_timesteps 970.
Path 41 | total_timesteps 988.
Path 42 | total_timesteps 1013.
Path 43 | total_timesteps 1042.
Path 44 | total_timesteps 1062.
Path 45 | total_timesteps 1082.
Path 46 | total_timesteps 1099.
Path 47 | total_timesteps 1131.
Path 48 | total_timesteps 1152.
Path 49 | total_timesteps 1179.
Path 50 | total_timesteps 1198.
Path 51 | total_timesteps 1216.
Path 52 | total_timesteps 1236.
Path 53 | total_timesteps 1253.
Path 54 | total_timesteps 1277.
Path 55 | total_timesteps 1297.
Path 56 | total_timesteps 1316.
Path 57 | total_timesteps 1338.
Path 58 | total_timesteps 1351.
Path 59 | total_timesteps 1363.
Path 60 | total_timesteps 1383.
Path 61 | total_timesteps 1408.
Path 62 | total_timesteps 1420.
Path 63 | total_timesteps 1431.
Path 64 | total_timesteps 1444.
Path 65 | total_timesteps 1462.
Path 66 | total_timesteps 1481.
Path 67 | total_timesteps 1511.
Path 68 | total_timesteps 1533.
Path 69 | total_timesteps 1557.
Path 70 | total_timesteps 1573.
Path 71 | total_timesteps 1589.
Path 72 | total_timesteps 1633.
Path 73 | total_timesteps 1656.
Path 74 | total_timesteps 1667.
Path 75 | total_timesteps 1698.
Path 76 | total_timesteps 1721.
Path 77 | total_timesteps 1735.
Path 78 | total_timesteps 1755.
Path 79 | total_timesteps 1772.
Path 80 | total_timesteps 1789.
Path 81 | total_timesteps 1821.
Path 82 | total_timesteps 1864.
Path 83 | total_timesteps 1875.
Path 84 | total_timesteps 1886.
Path 85 | total_timesteps 1904.
Path 86 | total_timesteps 1917.
Path 87 | total_timesteps 1930.
Path 88 | total_timesteps 1959.
Path 89 | total_timesteps 1971.
Path 90 | total_timesteps 1988.
Path 91 | total_timesteps 2008.
Path 92 | total_timesteps 2034.
Path 93 | total_timesteps 2051.
Path 94 | total_timesteps 2064.
Path 95 | total_timesteps 2088.
Path 96 | total_timesteps 2114.
Path 97 | total_timesteps 2134.
Path 98 | total_timesteps 2153.
Path 99 | total_timesteps 2183.
Path 100 | total_timesteps 2200.
Path 101 | total_timesteps 2219.
Path 102 | total_timesteps 2248.
Path 103 | total_timesteps 2268.
Path 104 | total_timesteps 2283.
Path 105 | total_timesteps 2311.
Path 106 | total_timesteps 2321.
Path 107 | total_timesteps 2340.
Path 108 | total_timesteps 2353.
Path 109 | total_timesteps 2372.
Path 110 | total_timesteps 2391.
Path 111 | total_timesteps 2437.
Path 112 | total_timesteps 2457.
Path 113 | total_timesteps 2481.
Path 114 | total_timesteps 2494.
Path 115 | total_timesteps 2516.
Path 116 | total_timesteps 2535.
Path 117 | total_timesteps 2549.
Path 118 | total_timesteps 2567.
Path 119 | total_timesteps 2592.
Path 120 | total_timesteps 2613.
Path 121 | total_timesteps 2632.
Path 122 | total_timesteps 2660.
Path 123 | total_timesteps 2679.
Path 124 | total_timesteps 2696.
Path 125 | total_timesteps 2722.
Path 126 | total_timesteps 2756.
Path 127 | total_timesteps 2783.
Path 128 | total_timesteps 2807.
Path 129 | total_timesteps 2831.
Path 130 | total_timesteps 2849.
Path 131 | total_timesteps 2867.
Path 132 | total_timesteps 2889.
Path 133 | total_timesteps 2904.
Path 134 | total_timesteps 2948.
Path 135 | total_timesteps 2971.
Path 136 | total_timesteps 2988.
Path 137 | total_timesteps 3007.
Path 138 | total_timesteps 3028.
Path 139 | total_timesteps 3057.
Path 140 | total_timesteps 3082.
Path 141 | total_timesteps 3097.
Path 142 | total_timesteps 3119.
Path 143 | total_timesteps 3143.
Path 144 | total_timesteps 3162.
Path 145 | total_timesteps 3186.
Path 146 | total_timesteps 3194.
Path 147 | total_timesteps 3226.
Path 148 | total_timesteps 3247.
Path 149 | total_timesteps 3262.
Path 150 | total_timesteps 3283.
Path 151 | total_timesteps 3305.
Path 152 | total_timesteps 3327.
Path 153 | total_timesteps 3345.
Path 154 | total_timesteps 3362.
Path 155 | total_timesteps 3382.
Path 156 | total_timesteps 3404.
Path 157 | total_timesteps 3448.
Path 158 | total_timesteps 3464.
Path 159 | total_timesteps 3475.
Path 160 | total_timesteps 3495.
Path 161 | total_timesteps 3509.
Path 162 | total_timesteps 3524.
Path 163 | total_timesteps 3543.
Path 164 | total_timesteps 3561.
Path 165 | total_timesteps 3580.
Path 166 | total_timesteps 3610.
Path 167 | total_timesteps 3629.
Path 168 | total_timesteps 3645.
Path 169 | total_timesteps 3669.
Path 170 | total_timesteps 3689.
Path 171 | total_timesteps 3698.
Path 172 | total_timesteps 3715.
Path 173 | total_timesteps 3747.
Path 174 | total_timesteps 3773.
Path 175 | total_timesteps 3796.
Path 176 | total_timesteps 3823.
Path 177 | total_timesteps 3839.
Path 178 | total_timesteps 3857.
Path 179 | total_timesteps 3872.
Path 180 | total_timesteps 3889.
Path 181 | total_timesteps 3906.
Path 182 | total_timesteps 3925.
Path 183 | total_timesteps 3949.
Path 184 | total_timesteps 3981.
Path 185 | total_timesteps 4002.
Path 186 | total_timesteps 4016.
Path 187 | total_timesteps 4034.
Path 188 | total_timesteps 4060.
Path 189 | total_timesteps 4081.
Path 190 | total_timesteps 4107.
Path 191 | total_timesteps 4135.
Path 192 | total_timesteps 4155.
Path 193 | total_timesteps 4165.
Path 194 | total_timesteps 4178.
Path 195 | total_timesteps 4197.
Path 196 | total_timesteps 4214.
Path 197 | total_timesteps 4237.
Path 198 | total_timesteps 4259.
Path 199 | total_timesteps 4278.
Path 200 | total_timesteps 4293.
Path 201 | total_timesteps 4311.
Path 202 | total_timesteps 4330.
Path 203 | total_timesteps 4351.
Path 204 | total_timesteps 4372.
Path 205 | total_timesteps 4400.
Path 206 | total_timesteps 4419.
Path 207 | total_timesteps 4437.
Path 208 | total_timesteps 4453.
Path 209 | total_timesteps 4472.
Path 210 | total_timesteps 4490.
Path 211 | total_timesteps 4505.
Path 212 | total_timesteps 4517.
Path 213 | total_timesteps 4538.
Path 214 | total_timesteps 4559.
Path 215 | total_timesteps 4589.
Path 216 | total_timesteps 4611.
Path 217 | total_timesteps 4636.
Path 218 | total_timesteps 4659.
Path 219 | total_timesteps 4685.
Path 220 | total_timesteps 4704.
Path 221 | total_timesteps 4718.
Path 222 | total_timesteps 4732.
Path 223 | total_timesteps 4761.
Path 224 | total_timesteps 4779.
Path 225 | total_timesteps 4805.
Path 226 | total_timesteps 4819.
Path 227 | total_timesteps 4835.
Path 228 | total_timesteps 4849.
Path 229 | total_timesteps 4876.
Path 230 | total_timesteps 4891.
Path 231 | total_timesteps 4912.
Path 232 | total_timesteps 4945.
Path 233 | total_timesteps 4971.
Path 234 | total_timesteps 4988.
Path 235 | total_timesteps 5008.
Path 236 | total_timesteps 5024.
Path 237 | total_timesteps 5051.
Path 238 | total_timesteps 5066.
Path 239 | total_timesteps 5130.
Path 240 | total_timesteps 5151.
Path 241 | total_timesteps 5164.
Path 242 | total_timesteps 5179.
Path 243 | total_timesteps 5194.
Path 244 | total_timesteps 5210.
Path 245 | total_timesteps 5225.
Path 246 | total_timesteps 5242.
Path 247 | total_timesteps 5260.
Path 248 | total_timesteps 5274.
Path 249 | total_timesteps 5300.
Path 250 | total_timesteps 5309.
Path 251 | total_timesteps 5341.
Path 252 | total_timesteps 5364.
Path 253 | total_timesteps 5381.
Path 254 | total_timesteps 5405.
Path 255 | total_timesteps 5430.
Path 256 | total_timesteps 5452.
Path 257 | total_timesteps 5469.
Path 258 | total_timesteps 5490.
Path 259 | total_timesteps 5501.
Path 260 | total_timesteps 5531.
Path 261 | total_timesteps 5550.
Path 262 | total_timesteps 5570.
Path 263 | total_timesteps 5609.
Path 264 | total_timesteps 5621.
Path 265 | total_timesteps 5637.
Path 266 | total_timesteps 5658.
Path 267 | total_timesteps 5668.
Path 268 | total_timesteps 5711.
Path 269 | total_timesteps 5734.
Path 270 | total_timesteps 5764.
Path 271 | total_timesteps 5781.
Path 272 | total_timesteps 5816.
Path 273 | total_timesteps 5828.
Path 274 | total_timesteps 5844.
Path 275 | total_timesteps 5873.
Path 276 | total_timesteps 5895.
Path 277 | total_timesteps 5904.
Path 278 | total_timesteps 5925.
Path 279 | total_timesteps 5954.
Path 280 | total_timesteps 5967.
Path 281 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.8    |
| Iteration     | 7        |
| MaximumReturn | 19.8     |
| MinimumReturn | -27.4    |
| TotalSamples  | 36090    |
----------------------------
itr #8 | 
Fitting dynamics.
Validation loss = 0.011094371788203716
Validation loss = 0.009061329998075962
Validation loss = 0.008172695524990559
Validation loss = 0.008460434153676033
Validation loss = 0.00845652911812067
Validation loss = 0.007991121150553226
Validation loss = 0.008640716783702374
Validation loss = 0.008324829861521721
Validation loss = 0.007661423645913601
Validation loss = 0.008404325693845749
Validation loss = 0.007826675660908222
Validation loss = 0.007384289521723986
Validation loss = 0.007763737812638283
Validation loss = 0.007438262924551964
Validation loss = 0.007367616053670645
Validation loss = 0.008134475909173489
Validation loss = 0.0077954065054655075
Validation loss = 0.008002929389476776
Validation loss = 0.007612364366650581
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 29.
Path 2 | total_timesteps 54.
Path 3 | total_timesteps 71.
Path 4 | total_timesteps 96.
Path 5 | total_timesteps 110.
Path 6 | total_timesteps 130.
Path 7 | total_timesteps 152.
Path 8 | total_timesteps 168.
Path 9 | total_timesteps 180.
Path 10 | total_timesteps 201.
Path 11 | total_timesteps 239.
Path 12 | total_timesteps 260.
Path 13 | total_timesteps 283.
Path 14 | total_timesteps 297.
Path 15 | total_timesteps 318.
Path 16 | total_timesteps 336.
Path 17 | total_timesteps 354.
Path 18 | total_timesteps 395.
Path 19 | total_timesteps 416.
Path 20 | total_timesteps 430.
Path 21 | total_timesteps 453.
Path 22 | total_timesteps 471.
Path 23 | total_timesteps 489.
Path 24 | total_timesteps 501.
Path 25 | total_timesteps 513.
Path 26 | total_timesteps 547.
Path 27 | total_timesteps 577.
Path 28 | total_timesteps 610.
Path 29 | total_timesteps 629.
Path 30 | total_timesteps 669.
Path 31 | total_timesteps 680.
Path 32 | total_timesteps 708.
Path 33 | total_timesteps 731.
Path 34 | total_timesteps 748.
Path 35 | total_timesteps 778.
Path 36 | total_timesteps 792.
Path 37 | total_timesteps 821.
Path 38 | total_timesteps 837.
Path 39 | total_timesteps 847.
Path 40 | total_timesteps 875.
Path 41 | total_timesteps 910.
Path 42 | total_timesteps 940.
Path 43 | total_timesteps 953.
Path 44 | total_timesteps 990.
Path 45 | total_timesteps 1005.
Path 46 | total_timesteps 1017.
Path 47 | total_timesteps 1036.
Path 48 | total_timesteps 1069.
Path 49 | total_timesteps 1098.
Path 50 | total_timesteps 1129.
Path 51 | total_timesteps 1142.
Path 52 | total_timesteps 1159.
Path 53 | total_timesteps 1183.
Path 54 | total_timesteps 1209.
Path 55 | total_timesteps 1228.
Path 56 | total_timesteps 1244.
Path 57 | total_timesteps 1279.
Path 58 | total_timesteps 1305.
Path 59 | total_timesteps 1324.
Path 60 | total_timesteps 1351.
Path 61 | total_timesteps 1375.
Path 62 | total_timesteps 1386.
Path 63 | total_timesteps 1403.
Path 64 | total_timesteps 1413.
Path 65 | total_timesteps 1431.
Path 66 | total_timesteps 1464.
Path 67 | total_timesteps 1490.
Path 68 | total_timesteps 1513.
Path 69 | total_timesteps 1525.
Path 70 | total_timesteps 1536.
Path 71 | total_timesteps 1551.
Path 72 | total_timesteps 1577.
Path 73 | total_timesteps 1597.
Path 74 | total_timesteps 1617.
Path 75 | total_timesteps 1633.
Path 76 | total_timesteps 1676.
Path 77 | total_timesteps 1691.
Path 78 | total_timesteps 1706.
Path 79 | total_timesteps 1733.
Path 80 | total_timesteps 1746.
Path 81 | total_timesteps 1762.
Path 82 | total_timesteps 1799.
Path 83 | total_timesteps 1813.
Path 84 | total_timesteps 1830.
Path 85 | total_timesteps 1842.
Path 86 | total_timesteps 1856.
Path 87 | total_timesteps 1876.
Path 88 | total_timesteps 1903.
Path 89 | total_timesteps 1919.
Path 90 | total_timesteps 1955.
Path 91 | total_timesteps 1974.
Path 92 | total_timesteps 1991.
Path 93 | total_timesteps 2009.
Path 94 | total_timesteps 2033.
Path 95 | total_timesteps 2066.
Path 96 | total_timesteps 2093.
Path 97 | total_timesteps 2123.
Path 98 | total_timesteps 2136.
Path 99 | total_timesteps 2164.
Path 100 | total_timesteps 2195.
Path 101 | total_timesteps 2213.
Path 102 | total_timesteps 2227.
Path 103 | total_timesteps 2243.
Path 104 | total_timesteps 2266.
Path 105 | total_timesteps 2289.
Path 106 | total_timesteps 2310.
Path 107 | total_timesteps 2343.
Path 108 | total_timesteps 2371.
Path 109 | total_timesteps 2388.
Path 110 | total_timesteps 2404.
Path 111 | total_timesteps 2430.
Path 112 | total_timesteps 2449.
Path 113 | total_timesteps 2466.
Path 114 | total_timesteps 2485.
Path 115 | total_timesteps 2522.
Path 116 | total_timesteps 2545.
Path 117 | total_timesteps 2566.
Path 118 | total_timesteps 2580.
Path 119 | total_timesteps 2595.
Path 120 | total_timesteps 2618.
Path 121 | total_timesteps 2636.
Path 122 | total_timesteps 2656.
Path 123 | total_timesteps 2685.
Path 124 | total_timesteps 2699.
Path 125 | total_timesteps 2715.
Path 126 | total_timesteps 2727.
Path 127 | total_timesteps 2741.
Path 128 | total_timesteps 2754.
Path 129 | total_timesteps 2786.
Path 130 | total_timesteps 2809.
Path 131 | total_timesteps 2840.
Path 132 | total_timesteps 2877.
Path 133 | total_timesteps 2898.
Path 134 | total_timesteps 2914.
Path 135 | total_timesteps 2940.
Path 136 | total_timesteps 2960.
Path 137 | total_timesteps 3003.
Path 138 | total_timesteps 3021.
Path 139 | total_timesteps 3037.
Path 140 | total_timesteps 3051.
Path 141 | total_timesteps 3069.
Path 142 | total_timesteps 3081.
Path 143 | total_timesteps 3096.
Path 144 | total_timesteps 3119.
Path 145 | total_timesteps 3136.
Path 146 | total_timesteps 3161.
Path 147 | total_timesteps 3181.
Path 148 | total_timesteps 3200.
Path 149 | total_timesteps 3225.
Path 150 | total_timesteps 3248.
Path 151 | total_timesteps 3295.
Path 152 | total_timesteps 3318.
Path 153 | total_timesteps 3340.
Path 154 | total_timesteps 3364.
Path 155 | total_timesteps 3397.
Path 156 | total_timesteps 3407.
Path 157 | total_timesteps 3426.
Path 158 | total_timesteps 3439.
Path 159 | total_timesteps 3448.
Path 160 | total_timesteps 3469.
Path 161 | total_timesteps 3487.
Path 162 | total_timesteps 3501.
Path 163 | total_timesteps 3521.
Path 164 | total_timesteps 3541.
Path 165 | total_timesteps 3554.
Path 166 | total_timesteps 3583.
Path 167 | total_timesteps 3606.
Path 168 | total_timesteps 3627.
Path 169 | total_timesteps 3651.
Path 170 | total_timesteps 3663.
Path 171 | total_timesteps 3672.
Path 172 | total_timesteps 3697.
Path 173 | total_timesteps 3713.
Path 174 | total_timesteps 3741.
Path 175 | total_timesteps 3788.
Path 176 | total_timesteps 3811.
Path 177 | total_timesteps 3822.
Path 178 | total_timesteps 3841.
Path 179 | total_timesteps 3859.
Path 180 | total_timesteps 3877.
Path 181 | total_timesteps 3886.
Path 182 | total_timesteps 3909.
Path 183 | total_timesteps 3936.
Path 184 | total_timesteps 3955.
Path 185 | total_timesteps 3976.
Path 186 | total_timesteps 3993.
Path 187 | total_timesteps 4025.
Path 188 | total_timesteps 4041.
Path 189 | total_timesteps 4065.
Path 190 | total_timesteps 4104.
Path 191 | total_timesteps 4119.
Path 192 | total_timesteps 4131.
Path 193 | total_timesteps 4160.
Path 194 | total_timesteps 4185.
Path 195 | total_timesteps 4209.
Path 196 | total_timesteps 4235.
Path 197 | total_timesteps 4257.
Path 198 | total_timesteps 4266.
Path 199 | total_timesteps 4283.
Path 200 | total_timesteps 4300.
Path 201 | total_timesteps 4312.
Path 202 | total_timesteps 4334.
Path 203 | total_timesteps 4357.
Path 204 | total_timesteps 4377.
Path 205 | total_timesteps 4397.
Path 206 | total_timesteps 4419.
Path 207 | total_timesteps 4436.
Path 208 | total_timesteps 4488.
Path 209 | total_timesteps 4510.
Path 210 | total_timesteps 4527.
Path 211 | total_timesteps 4548.
Path 212 | total_timesteps 4573.
Path 213 | total_timesteps 4592.
Path 214 | total_timesteps 4612.
Path 215 | total_timesteps 4629.
Path 216 | total_timesteps 4655.
Path 217 | total_timesteps 4667.
Path 218 | total_timesteps 4695.
Path 219 | total_timesteps 4715.
Path 220 | total_timesteps 4735.
Path 221 | total_timesteps 4755.
Path 222 | total_timesteps 4773.
Path 223 | total_timesteps 4792.
Path 224 | total_timesteps 4819.
Path 225 | total_timesteps 4832.
Path 226 | total_timesteps 4852.
Path 227 | total_timesteps 4864.
Path 228 | total_timesteps 4880.
Path 229 | total_timesteps 4898.
Path 230 | total_timesteps 4912.
Path 231 | total_timesteps 4931.
Path 232 | total_timesteps 4963.
Path 233 | total_timesteps 4974.
Path 234 | total_timesteps 4987.
Path 235 | total_timesteps 5014.
Path 236 | total_timesteps 5040.
Path 237 | total_timesteps 5070.
Path 238 | total_timesteps 5097.
Path 239 | total_timesteps 5114.
Path 240 | total_timesteps 5166.
Path 241 | total_timesteps 5186.
Path 242 | total_timesteps 5206.
Path 243 | total_timesteps 5223.
Path 244 | total_timesteps 5252.
Path 245 | total_timesteps 5265.
Path 246 | total_timesteps 5291.
Path 247 | total_timesteps 5304.
Path 248 | total_timesteps 5323.
Path 249 | total_timesteps 5345.
Path 250 | total_timesteps 5360.
Path 251 | total_timesteps 5377.
Path 252 | total_timesteps 5392.
Path 253 | total_timesteps 5419.
Path 254 | total_timesteps 5433.
Path 255 | total_timesteps 5472.
Path 256 | total_timesteps 5483.
Path 257 | total_timesteps 5502.
Path 258 | total_timesteps 5521.
Path 259 | total_timesteps 5530.
Path 260 | total_timesteps 5560.
Path 261 | total_timesteps 5579.
Path 262 | total_timesteps 5599.
Path 263 | total_timesteps 5624.
Path 264 | total_timesteps 5642.
Path 265 | total_timesteps 5671.
Path 266 | total_timesteps 5691.
Path 267 | total_timesteps 5720.
Path 268 | total_timesteps 5756.
Path 269 | total_timesteps 5766.
Path 270 | total_timesteps 5784.
Path 271 | total_timesteps 5806.
Path 272 | total_timesteps 5835.
Path 273 | total_timesteps 5850.
Path 274 | total_timesteps 5869.
Path 275 | total_timesteps 5891.
Path 276 | total_timesteps 5906.
Path 277 | total_timesteps 5932.
Path 278 | total_timesteps 5954.
Path 279 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.8    |
| Iteration     | 8        |
| MaximumReturn | 4.51     |
| MinimumReturn | -23      |
| TotalSamples  | 40096    |
----------------------------
itr #9 | 
Fitting dynamics.
Validation loss = 0.010211249813437462
Validation loss = 0.007330566644668579
Validation loss = 0.007560769561678171
Validation loss = 0.00825762189924717
Validation loss = 0.007676325738430023
Validation loss = 0.006448870059102774
Validation loss = 0.0078822560608387
Validation loss = 0.0072482191026210785
Validation loss = 0.00714719807729125
Validation loss = 0.008248920552432537
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 22.
Path 2 | total_timesteps 44.
Path 3 | total_timesteps 83.
Path 4 | total_timesteps 99.
Path 5 | total_timesteps 117.
Path 6 | total_timesteps 135.
Path 7 | total_timesteps 163.
Path 8 | total_timesteps 184.
Path 9 | total_timesteps 218.
Path 10 | total_timesteps 260.
Path 11 | total_timesteps 274.
Path 12 | total_timesteps 297.
Path 13 | total_timesteps 318.
Path 14 | total_timesteps 338.
Path 15 | total_timesteps 348.
Path 16 | total_timesteps 374.
Path 17 | total_timesteps 397.
Path 18 | total_timesteps 420.
Path 19 | total_timesteps 438.
Path 20 | total_timesteps 459.
Path 21 | total_timesteps 494.
Path 22 | total_timesteps 515.
Path 23 | total_timesteps 531.
Path 24 | total_timesteps 551.
Path 25 | total_timesteps 576.
Path 26 | total_timesteps 599.
Path 27 | total_timesteps 619.
Path 28 | total_timesteps 636.
Path 29 | total_timesteps 675.
Path 30 | total_timesteps 709.
Path 31 | total_timesteps 731.
Path 32 | total_timesteps 742.
Path 33 | total_timesteps 763.
Path 34 | total_timesteps 783.
Path 35 | total_timesteps 800.
Path 36 | total_timesteps 820.
Path 37 | total_timesteps 845.
Path 38 | total_timesteps 870.
Path 39 | total_timesteps 900.
Path 40 | total_timesteps 935.
Path 41 | total_timesteps 965.
Path 42 | total_timesteps 986.
Path 43 | total_timesteps 1008.
Path 44 | total_timesteps 1036.
Path 45 | total_timesteps 1049.
Path 46 | total_timesteps 1069.
Path 47 | total_timesteps 1087.
Path 48 | total_timesteps 1122.
Path 49 | total_timesteps 1139.
Path 50 | total_timesteps 1172.
Path 51 | total_timesteps 1196.
Path 52 | total_timesteps 1215.
Path 53 | total_timesteps 1227.
Path 54 | total_timesteps 1251.
Path 55 | total_timesteps 1267.
Path 56 | total_timesteps 1293.
Path 57 | total_timesteps 1305.
Path 58 | total_timesteps 1332.
Path 59 | total_timesteps 1349.
Path 60 | total_timesteps 1364.
Path 61 | total_timesteps 1397.
Path 62 | total_timesteps 1423.
Path 63 | total_timesteps 1439.
Path 64 | total_timesteps 1460.
Path 65 | total_timesteps 1479.
Path 66 | total_timesteps 1492.
Path 67 | total_timesteps 1507.
Path 68 | total_timesteps 1540.
Path 69 | total_timesteps 1571.
Path 70 | total_timesteps 1590.
Path 71 | total_timesteps 1607.
Path 72 | total_timesteps 1628.
Path 73 | total_timesteps 1653.
Path 74 | total_timesteps 1672.
Path 75 | total_timesteps 1692.
Path 76 | total_timesteps 1726.
Path 77 | total_timesteps 1741.
Path 78 | total_timesteps 1780.
Path 79 | total_timesteps 1794.
Path 80 | total_timesteps 1835.
Path 81 | total_timesteps 1862.
Path 82 | total_timesteps 1884.
Path 83 | total_timesteps 1904.
Path 84 | total_timesteps 1924.
Path 85 | total_timesteps 1942.
Path 86 | total_timesteps 1966.
Path 87 | total_timesteps 1980.
Path 88 | total_timesteps 2000.
Path 89 | total_timesteps 2032.
Path 90 | total_timesteps 2042.
Path 91 | total_timesteps 2059.
Path 92 | total_timesteps 2074.
Path 93 | total_timesteps 2105.
Path 94 | total_timesteps 2129.
Path 95 | total_timesteps 2151.
Path 96 | total_timesteps 2176.
Path 97 | total_timesteps 2194.
Path 98 | total_timesteps 2219.
Path 99 | total_timesteps 2246.
Path 100 | total_timesteps 2274.
Path 101 | total_timesteps 2290.
Path 102 | total_timesteps 2317.
Path 103 | total_timesteps 2330.
Path 104 | total_timesteps 2355.
Path 105 | total_timesteps 2385.
Path 106 | total_timesteps 2400.
Path 107 | total_timesteps 2437.
Path 108 | total_timesteps 2467.
Path 109 | total_timesteps 2498.
Path 110 | total_timesteps 2510.
Path 111 | total_timesteps 2540.
Path 112 | total_timesteps 2550.
Path 113 | total_timesteps 2601.
Path 114 | total_timesteps 2624.
Path 115 | total_timesteps 2650.
Path 116 | total_timesteps 2679.
Path 117 | total_timesteps 2706.
Path 118 | total_timesteps 2737.
Path 119 | total_timesteps 2752.
Path 120 | total_timesteps 2775.
Path 121 | total_timesteps 2787.
Path 122 | total_timesteps 2810.
Path 123 | total_timesteps 2826.
Path 124 | total_timesteps 2852.
Path 125 | total_timesteps 2872.
Path 126 | total_timesteps 2889.
Path 127 | total_timesteps 2931.
Path 128 | total_timesteps 2967.
Path 129 | total_timesteps 2982.
Path 130 | total_timesteps 3002.
Path 131 | total_timesteps 3018.
Path 132 | total_timesteps 3054.
Path 133 | total_timesteps 3070.
Path 134 | total_timesteps 3110.
Path 135 | total_timesteps 3120.
Path 136 | total_timesteps 3143.
Path 137 | total_timesteps 3161.
Path 138 | total_timesteps 3188.
Path 139 | total_timesteps 3210.
Path 140 | total_timesteps 3235.
Path 141 | total_timesteps 3248.
Path 142 | total_timesteps 3271.
Path 143 | total_timesteps 3292.
Path 144 | total_timesteps 3305.
Path 145 | total_timesteps 3329.
Path 146 | total_timesteps 3363.
Path 147 | total_timesteps 3391.
Path 148 | total_timesteps 3410.
Path 149 | total_timesteps 3439.
Path 150 | total_timesteps 3463.
Path 151 | total_timesteps 3484.
Path 152 | total_timesteps 3509.
Path 153 | total_timesteps 3553.
Path 154 | total_timesteps 3587.
Path 155 | total_timesteps 3611.
Path 156 | total_timesteps 3632.
Path 157 | total_timesteps 3644.
Path 158 | total_timesteps 3663.
Path 159 | total_timesteps 3684.
Path 160 | total_timesteps 3703.
Path 161 | total_timesteps 3723.
Path 162 | total_timesteps 3746.
Path 163 | total_timesteps 3766.
Path 164 | total_timesteps 3786.
Path 165 | total_timesteps 3800.
Path 166 | total_timesteps 3819.
Path 167 | total_timesteps 3848.
Path 168 | total_timesteps 3875.
Path 169 | total_timesteps 3897.
Path 170 | total_timesteps 3917.
Path 171 | total_timesteps 3932.
Path 172 | total_timesteps 3951.
Path 173 | total_timesteps 3966.
Path 174 | total_timesteps 3981.
Path 175 | total_timesteps 4002.
Path 176 | total_timesteps 4023.
Path 177 | total_timesteps 4068.
Path 178 | total_timesteps 4088.
Path 179 | total_timesteps 4107.
Path 180 | total_timesteps 4128.
Path 181 | total_timesteps 4148.
Path 182 | total_timesteps 4172.
Path 183 | total_timesteps 4194.
Path 184 | total_timesteps 4219.
Path 185 | total_timesteps 4285.
Path 186 | total_timesteps 4308.
Path 187 | total_timesteps 4325.
Path 188 | total_timesteps 4388.
Path 189 | total_timesteps 4408.
Path 190 | total_timesteps 4430.
Path 191 | total_timesteps 4448.
Path 192 | total_timesteps 4472.
Path 193 | total_timesteps 4505.
Path 194 | total_timesteps 4530.
Path 195 | total_timesteps 4575.
Path 196 | total_timesteps 4601.
Path 197 | total_timesteps 4623.
Path 198 | total_timesteps 4648.
Path 199 | total_timesteps 4669.
Path 200 | total_timesteps 4691.
Path 201 | total_timesteps 4708.
Path 202 | total_timesteps 4726.
Path 203 | total_timesteps 4750.
Path 204 | total_timesteps 4768.
Path 205 | total_timesteps 4789.
Path 206 | total_timesteps 4832.
Path 207 | total_timesteps 4853.
Path 208 | total_timesteps 4882.
Path 209 | total_timesteps 4902.
Path 210 | total_timesteps 4916.
Path 211 | total_timesteps 4936.
Path 212 | total_timesteps 4991.
Path 213 | total_timesteps 5020.
Path 214 | total_timesteps 5046.
Path 215 | total_timesteps 5065.
Path 216 | total_timesteps 5084.
Path 217 | total_timesteps 5114.
Path 218 | total_timesteps 5146.
Path 219 | total_timesteps 5175.
Path 220 | total_timesteps 5192.
Path 221 | total_timesteps 5219.
Path 222 | total_timesteps 5239.
Path 223 | total_timesteps 5253.
Path 224 | total_timesteps 5275.
Path 225 | total_timesteps 5287.
Path 226 | total_timesteps 5301.
Path 227 | total_timesteps 5327.
Path 228 | total_timesteps 5366.
Path 229 | total_timesteps 5391.
Path 230 | total_timesteps 5418.
Path 231 | total_timesteps 5453.
Path 232 | total_timesteps 5472.
Path 233 | total_timesteps 5498.
Path 234 | total_timesteps 5531.
Path 235 | total_timesteps 5548.
Path 236 | total_timesteps 5572.
Path 237 | total_timesteps 5586.
Path 238 | total_timesteps 5610.
Path 239 | total_timesteps 5620.
Path 240 | total_timesteps 5639.
Path 241 | total_timesteps 5664.
Path 242 | total_timesteps 5680.
Path 243 | total_timesteps 5701.
Path 244 | total_timesteps 5720.
Path 245 | total_timesteps 5740.
Path 246 | total_timesteps 5761.
Path 247 | total_timesteps 5782.
Path 248 | total_timesteps 5804.
Path 249 | total_timesteps 5826.
Path 250 | total_timesteps 5835.
Path 251 | total_timesteps 5860.
Path 252 | total_timesteps 5886.
Path 253 | total_timesteps 5900.
Path 254 | total_timesteps 5911.
Path 255 | total_timesteps 5929.
Path 256 | total_timesteps 5950.
Path 257 | total_timesteps 5970.
Path 258 | total_timesteps 5987.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.4    |
| Iteration     | 9        |
| MaximumReturn | 6.72     |
| MinimumReturn | -32.3    |
| TotalSamples  | 44102    |
----------------------------
itr #10 | 
Fitting dynamics.
Validation loss = 0.009562809020280838
Validation loss = 0.006887364201247692
Validation loss = 0.007491639349609613
Validation loss = 0.006537821609526873
Validation loss = 0.006396641954779625
Validation loss = 0.008593290112912655
Validation loss = 0.006868613883852959
Validation loss = 0.0070023490116000175
Validation loss = 0.006244773976504803
Validation loss = 0.00679402332752943
Validation loss = 0.006909829564392567
Validation loss = 0.006425740197300911
Validation loss = 0.006873913574963808
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 34.
Path 3 | total_timesteps 58.
Path 4 | total_timesteps 78.
Path 5 | total_timesteps 101.
Path 6 | total_timesteps 123.
Path 7 | total_timesteps 147.
Path 8 | total_timesteps 161.
Path 9 | total_timesteps 187.
Path 10 | total_timesteps 205.
Path 11 | total_timesteps 220.
Path 12 | total_timesteps 240.
Path 13 | total_timesteps 255.
Path 14 | total_timesteps 279.
Path 15 | total_timesteps 308.
Path 16 | total_timesteps 342.
Path 17 | total_timesteps 356.
Path 18 | total_timesteps 382.
Path 19 | total_timesteps 412.
Path 20 | total_timesteps 444.
Path 21 | total_timesteps 471.
Path 22 | total_timesteps 502.
Path 23 | total_timesteps 536.
Path 24 | total_timesteps 558.
Path 25 | total_timesteps 573.
Path 26 | total_timesteps 606.
Path 27 | total_timesteps 630.
Path 28 | total_timesteps 664.
Path 29 | total_timesteps 686.
Path 30 | total_timesteps 715.
Path 31 | total_timesteps 745.
Path 32 | total_timesteps 774.
Path 33 | total_timesteps 795.
Path 34 | total_timesteps 810.
Path 35 | total_timesteps 830.
Path 36 | total_timesteps 852.
Path 37 | total_timesteps 867.
Path 38 | total_timesteps 891.
Path 39 | total_timesteps 913.
Path 40 | total_timesteps 937.
Path 41 | total_timesteps 958.
Path 42 | total_timesteps 980.
Path 43 | total_timesteps 1002.
Path 44 | total_timesteps 1015.
Path 45 | total_timesteps 1037.
Path 46 | total_timesteps 1072.
Path 47 | total_timesteps 1088.
Path 48 | total_timesteps 1105.
Path 49 | total_timesteps 1135.
Path 50 | total_timesteps 1156.
Path 51 | total_timesteps 1182.
Path 52 | total_timesteps 1202.
Path 53 | total_timesteps 1227.
Path 54 | total_timesteps 1252.
Path 55 | total_timesteps 1283.
Path 56 | total_timesteps 1308.
Path 57 | total_timesteps 1348.
Path 58 | total_timesteps 1370.
Path 59 | total_timesteps 1382.
Path 60 | total_timesteps 1391.
Path 61 | total_timesteps 1406.
Path 62 | total_timesteps 1428.
Path 63 | total_timesteps 1451.
Path 64 | total_timesteps 1465.
Path 65 | total_timesteps 1482.
Path 66 | total_timesteps 1502.
Path 67 | total_timesteps 1519.
Path 68 | total_timesteps 1552.
Path 69 | total_timesteps 1579.
Path 70 | total_timesteps 1607.
Path 71 | total_timesteps 1624.
Path 72 | total_timesteps 1660.
Path 73 | total_timesteps 1682.
Path 74 | total_timesteps 1697.
Path 75 | total_timesteps 1711.
Path 76 | total_timesteps 1733.
Path 77 | total_timesteps 1762.
Path 78 | total_timesteps 1790.
Path 79 | total_timesteps 1817.
Path 80 | total_timesteps 1848.
Path 81 | total_timesteps 1863.
Path 82 | total_timesteps 1888.
Path 83 | total_timesteps 1914.
Path 84 | total_timesteps 1934.
Path 85 | total_timesteps 1951.
Path 86 | total_timesteps 1972.
Path 87 | total_timesteps 1984.
Path 88 | total_timesteps 2002.
Path 89 | total_timesteps 2049.
Path 90 | total_timesteps 2065.
Path 91 | total_timesteps 2080.
Path 92 | total_timesteps 2102.
Path 93 | total_timesteps 2123.
Path 94 | total_timesteps 2141.
Path 95 | total_timesteps 2170.
Path 96 | total_timesteps 2182.
Path 97 | total_timesteps 2205.
Path 98 | total_timesteps 2231.
Path 99 | total_timesteps 2271.
Path 100 | total_timesteps 2292.
Path 101 | total_timesteps 2314.
Path 102 | total_timesteps 2331.
Path 103 | total_timesteps 2352.
Path 104 | total_timesteps 2369.
Path 105 | total_timesteps 2398.
Path 106 | total_timesteps 2420.
Path 107 | total_timesteps 2445.
Path 108 | total_timesteps 2480.
Path 109 | total_timesteps 2498.
Path 110 | total_timesteps 2530.
Path 111 | total_timesteps 2554.
Path 112 | total_timesteps 2584.
Path 113 | total_timesteps 2603.
Path 114 | total_timesteps 2624.
Path 115 | total_timesteps 2645.
Path 116 | total_timesteps 2675.
Path 117 | total_timesteps 2699.
Path 118 | total_timesteps 2717.
Path 119 | total_timesteps 2741.
Path 120 | total_timesteps 2762.
Path 121 | total_timesteps 2773.
Path 122 | total_timesteps 2788.
Path 123 | total_timesteps 2803.
Path 124 | total_timesteps 2821.
Path 125 | total_timesteps 2838.
Path 126 | total_timesteps 2870.
Path 127 | total_timesteps 2892.
Path 128 | total_timesteps 2922.
Path 129 | total_timesteps 2944.
Path 130 | total_timesteps 2960.
Path 131 | total_timesteps 2985.
Path 132 | total_timesteps 2999.
Path 133 | total_timesteps 3023.
Path 134 | total_timesteps 3044.
Path 135 | total_timesteps 3057.
Path 136 | total_timesteps 3092.
Path 137 | total_timesteps 3116.
Path 138 | total_timesteps 3147.
Path 139 | total_timesteps 3163.
Path 140 | total_timesteps 3177.
Path 141 | total_timesteps 3188.
Path 142 | total_timesteps 3202.
Path 143 | total_timesteps 3239.
Path 144 | total_timesteps 3267.
Path 145 | total_timesteps 3284.
Path 146 | total_timesteps 3308.
Path 147 | total_timesteps 3332.
Path 148 | total_timesteps 3356.
Path 149 | total_timesteps 3369.
Path 150 | total_timesteps 3384.
Path 151 | total_timesteps 3413.
Path 152 | total_timesteps 3438.
Path 153 | total_timesteps 3475.
Path 154 | total_timesteps 3489.
Path 155 | total_timesteps 3503.
Path 156 | total_timesteps 3514.
Path 157 | total_timesteps 3538.
Path 158 | total_timesteps 3556.
Path 159 | total_timesteps 3566.
Path 160 | total_timesteps 3606.
Path 161 | total_timesteps 3634.
Path 162 | total_timesteps 3656.
Path 163 | total_timesteps 3681.
Path 164 | total_timesteps 3699.
Path 165 | total_timesteps 3736.
Path 166 | total_timesteps 3762.
Path 167 | total_timesteps 3779.
Path 168 | total_timesteps 3797.
Path 169 | total_timesteps 3817.
Path 170 | total_timesteps 3836.
Path 171 | total_timesteps 3872.
Path 172 | total_timesteps 3897.
Path 173 | total_timesteps 3919.
Path 174 | total_timesteps 3935.
Path 175 | total_timesteps 3957.
Path 176 | total_timesteps 3973.
Path 177 | total_timesteps 3993.
Path 178 | total_timesteps 4012.
Path 179 | total_timesteps 4032.
Path 180 | total_timesteps 4052.
Path 181 | total_timesteps 4072.
Path 182 | total_timesteps 4085.
Path 183 | total_timesteps 4107.
Path 184 | total_timesteps 4118.
Path 185 | total_timesteps 4129.
Path 186 | total_timesteps 4146.
Path 187 | total_timesteps 4167.
Path 188 | total_timesteps 4195.
Path 189 | total_timesteps 4213.
Path 190 | total_timesteps 4251.
Path 191 | total_timesteps 4263.
Path 192 | total_timesteps 4283.
Path 193 | total_timesteps 4295.
Path 194 | total_timesteps 4309.
Path 195 | total_timesteps 4321.
Path 196 | total_timesteps 4340.
Path 197 | total_timesteps 4351.
Path 198 | total_timesteps 4371.
Path 199 | total_timesteps 4391.
Path 200 | total_timesteps 4405.
Path 201 | total_timesteps 4432.
Path 202 | total_timesteps 4449.
Path 203 | total_timesteps 4474.
Path 204 | total_timesteps 4496.
Path 205 | total_timesteps 4508.
Path 206 | total_timesteps 4530.
Path 207 | total_timesteps 4554.
Path 208 | total_timesteps 4588.
Path 209 | total_timesteps 4620.
Path 210 | total_timesteps 4647.
Path 211 | total_timesteps 4687.
Path 212 | total_timesteps 4710.
Path 213 | total_timesteps 4724.
Path 214 | total_timesteps 4739.
Path 215 | total_timesteps 4760.
Path 216 | total_timesteps 4784.
Path 217 | total_timesteps 4812.
Path 218 | total_timesteps 4832.
Path 219 | total_timesteps 4878.
Path 220 | total_timesteps 4917.
Path 221 | total_timesteps 4955.
Path 222 | total_timesteps 4974.
Path 223 | total_timesteps 4986.
Path 224 | total_timesteps 5014.
Path 225 | total_timesteps 5038.
Path 226 | total_timesteps 5059.
Path 227 | total_timesteps 5072.
Path 228 | total_timesteps 5082.
Path 229 | total_timesteps 5115.
Path 230 | total_timesteps 5148.
Path 231 | total_timesteps 5167.
Path 232 | total_timesteps 5188.
Path 233 | total_timesteps 5203.
Path 234 | total_timesteps 5224.
Path 235 | total_timesteps 5239.
Path 236 | total_timesteps 5258.
Path 237 | total_timesteps 5286.
Path 238 | total_timesteps 5322.
Path 239 | total_timesteps 5343.
Path 240 | total_timesteps 5367.
Path 241 | total_timesteps 5397.
Path 242 | total_timesteps 5414.
Path 243 | total_timesteps 5430.
Path 244 | total_timesteps 5439.
Path 245 | total_timesteps 5462.
Path 246 | total_timesteps 5480.
Path 247 | total_timesteps 5498.
Path 248 | total_timesteps 5518.
Path 249 | total_timesteps 5542.
Path 250 | total_timesteps 5573.
Path 251 | total_timesteps 5599.
Path 252 | total_timesteps 5613.
Path 253 | total_timesteps 5627.
Path 254 | total_timesteps 5644.
Path 255 | total_timesteps 5656.
Path 256 | total_timesteps 5683.
Path 257 | total_timesteps 5700.
Path 258 | total_timesteps 5713.
Path 259 | total_timesteps 5742.
Path 260 | total_timesteps 5766.
Path 261 | total_timesteps 5783.
Path 262 | total_timesteps 5800.
Path 263 | total_timesteps 5823.
Path 264 | total_timesteps 5838.
Path 265 | total_timesteps 5852.
Path 266 | total_timesteps 5886.
Path 267 | total_timesteps 5906.
Path 268 | total_timesteps 5922.
Path 269 | total_timesteps 5930.
Path 270 | total_timesteps 5949.
Path 271 | total_timesteps 5962.
Path 272 | total_timesteps 5988.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.1    |
| Iteration     | 10       |
| MaximumReturn | 5.63     |
| MinimumReturn | -22.6    |
| TotalSamples  | 48103    |
----------------------------
itr #11 | 
Fitting dynamics.
Validation loss = 0.006300866603851318
Validation loss = 0.0066405716352164745
Validation loss = 0.0074595496989786625
Validation loss = 0.006778165698051453
Validation loss = 0.007355718407779932
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 41.
Path 3 | total_timesteps 65.
Path 4 | total_timesteps 81.
Path 5 | total_timesteps 98.
Path 6 | total_timesteps 129.
Path 7 | total_timesteps 139.
Path 8 | total_timesteps 152.
Path 9 | total_timesteps 178.
Path 10 | total_timesteps 193.
Path 11 | total_timesteps 206.
Path 12 | total_timesteps 232.
Path 13 | total_timesteps 255.
Path 14 | total_timesteps 285.
Path 15 | total_timesteps 330.
Path 16 | total_timesteps 369.
Path 17 | total_timesteps 387.
Path 18 | total_timesteps 402.
Path 19 | total_timesteps 428.
Path 20 | total_timesteps 445.
Path 21 | total_timesteps 468.
Path 22 | total_timesteps 493.
Path 23 | total_timesteps 516.
Path 24 | total_timesteps 540.
Path 25 | total_timesteps 566.
Path 26 | total_timesteps 597.
Path 27 | total_timesteps 615.
Path 28 | total_timesteps 646.
Path 29 | total_timesteps 683.
Path 30 | total_timesteps 699.
Path 31 | total_timesteps 718.
Path 32 | total_timesteps 749.
Path 33 | total_timesteps 760.
Path 34 | total_timesteps 790.
Path 35 | total_timesteps 809.
Path 36 | total_timesteps 830.
Path 37 | total_timesteps 858.
Path 38 | total_timesteps 878.
Path 39 | total_timesteps 901.
Path 40 | total_timesteps 920.
Path 41 | total_timesteps 933.
Path 42 | total_timesteps 947.
Path 43 | total_timesteps 969.
Path 44 | total_timesteps 986.
Path 45 | total_timesteps 999.
Path 46 | total_timesteps 1031.
Path 47 | total_timesteps 1059.
Path 48 | total_timesteps 1084.
Path 49 | total_timesteps 1108.
Path 50 | total_timesteps 1130.
Path 51 | total_timesteps 1141.
Path 52 | total_timesteps 1167.
Path 53 | total_timesteps 1183.
Path 54 | total_timesteps 1208.
Path 55 | total_timesteps 1227.
Path 56 | total_timesteps 1255.
Path 57 | total_timesteps 1297.
Path 58 | total_timesteps 1323.
Path 59 | total_timesteps 1335.
Path 60 | total_timesteps 1377.
Path 61 | total_timesteps 1394.
Path 62 | total_timesteps 1409.
Path 63 | total_timesteps 1435.
Path 64 | total_timesteps 1459.
Path 65 | total_timesteps 1495.
Path 66 | total_timesteps 1536.
Path 67 | total_timesteps 1571.
Path 68 | total_timesteps 1593.
Path 69 | total_timesteps 1611.
Path 70 | total_timesteps 1633.
Path 71 | total_timesteps 1653.
Path 72 | total_timesteps 1665.
Path 73 | total_timesteps 1683.
Path 74 | total_timesteps 1698.
Path 75 | total_timesteps 1722.
Path 76 | total_timesteps 1743.
Path 77 | total_timesteps 1763.
Path 78 | total_timesteps 1776.
Path 79 | total_timesteps 1787.
Path 80 | total_timesteps 1799.
Path 81 | total_timesteps 1810.
Path 82 | total_timesteps 1821.
Path 83 | total_timesteps 1848.
Path 84 | total_timesteps 1859.
Path 85 | total_timesteps 1877.
Path 86 | total_timesteps 1903.
Path 87 | total_timesteps 1924.
Path 88 | total_timesteps 1952.
Path 89 | total_timesteps 1969.
Path 90 | total_timesteps 1984.
Path 91 | total_timesteps 2002.
Path 92 | total_timesteps 2037.
Path 93 | total_timesteps 2051.
Path 94 | total_timesteps 2072.
Path 95 | total_timesteps 2105.
Path 96 | total_timesteps 2133.
Path 97 | total_timesteps 2166.
Path 98 | total_timesteps 2186.
Path 99 | total_timesteps 2197.
Path 100 | total_timesteps 2208.
Path 101 | total_timesteps 2227.
Path 102 | total_timesteps 2271.
Path 103 | total_timesteps 2282.
Path 104 | total_timesteps 2296.
Path 105 | total_timesteps 2312.
Path 106 | total_timesteps 2334.
Path 107 | total_timesteps 2348.
Path 108 | total_timesteps 2374.
Path 109 | total_timesteps 2412.
Path 110 | total_timesteps 2444.
Path 111 | total_timesteps 2460.
Path 112 | total_timesteps 2488.
Path 113 | total_timesteps 2505.
Path 114 | total_timesteps 2544.
Path 115 | total_timesteps 2577.
Path 116 | total_timesteps 2599.
Path 117 | total_timesteps 2622.
Path 118 | total_timesteps 2641.
Path 119 | total_timesteps 2658.
Path 120 | total_timesteps 2680.
Path 121 | total_timesteps 2696.
Path 122 | total_timesteps 2723.
Path 123 | total_timesteps 2731.
Path 124 | total_timesteps 2744.
Path 125 | total_timesteps 2785.
Path 126 | total_timesteps 2797.
Path 127 | total_timesteps 2817.
Path 128 | total_timesteps 2847.
Path 129 | total_timesteps 2862.
Path 130 | total_timesteps 2885.
Path 131 | total_timesteps 2902.
Path 132 | total_timesteps 2920.
Path 133 | total_timesteps 2931.
Path 134 | total_timesteps 2974.
Path 135 | total_timesteps 2988.
Path 136 | total_timesteps 3001.
Path 137 | total_timesteps 3028.
Path 138 | total_timesteps 3044.
Path 139 | total_timesteps 3070.
Path 140 | total_timesteps 3091.
Path 141 | total_timesteps 3109.
Path 142 | total_timesteps 3126.
Path 143 | total_timesteps 3137.
Path 144 | total_timesteps 3156.
Path 145 | total_timesteps 3176.
Path 146 | total_timesteps 3193.
Path 147 | total_timesteps 3208.
Path 148 | total_timesteps 3220.
Path 149 | total_timesteps 3262.
Path 150 | total_timesteps 3278.
Path 151 | total_timesteps 3309.
Path 152 | total_timesteps 3344.
Path 153 | total_timesteps 3360.
Path 154 | total_timesteps 3380.
Path 155 | total_timesteps 3392.
Path 156 | total_timesteps 3409.
Path 157 | total_timesteps 3435.
Path 158 | total_timesteps 3453.
Path 159 | total_timesteps 3474.
Path 160 | total_timesteps 3503.
Path 161 | total_timesteps 3521.
Path 162 | total_timesteps 3544.
Path 163 | total_timesteps 3554.
Path 164 | total_timesteps 3579.
Path 165 | total_timesteps 3588.
Path 166 | total_timesteps 3622.
Path 167 | total_timesteps 3637.
Path 168 | total_timesteps 3655.
Path 169 | total_timesteps 3669.
Path 170 | total_timesteps 3678.
Path 171 | total_timesteps 3692.
Path 172 | total_timesteps 3718.
Path 173 | total_timesteps 3739.
Path 174 | total_timesteps 3767.
Path 175 | total_timesteps 3779.
Path 176 | total_timesteps 3805.
Path 177 | total_timesteps 3820.
Path 178 | total_timesteps 3876.
Path 179 | total_timesteps 3896.
Path 180 | total_timesteps 3910.
Path 181 | total_timesteps 3935.
Path 182 | total_timesteps 3950.
Path 183 | total_timesteps 3990.
Path 184 | total_timesteps 4029.
Path 185 | total_timesteps 4085.
Path 186 | total_timesteps 4105.
Path 187 | total_timesteps 4119.
Path 188 | total_timesteps 4142.
Path 189 | total_timesteps 4163.
Path 190 | total_timesteps 4201.
Path 191 | total_timesteps 4227.
Path 192 | total_timesteps 4249.
Path 193 | total_timesteps 4270.
Path 194 | total_timesteps 4298.
Path 195 | total_timesteps 4314.
Path 196 | total_timesteps 4323.
Path 197 | total_timesteps 4339.
Path 198 | total_timesteps 4362.
Path 199 | total_timesteps 4385.
Path 200 | total_timesteps 4428.
Path 201 | total_timesteps 4443.
Path 202 | total_timesteps 4455.
Path 203 | total_timesteps 4475.
Path 204 | total_timesteps 4505.
Path 205 | total_timesteps 4534.
Path 206 | total_timesteps 4556.
Path 207 | total_timesteps 4568.
Path 208 | total_timesteps 4592.
Path 209 | total_timesteps 4609.
Path 210 | total_timesteps 4630.
Path 211 | total_timesteps 4645.
Path 212 | total_timesteps 4670.
Path 213 | total_timesteps 4687.
Path 214 | total_timesteps 4701.
Path 215 | total_timesteps 4721.
Path 216 | total_timesteps 4748.
Path 217 | total_timesteps 4770.
Path 218 | total_timesteps 4806.
Path 219 | total_timesteps 4824.
Path 220 | total_timesteps 4849.
Path 221 | total_timesteps 4876.
Path 222 | total_timesteps 4889.
Path 223 | total_timesteps 4907.
Path 224 | total_timesteps 4930.
Path 225 | total_timesteps 4940.
Path 226 | total_timesteps 4959.
Path 227 | total_timesteps 4978.
Path 228 | total_timesteps 4988.
Path 229 | total_timesteps 5019.
Path 230 | total_timesteps 5055.
Path 231 | total_timesteps 5083.
Path 232 | total_timesteps 5103.
Path 233 | total_timesteps 5120.
Path 234 | total_timesteps 5135.
Path 235 | total_timesteps 5155.
Path 236 | total_timesteps 5188.
Path 237 | total_timesteps 5221.
Path 238 | total_timesteps 5246.
Path 239 | total_timesteps 5273.
Path 240 | total_timesteps 5295.
Path 241 | total_timesteps 5323.
Path 242 | total_timesteps 5336.
Path 243 | total_timesteps 5365.
Path 244 | total_timesteps 5393.
Path 245 | total_timesteps 5410.
Path 246 | total_timesteps 5435.
Path 247 | total_timesteps 5464.
Path 248 | total_timesteps 5476.
Path 249 | total_timesteps 5515.
Path 250 | total_timesteps 5528.
Path 251 | total_timesteps 5548.
Path 252 | total_timesteps 5571.
Path 253 | total_timesteps 5592.
Path 254 | total_timesteps 5604.
Path 255 | total_timesteps 5620.
Path 256 | total_timesteps 5641.
Path 257 | total_timesteps 5662.
Path 258 | total_timesteps 5691.
Path 259 | total_timesteps 5703.
Path 260 | total_timesteps 5722.
Path 261 | total_timesteps 5756.
Path 262 | total_timesteps 5769.
Path 263 | total_timesteps 5789.
Path 264 | total_timesteps 5808.
Path 265 | total_timesteps 5830.
Path 266 | total_timesteps 5846.
Path 267 | total_timesteps 5881.
Path 268 | total_timesteps 5903.
Path 269 | total_timesteps 5914.
Path 270 | total_timesteps 5931.
Path 271 | total_timesteps 5953.
Path 272 | total_timesteps 5971.
Path 273 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -8.96    |
| Iteration     | 11       |
| MaximumReturn | 12.7     |
| MinimumReturn | -24.9    |
| TotalSamples  | 52124    |
----------------------------
itr #12 | 
Fitting dynamics.
Validation loss = 0.006154673174023628
Validation loss = 0.006445338949561119
Validation loss = 0.005785882472991943
Validation loss = 0.007103711366653442
Validation loss = 0.006159692537039518
Validation loss = 0.005540347658097744
Validation loss = 0.005692404694855213
Validation loss = 0.006502558011561632
Validation loss = 0.006059897597879171
Validation loss = 0.0056665982119739056
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 33.
Path 3 | total_timesteps 70.
Path 4 | total_timesteps 112.
Path 5 | total_timesteps 132.
Path 6 | total_timesteps 144.
Path 7 | total_timesteps 162.
Path 8 | total_timesteps 180.
Path 9 | total_timesteps 191.
Path 10 | total_timesteps 222.
Path 11 | total_timesteps 256.
Path 12 | total_timesteps 276.
Path 13 | total_timesteps 291.
Path 14 | total_timesteps 305.
Path 15 | total_timesteps 319.
Path 16 | total_timesteps 339.
Path 17 | total_timesteps 352.
Path 18 | total_timesteps 376.
Path 19 | total_timesteps 392.
Path 20 | total_timesteps 410.
Path 21 | total_timesteps 432.
Path 22 | total_timesteps 444.
Path 23 | total_timesteps 471.
Path 24 | total_timesteps 486.
Path 25 | total_timesteps 503.
Path 26 | total_timesteps 527.
Path 27 | total_timesteps 541.
Path 28 | total_timesteps 563.
Path 29 | total_timesteps 585.
Path 30 | total_timesteps 596.
Path 31 | total_timesteps 613.
Path 32 | total_timesteps 631.
Path 33 | total_timesteps 643.
Path 34 | total_timesteps 674.
Path 35 | total_timesteps 690.
Path 36 | total_timesteps 705.
Path 37 | total_timesteps 716.
Path 38 | total_timesteps 744.
Path 39 | total_timesteps 762.
Path 40 | total_timesteps 786.
Path 41 | total_timesteps 808.
Path 42 | total_timesteps 819.
Path 43 | total_timesteps 833.
Path 44 | total_timesteps 855.
Path 45 | total_timesteps 879.
Path 46 | total_timesteps 908.
Path 47 | total_timesteps 932.
Path 48 | total_timesteps 947.
Path 49 | total_timesteps 974.
Path 50 | total_timesteps 1000.
Path 51 | total_timesteps 1023.
Path 52 | total_timesteps 1034.
Path 53 | total_timesteps 1050.
Path 54 | total_timesteps 1058.
Path 55 | total_timesteps 1089.
Path 56 | total_timesteps 1102.
Path 57 | total_timesteps 1116.
Path 58 | total_timesteps 1132.
Path 59 | total_timesteps 1150.
Path 60 | total_timesteps 1161.
Path 61 | total_timesteps 1175.
Path 62 | total_timesteps 1199.
Path 63 | total_timesteps 1218.
Path 64 | total_timesteps 1262.
Path 65 | total_timesteps 1285.
Path 66 | total_timesteps 1297.
Path 67 | total_timesteps 1333.
Path 68 | total_timesteps 1351.
Path 69 | total_timesteps 1370.
Path 70 | total_timesteps 1387.
Path 71 | total_timesteps 1404.
Path 72 | total_timesteps 1425.
Path 73 | total_timesteps 1444.
Path 74 | total_timesteps 1464.
Path 75 | total_timesteps 1493.
Path 76 | total_timesteps 1520.
Path 77 | total_timesteps 1534.
Path 78 | total_timesteps 1547.
Path 79 | total_timesteps 1576.
Path 80 | total_timesteps 1585.
Path 81 | total_timesteps 1609.
Path 82 | total_timesteps 1623.
Path 83 | total_timesteps 1637.
Path 84 | total_timesteps 1667.
Path 85 | total_timesteps 1684.
Path 86 | total_timesteps 1717.
Path 87 | total_timesteps 1734.
Path 88 | total_timesteps 1748.
Path 89 | total_timesteps 1759.
Path 90 | total_timesteps 1777.
Path 91 | total_timesteps 1792.
Path 92 | total_timesteps 1810.
Path 93 | total_timesteps 1829.
Path 94 | total_timesteps 1841.
Path 95 | total_timesteps 1854.
Path 96 | total_timesteps 1873.
Path 97 | total_timesteps 1883.
Path 98 | total_timesteps 1897.
Path 99 | total_timesteps 1911.
Path 100 | total_timesteps 1929.
Path 101 | total_timesteps 1948.
Path 102 | total_timesteps 1969.
Path 103 | total_timesteps 1988.
Path 104 | total_timesteps 2018.
Path 105 | total_timesteps 2032.
Path 106 | total_timesteps 2046.
Path 107 | total_timesteps 2066.
Path 108 | total_timesteps 2078.
Path 109 | total_timesteps 2087.
Path 110 | total_timesteps 2108.
Path 111 | total_timesteps 2128.
Path 112 | total_timesteps 2150.
Path 113 | total_timesteps 2168.
Path 114 | total_timesteps 2179.
Path 115 | total_timesteps 2192.
Path 116 | total_timesteps 2207.
Path 117 | total_timesteps 2221.
Path 118 | total_timesteps 2243.
Path 119 | total_timesteps 2253.
Path 120 | total_timesteps 2283.
Path 121 | total_timesteps 2298.
Path 122 | total_timesteps 2318.
Path 123 | total_timesteps 2336.
Path 124 | total_timesteps 2365.
Path 125 | total_timesteps 2391.
Path 126 | total_timesteps 2412.
Path 127 | total_timesteps 2428.
Path 128 | total_timesteps 2446.
Path 129 | total_timesteps 2458.
Path 130 | total_timesteps 2471.
Path 131 | total_timesteps 2483.
Path 132 | total_timesteps 2516.
Path 133 | total_timesteps 2543.
Path 134 | total_timesteps 2561.
Path 135 | total_timesteps 2575.
Path 136 | total_timesteps 2595.
Path 137 | total_timesteps 2618.
Path 138 | total_timesteps 2644.
Path 139 | total_timesteps 2659.
Path 140 | total_timesteps 2668.
Path 141 | total_timesteps 2685.
Path 142 | total_timesteps 2697.
Path 143 | total_timesteps 2716.
Path 144 | total_timesteps 2737.
Path 145 | total_timesteps 2753.
Path 146 | total_timesteps 2776.
Path 147 | total_timesteps 2793.
Path 148 | total_timesteps 2806.
Path 149 | total_timesteps 2818.
Path 150 | total_timesteps 2830.
Path 151 | total_timesteps 2847.
Path 152 | total_timesteps 2868.
Path 153 | total_timesteps 2886.
Path 154 | total_timesteps 2908.
Path 155 | total_timesteps 2925.
Path 156 | total_timesteps 2943.
Path 157 | total_timesteps 2975.
Path 158 | total_timesteps 2988.
Path 159 | total_timesteps 3008.
Path 160 | total_timesteps 3032.
Path 161 | total_timesteps 3050.
Path 162 | total_timesteps 3069.
Path 163 | total_timesteps 3082.
Path 164 | total_timesteps 3097.
Path 165 | total_timesteps 3113.
Path 166 | total_timesteps 3131.
Path 167 | total_timesteps 3154.
Path 168 | total_timesteps 3190.
Path 169 | total_timesteps 3225.
Path 170 | total_timesteps 3239.
Path 171 | total_timesteps 3259.
Path 172 | total_timesteps 3276.
Path 173 | total_timesteps 3299.
Path 174 | total_timesteps 3315.
Path 175 | total_timesteps 3340.
Path 176 | total_timesteps 3353.
Path 177 | total_timesteps 3366.
Path 178 | total_timesteps 3388.
Path 179 | total_timesteps 3412.
Path 180 | total_timesteps 3427.
Path 181 | total_timesteps 3455.
Path 182 | total_timesteps 3470.
Path 183 | total_timesteps 3496.
Path 184 | total_timesteps 3528.
Path 185 | total_timesteps 3569.
Path 186 | total_timesteps 3593.
Path 187 | total_timesteps 3608.
Path 188 | total_timesteps 3626.
Path 189 | total_timesteps 3640.
Path 190 | total_timesteps 3664.
Path 191 | total_timesteps 3677.
Path 192 | total_timesteps 3702.
Path 193 | total_timesteps 3718.
Path 194 | total_timesteps 3751.
Path 195 | total_timesteps 3775.
Path 196 | total_timesteps 3794.
Path 197 | total_timesteps 3805.
Path 198 | total_timesteps 3818.
Path 199 | total_timesteps 3833.
Path 200 | total_timesteps 3855.
Path 201 | total_timesteps 3873.
Path 202 | total_timesteps 3901.
Path 203 | total_timesteps 3913.
Path 204 | total_timesteps 3929.
Path 205 | total_timesteps 3955.
Path 206 | total_timesteps 3971.
Path 207 | total_timesteps 3998.
Path 208 | total_timesteps 4010.
Path 209 | total_timesteps 4025.
Path 210 | total_timesteps 4045.
Path 211 | total_timesteps 4060.
Path 212 | total_timesteps 4082.
Path 213 | total_timesteps 4096.
Path 214 | total_timesteps 4116.
Path 215 | total_timesteps 4134.
Path 216 | total_timesteps 4149.
Path 217 | total_timesteps 4160.
Path 218 | total_timesteps 4177.
Path 219 | total_timesteps 4200.
Path 220 | total_timesteps 4219.
Path 221 | total_timesteps 4234.
Path 222 | total_timesteps 4260.
Path 223 | total_timesteps 4288.
Path 224 | total_timesteps 4306.
Path 225 | total_timesteps 4323.
Path 226 | total_timesteps 4343.
Path 227 | total_timesteps 4371.
Path 228 | total_timesteps 4390.
Path 229 | total_timesteps 4400.
Path 230 | total_timesteps 4411.
Path 231 | total_timesteps 4425.
Path 232 | total_timesteps 4442.
Path 233 | total_timesteps 4460.
Path 234 | total_timesteps 4480.
Path 235 | total_timesteps 4492.
Path 236 | total_timesteps 4533.
Path 237 | total_timesteps 4552.
Path 238 | total_timesteps 4576.
Path 239 | total_timesteps 4602.
Path 240 | total_timesteps 4620.
Path 241 | total_timesteps 4640.
Path 242 | total_timesteps 4660.
Path 243 | total_timesteps 4691.
Path 244 | total_timesteps 4714.
Path 245 | total_timesteps 4737.
Path 246 | total_timesteps 4754.
Path 247 | total_timesteps 4771.
Path 248 | total_timesteps 4801.
Path 249 | total_timesteps 4826.
Path 250 | total_timesteps 4846.
Path 251 | total_timesteps 4871.
Path 252 | total_timesteps 4892.
Path 253 | total_timesteps 4916.
Path 254 | total_timesteps 4933.
Path 255 | total_timesteps 4958.
Path 256 | total_timesteps 4984.
Path 257 | total_timesteps 4998.
Path 258 | total_timesteps 5030.
Path 259 | total_timesteps 5061.
Path 260 | total_timesteps 5077.
Path 261 | total_timesteps 5093.
Path 262 | total_timesteps 5109.
Path 263 | total_timesteps 5128.
Path 264 | total_timesteps 5155.
Path 265 | total_timesteps 5181.
Path 266 | total_timesteps 5205.
Path 267 | total_timesteps 5215.
Path 268 | total_timesteps 5231.
Path 269 | total_timesteps 5244.
Path 270 | total_timesteps 5270.
Path 271 | total_timesteps 5291.
Path 272 | total_timesteps 5308.
Path 273 | total_timesteps 5321.
Path 274 | total_timesteps 5338.
Path 275 | total_timesteps 5351.
Path 276 | total_timesteps 5374.
Path 277 | total_timesteps 5404.
Path 278 | total_timesteps 5423.
Path 279 | total_timesteps 5445.
Path 280 | total_timesteps 5475.
Path 281 | total_timesteps 5492.
Path 282 | total_timesteps 5517.
Path 283 | total_timesteps 5533.
Path 284 | total_timesteps 5568.
Path 285 | total_timesteps 5597.
Path 286 | total_timesteps 5611.
Path 287 | total_timesteps 5628.
Path 288 | total_timesteps 5644.
Path 289 | total_timesteps 5672.
Path 290 | total_timesteps 5681.
Path 291 | total_timesteps 5696.
Path 292 | total_timesteps 5725.
Path 293 | total_timesteps 5750.
Path 294 | total_timesteps 5765.
Path 295 | total_timesteps 5797.
Path 296 | total_timesteps 5820.
Path 297 | total_timesteps 5839.
Path 298 | total_timesteps 5854.
Path 299 | total_timesteps 5862.
Path 300 | total_timesteps 5879.
Path 301 | total_timesteps 5897.
Path 302 | total_timesteps 5928.
Path 303 | total_timesteps 5941.
Path 304 | total_timesteps 5953.
Path 305 | total_timesteps 5967.
Path 306 | total_timesteps 5989.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.25    |
| Iteration     | 12       |
| MaximumReturn | 7.36     |
| MinimumReturn | -25.3    |
| TotalSamples  | 56127    |
----------------------------
itr #13 | 
Fitting dynamics.
Validation loss = 0.005942132323980331
Validation loss = 0.005201689898967743
Validation loss = 0.005308171268552542
Validation loss = 0.005564613733440638
Validation loss = 0.0049511645920574665
Validation loss = 0.0056654768995940685
Validation loss = 0.0057583218440413475
Validation loss = 0.004908387083560228
Validation loss = 0.005571373738348484
Validation loss = 0.005268130451440811
Validation loss = 0.004927370231598616
Validation loss = 0.005061195697635412
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 38.
Path 3 | total_timesteps 53.
Path 4 | total_timesteps 70.
Path 5 | total_timesteps 84.
Path 6 | total_timesteps 93.
Path 7 | total_timesteps 123.
Path 8 | total_timesteps 138.
Path 9 | total_timesteps 159.
Path 10 | total_timesteps 192.
Path 11 | total_timesteps 212.
Path 12 | total_timesteps 232.
Path 13 | total_timesteps 273.
Path 14 | total_timesteps 291.
Path 15 | total_timesteps 319.
Path 16 | total_timesteps 340.
Path 17 | total_timesteps 373.
Path 18 | total_timesteps 410.
Path 19 | total_timesteps 430.
Path 20 | total_timesteps 468.
Path 21 | total_timesteps 486.
Path 22 | total_timesteps 504.
Path 23 | total_timesteps 528.
Path 24 | total_timesteps 567.
Path 25 | total_timesteps 595.
Path 26 | total_timesteps 610.
Path 27 | total_timesteps 652.
Path 28 | total_timesteps 661.
Path 29 | total_timesteps 681.
Path 30 | total_timesteps 699.
Path 31 | total_timesteps 718.
Path 32 | total_timesteps 744.
Path 33 | total_timesteps 767.
Path 34 | total_timesteps 781.
Path 35 | total_timesteps 794.
Path 36 | total_timesteps 804.
Path 37 | total_timesteps 819.
Path 38 | total_timesteps 834.
Path 39 | total_timesteps 853.
Path 40 | total_timesteps 876.
Path 41 | total_timesteps 923.
Path 42 | total_timesteps 967.
Path 43 | total_timesteps 984.
Path 44 | total_timesteps 1000.
Path 45 | total_timesteps 1027.
Path 46 | total_timesteps 1047.
Path 47 | total_timesteps 1067.
Path 48 | total_timesteps 1094.
Path 49 | total_timesteps 1120.
Path 50 | total_timesteps 1144.
Path 51 | total_timesteps 1170.
Path 52 | total_timesteps 1187.
Path 53 | total_timesteps 1204.
Path 54 | total_timesteps 1224.
Path 55 | total_timesteps 1243.
Path 56 | total_timesteps 1262.
Path 57 | total_timesteps 1281.
Path 58 | total_timesteps 1312.
Path 59 | total_timesteps 1337.
Path 60 | total_timesteps 1373.
Path 61 | total_timesteps 1404.
Path 62 | total_timesteps 1429.
Path 63 | total_timesteps 1450.
Path 64 | total_timesteps 1477.
Path 65 | total_timesteps 1503.
Path 66 | total_timesteps 1528.
Path 67 | total_timesteps 1551.
Path 68 | total_timesteps 1571.
Path 69 | total_timesteps 1584.
Path 70 | total_timesteps 1605.
Path 71 | total_timesteps 1629.
Path 72 | total_timesteps 1645.
Path 73 | total_timesteps 1678.
Path 74 | total_timesteps 1704.
Path 75 | total_timesteps 1739.
Path 76 | total_timesteps 1758.
Path 77 | total_timesteps 1774.
Path 78 | total_timesteps 1798.
Path 79 | total_timesteps 1821.
Path 80 | total_timesteps 1846.
Path 81 | total_timesteps 1868.
Path 82 | total_timesteps 1882.
Path 83 | total_timesteps 1906.
Path 84 | total_timesteps 1926.
Path 85 | total_timesteps 1952.
Path 86 | total_timesteps 1970.
Path 87 | total_timesteps 2005.
Path 88 | total_timesteps 2018.
Path 89 | total_timesteps 2038.
Path 90 | total_timesteps 2057.
Path 91 | total_timesteps 2075.
Path 92 | total_timesteps 2095.
Path 93 | total_timesteps 2122.
Path 94 | total_timesteps 2151.
Path 95 | total_timesteps 2169.
Path 96 | total_timesteps 2185.
Path 97 | total_timesteps 2200.
Path 98 | total_timesteps 2220.
Path 99 | total_timesteps 2237.
Path 100 | total_timesteps 2276.
Path 101 | total_timesteps 2303.
Path 102 | total_timesteps 2324.
Path 103 | total_timesteps 2339.
Path 104 | total_timesteps 2373.
Path 105 | total_timesteps 2392.
Path 106 | total_timesteps 2413.
Path 107 | total_timesteps 2428.
Path 108 | total_timesteps 2443.
Path 109 | total_timesteps 2456.
Path 110 | total_timesteps 2476.
Path 111 | total_timesteps 2490.
Path 112 | total_timesteps 2507.
Path 113 | total_timesteps 2540.
Path 114 | total_timesteps 2560.
Path 115 | total_timesteps 2577.
Path 116 | total_timesteps 2594.
Path 117 | total_timesteps 2609.
Path 118 | total_timesteps 2624.
Path 119 | total_timesteps 2642.
Path 120 | total_timesteps 2659.
Path 121 | total_timesteps 2674.
Path 122 | total_timesteps 2690.
Path 123 | total_timesteps 2723.
Path 124 | total_timesteps 2745.
Path 125 | total_timesteps 2761.
Path 126 | total_timesteps 2773.
Path 127 | total_timesteps 2814.
Path 128 | total_timesteps 2833.
Path 129 | total_timesteps 2869.
Path 130 | total_timesteps 2882.
Path 131 | total_timesteps 2898.
Path 132 | total_timesteps 2942.
Path 133 | total_timesteps 2955.
Path 134 | total_timesteps 2969.
Path 135 | total_timesteps 2987.
Path 136 | total_timesteps 3012.
Path 137 | total_timesteps 3039.
Path 138 | total_timesteps 3052.
Path 139 | total_timesteps 3066.
Path 140 | total_timesteps 3129.
Path 141 | total_timesteps 3152.
Path 142 | total_timesteps 3166.
Path 143 | total_timesteps 3193.
Path 144 | total_timesteps 3210.
Path 145 | total_timesteps 3217.
Path 146 | total_timesteps 3235.
Path 147 | total_timesteps 3256.
Path 148 | total_timesteps 3275.
Path 149 | total_timesteps 3297.
Path 150 | total_timesteps 3319.
Path 151 | total_timesteps 3339.
Path 152 | total_timesteps 3359.
Path 153 | total_timesteps 3380.
Path 154 | total_timesteps 3403.
Path 155 | total_timesteps 3428.
Path 156 | total_timesteps 3453.
Path 157 | total_timesteps 3468.
Path 158 | total_timesteps 3479.
Path 159 | total_timesteps 3496.
Path 160 | total_timesteps 3533.
Path 161 | total_timesteps 3557.
Path 162 | total_timesteps 3571.
Path 163 | total_timesteps 3585.
Path 164 | total_timesteps 3604.
Path 165 | total_timesteps 3623.
Path 166 | total_timesteps 3638.
Path 167 | total_timesteps 3655.
Path 168 | total_timesteps 3667.
Path 169 | total_timesteps 3704.
Path 170 | total_timesteps 3733.
Path 171 | total_timesteps 3748.
Path 172 | total_timesteps 3770.
Path 173 | total_timesteps 3783.
Path 174 | total_timesteps 3811.
Path 175 | total_timesteps 3825.
Path 176 | total_timesteps 3846.
Path 177 | total_timesteps 3867.
Path 178 | total_timesteps 3899.
Path 179 | total_timesteps 3909.
Path 180 | total_timesteps 3928.
Path 181 | total_timesteps 3977.
Path 182 | total_timesteps 4005.
Path 183 | total_timesteps 4042.
Path 184 | total_timesteps 4076.
Path 185 | total_timesteps 4096.
Path 186 | total_timesteps 4119.
Path 187 | total_timesteps 4137.
Path 188 | total_timesteps 4157.
Path 189 | total_timesteps 4181.
Path 190 | total_timesteps 4195.
Path 191 | total_timesteps 4213.
Path 192 | total_timesteps 4234.
Path 193 | total_timesteps 4246.
Path 194 | total_timesteps 4260.
Path 195 | total_timesteps 4283.
Path 196 | total_timesteps 4310.
Path 197 | total_timesteps 4343.
Path 198 | total_timesteps 4359.
Path 199 | total_timesteps 4379.
Path 200 | total_timesteps 4394.
Path 201 | total_timesteps 4407.
Path 202 | total_timesteps 4442.
Path 203 | total_timesteps 4458.
Path 204 | total_timesteps 4480.
Path 205 | total_timesteps 4504.
Path 206 | total_timesteps 4523.
Path 207 | total_timesteps 4549.
Path 208 | total_timesteps 4563.
Path 209 | total_timesteps 4596.
Path 210 | total_timesteps 4623.
Path 211 | total_timesteps 4648.
Path 212 | total_timesteps 4670.
Path 213 | total_timesteps 4701.
Path 214 | total_timesteps 4718.
Path 215 | total_timesteps 4740.
Path 216 | total_timesteps 4758.
Path 217 | total_timesteps 4772.
Path 218 | total_timesteps 4806.
Path 219 | total_timesteps 4829.
Path 220 | total_timesteps 4842.
Path 221 | total_timesteps 4856.
Path 222 | total_timesteps 4876.
Path 223 | total_timesteps 4892.
Path 224 | total_timesteps 4914.
Path 225 | total_timesteps 4931.
Path 226 | total_timesteps 4946.
Path 227 | total_timesteps 4975.
Path 228 | total_timesteps 5002.
Path 229 | total_timesteps 5042.
Path 230 | total_timesteps 5063.
Path 231 | total_timesteps 5089.
Path 232 | total_timesteps 5147.
Path 233 | total_timesteps 5163.
Path 234 | total_timesteps 5183.
Path 235 | total_timesteps 5195.
Path 236 | total_timesteps 5219.
Path 237 | total_timesteps 5237.
Path 238 | total_timesteps 5268.
Path 239 | total_timesteps 5297.
Path 240 | total_timesteps 5312.
Path 241 | total_timesteps 5324.
Path 242 | total_timesteps 5345.
Path 243 | total_timesteps 5365.
Path 244 | total_timesteps 5387.
Path 245 | total_timesteps 5406.
Path 246 | total_timesteps 5430.
Path 247 | total_timesteps 5460.
Path 248 | total_timesteps 5474.
Path 249 | total_timesteps 5503.
Path 250 | total_timesteps 5532.
Path 251 | total_timesteps 5545.
Path 252 | total_timesteps 5561.
Path 253 | total_timesteps 5577.
Path 254 | total_timesteps 5597.
Path 255 | total_timesteps 5616.
Path 256 | total_timesteps 5636.
Path 257 | total_timesteps 5652.
Path 258 | total_timesteps 5671.
Path 259 | total_timesteps 5690.
Path 260 | total_timesteps 5708.
Path 261 | total_timesteps 5725.
Path 262 | total_timesteps 5751.
Path 263 | total_timesteps 5773.
Path 264 | total_timesteps 5799.
Path 265 | total_timesteps 5818.
Path 266 | total_timesteps 5833.
Path 267 | total_timesteps 5856.
Path 268 | total_timesteps 5880.
Path 269 | total_timesteps 5891.
Path 270 | total_timesteps 5905.
Path 271 | total_timesteps 5924.
Path 272 | total_timesteps 5945.
Path 273 | total_timesteps 5972.
Path 274 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -10.9    |
| Iteration     | 13       |
| MaximumReturn | 10.8     |
| MinimumReturn | -24.5    |
| TotalSamples  | 60135    |
----------------------------
itr #14 | 
Fitting dynamics.
Validation loss = 0.005716204643249512
Validation loss = 0.004991470370441675
Validation loss = 0.005330224521458149
Validation loss = 0.004765039309859276
Validation loss = 0.004696333315223455
Validation loss = 0.006648672744631767
Validation loss = 0.004583986476063728
Validation loss = 0.0048878733068704605
Validation loss = 0.004328439943492413
Validation loss = 0.0052185822278261185
Validation loss = 0.004896511323750019
Validation loss = 0.004296896513551474
Validation loss = 0.005027763079851866
Validation loss = 0.004649037029594183
Validation loss = 0.004824905656278133
Validation loss = 0.004699418321251869
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 21.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 71.
Path 4 | total_timesteps 118.
Path 5 | total_timesteps 140.
Path 6 | total_timesteps 159.
Path 7 | total_timesteps 171.
Path 8 | total_timesteps 185.
Path 9 | total_timesteps 198.
Path 10 | total_timesteps 211.
Path 11 | total_timesteps 243.
Path 12 | total_timesteps 253.
Path 13 | total_timesteps 274.
Path 14 | total_timesteps 301.
Path 15 | total_timesteps 327.
Path 16 | total_timesteps 339.
Path 17 | total_timesteps 359.
Path 18 | total_timesteps 386.
Path 19 | total_timesteps 417.
Path 20 | total_timesteps 430.
Path 21 | total_timesteps 456.
Path 22 | total_timesteps 475.
Path 23 | total_timesteps 511.
Path 24 | total_timesteps 522.
Path 25 | total_timesteps 538.
Path 26 | total_timesteps 566.
Path 27 | total_timesteps 594.
Path 28 | total_timesteps 614.
Path 29 | total_timesteps 627.
Path 30 | total_timesteps 650.
Path 31 | total_timesteps 671.
Path 32 | total_timesteps 690.
Path 33 | total_timesteps 714.
Path 34 | total_timesteps 730.
Path 35 | total_timesteps 757.
Path 36 | total_timesteps 777.
Path 37 | total_timesteps 802.
Path 38 | total_timesteps 839.
Path 39 | total_timesteps 861.
Path 40 | total_timesteps 880.
Path 41 | total_timesteps 895.
Path 42 | total_timesteps 919.
Path 43 | total_timesteps 939.
Path 44 | total_timesteps 958.
Path 45 | total_timesteps 1010.
Path 46 | total_timesteps 1029.
Path 47 | total_timesteps 1048.
Path 48 | total_timesteps 1064.
Path 49 | total_timesteps 1091.
Path 50 | total_timesteps 1105.
Path 51 | total_timesteps 1131.
Path 52 | total_timesteps 1150.
Path 53 | total_timesteps 1171.
Path 54 | total_timesteps 1195.
Path 55 | total_timesteps 1238.
Path 56 | total_timesteps 1257.
Path 57 | total_timesteps 1284.
Path 58 | total_timesteps 1342.
Path 59 | total_timesteps 1356.
Path 60 | total_timesteps 1375.
Path 61 | total_timesteps 1415.
Path 62 | total_timesteps 1439.
Path 63 | total_timesteps 1465.
Path 64 | total_timesteps 1475.
Path 65 | total_timesteps 1507.
Path 66 | total_timesteps 1532.
Path 67 | total_timesteps 1553.
Path 68 | total_timesteps 1584.
Path 69 | total_timesteps 1623.
Path 70 | total_timesteps 1646.
Path 71 | total_timesteps 1691.
Path 72 | total_timesteps 1710.
Path 73 | total_timesteps 1735.
Path 74 | total_timesteps 1756.
Path 75 | total_timesteps 1780.
Path 76 | total_timesteps 1791.
Path 77 | total_timesteps 1816.
Path 78 | total_timesteps 1838.
Path 79 | total_timesteps 1860.
Path 80 | total_timesteps 1877.
Path 81 | total_timesteps 1903.
Path 82 | total_timesteps 1925.
Path 83 | total_timesteps 1951.
Path 84 | total_timesteps 1973.
Path 85 | total_timesteps 1998.
Path 86 | total_timesteps 2021.
Path 87 | total_timesteps 2042.
Path 88 | total_timesteps 2073.
Path 89 | total_timesteps 2093.
Path 90 | total_timesteps 2110.
Path 91 | total_timesteps 2128.
Path 92 | total_timesteps 2158.
Path 93 | total_timesteps 2185.
Path 94 | total_timesteps 2206.
Path 95 | total_timesteps 2224.
Path 96 | total_timesteps 2266.
Path 97 | total_timesteps 2276.
Path 98 | total_timesteps 2301.
Path 99 | total_timesteps 2325.
Path 100 | total_timesteps 2341.
Path 101 | total_timesteps 2372.
Path 102 | total_timesteps 2409.
Path 103 | total_timesteps 2456.
Path 104 | total_timesteps 2476.
Path 105 | total_timesteps 2488.
Path 106 | total_timesteps 2511.
Path 107 | total_timesteps 2523.
Path 108 | total_timesteps 2542.
Path 109 | total_timesteps 2554.
Path 110 | total_timesteps 2571.
Path 111 | total_timesteps 2606.
Path 112 | total_timesteps 2623.
Path 113 | total_timesteps 2648.
Path 114 | total_timesteps 2668.
Path 115 | total_timesteps 2690.
Path 116 | total_timesteps 2703.
Path 117 | total_timesteps 2719.
Path 118 | total_timesteps 2740.
Path 119 | total_timesteps 2761.
Path 120 | total_timesteps 2778.
Path 121 | total_timesteps 2787.
Path 122 | total_timesteps 2810.
Path 123 | total_timesteps 2830.
Path 124 | total_timesteps 2853.
Path 125 | total_timesteps 2870.
Path 126 | total_timesteps 2902.
Path 127 | total_timesteps 2916.
Path 128 | total_timesteps 2935.
Path 129 | total_timesteps 2955.
Path 130 | total_timesteps 2970.
Path 131 | total_timesteps 2991.
Path 132 | total_timesteps 3003.
Path 133 | total_timesteps 3029.
Path 134 | total_timesteps 3070.
Path 135 | total_timesteps 3108.
Path 136 | total_timesteps 3150.
Path 137 | total_timesteps 3169.
Path 138 | total_timesteps 3198.
Path 139 | total_timesteps 3211.
Path 140 | total_timesteps 3241.
Path 141 | total_timesteps 3255.
Path 142 | total_timesteps 3273.
Path 143 | total_timesteps 3284.
Path 144 | total_timesteps 3301.
Path 145 | total_timesteps 3337.
Path 146 | total_timesteps 3365.
Path 147 | total_timesteps 3402.
Path 148 | total_timesteps 3423.
Path 149 | total_timesteps 3440.
Path 150 | total_timesteps 3464.
Path 151 | total_timesteps 3488.
Path 152 | total_timesteps 3526.
Path 153 | total_timesteps 3545.
Path 154 | total_timesteps 3569.
Path 155 | total_timesteps 3584.
Path 156 | total_timesteps 3601.
Path 157 | total_timesteps 3629.
Path 158 | total_timesteps 3672.
Path 159 | total_timesteps 3697.
Path 160 | total_timesteps 3741.
Path 161 | total_timesteps 3758.
Path 162 | total_timesteps 3782.
Path 163 | total_timesteps 3823.
Path 164 | total_timesteps 3855.
Path 165 | total_timesteps 3867.
Path 166 | total_timesteps 3890.
Path 167 | total_timesteps 3935.
Path 168 | total_timesteps 3957.
Path 169 | total_timesteps 3978.
Path 170 | total_timesteps 3996.
Path 171 | total_timesteps 4006.
Path 172 | total_timesteps 4016.
Path 173 | total_timesteps 4038.
Path 174 | total_timesteps 4069.
Path 175 | total_timesteps 4082.
Path 176 | total_timesteps 4101.
Path 177 | total_timesteps 4118.
Path 178 | total_timesteps 4133.
Path 179 | total_timesteps 4149.
Path 180 | total_timesteps 4167.
Path 181 | total_timesteps 4184.
Path 182 | total_timesteps 4211.
Path 183 | total_timesteps 4219.
Path 184 | total_timesteps 4230.
Path 185 | total_timesteps 4251.
Path 186 | total_timesteps 4275.
Path 187 | total_timesteps 4297.
Path 188 | total_timesteps 4311.
Path 189 | total_timesteps 4347.
Path 190 | total_timesteps 4380.
Path 191 | total_timesteps 4392.
Path 192 | total_timesteps 4407.
Path 193 | total_timesteps 4426.
Path 194 | total_timesteps 4449.
Path 195 | total_timesteps 4476.
Path 196 | total_timesteps 4504.
Path 197 | total_timesteps 4525.
Path 198 | total_timesteps 4547.
Path 199 | total_timesteps 4587.
Path 200 | total_timesteps 4607.
Path 201 | total_timesteps 4624.
Path 202 | total_timesteps 4658.
Path 203 | total_timesteps 4672.
Path 204 | total_timesteps 4687.
Path 205 | total_timesteps 4705.
Path 206 | total_timesteps 4727.
Path 207 | total_timesteps 4741.
Path 208 | total_timesteps 4761.
Path 209 | total_timesteps 4778.
Path 210 | total_timesteps 4805.
Path 211 | total_timesteps 4818.
Path 212 | total_timesteps 4858.
Path 213 | total_timesteps 4884.
Path 214 | total_timesteps 4912.
Path 215 | total_timesteps 4931.
Path 216 | total_timesteps 4944.
Path 217 | total_timesteps 4969.
Path 218 | total_timesteps 4987.
Path 219 | total_timesteps 5005.
Path 220 | total_timesteps 5021.
Path 221 | total_timesteps 5048.
Path 222 | total_timesteps 5070.
Path 223 | total_timesteps 5099.
Path 224 | total_timesteps 5117.
Path 225 | total_timesteps 5138.
Path 226 | total_timesteps 5147.
Path 227 | total_timesteps 5156.
Path 228 | total_timesteps 5174.
Path 229 | total_timesteps 5195.
Path 230 | total_timesteps 5214.
Path 231 | total_timesteps 5231.
Path 232 | total_timesteps 5239.
Path 233 | total_timesteps 5269.
Path 234 | total_timesteps 5284.
Path 235 | total_timesteps 5303.
Path 236 | total_timesteps 5323.
Path 237 | total_timesteps 5332.
Path 238 | total_timesteps 5355.
Path 239 | total_timesteps 5385.
Path 240 | total_timesteps 5409.
Path 241 | total_timesteps 5433.
Path 242 | total_timesteps 5459.
Path 243 | total_timesteps 5472.
Path 244 | total_timesteps 5490.
Path 245 | total_timesteps 5511.
Path 246 | total_timesteps 5537.
Path 247 | total_timesteps 5561.
Path 248 | total_timesteps 5588.
Path 249 | total_timesteps 5612.
Path 250 | total_timesteps 5628.
Path 251 | total_timesteps 5653.
Path 252 | total_timesteps 5670.
Path 253 | total_timesteps 5698.
Path 254 | total_timesteps 5715.
Path 255 | total_timesteps 5741.
Path 256 | total_timesteps 5773.
Path 257 | total_timesteps 5783.
Path 258 | total_timesteps 5856.
Path 259 | total_timesteps 5884.
Path 260 | total_timesteps 5920.
Path 261 | total_timesteps 5946.
Path 262 | total_timesteps 5968.
Path 263 | total_timesteps 5980.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.4    |
| Iteration     | 14       |
| MaximumReturn | 2.61     |
| MinimumReturn | -35.6    |
| TotalSamples  | 64142    |
----------------------------
itr #15 | 
Fitting dynamics.
Validation loss = 0.006222822237759829
Validation loss = 0.004331394098699093
Validation loss = 0.0046363938599824905
Validation loss = 0.00476814853027463
Validation loss = 0.0043169427663087845
Validation loss = 0.0044121332466602325
Validation loss = 0.0041597820818424225
Validation loss = 0.0043196785263717175
Validation loss = 0.004232224076986313
Validation loss = 0.004662375431507826
Validation loss = 0.004279386252164841
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 53.
Path 2 | total_timesteps 81.
Path 3 | total_timesteps 99.
Path 4 | total_timesteps 128.
Path 5 | total_timesteps 152.
Path 6 | total_timesteps 193.
Path 7 | total_timesteps 213.
Path 8 | total_timesteps 235.
Path 9 | total_timesteps 255.
Path 10 | total_timesteps 272.
Path 11 | total_timesteps 303.
Path 12 | total_timesteps 319.
Path 13 | total_timesteps 344.
Path 14 | total_timesteps 358.
Path 15 | total_timesteps 383.
Path 16 | total_timesteps 397.
Path 17 | total_timesteps 418.
Path 18 | total_timesteps 438.
Path 19 | total_timesteps 455.
Path 20 | total_timesteps 475.
Path 21 | total_timesteps 497.
Path 22 | total_timesteps 513.
Path 23 | total_timesteps 533.
Path 24 | total_timesteps 558.
Path 25 | total_timesteps 584.
Path 26 | total_timesteps 607.
Path 27 | total_timesteps 626.
Path 28 | total_timesteps 645.
Path 29 | total_timesteps 671.
Path 30 | total_timesteps 689.
Path 31 | total_timesteps 708.
Path 32 | total_timesteps 743.
Path 33 | total_timesteps 758.
Path 34 | total_timesteps 778.
Path 35 | total_timesteps 808.
Path 36 | total_timesteps 823.
Path 37 | total_timesteps 855.
Path 38 | total_timesteps 874.
Path 39 | total_timesteps 899.
Path 40 | total_timesteps 929.
Path 41 | total_timesteps 944.
Path 42 | total_timesteps 966.
Path 43 | total_timesteps 994.
Path 44 | total_timesteps 1029.
Path 45 | total_timesteps 1056.
Path 46 | total_timesteps 1084.
Path 47 | total_timesteps 1114.
Path 48 | total_timesteps 1165.
Path 49 | total_timesteps 1178.
Path 50 | total_timesteps 1191.
Path 51 | total_timesteps 1215.
Path 52 | total_timesteps 1226.
Path 53 | total_timesteps 1241.
Path 54 | total_timesteps 1267.
Path 55 | total_timesteps 1287.
Path 56 | total_timesteps 1301.
Path 57 | total_timesteps 1332.
Path 58 | total_timesteps 1347.
Path 59 | total_timesteps 1377.
Path 60 | total_timesteps 1394.
Path 61 | total_timesteps 1419.
Path 62 | total_timesteps 1449.
Path 63 | total_timesteps 1467.
Path 64 | total_timesteps 1480.
Path 65 | total_timesteps 1501.
Path 66 | total_timesteps 1524.
Path 67 | total_timesteps 1551.
Path 68 | total_timesteps 1572.
Path 69 | total_timesteps 1618.
Path 70 | total_timesteps 1641.
Path 71 | total_timesteps 1659.
Path 72 | total_timesteps 1679.
Path 73 | total_timesteps 1699.
Path 74 | total_timesteps 1725.
Path 75 | total_timesteps 1753.
Path 76 | total_timesteps 1776.
Path 77 | total_timesteps 1800.
Path 78 | total_timesteps 1819.
Path 79 | total_timesteps 1837.
Path 80 | total_timesteps 1855.
Path 81 | total_timesteps 1897.
Path 82 | total_timesteps 1920.
Path 83 | total_timesteps 1960.
Path 84 | total_timesteps 1972.
Path 85 | total_timesteps 1999.
Path 86 | total_timesteps 2025.
Path 87 | total_timesteps 2057.
Path 88 | total_timesteps 2068.
Path 89 | total_timesteps 2099.
Path 90 | total_timesteps 2122.
Path 91 | total_timesteps 2136.
Path 92 | total_timesteps 2156.
Path 93 | total_timesteps 2180.
Path 94 | total_timesteps 2201.
Path 95 | total_timesteps 2227.
Path 96 | total_timesteps 2254.
Path 97 | total_timesteps 2275.
Path 98 | total_timesteps 2305.
Path 99 | total_timesteps 2331.
Path 100 | total_timesteps 2350.
Path 101 | total_timesteps 2369.
Path 102 | total_timesteps 2392.
Path 103 | total_timesteps 2420.
Path 104 | total_timesteps 2436.
Path 105 | total_timesteps 2455.
Path 106 | total_timesteps 2484.
Path 107 | total_timesteps 2511.
Path 108 | total_timesteps 2521.
Path 109 | total_timesteps 2545.
Path 110 | total_timesteps 2567.
Path 111 | total_timesteps 2597.
Path 112 | total_timesteps 2621.
Path 113 | total_timesteps 2644.
Path 114 | total_timesteps 2670.
Path 115 | total_timesteps 2689.
Path 116 | total_timesteps 2720.
Path 117 | total_timesteps 2751.
Path 118 | total_timesteps 2771.
Path 119 | total_timesteps 2787.
Path 120 | total_timesteps 2800.
Path 121 | total_timesteps 2825.
Path 122 | total_timesteps 2844.
Path 123 | total_timesteps 2863.
Path 124 | total_timesteps 2882.
Path 125 | total_timesteps 2917.
Path 126 | total_timesteps 2932.
Path 127 | total_timesteps 2962.
Path 128 | total_timesteps 2985.
Path 129 | total_timesteps 3009.
Path 130 | total_timesteps 3035.
Path 131 | total_timesteps 3053.
Path 132 | total_timesteps 3078.
Path 133 | total_timesteps 3093.
Path 134 | total_timesteps 3128.
Path 135 | total_timesteps 3151.
Path 136 | total_timesteps 3172.
Path 137 | total_timesteps 3189.
Path 138 | total_timesteps 3221.
Path 139 | total_timesteps 3239.
Path 140 | total_timesteps 3260.
Path 141 | total_timesteps 3273.
Path 142 | total_timesteps 3296.
Path 143 | total_timesteps 3314.
Path 144 | total_timesteps 3356.
Path 145 | total_timesteps 3378.
Path 146 | total_timesteps 3393.
Path 147 | total_timesteps 3416.
Path 148 | total_timesteps 3434.
Path 149 | total_timesteps 3469.
Path 150 | total_timesteps 3491.
Path 151 | total_timesteps 3515.
Path 152 | total_timesteps 3525.
Path 153 | total_timesteps 3549.
Path 154 | total_timesteps 3568.
Path 155 | total_timesteps 3596.
Path 156 | total_timesteps 3608.
Path 157 | total_timesteps 3635.
Path 158 | total_timesteps 3674.
Path 159 | total_timesteps 3692.
Path 160 | total_timesteps 3711.
Path 161 | total_timesteps 3734.
Path 162 | total_timesteps 3756.
Path 163 | total_timesteps 3782.
Path 164 | total_timesteps 3817.
Path 165 | total_timesteps 3840.
Path 166 | total_timesteps 3852.
Path 167 | total_timesteps 3870.
Path 168 | total_timesteps 3897.
Path 169 | total_timesteps 3912.
Path 170 | total_timesteps 3935.
Path 171 | total_timesteps 3963.
Path 172 | total_timesteps 3982.
Path 173 | total_timesteps 4001.
Path 174 | total_timesteps 4016.
Path 175 | total_timesteps 4034.
Path 176 | total_timesteps 4047.
Path 177 | total_timesteps 4082.
Path 178 | total_timesteps 4114.
Path 179 | total_timesteps 4140.
Path 180 | total_timesteps 4157.
Path 181 | total_timesteps 4177.
Path 182 | total_timesteps 4191.
Path 183 | total_timesteps 4209.
Path 184 | total_timesteps 4228.
Path 185 | total_timesteps 4255.
Path 186 | total_timesteps 4269.
Path 187 | total_timesteps 4287.
Path 188 | total_timesteps 4298.
Path 189 | total_timesteps 4342.
Path 190 | total_timesteps 4369.
Path 191 | total_timesteps 4404.
Path 192 | total_timesteps 4422.
Path 193 | total_timesteps 4462.
Path 194 | total_timesteps 4481.
Path 195 | total_timesteps 4502.
Path 196 | total_timesteps 4521.
Path 197 | total_timesteps 4540.
Path 198 | total_timesteps 4556.
Path 199 | total_timesteps 4579.
Path 200 | total_timesteps 4591.
Path 201 | total_timesteps 4609.
Path 202 | total_timesteps 4630.
Path 203 | total_timesteps 4665.
Path 204 | total_timesteps 4681.
Path 205 | total_timesteps 4699.
Path 206 | total_timesteps 4716.
Path 207 | total_timesteps 4739.
Path 208 | total_timesteps 4758.
Path 209 | total_timesteps 4776.
Path 210 | total_timesteps 4792.
Path 211 | total_timesteps 4808.
Path 212 | total_timesteps 4827.
Path 213 | total_timesteps 4850.
Path 214 | total_timesteps 4876.
Path 215 | total_timesteps 4889.
Path 216 | total_timesteps 4909.
Path 217 | total_timesteps 4923.
Path 218 | total_timesteps 4944.
Path 219 | total_timesteps 4969.
Path 220 | total_timesteps 4987.
Path 221 | total_timesteps 4998.
Path 222 | total_timesteps 5016.
Path 223 | total_timesteps 5037.
Path 224 | total_timesteps 5056.
Path 225 | total_timesteps 5076.
Path 226 | total_timesteps 5093.
Path 227 | total_timesteps 5110.
Path 228 | total_timesteps 5120.
Path 229 | total_timesteps 5154.
Path 230 | total_timesteps 5168.
Path 231 | total_timesteps 5203.
Path 232 | total_timesteps 5254.
Path 233 | total_timesteps 5272.
Path 234 | total_timesteps 5297.
Path 235 | total_timesteps 5315.
Path 236 | total_timesteps 5324.
Path 237 | total_timesteps 5342.
Path 238 | total_timesteps 5368.
Path 239 | total_timesteps 5394.
Path 240 | total_timesteps 5424.
Path 241 | total_timesteps 5459.
Path 242 | total_timesteps 5477.
Path 243 | total_timesteps 5498.
Path 244 | total_timesteps 5529.
Path 245 | total_timesteps 5551.
Path 246 | total_timesteps 5571.
Path 247 | total_timesteps 5586.
Path 248 | total_timesteps 5604.
Path 249 | total_timesteps 5629.
Path 250 | total_timesteps 5642.
Path 251 | total_timesteps 5665.
Path 252 | total_timesteps 5684.
Path 253 | total_timesteps 5717.
Path 254 | total_timesteps 5733.
Path 255 | total_timesteps 5768.
Path 256 | total_timesteps 5800.
Path 257 | total_timesteps 5822.
Path 258 | total_timesteps 5839.
Path 259 | total_timesteps 5858.
Path 260 | total_timesteps 5880.
Path 261 | total_timesteps 5902.
Path 262 | total_timesteps 5923.
Path 263 | total_timesteps 5956.
Path 264 | total_timesteps 5967.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.2    |
| Iteration     | 15       |
| MaximumReturn | 9.31     |
| MinimumReturn | -22.6    |
| TotalSamples  | 68148    |
----------------------------
itr #16 | 
Fitting dynamics.
Validation loss = 0.00466889375820756
Validation loss = 0.00433807447552681
Validation loss = 0.004821231123059988
Validation loss = 0.004752850625663996
Validation loss = 0.00400520721450448
Validation loss = 0.004394508898258209
Validation loss = 0.004165917169302702
Validation loss = 0.00395415723323822
Validation loss = 0.0047187963500618935
Validation loss = 0.0039376127533614635
Validation loss = 0.0045608351938426495
Validation loss = 0.004170957487076521
Validation loss = 0.003759038867428899
Validation loss = 0.0038666408509016037
Validation loss = 0.004066385794430971
Validation loss = 0.003932995721697807
Validation loss = 0.0036723949015140533
Validation loss = 0.0037909250240772963
Validation loss = 0.004002462141215801
Validation loss = 0.0036210573744028807
Validation loss = 0.0038610894698649645
Validation loss = 0.0037952333223074675
Validation loss = 0.0036917091347277164
Validation loss = 0.003999502398073673
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 18.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 49.
Path 4 | total_timesteps 68.
Path 5 | total_timesteps 80.
Path 6 | total_timesteps 95.
Path 7 | total_timesteps 115.
Path 8 | total_timesteps 128.
Path 9 | total_timesteps 147.
Path 10 | total_timesteps 177.
Path 11 | total_timesteps 200.
Path 12 | total_timesteps 238.
Path 13 | total_timesteps 268.
Path 14 | total_timesteps 295.
Path 15 | total_timesteps 312.
Path 16 | total_timesteps 325.
Path 17 | total_timesteps 340.
Path 18 | total_timesteps 387.
Path 19 | total_timesteps 402.
Path 20 | total_timesteps 425.
Path 21 | total_timesteps 441.
Path 22 | total_timesteps 462.
Path 23 | total_timesteps 475.
Path 24 | total_timesteps 485.
Path 25 | total_timesteps 498.
Path 26 | total_timesteps 517.
Path 27 | total_timesteps 536.
Path 28 | total_timesteps 558.
Path 29 | total_timesteps 603.
Path 30 | total_timesteps 621.
Path 31 | total_timesteps 653.
Path 32 | total_timesteps 664.
Path 33 | total_timesteps 678.
Path 34 | total_timesteps 692.
Path 35 | total_timesteps 715.
Path 36 | total_timesteps 723.
Path 37 | total_timesteps 737.
Path 38 | total_timesteps 765.
Path 39 | total_timesteps 790.
Path 40 | total_timesteps 813.
Path 41 | total_timesteps 831.
Path 42 | total_timesteps 847.
Path 43 | total_timesteps 872.
Path 44 | total_timesteps 886.
Path 45 | total_timesteps 906.
Path 46 | total_timesteps 927.
Path 47 | total_timesteps 942.
Path 48 | total_timesteps 983.
Path 49 | total_timesteps 1004.
Path 50 | total_timesteps 1022.
Path 51 | total_timesteps 1040.
Path 52 | total_timesteps 1061.
Path 53 | total_timesteps 1094.
Path 54 | total_timesteps 1110.
Path 55 | total_timesteps 1128.
Path 56 | total_timesteps 1148.
Path 57 | total_timesteps 1169.
Path 58 | total_timesteps 1189.
Path 59 | total_timesteps 1208.
Path 60 | total_timesteps 1229.
Path 61 | total_timesteps 1252.
Path 62 | total_timesteps 1275.
Path 63 | total_timesteps 1292.
Path 64 | total_timesteps 1303.
Path 65 | total_timesteps 1318.
Path 66 | total_timesteps 1340.
Path 67 | total_timesteps 1367.
Path 68 | total_timesteps 1392.
Path 69 | total_timesteps 1415.
Path 70 | total_timesteps 1444.
Path 71 | total_timesteps 1471.
Path 72 | total_timesteps 1489.
Path 73 | total_timesteps 1509.
Path 74 | total_timesteps 1529.
Path 75 | total_timesteps 1538.
Path 76 | total_timesteps 1551.
Path 77 | total_timesteps 1571.
Path 78 | total_timesteps 1589.
Path 79 | total_timesteps 1610.
Path 80 | total_timesteps 1623.
Path 81 | total_timesteps 1639.
Path 82 | total_timesteps 1654.
Path 83 | total_timesteps 1677.
Path 84 | total_timesteps 1710.
Path 85 | total_timesteps 1732.
Path 86 | total_timesteps 1752.
Path 87 | total_timesteps 1762.
Path 88 | total_timesteps 1780.
Path 89 | total_timesteps 1797.
Path 90 | total_timesteps 1822.
Path 91 | total_timesteps 1850.
Path 92 | total_timesteps 1879.
Path 93 | total_timesteps 1901.
Path 94 | total_timesteps 1919.
Path 95 | total_timesteps 1933.
Path 96 | total_timesteps 1952.
Path 97 | total_timesteps 1988.
Path 98 | total_timesteps 2017.
Path 99 | total_timesteps 2034.
Path 100 | total_timesteps 2059.
Path 101 | total_timesteps 2070.
Path 102 | total_timesteps 2094.
Path 103 | total_timesteps 2109.
Path 104 | total_timesteps 2128.
Path 105 | total_timesteps 2152.
Path 106 | total_timesteps 2174.
Path 107 | total_timesteps 2191.
Path 108 | total_timesteps 2213.
Path 109 | total_timesteps 2232.
Path 110 | total_timesteps 2271.
Path 111 | total_timesteps 2298.
Path 112 | total_timesteps 2309.
Path 113 | total_timesteps 2326.
Path 114 | total_timesteps 2345.
Path 115 | total_timesteps 2358.
Path 116 | total_timesteps 2373.
Path 117 | total_timesteps 2397.
Path 118 | total_timesteps 2420.
Path 119 | total_timesteps 2442.
Path 120 | total_timesteps 2467.
Path 121 | total_timesteps 2498.
Path 122 | total_timesteps 2525.
Path 123 | total_timesteps 2548.
Path 124 | total_timesteps 2573.
Path 125 | total_timesteps 2593.
Path 126 | total_timesteps 2611.
Path 127 | total_timesteps 2625.
Path 128 | total_timesteps 2646.
Path 129 | total_timesteps 2664.
Path 130 | total_timesteps 2687.
Path 131 | total_timesteps 2709.
Path 132 | total_timesteps 2722.
Path 133 | total_timesteps 2737.
Path 134 | total_timesteps 2755.
Path 135 | total_timesteps 2779.
Path 136 | total_timesteps 2801.
Path 137 | total_timesteps 2826.
Path 138 | total_timesteps 2848.
Path 139 | total_timesteps 2857.
Path 140 | total_timesteps 2882.
Path 141 | total_timesteps 2905.
Path 142 | total_timesteps 2925.
Path 143 | total_timesteps 2949.
Path 144 | total_timesteps 2982.
Path 145 | total_timesteps 2993.
Path 146 | total_timesteps 3013.
Path 147 | total_timesteps 3027.
Path 148 | total_timesteps 3051.
Path 149 | total_timesteps 3061.
Path 150 | total_timesteps 3086.
Path 151 | total_timesteps 3104.
Path 152 | total_timesteps 3120.
Path 153 | total_timesteps 3143.
Path 154 | total_timesteps 3168.
Path 155 | total_timesteps 3189.
Path 156 | total_timesteps 3224.
Path 157 | total_timesteps 3260.
Path 158 | total_timesteps 3276.
Path 159 | total_timesteps 3295.
Path 160 | total_timesteps 3329.
Path 161 | total_timesteps 3352.
Path 162 | total_timesteps 3368.
Path 163 | total_timesteps 3379.
Path 164 | total_timesteps 3405.
Path 165 | total_timesteps 3432.
Path 166 | total_timesteps 3454.
Path 167 | total_timesteps 3473.
Path 168 | total_timesteps 3489.
Path 169 | total_timesteps 3506.
Path 170 | total_timesteps 3545.
Path 171 | total_timesteps 3566.
Path 172 | total_timesteps 3580.
Path 173 | total_timesteps 3604.
Path 174 | total_timesteps 3646.
Path 175 | total_timesteps 3664.
Path 176 | total_timesteps 3675.
Path 177 | total_timesteps 3693.
Path 178 | total_timesteps 3709.
Path 179 | total_timesteps 3725.
Path 180 | total_timesteps 3744.
Path 181 | total_timesteps 3763.
Path 182 | total_timesteps 3781.
Path 183 | total_timesteps 3804.
Path 184 | total_timesteps 3831.
Path 185 | total_timesteps 3843.
Path 186 | total_timesteps 3857.
Path 187 | total_timesteps 3872.
Path 188 | total_timesteps 3889.
Path 189 | total_timesteps 3918.
Path 190 | total_timesteps 3948.
Path 191 | total_timesteps 3968.
Path 192 | total_timesteps 3987.
Path 193 | total_timesteps 4018.
Path 194 | total_timesteps 4033.
Path 195 | total_timesteps 4049.
Path 196 | total_timesteps 4069.
Path 197 | total_timesteps 4098.
Path 198 | total_timesteps 4125.
Path 199 | total_timesteps 4150.
Path 200 | total_timesteps 4166.
Path 201 | total_timesteps 4186.
Path 202 | total_timesteps 4195.
Path 203 | total_timesteps 4203.
Path 204 | total_timesteps 4220.
Path 205 | total_timesteps 4231.
Path 206 | total_timesteps 4251.
Path 207 | total_timesteps 4288.
Path 208 | total_timesteps 4310.
Path 209 | total_timesteps 4329.
Path 210 | total_timesteps 4339.
Path 211 | total_timesteps 4348.
Path 212 | total_timesteps 4363.
Path 213 | total_timesteps 4387.
Path 214 | total_timesteps 4406.
Path 215 | total_timesteps 4430.
Path 216 | total_timesteps 4455.
Path 217 | total_timesteps 4464.
Path 218 | total_timesteps 4475.
Path 219 | total_timesteps 4499.
Path 220 | total_timesteps 4519.
Path 221 | total_timesteps 4530.
Path 222 | total_timesteps 4550.
Path 223 | total_timesteps 4562.
Path 224 | total_timesteps 4582.
Path 225 | total_timesteps 4599.
Path 226 | total_timesteps 4615.
Path 227 | total_timesteps 4652.
Path 228 | total_timesteps 4674.
Path 229 | total_timesteps 4684.
Path 230 | total_timesteps 4710.
Path 231 | total_timesteps 4745.
Path 232 | total_timesteps 4783.
Path 233 | total_timesteps 4807.
Path 234 | total_timesteps 4833.
Path 235 | total_timesteps 4866.
Path 236 | total_timesteps 4891.
Path 237 | total_timesteps 4912.
Path 238 | total_timesteps 4935.
Path 239 | total_timesteps 4965.
Path 240 | total_timesteps 4986.
Path 241 | total_timesteps 5004.
Path 242 | total_timesteps 5015.
Path 243 | total_timesteps 5024.
Path 244 | total_timesteps 5048.
Path 245 | total_timesteps 5063.
Path 246 | total_timesteps 5094.
Path 247 | total_timesteps 5106.
Path 248 | total_timesteps 5121.
Path 249 | total_timesteps 5129.
Path 250 | total_timesteps 5143.
Path 251 | total_timesteps 5167.
Path 252 | total_timesteps 5192.
Path 253 | total_timesteps 5206.
Path 254 | total_timesteps 5223.
Path 255 | total_timesteps 5241.
Path 256 | total_timesteps 5255.
Path 257 | total_timesteps 5276.
Path 258 | total_timesteps 5297.
Path 259 | total_timesteps 5322.
Path 260 | total_timesteps 5342.
Path 261 | total_timesteps 5366.
Path 262 | total_timesteps 5375.
Path 263 | total_timesteps 5398.
Path 264 | total_timesteps 5409.
Path 265 | total_timesteps 5423.
Path 266 | total_timesteps 5446.
Path 267 | total_timesteps 5477.
Path 268 | total_timesteps 5515.
Path 269 | total_timesteps 5564.
Path 270 | total_timesteps 5587.
Path 271 | total_timesteps 5610.
Path 272 | total_timesteps 5627.
Path 273 | total_timesteps 5644.
Path 274 | total_timesteps 5655.
Path 275 | total_timesteps 5681.
Path 276 | total_timesteps 5707.
Path 277 | total_timesteps 5722.
Path 278 | total_timesteps 5747.
Path 279 | total_timesteps 5762.
Path 280 | total_timesteps 5779.
Path 281 | total_timesteps 5791.
Path 282 | total_timesteps 5807.
Path 283 | total_timesteps 5824.
Path 284 | total_timesteps 5852.
Path 285 | total_timesteps 5872.
Path 286 | total_timesteps 5899.
Path 287 | total_timesteps 5920.
Path 288 | total_timesteps 5959.
Path 289 | total_timesteps 5980.
Path 290 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.4    |
| Iteration     | 16       |
| MaximumReturn | 3.96     |
| MinimumReturn | -22.6    |
| TotalSamples  | 72158    |
----------------------------
itr #17 | 
Fitting dynamics.
Validation loss = 0.003923047799617052
Validation loss = 0.0038327069487422705
Validation loss = 0.0035014981403946877
Validation loss = 0.0037390287034213543
Validation loss = 0.003811985719949007
Validation loss = 0.0034917050506919622
Validation loss = 0.0033261701464653015
Validation loss = 0.0038115556817501783
Validation loss = 0.0035277302376925945
Validation loss = 0.003533640643581748
Validation loss = 0.0036508904304355383
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 60.
Path 3 | total_timesteps 88.
Path 4 | total_timesteps 100.
Path 5 | total_timesteps 121.
Path 6 | total_timesteps 154.
Path 7 | total_timesteps 169.
Path 8 | total_timesteps 187.
Path 9 | total_timesteps 208.
Path 10 | total_timesteps 228.
Path 11 | total_timesteps 250.
Path 12 | total_timesteps 270.
Path 13 | total_timesteps 285.
Path 14 | total_timesteps 303.
Path 15 | total_timesteps 323.
Path 16 | total_timesteps 344.
Path 17 | total_timesteps 371.
Path 18 | total_timesteps 393.
Path 19 | total_timesteps 422.
Path 20 | total_timesteps 437.
Path 21 | total_timesteps 450.
Path 22 | total_timesteps 461.
Path 23 | total_timesteps 475.
Path 24 | total_timesteps 492.
Path 25 | total_timesteps 510.
Path 26 | total_timesteps 535.
Path 27 | total_timesteps 556.
Path 28 | total_timesteps 581.
Path 29 | total_timesteps 594.
Path 30 | total_timesteps 611.
Path 31 | total_timesteps 623.
Path 32 | total_timesteps 660.
Path 33 | total_timesteps 696.
Path 34 | total_timesteps 709.
Path 35 | total_timesteps 749.
Path 36 | total_timesteps 761.
Path 37 | total_timesteps 772.
Path 38 | total_timesteps 786.
Path 39 | total_timesteps 803.
Path 40 | total_timesteps 823.
Path 41 | total_timesteps 845.
Path 42 | total_timesteps 858.
Path 43 | total_timesteps 886.
Path 44 | total_timesteps 907.
Path 45 | total_timesteps 931.
Path 46 | total_timesteps 949.
Path 47 | total_timesteps 963.
Path 48 | total_timesteps 982.
Path 49 | total_timesteps 1010.
Path 50 | total_timesteps 1018.
Path 51 | total_timesteps 1036.
Path 52 | total_timesteps 1060.
Path 53 | total_timesteps 1090.
Path 54 | total_timesteps 1109.
Path 55 | total_timesteps 1128.
Path 56 | total_timesteps 1151.
Path 57 | total_timesteps 1165.
Path 58 | total_timesteps 1187.
Path 59 | total_timesteps 1199.
Path 60 | total_timesteps 1238.
Path 61 | total_timesteps 1254.
Path 62 | total_timesteps 1271.
Path 63 | total_timesteps 1296.
Path 64 | total_timesteps 1316.
Path 65 | total_timesteps 1325.
Path 66 | total_timesteps 1340.
Path 67 | total_timesteps 1354.
Path 68 | total_timesteps 1383.
Path 69 | total_timesteps 1394.
Path 70 | total_timesteps 1412.
Path 71 | total_timesteps 1435.
Path 72 | total_timesteps 1455.
Path 73 | total_timesteps 1468.
Path 74 | total_timesteps 1493.
Path 75 | total_timesteps 1512.
Path 76 | total_timesteps 1532.
Path 77 | total_timesteps 1546.
Path 78 | total_timesteps 1564.
Path 79 | total_timesteps 1583.
Path 80 | total_timesteps 1598.
Path 81 | total_timesteps 1613.
Path 82 | total_timesteps 1632.
Path 83 | total_timesteps 1648.
Path 84 | total_timesteps 1661.
Path 85 | total_timesteps 1680.
Path 86 | total_timesteps 1708.
Path 87 | total_timesteps 1730.
Path 88 | total_timesteps 1750.
Path 89 | total_timesteps 1778.
Path 90 | total_timesteps 1807.
Path 91 | total_timesteps 1816.
Path 92 | total_timesteps 1834.
Path 93 | total_timesteps 1850.
Path 94 | total_timesteps 1870.
Path 95 | total_timesteps 1897.
Path 96 | total_timesteps 1917.
Path 97 | total_timesteps 1935.
Path 98 | total_timesteps 1946.
Path 99 | total_timesteps 1970.
Path 100 | total_timesteps 2001.
Path 101 | total_timesteps 2013.
Path 102 | total_timesteps 2023.
Path 103 | total_timesteps 2045.
Path 104 | total_timesteps 2061.
Path 105 | total_timesteps 2073.
Path 106 | total_timesteps 2088.
Path 107 | total_timesteps 2106.
Path 108 | total_timesteps 2127.
Path 109 | total_timesteps 2144.
Path 110 | total_timesteps 2155.
Path 111 | total_timesteps 2173.
Path 112 | total_timesteps 2193.
Path 113 | total_timesteps 2220.
Path 114 | total_timesteps 2243.
Path 115 | total_timesteps 2261.
Path 116 | total_timesteps 2275.
Path 117 | total_timesteps 2291.
Path 118 | total_timesteps 2308.
Path 119 | total_timesteps 2324.
Path 120 | total_timesteps 2337.
Path 121 | total_timesteps 2363.
Path 122 | total_timesteps 2380.
Path 123 | total_timesteps 2403.
Path 124 | total_timesteps 2433.
Path 125 | total_timesteps 2443.
Path 126 | total_timesteps 2458.
Path 127 | total_timesteps 2471.
Path 128 | total_timesteps 2497.
Path 129 | total_timesteps 2534.
Path 130 | total_timesteps 2550.
Path 131 | total_timesteps 2569.
Path 132 | total_timesteps 2592.
Path 133 | total_timesteps 2602.
Path 134 | total_timesteps 2618.
Path 135 | total_timesteps 2628.
Path 136 | total_timesteps 2646.
Path 137 | total_timesteps 2677.
Path 138 | total_timesteps 2693.
Path 139 | total_timesteps 2715.
Path 140 | total_timesteps 2754.
Path 141 | total_timesteps 2780.
Path 142 | total_timesteps 2808.
Path 143 | total_timesteps 2826.
Path 144 | total_timesteps 2844.
Path 145 | total_timesteps 2855.
Path 146 | total_timesteps 2870.
Path 147 | total_timesteps 2890.
Path 148 | total_timesteps 2921.
Path 149 | total_timesteps 2948.
Path 150 | total_timesteps 2966.
Path 151 | total_timesteps 2983.
Path 152 | total_timesteps 3004.
Path 153 | total_timesteps 3028.
Path 154 | total_timesteps 3048.
Path 155 | total_timesteps 3061.
Path 156 | total_timesteps 3074.
Path 157 | total_timesteps 3094.
Path 158 | total_timesteps 3108.
Path 159 | total_timesteps 3125.
Path 160 | total_timesteps 3155.
Path 161 | total_timesteps 3171.
Path 162 | total_timesteps 3189.
Path 163 | total_timesteps 3211.
Path 164 | total_timesteps 3234.
Path 165 | total_timesteps 3248.
Path 166 | total_timesteps 3263.
Path 167 | total_timesteps 3271.
Path 168 | total_timesteps 3285.
Path 169 | total_timesteps 3298.
Path 170 | total_timesteps 3319.
Path 171 | total_timesteps 3342.
Path 172 | total_timesteps 3358.
Path 173 | total_timesteps 3385.
Path 174 | total_timesteps 3399.
Path 175 | total_timesteps 3415.
Path 176 | total_timesteps 3434.
Path 177 | total_timesteps 3454.
Path 178 | total_timesteps 3478.
Path 179 | total_timesteps 3495.
Path 180 | total_timesteps 3511.
Path 181 | total_timesteps 3530.
Path 182 | total_timesteps 3554.
Path 183 | total_timesteps 3569.
Path 184 | total_timesteps 3597.
Path 185 | total_timesteps 3612.
Path 186 | total_timesteps 3636.
Path 187 | total_timesteps 3653.
Path 188 | total_timesteps 3669.
Path 189 | total_timesteps 3683.
Path 190 | total_timesteps 3702.
Path 191 | total_timesteps 3710.
Path 192 | total_timesteps 3734.
Path 193 | total_timesteps 3755.
Path 194 | total_timesteps 3783.
Path 195 | total_timesteps 3797.
Path 196 | total_timesteps 3808.
Path 197 | total_timesteps 3823.
Path 198 | total_timesteps 3839.
Path 199 | total_timesteps 3849.
Path 200 | total_timesteps 3872.
Path 201 | total_timesteps 3892.
Path 202 | total_timesteps 3919.
Path 203 | total_timesteps 3939.
Path 204 | total_timesteps 3969.
Path 205 | total_timesteps 3991.
Path 206 | total_timesteps 4011.
Path 207 | total_timesteps 4043.
Path 208 | total_timesteps 4059.
Path 209 | total_timesteps 4084.
Path 210 | total_timesteps 4103.
Path 211 | total_timesteps 4120.
Path 212 | total_timesteps 4149.
Path 213 | total_timesteps 4173.
Path 214 | total_timesteps 4196.
Path 215 | total_timesteps 4222.
Path 216 | total_timesteps 4238.
Path 217 | total_timesteps 4261.
Path 218 | total_timesteps 4281.
Path 219 | total_timesteps 4289.
Path 220 | total_timesteps 4313.
Path 221 | total_timesteps 4351.
Path 222 | total_timesteps 4368.
Path 223 | total_timesteps 4383.
Path 224 | total_timesteps 4402.
Path 225 | total_timesteps 4438.
Path 226 | total_timesteps 4455.
Path 227 | total_timesteps 4474.
Path 228 | total_timesteps 4499.
Path 229 | total_timesteps 4511.
Path 230 | total_timesteps 4530.
Path 231 | total_timesteps 4558.
Path 232 | total_timesteps 4570.
Path 233 | total_timesteps 4597.
Path 234 | total_timesteps 4628.
Path 235 | total_timesteps 4651.
Path 236 | total_timesteps 4670.
Path 237 | total_timesteps 4693.
Path 238 | total_timesteps 4712.
Path 239 | total_timesteps 4731.
Path 240 | total_timesteps 4742.
Path 241 | total_timesteps 4755.
Path 242 | total_timesteps 4779.
Path 243 | total_timesteps 4798.
Path 244 | total_timesteps 4822.
Path 245 | total_timesteps 4840.
Path 246 | total_timesteps 4860.
Path 247 | total_timesteps 4880.
Path 248 | total_timesteps 4893.
Path 249 | total_timesteps 4911.
Path 250 | total_timesteps 4946.
Path 251 | total_timesteps 4967.
Path 252 | total_timesteps 4983.
Path 253 | total_timesteps 4994.
Path 254 | total_timesteps 5014.
Path 255 | total_timesteps 5029.
Path 256 | total_timesteps 5041.
Path 257 | total_timesteps 5064.
Path 258 | total_timesteps 5079.
Path 259 | total_timesteps 5086.
Path 260 | total_timesteps 5103.
Path 261 | total_timesteps 5123.
Path 262 | total_timesteps 5142.
Path 263 | total_timesteps 5161.
Path 264 | total_timesteps 5188.
Path 265 | total_timesteps 5204.
Path 266 | total_timesteps 5223.
Path 267 | total_timesteps 5252.
Path 268 | total_timesteps 5276.
Path 269 | total_timesteps 5294.
Path 270 | total_timesteps 5314.
Path 271 | total_timesteps 5327.
Path 272 | total_timesteps 5343.
Path 273 | total_timesteps 5360.
Path 274 | total_timesteps 5394.
Path 275 | total_timesteps 5414.
Path 276 | total_timesteps 5432.
Path 277 | total_timesteps 5454.
Path 278 | total_timesteps 5472.
Path 279 | total_timesteps 5494.
Path 280 | total_timesteps 5504.
Path 281 | total_timesteps 5529.
Path 282 | total_timesteps 5560.
Path 283 | total_timesteps 5578.
Path 284 | total_timesteps 5601.
Path 285 | total_timesteps 5643.
Path 286 | total_timesteps 5670.
Path 287 | total_timesteps 5697.
Path 288 | total_timesteps 5712.
Path 289 | total_timesteps 5758.
Path 290 | total_timesteps 5785.
Path 291 | total_timesteps 5803.
Path 292 | total_timesteps 5829.
Path 293 | total_timesteps 5859.
Path 294 | total_timesteps 5877.
Path 295 | total_timesteps 5896.
Path 296 | total_timesteps 5917.
Path 297 | total_timesteps 5944.
Path 298 | total_timesteps 5964.
Path 299 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.2    |
| Iteration     | 17       |
| MaximumReturn | 13.1     |
| MinimumReturn | -24.3    |
| TotalSamples  | 76164    |
----------------------------
itr #18 | 
Fitting dynamics.
Validation loss = 0.003964120056480169
Validation loss = 0.003471291856840253
Validation loss = 0.003461627522483468
Validation loss = 0.003821551101282239
Validation loss = 0.0032078418880701065
Validation loss = 0.003388078883290291
Validation loss = 0.003568999469280243
Validation loss = 0.003392465878278017
Validation loss = 0.0033683013170957565
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 17.
Path 2 | total_timesteps 35.
Path 3 | total_timesteps 62.
Path 4 | total_timesteps 86.
Path 5 | total_timesteps 105.
Path 6 | total_timesteps 132.
Path 7 | total_timesteps 140.
Path 8 | total_timesteps 162.
Path 9 | total_timesteps 185.
Path 10 | total_timesteps 200.
Path 11 | total_timesteps 217.
Path 12 | total_timesteps 235.
Path 13 | total_timesteps 246.
Path 14 | total_timesteps 268.
Path 15 | total_timesteps 284.
Path 16 | total_timesteps 302.
Path 17 | total_timesteps 316.
Path 18 | total_timesteps 335.
Path 19 | total_timesteps 359.
Path 20 | total_timesteps 396.
Path 21 | total_timesteps 414.
Path 22 | total_timesteps 431.
Path 23 | total_timesteps 445.
Path 24 | total_timesteps 458.
Path 25 | total_timesteps 474.
Path 26 | total_timesteps 488.
Path 27 | total_timesteps 506.
Path 28 | total_timesteps 526.
Path 29 | total_timesteps 551.
Path 30 | total_timesteps 567.
Path 31 | total_timesteps 585.
Path 32 | total_timesteps 606.
Path 33 | total_timesteps 634.
Path 34 | total_timesteps 657.
Path 35 | total_timesteps 669.
Path 36 | total_timesteps 683.
Path 37 | total_timesteps 693.
Path 38 | total_timesteps 708.
Path 39 | total_timesteps 724.
Path 40 | total_timesteps 751.
Path 41 | total_timesteps 767.
Path 42 | total_timesteps 784.
Path 43 | total_timesteps 811.
Path 44 | total_timesteps 837.
Path 45 | total_timesteps 873.
Path 46 | total_timesteps 901.
Path 47 | total_timesteps 923.
Path 48 | total_timesteps 936.
Path 49 | total_timesteps 953.
Path 50 | total_timesteps 963.
Path 51 | total_timesteps 978.
Path 52 | total_timesteps 1007.
Path 53 | total_timesteps 1019.
Path 54 | total_timesteps 1054.
Path 55 | total_timesteps 1078.
Path 56 | total_timesteps 1097.
Path 57 | total_timesteps 1119.
Path 58 | total_timesteps 1145.
Path 59 | total_timesteps 1154.
Path 60 | total_timesteps 1177.
Path 61 | total_timesteps 1194.
Path 62 | total_timesteps 1223.
Path 63 | total_timesteps 1241.
Path 64 | total_timesteps 1259.
Path 65 | total_timesteps 1270.
Path 66 | total_timesteps 1283.
Path 67 | total_timesteps 1304.
Path 68 | total_timesteps 1322.
Path 69 | total_timesteps 1348.
Path 70 | total_timesteps 1386.
Path 71 | total_timesteps 1411.
Path 72 | total_timesteps 1422.
Path 73 | total_timesteps 1443.
Path 74 | total_timesteps 1460.
Path 75 | total_timesteps 1483.
Path 76 | total_timesteps 1506.
Path 77 | total_timesteps 1520.
Path 78 | total_timesteps 1527.
Path 79 | total_timesteps 1546.
Path 80 | total_timesteps 1553.
Path 81 | total_timesteps 1570.
Path 82 | total_timesteps 1590.
Path 83 | total_timesteps 1603.
Path 84 | total_timesteps 1630.
Path 85 | total_timesteps 1649.
Path 86 | total_timesteps 1660.
Path 87 | total_timesteps 1681.
Path 88 | total_timesteps 1705.
Path 89 | total_timesteps 1726.
Path 90 | total_timesteps 1740.
Path 91 | total_timesteps 1762.
Path 92 | total_timesteps 1788.
Path 93 | total_timesteps 1805.
Path 94 | total_timesteps 1819.
Path 95 | total_timesteps 1838.
Path 96 | total_timesteps 1857.
Path 97 | total_timesteps 1866.
Path 98 | total_timesteps 1899.
Path 99 | total_timesteps 1919.
Path 100 | total_timesteps 1939.
Path 101 | total_timesteps 1960.
Path 102 | total_timesteps 1982.
Path 103 | total_timesteps 1994.
Path 104 | total_timesteps 2022.
Path 105 | total_timesteps 2041.
Path 106 | total_timesteps 2053.
Path 107 | total_timesteps 2075.
Path 108 | total_timesteps 2092.
Path 109 | total_timesteps 2111.
Path 110 | total_timesteps 2128.
Path 111 | total_timesteps 2140.
Path 112 | total_timesteps 2161.
Path 113 | total_timesteps 2182.
Path 114 | total_timesteps 2195.
Path 115 | total_timesteps 2212.
Path 116 | total_timesteps 2227.
Path 117 | total_timesteps 2256.
Path 118 | total_timesteps 2292.
Path 119 | total_timesteps 2313.
Path 120 | total_timesteps 2346.
Path 121 | total_timesteps 2373.
Path 122 | total_timesteps 2391.
Path 123 | total_timesteps 2400.
Path 124 | total_timesteps 2429.
Path 125 | total_timesteps 2454.
Path 126 | total_timesteps 2463.
Path 127 | total_timesteps 2476.
Path 128 | total_timesteps 2496.
Path 129 | total_timesteps 2530.
Path 130 | total_timesteps 2554.
Path 131 | total_timesteps 2568.
Path 132 | total_timesteps 2592.
Path 133 | total_timesteps 2622.
Path 134 | total_timesteps 2648.
Path 135 | total_timesteps 2668.
Path 136 | total_timesteps 2693.
Path 137 | total_timesteps 2734.
Path 138 | total_timesteps 2747.
Path 139 | total_timesteps 2772.
Path 140 | total_timesteps 2788.
Path 141 | total_timesteps 2801.
Path 142 | total_timesteps 2819.
Path 143 | total_timesteps 2836.
Path 144 | total_timesteps 2859.
Path 145 | total_timesteps 2873.
Path 146 | total_timesteps 2891.
Path 147 | total_timesteps 2904.
Path 148 | total_timesteps 2925.
Path 149 | total_timesteps 2938.
Path 150 | total_timesteps 2955.
Path 151 | total_timesteps 2980.
Path 152 | total_timesteps 2996.
Path 153 | total_timesteps 3009.
Path 154 | total_timesteps 3039.
Path 155 | total_timesteps 3052.
Path 156 | total_timesteps 3079.
Path 157 | total_timesteps 3109.
Path 158 | total_timesteps 3125.
Path 159 | total_timesteps 3146.
Path 160 | total_timesteps 3189.
Path 161 | total_timesteps 3207.
Path 162 | total_timesteps 3233.
Path 163 | total_timesteps 3254.
Path 164 | total_timesteps 3268.
Path 165 | total_timesteps 3280.
Path 166 | total_timesteps 3298.
Path 167 | total_timesteps 3316.
Path 168 | total_timesteps 3332.
Path 169 | total_timesteps 3351.
Path 170 | total_timesteps 3371.
Path 171 | total_timesteps 3380.
Path 172 | total_timesteps 3393.
Path 173 | total_timesteps 3403.
Path 174 | total_timesteps 3413.
Path 175 | total_timesteps 3425.
Path 176 | total_timesteps 3442.
Path 177 | total_timesteps 3457.
Path 178 | total_timesteps 3481.
Path 179 | total_timesteps 3511.
Path 180 | total_timesteps 3522.
Path 181 | total_timesteps 3549.
Path 182 | total_timesteps 3559.
Path 183 | total_timesteps 3574.
Path 184 | total_timesteps 3585.
Path 185 | total_timesteps 3613.
Path 186 | total_timesteps 3625.
Path 187 | total_timesteps 3648.
Path 188 | total_timesteps 3662.
Path 189 | total_timesteps 3682.
Path 190 | total_timesteps 3697.
Path 191 | total_timesteps 3723.
Path 192 | total_timesteps 3736.
Path 193 | total_timesteps 3759.
Path 194 | total_timesteps 3788.
Path 195 | total_timesteps 3804.
Path 196 | total_timesteps 3828.
Path 197 | total_timesteps 3850.
Path 198 | total_timesteps 3869.
Path 199 | total_timesteps 3883.
Path 200 | total_timesteps 3918.
Path 201 | total_timesteps 3946.
Path 202 | total_timesteps 3974.
Path 203 | total_timesteps 3990.
Path 204 | total_timesteps 3998.
Path 205 | total_timesteps 4029.
Path 206 | total_timesteps 4050.
Path 207 | total_timesteps 4072.
Path 208 | total_timesteps 4091.
Path 209 | total_timesteps 4109.
Path 210 | total_timesteps 4136.
Path 211 | total_timesteps 4150.
Path 212 | total_timesteps 4170.
Path 213 | total_timesteps 4187.
Path 214 | total_timesteps 4207.
Path 215 | total_timesteps 4228.
Path 216 | total_timesteps 4241.
Path 217 | total_timesteps 4268.
Path 218 | total_timesteps 4287.
Path 219 | total_timesteps 4306.
Path 220 | total_timesteps 4323.
Path 221 | total_timesteps 4348.
Path 222 | total_timesteps 4372.
Path 223 | total_timesteps 4384.
Path 224 | total_timesteps 4407.
Path 225 | total_timesteps 4444.
Path 226 | total_timesteps 4458.
Path 227 | total_timesteps 4467.
Path 228 | total_timesteps 4484.
Path 229 | total_timesteps 4505.
Path 230 | total_timesteps 4521.
Path 231 | total_timesteps 4535.
Path 232 | total_timesteps 4555.
Path 233 | total_timesteps 4574.
Path 234 | total_timesteps 4609.
Path 235 | total_timesteps 4624.
Path 236 | total_timesteps 4639.
Path 237 | total_timesteps 4661.
Path 238 | total_timesteps 4672.
Path 239 | total_timesteps 4685.
Path 240 | total_timesteps 4700.
Path 241 | total_timesteps 4713.
Path 242 | total_timesteps 4753.
Path 243 | total_timesteps 4762.
Path 244 | total_timesteps 4779.
Path 245 | total_timesteps 4795.
Path 246 | total_timesteps 4823.
Path 247 | total_timesteps 4839.
Path 248 | total_timesteps 4876.
Path 249 | total_timesteps 4894.
Path 250 | total_timesteps 4912.
Path 251 | total_timesteps 4937.
Path 252 | total_timesteps 4949.
Path 253 | total_timesteps 4965.
Path 254 | total_timesteps 4974.
Path 255 | total_timesteps 4990.
Path 256 | total_timesteps 5015.
Path 257 | total_timesteps 5034.
Path 258 | total_timesteps 5051.
Path 259 | total_timesteps 5071.
Path 260 | total_timesteps 5097.
Path 261 | total_timesteps 5121.
Path 262 | total_timesteps 5153.
Path 263 | total_timesteps 5169.
Path 264 | total_timesteps 5179.
Path 265 | total_timesteps 5202.
Path 266 | total_timesteps 5213.
Path 267 | total_timesteps 5231.
Path 268 | total_timesteps 5247.
Path 269 | total_timesteps 5272.
Path 270 | total_timesteps 5295.
Path 271 | total_timesteps 5321.
Path 272 | total_timesteps 5340.
Path 273 | total_timesteps 5351.
Path 274 | total_timesteps 5377.
Path 275 | total_timesteps 5392.
Path 276 | total_timesteps 5413.
Path 277 | total_timesteps 5437.
Path 278 | total_timesteps 5464.
Path 279 | total_timesteps 5490.
Path 280 | total_timesteps 5507.
Path 281 | total_timesteps 5517.
Path 282 | total_timesteps 5533.
Path 283 | total_timesteps 5548.
Path 284 | total_timesteps 5574.
Path 285 | total_timesteps 5592.
Path 286 | total_timesteps 5605.
Path 287 | total_timesteps 5630.
Path 288 | total_timesteps 5645.
Path 289 | total_timesteps 5663.
Path 290 | total_timesteps 5678.
Path 291 | total_timesteps 5705.
Path 292 | total_timesteps 5722.
Path 293 | total_timesteps 5738.
Path 294 | total_timesteps 5747.
Path 295 | total_timesteps 5767.
Path 296 | total_timesteps 5788.
Path 297 | total_timesteps 5805.
Path 298 | total_timesteps 5816.
Path 299 | total_timesteps 5835.
Path 300 | total_timesteps 5849.
Path 301 | total_timesteps 5883.
Path 302 | total_timesteps 5907.
Path 303 | total_timesteps 5924.
Path 304 | total_timesteps 5937.
Path 305 | total_timesteps 5957.
Path 306 | total_timesteps 5977.
Path 307 | total_timesteps 5997.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -11.1    |
| Iteration     | 18       |
| MaximumReturn | 6.67     |
| MinimumReturn | -24.2    |
| TotalSamples  | 80169    |
----------------------------
itr #19 | 
Fitting dynamics.
Validation loss = 0.003614076878875494
Validation loss = 0.0033674463629722595
Validation loss = 0.0031341940630227327
Validation loss = 0.003314099507406354
Validation loss = 0.0033739195205271244
Validation loss = 0.0033804611302912235
Validation loss = 0.0034394010435789824
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 33.
Path 4 | total_timesteps 54.
Path 5 | total_timesteps 71.
Path 6 | total_timesteps 97.
Path 7 | total_timesteps 118.
Path 8 | total_timesteps 138.
Path 9 | total_timesteps 158.
Path 10 | total_timesteps 173.
Path 11 | total_timesteps 182.
Path 12 | total_timesteps 205.
Path 13 | total_timesteps 215.
Path 14 | total_timesteps 232.
Path 15 | total_timesteps 242.
Path 16 | total_timesteps 259.
Path 17 | total_timesteps 285.
Path 18 | total_timesteps 296.
Path 19 | total_timesteps 327.
Path 20 | total_timesteps 351.
Path 21 | total_timesteps 367.
Path 22 | total_timesteps 381.
Path 23 | total_timesteps 407.
Path 24 | total_timesteps 421.
Path 25 | total_timesteps 448.
Path 26 | total_timesteps 464.
Path 27 | total_timesteps 482.
Path 28 | total_timesteps 501.
Path 29 | total_timesteps 517.
Path 30 | total_timesteps 543.
Path 31 | total_timesteps 556.
Path 32 | total_timesteps 567.
Path 33 | total_timesteps 585.
Path 34 | total_timesteps 599.
Path 35 | total_timesteps 612.
Path 36 | total_timesteps 632.
Path 37 | total_timesteps 648.
Path 38 | total_timesteps 671.
Path 39 | total_timesteps 693.
Path 40 | total_timesteps 705.
Path 41 | total_timesteps 725.
Path 42 | total_timesteps 750.
Path 43 | total_timesteps 765.
Path 44 | total_timesteps 775.
Path 45 | total_timesteps 789.
Path 46 | total_timesteps 809.
Path 47 | total_timesteps 832.
Path 48 | total_timesteps 860.
Path 49 | total_timesteps 890.
Path 50 | total_timesteps 908.
Path 51 | total_timesteps 930.
Path 52 | total_timesteps 950.
Path 53 | total_timesteps 960.
Path 54 | total_timesteps 981.
Path 55 | total_timesteps 1004.
Path 56 | total_timesteps 1013.
Path 57 | total_timesteps 1029.
Path 58 | total_timesteps 1040.
Path 59 | total_timesteps 1062.
Path 60 | total_timesteps 1076.
Path 61 | total_timesteps 1092.
Path 62 | total_timesteps 1100.
Path 63 | total_timesteps 1110.
Path 64 | total_timesteps 1138.
Path 65 | total_timesteps 1156.
Path 66 | total_timesteps 1166.
Path 67 | total_timesteps 1192.
Path 68 | total_timesteps 1207.
Path 69 | total_timesteps 1228.
Path 70 | total_timesteps 1259.
Path 71 | total_timesteps 1277.
Path 72 | total_timesteps 1290.
Path 73 | total_timesteps 1310.
Path 74 | total_timesteps 1326.
Path 75 | total_timesteps 1350.
Path 76 | total_timesteps 1374.
Path 77 | total_timesteps 1389.
Path 78 | total_timesteps 1407.
Path 79 | total_timesteps 1428.
Path 80 | total_timesteps 1453.
Path 81 | total_timesteps 1469.
Path 82 | total_timesteps 1497.
Path 83 | total_timesteps 1542.
Path 84 | total_timesteps 1556.
Path 85 | total_timesteps 1586.
Path 86 | total_timesteps 1599.
Path 87 | total_timesteps 1609.
Path 88 | total_timesteps 1633.
Path 89 | total_timesteps 1655.
Path 90 | total_timesteps 1669.
Path 91 | total_timesteps 1686.
Path 92 | total_timesteps 1699.
Path 93 | total_timesteps 1714.
Path 94 | total_timesteps 1740.
Path 95 | total_timesteps 1758.
Path 96 | total_timesteps 1772.
Path 97 | total_timesteps 1783.
Path 98 | total_timesteps 1806.
Path 99 | total_timesteps 1823.
Path 100 | total_timesteps 1841.
Path 101 | total_timesteps 1867.
Path 102 | total_timesteps 1878.
Path 103 | total_timesteps 1888.
Path 104 | total_timesteps 1921.
Path 105 | total_timesteps 1938.
Path 106 | total_timesteps 1955.
Path 107 | total_timesteps 1983.
Path 108 | total_timesteps 1996.
Path 109 | total_timesteps 2007.
Path 110 | total_timesteps 2019.
Path 111 | total_timesteps 2038.
Path 112 | total_timesteps 2064.
Path 113 | total_timesteps 2080.
Path 114 | total_timesteps 2102.
Path 115 | total_timesteps 2122.
Path 116 | total_timesteps 2140.
Path 117 | total_timesteps 2160.
Path 118 | total_timesteps 2179.
Path 119 | total_timesteps 2204.
Path 120 | total_timesteps 2214.
Path 121 | total_timesteps 2244.
Path 122 | total_timesteps 2255.
Path 123 | total_timesteps 2269.
Path 124 | total_timesteps 2277.
Path 125 | total_timesteps 2292.
Path 126 | total_timesteps 2307.
Path 127 | total_timesteps 2326.
Path 128 | total_timesteps 2337.
Path 129 | total_timesteps 2365.
Path 130 | total_timesteps 2388.
Path 131 | total_timesteps 2398.
Path 132 | total_timesteps 2412.
Path 133 | total_timesteps 2445.
Path 134 | total_timesteps 2455.
Path 135 | total_timesteps 2466.
Path 136 | total_timesteps 2489.
Path 137 | total_timesteps 2504.
Path 138 | total_timesteps 2518.
Path 139 | total_timesteps 2526.
Path 140 | total_timesteps 2553.
Path 141 | total_timesteps 2572.
Path 142 | total_timesteps 2605.
Path 143 | total_timesteps 2621.
Path 144 | total_timesteps 2643.
Path 145 | total_timesteps 2668.
Path 146 | total_timesteps 2691.
Path 147 | total_timesteps 2701.
Path 148 | total_timesteps 2721.
Path 149 | total_timesteps 2740.
Path 150 | total_timesteps 2763.
Path 151 | total_timesteps 2783.
Path 152 | total_timesteps 2799.
Path 153 | total_timesteps 2826.
Path 154 | total_timesteps 2845.
Path 155 | total_timesteps 2869.
Path 156 | total_timesteps 2881.
Path 157 | total_timesteps 2896.
Path 158 | total_timesteps 2927.
Path 159 | total_timesteps 2940.
Path 160 | total_timesteps 2954.
Path 161 | total_timesteps 2962.
Path 162 | total_timesteps 2976.
Path 163 | total_timesteps 2992.
Path 164 | total_timesteps 3009.
Path 165 | total_timesteps 3035.
Path 166 | total_timesteps 3046.
Path 167 | total_timesteps 3066.
Path 168 | total_timesteps 3093.
Path 169 | total_timesteps 3118.
Path 170 | total_timesteps 3140.
Path 171 | total_timesteps 3162.
Path 172 | total_timesteps 3189.
Path 173 | total_timesteps 3205.
Path 174 | total_timesteps 3236.
Path 175 | total_timesteps 3255.
Path 176 | total_timesteps 3279.
Path 177 | total_timesteps 3296.
Path 178 | total_timesteps 3311.
Path 179 | total_timesteps 3323.
Path 180 | total_timesteps 3335.
Path 181 | total_timesteps 3343.
Path 182 | total_timesteps 3358.
Path 183 | total_timesteps 3374.
Path 184 | total_timesteps 3399.
Path 185 | total_timesteps 3415.
Path 186 | total_timesteps 3441.
Path 187 | total_timesteps 3462.
Path 188 | total_timesteps 3475.
Path 189 | total_timesteps 3494.
Path 190 | total_timesteps 3504.
Path 191 | total_timesteps 3518.
Path 192 | total_timesteps 3530.
Path 193 | total_timesteps 3561.
Path 194 | total_timesteps 3577.
Path 195 | total_timesteps 3597.
Path 196 | total_timesteps 3614.
Path 197 | total_timesteps 3636.
Path 198 | total_timesteps 3662.
Path 199 | total_timesteps 3681.
Path 200 | total_timesteps 3699.
Path 201 | total_timesteps 3721.
Path 202 | total_timesteps 3733.
Path 203 | total_timesteps 3762.
Path 204 | total_timesteps 3771.
Path 205 | total_timesteps 3785.
Path 206 | total_timesteps 3807.
Path 207 | total_timesteps 3834.
Path 208 | total_timesteps 3860.
Path 209 | total_timesteps 3878.
Path 210 | total_timesteps 3909.
Path 211 | total_timesteps 3928.
Path 212 | total_timesteps 3937.
Path 213 | total_timesteps 3949.
Path 214 | total_timesteps 3964.
Path 215 | total_timesteps 3979.
Path 216 | total_timesteps 4002.
Path 217 | total_timesteps 4028.
Path 218 | total_timesteps 4044.
Path 219 | total_timesteps 4057.
Path 220 | total_timesteps 4077.
Path 221 | total_timesteps 4104.
Path 222 | total_timesteps 4132.
Path 223 | total_timesteps 4149.
Path 224 | total_timesteps 4175.
Path 225 | total_timesteps 4195.
Path 226 | total_timesteps 4207.
Path 227 | total_timesteps 4232.
Path 228 | total_timesteps 4262.
Path 229 | total_timesteps 4283.
Path 230 | total_timesteps 4306.
Path 231 | total_timesteps 4330.
Path 232 | total_timesteps 4348.
Path 233 | total_timesteps 4369.
Path 234 | total_timesteps 4384.
Path 235 | total_timesteps 4403.
Path 236 | total_timesteps 4421.
Path 237 | total_timesteps 4436.
Path 238 | total_timesteps 4454.
Path 239 | total_timesteps 4475.
Path 240 | total_timesteps 4506.
Path 241 | total_timesteps 4519.
Path 242 | total_timesteps 4548.
Path 243 | total_timesteps 4558.
Path 244 | total_timesteps 4579.
Path 245 | total_timesteps 4595.
Path 246 | total_timesteps 4615.
Path 247 | total_timesteps 4626.
Path 248 | total_timesteps 4658.
Path 249 | total_timesteps 4666.
Path 250 | total_timesteps 4680.
Path 251 | total_timesteps 4709.
Path 252 | total_timesteps 4727.
Path 253 | total_timesteps 4745.
Path 254 | total_timesteps 4776.
Path 255 | total_timesteps 4790.
Path 256 | total_timesteps 4799.
Path 257 | total_timesteps 4817.
Path 258 | total_timesteps 4834.
Path 259 | total_timesteps 4851.
Path 260 | total_timesteps 4864.
Path 261 | total_timesteps 4878.
Path 262 | total_timesteps 4897.
Path 263 | total_timesteps 4905.
Path 264 | total_timesteps 4923.
Path 265 | total_timesteps 4939.
Path 266 | total_timesteps 4955.
Path 267 | total_timesteps 4974.
Path 268 | total_timesteps 4986.
Path 269 | total_timesteps 4997.
Path 270 | total_timesteps 5014.
Path 271 | total_timesteps 5030.
Path 272 | total_timesteps 5063.
Path 273 | total_timesteps 5077.
Path 274 | total_timesteps 5095.
Path 275 | total_timesteps 5113.
Path 276 | total_timesteps 5131.
Path 277 | total_timesteps 5144.
Path 278 | total_timesteps 5161.
Path 279 | total_timesteps 5184.
Path 280 | total_timesteps 5196.
Path 281 | total_timesteps 5215.
Path 282 | total_timesteps 5227.
Path 283 | total_timesteps 5233.
Path 284 | total_timesteps 5261.
Path 285 | total_timesteps 5289.
Path 286 | total_timesteps 5322.
Path 287 | total_timesteps 5339.
Path 288 | total_timesteps 5352.
Path 289 | total_timesteps 5371.
Path 290 | total_timesteps 5390.
Path 291 | total_timesteps 5406.
Path 292 | total_timesteps 5419.
Path 293 | total_timesteps 5433.
Path 294 | total_timesteps 5450.
Path 295 | total_timesteps 5464.
Path 296 | total_timesteps 5477.
Path 297 | total_timesteps 5492.
Path 298 | total_timesteps 5502.
Path 299 | total_timesteps 5515.
Path 300 | total_timesteps 5530.
Path 301 | total_timesteps 5544.
Path 302 | total_timesteps 5570.
Path 303 | total_timesteps 5596.
Path 304 | total_timesteps 5615.
Path 305 | total_timesteps 5624.
Path 306 | total_timesteps 5636.
Path 307 | total_timesteps 5655.
Path 308 | total_timesteps 5672.
Path 309 | total_timesteps 5693.
Path 310 | total_timesteps 5707.
Path 311 | total_timesteps 5734.
Path 312 | total_timesteps 5748.
Path 313 | total_timesteps 5759.
Path 314 | total_timesteps 5777.
Path 315 | total_timesteps 5794.
Path 316 | total_timesteps 5811.
Path 317 | total_timesteps 5826.
Path 318 | total_timesteps 5838.
Path 319 | total_timesteps 5854.
Path 320 | total_timesteps 5869.
Path 321 | total_timesteps 5885.
Path 322 | total_timesteps 5902.
Path 323 | total_timesteps 5921.
Path 324 | total_timesteps 5951.
Path 325 | total_timesteps 5975.
Path 326 | total_timesteps 5986.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.98    |
| Iteration     | 19       |
| MaximumReturn | 4.13     |
| MinimumReturn | -24.3    |
| TotalSamples  | 84173    |
----------------------------
itr #20 | 
Fitting dynamics.
Validation loss = 0.003193838521838188
Validation loss = 0.0034412185195833445
Validation loss = 0.0034769917838275433
Validation loss = 0.0031043486669659615
Validation loss = 0.0033151370007544756
Validation loss = 0.00308802118524909
Validation loss = 0.0034255413338541985
Validation loss = 0.003304104320704937
Validation loss = 0.0029211919754743576
Validation loss = 0.0037359013222157955
Validation loss = 0.003105852287262678
Validation loss = 0.003247897606343031
Validation loss = 0.003094879910349846
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 31.
Path 3 | total_timesteps 51.
Path 4 | total_timesteps 72.
Path 5 | total_timesteps 89.
Path 6 | total_timesteps 108.
Path 7 | total_timesteps 126.
Path 8 | total_timesteps 144.
Path 9 | total_timesteps 179.
Path 10 | total_timesteps 192.
Path 11 | total_timesteps 202.
Path 12 | total_timesteps 209.
Path 13 | total_timesteps 230.
Path 14 | total_timesteps 239.
Path 15 | total_timesteps 256.
Path 16 | total_timesteps 272.
Path 17 | total_timesteps 289.
Path 18 | total_timesteps 303.
Path 19 | total_timesteps 319.
Path 20 | total_timesteps 335.
Path 21 | total_timesteps 357.
Path 22 | total_timesteps 379.
Path 23 | total_timesteps 388.
Path 24 | total_timesteps 401.
Path 25 | total_timesteps 418.
Path 26 | total_timesteps 429.
Path 27 | total_timesteps 442.
Path 28 | total_timesteps 462.
Path 29 | total_timesteps 491.
Path 30 | total_timesteps 512.
Path 31 | total_timesteps 521.
Path 32 | total_timesteps 540.
Path 33 | total_timesteps 555.
Path 34 | total_timesteps 577.
Path 35 | total_timesteps 588.
Path 36 | total_timesteps 607.
Path 37 | total_timesteps 620.
Path 38 | total_timesteps 630.
Path 39 | total_timesteps 659.
Path 40 | total_timesteps 672.
Path 41 | total_timesteps 693.
Path 42 | total_timesteps 710.
Path 43 | total_timesteps 723.
Path 44 | total_timesteps 736.
Path 45 | total_timesteps 765.
Path 46 | total_timesteps 782.
Path 47 | total_timesteps 802.
Path 48 | total_timesteps 835.
Path 49 | total_timesteps 849.
Path 50 | total_timesteps 858.
Path 51 | total_timesteps 886.
Path 52 | total_timesteps 901.
Path 53 | total_timesteps 912.
Path 54 | total_timesteps 928.
Path 55 | total_timesteps 943.
Path 56 | total_timesteps 958.
Path 57 | total_timesteps 974.
Path 58 | total_timesteps 990.
Path 59 | total_timesteps 1004.
Path 60 | total_timesteps 1023.
Path 61 | total_timesteps 1046.
Path 62 | total_timesteps 1077.
Path 63 | total_timesteps 1092.
Path 64 | total_timesteps 1112.
Path 65 | total_timesteps 1133.
Path 66 | total_timesteps 1151.
Path 67 | total_timesteps 1161.
Path 68 | total_timesteps 1176.
Path 69 | total_timesteps 1193.
Path 70 | total_timesteps 1208.
Path 71 | total_timesteps 1226.
Path 72 | total_timesteps 1256.
Path 73 | total_timesteps 1272.
Path 74 | total_timesteps 1289.
Path 75 | total_timesteps 1302.
Path 76 | total_timesteps 1340.
Path 77 | total_timesteps 1354.
Path 78 | total_timesteps 1371.
Path 79 | total_timesteps 1396.
Path 80 | total_timesteps 1409.
Path 81 | total_timesteps 1431.
Path 82 | total_timesteps 1440.
Path 83 | total_timesteps 1449.
Path 84 | total_timesteps 1462.
Path 85 | total_timesteps 1470.
Path 86 | total_timesteps 1480.
Path 87 | total_timesteps 1523.
Path 88 | total_timesteps 1533.
Path 89 | total_timesteps 1569.
Path 90 | total_timesteps 1586.
Path 91 | total_timesteps 1601.
Path 92 | total_timesteps 1612.
Path 93 | total_timesteps 1641.
Path 94 | total_timesteps 1656.
Path 95 | total_timesteps 1671.
Path 96 | total_timesteps 1696.
Path 97 | total_timesteps 1721.
Path 98 | total_timesteps 1732.
Path 99 | total_timesteps 1744.
Path 100 | total_timesteps 1764.
Path 101 | total_timesteps 1779.
Path 102 | total_timesteps 1797.
Path 103 | total_timesteps 1819.
Path 104 | total_timesteps 1841.
Path 105 | total_timesteps 1850.
Path 106 | total_timesteps 1870.
Path 107 | total_timesteps 1889.
Path 108 | total_timesteps 1903.
Path 109 | total_timesteps 1920.
Path 110 | total_timesteps 1936.
Path 111 | total_timesteps 1944.
Path 112 | total_timesteps 1954.
Path 113 | total_timesteps 1966.
Path 114 | total_timesteps 1979.
Path 115 | total_timesteps 1995.
Path 116 | total_timesteps 2027.
Path 117 | total_timesteps 2038.
Path 118 | total_timesteps 2057.
Path 119 | total_timesteps 2070.
Path 120 | total_timesteps 2091.
Path 121 | total_timesteps 2101.
Path 122 | total_timesteps 2111.
Path 123 | total_timesteps 2132.
Path 124 | total_timesteps 2145.
Path 125 | total_timesteps 2160.
Path 126 | total_timesteps 2177.
Path 127 | total_timesteps 2192.
Path 128 | total_timesteps 2202.
Path 129 | total_timesteps 2223.
Path 130 | total_timesteps 2235.
Path 131 | total_timesteps 2246.
Path 132 | total_timesteps 2264.
Path 133 | total_timesteps 2279.
Path 134 | total_timesteps 2301.
Path 135 | total_timesteps 2316.
Path 136 | total_timesteps 2336.
Path 137 | total_timesteps 2353.
Path 138 | total_timesteps 2372.
Path 139 | total_timesteps 2392.
Path 140 | total_timesteps 2411.
Path 141 | total_timesteps 2431.
Path 142 | total_timesteps 2449.
Path 143 | total_timesteps 2470.
Path 144 | total_timesteps 2485.
Path 145 | total_timesteps 2495.
Path 146 | total_timesteps 2510.
Path 147 | total_timesteps 2534.
Path 148 | total_timesteps 2553.
Path 149 | total_timesteps 2576.
Path 150 | total_timesteps 2588.
Path 151 | total_timesteps 2598.
Path 152 | total_timesteps 2614.
Path 153 | total_timesteps 2633.
Path 154 | total_timesteps 2647.
Path 155 | total_timesteps 2659.
Path 156 | total_timesteps 2668.
Path 157 | total_timesteps 2686.
Path 158 | total_timesteps 2696.
Path 159 | total_timesteps 2705.
Path 160 | total_timesteps 2735.
Path 161 | total_timesteps 2754.
Path 162 | total_timesteps 2790.
Path 163 | total_timesteps 2819.
Path 164 | total_timesteps 2832.
Path 165 | total_timesteps 2856.
Path 166 | total_timesteps 2876.
Path 167 | total_timesteps 2888.
Path 168 | total_timesteps 2905.
Path 169 | total_timesteps 2915.
Path 170 | total_timesteps 2928.
Path 171 | total_timesteps 2949.
Path 172 | total_timesteps 2968.
Path 173 | total_timesteps 2976.
Path 174 | total_timesteps 3006.
Path 175 | total_timesteps 3026.
Path 176 | total_timesteps 3046.
Path 177 | total_timesteps 3057.
Path 178 | total_timesteps 3076.
Path 179 | total_timesteps 3093.
Path 180 | total_timesteps 3107.
Path 181 | total_timesteps 3123.
Path 182 | total_timesteps 3135.
Path 183 | total_timesteps 3154.
Path 184 | total_timesteps 3165.
Path 185 | total_timesteps 3189.
Path 186 | total_timesteps 3205.
Path 187 | total_timesteps 3217.
Path 188 | total_timesteps 3240.
Path 189 | total_timesteps 3261.
Path 190 | total_timesteps 3278.
Path 191 | total_timesteps 3300.
Path 192 | total_timesteps 3323.
Path 193 | total_timesteps 3342.
Path 194 | total_timesteps 3356.
Path 195 | total_timesteps 3374.
Path 196 | total_timesteps 3397.
Path 197 | total_timesteps 3420.
Path 198 | total_timesteps 3432.
Path 199 | total_timesteps 3450.
Path 200 | total_timesteps 3465.
Path 201 | total_timesteps 3482.
Path 202 | total_timesteps 3502.
Path 203 | total_timesteps 3515.
Path 204 | total_timesteps 3529.
Path 205 | total_timesteps 3539.
Path 206 | total_timesteps 3550.
Path 207 | total_timesteps 3568.
Path 208 | total_timesteps 3586.
Path 209 | total_timesteps 3605.
Path 210 | total_timesteps 3622.
Path 211 | total_timesteps 3638.
Path 212 | total_timesteps 3655.
Path 213 | total_timesteps 3673.
Path 214 | total_timesteps 3695.
Path 215 | total_timesteps 3705.
Path 216 | total_timesteps 3733.
Path 217 | total_timesteps 3759.
Path 218 | total_timesteps 3779.
Path 219 | total_timesteps 3795.
Path 220 | total_timesteps 3816.
Path 221 | total_timesteps 3829.
Path 222 | total_timesteps 3856.
Path 223 | total_timesteps 3882.
Path 224 | total_timesteps 3901.
Path 225 | total_timesteps 3910.
Path 226 | total_timesteps 3929.
Path 227 | total_timesteps 3942.
Path 228 | total_timesteps 3956.
Path 229 | total_timesteps 3984.
Path 230 | total_timesteps 4005.
Path 231 | total_timesteps 4033.
Path 232 | total_timesteps 4051.
Path 233 | total_timesteps 4068.
Path 234 | total_timesteps 4088.
Path 235 | total_timesteps 4108.
Path 236 | total_timesteps 4118.
Path 237 | total_timesteps 4130.
Path 238 | total_timesteps 4149.
Path 239 | total_timesteps 4175.
Path 240 | total_timesteps 4189.
Path 241 | total_timesteps 4203.
Path 242 | total_timesteps 4228.
Path 243 | total_timesteps 4240.
Path 244 | total_timesteps 4255.
Path 245 | total_timesteps 4269.
Path 246 | total_timesteps 4280.
Path 247 | total_timesteps 4293.
Path 248 | total_timesteps 4319.
Path 249 | total_timesteps 4334.
Path 250 | total_timesteps 4348.
Path 251 | total_timesteps 4364.
Path 252 | total_timesteps 4383.
Path 253 | total_timesteps 4401.
Path 254 | total_timesteps 4423.
Path 255 | total_timesteps 4443.
Path 256 | total_timesteps 4456.
Path 257 | total_timesteps 4475.
Path 258 | total_timesteps 4506.
Path 259 | total_timesteps 4533.
Path 260 | total_timesteps 4559.
Path 261 | total_timesteps 4574.
Path 262 | total_timesteps 4596.
Path 263 | total_timesteps 4612.
Path 264 | total_timesteps 4621.
Path 265 | total_timesteps 4633.
Path 266 | total_timesteps 4651.
Path 267 | total_timesteps 4664.
Path 268 | total_timesteps 4673.
Path 269 | total_timesteps 4681.
Path 270 | total_timesteps 4699.
Path 271 | total_timesteps 4710.
Path 272 | total_timesteps 4722.
Path 273 | total_timesteps 4735.
Path 274 | total_timesteps 4749.
Path 275 | total_timesteps 4775.
Path 276 | total_timesteps 4786.
Path 277 | total_timesteps 4810.
Path 278 | total_timesteps 4832.
Path 279 | total_timesteps 4849.
Path 280 | total_timesteps 4860.
Path 281 | total_timesteps 4881.
Path 282 | total_timesteps 4903.
Path 283 | total_timesteps 4915.
Path 284 | total_timesteps 4927.
Path 285 | total_timesteps 4940.
Path 286 | total_timesteps 4960.
Path 287 | total_timesteps 4978.
Path 288 | total_timesteps 5000.
Path 289 | total_timesteps 5011.
Path 290 | total_timesteps 5040.
Path 291 | total_timesteps 5057.
Path 292 | total_timesteps 5072.
Path 293 | total_timesteps 5092.
Path 294 | total_timesteps 5110.
Path 295 | total_timesteps 5138.
Path 296 | total_timesteps 5149.
Path 297 | total_timesteps 5171.
Path 298 | total_timesteps 5200.
Path 299 | total_timesteps 5214.
Path 300 | total_timesteps 5241.
Path 301 | total_timesteps 5257.
Path 302 | total_timesteps 5267.
Path 303 | total_timesteps 5294.
Path 304 | total_timesteps 5309.
Path 305 | total_timesteps 5325.
Path 306 | total_timesteps 5337.
Path 307 | total_timesteps 5362.
Path 308 | total_timesteps 5382.
Path 309 | total_timesteps 5403.
Path 310 | total_timesteps 5415.
Path 311 | total_timesteps 5442.
Path 312 | total_timesteps 5460.
Path 313 | total_timesteps 5482.
Path 314 | total_timesteps 5496.
Path 315 | total_timesteps 5523.
Path 316 | total_timesteps 5548.
Path 317 | total_timesteps 5565.
Path 318 | total_timesteps 5592.
Path 319 | total_timesteps 5602.
Path 320 | total_timesteps 5617.
Path 321 | total_timesteps 5630.
Path 322 | total_timesteps 5644.
Path 323 | total_timesteps 5660.
Path 324 | total_timesteps 5674.
Path 325 | total_timesteps 5692.
Path 326 | total_timesteps 5707.
Path 327 | total_timesteps 5716.
Path 328 | total_timesteps 5742.
Path 329 | total_timesteps 5758.
Path 330 | total_timesteps 5773.
Path 331 | total_timesteps 5783.
Path 332 | total_timesteps 5804.
Path 333 | total_timesteps 5822.
Path 334 | total_timesteps 5835.
Path 335 | total_timesteps 5862.
Path 336 | total_timesteps 5890.
Path 337 | total_timesteps 5902.
Path 338 | total_timesteps 5926.
Path 339 | total_timesteps 5936.
Path 340 | total_timesteps 5946.
Path 341 | total_timesteps 5977.
Path 342 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.92    |
| Iteration     | 20       |
| MaximumReturn | 3.51     |
| MinimumReturn | -24.7    |
| TotalSamples  | 88177    |
----------------------------
itr #21 | 
Fitting dynamics.
Validation loss = 0.0030095239635556936
Validation loss = 0.0033593676052987576
Validation loss = 0.003135587787255645
Validation loss = 0.002912989817559719
Validation loss = 0.002882482251152396
Validation loss = 0.0029957338701933622
Validation loss = 0.0028949992265552282
Validation loss = 0.0028245090506970882
Validation loss = 0.003113830229267478
Validation loss = 0.0027800307143479586
Validation loss = 0.0032515674829483032
Validation loss = 0.0026895261835306883
Validation loss = 0.0028328325133770704
Validation loss = 0.0030923672020435333
Validation loss = 0.002867087023332715
Validation loss = 0.0027546780183911324
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 20.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 48.
Path 5 | total_timesteps 80.
Path 6 | total_timesteps 95.
Path 7 | total_timesteps 105.
Path 8 | total_timesteps 125.
Path 9 | total_timesteps 137.
Path 10 | total_timesteps 149.
Path 11 | total_timesteps 171.
Path 12 | total_timesteps 192.
Path 13 | total_timesteps 206.
Path 14 | total_timesteps 220.
Path 15 | total_timesteps 240.
Path 16 | total_timesteps 251.
Path 17 | total_timesteps 272.
Path 18 | total_timesteps 288.
Path 19 | total_timesteps 299.
Path 20 | total_timesteps 312.
Path 21 | total_timesteps 337.
Path 22 | total_timesteps 365.
Path 23 | total_timesteps 384.
Path 24 | total_timesteps 397.
Path 25 | total_timesteps 407.
Path 26 | total_timesteps 428.
Path 27 | total_timesteps 455.
Path 28 | total_timesteps 478.
Path 29 | total_timesteps 486.
Path 30 | total_timesteps 495.
Path 31 | total_timesteps 502.
Path 32 | total_timesteps 515.
Path 33 | total_timesteps 526.
Path 34 | total_timesteps 558.
Path 35 | total_timesteps 580.
Path 36 | total_timesteps 598.
Path 37 | total_timesteps 615.
Path 38 | total_timesteps 625.
Path 39 | total_timesteps 641.
Path 40 | total_timesteps 657.
Path 41 | total_timesteps 679.
Path 42 | total_timesteps 690.
Path 43 | total_timesteps 699.
Path 44 | total_timesteps 715.
Path 45 | total_timesteps 735.
Path 46 | total_timesteps 744.
Path 47 | total_timesteps 762.
Path 48 | total_timesteps 786.
Path 49 | total_timesteps 801.
Path 50 | total_timesteps 819.
Path 51 | total_timesteps 836.
Path 52 | total_timesteps 856.
Path 53 | total_timesteps 872.
Path 54 | total_timesteps 894.
Path 55 | total_timesteps 908.
Path 56 | total_timesteps 929.
Path 57 | total_timesteps 942.
Path 58 | total_timesteps 954.
Path 59 | total_timesteps 969.
Path 60 | total_timesteps 1007.
Path 61 | total_timesteps 1029.
Path 62 | total_timesteps 1046.
Path 63 | total_timesteps 1067.
Path 64 | total_timesteps 1083.
Path 65 | total_timesteps 1101.
Path 66 | total_timesteps 1109.
Path 67 | total_timesteps 1124.
Path 68 | total_timesteps 1130.
Path 69 | total_timesteps 1155.
Path 70 | total_timesteps 1166.
Path 71 | total_timesteps 1188.
Path 72 | total_timesteps 1209.
Path 73 | total_timesteps 1216.
Path 74 | total_timesteps 1238.
Path 75 | total_timesteps 1249.
Path 76 | total_timesteps 1258.
Path 77 | total_timesteps 1271.
Path 78 | total_timesteps 1279.
Path 79 | total_timesteps 1288.
Path 80 | total_timesteps 1299.
Path 81 | total_timesteps 1326.
Path 82 | total_timesteps 1349.
Path 83 | total_timesteps 1357.
Path 84 | total_timesteps 1367.
Path 85 | total_timesteps 1383.
Path 86 | total_timesteps 1406.
Path 87 | total_timesteps 1424.
Path 88 | total_timesteps 1435.
Path 89 | total_timesteps 1454.
Path 90 | total_timesteps 1480.
Path 91 | total_timesteps 1509.
Path 92 | total_timesteps 1522.
Path 93 | total_timesteps 1544.
Path 94 | total_timesteps 1559.
Path 95 | total_timesteps 1571.
Path 96 | total_timesteps 1588.
Path 97 | total_timesteps 1604.
Path 98 | total_timesteps 1612.
Path 99 | total_timesteps 1634.
Path 100 | total_timesteps 1647.
Path 101 | total_timesteps 1668.
Path 102 | total_timesteps 1689.
Path 103 | total_timesteps 1706.
Path 104 | total_timesteps 1715.
Path 105 | total_timesteps 1722.
Path 106 | total_timesteps 1747.
Path 107 | total_timesteps 1762.
Path 108 | total_timesteps 1784.
Path 109 | total_timesteps 1810.
Path 110 | total_timesteps 1834.
Path 111 | total_timesteps 1869.
Path 112 | total_timesteps 1896.
Path 113 | total_timesteps 1904.
Path 114 | total_timesteps 1926.
Path 115 | total_timesteps 1953.
Path 116 | total_timesteps 1973.
Path 117 | total_timesteps 1986.
Path 118 | total_timesteps 2019.
Path 119 | total_timesteps 2030.
Path 120 | total_timesteps 2041.
Path 121 | total_timesteps 2051.
Path 122 | total_timesteps 2072.
Path 123 | total_timesteps 2083.
Path 124 | total_timesteps 2102.
Path 125 | total_timesteps 2123.
Path 126 | total_timesteps 2145.
Path 127 | total_timesteps 2165.
Path 128 | total_timesteps 2190.
Path 129 | total_timesteps 2200.
Path 130 | total_timesteps 2222.
Path 131 | total_timesteps 2235.
Path 132 | total_timesteps 2245.
Path 133 | total_timesteps 2261.
Path 134 | total_timesteps 2285.
Path 135 | total_timesteps 2305.
Path 136 | total_timesteps 2320.
Path 137 | total_timesteps 2341.
Path 138 | total_timesteps 2353.
Path 139 | total_timesteps 2364.
Path 140 | total_timesteps 2385.
Path 141 | total_timesteps 2396.
Path 142 | total_timesteps 2404.
Path 143 | total_timesteps 2410.
Path 144 | total_timesteps 2419.
Path 145 | total_timesteps 2445.
Path 146 | total_timesteps 2453.
Path 147 | total_timesteps 2464.
Path 148 | total_timesteps 2478.
Path 149 | total_timesteps 2489.
Path 150 | total_timesteps 2501.
Path 151 | total_timesteps 2523.
Path 152 | total_timesteps 2543.
Path 153 | total_timesteps 2563.
Path 154 | total_timesteps 2588.
Path 155 | total_timesteps 2613.
Path 156 | total_timesteps 2626.
Path 157 | total_timesteps 2650.
Path 158 | total_timesteps 2661.
Path 159 | total_timesteps 2683.
Path 160 | total_timesteps 2691.
Path 161 | total_timesteps 2706.
Path 162 | total_timesteps 2721.
Path 163 | total_timesteps 2740.
Path 164 | total_timesteps 2752.
Path 165 | total_timesteps 2761.
Path 166 | total_timesteps 2785.
Path 167 | total_timesteps 2793.
Path 168 | total_timesteps 2806.
Path 169 | total_timesteps 2825.
Path 170 | total_timesteps 2839.
Path 171 | total_timesteps 2856.
Path 172 | total_timesteps 2874.
Path 173 | total_timesteps 2888.
Path 174 | total_timesteps 2905.
Path 175 | total_timesteps 2917.
Path 176 | total_timesteps 2928.
Path 177 | total_timesteps 2948.
Path 178 | total_timesteps 2964.
Path 179 | total_timesteps 2998.
Path 180 | total_timesteps 3020.
Path 181 | total_timesteps 3033.
Path 182 | total_timesteps 3048.
Path 183 | total_timesteps 3060.
Path 184 | total_timesteps 3071.
Path 185 | total_timesteps 3093.
Path 186 | total_timesteps 3116.
Path 187 | total_timesteps 3148.
Path 188 | total_timesteps 3157.
Path 189 | total_timesteps 3173.
Path 190 | total_timesteps 3195.
Path 191 | total_timesteps 3219.
Path 192 | total_timesteps 3229.
Path 193 | total_timesteps 3242.
Path 194 | total_timesteps 3259.
Path 195 | total_timesteps 3271.
Path 196 | total_timesteps 3297.
Path 197 | total_timesteps 3321.
Path 198 | total_timesteps 3347.
Path 199 | total_timesteps 3363.
Path 200 | total_timesteps 3372.
Path 201 | total_timesteps 3396.
Path 202 | total_timesteps 3421.
Path 203 | total_timesteps 3432.
Path 204 | total_timesteps 3442.
Path 205 | total_timesteps 3466.
Path 206 | total_timesteps 3490.
Path 207 | total_timesteps 3511.
Path 208 | total_timesteps 3532.
Path 209 | total_timesteps 3554.
Path 210 | total_timesteps 3562.
Path 211 | total_timesteps 3580.
Path 212 | total_timesteps 3590.
Path 213 | total_timesteps 3608.
Path 214 | total_timesteps 3619.
Path 215 | total_timesteps 3631.
Path 216 | total_timesteps 3656.
Path 217 | total_timesteps 3690.
Path 218 | total_timesteps 3707.
Path 219 | total_timesteps 3726.
Path 220 | total_timesteps 3741.
Path 221 | total_timesteps 3756.
Path 222 | total_timesteps 3775.
Path 223 | total_timesteps 3796.
Path 224 | total_timesteps 3830.
Path 225 | total_timesteps 3857.
Path 226 | total_timesteps 3877.
Path 227 | total_timesteps 3894.
Path 228 | total_timesteps 3912.
Path 229 | total_timesteps 3931.
Path 230 | total_timesteps 3943.
Path 231 | total_timesteps 3960.
Path 232 | total_timesteps 3992.
Path 233 | total_timesteps 4001.
Path 234 | total_timesteps 4014.
Path 235 | total_timesteps 4027.
Path 236 | total_timesteps 4046.
Path 237 | total_timesteps 4066.
Path 238 | total_timesteps 4090.
Path 239 | total_timesteps 4116.
Path 240 | total_timesteps 4129.
Path 241 | total_timesteps 4141.
Path 242 | total_timesteps 4165.
Path 243 | total_timesteps 4188.
Path 244 | total_timesteps 4201.
Path 245 | total_timesteps 4221.
Path 246 | total_timesteps 4232.
Path 247 | total_timesteps 4243.
Path 248 | total_timesteps 4255.
Path 249 | total_timesteps 4264.
Path 250 | total_timesteps 4276.
Path 251 | total_timesteps 4293.
Path 252 | total_timesteps 4307.
Path 253 | total_timesteps 4332.
Path 254 | total_timesteps 4361.
Path 255 | total_timesteps 4376.
Path 256 | total_timesteps 4398.
Path 257 | total_timesteps 4415.
Path 258 | total_timesteps 4426.
Path 259 | total_timesteps 4455.
Path 260 | total_timesteps 4475.
Path 261 | total_timesteps 4484.
Path 262 | total_timesteps 4496.
Path 263 | total_timesteps 4522.
Path 264 | total_timesteps 4538.
Path 265 | total_timesteps 4557.
Path 266 | total_timesteps 4574.
Path 267 | total_timesteps 4583.
Path 268 | total_timesteps 4592.
Path 269 | total_timesteps 4608.
Path 270 | total_timesteps 4624.
Path 271 | total_timesteps 4643.
Path 272 | total_timesteps 4661.
Path 273 | total_timesteps 4692.
Path 274 | total_timesteps 4700.
Path 275 | total_timesteps 4720.
Path 276 | total_timesteps 4729.
Path 277 | total_timesteps 4750.
Path 278 | total_timesteps 4768.
Path 279 | total_timesteps 4786.
Path 280 | total_timesteps 4800.
Path 281 | total_timesteps 4809.
Path 282 | total_timesteps 4823.
Path 283 | total_timesteps 4833.
Path 284 | total_timesteps 4843.
Path 285 | total_timesteps 4861.
Path 286 | total_timesteps 4886.
Path 287 | total_timesteps 4900.
Path 288 | total_timesteps 4916.
Path 289 | total_timesteps 4935.
Path 290 | total_timesteps 4962.
Path 291 | total_timesteps 4974.
Path 292 | total_timesteps 4990.
Path 293 | total_timesteps 5012.
Path 294 | total_timesteps 5031.
Path 295 | total_timesteps 5045.
Path 296 | total_timesteps 5065.
Path 297 | total_timesteps 5079.
Path 298 | total_timesteps 5090.
Path 299 | total_timesteps 5105.
Path 300 | total_timesteps 5119.
Path 301 | total_timesteps 5133.
Path 302 | total_timesteps 5156.
Path 303 | total_timesteps 5168.
Path 304 | total_timesteps 5178.
Path 305 | total_timesteps 5189.
Path 306 | total_timesteps 5205.
Path 307 | total_timesteps 5224.
Path 308 | total_timesteps 5235.
Path 309 | total_timesteps 5260.
Path 310 | total_timesteps 5280.
Path 311 | total_timesteps 5306.
Path 312 | total_timesteps 5323.
Path 313 | total_timesteps 5331.
Path 314 | total_timesteps 5343.
Path 315 | total_timesteps 5352.
Path 316 | total_timesteps 5364.
Path 317 | total_timesteps 5380.
Path 318 | total_timesteps 5398.
Path 319 | total_timesteps 5411.
Path 320 | total_timesteps 5431.
Path 321 | total_timesteps 5444.
Path 322 | total_timesteps 5456.
Path 323 | total_timesteps 5471.
Path 324 | total_timesteps 5492.
Path 325 | total_timesteps 5504.
Path 326 | total_timesteps 5528.
Path 327 | total_timesteps 5544.
Path 328 | total_timesteps 5566.
Path 329 | total_timesteps 5578.
Path 330 | total_timesteps 5604.
Path 331 | total_timesteps 5619.
Path 332 | total_timesteps 5641.
Path 333 | total_timesteps 5657.
Path 334 | total_timesteps 5673.
Path 335 | total_timesteps 5695.
Path 336 | total_timesteps 5712.
Path 337 | total_timesteps 5721.
Path 338 | total_timesteps 5732.
Path 339 | total_timesteps 5755.
Path 340 | total_timesteps 5771.
Path 341 | total_timesteps 5784.
Path 342 | total_timesteps 5796.
Path 343 | total_timesteps 5810.
Path 344 | total_timesteps 5827.
Path 345 | total_timesteps 5842.
Path 346 | total_timesteps 5857.
Path 347 | total_timesteps 5882.
Path 348 | total_timesteps 5893.
Path 349 | total_timesteps 5907.
Path 350 | total_timesteps 5916.
Path 351 | total_timesteps 5930.
Path 352 | total_timesteps 5945.
Path 353 | total_timesteps 5967.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -9.12    |
| Iteration     | 21       |
| MaximumReturn | 5.93     |
| MinimumReturn | -23.7    |
| TotalSamples  | 92177    |
----------------------------
itr #22 | 
Fitting dynamics.
Validation loss = 0.0027105771005153656
Validation loss = 0.0032328609377145767
Validation loss = 0.0026181538123637438
Validation loss = 0.0026362210046499968
Validation loss = 0.002681794110685587
Validation loss = 0.0028769862838089466
Validation loss = 0.0026994969230145216
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 16.
Path 2 | total_timesteps 48.
Path 3 | total_timesteps 59.
Path 4 | total_timesteps 71.
Path 5 | total_timesteps 83.
Path 6 | total_timesteps 102.
Path 7 | total_timesteps 112.
Path 8 | total_timesteps 125.
Path 9 | total_timesteps 145.
Path 10 | total_timesteps 153.
Path 11 | total_timesteps 167.
Path 12 | total_timesteps 181.
Path 13 | total_timesteps 200.
Path 14 | total_timesteps 215.
Path 15 | total_timesteps 229.
Path 16 | total_timesteps 248.
Path 17 | total_timesteps 264.
Path 18 | total_timesteps 290.
Path 19 | total_timesteps 302.
Path 20 | total_timesteps 310.
Path 21 | total_timesteps 323.
Path 22 | total_timesteps 339.
Path 23 | total_timesteps 347.
Path 24 | total_timesteps 355.
Path 25 | total_timesteps 374.
Path 26 | total_timesteps 384.
Path 27 | total_timesteps 396.
Path 28 | total_timesteps 410.
Path 29 | total_timesteps 417.
Path 30 | total_timesteps 435.
Path 31 | total_timesteps 445.
Path 32 | total_timesteps 462.
Path 33 | total_timesteps 477.
Path 34 | total_timesteps 492.
Path 35 | total_timesteps 500.
Path 36 | total_timesteps 511.
Path 37 | total_timesteps 522.
Path 38 | total_timesteps 538.
Path 39 | total_timesteps 556.
Path 40 | total_timesteps 563.
Path 41 | total_timesteps 570.
Path 42 | total_timesteps 578.
Path 43 | total_timesteps 589.
Path 44 | total_timesteps 598.
Path 45 | total_timesteps 608.
Path 46 | total_timesteps 617.
Path 47 | total_timesteps 627.
Path 48 | total_timesteps 644.
Path 49 | total_timesteps 655.
Path 50 | total_timesteps 665.
Path 51 | total_timesteps 683.
Path 52 | total_timesteps 692.
Path 53 | total_timesteps 707.
Path 54 | total_timesteps 717.
Path 55 | total_timesteps 725.
Path 56 | total_timesteps 733.
Path 57 | total_timesteps 749.
Path 58 | total_timesteps 775.
Path 59 | total_timesteps 788.
Path 60 | total_timesteps 803.
Path 61 | total_timesteps 813.
Path 62 | total_timesteps 835.
Path 63 | total_timesteps 847.
Path 64 | total_timesteps 864.
Path 65 | total_timesteps 887.
Path 66 | total_timesteps 898.
Path 67 | total_timesteps 905.
Path 68 | total_timesteps 914.
Path 69 | total_timesteps 922.
Path 70 | total_timesteps 930.
Path 71 | total_timesteps 938.
Path 72 | total_timesteps 952.
Path 73 | total_timesteps 976.
Path 74 | total_timesteps 995.
Path 75 | total_timesteps 1016.
Path 76 | total_timesteps 1030.
Path 77 | total_timesteps 1043.
Path 78 | total_timesteps 1052.
Path 79 | total_timesteps 1072.
Path 80 | total_timesteps 1085.
Path 81 | total_timesteps 1110.
Path 82 | total_timesteps 1130.
Path 83 | total_timesteps 1138.
Path 84 | total_timesteps 1159.
Path 85 | total_timesteps 1167.
Path 86 | total_timesteps 1179.
Path 87 | total_timesteps 1191.
Path 88 | total_timesteps 1205.
Path 89 | total_timesteps 1218.
Path 90 | total_timesteps 1228.
Path 91 | total_timesteps 1242.
Path 92 | total_timesteps 1250.
Path 93 | total_timesteps 1263.
Path 94 | total_timesteps 1274.
Path 95 | total_timesteps 1283.
Path 96 | total_timesteps 1300.
Path 97 | total_timesteps 1312.
Path 98 | total_timesteps 1328.
Path 99 | total_timesteps 1344.
Path 100 | total_timesteps 1362.
Path 101 | total_timesteps 1371.
Path 102 | total_timesteps 1386.
Path 103 | total_timesteps 1402.
Path 104 | total_timesteps 1428.
Path 105 | total_timesteps 1435.
Path 106 | total_timesteps 1449.
Path 107 | total_timesteps 1465.
Path 108 | total_timesteps 1473.
Path 109 | total_timesteps 1499.
Path 110 | total_timesteps 1507.
Path 111 | total_timesteps 1519.
Path 112 | total_timesteps 1540.
Path 113 | total_timesteps 1553.
Path 114 | total_timesteps 1564.
Path 115 | total_timesteps 1573.
Path 116 | total_timesteps 1600.
Path 117 | total_timesteps 1620.
Path 118 | total_timesteps 1628.
Path 119 | total_timesteps 1637.
Path 120 | total_timesteps 1645.
Path 121 | total_timesteps 1662.
Path 122 | total_timesteps 1672.
Path 123 | total_timesteps 1682.
Path 124 | total_timesteps 1697.
Path 125 | total_timesteps 1707.
Path 126 | total_timesteps 1720.
Path 127 | total_timesteps 1739.
Path 128 | total_timesteps 1746.
Path 129 | total_timesteps 1762.
Path 130 | total_timesteps 1781.
Path 131 | total_timesteps 1803.
Path 132 | total_timesteps 1812.
Path 133 | total_timesteps 1832.
Path 134 | total_timesteps 1843.
Path 135 | total_timesteps 1857.
Path 136 | total_timesteps 1865.
Path 137 | total_timesteps 1891.
Path 138 | total_timesteps 1902.
Path 139 | total_timesteps 1916.
Path 140 | total_timesteps 1926.
Path 141 | total_timesteps 1935.
Path 142 | total_timesteps 1942.
Path 143 | total_timesteps 1951.
Path 144 | total_timesteps 1959.
Path 145 | total_timesteps 1968.
Path 146 | total_timesteps 1979.
Path 147 | total_timesteps 1998.
Path 148 | total_timesteps 2015.
Path 149 | total_timesteps 2023.
Path 150 | total_timesteps 2030.
Path 151 | total_timesteps 2043.
Path 152 | total_timesteps 2059.
Path 153 | total_timesteps 2075.
Path 154 | total_timesteps 2088.
Path 155 | total_timesteps 2103.
Path 156 | total_timesteps 2112.
Path 157 | total_timesteps 2135.
Path 158 | total_timesteps 2144.
Path 159 | total_timesteps 2155.
Path 160 | total_timesteps 2165.
Path 161 | total_timesteps 2180.
Path 162 | total_timesteps 2193.
Path 163 | total_timesteps 2199.
Path 164 | total_timesteps 2207.
Path 165 | total_timesteps 2219.
Path 166 | total_timesteps 2236.
Path 167 | total_timesteps 2245.
Path 168 | total_timesteps 2256.
Path 169 | total_timesteps 2267.
Path 170 | total_timesteps 2288.
Path 171 | total_timesteps 2301.
Path 172 | total_timesteps 2313.
Path 173 | total_timesteps 2320.
Path 174 | total_timesteps 2327.
Path 175 | total_timesteps 2337.
Path 176 | total_timesteps 2350.
Path 177 | total_timesteps 2369.
Path 178 | total_timesteps 2384.
Path 179 | total_timesteps 2405.
Path 180 | total_timesteps 2414.
Path 181 | total_timesteps 2425.
Path 182 | total_timesteps 2435.
Path 183 | total_timesteps 2446.
Path 184 | total_timesteps 2460.
Path 185 | total_timesteps 2481.
Path 186 | total_timesteps 2491.
Path 187 | total_timesteps 2500.
Path 188 | total_timesteps 2509.
Path 189 | total_timesteps 2516.
Path 190 | total_timesteps 2533.
Path 191 | total_timesteps 2545.
Path 192 | total_timesteps 2554.
Path 193 | total_timesteps 2564.
Path 194 | total_timesteps 2574.
Path 195 | total_timesteps 2581.
Path 196 | total_timesteps 2589.
Path 197 | total_timesteps 2614.
Path 198 | total_timesteps 2624.
Path 199 | total_timesteps 2633.
Path 200 | total_timesteps 2654.
Path 201 | total_timesteps 2665.
Path 202 | total_timesteps 2680.
Path 203 | total_timesteps 2691.
Path 204 | total_timesteps 2701.
Path 205 | total_timesteps 2726.
Path 206 | total_timesteps 2748.
Path 207 | total_timesteps 2756.
Path 208 | total_timesteps 2770.
Path 209 | total_timesteps 2790.
Path 210 | total_timesteps 2801.
Path 211 | total_timesteps 2812.
Path 212 | total_timesteps 2819.
Path 213 | total_timesteps 2839.
Path 214 | total_timesteps 2857.
Path 215 | total_timesteps 2867.
Path 216 | total_timesteps 2879.
Path 217 | total_timesteps 2893.
Path 218 | total_timesteps 2907.
Path 219 | total_timesteps 2940.
Path 220 | total_timesteps 2957.
Path 221 | total_timesteps 2967.
Path 222 | total_timesteps 2987.
Path 223 | total_timesteps 2998.
Path 224 | total_timesteps 3007.
Path 225 | total_timesteps 3017.
Path 226 | total_timesteps 3029.
Path 227 | total_timesteps 3040.
Path 228 | total_timesteps 3080.
Path 229 | total_timesteps 3090.
Path 230 | total_timesteps 3104.
Path 231 | total_timesteps 3116.
Path 232 | total_timesteps 3129.
Path 233 | total_timesteps 3149.
Path 234 | total_timesteps 3157.
Path 235 | total_timesteps 3175.
Path 236 | total_timesteps 3193.
Path 237 | total_timesteps 3203.
Path 238 | total_timesteps 3216.
Path 239 | total_timesteps 3238.
Path 240 | total_timesteps 3250.
Path 241 | total_timesteps 3260.
Path 242 | total_timesteps 3268.
Path 243 | total_timesteps 3280.
Path 244 | total_timesteps 3298.
Path 245 | total_timesteps 3305.
Path 246 | total_timesteps 3326.
Path 247 | total_timesteps 3334.
Path 248 | total_timesteps 3346.
Path 249 | total_timesteps 3355.
Path 250 | total_timesteps 3365.
Path 251 | total_timesteps 3374.
Path 252 | total_timesteps 3382.
Path 253 | total_timesteps 3392.
Path 254 | total_timesteps 3410.
Path 255 | total_timesteps 3425.
Path 256 | total_timesteps 3448.
Path 257 | total_timesteps 3458.
Path 258 | total_timesteps 3467.
Path 259 | total_timesteps 3481.
Path 260 | total_timesteps 3490.
Path 261 | total_timesteps 3515.
Path 262 | total_timesteps 3525.
Path 263 | total_timesteps 3548.
Path 264 | total_timesteps 3561.
Path 265 | total_timesteps 3582.
Path 266 | total_timesteps 3600.
Path 267 | total_timesteps 3615.
Path 268 | total_timesteps 3635.
Path 269 | total_timesteps 3643.
Path 270 | total_timesteps 3657.
Path 271 | total_timesteps 3666.
Path 272 | total_timesteps 3678.
Path 273 | total_timesteps 3696.
Path 274 | total_timesteps 3709.
Path 275 | total_timesteps 3719.
Path 276 | total_timesteps 3731.
Path 277 | total_timesteps 3748.
Path 278 | total_timesteps 3759.
Path 279 | total_timesteps 3772.
Path 280 | total_timesteps 3785.
Path 281 | total_timesteps 3797.
Path 282 | total_timesteps 3807.
Path 283 | total_timesteps 3816.
Path 284 | total_timesteps 3827.
Path 285 | total_timesteps 3839.
Path 286 | total_timesteps 3855.
Path 287 | total_timesteps 3863.
Path 288 | total_timesteps 3870.
Path 289 | total_timesteps 3878.
Path 290 | total_timesteps 3887.
Path 291 | total_timesteps 3899.
Path 292 | total_timesteps 3909.
Path 293 | total_timesteps 3919.
Path 294 | total_timesteps 3934.
Path 295 | total_timesteps 3944.
Path 296 | total_timesteps 3972.
Path 297 | total_timesteps 3986.
Path 298 | total_timesteps 3994.
Path 299 | total_timesteps 4017.
Path 300 | total_timesteps 4025.
Path 301 | total_timesteps 4038.
Path 302 | total_timesteps 4047.
Path 303 | total_timesteps 4059.
Path 304 | total_timesteps 4074.
Path 305 | total_timesteps 4089.
Path 306 | total_timesteps 4101.
Path 307 | total_timesteps 4110.
Path 308 | total_timesteps 4125.
Path 309 | total_timesteps 4140.
Path 310 | total_timesteps 4150.
Path 311 | total_timesteps 4171.
Path 312 | total_timesteps 4181.
Path 313 | total_timesteps 4201.
Path 314 | total_timesteps 4212.
Path 315 | total_timesteps 4230.
Path 316 | total_timesteps 4246.
Path 317 | total_timesteps 4263.
Path 318 | total_timesteps 4270.
Path 319 | total_timesteps 4279.
Path 320 | total_timesteps 4293.
Path 321 | total_timesteps 4304.
Path 322 | total_timesteps 4312.
Path 323 | total_timesteps 4325.
Path 324 | total_timesteps 4332.
Path 325 | total_timesteps 4353.
Path 326 | total_timesteps 4372.
Path 327 | total_timesteps 4396.
Path 328 | total_timesteps 4407.
Path 329 | total_timesteps 4417.
Path 330 | total_timesteps 4428.
Path 331 | total_timesteps 4453.
Path 332 | total_timesteps 4464.
Path 333 | total_timesteps 4477.
Path 334 | total_timesteps 4484.
Path 335 | total_timesteps 4499.
Path 336 | total_timesteps 4509.
Path 337 | total_timesteps 4532.
Path 338 | total_timesteps 4559.
Path 339 | total_timesteps 4573.
Path 340 | total_timesteps 4585.
Path 341 | total_timesteps 4608.
Path 342 | total_timesteps 4615.
Path 343 | total_timesteps 4624.
Path 344 | total_timesteps 4650.
Path 345 | total_timesteps 4662.
Path 346 | total_timesteps 4679.
Path 347 | total_timesteps 4690.
Path 348 | total_timesteps 4707.
Path 349 | total_timesteps 4721.
Path 350 | total_timesteps 4737.
Path 351 | total_timesteps 4747.
Path 352 | total_timesteps 4762.
Path 353 | total_timesteps 4783.
Path 354 | total_timesteps 4802.
Path 355 | total_timesteps 4814.
Path 356 | total_timesteps 4824.
Path 357 | total_timesteps 4839.
Path 358 | total_timesteps 4852.
Path 359 | total_timesteps 4873.
Path 360 | total_timesteps 4892.
Path 361 | total_timesteps 4908.
Path 362 | total_timesteps 4930.
Path 363 | total_timesteps 4943.
Path 364 | total_timesteps 4958.
Path 365 | total_timesteps 4966.
Path 366 | total_timesteps 4981.
Path 367 | total_timesteps 4990.
Path 368 | total_timesteps 5000.
Path 369 | total_timesteps 5010.
Path 370 | total_timesteps 5020.
Path 371 | total_timesteps 5033.
Path 372 | total_timesteps 5042.
Path 373 | total_timesteps 5053.
Path 374 | total_timesteps 5063.
Path 375 | total_timesteps 5072.
Path 376 | total_timesteps 5088.
Path 377 | total_timesteps 5100.
Path 378 | total_timesteps 5119.
Path 379 | total_timesteps 5137.
Path 380 | total_timesteps 5145.
Path 381 | total_timesteps 5156.
Path 382 | total_timesteps 5173.
Path 383 | total_timesteps 5184.
Path 384 | total_timesteps 5193.
Path 385 | total_timesteps 5203.
Path 386 | total_timesteps 5212.
Path 387 | total_timesteps 5238.
Path 388 | total_timesteps 5247.
Path 389 | total_timesteps 5267.
Path 390 | total_timesteps 5288.
Path 391 | total_timesteps 5302.
Path 392 | total_timesteps 5322.
Path 393 | total_timesteps 5342.
Path 394 | total_timesteps 5357.
Path 395 | total_timesteps 5376.
Path 396 | total_timesteps 5382.
Path 397 | total_timesteps 5399.
Path 398 | total_timesteps 5417.
Path 399 | total_timesteps 5429.
Path 400 | total_timesteps 5441.
Path 401 | total_timesteps 5455.
Path 402 | total_timesteps 5463.
Path 403 | total_timesteps 5472.
Path 404 | total_timesteps 5481.
Path 405 | total_timesteps 5493.
Path 406 | total_timesteps 5507.
Path 407 | total_timesteps 5518.
Path 408 | total_timesteps 5528.
Path 409 | total_timesteps 5537.
Path 410 | total_timesteps 5547.
Path 411 | total_timesteps 5562.
Path 412 | total_timesteps 5574.
Path 413 | total_timesteps 5587.
Path 414 | total_timesteps 5605.
Path 415 | total_timesteps 5613.
Path 416 | total_timesteps 5635.
Path 417 | total_timesteps 5646.
Path 418 | total_timesteps 5668.
Path 419 | total_timesteps 5674.
Path 420 | total_timesteps 5690.
Path 421 | total_timesteps 5702.
Path 422 | total_timesteps 5717.
Path 423 | total_timesteps 5732.
Path 424 | total_timesteps 5751.
Path 425 | total_timesteps 5762.
Path 426 | total_timesteps 5772.
Path 427 | total_timesteps 5781.
Path 428 | total_timesteps 5789.
Path 429 | total_timesteps 5801.
Path 430 | total_timesteps 5814.
Path 431 | total_timesteps 5830.
Path 432 | total_timesteps 5851.
Path 433 | total_timesteps 5873.
Path 434 | total_timesteps 5882.
Path 435 | total_timesteps 5893.
Path 436 | total_timesteps 5903.
Path 437 | total_timesteps 5923.
Path 438 | total_timesteps 5947.
Path 439 | total_timesteps 5954.
Path 440 | total_timesteps 5961.
Path 441 | total_timesteps 5977.
Path 442 | total_timesteps 5987.
Path 443 | total_timesteps 5996.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.82    |
| Iteration     | 22       |
| MaximumReturn | 2.87     |
| MinimumReturn | -22      |
| TotalSamples  | 96186    |
----------------------------
itr #23 | 
Fitting dynamics.
Validation loss = 0.002613479970023036
Validation loss = 0.00263921613804996
Validation loss = 0.0027613018173724413
Validation loss = 0.0025609107688069344
Validation loss = 0.0026463938411325216
Validation loss = 0.002545325318351388
Validation loss = 0.002893113764002919
Validation loss = 0.0025513421278446913
Validation loss = 0.0028125743847340345
Validation loss = 0.002647513523697853
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 25.
Path 3 | total_timesteps 39.
Path 4 | total_timesteps 47.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 70.
Path 7 | total_timesteps 84.
Path 8 | total_timesteps 94.
Path 9 | total_timesteps 104.
Path 10 | total_timesteps 126.
Path 11 | total_timesteps 133.
Path 12 | total_timesteps 140.
Path 13 | total_timesteps 157.
Path 14 | total_timesteps 168.
Path 15 | total_timesteps 177.
Path 16 | total_timesteps 188.
Path 17 | total_timesteps 199.
Path 18 | total_timesteps 214.
Path 19 | total_timesteps 226.
Path 20 | total_timesteps 233.
Path 21 | total_timesteps 254.
Path 22 | total_timesteps 267.
Path 23 | total_timesteps 283.
Path 24 | total_timesteps 298.
Path 25 | total_timesteps 315.
Path 26 | total_timesteps 322.
Path 27 | total_timesteps 342.
Path 28 | total_timesteps 352.
Path 29 | total_timesteps 361.
Path 30 | total_timesteps 372.
Path 31 | total_timesteps 383.
Path 32 | total_timesteps 391.
Path 33 | total_timesteps 402.
Path 34 | total_timesteps 418.
Path 35 | total_timesteps 430.
Path 36 | total_timesteps 440.
Path 37 | total_timesteps 456.
Path 38 | total_timesteps 470.
Path 39 | total_timesteps 483.
Path 40 | total_timesteps 498.
Path 41 | total_timesteps 505.
Path 42 | total_timesteps 519.
Path 43 | total_timesteps 538.
Path 44 | total_timesteps 551.
Path 45 | total_timesteps 577.
Path 46 | total_timesteps 591.
Path 47 | total_timesteps 599.
Path 48 | total_timesteps 607.
Path 49 | total_timesteps 620.
Path 50 | total_timesteps 630.
Path 51 | total_timesteps 644.
Path 52 | total_timesteps 657.
Path 53 | total_timesteps 670.
Path 54 | total_timesteps 679.
Path 55 | total_timesteps 687.
Path 56 | total_timesteps 704.
Path 57 | total_timesteps 712.
Path 58 | total_timesteps 721.
Path 59 | total_timesteps 737.
Path 60 | total_timesteps 745.
Path 61 | total_timesteps 758.
Path 62 | total_timesteps 777.
Path 63 | total_timesteps 796.
Path 64 | total_timesteps 806.
Path 65 | total_timesteps 819.
Path 66 | total_timesteps 831.
Path 67 | total_timesteps 841.
Path 68 | total_timesteps 851.
Path 69 | total_timesteps 860.
Path 70 | total_timesteps 874.
Path 71 | total_timesteps 892.
Path 72 | total_timesteps 902.
Path 73 | total_timesteps 915.
Path 74 | total_timesteps 925.
Path 75 | total_timesteps 948.
Path 76 | total_timesteps 956.
Path 77 | total_timesteps 963.
Path 78 | total_timesteps 974.
Path 79 | total_timesteps 982.
Path 80 | total_timesteps 1000.
Path 81 | total_timesteps 1007.
Path 82 | total_timesteps 1019.
Path 83 | total_timesteps 1036.
Path 84 | total_timesteps 1045.
Path 85 | total_timesteps 1058.
Path 86 | total_timesteps 1067.
Path 87 | total_timesteps 1080.
Path 88 | total_timesteps 1092.
Path 89 | total_timesteps 1105.
Path 90 | total_timesteps 1124.
Path 91 | total_timesteps 1134.
Path 92 | total_timesteps 1143.
Path 93 | total_timesteps 1158.
Path 94 | total_timesteps 1165.
Path 95 | total_timesteps 1184.
Path 96 | total_timesteps 1191.
Path 97 | total_timesteps 1202.
Path 98 | total_timesteps 1214.
Path 99 | total_timesteps 1225.
Path 100 | total_timesteps 1238.
Path 101 | total_timesteps 1249.
Path 102 | total_timesteps 1269.
Path 103 | total_timesteps 1280.
Path 104 | total_timesteps 1290.
Path 105 | total_timesteps 1302.
Path 106 | total_timesteps 1310.
Path 107 | total_timesteps 1321.
Path 108 | total_timesteps 1333.
Path 109 | total_timesteps 1341.
Path 110 | total_timesteps 1350.
Path 111 | total_timesteps 1366.
Path 112 | total_timesteps 1378.
Path 113 | total_timesteps 1393.
Path 114 | total_timesteps 1409.
Path 115 | total_timesteps 1419.
Path 116 | total_timesteps 1432.
Path 117 | total_timesteps 1454.
Path 118 | total_timesteps 1466.
Path 119 | total_timesteps 1479.
Path 120 | total_timesteps 1494.
Path 121 | total_timesteps 1516.
Path 122 | total_timesteps 1526.
Path 123 | total_timesteps 1535.
Path 124 | total_timesteps 1550.
Path 125 | total_timesteps 1561.
Path 126 | total_timesteps 1571.
Path 127 | total_timesteps 1579.
Path 128 | total_timesteps 1589.
Path 129 | total_timesteps 1602.
Path 130 | total_timesteps 1615.
Path 131 | total_timesteps 1625.
Path 132 | total_timesteps 1632.
Path 133 | total_timesteps 1651.
Path 134 | total_timesteps 1660.
Path 135 | total_timesteps 1673.
Path 136 | total_timesteps 1690.
Path 137 | total_timesteps 1706.
Path 138 | total_timesteps 1728.
Path 139 | total_timesteps 1749.
Path 140 | total_timesteps 1765.
Path 141 | total_timesteps 1776.
Path 142 | total_timesteps 1795.
Path 143 | total_timesteps 1814.
Path 144 | total_timesteps 1838.
Path 145 | total_timesteps 1856.
Path 146 | total_timesteps 1863.
Path 147 | total_timesteps 1874.
Path 148 | total_timesteps 1883.
Path 149 | total_timesteps 1892.
Path 150 | total_timesteps 1907.
Path 151 | total_timesteps 1924.
Path 152 | total_timesteps 1933.
Path 153 | total_timesteps 1947.
Path 154 | total_timesteps 1958.
Path 155 | total_timesteps 1967.
Path 156 | total_timesteps 1979.
Path 157 | total_timesteps 1995.
Path 158 | total_timesteps 2003.
Path 159 | total_timesteps 2011.
Path 160 | total_timesteps 2021.
Path 161 | total_timesteps 2030.
Path 162 | total_timesteps 2045.
Path 163 | total_timesteps 2054.
Path 164 | total_timesteps 2065.
Path 165 | total_timesteps 2077.
Path 166 | total_timesteps 2095.
Path 167 | total_timesteps 2104.
Path 168 | total_timesteps 2117.
Path 169 | total_timesteps 2134.
Path 170 | total_timesteps 2152.
Path 171 | total_timesteps 2163.
Path 172 | total_timesteps 2187.
Path 173 | total_timesteps 2210.
Path 174 | total_timesteps 2221.
Path 175 | total_timesteps 2232.
Path 176 | total_timesteps 2242.
Path 177 | total_timesteps 2251.
Path 178 | total_timesteps 2263.
Path 179 | total_timesteps 2280.
Path 180 | total_timesteps 2298.
Path 181 | total_timesteps 2308.
Path 182 | total_timesteps 2322.
Path 183 | total_timesteps 2339.
Path 184 | total_timesteps 2347.
Path 185 | total_timesteps 2356.
Path 186 | total_timesteps 2366.
Path 187 | total_timesteps 2378.
Path 188 | total_timesteps 2397.
Path 189 | total_timesteps 2411.
Path 190 | total_timesteps 2432.
Path 191 | total_timesteps 2446.
Path 192 | total_timesteps 2454.
Path 193 | total_timesteps 2463.
Path 194 | total_timesteps 2480.
Path 195 | total_timesteps 2502.
Path 196 | total_timesteps 2516.
Path 197 | total_timesteps 2525.
Path 198 | total_timesteps 2544.
Path 199 | total_timesteps 2557.
Path 200 | total_timesteps 2570.
Path 201 | total_timesteps 2583.
Path 202 | total_timesteps 2594.
Path 203 | total_timesteps 2602.
Path 204 | total_timesteps 2613.
Path 205 | total_timesteps 2627.
Path 206 | total_timesteps 2638.
Path 207 | total_timesteps 2660.
Path 208 | total_timesteps 2667.
Path 209 | total_timesteps 2680.
Path 210 | total_timesteps 2692.
Path 211 | total_timesteps 2700.
Path 212 | total_timesteps 2718.
Path 213 | total_timesteps 2727.
Path 214 | total_timesteps 2737.
Path 215 | total_timesteps 2751.
Path 216 | total_timesteps 2760.
Path 217 | total_timesteps 2772.
Path 218 | total_timesteps 2786.
Path 219 | total_timesteps 2799.
Path 220 | total_timesteps 2814.
Path 221 | total_timesteps 2836.
Path 222 | total_timesteps 2845.
Path 223 | total_timesteps 2856.
Path 224 | total_timesteps 2878.
Path 225 | total_timesteps 2888.
Path 226 | total_timesteps 2913.
Path 227 | total_timesteps 2924.
Path 228 | total_timesteps 2931.
Path 229 | total_timesteps 2940.
Path 230 | total_timesteps 2952.
Path 231 | total_timesteps 2970.
Path 232 | total_timesteps 2981.
Path 233 | total_timesteps 2999.
Path 234 | total_timesteps 3013.
Path 235 | total_timesteps 3028.
Path 236 | total_timesteps 3036.
Path 237 | total_timesteps 3043.
Path 238 | total_timesteps 3051.
Path 239 | total_timesteps 3063.
Path 240 | total_timesteps 3078.
Path 241 | total_timesteps 3089.
Path 242 | total_timesteps 3104.
Path 243 | total_timesteps 3122.
Path 244 | total_timesteps 3142.
Path 245 | total_timesteps 3163.
Path 246 | total_timesteps 3179.
Path 247 | total_timesteps 3187.
Path 248 | total_timesteps 3198.
Path 249 | total_timesteps 3206.
Path 250 | total_timesteps 3224.
Path 251 | total_timesteps 3240.
Path 252 | total_timesteps 3251.
Path 253 | total_timesteps 3268.
Path 254 | total_timesteps 3292.
Path 255 | total_timesteps 3299.
Path 256 | total_timesteps 3310.
Path 257 | total_timesteps 3323.
Path 258 | total_timesteps 3333.
Path 259 | total_timesteps 3351.
Path 260 | total_timesteps 3359.
Path 261 | total_timesteps 3378.
Path 262 | total_timesteps 3403.
Path 263 | total_timesteps 3415.
Path 264 | total_timesteps 3426.
Path 265 | total_timesteps 3432.
Path 266 | total_timesteps 3442.
Path 267 | total_timesteps 3452.
Path 268 | total_timesteps 3463.
Path 269 | total_timesteps 3474.
Path 270 | total_timesteps 3495.
Path 271 | total_timesteps 3503.
Path 272 | total_timesteps 3511.
Path 273 | total_timesteps 3532.
Path 274 | total_timesteps 3548.
Path 275 | total_timesteps 3570.
Path 276 | total_timesteps 3582.
Path 277 | total_timesteps 3591.
Path 278 | total_timesteps 3600.
Path 279 | total_timesteps 3615.
Path 280 | total_timesteps 3638.
Path 281 | total_timesteps 3648.
Path 282 | total_timesteps 3660.
Path 283 | total_timesteps 3669.
Path 284 | total_timesteps 3681.
Path 285 | total_timesteps 3692.
Path 286 | total_timesteps 3703.
Path 287 | total_timesteps 3725.
Path 288 | total_timesteps 3736.
Path 289 | total_timesteps 3746.
Path 290 | total_timesteps 3753.
Path 291 | total_timesteps 3764.
Path 292 | total_timesteps 3771.
Path 293 | total_timesteps 3794.
Path 294 | total_timesteps 3804.
Path 295 | total_timesteps 3820.
Path 296 | total_timesteps 3832.
Path 297 | total_timesteps 3848.
Path 298 | total_timesteps 3867.
Path 299 | total_timesteps 3879.
Path 300 | total_timesteps 3887.
Path 301 | total_timesteps 3896.
Path 302 | total_timesteps 3911.
Path 303 | total_timesteps 3920.
Path 304 | total_timesteps 3930.
Path 305 | total_timesteps 3939.
Path 306 | total_timesteps 3947.
Path 307 | total_timesteps 3957.
Path 308 | total_timesteps 3966.
Path 309 | total_timesteps 3988.
Path 310 | total_timesteps 3999.
Path 311 | total_timesteps 4007.
Path 312 | total_timesteps 4018.
Path 313 | total_timesteps 4033.
Path 314 | total_timesteps 4051.
Path 315 | total_timesteps 4071.
Path 316 | total_timesteps 4085.
Path 317 | total_timesteps 4102.
Path 318 | total_timesteps 4111.
Path 319 | total_timesteps 4124.
Path 320 | total_timesteps 4135.
Path 321 | total_timesteps 4149.
Path 322 | total_timesteps 4162.
Path 323 | total_timesteps 4175.
Path 324 | total_timesteps 4192.
Path 325 | total_timesteps 4201.
Path 326 | total_timesteps 4214.
Path 327 | total_timesteps 4229.
Path 328 | total_timesteps 4243.
Path 329 | total_timesteps 4262.
Path 330 | total_timesteps 4275.
Path 331 | total_timesteps 4287.
Path 332 | total_timesteps 4296.
Path 333 | total_timesteps 4319.
Path 334 | total_timesteps 4328.
Path 335 | total_timesteps 4338.
Path 336 | total_timesteps 4371.
Path 337 | total_timesteps 4383.
Path 338 | total_timesteps 4406.
Path 339 | total_timesteps 4426.
Path 340 | total_timesteps 4433.
Path 341 | total_timesteps 4449.
Path 342 | total_timesteps 4461.
Path 343 | total_timesteps 4474.
Path 344 | total_timesteps 4482.
Path 345 | total_timesteps 4504.
Path 346 | total_timesteps 4513.
Path 347 | total_timesteps 4534.
Path 348 | total_timesteps 4548.
Path 349 | total_timesteps 4561.
Path 350 | total_timesteps 4575.
Path 351 | total_timesteps 4585.
Path 352 | total_timesteps 4602.
Path 353 | total_timesteps 4611.
Path 354 | total_timesteps 4625.
Path 355 | total_timesteps 4636.
Path 356 | total_timesteps 4649.
Path 357 | total_timesteps 4669.
Path 358 | total_timesteps 4679.
Path 359 | total_timesteps 4691.
Path 360 | total_timesteps 4699.
Path 361 | total_timesteps 4709.
Path 362 | total_timesteps 4720.
Path 363 | total_timesteps 4729.
Path 364 | total_timesteps 4737.
Path 365 | total_timesteps 4752.
Path 366 | total_timesteps 4768.
Path 367 | total_timesteps 4783.
Path 368 | total_timesteps 4796.
Path 369 | total_timesteps 4813.
Path 370 | total_timesteps 4824.
Path 371 | total_timesteps 4839.
Path 372 | total_timesteps 4853.
Path 373 | total_timesteps 4861.
Path 374 | total_timesteps 4880.
Path 375 | total_timesteps 4893.
Path 376 | total_timesteps 4904.
Path 377 | total_timesteps 4912.
Path 378 | total_timesteps 4926.
Path 379 | total_timesteps 4938.
Path 380 | total_timesteps 4947.
Path 381 | total_timesteps 4966.
Path 382 | total_timesteps 4980.
Path 383 | total_timesteps 4988.
Path 384 | total_timesteps 5008.
Path 385 | total_timesteps 5017.
Path 386 | total_timesteps 5027.
Path 387 | total_timesteps 5042.
Path 388 | total_timesteps 5059.
Path 389 | total_timesteps 5066.
Path 390 | total_timesteps 5075.
Path 391 | total_timesteps 5096.
Path 392 | total_timesteps 5103.
Path 393 | total_timesteps 5114.
Path 394 | total_timesteps 5122.
Path 395 | total_timesteps 5135.
Path 396 | total_timesteps 5153.
Path 397 | total_timesteps 5170.
Path 398 | total_timesteps 5180.
Path 399 | total_timesteps 5189.
Path 400 | total_timesteps 5208.
Path 401 | total_timesteps 5216.
Path 402 | total_timesteps 5223.
Path 403 | total_timesteps 5237.
Path 404 | total_timesteps 5245.
Path 405 | total_timesteps 5253.
Path 406 | total_timesteps 5265.
Path 407 | total_timesteps 5273.
Path 408 | total_timesteps 5288.
Path 409 | total_timesteps 5300.
Path 410 | total_timesteps 5322.
Path 411 | total_timesteps 5342.
Path 412 | total_timesteps 5354.
Path 413 | total_timesteps 5363.
Path 414 | total_timesteps 5377.
Path 415 | total_timesteps 5385.
Path 416 | total_timesteps 5398.
Path 417 | total_timesteps 5407.
Path 418 | total_timesteps 5423.
Path 419 | total_timesteps 5434.
Path 420 | total_timesteps 5451.
Path 421 | total_timesteps 5460.
Path 422 | total_timesteps 5482.
Path 423 | total_timesteps 5490.
Path 424 | total_timesteps 5502.
Path 425 | total_timesteps 5511.
Path 426 | total_timesteps 5526.
Path 427 | total_timesteps 5535.
Path 428 | total_timesteps 5552.
Path 429 | total_timesteps 5565.
Path 430 | total_timesteps 5588.
Path 431 | total_timesteps 5598.
Path 432 | total_timesteps 5616.
Path 433 | total_timesteps 5627.
Path 434 | total_timesteps 5646.
Path 435 | total_timesteps 5656.
Path 436 | total_timesteps 5664.
Path 437 | total_timesteps 5672.
Path 438 | total_timesteps 5683.
Path 439 | total_timesteps 5691.
Path 440 | total_timesteps 5706.
Path 441 | total_timesteps 5717.
Path 442 | total_timesteps 5747.
Path 443 | total_timesteps 5761.
Path 444 | total_timesteps 5781.
Path 445 | total_timesteps 5800.
Path 446 | total_timesteps 5811.
Path 447 | total_timesteps 5825.
Path 448 | total_timesteps 5836.
Path 449 | total_timesteps 5856.
Path 450 | total_timesteps 5869.
Path 451 | total_timesteps 5878.
Path 452 | total_timesteps 5887.
Path 453 | total_timesteps 5895.
Path 454 | total_timesteps 5911.
Path 455 | total_timesteps 5926.
Path 456 | total_timesteps 5937.
Path 457 | total_timesteps 5951.
Path 458 | total_timesteps 5963.
Path 459 | total_timesteps 5977.
Path 460 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.65    |
| Iteration     | 23       |
| MaximumReturn | 3.84     |
| MinimumReturn | -23.9    |
| TotalSamples  | 100187   |
----------------------------
itr #24 | 
Fitting dynamics.
Validation loss = 0.0027441626880317926
Validation loss = 0.002564390189945698
Validation loss = 0.002470004605129361
Validation loss = 0.002813524566590786
Validation loss = 0.0025007901713252068
Validation loss = 0.0025705357547849417
Validation loss = 0.002557275351136923
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 8.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 36.
Path 4 | total_timesteps 59.
Path 5 | total_timesteps 73.
Path 6 | total_timesteps 84.
Path 7 | total_timesteps 98.
Path 8 | total_timesteps 111.
Path 9 | total_timesteps 119.
Path 10 | total_timesteps 130.
Path 11 | total_timesteps 138.
Path 12 | total_timesteps 147.
Path 13 | total_timesteps 162.
Path 14 | total_timesteps 173.
Path 15 | total_timesteps 180.
Path 16 | total_timesteps 199.
Path 17 | total_timesteps 211.
Path 18 | total_timesteps 221.
Path 19 | total_timesteps 239.
Path 20 | total_timesteps 257.
Path 21 | total_timesteps 265.
Path 22 | total_timesteps 286.
Path 23 | total_timesteps 294.
Path 24 | total_timesteps 304.
Path 25 | total_timesteps 315.
Path 26 | total_timesteps 329.
Path 27 | total_timesteps 340.
Path 28 | total_timesteps 358.
Path 29 | total_timesteps 377.
Path 30 | total_timesteps 387.
Path 31 | total_timesteps 397.
Path 32 | total_timesteps 412.
Path 33 | total_timesteps 425.
Path 34 | total_timesteps 433.
Path 35 | total_timesteps 448.
Path 36 | total_timesteps 457.
Path 37 | total_timesteps 479.
Path 38 | total_timesteps 491.
Path 39 | total_timesteps 499.
Path 40 | total_timesteps 517.
Path 41 | total_timesteps 526.
Path 42 | total_timesteps 537.
Path 43 | total_timesteps 548.
Path 44 | total_timesteps 557.
Path 45 | total_timesteps 573.
Path 46 | total_timesteps 592.
Path 47 | total_timesteps 609.
Path 48 | total_timesteps 621.
Path 49 | total_timesteps 632.
Path 50 | total_timesteps 646.
Path 51 | total_timesteps 669.
Path 52 | total_timesteps 685.
Path 53 | total_timesteps 695.
Path 54 | total_timesteps 704.
Path 55 | total_timesteps 716.
Path 56 | total_timesteps 725.
Path 57 | total_timesteps 735.
Path 58 | total_timesteps 745.
Path 59 | total_timesteps 767.
Path 60 | total_timesteps 776.
Path 61 | total_timesteps 784.
Path 62 | total_timesteps 804.
Path 63 | total_timesteps 823.
Path 64 | total_timesteps 838.
Path 65 | total_timesteps 852.
Path 66 | total_timesteps 861.
Path 67 | total_timesteps 871.
Path 68 | total_timesteps 890.
Path 69 | total_timesteps 899.
Path 70 | total_timesteps 908.
Path 71 | total_timesteps 918.
Path 72 | total_timesteps 928.
Path 73 | total_timesteps 941.
Path 74 | total_timesteps 961.
Path 75 | total_timesteps 974.
Path 76 | total_timesteps 987.
Path 77 | total_timesteps 994.
Path 78 | total_timesteps 1002.
Path 79 | total_timesteps 1019.
Path 80 | total_timesteps 1028.
Path 81 | total_timesteps 1041.
Path 82 | total_timesteps 1053.
Path 83 | total_timesteps 1061.
Path 84 | total_timesteps 1072.
Path 85 | total_timesteps 1079.
Path 86 | total_timesteps 1092.
Path 87 | total_timesteps 1102.
Path 88 | total_timesteps 1118.
Path 89 | total_timesteps 1129.
Path 90 | total_timesteps 1143.
Path 91 | total_timesteps 1150.
Path 92 | total_timesteps 1161.
Path 93 | total_timesteps 1174.
Path 94 | total_timesteps 1187.
Path 95 | total_timesteps 1211.
Path 96 | total_timesteps 1222.
Path 97 | total_timesteps 1230.
Path 98 | total_timesteps 1237.
Path 99 | total_timesteps 1247.
Path 100 | total_timesteps 1266.
Path 101 | total_timesteps 1276.
Path 102 | total_timesteps 1290.
Path 103 | total_timesteps 1323.
Path 104 | total_timesteps 1336.
Path 105 | total_timesteps 1344.
Path 106 | total_timesteps 1353.
Path 107 | total_timesteps 1370.
Path 108 | total_timesteps 1377.
Path 109 | total_timesteps 1387.
Path 110 | total_timesteps 1398.
Path 111 | total_timesteps 1422.
Path 112 | total_timesteps 1431.
Path 113 | total_timesteps 1443.
Path 114 | total_timesteps 1453.
Path 115 | total_timesteps 1461.
Path 116 | total_timesteps 1470.
Path 117 | total_timesteps 1495.
Path 118 | total_timesteps 1505.
Path 119 | total_timesteps 1519.
Path 120 | total_timesteps 1535.
Path 121 | total_timesteps 1543.
Path 122 | total_timesteps 1551.
Path 123 | total_timesteps 1568.
Path 124 | total_timesteps 1579.
Path 125 | total_timesteps 1587.
Path 126 | total_timesteps 1603.
Path 127 | total_timesteps 1619.
Path 128 | total_timesteps 1634.
Path 129 | total_timesteps 1645.
Path 130 | total_timesteps 1660.
Path 131 | total_timesteps 1667.
Path 132 | total_timesteps 1678.
Path 133 | total_timesteps 1693.
Path 134 | total_timesteps 1701.
Path 135 | total_timesteps 1716.
Path 136 | total_timesteps 1728.
Path 137 | total_timesteps 1745.
Path 138 | total_timesteps 1755.
Path 139 | total_timesteps 1764.
Path 140 | total_timesteps 1772.
Path 141 | total_timesteps 1780.
Path 142 | total_timesteps 1803.
Path 143 | total_timesteps 1812.
Path 144 | total_timesteps 1826.
Path 145 | total_timesteps 1836.
Path 146 | total_timesteps 1850.
Path 147 | total_timesteps 1858.
Path 148 | total_timesteps 1867.
Path 149 | total_timesteps 1874.
Path 150 | total_timesteps 1895.
Path 151 | total_timesteps 1908.
Path 152 | total_timesteps 1916.
Path 153 | total_timesteps 1926.
Path 154 | total_timesteps 1945.
Path 155 | total_timesteps 1956.
Path 156 | total_timesteps 1967.
Path 157 | total_timesteps 1985.
Path 158 | total_timesteps 1998.
Path 159 | total_timesteps 2006.
Path 160 | total_timesteps 2015.
Path 161 | total_timesteps 2031.
Path 162 | total_timesteps 2040.
Path 163 | total_timesteps 2049.
Path 164 | total_timesteps 2064.
Path 165 | total_timesteps 2082.
Path 166 | total_timesteps 2090.
Path 167 | total_timesteps 2101.
Path 168 | total_timesteps 2112.
Path 169 | total_timesteps 2122.
Path 170 | total_timesteps 2140.
Path 171 | total_timesteps 2147.
Path 172 | total_timesteps 2156.
Path 173 | total_timesteps 2166.
Path 174 | total_timesteps 2174.
Path 175 | total_timesteps 2183.
Path 176 | total_timesteps 2199.
Path 177 | total_timesteps 2215.
Path 178 | total_timesteps 2224.
Path 179 | total_timesteps 2240.
Path 180 | total_timesteps 2261.
Path 181 | total_timesteps 2271.
Path 182 | total_timesteps 2284.
Path 183 | total_timesteps 2299.
Path 184 | total_timesteps 2306.
Path 185 | total_timesteps 2316.
Path 186 | total_timesteps 2334.
Path 187 | total_timesteps 2341.
Path 188 | total_timesteps 2359.
Path 189 | total_timesteps 2372.
Path 190 | total_timesteps 2382.
Path 191 | total_timesteps 2390.
Path 192 | total_timesteps 2398.
Path 193 | total_timesteps 2405.
Path 194 | total_timesteps 2418.
Path 195 | total_timesteps 2426.
Path 196 | total_timesteps 2435.
Path 197 | total_timesteps 2443.
Path 198 | total_timesteps 2459.
Path 199 | total_timesteps 2467.
Path 200 | total_timesteps 2485.
Path 201 | total_timesteps 2501.
Path 202 | total_timesteps 2511.
Path 203 | total_timesteps 2524.
Path 204 | total_timesteps 2543.
Path 205 | total_timesteps 2552.
Path 206 | total_timesteps 2564.
Path 207 | total_timesteps 2576.
Path 208 | total_timesteps 2599.
Path 209 | total_timesteps 2619.
Path 210 | total_timesteps 2627.
Path 211 | total_timesteps 2637.
Path 212 | total_timesteps 2647.
Path 213 | total_timesteps 2662.
Path 214 | total_timesteps 2673.
Path 215 | total_timesteps 2684.
Path 216 | total_timesteps 2702.
Path 217 | total_timesteps 2711.
Path 218 | total_timesteps 2721.
Path 219 | total_timesteps 2739.
Path 220 | total_timesteps 2747.
Path 221 | total_timesteps 2762.
Path 222 | total_timesteps 2781.
Path 223 | total_timesteps 2791.
Path 224 | total_timesteps 2821.
Path 225 | total_timesteps 2833.
Path 226 | total_timesteps 2848.
Path 227 | total_timesteps 2859.
Path 228 | total_timesteps 2868.
Path 229 | total_timesteps 2876.
Path 230 | total_timesteps 2888.
Path 231 | total_timesteps 2898.
Path 232 | total_timesteps 2907.
Path 233 | total_timesteps 2921.
Path 234 | total_timesteps 2938.
Path 235 | total_timesteps 2947.
Path 236 | total_timesteps 2959.
Path 237 | total_timesteps 2969.
Path 238 | total_timesteps 2990.
Path 239 | total_timesteps 3003.
Path 240 | total_timesteps 3013.
Path 241 | total_timesteps 3021.
Path 242 | total_timesteps 3031.
Path 243 | total_timesteps 3040.
Path 244 | total_timesteps 3047.
Path 245 | total_timesteps 3059.
Path 246 | total_timesteps 3068.
Path 247 | total_timesteps 3077.
Path 248 | total_timesteps 3084.
Path 249 | total_timesteps 3095.
Path 250 | total_timesteps 3106.
Path 251 | total_timesteps 3119.
Path 252 | total_timesteps 3133.
Path 253 | total_timesteps 3155.
Path 254 | total_timesteps 3168.
Path 255 | total_timesteps 3181.
Path 256 | total_timesteps 3203.
Path 257 | total_timesteps 3221.
Path 258 | total_timesteps 3228.
Path 259 | total_timesteps 3238.
Path 260 | total_timesteps 3251.
Path 261 | total_timesteps 3261.
Path 262 | total_timesteps 3275.
Path 263 | total_timesteps 3285.
Path 264 | total_timesteps 3299.
Path 265 | total_timesteps 3314.
Path 266 | total_timesteps 3322.
Path 267 | total_timesteps 3331.
Path 268 | total_timesteps 3341.
Path 269 | total_timesteps 3357.
Path 270 | total_timesteps 3396.
Path 271 | total_timesteps 3405.
Path 272 | total_timesteps 3415.
Path 273 | total_timesteps 3425.
Path 274 | total_timesteps 3442.
Path 275 | total_timesteps 3449.
Path 276 | total_timesteps 3458.
Path 277 | total_timesteps 3476.
Path 278 | total_timesteps 3486.
Path 279 | total_timesteps 3496.
Path 280 | total_timesteps 3511.
Path 281 | total_timesteps 3527.
Path 282 | total_timesteps 3540.
Path 283 | total_timesteps 3560.
Path 284 | total_timesteps 3571.
Path 285 | total_timesteps 3585.
Path 286 | total_timesteps 3597.
Path 287 | total_timesteps 3616.
Path 288 | total_timesteps 3633.
Path 289 | total_timesteps 3645.
Path 290 | total_timesteps 3659.
Path 291 | total_timesteps 3677.
Path 292 | total_timesteps 3689.
Path 293 | total_timesteps 3699.
Path 294 | total_timesteps 3718.
Path 295 | total_timesteps 3726.
Path 296 | total_timesteps 3736.
Path 297 | total_timesteps 3745.
Path 298 | total_timesteps 3762.
Path 299 | total_timesteps 3772.
Path 300 | total_timesteps 3782.
Path 301 | total_timesteps 3791.
Path 302 | total_timesteps 3812.
Path 303 | total_timesteps 3827.
Path 304 | total_timesteps 3838.
Path 305 | total_timesteps 3846.
Path 306 | total_timesteps 3862.
Path 307 | total_timesteps 3877.
Path 308 | total_timesteps 3887.
Path 309 | total_timesteps 3908.
Path 310 | total_timesteps 3923.
Path 311 | total_timesteps 3930.
Path 312 | total_timesteps 3952.
Path 313 | total_timesteps 3962.
Path 314 | total_timesteps 3971.
Path 315 | total_timesteps 3978.
Path 316 | total_timesteps 3989.
Path 317 | total_timesteps 4005.
Path 318 | total_timesteps 4012.
Path 319 | total_timesteps 4023.
Path 320 | total_timesteps 4030.
Path 321 | total_timesteps 4060.
Path 322 | total_timesteps 4076.
Path 323 | total_timesteps 4085.
Path 324 | total_timesteps 4094.
Path 325 | total_timesteps 4106.
Path 326 | total_timesteps 4118.
Path 327 | total_timesteps 4129.
Path 328 | total_timesteps 4138.
Path 329 | total_timesteps 4148.
Path 330 | total_timesteps 4163.
Path 331 | total_timesteps 4180.
Path 332 | total_timesteps 4193.
Path 333 | total_timesteps 4205.
Path 334 | total_timesteps 4213.
Path 335 | total_timesteps 4231.
Path 336 | total_timesteps 4256.
Path 337 | total_timesteps 4266.
Path 338 | total_timesteps 4275.
Path 339 | total_timesteps 4283.
Path 340 | total_timesteps 4291.
Path 341 | total_timesteps 4300.
Path 342 | total_timesteps 4311.
Path 343 | total_timesteps 4321.
Path 344 | total_timesteps 4341.
Path 345 | total_timesteps 4358.
Path 346 | total_timesteps 4367.
Path 347 | total_timesteps 4384.
Path 348 | total_timesteps 4393.
Path 349 | total_timesteps 4403.
Path 350 | total_timesteps 4413.
Path 351 | total_timesteps 4431.
Path 352 | total_timesteps 4446.
Path 353 | total_timesteps 4453.
Path 354 | total_timesteps 4470.
Path 355 | total_timesteps 4481.
Path 356 | total_timesteps 4492.
Path 357 | total_timesteps 4502.
Path 358 | total_timesteps 4522.
Path 359 | total_timesteps 4534.
Path 360 | total_timesteps 4544.
Path 361 | total_timesteps 4568.
Path 362 | total_timesteps 4585.
Path 363 | total_timesteps 4607.
Path 364 | total_timesteps 4621.
Path 365 | total_timesteps 4634.
Path 366 | total_timesteps 4641.
Path 367 | total_timesteps 4650.
Path 368 | total_timesteps 4662.
Path 369 | total_timesteps 4678.
Path 370 | total_timesteps 4688.
Path 371 | total_timesteps 4702.
Path 372 | total_timesteps 4719.
Path 373 | total_timesteps 4728.
Path 374 | total_timesteps 4748.
Path 375 | total_timesteps 4758.
Path 376 | total_timesteps 4768.
Path 377 | total_timesteps 4781.
Path 378 | total_timesteps 4800.
Path 379 | total_timesteps 4814.
Path 380 | total_timesteps 4822.
Path 381 | total_timesteps 4832.
Path 382 | total_timesteps 4845.
Path 383 | total_timesteps 4856.
Path 384 | total_timesteps 4867.
Path 385 | total_timesteps 4888.
Path 386 | total_timesteps 4904.
Path 387 | total_timesteps 4919.
Path 388 | total_timesteps 4929.
Path 389 | total_timesteps 4945.
Path 390 | total_timesteps 4958.
Path 391 | total_timesteps 4971.
Path 392 | total_timesteps 4996.
Path 393 | total_timesteps 5004.
Path 394 | total_timesteps 5024.
Path 395 | total_timesteps 5035.
Path 396 | total_timesteps 5044.
Path 397 | total_timesteps 5055.
Path 398 | total_timesteps 5067.
Path 399 | total_timesteps 5087.
Path 400 | total_timesteps 5101.
Path 401 | total_timesteps 5108.
Path 402 | total_timesteps 5118.
Path 403 | total_timesteps 5134.
Path 404 | total_timesteps 5151.
Path 405 | total_timesteps 5162.
Path 406 | total_timesteps 5172.
Path 407 | total_timesteps 5181.
Path 408 | total_timesteps 5200.
Path 409 | total_timesteps 5213.
Path 410 | total_timesteps 5227.
Path 411 | total_timesteps 5238.
Path 412 | total_timesteps 5249.
Path 413 | total_timesteps 5261.
Path 414 | total_timesteps 5274.
Path 415 | total_timesteps 5287.
Path 416 | total_timesteps 5294.
Path 417 | total_timesteps 5310.
Path 418 | total_timesteps 5318.
Path 419 | total_timesteps 5328.
Path 420 | total_timesteps 5338.
Path 421 | total_timesteps 5352.
Path 422 | total_timesteps 5369.
Path 423 | total_timesteps 5379.
Path 424 | total_timesteps 5392.
Path 425 | total_timesteps 5406.
Path 426 | total_timesteps 5414.
Path 427 | total_timesteps 5423.
Path 428 | total_timesteps 5435.
Path 429 | total_timesteps 5444.
Path 430 | total_timesteps 5452.
Path 431 | total_timesteps 5471.
Path 432 | total_timesteps 5481.
Path 433 | total_timesteps 5491.
Path 434 | total_timesteps 5514.
Path 435 | total_timesteps 5526.
Path 436 | total_timesteps 5537.
Path 437 | total_timesteps 5543.
Path 438 | total_timesteps 5550.
Path 439 | total_timesteps 5563.
Path 440 | total_timesteps 5572.
Path 441 | total_timesteps 5585.
Path 442 | total_timesteps 5593.
Path 443 | total_timesteps 5610.
Path 444 | total_timesteps 5625.
Path 445 | total_timesteps 5637.
Path 446 | total_timesteps 5646.
Path 447 | total_timesteps 5661.
Path 448 | total_timesteps 5673.
Path 449 | total_timesteps 5680.
Path 450 | total_timesteps 5693.
Path 451 | total_timesteps 5704.
Path 452 | total_timesteps 5717.
Path 453 | total_timesteps 5734.
Path 454 | total_timesteps 5756.
Path 455 | total_timesteps 5763.
Path 456 | total_timesteps 5773.
Path 457 | total_timesteps 5792.
Path 458 | total_timesteps 5812.
Path 459 | total_timesteps 5822.
Path 460 | total_timesteps 5830.
Path 461 | total_timesteps 5844.
Path 462 | total_timesteps 5856.
Path 463 | total_timesteps 5874.
Path 464 | total_timesteps 5894.
Path 465 | total_timesteps 5908.
Path 466 | total_timesteps 5915.
Path 467 | total_timesteps 5925.
Path 468 | total_timesteps 5936.
Path 469 | total_timesteps 5946.
Path 470 | total_timesteps 5961.
Path 471 | total_timesteps 5970.
Path 472 | total_timesteps 5984.
Path 473 | total_timesteps 5992.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -7.48    |
| Iteration     | 24       |
| MaximumReturn | 0.665    |
| MinimumReturn | -20.9    |
| TotalSamples  | 104197   |
----------------------------
itr #25 | 
Fitting dynamics.
Validation loss = 0.002484718104824424
Validation loss = 0.0025140256620943546
Validation loss = 0.0024749678559601307
Validation loss = 0.0024784482084214687
Validation loss = 0.002285108668729663
Validation loss = 0.0025979806669056416
Validation loss = 0.0024577677249908447
Validation loss = 0.002674804301932454
Validation loss = 0.002950353780761361
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 9.
Path 2 | total_timesteps 18.
Path 3 | total_timesteps 40.
Path 4 | total_timesteps 50.
Path 5 | total_timesteps 58.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 92.
Path 8 | total_timesteps 104.
Path 9 | total_timesteps 115.
Path 10 | total_timesteps 126.
Path 11 | total_timesteps 136.
Path 12 | total_timesteps 150.
Path 13 | total_timesteps 156.
Path 14 | total_timesteps 170.
Path 15 | total_timesteps 182.
Path 16 | total_timesteps 190.
Path 17 | total_timesteps 204.
Path 18 | total_timesteps 212.
Path 19 | total_timesteps 224.
Path 20 | total_timesteps 235.
Path 21 | total_timesteps 245.
Path 22 | total_timesteps 258.
Path 23 | total_timesteps 269.
Path 24 | total_timesteps 278.
Path 25 | total_timesteps 292.
Path 26 | total_timesteps 302.
Path 27 | total_timesteps 311.
Path 28 | total_timesteps 322.
Path 29 | total_timesteps 338.
Path 30 | total_timesteps 355.
Path 31 | total_timesteps 368.
Path 32 | total_timesteps 379.
Path 33 | total_timesteps 387.
Path 34 | total_timesteps 402.
Path 35 | total_timesteps 423.
Path 36 | total_timesteps 433.
Path 37 | total_timesteps 445.
Path 38 | total_timesteps 454.
Path 39 | total_timesteps 464.
Path 40 | total_timesteps 472.
Path 41 | total_timesteps 481.
Path 42 | total_timesteps 496.
Path 43 | total_timesteps 514.
Path 44 | total_timesteps 526.
Path 45 | total_timesteps 557.
Path 46 | total_timesteps 567.
Path 47 | total_timesteps 578.
Path 48 | total_timesteps 589.
Path 49 | total_timesteps 598.
Path 50 | total_timesteps 616.
Path 51 | total_timesteps 633.
Path 52 | total_timesteps 643.
Path 53 | total_timesteps 653.
Path 54 | total_timesteps 661.
Path 55 | total_timesteps 677.
Path 56 | total_timesteps 691.
Path 57 | total_timesteps 704.
Path 58 | total_timesteps 724.
Path 59 | total_timesteps 736.
Path 60 | total_timesteps 759.
Path 61 | total_timesteps 769.
Path 62 | total_timesteps 783.
Path 63 | total_timesteps 791.
Path 64 | total_timesteps 802.
Path 65 | total_timesteps 810.
Path 66 | total_timesteps 820.
Path 67 | total_timesteps 829.
Path 68 | total_timesteps 838.
Path 69 | total_timesteps 845.
Path 70 | total_timesteps 859.
Path 71 | total_timesteps 866.
Path 72 | total_timesteps 875.
Path 73 | total_timesteps 885.
Path 74 | total_timesteps 893.
Path 75 | total_timesteps 901.
Path 76 | total_timesteps 911.
Path 77 | total_timesteps 921.
Path 78 | total_timesteps 930.
Path 79 | total_timesteps 947.
Path 80 | total_timesteps 954.
Path 81 | total_timesteps 974.
Path 82 | total_timesteps 983.
Path 83 | total_timesteps 1009.
Path 84 | total_timesteps 1028.
Path 85 | total_timesteps 1041.
Path 86 | total_timesteps 1049.
Path 87 | total_timesteps 1064.
Path 88 | total_timesteps 1078.
Path 89 | total_timesteps 1088.
Path 90 | total_timesteps 1097.
Path 91 | total_timesteps 1107.
Path 92 | total_timesteps 1121.
Path 93 | total_timesteps 1129.
Path 94 | total_timesteps 1135.
Path 95 | total_timesteps 1161.
Path 96 | total_timesteps 1173.
Path 97 | total_timesteps 1191.
Path 98 | total_timesteps 1202.
Path 99 | total_timesteps 1209.
Path 100 | total_timesteps 1228.
Path 101 | total_timesteps 1236.
Path 102 | total_timesteps 1251.
Path 103 | total_timesteps 1261.
Path 104 | total_timesteps 1268.
Path 105 | total_timesteps 1279.
Path 106 | total_timesteps 1287.
Path 107 | total_timesteps 1310.
Path 108 | total_timesteps 1318.
Path 109 | total_timesteps 1330.
Path 110 | total_timesteps 1342.
Path 111 | total_timesteps 1353.
Path 112 | total_timesteps 1361.
Path 113 | total_timesteps 1371.
Path 114 | total_timesteps 1381.
Path 115 | total_timesteps 1390.
Path 116 | total_timesteps 1404.
Path 117 | total_timesteps 1414.
Path 118 | total_timesteps 1425.
Path 119 | total_timesteps 1444.
Path 120 | total_timesteps 1457.
Path 121 | total_timesteps 1475.
Path 122 | total_timesteps 1486.
Path 123 | total_timesteps 1499.
Path 124 | total_timesteps 1508.
Path 125 | total_timesteps 1520.
Path 126 | total_timesteps 1527.
Path 127 | total_timesteps 1549.
Path 128 | total_timesteps 1557.
Path 129 | total_timesteps 1569.
Path 130 | total_timesteps 1580.
Path 131 | total_timesteps 1589.
Path 132 | total_timesteps 1597.
Path 133 | total_timesteps 1605.
Path 134 | total_timesteps 1614.
Path 135 | total_timesteps 1623.
Path 136 | total_timesteps 1630.
Path 137 | total_timesteps 1639.
Path 138 | total_timesteps 1649.
Path 139 | total_timesteps 1660.
Path 140 | total_timesteps 1673.
Path 141 | total_timesteps 1688.
Path 142 | total_timesteps 1706.
Path 143 | total_timesteps 1714.
Path 144 | total_timesteps 1730.
Path 145 | total_timesteps 1738.
Path 146 | total_timesteps 1764.
Path 147 | total_timesteps 1778.
Path 148 | total_timesteps 1791.
Path 149 | total_timesteps 1804.
Path 150 | total_timesteps 1813.
Path 151 | total_timesteps 1827.
Path 152 | total_timesteps 1838.
Path 153 | total_timesteps 1849.
Path 154 | total_timesteps 1862.
Path 155 | total_timesteps 1875.
Path 156 | total_timesteps 1894.
Path 157 | total_timesteps 1907.
Path 158 | total_timesteps 1920.
Path 159 | total_timesteps 1939.
Path 160 | total_timesteps 1947.
Path 161 | total_timesteps 1967.
Path 162 | total_timesteps 1986.
Path 163 | total_timesteps 1998.
Path 164 | total_timesteps 2022.
Path 165 | total_timesteps 2035.
Path 166 | total_timesteps 2048.
Path 167 | total_timesteps 2057.
Path 168 | total_timesteps 2066.
Path 169 | total_timesteps 2074.
Path 170 | total_timesteps 2082.
Path 171 | total_timesteps 2097.
Path 172 | total_timesteps 2119.
Path 173 | total_timesteps 2130.
Path 174 | total_timesteps 2138.
Path 175 | total_timesteps 2153.
Path 176 | total_timesteps 2167.
Path 177 | total_timesteps 2179.
Path 178 | total_timesteps 2192.
Path 179 | total_timesteps 2207.
Path 180 | total_timesteps 2220.
Path 181 | total_timesteps 2233.
Path 182 | total_timesteps 2243.
Path 183 | total_timesteps 2253.
Path 184 | total_timesteps 2267.
Path 185 | total_timesteps 2274.
Path 186 | total_timesteps 2299.
Path 187 | total_timesteps 2309.
Path 188 | total_timesteps 2321.
Path 189 | total_timesteps 2332.
Path 190 | total_timesteps 2343.
Path 191 | total_timesteps 2360.
Path 192 | total_timesteps 2372.
Path 193 | total_timesteps 2381.
Path 194 | total_timesteps 2391.
Path 195 | total_timesteps 2398.
Path 196 | total_timesteps 2414.
Path 197 | total_timesteps 2421.
Path 198 | total_timesteps 2429.
Path 199 | total_timesteps 2441.
Path 200 | total_timesteps 2448.
Path 201 | total_timesteps 2472.
Path 202 | total_timesteps 2481.
Path 203 | total_timesteps 2493.
Path 204 | total_timesteps 2507.
Path 205 | total_timesteps 2516.
Path 206 | total_timesteps 2530.
Path 207 | total_timesteps 2544.
Path 208 | total_timesteps 2555.
Path 209 | total_timesteps 2561.
Path 210 | total_timesteps 2573.
Path 211 | total_timesteps 2584.
Path 212 | total_timesteps 2595.
Path 213 | total_timesteps 2615.
Path 214 | total_timesteps 2624.
Path 215 | total_timesteps 2635.
Path 216 | total_timesteps 2647.
Path 217 | total_timesteps 2656.
Path 218 | total_timesteps 2666.
Path 219 | total_timesteps 2678.
Path 220 | total_timesteps 2685.
Path 221 | total_timesteps 2697.
Path 222 | total_timesteps 2705.
Path 223 | total_timesteps 2714.
Path 224 | total_timesteps 2721.
Path 225 | total_timesteps 2737.
Path 226 | total_timesteps 2752.
Path 227 | total_timesteps 2762.
Path 228 | total_timesteps 2783.
Path 229 | total_timesteps 2796.
Path 230 | total_timesteps 2810.
Path 231 | total_timesteps 2828.
Path 232 | total_timesteps 2839.
Path 233 | total_timesteps 2851.
Path 234 | total_timesteps 2861.
Path 235 | total_timesteps 2869.
Path 236 | total_timesteps 2882.
Path 237 | total_timesteps 2893.
Path 238 | total_timesteps 2904.
Path 239 | total_timesteps 2912.
Path 240 | total_timesteps 2919.
Path 241 | total_timesteps 2931.
Path 242 | total_timesteps 2944.
Path 243 | total_timesteps 2952.
Path 244 | total_timesteps 2966.
Path 245 | total_timesteps 2979.
Path 246 | total_timesteps 2986.
Path 247 | total_timesteps 2999.
Path 248 | total_timesteps 3009.
Path 249 | total_timesteps 3016.
Path 250 | total_timesteps 3025.
Path 251 | total_timesteps 3033.
Path 252 | total_timesteps 3040.
Path 253 | total_timesteps 3052.
Path 254 | total_timesteps 3059.
Path 255 | total_timesteps 3069.
Path 256 | total_timesteps 3076.
Path 257 | total_timesteps 3086.
Path 258 | total_timesteps 3094.
Path 259 | total_timesteps 3102.
Path 260 | total_timesteps 3119.
Path 261 | total_timesteps 3133.
Path 262 | total_timesteps 3153.
Path 263 | total_timesteps 3166.
Path 264 | total_timesteps 3178.
Path 265 | total_timesteps 3187.
Path 266 | total_timesteps 3197.
Path 267 | total_timesteps 3215.
Path 268 | total_timesteps 3226.
Path 269 | total_timesteps 3240.
Path 270 | total_timesteps 3264.
Path 271 | total_timesteps 3274.
Path 272 | total_timesteps 3294.
Path 273 | total_timesteps 3303.
Path 274 | total_timesteps 3324.
Path 275 | total_timesteps 3342.
Path 276 | total_timesteps 3349.
Path 277 | total_timesteps 3358.
Path 278 | total_timesteps 3368.
Path 279 | total_timesteps 3380.
Path 280 | total_timesteps 3389.
Path 281 | total_timesteps 3398.
Path 282 | total_timesteps 3409.
Path 283 | total_timesteps 3418.
Path 284 | total_timesteps 3430.
Path 285 | total_timesteps 3440.
Path 286 | total_timesteps 3451.
Path 287 | total_timesteps 3459.
Path 288 | total_timesteps 3467.
Path 289 | total_timesteps 3474.
Path 290 | total_timesteps 3494.
Path 291 | total_timesteps 3508.
Path 292 | total_timesteps 3515.
Path 293 | total_timesteps 3532.
Path 294 | total_timesteps 3542.
Path 295 | total_timesteps 3550.
Path 296 | total_timesteps 3559.
Path 297 | total_timesteps 3568.
Path 298 | total_timesteps 3578.
Path 299 | total_timesteps 3588.
Path 300 | total_timesteps 3596.
Path 301 | total_timesteps 3611.
Path 302 | total_timesteps 3620.
Path 303 | total_timesteps 3627.
Path 304 | total_timesteps 3639.
Path 305 | total_timesteps 3649.
Path 306 | total_timesteps 3657.
Path 307 | total_timesteps 3669.
Path 308 | total_timesteps 3681.
Path 309 | total_timesteps 3692.
Path 310 | total_timesteps 3707.
Path 311 | total_timesteps 3720.
Path 312 | total_timesteps 3732.
Path 313 | total_timesteps 3741.
Path 314 | total_timesteps 3750.
Path 315 | total_timesteps 3759.
Path 316 | total_timesteps 3768.
Path 317 | total_timesteps 3776.
Path 318 | total_timesteps 3788.
Path 319 | total_timesteps 3809.
Path 320 | total_timesteps 3816.
Path 321 | total_timesteps 3825.
Path 322 | total_timesteps 3840.
Path 323 | total_timesteps 3851.
Path 324 | total_timesteps 3859.
Path 325 | total_timesteps 3878.
Path 326 | total_timesteps 3889.
Path 327 | total_timesteps 3898.
Path 328 | total_timesteps 3910.
Path 329 | total_timesteps 3919.
Path 330 | total_timesteps 3930.
Path 331 | total_timesteps 3940.
Path 332 | total_timesteps 3952.
Path 333 | total_timesteps 3963.
Path 334 | total_timesteps 3976.
Path 335 | total_timesteps 3984.
Path 336 | total_timesteps 3996.
Path 337 | total_timesteps 4007.
Path 338 | total_timesteps 4022.
Path 339 | total_timesteps 4031.
Path 340 | total_timesteps 4040.
Path 341 | total_timesteps 4052.
Path 342 | total_timesteps 4083.
Path 343 | total_timesteps 4091.
Path 344 | total_timesteps 4111.
Path 345 | total_timesteps 4121.
Path 346 | total_timesteps 4134.
Path 347 | total_timesteps 4149.
Path 348 | total_timesteps 4160.
Path 349 | total_timesteps 4174.
Path 350 | total_timesteps 4189.
Path 351 | total_timesteps 4204.
Path 352 | total_timesteps 4228.
Path 353 | total_timesteps 4236.
Path 354 | total_timesteps 4249.
Path 355 | total_timesteps 4259.
Path 356 | total_timesteps 4274.
Path 357 | total_timesteps 4282.
Path 358 | total_timesteps 4294.
Path 359 | total_timesteps 4305.
Path 360 | total_timesteps 4313.
Path 361 | total_timesteps 4334.
Path 362 | total_timesteps 4346.
Path 363 | total_timesteps 4360.
Path 364 | total_timesteps 4373.
Path 365 | total_timesteps 4384.
Path 366 | total_timesteps 4392.
Path 367 | total_timesteps 4405.
Path 368 | total_timesteps 4424.
Path 369 | total_timesteps 4433.
Path 370 | total_timesteps 4445.
Path 371 | total_timesteps 4456.
Path 372 | total_timesteps 4465.
Path 373 | total_timesteps 4472.
Path 374 | total_timesteps 4482.
Path 375 | total_timesteps 4496.
Path 376 | total_timesteps 4507.
Path 377 | total_timesteps 4518.
Path 378 | total_timesteps 4527.
Path 379 | total_timesteps 4537.
Path 380 | total_timesteps 4549.
Path 381 | total_timesteps 4560.
Path 382 | total_timesteps 4569.
Path 383 | total_timesteps 4588.
Path 384 | total_timesteps 4594.
Path 385 | total_timesteps 4605.
Path 386 | total_timesteps 4613.
Path 387 | total_timesteps 4629.
Path 388 | total_timesteps 4642.
Path 389 | total_timesteps 4650.
Path 390 | total_timesteps 4657.
Path 391 | total_timesteps 4665.
Path 392 | total_timesteps 4681.
Path 393 | total_timesteps 4693.
Path 394 | total_timesteps 4708.
Path 395 | total_timesteps 4716.
Path 396 | total_timesteps 4729.
Path 397 | total_timesteps 4738.
Path 398 | total_timesteps 4750.
Path 399 | total_timesteps 4756.
Path 400 | total_timesteps 4772.
Path 401 | total_timesteps 4796.
Path 402 | total_timesteps 4807.
Path 403 | total_timesteps 4825.
Path 404 | total_timesteps 4838.
Path 405 | total_timesteps 4851.
Path 406 | total_timesteps 4862.
Path 407 | total_timesteps 4871.
Path 408 | total_timesteps 4885.
Path 409 | total_timesteps 4895.
Path 410 | total_timesteps 4905.
Path 411 | total_timesteps 4915.
Path 412 | total_timesteps 4925.
Path 413 | total_timesteps 4932.
Path 414 | total_timesteps 4942.
Path 415 | total_timesteps 4954.
Path 416 | total_timesteps 4963.
Path 417 | total_timesteps 4972.
Path 418 | total_timesteps 4982.
Path 419 | total_timesteps 4996.
Path 420 | total_timesteps 5007.
Path 421 | total_timesteps 5018.
Path 422 | total_timesteps 5030.
Path 423 | total_timesteps 5042.
Path 424 | total_timesteps 5061.
Path 425 | total_timesteps 5068.
Path 426 | total_timesteps 5076.
Path 427 | total_timesteps 5088.
Path 428 | total_timesteps 5096.
Path 429 | total_timesteps 5104.
Path 430 | total_timesteps 5117.
Path 431 | total_timesteps 5129.
Path 432 | total_timesteps 5137.
Path 433 | total_timesteps 5150.
Path 434 | total_timesteps 5158.
Path 435 | total_timesteps 5174.
Path 436 | total_timesteps 5191.
Path 437 | total_timesteps 5203.
Path 438 | total_timesteps 5212.
Path 439 | total_timesteps 5223.
Path 440 | total_timesteps 5234.
Path 441 | total_timesteps 5242.
Path 442 | total_timesteps 5252.
Path 443 | total_timesteps 5271.
Path 444 | total_timesteps 5282.
Path 445 | total_timesteps 5306.
Path 446 | total_timesteps 5320.
Path 447 | total_timesteps 5330.
Path 448 | total_timesteps 5340.
Path 449 | total_timesteps 5349.
Path 450 | total_timesteps 5356.
Path 451 | total_timesteps 5372.
Path 452 | total_timesteps 5388.
Path 453 | total_timesteps 5396.
Path 454 | total_timesteps 5415.
Path 455 | total_timesteps 5422.
Path 456 | total_timesteps 5443.
Path 457 | total_timesteps 5455.
Path 458 | total_timesteps 5471.
Path 459 | total_timesteps 5479.
Path 460 | total_timesteps 5495.
Path 461 | total_timesteps 5503.
Path 462 | total_timesteps 5512.
Path 463 | total_timesteps 5521.
Path 464 | total_timesteps 5530.
Path 465 | total_timesteps 5539.
Path 466 | total_timesteps 5547.
Path 467 | total_timesteps 5555.
Path 468 | total_timesteps 5575.
Path 469 | total_timesteps 5588.
Path 470 | total_timesteps 5600.
Path 471 | total_timesteps 5611.
Path 472 | total_timesteps 5619.
Path 473 | total_timesteps 5628.
Path 474 | total_timesteps 5635.
Path 475 | total_timesteps 5645.
Path 476 | total_timesteps 5662.
Path 477 | total_timesteps 5671.
Path 478 | total_timesteps 5683.
Path 479 | total_timesteps 5697.
Path 480 | total_timesteps 5710.
Path 481 | total_timesteps 5728.
Path 482 | total_timesteps 5738.
Path 483 | total_timesteps 5751.
Path 484 | total_timesteps 5765.
Path 485 | total_timesteps 5786.
Path 486 | total_timesteps 5803.
Path 487 | total_timesteps 5819.
Path 488 | total_timesteps 5828.
Path 489 | total_timesteps 5836.
Path 490 | total_timesteps 5847.
Path 491 | total_timesteps 5861.
Path 492 | total_timesteps 5869.
Path 493 | total_timesteps 5881.
Path 494 | total_timesteps 5895.
Path 495 | total_timesteps 5905.
Path 496 | total_timesteps 5914.
Path 497 | total_timesteps 5930.
Path 498 | total_timesteps 5946.
Path 499 | total_timesteps 5957.
Path 500 | total_timesteps 5968.
Path 501 | total_timesteps 5986.
Path 502 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.73    |
| Iteration     | 25       |
| MaximumReturn | 0.892    |
| MinimumReturn | -19.9    |
| TotalSamples  | 108201   |
----------------------------
itr #26 | 
Fitting dynamics.
Validation loss = 0.002362331375479698
Validation loss = 0.002404857659712434
Validation loss = 0.0024670108687132597
Validation loss = 0.0023492726031690836
Validation loss = 0.0026414378080517054
Validation loss = 0.002204892924055457
Validation loss = 0.0024225274100899696
Validation loss = 0.0023147703614085913
Validation loss = 0.0023346596863120794
Validation loss = 0.00246990448795259
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 53.
Path 6 | total_timesteps 65.
Path 7 | total_timesteps 71.
Path 8 | total_timesteps 81.
Path 9 | total_timesteps 88.
Path 10 | total_timesteps 100.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 120.
Path 13 | total_timesteps 127.
Path 14 | total_timesteps 135.
Path 15 | total_timesteps 150.
Path 16 | total_timesteps 161.
Path 17 | total_timesteps 168.
Path 18 | total_timesteps 177.
Path 19 | total_timesteps 184.
Path 20 | total_timesteps 193.
Path 21 | total_timesteps 203.
Path 22 | total_timesteps 209.
Path 23 | total_timesteps 217.
Path 24 | total_timesteps 238.
Path 25 | total_timesteps 245.
Path 26 | total_timesteps 253.
Path 27 | total_timesteps 268.
Path 28 | total_timesteps 283.
Path 29 | total_timesteps 291.
Path 30 | total_timesteps 303.
Path 31 | total_timesteps 314.
Path 32 | total_timesteps 326.
Path 33 | total_timesteps 339.
Path 34 | total_timesteps 347.
Path 35 | total_timesteps 357.
Path 36 | total_timesteps 367.
Path 37 | total_timesteps 379.
Path 38 | total_timesteps 390.
Path 39 | total_timesteps 397.
Path 40 | total_timesteps 407.
Path 41 | total_timesteps 415.
Path 42 | total_timesteps 425.
Path 43 | total_timesteps 434.
Path 44 | total_timesteps 444.
Path 45 | total_timesteps 453.
Path 46 | total_timesteps 467.
Path 47 | total_timesteps 479.
Path 48 | total_timesteps 489.
Path 49 | total_timesteps 496.
Path 50 | total_timesteps 503.
Path 51 | total_timesteps 513.
Path 52 | total_timesteps 526.
Path 53 | total_timesteps 534.
Path 54 | total_timesteps 541.
Path 55 | total_timesteps 548.
Path 56 | total_timesteps 557.
Path 57 | total_timesteps 564.
Path 58 | total_timesteps 573.
Path 59 | total_timesteps 587.
Path 60 | total_timesteps 597.
Path 61 | total_timesteps 606.
Path 62 | total_timesteps 613.
Path 63 | total_timesteps 626.
Path 64 | total_timesteps 642.
Path 65 | total_timesteps 649.
Path 66 | total_timesteps 658.
Path 67 | total_timesteps 668.
Path 68 | total_timesteps 680.
Path 69 | total_timesteps 694.
Path 70 | total_timesteps 704.
Path 71 | total_timesteps 713.
Path 72 | total_timesteps 728.
Path 73 | total_timesteps 740.
Path 74 | total_timesteps 752.
Path 75 | total_timesteps 764.
Path 76 | total_timesteps 772.
Path 77 | total_timesteps 782.
Path 78 | total_timesteps 795.
Path 79 | total_timesteps 807.
Path 80 | total_timesteps 814.
Path 81 | total_timesteps 832.
Path 82 | total_timesteps 841.
Path 83 | total_timesteps 852.
Path 84 | total_timesteps 860.
Path 85 | total_timesteps 870.
Path 86 | total_timesteps 878.
Path 87 | total_timesteps 888.
Path 88 | total_timesteps 910.
Path 89 | total_timesteps 920.
Path 90 | total_timesteps 932.
Path 91 | total_timesteps 939.
Path 92 | total_timesteps 948.
Path 93 | total_timesteps 955.
Path 94 | total_timesteps 969.
Path 95 | total_timesteps 976.
Path 96 | total_timesteps 987.
Path 97 | total_timesteps 998.
Path 98 | total_timesteps 1008.
Path 99 | total_timesteps 1020.
Path 100 | total_timesteps 1035.
Path 101 | total_timesteps 1049.
Path 102 | total_timesteps 1058.
Path 103 | total_timesteps 1073.
Path 104 | total_timesteps 1086.
Path 105 | total_timesteps 1096.
Path 106 | total_timesteps 1113.
Path 107 | total_timesteps 1120.
Path 108 | total_timesteps 1129.
Path 109 | total_timesteps 1142.
Path 110 | total_timesteps 1161.
Path 111 | total_timesteps 1174.
Path 112 | total_timesteps 1183.
Path 113 | total_timesteps 1195.
Path 114 | total_timesteps 1210.
Path 115 | total_timesteps 1217.
Path 116 | total_timesteps 1227.
Path 117 | total_timesteps 1237.
Path 118 | total_timesteps 1244.
Path 119 | total_timesteps 1255.
Path 120 | total_timesteps 1266.
Path 121 | total_timesteps 1275.
Path 122 | total_timesteps 1287.
Path 123 | total_timesteps 1296.
Path 124 | total_timesteps 1306.
Path 125 | total_timesteps 1316.
Path 126 | total_timesteps 1338.
Path 127 | total_timesteps 1350.
Path 128 | total_timesteps 1361.
Path 129 | total_timesteps 1369.
Path 130 | total_timesteps 1381.
Path 131 | total_timesteps 1391.
Path 132 | total_timesteps 1403.
Path 133 | total_timesteps 1425.
Path 134 | total_timesteps 1431.
Path 135 | total_timesteps 1442.
Path 136 | total_timesteps 1455.
Path 137 | total_timesteps 1467.
Path 138 | total_timesteps 1480.
Path 139 | total_timesteps 1488.
Path 140 | total_timesteps 1498.
Path 141 | total_timesteps 1513.
Path 142 | total_timesteps 1520.
Path 143 | total_timesteps 1530.
Path 144 | total_timesteps 1539.
Path 145 | total_timesteps 1554.
Path 146 | total_timesteps 1565.
Path 147 | total_timesteps 1577.
Path 148 | total_timesteps 1585.
Path 149 | total_timesteps 1595.
Path 150 | total_timesteps 1605.
Path 151 | total_timesteps 1615.
Path 152 | total_timesteps 1623.
Path 153 | total_timesteps 1633.
Path 154 | total_timesteps 1645.
Path 155 | total_timesteps 1653.
Path 156 | total_timesteps 1664.
Path 157 | total_timesteps 1670.
Path 158 | total_timesteps 1677.
Path 159 | total_timesteps 1688.
Path 160 | total_timesteps 1697.
Path 161 | total_timesteps 1705.
Path 162 | total_timesteps 1715.
Path 163 | total_timesteps 1725.
Path 164 | total_timesteps 1733.
Path 165 | total_timesteps 1743.
Path 166 | total_timesteps 1755.
Path 167 | total_timesteps 1764.
Path 168 | total_timesteps 1774.
Path 169 | total_timesteps 1783.
Path 170 | total_timesteps 1792.
Path 171 | total_timesteps 1800.
Path 172 | total_timesteps 1809.
Path 173 | total_timesteps 1817.
Path 174 | total_timesteps 1831.
Path 175 | total_timesteps 1845.
Path 176 | total_timesteps 1858.
Path 177 | total_timesteps 1867.
Path 178 | total_timesteps 1879.
Path 179 | total_timesteps 1898.
Path 180 | total_timesteps 1908.
Path 181 | total_timesteps 1916.
Path 182 | total_timesteps 1925.
Path 183 | total_timesteps 1938.
Path 184 | total_timesteps 1949.
Path 185 | total_timesteps 1959.
Path 186 | total_timesteps 1968.
Path 187 | total_timesteps 1979.
Path 188 | total_timesteps 1985.
Path 189 | total_timesteps 1993.
Path 190 | total_timesteps 2003.
Path 191 | total_timesteps 2011.
Path 192 | total_timesteps 2026.
Path 193 | total_timesteps 2036.
Path 194 | total_timesteps 2044.
Path 195 | total_timesteps 2058.
Path 196 | total_timesteps 2065.
Path 197 | total_timesteps 2074.
Path 198 | total_timesteps 2082.
Path 199 | total_timesteps 2091.
Path 200 | total_timesteps 2102.
Path 201 | total_timesteps 2113.
Path 202 | total_timesteps 2121.
Path 203 | total_timesteps 2130.
Path 204 | total_timesteps 2140.
Path 205 | total_timesteps 2148.
Path 206 | total_timesteps 2156.
Path 207 | total_timesteps 2162.
Path 208 | total_timesteps 2173.
Path 209 | total_timesteps 2180.
Path 210 | total_timesteps 2188.
Path 211 | total_timesteps 2195.
Path 212 | total_timesteps 2202.
Path 213 | total_timesteps 2214.
Path 214 | total_timesteps 2231.
Path 215 | total_timesteps 2245.
Path 216 | total_timesteps 2258.
Path 217 | total_timesteps 2265.
Path 218 | total_timesteps 2276.
Path 219 | total_timesteps 2295.
Path 220 | total_timesteps 2305.
Path 221 | total_timesteps 2316.
Path 222 | total_timesteps 2326.
Path 223 | total_timesteps 2341.
Path 224 | total_timesteps 2351.
Path 225 | total_timesteps 2358.
Path 226 | total_timesteps 2372.
Path 227 | total_timesteps 2382.
Path 228 | total_timesteps 2391.
Path 229 | total_timesteps 2402.
Path 230 | total_timesteps 2411.
Path 231 | total_timesteps 2420.
Path 232 | total_timesteps 2428.
Path 233 | total_timesteps 2448.
Path 234 | total_timesteps 2463.
Path 235 | total_timesteps 2473.
Path 236 | total_timesteps 2480.
Path 237 | total_timesteps 2495.
Path 238 | total_timesteps 2503.
Path 239 | total_timesteps 2515.
Path 240 | total_timesteps 2523.
Path 241 | total_timesteps 2532.
Path 242 | total_timesteps 2545.
Path 243 | total_timesteps 2554.
Path 244 | total_timesteps 2565.
Path 245 | total_timesteps 2576.
Path 246 | total_timesteps 2585.
Path 247 | total_timesteps 2592.
Path 248 | total_timesteps 2606.
Path 249 | total_timesteps 2614.
Path 250 | total_timesteps 2621.
Path 251 | total_timesteps 2643.
Path 252 | total_timesteps 2654.
Path 253 | total_timesteps 2668.
Path 254 | total_timesteps 2691.
Path 255 | total_timesteps 2699.
Path 256 | total_timesteps 2713.
Path 257 | total_timesteps 2729.
Path 258 | total_timesteps 2738.
Path 259 | total_timesteps 2749.
Path 260 | total_timesteps 2756.
Path 261 | total_timesteps 2763.
Path 262 | total_timesteps 2770.
Path 263 | total_timesteps 2777.
Path 264 | total_timesteps 2786.
Path 265 | total_timesteps 2797.
Path 266 | total_timesteps 2805.
Path 267 | total_timesteps 2814.
Path 268 | total_timesteps 2823.
Path 269 | total_timesteps 2833.
Path 270 | total_timesteps 2845.
Path 271 | total_timesteps 2858.
Path 272 | total_timesteps 2867.
Path 273 | total_timesteps 2881.
Path 274 | total_timesteps 2893.
Path 275 | total_timesteps 2903.
Path 276 | total_timesteps 2920.
Path 277 | total_timesteps 2928.
Path 278 | total_timesteps 2939.
Path 279 | total_timesteps 2946.
Path 280 | total_timesteps 2954.
Path 281 | total_timesteps 2962.
Path 282 | total_timesteps 2974.
Path 283 | total_timesteps 2984.
Path 284 | total_timesteps 2992.
Path 285 | total_timesteps 3000.
Path 286 | total_timesteps 3012.
Path 287 | total_timesteps 3019.
Path 288 | total_timesteps 3040.
Path 289 | total_timesteps 3052.
Path 290 | total_timesteps 3062.
Path 291 | total_timesteps 3069.
Path 292 | total_timesteps 3076.
Path 293 | total_timesteps 3091.
Path 294 | total_timesteps 3097.
Path 295 | total_timesteps 3105.
Path 296 | total_timesteps 3116.
Path 297 | total_timesteps 3126.
Path 298 | total_timesteps 3134.
Path 299 | total_timesteps 3148.
Path 300 | total_timesteps 3156.
Path 301 | total_timesteps 3164.
Path 302 | total_timesteps 3175.
Path 303 | total_timesteps 3189.
Path 304 | total_timesteps 3203.
Path 305 | total_timesteps 3212.
Path 306 | total_timesteps 3224.
Path 307 | total_timesteps 3235.
Path 308 | total_timesteps 3244.
Path 309 | total_timesteps 3251.
Path 310 | total_timesteps 3259.
Path 311 | total_timesteps 3267.
Path 312 | total_timesteps 3278.
Path 313 | total_timesteps 3286.
Path 314 | total_timesteps 3294.
Path 315 | total_timesteps 3301.
Path 316 | total_timesteps 3309.
Path 317 | total_timesteps 3317.
Path 318 | total_timesteps 3325.
Path 319 | total_timesteps 3335.
Path 320 | total_timesteps 3347.
Path 321 | total_timesteps 3355.
Path 322 | total_timesteps 3365.
Path 323 | total_timesteps 3377.
Path 324 | total_timesteps 3387.
Path 325 | total_timesteps 3395.
Path 326 | total_timesteps 3409.
Path 327 | total_timesteps 3434.
Path 328 | total_timesteps 3446.
Path 329 | total_timesteps 3453.
Path 330 | total_timesteps 3459.
Path 331 | total_timesteps 3471.
Path 332 | total_timesteps 3482.
Path 333 | total_timesteps 3492.
Path 334 | total_timesteps 3500.
Path 335 | total_timesteps 3511.
Path 336 | total_timesteps 3524.
Path 337 | total_timesteps 3533.
Path 338 | total_timesteps 3542.
Path 339 | total_timesteps 3553.
Path 340 | total_timesteps 3560.
Path 341 | total_timesteps 3572.
Path 342 | total_timesteps 3585.
Path 343 | total_timesteps 3597.
Path 344 | total_timesteps 3615.
Path 345 | total_timesteps 3632.
Path 346 | total_timesteps 3638.
Path 347 | total_timesteps 3646.
Path 348 | total_timesteps 3653.
Path 349 | total_timesteps 3666.
Path 350 | total_timesteps 3680.
Path 351 | total_timesteps 3689.
Path 352 | total_timesteps 3700.
Path 353 | total_timesteps 3712.
Path 354 | total_timesteps 3726.
Path 355 | total_timesteps 3737.
Path 356 | total_timesteps 3752.
Path 357 | total_timesteps 3763.
Path 358 | total_timesteps 3774.
Path 359 | total_timesteps 3783.
Path 360 | total_timesteps 3791.
Path 361 | total_timesteps 3799.
Path 362 | total_timesteps 3810.
Path 363 | total_timesteps 3819.
Path 364 | total_timesteps 3827.
Path 365 | total_timesteps 3840.
Path 366 | total_timesteps 3851.
Path 367 | total_timesteps 3862.
Path 368 | total_timesteps 3870.
Path 369 | total_timesteps 3879.
Path 370 | total_timesteps 3886.
Path 371 | total_timesteps 3900.
Path 372 | total_timesteps 3908.
Path 373 | total_timesteps 3918.
Path 374 | total_timesteps 3928.
Path 375 | total_timesteps 3944.
Path 376 | total_timesteps 3951.
Path 377 | total_timesteps 3964.
Path 378 | total_timesteps 3979.
Path 379 | total_timesteps 3991.
Path 380 | total_timesteps 4003.
Path 381 | total_timesteps 4021.
Path 382 | total_timesteps 4035.
Path 383 | total_timesteps 4043.
Path 384 | total_timesteps 4050.
Path 385 | total_timesteps 4059.
Path 386 | total_timesteps 4069.
Path 387 | total_timesteps 4078.
Path 388 | total_timesteps 4090.
Path 389 | total_timesteps 4100.
Path 390 | total_timesteps 4108.
Path 391 | total_timesteps 4122.
Path 392 | total_timesteps 4130.
Path 393 | total_timesteps 4139.
Path 394 | total_timesteps 4148.
Path 395 | total_timesteps 4162.
Path 396 | total_timesteps 4177.
Path 397 | total_timesteps 4185.
Path 398 | total_timesteps 4194.
Path 399 | total_timesteps 4212.
Path 400 | total_timesteps 4222.
Path 401 | total_timesteps 4232.
Path 402 | total_timesteps 4242.
Path 403 | total_timesteps 4253.
Path 404 | total_timesteps 4260.
Path 405 | total_timesteps 4272.
Path 406 | total_timesteps 4283.
Path 407 | total_timesteps 4290.
Path 408 | total_timesteps 4297.
Path 409 | total_timesteps 4308.
Path 410 | total_timesteps 4317.
Path 411 | total_timesteps 4326.
Path 412 | total_timesteps 4333.
Path 413 | total_timesteps 4342.
Path 414 | total_timesteps 4349.
Path 415 | total_timesteps 4356.
Path 416 | total_timesteps 4368.
Path 417 | total_timesteps 4376.
Path 418 | total_timesteps 4384.
Path 419 | total_timesteps 4393.
Path 420 | total_timesteps 4402.
Path 421 | total_timesteps 4412.
Path 422 | total_timesteps 4427.
Path 423 | total_timesteps 4436.
Path 424 | total_timesteps 4445.
Path 425 | total_timesteps 4461.
Path 426 | total_timesteps 4470.
Path 427 | total_timesteps 4481.
Path 428 | total_timesteps 4501.
Path 429 | total_timesteps 4513.
Path 430 | total_timesteps 4523.
Path 431 | total_timesteps 4538.
Path 432 | total_timesteps 4559.
Path 433 | total_timesteps 4570.
Path 434 | total_timesteps 4581.
Path 435 | total_timesteps 4595.
Path 436 | total_timesteps 4601.
Path 437 | total_timesteps 4615.
Path 438 | total_timesteps 4627.
Path 439 | total_timesteps 4638.
Path 440 | total_timesteps 4646.
Path 441 | total_timesteps 4653.
Path 442 | total_timesteps 4663.
Path 443 | total_timesteps 4673.
Path 444 | total_timesteps 4681.
Path 445 | total_timesteps 4689.
Path 446 | total_timesteps 4705.
Path 447 | total_timesteps 4715.
Path 448 | total_timesteps 4724.
Path 449 | total_timesteps 4743.
Path 450 | total_timesteps 4756.
Path 451 | total_timesteps 4767.
Path 452 | total_timesteps 4783.
Path 453 | total_timesteps 4790.
Path 454 | total_timesteps 4800.
Path 455 | total_timesteps 4808.
Path 456 | total_timesteps 4828.
Path 457 | total_timesteps 4839.
Path 458 | total_timesteps 4849.
Path 459 | total_timesteps 4862.
Path 460 | total_timesteps 4870.
Path 461 | total_timesteps 4882.
Path 462 | total_timesteps 4892.
Path 463 | total_timesteps 4899.
Path 464 | total_timesteps 4907.
Path 465 | total_timesteps 4934.
Path 466 | total_timesteps 4941.
Path 467 | total_timesteps 4950.
Path 468 | total_timesteps 4959.
Path 469 | total_timesteps 4971.
Path 470 | total_timesteps 4979.
Path 471 | total_timesteps 4986.
Path 472 | total_timesteps 4992.
Path 473 | total_timesteps 5015.
Path 474 | total_timesteps 5023.
Path 475 | total_timesteps 5031.
Path 476 | total_timesteps 5040.
Path 477 | total_timesteps 5048.
Path 478 | total_timesteps 5061.
Path 479 | total_timesteps 5069.
Path 480 | total_timesteps 5076.
Path 481 | total_timesteps 5087.
Path 482 | total_timesteps 5099.
Path 483 | total_timesteps 5111.
Path 484 | total_timesteps 5119.
Path 485 | total_timesteps 5128.
Path 486 | total_timesteps 5134.
Path 487 | total_timesteps 5144.
Path 488 | total_timesteps 5157.
Path 489 | total_timesteps 5165.
Path 490 | total_timesteps 5173.
Path 491 | total_timesteps 5185.
Path 492 | total_timesteps 5196.
Path 493 | total_timesteps 5206.
Path 494 | total_timesteps 5212.
Path 495 | total_timesteps 5229.
Path 496 | total_timesteps 5245.
Path 497 | total_timesteps 5253.
Path 498 | total_timesteps 5269.
Path 499 | total_timesteps 5283.
Path 500 | total_timesteps 5290.
Path 501 | total_timesteps 5298.
Path 502 | total_timesteps 5310.
Path 503 | total_timesteps 5318.
Path 504 | total_timesteps 5328.
Path 505 | total_timesteps 5342.
Path 506 | total_timesteps 5352.
Path 507 | total_timesteps 5370.
Path 508 | total_timesteps 5380.
Path 509 | total_timesteps 5397.
Path 510 | total_timesteps 5406.
Path 511 | total_timesteps 5417.
Path 512 | total_timesteps 5431.
Path 513 | total_timesteps 5439.
Path 514 | total_timesteps 5449.
Path 515 | total_timesteps 5456.
Path 516 | total_timesteps 5471.
Path 517 | total_timesteps 5481.
Path 518 | total_timesteps 5493.
Path 519 | total_timesteps 5500.
Path 520 | total_timesteps 5507.
Path 521 | total_timesteps 5517.
Path 522 | total_timesteps 5529.
Path 523 | total_timesteps 5538.
Path 524 | total_timesteps 5547.
Path 525 | total_timesteps 5555.
Path 526 | total_timesteps 5566.
Path 527 | total_timesteps 5580.
Path 528 | total_timesteps 5592.
Path 529 | total_timesteps 5600.
Path 530 | total_timesteps 5617.
Path 531 | total_timesteps 5627.
Path 532 | total_timesteps 5634.
Path 533 | total_timesteps 5645.
Path 534 | total_timesteps 5655.
Path 535 | total_timesteps 5663.
Path 536 | total_timesteps 5669.
Path 537 | total_timesteps 5681.
Path 538 | total_timesteps 5692.
Path 539 | total_timesteps 5703.
Path 540 | total_timesteps 5716.
Path 541 | total_timesteps 5726.
Path 542 | total_timesteps 5745.
Path 543 | total_timesteps 5755.
Path 544 | total_timesteps 5762.
Path 545 | total_timesteps 5776.
Path 546 | total_timesteps 5785.
Path 547 | total_timesteps 5796.
Path 548 | total_timesteps 5805.
Path 549 | total_timesteps 5813.
Path 550 | total_timesteps 5822.
Path 551 | total_timesteps 5829.
Path 552 | total_timesteps 5846.
Path 553 | total_timesteps 5857.
Path 554 | total_timesteps 5867.
Path 555 | total_timesteps 5882.
Path 556 | total_timesteps 5895.
Path 557 | total_timesteps 5902.
Path 558 | total_timesteps 5909.
Path 559 | total_timesteps 5920.
Path 560 | total_timesteps 5928.
Path 561 | total_timesteps 5936.
Path 562 | total_timesteps 5948.
Path 563 | total_timesteps 5955.
Path 564 | total_timesteps 5967.
Path 565 | total_timesteps 5977.
Path 566 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.2     |
| Iteration     | 26       |
| MaximumReturn | 0.286    |
| MinimumReturn | -20.4    |
| TotalSamples  | 112202   |
----------------------------
itr #27 | 
Fitting dynamics.
Validation loss = 0.0023344617802649736
Validation loss = 0.0022094722371548414
Validation loss = 0.0022190262097865343
Validation loss = 0.002232916187494993
Validation loss = 0.0023935481440275908
Validation loss = 0.002357250778004527
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 15.
Path 2 | total_timesteps 26.
Path 3 | total_timesteps 43.
Path 4 | total_timesteps 56.
Path 5 | total_timesteps 68.
Path 6 | total_timesteps 75.
Path 7 | total_timesteps 81.
Path 8 | total_timesteps 87.
Path 9 | total_timesteps 95.
Path 10 | total_timesteps 104.
Path 11 | total_timesteps 113.
Path 12 | total_timesteps 122.
Path 13 | total_timesteps 133.
Path 14 | total_timesteps 142.
Path 15 | total_timesteps 152.
Path 16 | total_timesteps 163.
Path 17 | total_timesteps 171.
Path 18 | total_timesteps 182.
Path 19 | total_timesteps 193.
Path 20 | total_timesteps 205.
Path 21 | total_timesteps 212.
Path 22 | total_timesteps 219.
Path 23 | total_timesteps 240.
Path 24 | total_timesteps 253.
Path 25 | total_timesteps 260.
Path 26 | total_timesteps 266.
Path 27 | total_timesteps 274.
Path 28 | total_timesteps 285.
Path 29 | total_timesteps 307.
Path 30 | total_timesteps 317.
Path 31 | total_timesteps 323.
Path 32 | total_timesteps 337.
Path 33 | total_timesteps 345.
Path 34 | total_timesteps 352.
Path 35 | total_timesteps 367.
Path 36 | total_timesteps 378.
Path 37 | total_timesteps 388.
Path 38 | total_timesteps 397.
Path 39 | total_timesteps 403.
Path 40 | total_timesteps 414.
Path 41 | total_timesteps 421.
Path 42 | total_timesteps 435.
Path 43 | total_timesteps 443.
Path 44 | total_timesteps 452.
Path 45 | total_timesteps 464.
Path 46 | total_timesteps 477.
Path 47 | total_timesteps 490.
Path 48 | total_timesteps 497.
Path 49 | total_timesteps 507.
Path 50 | total_timesteps 518.
Path 51 | total_timesteps 526.
Path 52 | total_timesteps 534.
Path 53 | total_timesteps 546.
Path 54 | total_timesteps 558.
Path 55 | total_timesteps 567.
Path 56 | total_timesteps 576.
Path 57 | total_timesteps 588.
Path 58 | total_timesteps 596.
Path 59 | total_timesteps 608.
Path 60 | total_timesteps 619.
Path 61 | total_timesteps 627.
Path 62 | total_timesteps 643.
Path 63 | total_timesteps 653.
Path 64 | total_timesteps 665.
Path 65 | total_timesteps 672.
Path 66 | total_timesteps 678.
Path 67 | total_timesteps 687.
Path 68 | total_timesteps 698.
Path 69 | total_timesteps 714.
Path 70 | total_timesteps 724.
Path 71 | total_timesteps 736.
Path 72 | total_timesteps 748.
Path 73 | total_timesteps 755.
Path 74 | total_timesteps 762.
Path 75 | total_timesteps 769.
Path 76 | total_timesteps 778.
Path 77 | total_timesteps 789.
Path 78 | total_timesteps 797.
Path 79 | total_timesteps 807.
Path 80 | total_timesteps 813.
Path 81 | total_timesteps 823.
Path 82 | total_timesteps 838.
Path 83 | total_timesteps 850.
Path 84 | total_timesteps 858.
Path 85 | total_timesteps 872.
Path 86 | total_timesteps 879.
Path 87 | total_timesteps 889.
Path 88 | total_timesteps 901.
Path 89 | total_timesteps 909.
Path 90 | total_timesteps 921.
Path 91 | total_timesteps 928.
Path 92 | total_timesteps 936.
Path 93 | total_timesteps 943.
Path 94 | total_timesteps 950.
Path 95 | total_timesteps 959.
Path 96 | total_timesteps 965.
Path 97 | total_timesteps 974.
Path 98 | total_timesteps 982.
Path 99 | total_timesteps 989.
Path 100 | total_timesteps 1003.
Path 101 | total_timesteps 1015.
Path 102 | total_timesteps 1023.
Path 103 | total_timesteps 1029.
Path 104 | total_timesteps 1037.
Path 105 | total_timesteps 1046.
Path 106 | total_timesteps 1055.
Path 107 | total_timesteps 1068.
Path 108 | total_timesteps 1077.
Path 109 | total_timesteps 1088.
Path 110 | total_timesteps 1107.
Path 111 | total_timesteps 1118.
Path 112 | total_timesteps 1129.
Path 113 | total_timesteps 1137.
Path 114 | total_timesteps 1145.
Path 115 | total_timesteps 1162.
Path 116 | total_timesteps 1170.
Path 117 | total_timesteps 1177.
Path 118 | total_timesteps 1187.
Path 119 | total_timesteps 1197.
Path 120 | total_timesteps 1208.
Path 121 | total_timesteps 1215.
Path 122 | total_timesteps 1225.
Path 123 | total_timesteps 1232.
Path 124 | total_timesteps 1239.
Path 125 | total_timesteps 1253.
Path 126 | total_timesteps 1262.
Path 127 | total_timesteps 1273.
Path 128 | total_timesteps 1289.
Path 129 | total_timesteps 1297.
Path 130 | total_timesteps 1304.
Path 131 | total_timesteps 1315.
Path 132 | total_timesteps 1325.
Path 133 | total_timesteps 1333.
Path 134 | total_timesteps 1340.
Path 135 | total_timesteps 1351.
Path 136 | total_timesteps 1362.
Path 137 | total_timesteps 1369.
Path 138 | total_timesteps 1380.
Path 139 | total_timesteps 1389.
Path 140 | total_timesteps 1397.
Path 141 | total_timesteps 1404.
Path 142 | total_timesteps 1416.
Path 143 | total_timesteps 1422.
Path 144 | total_timesteps 1432.
Path 145 | total_timesteps 1441.
Path 146 | total_timesteps 1458.
Path 147 | total_timesteps 1471.
Path 148 | total_timesteps 1482.
Path 149 | total_timesteps 1490.
Path 150 | total_timesteps 1500.
Path 151 | total_timesteps 1509.
Path 152 | total_timesteps 1520.
Path 153 | total_timesteps 1530.
Path 154 | total_timesteps 1541.
Path 155 | total_timesteps 1548.
Path 156 | total_timesteps 1570.
Path 157 | total_timesteps 1584.
Path 158 | total_timesteps 1600.
Path 159 | total_timesteps 1607.
Path 160 | total_timesteps 1615.
Path 161 | total_timesteps 1622.
Path 162 | total_timesteps 1632.
Path 163 | total_timesteps 1645.
Path 164 | total_timesteps 1652.
Path 165 | total_timesteps 1664.
Path 166 | total_timesteps 1673.
Path 167 | total_timesteps 1681.
Path 168 | total_timesteps 1691.
Path 169 | total_timesteps 1699.
Path 170 | total_timesteps 1710.
Path 171 | total_timesteps 1721.
Path 172 | total_timesteps 1729.
Path 173 | total_timesteps 1736.
Path 174 | total_timesteps 1746.
Path 175 | total_timesteps 1753.
Path 176 | total_timesteps 1763.
Path 177 | total_timesteps 1771.
Path 178 | total_timesteps 1782.
Path 179 | total_timesteps 1801.
Path 180 | total_timesteps 1811.
Path 181 | total_timesteps 1822.
Path 182 | total_timesteps 1840.
Path 183 | total_timesteps 1847.
Path 184 | total_timesteps 1857.
Path 185 | total_timesteps 1863.
Path 186 | total_timesteps 1870.
Path 187 | total_timesteps 1880.
Path 188 | total_timesteps 1889.
Path 189 | total_timesteps 1904.
Path 190 | total_timesteps 1914.
Path 191 | total_timesteps 1924.
Path 192 | total_timesteps 1936.
Path 193 | total_timesteps 1946.
Path 194 | total_timesteps 1960.
Path 195 | total_timesteps 1970.
Path 196 | total_timesteps 1988.
Path 197 | total_timesteps 2005.
Path 198 | total_timesteps 2014.
Path 199 | total_timesteps 2021.
Path 200 | total_timesteps 2029.
Path 201 | total_timesteps 2038.
Path 202 | total_timesteps 2052.
Path 203 | total_timesteps 2063.
Path 204 | total_timesteps 2070.
Path 205 | total_timesteps 2080.
Path 206 | total_timesteps 2088.
Path 207 | total_timesteps 2099.
Path 208 | total_timesteps 2109.
Path 209 | total_timesteps 2122.
Path 210 | total_timesteps 2131.
Path 211 | total_timesteps 2140.
Path 212 | total_timesteps 2148.
Path 213 | total_timesteps 2162.
Path 214 | total_timesteps 2179.
Path 215 | total_timesteps 2187.
Path 216 | total_timesteps 2193.
Path 217 | total_timesteps 2204.
Path 218 | total_timesteps 2216.
Path 219 | total_timesteps 2229.
Path 220 | total_timesteps 2242.
Path 221 | total_timesteps 2249.
Path 222 | total_timesteps 2257.
Path 223 | total_timesteps 2272.
Path 224 | total_timesteps 2278.
Path 225 | total_timesteps 2286.
Path 226 | total_timesteps 2298.
Path 227 | total_timesteps 2307.
Path 228 | total_timesteps 2322.
Path 229 | total_timesteps 2331.
Path 230 | total_timesteps 2349.
Path 231 | total_timesteps 2360.
Path 232 | total_timesteps 2373.
Path 233 | total_timesteps 2385.
Path 234 | total_timesteps 2392.
Path 235 | total_timesteps 2400.
Path 236 | total_timesteps 2408.
Path 237 | total_timesteps 2422.
Path 238 | total_timesteps 2431.
Path 239 | total_timesteps 2440.
Path 240 | total_timesteps 2448.
Path 241 | total_timesteps 2455.
Path 242 | total_timesteps 2465.
Path 243 | total_timesteps 2482.
Path 244 | total_timesteps 2490.
Path 245 | total_timesteps 2502.
Path 246 | total_timesteps 2512.
Path 247 | total_timesteps 2518.
Path 248 | total_timesteps 2534.
Path 249 | total_timesteps 2543.
Path 250 | total_timesteps 2551.
Path 251 | total_timesteps 2558.
Path 252 | total_timesteps 2567.
Path 253 | total_timesteps 2575.
Path 254 | total_timesteps 2583.
Path 255 | total_timesteps 2602.
Path 256 | total_timesteps 2613.
Path 257 | total_timesteps 2620.
Path 258 | total_timesteps 2636.
Path 259 | total_timesteps 2644.
Path 260 | total_timesteps 2652.
Path 261 | total_timesteps 2659.
Path 262 | total_timesteps 2669.
Path 263 | total_timesteps 2679.
Path 264 | total_timesteps 2692.
Path 265 | total_timesteps 2700.
Path 266 | total_timesteps 2710.
Path 267 | total_timesteps 2718.
Path 268 | total_timesteps 2726.
Path 269 | total_timesteps 2739.
Path 270 | total_timesteps 2747.
Path 271 | total_timesteps 2755.
Path 272 | total_timesteps 2763.
Path 273 | total_timesteps 2771.
Path 274 | total_timesteps 2779.
Path 275 | total_timesteps 2793.
Path 276 | total_timesteps 2801.
Path 277 | total_timesteps 2811.
Path 278 | total_timesteps 2819.
Path 279 | total_timesteps 2827.
Path 280 | total_timesteps 2837.
Path 281 | total_timesteps 2843.
Path 282 | total_timesteps 2851.
Path 283 | total_timesteps 2861.
Path 284 | total_timesteps 2869.
Path 285 | total_timesteps 2878.
Path 286 | total_timesteps 2889.
Path 287 | total_timesteps 2898.
Path 288 | total_timesteps 2914.
Path 289 | total_timesteps 2921.
Path 290 | total_timesteps 2928.
Path 291 | total_timesteps 2938.
Path 292 | total_timesteps 2953.
Path 293 | total_timesteps 2960.
Path 294 | total_timesteps 2973.
Path 295 | total_timesteps 2982.
Path 296 | total_timesteps 2994.
Path 297 | total_timesteps 3008.
Path 298 | total_timesteps 3023.
Path 299 | total_timesteps 3031.
Path 300 | total_timesteps 3042.
Path 301 | total_timesteps 3048.
Path 302 | total_timesteps 3056.
Path 303 | total_timesteps 3066.
Path 304 | total_timesteps 3077.
Path 305 | total_timesteps 3084.
Path 306 | total_timesteps 3091.
Path 307 | total_timesteps 3105.
Path 308 | total_timesteps 3115.
Path 309 | total_timesteps 3123.
Path 310 | total_timesteps 3131.
Path 311 | total_timesteps 3140.
Path 312 | total_timesteps 3148.
Path 313 | total_timesteps 3155.
Path 314 | total_timesteps 3164.
Path 315 | total_timesteps 3175.
Path 316 | total_timesteps 3186.
Path 317 | total_timesteps 3199.
Path 318 | total_timesteps 3209.
Path 319 | total_timesteps 3221.
Path 320 | total_timesteps 3229.
Path 321 | total_timesteps 3243.
Path 322 | total_timesteps 3253.
Path 323 | total_timesteps 3262.
Path 324 | total_timesteps 3270.
Path 325 | total_timesteps 3281.
Path 326 | total_timesteps 3291.
Path 327 | total_timesteps 3303.
Path 328 | total_timesteps 3317.
Path 329 | total_timesteps 3335.
Path 330 | total_timesteps 3342.
Path 331 | total_timesteps 3350.
Path 332 | total_timesteps 3357.
Path 333 | total_timesteps 3367.
Path 334 | total_timesteps 3374.
Path 335 | total_timesteps 3382.
Path 336 | total_timesteps 3392.
Path 337 | total_timesteps 3400.
Path 338 | total_timesteps 3407.
Path 339 | total_timesteps 3414.
Path 340 | total_timesteps 3421.
Path 341 | total_timesteps 3436.
Path 342 | total_timesteps 3447.
Path 343 | total_timesteps 3455.
Path 344 | total_timesteps 3464.
Path 345 | total_timesteps 3474.
Path 346 | total_timesteps 3483.
Path 347 | total_timesteps 3494.
Path 348 | total_timesteps 3506.
Path 349 | total_timesteps 3514.
Path 350 | total_timesteps 3528.
Path 351 | total_timesteps 3536.
Path 352 | total_timesteps 3544.
Path 353 | total_timesteps 3555.
Path 354 | total_timesteps 3566.
Path 355 | total_timesteps 3582.
Path 356 | total_timesteps 3598.
Path 357 | total_timesteps 3610.
Path 358 | total_timesteps 3626.
Path 359 | total_timesteps 3635.
Path 360 | total_timesteps 3644.
Path 361 | total_timesteps 3656.
Path 362 | total_timesteps 3666.
Path 363 | total_timesteps 3677.
Path 364 | total_timesteps 3685.
Path 365 | total_timesteps 3706.
Path 366 | total_timesteps 3715.
Path 367 | total_timesteps 3728.
Path 368 | total_timesteps 3737.
Path 369 | total_timesteps 3750.
Path 370 | total_timesteps 3765.
Path 371 | total_timesteps 3777.
Path 372 | total_timesteps 3786.
Path 373 | total_timesteps 3808.
Path 374 | total_timesteps 3819.
Path 375 | total_timesteps 3826.
Path 376 | total_timesteps 3832.
Path 377 | total_timesteps 3846.
Path 378 | total_timesteps 3852.
Path 379 | total_timesteps 3865.
Path 380 | total_timesteps 3875.
Path 381 | total_timesteps 3885.
Path 382 | total_timesteps 3892.
Path 383 | total_timesteps 3904.
Path 384 | total_timesteps 3914.
Path 385 | total_timesteps 3923.
Path 386 | total_timesteps 3931.
Path 387 | total_timesteps 3937.
Path 388 | total_timesteps 3947.
Path 389 | total_timesteps 3953.
Path 390 | total_timesteps 3971.
Path 391 | total_timesteps 3983.
Path 392 | total_timesteps 3993.
Path 393 | total_timesteps 4000.
Path 394 | total_timesteps 4010.
Path 395 | total_timesteps 4019.
Path 396 | total_timesteps 4025.
Path 397 | total_timesteps 4033.
Path 398 | total_timesteps 4040.
Path 399 | total_timesteps 4047.
Path 400 | total_timesteps 4057.
Path 401 | total_timesteps 4067.
Path 402 | total_timesteps 4083.
Path 403 | total_timesteps 4091.
Path 404 | total_timesteps 4097.
Path 405 | total_timesteps 4104.
Path 406 | total_timesteps 4120.
Path 407 | total_timesteps 4130.
Path 408 | total_timesteps 4140.
Path 409 | total_timesteps 4147.
Path 410 | total_timesteps 4153.
Path 411 | total_timesteps 4160.
Path 412 | total_timesteps 4169.
Path 413 | total_timesteps 4177.
Path 414 | total_timesteps 4189.
Path 415 | total_timesteps 4198.
Path 416 | total_timesteps 4207.
Path 417 | total_timesteps 4214.
Path 418 | total_timesteps 4224.
Path 419 | total_timesteps 4236.
Path 420 | total_timesteps 4249.
Path 421 | total_timesteps 4257.
Path 422 | total_timesteps 4274.
Path 423 | total_timesteps 4281.
Path 424 | total_timesteps 4291.
Path 425 | total_timesteps 4298.
Path 426 | total_timesteps 4306.
Path 427 | total_timesteps 4312.
Path 428 | total_timesteps 4324.
Path 429 | total_timesteps 4332.
Path 430 | total_timesteps 4341.
Path 431 | total_timesteps 4349.
Path 432 | total_timesteps 4357.
Path 433 | total_timesteps 4367.
Path 434 | total_timesteps 4375.
Path 435 | total_timesteps 4391.
Path 436 | total_timesteps 4403.
Path 437 | total_timesteps 4411.
Path 438 | total_timesteps 4420.
Path 439 | total_timesteps 4430.
Path 440 | total_timesteps 4437.
Path 441 | total_timesteps 4450.
Path 442 | total_timesteps 4460.
Path 443 | total_timesteps 4482.
Path 444 | total_timesteps 4489.
Path 445 | total_timesteps 4498.
Path 446 | total_timesteps 4511.
Path 447 | total_timesteps 4520.
Path 448 | total_timesteps 4537.
Path 449 | total_timesteps 4544.
Path 450 | total_timesteps 4555.
Path 451 | total_timesteps 4564.
Path 452 | total_timesteps 4574.
Path 453 | total_timesteps 4583.
Path 454 | total_timesteps 4592.
Path 455 | total_timesteps 4603.
Path 456 | total_timesteps 4616.
Path 457 | total_timesteps 4625.
Path 458 | total_timesteps 4635.
Path 459 | total_timesteps 4648.
Path 460 | total_timesteps 4658.
Path 461 | total_timesteps 4666.
Path 462 | total_timesteps 4678.
Path 463 | total_timesteps 4686.
Path 464 | total_timesteps 4694.
Path 465 | total_timesteps 4705.
Path 466 | total_timesteps 4713.
Path 467 | total_timesteps 4724.
Path 468 | total_timesteps 4731.
Path 469 | total_timesteps 4737.
Path 470 | total_timesteps 4745.
Path 471 | total_timesteps 4756.
Path 472 | total_timesteps 4765.
Path 473 | total_timesteps 4773.
Path 474 | total_timesteps 4786.
Path 475 | total_timesteps 4793.
Path 476 | total_timesteps 4811.
Path 477 | total_timesteps 4827.
Path 478 | total_timesteps 4839.
Path 479 | total_timesteps 4849.
Path 480 | total_timesteps 4862.
Path 481 | total_timesteps 4876.
Path 482 | total_timesteps 4882.
Path 483 | total_timesteps 4891.
Path 484 | total_timesteps 4898.
Path 485 | total_timesteps 4907.
Path 486 | total_timesteps 4915.
Path 487 | total_timesteps 4924.
Path 488 | total_timesteps 4933.
Path 489 | total_timesteps 4945.
Path 490 | total_timesteps 4956.
Path 491 | total_timesteps 4965.
Path 492 | total_timesteps 4973.
Path 493 | total_timesteps 4982.
Path 494 | total_timesteps 4990.
Path 495 | total_timesteps 4998.
Path 496 | total_timesteps 5004.
Path 497 | total_timesteps 5012.
Path 498 | total_timesteps 5018.
Path 499 | total_timesteps 5030.
Path 500 | total_timesteps 5040.
Path 501 | total_timesteps 5047.
Path 502 | total_timesteps 5056.
Path 503 | total_timesteps 5067.
Path 504 | total_timesteps 5077.
Path 505 | total_timesteps 5086.
Path 506 | total_timesteps 5098.
Path 507 | total_timesteps 5109.
Path 508 | total_timesteps 5122.
Path 509 | total_timesteps 5132.
Path 510 | total_timesteps 5144.
Path 511 | total_timesteps 5152.
Path 512 | total_timesteps 5158.
Path 513 | total_timesteps 5172.
Path 514 | total_timesteps 5182.
Path 515 | total_timesteps 5189.
Path 516 | total_timesteps 5202.
Path 517 | total_timesteps 5209.
Path 518 | total_timesteps 5218.
Path 519 | total_timesteps 5228.
Path 520 | total_timesteps 5243.
Path 521 | total_timesteps 5251.
Path 522 | total_timesteps 5261.
Path 523 | total_timesteps 5269.
Path 524 | total_timesteps 5284.
Path 525 | total_timesteps 5295.
Path 526 | total_timesteps 5302.
Path 527 | total_timesteps 5309.
Path 528 | total_timesteps 5321.
Path 529 | total_timesteps 5334.
Path 530 | total_timesteps 5344.
Path 531 | total_timesteps 5351.
Path 532 | total_timesteps 5362.
Path 533 | total_timesteps 5370.
Path 534 | total_timesteps 5379.
Path 535 | total_timesteps 5396.
Path 536 | total_timesteps 5412.
Path 537 | total_timesteps 5426.
Path 538 | total_timesteps 5435.
Path 539 | total_timesteps 5447.
Path 540 | total_timesteps 5453.
Path 541 | total_timesteps 5463.
Path 542 | total_timesteps 5472.
Path 543 | total_timesteps 5480.
Path 544 | total_timesteps 5490.
Path 545 | total_timesteps 5499.
Path 546 | total_timesteps 5505.
Path 547 | total_timesteps 5514.
Path 548 | total_timesteps 5524.
Path 549 | total_timesteps 5531.
Path 550 | total_timesteps 5538.
Path 551 | total_timesteps 5548.
Path 552 | total_timesteps 5557.
Path 553 | total_timesteps 5565.
Path 554 | total_timesteps 5578.
Path 555 | total_timesteps 5586.
Path 556 | total_timesteps 5592.
Path 557 | total_timesteps 5601.
Path 558 | total_timesteps 5607.
Path 559 | total_timesteps 5617.
Path 560 | total_timesteps 5632.
Path 561 | total_timesteps 5642.
Path 562 | total_timesteps 5651.
Path 563 | total_timesteps 5663.
Path 564 | total_timesteps 5671.
Path 565 | total_timesteps 5688.
Path 566 | total_timesteps 5695.
Path 567 | total_timesteps 5705.
Path 568 | total_timesteps 5718.
Path 569 | total_timesteps 5728.
Path 570 | total_timesteps 5743.
Path 571 | total_timesteps 5753.
Path 572 | total_timesteps 5760.
Path 573 | total_timesteps 5768.
Path 574 | total_timesteps 5778.
Path 575 | total_timesteps 5787.
Path 576 | total_timesteps 5796.
Path 577 | total_timesteps 5804.
Path 578 | total_timesteps 5813.
Path 579 | total_timesteps 5821.
Path 580 | total_timesteps 5829.
Path 581 | total_timesteps 5836.
Path 582 | total_timesteps 5843.
Path 583 | total_timesteps 5854.
Path 584 | total_timesteps 5865.
Path 585 | total_timesteps 5873.
Path 586 | total_timesteps 5884.
Path 587 | total_timesteps 5898.
Path 588 | total_timesteps 5905.
Path 589 | total_timesteps 5919.
Path 590 | total_timesteps 5929.
Path 591 | total_timesteps 5942.
Path 592 | total_timesteps 5949.
Path 593 | total_timesteps 5967.
Path 594 | total_timesteps 5976.
Path 595 | total_timesteps 5990.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.26    |
| Iteration     | 27       |
| MaximumReturn | 1.36     |
| MinimumReturn | -19.3    |
| TotalSamples  | 116206   |
----------------------------
itr #28 | 
Fitting dynamics.
Validation loss = 0.0027674741577357054
Validation loss = 0.0025118673220276833
Validation loss = 0.0022901026532053947
Validation loss = 0.0024399326648563147
Validation loss = 0.002187025034800172
Validation loss = 0.002144057769328356
Validation loss = 0.0026299944147467613
Validation loss = 0.0024633025750517845
Validation loss = 0.002151395194232464
Validation loss = 0.0022349669598042965
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 21.
Path 3 | total_timesteps 31.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 56.
Path 6 | total_timesteps 65.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 85.
Path 9 | total_timesteps 103.
Path 10 | total_timesteps 116.
Path 11 | total_timesteps 124.
Path 12 | total_timesteps 131.
Path 13 | total_timesteps 139.
Path 14 | total_timesteps 146.
Path 15 | total_timesteps 156.
Path 16 | total_timesteps 165.
Path 17 | total_timesteps 174.
Path 18 | total_timesteps 185.
Path 19 | total_timesteps 194.
Path 20 | total_timesteps 202.
Path 21 | total_timesteps 211.
Path 22 | total_timesteps 219.
Path 23 | total_timesteps 229.
Path 24 | total_timesteps 241.
Path 25 | total_timesteps 251.
Path 26 | total_timesteps 258.
Path 27 | total_timesteps 265.
Path 28 | total_timesteps 275.
Path 29 | total_timesteps 285.
Path 30 | total_timesteps 294.
Path 31 | total_timesteps 303.
Path 32 | total_timesteps 315.
Path 33 | total_timesteps 323.
Path 34 | total_timesteps 334.
Path 35 | total_timesteps 342.
Path 36 | total_timesteps 353.
Path 37 | total_timesteps 360.
Path 38 | total_timesteps 367.
Path 39 | total_timesteps 375.
Path 40 | total_timesteps 386.
Path 41 | total_timesteps 399.
Path 42 | total_timesteps 407.
Path 43 | total_timesteps 416.
Path 44 | total_timesteps 425.
Path 45 | total_timesteps 437.
Path 46 | total_timesteps 445.
Path 47 | total_timesteps 453.
Path 48 | total_timesteps 461.
Path 49 | total_timesteps 468.
Path 50 | total_timesteps 487.
Path 51 | total_timesteps 495.
Path 52 | total_timesteps 504.
Path 53 | total_timesteps 520.
Path 54 | total_timesteps 528.
Path 55 | total_timesteps 541.
Path 56 | total_timesteps 550.
Path 57 | total_timesteps 563.
Path 58 | total_timesteps 575.
Path 59 | total_timesteps 583.
Path 60 | total_timesteps 590.
Path 61 | total_timesteps 603.
Path 62 | total_timesteps 615.
Path 63 | total_timesteps 626.
Path 64 | total_timesteps 636.
Path 65 | total_timesteps 646.
Path 66 | total_timesteps 658.
Path 67 | total_timesteps 676.
Path 68 | total_timesteps 685.
Path 69 | total_timesteps 700.
Path 70 | total_timesteps 713.
Path 71 | total_timesteps 724.
Path 72 | total_timesteps 731.
Path 73 | total_timesteps 742.
Path 74 | total_timesteps 750.
Path 75 | total_timesteps 758.
Path 76 | total_timesteps 766.
Path 77 | total_timesteps 777.
Path 78 | total_timesteps 792.
Path 79 | total_timesteps 799.
Path 80 | total_timesteps 813.
Path 81 | total_timesteps 819.
Path 82 | total_timesteps 827.
Path 83 | total_timesteps 840.
Path 84 | total_timesteps 850.
Path 85 | total_timesteps 857.
Path 86 | total_timesteps 870.
Path 87 | total_timesteps 884.
Path 88 | total_timesteps 892.
Path 89 | total_timesteps 903.
Path 90 | total_timesteps 912.
Path 91 | total_timesteps 918.
Path 92 | total_timesteps 931.
Path 93 | total_timesteps 942.
Path 94 | total_timesteps 954.
Path 95 | total_timesteps 962.
Path 96 | total_timesteps 971.
Path 97 | total_timesteps 977.
Path 98 | total_timesteps 986.
Path 99 | total_timesteps 996.
Path 100 | total_timesteps 1009.
Path 101 | total_timesteps 1016.
Path 102 | total_timesteps 1023.
Path 103 | total_timesteps 1034.
Path 104 | total_timesteps 1054.
Path 105 | total_timesteps 1062.
Path 106 | total_timesteps 1070.
Path 107 | total_timesteps 1079.
Path 108 | total_timesteps 1091.
Path 109 | total_timesteps 1103.
Path 110 | total_timesteps 1122.
Path 111 | total_timesteps 1131.
Path 112 | total_timesteps 1140.
Path 113 | total_timesteps 1152.
Path 114 | total_timesteps 1161.
Path 115 | total_timesteps 1173.
Path 116 | total_timesteps 1183.
Path 117 | total_timesteps 1194.
Path 118 | total_timesteps 1203.
Path 119 | total_timesteps 1212.
Path 120 | total_timesteps 1220.
Path 121 | total_timesteps 1229.
Path 122 | total_timesteps 1240.
Path 123 | total_timesteps 1248.
Path 124 | total_timesteps 1261.
Path 125 | total_timesteps 1277.
Path 126 | total_timesteps 1284.
Path 127 | total_timesteps 1299.
Path 128 | total_timesteps 1312.
Path 129 | total_timesteps 1332.
Path 130 | total_timesteps 1340.
Path 131 | total_timesteps 1350.
Path 132 | total_timesteps 1361.
Path 133 | total_timesteps 1370.
Path 134 | total_timesteps 1382.
Path 135 | total_timesteps 1389.
Path 136 | total_timesteps 1397.
Path 137 | total_timesteps 1406.
Path 138 | total_timesteps 1416.
Path 139 | total_timesteps 1425.
Path 140 | total_timesteps 1437.
Path 141 | total_timesteps 1445.
Path 142 | total_timesteps 1458.
Path 143 | total_timesteps 1471.
Path 144 | total_timesteps 1478.
Path 145 | total_timesteps 1486.
Path 146 | total_timesteps 1493.
Path 147 | total_timesteps 1501.
Path 148 | total_timesteps 1512.
Path 149 | total_timesteps 1528.
Path 150 | total_timesteps 1537.
Path 151 | total_timesteps 1546.
Path 152 | total_timesteps 1562.
Path 153 | total_timesteps 1571.
Path 154 | total_timesteps 1578.
Path 155 | total_timesteps 1590.
Path 156 | total_timesteps 1602.
Path 157 | total_timesteps 1611.
Path 158 | total_timesteps 1622.
Path 159 | total_timesteps 1637.
Path 160 | total_timesteps 1647.
Path 161 | total_timesteps 1658.
Path 162 | total_timesteps 1666.
Path 163 | total_timesteps 1676.
Path 164 | total_timesteps 1685.
Path 165 | total_timesteps 1692.
Path 166 | total_timesteps 1703.
Path 167 | total_timesteps 1712.
Path 168 | total_timesteps 1723.
Path 169 | total_timesteps 1731.
Path 170 | total_timesteps 1741.
Path 171 | total_timesteps 1754.
Path 172 | total_timesteps 1769.
Path 173 | total_timesteps 1780.
Path 174 | total_timesteps 1787.
Path 175 | total_timesteps 1794.
Path 176 | total_timesteps 1802.
Path 177 | total_timesteps 1808.
Path 178 | total_timesteps 1821.
Path 179 | total_timesteps 1832.
Path 180 | total_timesteps 1843.
Path 181 | total_timesteps 1854.
Path 182 | total_timesteps 1862.
Path 183 | total_timesteps 1879.
Path 184 | total_timesteps 1886.
Path 185 | total_timesteps 1893.
Path 186 | total_timesteps 1907.
Path 187 | total_timesteps 1915.
Path 188 | total_timesteps 1927.
Path 189 | total_timesteps 1935.
Path 190 | total_timesteps 1943.
Path 191 | total_timesteps 1952.
Path 192 | total_timesteps 1964.
Path 193 | total_timesteps 1975.
Path 194 | total_timesteps 1984.
Path 195 | total_timesteps 1991.
Path 196 | total_timesteps 2001.
Path 197 | total_timesteps 2009.
Path 198 | total_timesteps 2018.
Path 199 | total_timesteps 2027.
Path 200 | total_timesteps 2039.
Path 201 | total_timesteps 2047.
Path 202 | total_timesteps 2057.
Path 203 | total_timesteps 2069.
Path 204 | total_timesteps 2079.
Path 205 | total_timesteps 2086.
Path 206 | total_timesteps 2096.
Path 207 | total_timesteps 2108.
Path 208 | total_timesteps 2115.
Path 209 | total_timesteps 2125.
Path 210 | total_timesteps 2141.
Path 211 | total_timesteps 2150.
Path 212 | total_timesteps 2157.
Path 213 | total_timesteps 2166.
Path 214 | total_timesteps 2174.
Path 215 | total_timesteps 2180.
Path 216 | total_timesteps 2189.
Path 217 | total_timesteps 2206.
Path 218 | total_timesteps 2215.
Path 219 | total_timesteps 2224.
Path 220 | total_timesteps 2234.
Path 221 | total_timesteps 2242.
Path 222 | total_timesteps 2249.
Path 223 | total_timesteps 2258.
Path 224 | total_timesteps 2268.
Path 225 | total_timesteps 2282.
Path 226 | total_timesteps 2292.
Path 227 | total_timesteps 2298.
Path 228 | total_timesteps 2306.
Path 229 | total_timesteps 2314.
Path 230 | total_timesteps 2323.
Path 231 | total_timesteps 2334.
Path 232 | total_timesteps 2345.
Path 233 | total_timesteps 2353.
Path 234 | total_timesteps 2362.
Path 235 | total_timesteps 2373.
Path 236 | total_timesteps 2382.
Path 237 | total_timesteps 2392.
Path 238 | total_timesteps 2403.
Path 239 | total_timesteps 2409.
Path 240 | total_timesteps 2416.
Path 241 | total_timesteps 2427.
Path 242 | total_timesteps 2440.
Path 243 | total_timesteps 2448.
Path 244 | total_timesteps 2458.
Path 245 | total_timesteps 2466.
Path 246 | total_timesteps 2475.
Path 247 | total_timesteps 2488.
Path 248 | total_timesteps 2499.
Path 249 | total_timesteps 2505.
Path 250 | total_timesteps 2518.
Path 251 | total_timesteps 2530.
Path 252 | total_timesteps 2538.
Path 253 | total_timesteps 2547.
Path 254 | total_timesteps 2557.
Path 255 | total_timesteps 2569.
Path 256 | total_timesteps 2584.
Path 257 | total_timesteps 2590.
Path 258 | total_timesteps 2599.
Path 259 | total_timesteps 2611.
Path 260 | total_timesteps 2625.
Path 261 | total_timesteps 2643.
Path 262 | total_timesteps 2652.
Path 263 | total_timesteps 2671.
Path 264 | total_timesteps 2682.
Path 265 | total_timesteps 2690.
Path 266 | total_timesteps 2701.
Path 267 | total_timesteps 2709.
Path 268 | total_timesteps 2719.
Path 269 | total_timesteps 2730.
Path 270 | total_timesteps 2738.
Path 271 | total_timesteps 2749.
Path 272 | total_timesteps 2758.
Path 273 | total_timesteps 2765.
Path 274 | total_timesteps 2774.
Path 275 | total_timesteps 2782.
Path 276 | total_timesteps 2789.
Path 277 | total_timesteps 2798.
Path 278 | total_timesteps 2807.
Path 279 | total_timesteps 2819.
Path 280 | total_timesteps 2829.
Path 281 | total_timesteps 2843.
Path 282 | total_timesteps 2854.
Path 283 | total_timesteps 2861.
Path 284 | total_timesteps 2879.
Path 285 | total_timesteps 2888.
Path 286 | total_timesteps 2897.
Path 287 | total_timesteps 2909.
Path 288 | total_timesteps 2918.
Path 289 | total_timesteps 2926.
Path 290 | total_timesteps 2940.
Path 291 | total_timesteps 2960.
Path 292 | total_timesteps 2966.
Path 293 | total_timesteps 2977.
Path 294 | total_timesteps 2985.
Path 295 | total_timesteps 2994.
Path 296 | total_timesteps 3005.
Path 297 | total_timesteps 3016.
Path 298 | total_timesteps 3033.
Path 299 | total_timesteps 3043.
Path 300 | total_timesteps 3054.
Path 301 | total_timesteps 3062.
Path 302 | total_timesteps 3073.
Path 303 | total_timesteps 3081.
Path 304 | total_timesteps 3090.
Path 305 | total_timesteps 3102.
Path 306 | total_timesteps 3110.
Path 307 | total_timesteps 3119.
Path 308 | total_timesteps 3129.
Path 309 | total_timesteps 3140.
Path 310 | total_timesteps 3153.
Path 311 | total_timesteps 3163.
Path 312 | total_timesteps 3171.
Path 313 | total_timesteps 3179.
Path 314 | total_timesteps 3189.
Path 315 | total_timesteps 3197.
Path 316 | total_timesteps 3213.
Path 317 | total_timesteps 3225.
Path 318 | total_timesteps 3236.
Path 319 | total_timesteps 3246.
Path 320 | total_timesteps 3263.
Path 321 | total_timesteps 3271.
Path 322 | total_timesteps 3278.
Path 323 | total_timesteps 3286.
Path 324 | total_timesteps 3297.
Path 325 | total_timesteps 3313.
Path 326 | total_timesteps 3319.
Path 327 | total_timesteps 3328.
Path 328 | total_timesteps 3339.
Path 329 | total_timesteps 3347.
Path 330 | total_timesteps 3355.
Path 331 | total_timesteps 3362.
Path 332 | total_timesteps 3377.
Path 333 | total_timesteps 3396.
Path 334 | total_timesteps 3406.
Path 335 | total_timesteps 3414.
Path 336 | total_timesteps 3425.
Path 337 | total_timesteps 3433.
Path 338 | total_timesteps 3442.
Path 339 | total_timesteps 3456.
Path 340 | total_timesteps 3469.
Path 341 | total_timesteps 3477.
Path 342 | total_timesteps 3484.
Path 343 | total_timesteps 3492.
Path 344 | total_timesteps 3500.
Path 345 | total_timesteps 3514.
Path 346 | total_timesteps 3521.
Path 347 | total_timesteps 3530.
Path 348 | total_timesteps 3537.
Path 349 | total_timesteps 3545.
Path 350 | total_timesteps 3552.
Path 351 | total_timesteps 3560.
Path 352 | total_timesteps 3569.
Path 353 | total_timesteps 3577.
Path 354 | total_timesteps 3584.
Path 355 | total_timesteps 3592.
Path 356 | total_timesteps 3615.
Path 357 | total_timesteps 3622.
Path 358 | total_timesteps 3640.
Path 359 | total_timesteps 3648.
Path 360 | total_timesteps 3658.
Path 361 | total_timesteps 3668.
Path 362 | total_timesteps 3675.
Path 363 | total_timesteps 3684.
Path 364 | total_timesteps 3692.
Path 365 | total_timesteps 3700.
Path 366 | total_timesteps 3708.
Path 367 | total_timesteps 3717.
Path 368 | total_timesteps 3724.
Path 369 | total_timesteps 3733.
Path 370 | total_timesteps 3744.
Path 371 | total_timesteps 3751.
Path 372 | total_timesteps 3763.
Path 373 | total_timesteps 3773.
Path 374 | total_timesteps 3791.
Path 375 | total_timesteps 3799.
Path 376 | total_timesteps 3807.
Path 377 | total_timesteps 3821.
Path 378 | total_timesteps 3837.
Path 379 | total_timesteps 3846.
Path 380 | total_timesteps 3861.
Path 381 | total_timesteps 3869.
Path 382 | total_timesteps 3876.
Path 383 | total_timesteps 3889.
Path 384 | total_timesteps 3898.
Path 385 | total_timesteps 3913.
Path 386 | total_timesteps 3930.
Path 387 | total_timesteps 3940.
Path 388 | total_timesteps 3948.
Path 389 | total_timesteps 3962.
Path 390 | total_timesteps 3970.
Path 391 | total_timesteps 3979.
Path 392 | total_timesteps 3992.
Path 393 | total_timesteps 4002.
Path 394 | total_timesteps 4008.
Path 395 | total_timesteps 4015.
Path 396 | total_timesteps 4023.
Path 397 | total_timesteps 4031.
Path 398 | total_timesteps 4049.
Path 399 | total_timesteps 4059.
Path 400 | total_timesteps 4069.
Path 401 | total_timesteps 4078.
Path 402 | total_timesteps 4090.
Path 403 | total_timesteps 4100.
Path 404 | total_timesteps 4113.
Path 405 | total_timesteps 4126.
Path 406 | total_timesteps 4134.
Path 407 | total_timesteps 4141.
Path 408 | total_timesteps 4151.
Path 409 | total_timesteps 4158.
Path 410 | total_timesteps 4165.
Path 411 | total_timesteps 4175.
Path 412 | total_timesteps 4186.
Path 413 | total_timesteps 4194.
Path 414 | total_timesteps 4204.
Path 415 | total_timesteps 4215.
Path 416 | total_timesteps 4222.
Path 417 | total_timesteps 4236.
Path 418 | total_timesteps 4252.
Path 419 | total_timesteps 4261.
Path 420 | total_timesteps 4270.
Path 421 | total_timesteps 4278.
Path 422 | total_timesteps 4286.
Path 423 | total_timesteps 4293.
Path 424 | total_timesteps 4302.
Path 425 | total_timesteps 4310.
Path 426 | total_timesteps 4318.
Path 427 | total_timesteps 4326.
Path 428 | total_timesteps 4335.
Path 429 | total_timesteps 4345.
Path 430 | total_timesteps 4359.
Path 431 | total_timesteps 4367.
Path 432 | total_timesteps 4375.
Path 433 | total_timesteps 4385.
Path 434 | total_timesteps 4401.
Path 435 | total_timesteps 4417.
Path 436 | total_timesteps 4426.
Path 437 | total_timesteps 4440.
Path 438 | total_timesteps 4447.
Path 439 | total_timesteps 4455.
Path 440 | total_timesteps 4473.
Path 441 | total_timesteps 4487.
Path 442 | total_timesteps 4503.
Path 443 | total_timesteps 4514.
Path 444 | total_timesteps 4520.
Path 445 | total_timesteps 4534.
Path 446 | total_timesteps 4544.
Path 447 | total_timesteps 4553.
Path 448 | total_timesteps 4567.
Path 449 | total_timesteps 4575.
Path 450 | total_timesteps 4584.
Path 451 | total_timesteps 4594.
Path 452 | total_timesteps 4607.
Path 453 | total_timesteps 4615.
Path 454 | total_timesteps 4626.
Path 455 | total_timesteps 4642.
Path 456 | total_timesteps 4654.
Path 457 | total_timesteps 4669.
Path 458 | total_timesteps 4682.
Path 459 | total_timesteps 4690.
Path 460 | total_timesteps 4700.
Path 461 | total_timesteps 4710.
Path 462 | total_timesteps 4718.
Path 463 | total_timesteps 4734.
Path 464 | total_timesteps 4746.
Path 465 | total_timesteps 4753.
Path 466 | total_timesteps 4762.
Path 467 | total_timesteps 4769.
Path 468 | total_timesteps 4777.
Path 469 | total_timesteps 4784.
Path 470 | total_timesteps 4792.
Path 471 | total_timesteps 4800.
Path 472 | total_timesteps 4808.
Path 473 | total_timesteps 4824.
Path 474 | total_timesteps 4841.
Path 475 | total_timesteps 4851.
Path 476 | total_timesteps 4859.
Path 477 | total_timesteps 4865.
Path 478 | total_timesteps 4874.
Path 479 | total_timesteps 4884.
Path 480 | total_timesteps 4894.
Path 481 | total_timesteps 4905.
Path 482 | total_timesteps 4912.
Path 483 | total_timesteps 4920.
Path 484 | total_timesteps 4932.
Path 485 | total_timesteps 4939.
Path 486 | total_timesteps 4947.
Path 487 | total_timesteps 4958.
Path 488 | total_timesteps 4965.
Path 489 | total_timesteps 4980.
Path 490 | total_timesteps 4986.
Path 491 | total_timesteps 4995.
Path 492 | total_timesteps 5006.
Path 493 | total_timesteps 5023.
Path 494 | total_timesteps 5029.
Path 495 | total_timesteps 5043.
Path 496 | total_timesteps 5053.
Path 497 | total_timesteps 5061.
Path 498 | total_timesteps 5071.
Path 499 | total_timesteps 5079.
Path 500 | total_timesteps 5087.
Path 501 | total_timesteps 5097.
Path 502 | total_timesteps 5110.
Path 503 | total_timesteps 5123.
Path 504 | total_timesteps 5132.
Path 505 | total_timesteps 5141.
Path 506 | total_timesteps 5152.
Path 507 | total_timesteps 5162.
Path 508 | total_timesteps 5170.
Path 509 | total_timesteps 5177.
Path 510 | total_timesteps 5184.
Path 511 | total_timesteps 5193.
Path 512 | total_timesteps 5200.
Path 513 | total_timesteps 5209.
Path 514 | total_timesteps 5218.
Path 515 | total_timesteps 5230.
Path 516 | total_timesteps 5238.
Path 517 | total_timesteps 5250.
Path 518 | total_timesteps 5269.
Path 519 | total_timesteps 5277.
Path 520 | total_timesteps 5286.
Path 521 | total_timesteps 5293.
Path 522 | total_timesteps 5301.
Path 523 | total_timesteps 5313.
Path 524 | total_timesteps 5326.
Path 525 | total_timesteps 5343.
Path 526 | total_timesteps 5351.
Path 527 | total_timesteps 5361.
Path 528 | total_timesteps 5368.
Path 529 | total_timesteps 5377.
Path 530 | total_timesteps 5385.
Path 531 | total_timesteps 5392.
Path 532 | total_timesteps 5403.
Path 533 | total_timesteps 5412.
Path 534 | total_timesteps 5425.
Path 535 | total_timesteps 5440.
Path 536 | total_timesteps 5446.
Path 537 | total_timesteps 5461.
Path 538 | total_timesteps 5470.
Path 539 | total_timesteps 5478.
Path 540 | total_timesteps 5486.
Path 541 | total_timesteps 5497.
Path 542 | total_timesteps 5507.
Path 543 | total_timesteps 5518.
Path 544 | total_timesteps 5525.
Path 545 | total_timesteps 5536.
Path 546 | total_timesteps 5545.
Path 547 | total_timesteps 5552.
Path 548 | total_timesteps 5559.
Path 549 | total_timesteps 5566.
Path 550 | total_timesteps 5576.
Path 551 | total_timesteps 5589.
Path 552 | total_timesteps 5601.
Path 553 | total_timesteps 5609.
Path 554 | total_timesteps 5617.
Path 555 | total_timesteps 5626.
Path 556 | total_timesteps 5636.
Path 557 | total_timesteps 5645.
Path 558 | total_timesteps 5669.
Path 559 | total_timesteps 5683.
Path 560 | total_timesteps 5693.
Path 561 | total_timesteps 5705.
Path 562 | total_timesteps 5714.
Path 563 | total_timesteps 5723.
Path 564 | total_timesteps 5731.
Path 565 | total_timesteps 5738.
Path 566 | total_timesteps 5748.
Path 567 | total_timesteps 5758.
Path 568 | total_timesteps 5768.
Path 569 | total_timesteps 5775.
Path 570 | total_timesteps 5784.
Path 571 | total_timesteps 5802.
Path 572 | total_timesteps 5809.
Path 573 | total_timesteps 5828.
Path 574 | total_timesteps 5836.
Path 575 | total_timesteps 5845.
Path 576 | total_timesteps 5851.
Path 577 | total_timesteps 5861.
Path 578 | total_timesteps 5875.
Path 579 | total_timesteps 5894.
Path 580 | total_timesteps 5901.
Path 581 | total_timesteps 5911.
Path 582 | total_timesteps 5918.
Path 583 | total_timesteps 5925.
Path 584 | total_timesteps 5948.
Path 585 | total_timesteps 5954.
Path 586 | total_timesteps 5964.
Path 587 | total_timesteps 5972.
Path 588 | total_timesteps 5984.
Path 589 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.03    |
| Iteration     | 28       |
| MaximumReturn | 1.36     |
| MinimumReturn | -18      |
| TotalSamples  | 120210   |
----------------------------
itr #29 | 
Fitting dynamics.
Validation loss = 0.0020488810259848833
Validation loss = 0.002537777414545417
Validation loss = 0.0024666828103363514
Validation loss = 0.001936799963004887
Validation loss = 0.001944838440977037
Validation loss = 0.002054725307971239
Validation loss = 0.001980929635465145
Validation loss = 0.0021829656325280666
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 14.
Path 2 | total_timesteps 22.
Path 3 | total_timesteps 29.
Path 4 | total_timesteps 37.
Path 5 | total_timesteps 45.
Path 6 | total_timesteps 53.
Path 7 | total_timesteps 69.
Path 8 | total_timesteps 76.
Path 9 | total_timesteps 84.
Path 10 | total_timesteps 91.
Path 11 | total_timesteps 103.
Path 12 | total_timesteps 117.
Path 13 | total_timesteps 125.
Path 14 | total_timesteps 136.
Path 15 | total_timesteps 145.
Path 16 | total_timesteps 156.
Path 17 | total_timesteps 169.
Path 18 | total_timesteps 177.
Path 19 | total_timesteps 185.
Path 20 | total_timesteps 192.
Path 21 | total_timesteps 198.
Path 22 | total_timesteps 208.
Path 23 | total_timesteps 218.
Path 24 | total_timesteps 232.
Path 25 | total_timesteps 248.
Path 26 | total_timesteps 255.
Path 27 | total_timesteps 264.
Path 28 | total_timesteps 274.
Path 29 | total_timesteps 283.
Path 30 | total_timesteps 294.
Path 31 | total_timesteps 310.
Path 32 | total_timesteps 316.
Path 33 | total_timesteps 326.
Path 34 | total_timesteps 335.
Path 35 | total_timesteps 345.
Path 36 | total_timesteps 355.
Path 37 | total_timesteps 365.
Path 38 | total_timesteps 375.
Path 39 | total_timesteps 383.
Path 40 | total_timesteps 392.
Path 41 | total_timesteps 406.
Path 42 | total_timesteps 414.
Path 43 | total_timesteps 422.
Path 44 | total_timesteps 435.
Path 45 | total_timesteps 456.
Path 46 | total_timesteps 465.
Path 47 | total_timesteps 472.
Path 48 | total_timesteps 478.
Path 49 | total_timesteps 486.
Path 50 | total_timesteps 494.
Path 51 | total_timesteps 503.
Path 52 | total_timesteps 514.
Path 53 | total_timesteps 522.
Path 54 | total_timesteps 538.
Path 55 | total_timesteps 553.
Path 56 | total_timesteps 561.
Path 57 | total_timesteps 569.
Path 58 | total_timesteps 580.
Path 59 | total_timesteps 595.
Path 60 | total_timesteps 604.
Path 61 | total_timesteps 611.
Path 62 | total_timesteps 621.
Path 63 | total_timesteps 632.
Path 64 | total_timesteps 644.
Path 65 | total_timesteps 653.
Path 66 | total_timesteps 665.
Path 67 | total_timesteps 672.
Path 68 | total_timesteps 684.
Path 69 | total_timesteps 696.
Path 70 | total_timesteps 703.
Path 71 | total_timesteps 711.
Path 72 | total_timesteps 718.
Path 73 | total_timesteps 726.
Path 74 | total_timesteps 734.
Path 75 | total_timesteps 746.
Path 76 | total_timesteps 756.
Path 77 | total_timesteps 768.
Path 78 | total_timesteps 777.
Path 79 | total_timesteps 786.
Path 80 | total_timesteps 793.
Path 81 | total_timesteps 800.
Path 82 | total_timesteps 808.
Path 83 | total_timesteps 819.
Path 84 | total_timesteps 826.
Path 85 | total_timesteps 834.
Path 86 | total_timesteps 844.
Path 87 | total_timesteps 852.
Path 88 | total_timesteps 861.
Path 89 | total_timesteps 871.
Path 90 | total_timesteps 880.
Path 91 | total_timesteps 891.
Path 92 | total_timesteps 899.
Path 93 | total_timesteps 912.
Path 94 | total_timesteps 929.
Path 95 | total_timesteps 936.
Path 96 | total_timesteps 945.
Path 97 | total_timesteps 958.
Path 98 | total_timesteps 967.
Path 99 | total_timesteps 974.
Path 100 | total_timesteps 985.
Path 101 | total_timesteps 992.
Path 102 | total_timesteps 1001.
Path 103 | total_timesteps 1011.
Path 104 | total_timesteps 1018.
Path 105 | total_timesteps 1027.
Path 106 | total_timesteps 1037.
Path 107 | total_timesteps 1048.
Path 108 | total_timesteps 1058.
Path 109 | total_timesteps 1066.
Path 110 | total_timesteps 1073.
Path 111 | total_timesteps 1080.
Path 112 | total_timesteps 1087.
Path 113 | total_timesteps 1094.
Path 114 | total_timesteps 1102.
Path 115 | total_timesteps 1115.
Path 116 | total_timesteps 1123.
Path 117 | total_timesteps 1136.
Path 118 | total_timesteps 1151.
Path 119 | total_timesteps 1158.
Path 120 | total_timesteps 1168.
Path 121 | total_timesteps 1176.
Path 122 | total_timesteps 1183.
Path 123 | total_timesteps 1192.
Path 124 | total_timesteps 1200.
Path 125 | total_timesteps 1210.
Path 126 | total_timesteps 1221.
Path 127 | total_timesteps 1237.
Path 128 | total_timesteps 1245.
Path 129 | total_timesteps 1255.
Path 130 | total_timesteps 1266.
Path 131 | total_timesteps 1274.
Path 132 | total_timesteps 1283.
Path 133 | total_timesteps 1295.
Path 134 | total_timesteps 1301.
Path 135 | total_timesteps 1310.
Path 136 | total_timesteps 1317.
Path 137 | total_timesteps 1328.
Path 138 | total_timesteps 1341.
Path 139 | total_timesteps 1354.
Path 140 | total_timesteps 1364.
Path 141 | total_timesteps 1377.
Path 142 | total_timesteps 1390.
Path 143 | total_timesteps 1397.
Path 144 | total_timesteps 1409.
Path 145 | total_timesteps 1419.
Path 146 | total_timesteps 1425.
Path 147 | total_timesteps 1433.
Path 148 | total_timesteps 1440.
Path 149 | total_timesteps 1451.
Path 150 | total_timesteps 1459.
Path 151 | total_timesteps 1475.
Path 152 | total_timesteps 1487.
Path 153 | total_timesteps 1495.
Path 154 | total_timesteps 1504.
Path 155 | total_timesteps 1515.
Path 156 | total_timesteps 1523.
Path 157 | total_timesteps 1531.
Path 158 | total_timesteps 1538.
Path 159 | total_timesteps 1547.
Path 160 | total_timesteps 1554.
Path 161 | total_timesteps 1565.
Path 162 | total_timesteps 1578.
Path 163 | total_timesteps 1587.
Path 164 | total_timesteps 1598.
Path 165 | total_timesteps 1606.
Path 166 | total_timesteps 1614.
Path 167 | total_timesteps 1622.
Path 168 | total_timesteps 1631.
Path 169 | total_timesteps 1647.
Path 170 | total_timesteps 1653.
Path 171 | total_timesteps 1662.
Path 172 | total_timesteps 1671.
Path 173 | total_timesteps 1679.
Path 174 | total_timesteps 1687.
Path 175 | total_timesteps 1696.
Path 176 | total_timesteps 1704.
Path 177 | total_timesteps 1712.
Path 178 | total_timesteps 1720.
Path 179 | total_timesteps 1727.
Path 180 | total_timesteps 1734.
Path 181 | total_timesteps 1742.
Path 182 | total_timesteps 1751.
Path 183 | total_timesteps 1758.
Path 184 | total_timesteps 1767.
Path 185 | total_timesteps 1788.
Path 186 | total_timesteps 1796.
Path 187 | total_timesteps 1804.
Path 188 | total_timesteps 1813.
Path 189 | total_timesteps 1832.
Path 190 | total_timesteps 1843.
Path 191 | total_timesteps 1852.
Path 192 | total_timesteps 1859.
Path 193 | total_timesteps 1866.
Path 194 | total_timesteps 1876.
Path 195 | total_timesteps 1885.
Path 196 | total_timesteps 1902.
Path 197 | total_timesteps 1911.
Path 198 | total_timesteps 1920.
Path 199 | total_timesteps 1927.
Path 200 | total_timesteps 1936.
Path 201 | total_timesteps 1943.
Path 202 | total_timesteps 1950.
Path 203 | total_timesteps 1961.
Path 204 | total_timesteps 1969.
Path 205 | total_timesteps 1976.
Path 206 | total_timesteps 1983.
Path 207 | total_timesteps 1995.
Path 208 | total_timesteps 2001.
Path 209 | total_timesteps 2013.
Path 210 | total_timesteps 2020.
Path 211 | total_timesteps 2028.
Path 212 | total_timesteps 2037.
Path 213 | total_timesteps 2047.
Path 214 | total_timesteps 2060.
Path 215 | total_timesteps 2066.
Path 216 | total_timesteps 2076.
Path 217 | total_timesteps 2087.
Path 218 | total_timesteps 2100.
Path 219 | total_timesteps 2110.
Path 220 | total_timesteps 2117.
Path 221 | total_timesteps 2126.
Path 222 | total_timesteps 2134.
Path 223 | total_timesteps 2141.
Path 224 | total_timesteps 2151.
Path 225 | total_timesteps 2162.
Path 226 | total_timesteps 2170.
Path 227 | total_timesteps 2178.
Path 228 | total_timesteps 2185.
Path 229 | total_timesteps 2192.
Path 230 | total_timesteps 2198.
Path 231 | total_timesteps 2205.
Path 232 | total_timesteps 2212.
Path 233 | total_timesteps 2220.
Path 234 | total_timesteps 2230.
Path 235 | total_timesteps 2240.
Path 236 | total_timesteps 2248.
Path 237 | total_timesteps 2255.
Path 238 | total_timesteps 2266.
Path 239 | total_timesteps 2275.
Path 240 | total_timesteps 2285.
Path 241 | total_timesteps 2291.
Path 242 | total_timesteps 2299.
Path 243 | total_timesteps 2306.
Path 244 | total_timesteps 2317.
Path 245 | total_timesteps 2329.
Path 246 | total_timesteps 2338.
Path 247 | total_timesteps 2350.
Path 248 | total_timesteps 2358.
Path 249 | total_timesteps 2365.
Path 250 | total_timesteps 2376.
Path 251 | total_timesteps 2388.
Path 252 | total_timesteps 2399.
Path 253 | total_timesteps 2408.
Path 254 | total_timesteps 2415.
Path 255 | total_timesteps 2428.
Path 256 | total_timesteps 2437.
Path 257 | total_timesteps 2444.
Path 258 | total_timesteps 2454.
Path 259 | total_timesteps 2462.
Path 260 | total_timesteps 2473.
Path 261 | total_timesteps 2482.
Path 262 | total_timesteps 2497.
Path 263 | total_timesteps 2508.
Path 264 | total_timesteps 2517.
Path 265 | total_timesteps 2527.
Path 266 | total_timesteps 2537.
Path 267 | total_timesteps 2547.
Path 268 | total_timesteps 2555.
Path 269 | total_timesteps 2565.
Path 270 | total_timesteps 2575.
Path 271 | total_timesteps 2589.
Path 272 | total_timesteps 2597.
Path 273 | total_timesteps 2608.
Path 274 | total_timesteps 2615.
Path 275 | total_timesteps 2622.
Path 276 | total_timesteps 2630.
Path 277 | total_timesteps 2641.
Path 278 | total_timesteps 2650.
Path 279 | total_timesteps 2659.
Path 280 | total_timesteps 2667.
Path 281 | total_timesteps 2676.
Path 282 | total_timesteps 2685.
Path 283 | total_timesteps 2694.
Path 284 | total_timesteps 2702.
Path 285 | total_timesteps 2709.
Path 286 | total_timesteps 2717.
Path 287 | total_timesteps 2724.
Path 288 | total_timesteps 2733.
Path 289 | total_timesteps 2744.
Path 290 | total_timesteps 2754.
Path 291 | total_timesteps 2762.
Path 292 | total_timesteps 2773.
Path 293 | total_timesteps 2782.
Path 294 | total_timesteps 2798.
Path 295 | total_timesteps 2805.
Path 296 | total_timesteps 2813.
Path 297 | total_timesteps 2829.
Path 298 | total_timesteps 2838.
Path 299 | total_timesteps 2849.
Path 300 | total_timesteps 2856.
Path 301 | total_timesteps 2864.
Path 302 | total_timesteps 2871.
Path 303 | total_timesteps 2879.
Path 304 | total_timesteps 2891.
Path 305 | total_timesteps 2898.
Path 306 | total_timesteps 2907.
Path 307 | total_timesteps 2914.
Path 308 | total_timesteps 2925.
Path 309 | total_timesteps 2931.
Path 310 | total_timesteps 2940.
Path 311 | total_timesteps 2949.
Path 312 | total_timesteps 2957.
Path 313 | total_timesteps 2966.
Path 314 | total_timesteps 2974.
Path 315 | total_timesteps 2983.
Path 316 | total_timesteps 2990.
Path 317 | total_timesteps 2998.
Path 318 | total_timesteps 3007.
Path 319 | total_timesteps 3020.
Path 320 | total_timesteps 3030.
Path 321 | total_timesteps 3038.
Path 322 | total_timesteps 3045.
Path 323 | total_timesteps 3058.
Path 324 | total_timesteps 3064.
Path 325 | total_timesteps 3074.
Path 326 | total_timesteps 3085.
Path 327 | total_timesteps 3092.
Path 328 | total_timesteps 3102.
Path 329 | total_timesteps 3113.
Path 330 | total_timesteps 3120.
Path 331 | total_timesteps 3127.
Path 332 | total_timesteps 3136.
Path 333 | total_timesteps 3144.
Path 334 | total_timesteps 3156.
Path 335 | total_timesteps 3167.
Path 336 | total_timesteps 3174.
Path 337 | total_timesteps 3181.
Path 338 | total_timesteps 3193.
Path 339 | total_timesteps 3204.
Path 340 | total_timesteps 3215.
Path 341 | total_timesteps 3224.
Path 342 | total_timesteps 3231.
Path 343 | total_timesteps 3239.
Path 344 | total_timesteps 3252.
Path 345 | total_timesteps 3259.
Path 346 | total_timesteps 3268.
Path 347 | total_timesteps 3281.
Path 348 | total_timesteps 3288.
Path 349 | total_timesteps 3296.
Path 350 | total_timesteps 3304.
Path 351 | total_timesteps 3319.
Path 352 | total_timesteps 3327.
Path 353 | total_timesteps 3337.
Path 354 | total_timesteps 3348.
Path 355 | total_timesteps 3356.
Path 356 | total_timesteps 3366.
Path 357 | total_timesteps 3374.
Path 358 | total_timesteps 3383.
Path 359 | total_timesteps 3392.
Path 360 | total_timesteps 3404.
Path 361 | total_timesteps 3412.
Path 362 | total_timesteps 3421.
Path 363 | total_timesteps 3430.
Path 364 | total_timesteps 3440.
Path 365 | total_timesteps 3457.
Path 366 | total_timesteps 3469.
Path 367 | total_timesteps 3479.
Path 368 | total_timesteps 3489.
Path 369 | total_timesteps 3501.
Path 370 | total_timesteps 3513.
Path 371 | total_timesteps 3520.
Path 372 | total_timesteps 3531.
Path 373 | total_timesteps 3537.
Path 374 | total_timesteps 3554.
Path 375 | total_timesteps 3561.
Path 376 | total_timesteps 3568.
Path 377 | total_timesteps 3574.
Path 378 | total_timesteps 3582.
Path 379 | total_timesteps 3590.
Path 380 | total_timesteps 3601.
Path 381 | total_timesteps 3610.
Path 382 | total_timesteps 3618.
Path 383 | total_timesteps 3627.
Path 384 | total_timesteps 3635.
Path 385 | total_timesteps 3643.
Path 386 | total_timesteps 3650.
Path 387 | total_timesteps 3661.
Path 388 | total_timesteps 3672.
Path 389 | total_timesteps 3678.
Path 390 | total_timesteps 3686.
Path 391 | total_timesteps 3694.
Path 392 | total_timesteps 3702.
Path 393 | total_timesteps 3714.
Path 394 | total_timesteps 3723.
Path 395 | total_timesteps 3736.
Path 396 | total_timesteps 3747.
Path 397 | total_timesteps 3757.
Path 398 | total_timesteps 3765.
Path 399 | total_timesteps 3776.
Path 400 | total_timesteps 3784.
Path 401 | total_timesteps 3792.
Path 402 | total_timesteps 3800.
Path 403 | total_timesteps 3810.
Path 404 | total_timesteps 3818.
Path 405 | total_timesteps 3827.
Path 406 | total_timesteps 3834.
Path 407 | total_timesteps 3840.
Path 408 | total_timesteps 3847.
Path 409 | total_timesteps 3860.
Path 410 | total_timesteps 3873.
Path 411 | total_timesteps 3881.
Path 412 | total_timesteps 3891.
Path 413 | total_timesteps 3897.
Path 414 | total_timesteps 3907.
Path 415 | total_timesteps 3916.
Path 416 | total_timesteps 3923.
Path 417 | total_timesteps 3932.
Path 418 | total_timesteps 3944.
Path 419 | total_timesteps 3951.
Path 420 | total_timesteps 3960.
Path 421 | total_timesteps 3967.
Path 422 | total_timesteps 3974.
Path 423 | total_timesteps 3985.
Path 424 | total_timesteps 3995.
Path 425 | total_timesteps 4003.
Path 426 | total_timesteps 4014.
Path 427 | total_timesteps 4030.
Path 428 | total_timesteps 4038.
Path 429 | total_timesteps 4053.
Path 430 | total_timesteps 4061.
Path 431 | total_timesteps 4067.
Path 432 | total_timesteps 4074.
Path 433 | total_timesteps 4082.
Path 434 | total_timesteps 4089.
Path 435 | total_timesteps 4098.
Path 436 | total_timesteps 4106.
Path 437 | total_timesteps 4112.
Path 438 | total_timesteps 4126.
Path 439 | total_timesteps 4133.
Path 440 | total_timesteps 4148.
Path 441 | total_timesteps 4156.
Path 442 | total_timesteps 4163.
Path 443 | total_timesteps 4173.
Path 444 | total_timesteps 4187.
Path 445 | total_timesteps 4195.
Path 446 | total_timesteps 4201.
Path 447 | total_timesteps 4209.
Path 448 | total_timesteps 4219.
Path 449 | total_timesteps 4228.
Path 450 | total_timesteps 4236.
Path 451 | total_timesteps 4244.
Path 452 | total_timesteps 4252.
Path 453 | total_timesteps 4265.
Path 454 | total_timesteps 4277.
Path 455 | total_timesteps 4285.
Path 456 | total_timesteps 4300.
Path 457 | total_timesteps 4307.
Path 458 | total_timesteps 4317.
Path 459 | total_timesteps 4333.
Path 460 | total_timesteps 4340.
Path 461 | total_timesteps 4351.
Path 462 | total_timesteps 4361.
Path 463 | total_timesteps 4369.
Path 464 | total_timesteps 4386.
Path 465 | total_timesteps 4395.
Path 466 | total_timesteps 4409.
Path 467 | total_timesteps 4422.
Path 468 | total_timesteps 4431.
Path 469 | total_timesteps 4444.
Path 470 | total_timesteps 4455.
Path 471 | total_timesteps 4464.
Path 472 | total_timesteps 4470.
Path 473 | total_timesteps 4484.
Path 474 | total_timesteps 4493.
Path 475 | total_timesteps 4500.
Path 476 | total_timesteps 4511.
Path 477 | total_timesteps 4520.
Path 478 | total_timesteps 4528.
Path 479 | total_timesteps 4536.
Path 480 | total_timesteps 4544.
Path 481 | total_timesteps 4551.
Path 482 | total_timesteps 4558.
Path 483 | total_timesteps 4567.
Path 484 | total_timesteps 4576.
Path 485 | total_timesteps 4582.
Path 486 | total_timesteps 4593.
Path 487 | total_timesteps 4599.
Path 488 | total_timesteps 4609.
Path 489 | total_timesteps 4617.
Path 490 | total_timesteps 4628.
Path 491 | total_timesteps 4634.
Path 492 | total_timesteps 4648.
Path 493 | total_timesteps 4655.
Path 494 | total_timesteps 4669.
Path 495 | total_timesteps 4679.
Path 496 | total_timesteps 4688.
Path 497 | total_timesteps 4703.
Path 498 | total_timesteps 4711.
Path 499 | total_timesteps 4721.
Path 500 | total_timesteps 4728.
Path 501 | total_timesteps 4737.
Path 502 | total_timesteps 4745.
Path 503 | total_timesteps 4753.
Path 504 | total_timesteps 4760.
Path 505 | total_timesteps 4770.
Path 506 | total_timesteps 4782.
Path 507 | total_timesteps 4792.
Path 508 | total_timesteps 4805.
Path 509 | total_timesteps 4819.
Path 510 | total_timesteps 4827.
Path 511 | total_timesteps 4834.
Path 512 | total_timesteps 4844.
Path 513 | total_timesteps 4851.
Path 514 | total_timesteps 4862.
Path 515 | total_timesteps 4871.
Path 516 | total_timesteps 4879.
Path 517 | total_timesteps 4890.
Path 518 | total_timesteps 4898.
Path 519 | total_timesteps 4909.
Path 520 | total_timesteps 4919.
Path 521 | total_timesteps 4928.
Path 522 | total_timesteps 4936.
Path 523 | total_timesteps 4950.
Path 524 | total_timesteps 4957.
Path 525 | total_timesteps 4982.
Path 526 | total_timesteps 4992.
Path 527 | total_timesteps 5004.
Path 528 | total_timesteps 5014.
Path 529 | total_timesteps 5025.
Path 530 | total_timesteps 5036.
Path 531 | total_timesteps 5053.
Path 532 | total_timesteps 5063.
Path 533 | total_timesteps 5071.
Path 534 | total_timesteps 5080.
Path 535 | total_timesteps 5093.
Path 536 | total_timesteps 5105.
Path 537 | total_timesteps 5116.
Path 538 | total_timesteps 5124.
Path 539 | total_timesteps 5133.
Path 540 | total_timesteps 5141.
Path 541 | total_timesteps 5149.
Path 542 | total_timesteps 5165.
Path 543 | total_timesteps 5173.
Path 544 | total_timesteps 5183.
Path 545 | total_timesteps 5192.
Path 546 | total_timesteps 5201.
Path 547 | total_timesteps 5212.
Path 548 | total_timesteps 5220.
Path 549 | total_timesteps 5228.
Path 550 | total_timesteps 5238.
Path 551 | total_timesteps 5246.
Path 552 | total_timesteps 5254.
Path 553 | total_timesteps 5262.
Path 554 | total_timesteps 5269.
Path 555 | total_timesteps 5278.
Path 556 | total_timesteps 5289.
Path 557 | total_timesteps 5297.
Path 558 | total_timesteps 5305.
Path 559 | total_timesteps 5313.
Path 560 | total_timesteps 5320.
Path 561 | total_timesteps 5330.
Path 562 | total_timesteps 5338.
Path 563 | total_timesteps 5346.
Path 564 | total_timesteps 5360.
Path 565 | total_timesteps 5374.
Path 566 | total_timesteps 5382.
Path 567 | total_timesteps 5395.
Path 568 | total_timesteps 5404.
Path 569 | total_timesteps 5411.
Path 570 | total_timesteps 5420.
Path 571 | total_timesteps 5430.
Path 572 | total_timesteps 5437.
Path 573 | total_timesteps 5446.
Path 574 | total_timesteps 5460.
Path 575 | total_timesteps 5479.
Path 576 | total_timesteps 5486.
Path 577 | total_timesteps 5498.
Path 578 | total_timesteps 5506.
Path 579 | total_timesteps 5513.
Path 580 | total_timesteps 5522.
Path 581 | total_timesteps 5532.
Path 582 | total_timesteps 5544.
Path 583 | total_timesteps 5554.
Path 584 | total_timesteps 5570.
Path 585 | total_timesteps 5578.
Path 586 | total_timesteps 5585.
Path 587 | total_timesteps 5593.
Path 588 | total_timesteps 5601.
Path 589 | total_timesteps 5611.
Path 590 | total_timesteps 5618.
Path 591 | total_timesteps 5628.
Path 592 | total_timesteps 5637.
Path 593 | total_timesteps 5651.
Path 594 | total_timesteps 5665.
Path 595 | total_timesteps 5678.
Path 596 | total_timesteps 5685.
Path 597 | total_timesteps 5696.
Path 598 | total_timesteps 5709.
Path 599 | total_timesteps 5721.
Path 600 | total_timesteps 5729.
Path 601 | total_timesteps 5741.
Path 602 | total_timesteps 5750.
Path 603 | total_timesteps 5759.
Path 604 | total_timesteps 5770.
Path 605 | total_timesteps 5781.
Path 606 | total_timesteps 5793.
Path 607 | total_timesteps 5800.
Path 608 | total_timesteps 5809.
Path 609 | total_timesteps 5825.
Path 610 | total_timesteps 5834.
Path 611 | total_timesteps 5843.
Path 612 | total_timesteps 5850.
Path 613 | total_timesteps 5859.
Path 614 | total_timesteps 5871.
Path 615 | total_timesteps 5879.
Path 616 | total_timesteps 5889.
Path 617 | total_timesteps 5897.
Path 618 | total_timesteps 5905.
Path 619 | total_timesteps 5914.
Path 620 | total_timesteps 5928.
Path 621 | total_timesteps 5935.
Path 622 | total_timesteps 5947.
Path 623 | total_timesteps 5955.
Path 624 | total_timesteps 5961.
Path 625 | total_timesteps 5971.
Path 626 | total_timesteps 5980.
Path 627 | total_timesteps 5993.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.25    |
| Iteration     | 29       |
| MaximumReturn | -0.406   |
| MinimumReturn | -18.4    |
| TotalSamples  | 124211   |
----------------------------
itr #30 | 
Fitting dynamics.
Validation loss = 0.0021778931841254234
Validation loss = 0.002040792489424348
Validation loss = 0.0022467058151960373
Validation loss = 0.0020037940703332424
Validation loss = 0.002207097364589572
Validation loss = 0.0026410198770463467
Validation loss = 0.0019755568355321884
Validation loss = 0.0022511205170303583
Validation loss = 0.0020031246822327375
Validation loss = 0.0019634850323200226
Validation loss = 0.002056234749034047
Validation loss = 0.002044705906882882
Validation loss = 0.001831844449043274
Validation loss = 0.002152189612388611
Validation loss = 0.0022029161918908358
Validation loss = 0.0021449110936373472
Validation loss = 0.002111258450895548
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 12.
Path 2 | total_timesteps 24.
Path 3 | total_timesteps 34.
Path 4 | total_timesteps 42.
Path 5 | total_timesteps 54.
Path 6 | total_timesteps 66.
Path 7 | total_timesteps 74.
Path 8 | total_timesteps 83.
Path 9 | total_timesteps 93.
Path 10 | total_timesteps 103.
Path 11 | total_timesteps 109.
Path 12 | total_timesteps 116.
Path 13 | total_timesteps 123.
Path 14 | total_timesteps 134.
Path 15 | total_timesteps 141.
Path 16 | total_timesteps 149.
Path 17 | total_timesteps 159.
Path 18 | total_timesteps 169.
Path 19 | total_timesteps 177.
Path 20 | total_timesteps 185.
Path 21 | total_timesteps 192.
Path 22 | total_timesteps 203.
Path 23 | total_timesteps 211.
Path 24 | total_timesteps 218.
Path 25 | total_timesteps 225.
Path 26 | total_timesteps 232.
Path 27 | total_timesteps 240.
Path 28 | total_timesteps 247.
Path 29 | total_timesteps 257.
Path 30 | total_timesteps 270.
Path 31 | total_timesteps 281.
Path 32 | total_timesteps 288.
Path 33 | total_timesteps 296.
Path 34 | total_timesteps 303.
Path 35 | total_timesteps 315.
Path 36 | total_timesteps 328.
Path 37 | total_timesteps 340.
Path 38 | total_timesteps 347.
Path 39 | total_timesteps 355.
Path 40 | total_timesteps 367.
Path 41 | total_timesteps 387.
Path 42 | total_timesteps 395.
Path 43 | total_timesteps 403.
Path 44 | total_timesteps 410.
Path 45 | total_timesteps 422.
Path 46 | total_timesteps 430.
Path 47 | total_timesteps 438.
Path 48 | total_timesteps 445.
Path 49 | total_timesteps 452.
Path 50 | total_timesteps 462.
Path 51 | total_timesteps 469.
Path 52 | total_timesteps 479.
Path 53 | total_timesteps 487.
Path 54 | total_timesteps 495.
Path 55 | total_timesteps 508.
Path 56 | total_timesteps 520.
Path 57 | total_timesteps 532.
Path 58 | total_timesteps 544.
Path 59 | total_timesteps 554.
Path 60 | total_timesteps 565.
Path 61 | total_timesteps 573.
Path 62 | total_timesteps 581.
Path 63 | total_timesteps 590.
Path 64 | total_timesteps 597.
Path 65 | total_timesteps 609.
Path 66 | total_timesteps 616.
Path 67 | total_timesteps 624.
Path 68 | total_timesteps 631.
Path 69 | total_timesteps 640.
Path 70 | total_timesteps 647.
Path 71 | total_timesteps 656.
Path 72 | total_timesteps 663.
Path 73 | total_timesteps 671.
Path 74 | total_timesteps 680.
Path 75 | total_timesteps 687.
Path 76 | total_timesteps 698.
Path 77 | total_timesteps 706.
Path 78 | total_timesteps 717.
Path 79 | total_timesteps 728.
Path 80 | total_timesteps 736.
Path 81 | total_timesteps 743.
Path 82 | total_timesteps 753.
Path 83 | total_timesteps 764.
Path 84 | total_timesteps 781.
Path 85 | total_timesteps 790.
Path 86 | total_timesteps 798.
Path 87 | total_timesteps 806.
Path 88 | total_timesteps 816.
Path 89 | total_timesteps 823.
Path 90 | total_timesteps 831.
Path 91 | total_timesteps 840.
Path 92 | total_timesteps 849.
Path 93 | total_timesteps 864.
Path 94 | total_timesteps 873.
Path 95 | total_timesteps 887.
Path 96 | total_timesteps 895.
Path 97 | total_timesteps 905.
Path 98 | total_timesteps 913.
Path 99 | total_timesteps 919.
Path 100 | total_timesteps 931.
Path 101 | total_timesteps 942.
Path 102 | total_timesteps 949.
Path 103 | total_timesteps 957.
Path 104 | total_timesteps 965.
Path 105 | total_timesteps 973.
Path 106 | total_timesteps 980.
Path 107 | total_timesteps 988.
Path 108 | total_timesteps 1010.
Path 109 | total_timesteps 1018.
Path 110 | total_timesteps 1027.
Path 111 | total_timesteps 1036.
Path 112 | total_timesteps 1043.
Path 113 | total_timesteps 1053.
Path 114 | total_timesteps 1059.
Path 115 | total_timesteps 1073.
Path 116 | total_timesteps 1079.
Path 117 | total_timesteps 1085.
Path 118 | total_timesteps 1095.
Path 119 | total_timesteps 1104.
Path 120 | total_timesteps 1112.
Path 121 | total_timesteps 1119.
Path 122 | total_timesteps 1127.
Path 123 | total_timesteps 1134.
Path 124 | total_timesteps 1146.
Path 125 | total_timesteps 1158.
Path 126 | total_timesteps 1169.
Path 127 | total_timesteps 1176.
Path 128 | total_timesteps 1182.
Path 129 | total_timesteps 1190.
Path 130 | total_timesteps 1198.
Path 131 | total_timesteps 1207.
Path 132 | total_timesteps 1218.
Path 133 | total_timesteps 1229.
Path 134 | total_timesteps 1235.
Path 135 | total_timesteps 1252.
Path 136 | total_timesteps 1265.
Path 137 | total_timesteps 1272.
Path 138 | total_timesteps 1283.
Path 139 | total_timesteps 1290.
Path 140 | total_timesteps 1297.
Path 141 | total_timesteps 1303.
Path 142 | total_timesteps 1310.
Path 143 | total_timesteps 1322.
Path 144 | total_timesteps 1330.
Path 145 | total_timesteps 1341.
Path 146 | total_timesteps 1349.
Path 147 | total_timesteps 1356.
Path 148 | total_timesteps 1362.
Path 149 | total_timesteps 1370.
Path 150 | total_timesteps 1379.
Path 151 | total_timesteps 1391.
Path 152 | total_timesteps 1397.
Path 153 | total_timesteps 1407.
Path 154 | total_timesteps 1414.
Path 155 | total_timesteps 1423.
Path 156 | total_timesteps 1440.
Path 157 | total_timesteps 1449.
Path 158 | total_timesteps 1459.
Path 159 | total_timesteps 1473.
Path 160 | total_timesteps 1480.
Path 161 | total_timesteps 1487.
Path 162 | total_timesteps 1496.
Path 163 | total_timesteps 1508.
Path 164 | total_timesteps 1519.
Path 165 | total_timesteps 1529.
Path 166 | total_timesteps 1543.
Path 167 | total_timesteps 1553.
Path 168 | total_timesteps 1561.
Path 169 | total_timesteps 1568.
Path 170 | total_timesteps 1577.
Path 171 | total_timesteps 1590.
Path 172 | total_timesteps 1603.
Path 173 | total_timesteps 1613.
Path 174 | total_timesteps 1620.
Path 175 | total_timesteps 1630.
Path 176 | total_timesteps 1637.
Path 177 | total_timesteps 1645.
Path 178 | total_timesteps 1654.
Path 179 | total_timesteps 1662.
Path 180 | total_timesteps 1675.
Path 181 | total_timesteps 1682.
Path 182 | total_timesteps 1694.
Path 183 | total_timesteps 1703.
Path 184 | total_timesteps 1718.
Path 185 | total_timesteps 1736.
Path 186 | total_timesteps 1749.
Path 187 | total_timesteps 1759.
Path 188 | total_timesteps 1766.
Path 189 | total_timesteps 1773.
Path 190 | total_timesteps 1781.
Path 191 | total_timesteps 1788.
Path 192 | total_timesteps 1795.
Path 193 | total_timesteps 1808.
Path 194 | total_timesteps 1816.
Path 195 | total_timesteps 1823.
Path 196 | total_timesteps 1832.
Path 197 | total_timesteps 1840.
Path 198 | total_timesteps 1849.
Path 199 | total_timesteps 1858.
Path 200 | total_timesteps 1866.
Path 201 | total_timesteps 1875.
Path 202 | total_timesteps 1884.
Path 203 | total_timesteps 1893.
Path 204 | total_timesteps 1902.
Path 205 | total_timesteps 1909.
Path 206 | total_timesteps 1919.
Path 207 | total_timesteps 1926.
Path 208 | total_timesteps 1936.
Path 209 | total_timesteps 1944.
Path 210 | total_timesteps 1952.
Path 211 | total_timesteps 1961.
Path 212 | total_timesteps 1969.
Path 213 | total_timesteps 1976.
Path 214 | total_timesteps 1989.
Path 215 | total_timesteps 1996.
Path 216 | total_timesteps 2005.
Path 217 | total_timesteps 2016.
Path 218 | total_timesteps 2022.
Path 219 | total_timesteps 2028.
Path 220 | total_timesteps 2035.
Path 221 | total_timesteps 2043.
Path 222 | total_timesteps 2056.
Path 223 | total_timesteps 2070.
Path 224 | total_timesteps 2078.
Path 225 | total_timesteps 2091.
Path 226 | total_timesteps 2099.
Path 227 | total_timesteps 2113.
Path 228 | total_timesteps 2125.
Path 229 | total_timesteps 2134.
Path 230 | total_timesteps 2145.
Path 231 | total_timesteps 2154.
Path 232 | total_timesteps 2161.
Path 233 | total_timesteps 2168.
Path 234 | total_timesteps 2176.
Path 235 | total_timesteps 2184.
Path 236 | total_timesteps 2196.
Path 237 | total_timesteps 2208.
Path 238 | total_timesteps 2216.
Path 239 | total_timesteps 2225.
Path 240 | total_timesteps 2232.
Path 241 | total_timesteps 2243.
Path 242 | total_timesteps 2255.
Path 243 | total_timesteps 2269.
Path 244 | total_timesteps 2280.
Path 245 | total_timesteps 2288.
Path 246 | total_timesteps 2298.
Path 247 | total_timesteps 2311.
Path 248 | total_timesteps 2320.
Path 249 | total_timesteps 2327.
Path 250 | total_timesteps 2334.
Path 251 | total_timesteps 2344.
Path 252 | total_timesteps 2352.
Path 253 | total_timesteps 2361.
Path 254 | total_timesteps 2370.
Path 255 | total_timesteps 2379.
Path 256 | total_timesteps 2385.
Path 257 | total_timesteps 2393.
Path 258 | total_timesteps 2401.
Path 259 | total_timesteps 2410.
Path 260 | total_timesteps 2417.
Path 261 | total_timesteps 2423.
Path 262 | total_timesteps 2432.
Path 263 | total_timesteps 2439.
Path 264 | total_timesteps 2447.
Path 265 | total_timesteps 2453.
Path 266 | total_timesteps 2460.
Path 267 | total_timesteps 2471.
Path 268 | total_timesteps 2478.
Path 269 | total_timesteps 2491.
Path 270 | total_timesteps 2500.
Path 271 | total_timesteps 2508.
Path 272 | total_timesteps 2517.
Path 273 | total_timesteps 2530.
Path 274 | total_timesteps 2543.
Path 275 | total_timesteps 2551.
Path 276 | total_timesteps 2560.
Path 277 | total_timesteps 2576.
Path 278 | total_timesteps 2585.
Path 279 | total_timesteps 2596.
Path 280 | total_timesteps 2604.
Path 281 | total_timesteps 2616.
Path 282 | total_timesteps 2630.
Path 283 | total_timesteps 2649.
Path 284 | total_timesteps 2664.
Path 285 | total_timesteps 2673.
Path 286 | total_timesteps 2685.
Path 287 | total_timesteps 2693.
Path 288 | total_timesteps 2704.
Path 289 | total_timesteps 2711.
Path 290 | total_timesteps 2719.
Path 291 | total_timesteps 2729.
Path 292 | total_timesteps 2737.
Path 293 | total_timesteps 2745.
Path 294 | total_timesteps 2755.
Path 295 | total_timesteps 2765.
Path 296 | total_timesteps 2773.
Path 297 | total_timesteps 2782.
Path 298 | total_timesteps 2793.
Path 299 | total_timesteps 2800.
Path 300 | total_timesteps 2809.
Path 301 | total_timesteps 2818.
Path 302 | total_timesteps 2824.
Path 303 | total_timesteps 2830.
Path 304 | total_timesteps 2838.
Path 305 | total_timesteps 2849.
Path 306 | total_timesteps 2857.
Path 307 | total_timesteps 2868.
Path 308 | total_timesteps 2878.
Path 309 | total_timesteps 2892.
Path 310 | total_timesteps 2901.
Path 311 | total_timesteps 2910.
Path 312 | total_timesteps 2917.
Path 313 | total_timesteps 2927.
Path 314 | total_timesteps 2939.
Path 315 | total_timesteps 2949.
Path 316 | total_timesteps 2962.
Path 317 | total_timesteps 2971.
Path 318 | total_timesteps 2986.
Path 319 | total_timesteps 2993.
Path 320 | total_timesteps 3002.
Path 321 | total_timesteps 3009.
Path 322 | total_timesteps 3018.
Path 323 | total_timesteps 3033.
Path 324 | total_timesteps 3039.
Path 325 | total_timesteps 3046.
Path 326 | total_timesteps 3055.
Path 327 | total_timesteps 3065.
Path 328 | total_timesteps 3074.
Path 329 | total_timesteps 3089.
Path 330 | total_timesteps 3097.
Path 331 | total_timesteps 3104.
Path 332 | total_timesteps 3116.
Path 333 | total_timesteps 3127.
Path 334 | total_timesteps 3137.
Path 335 | total_timesteps 3145.
Path 336 | total_timesteps 3153.
Path 337 | total_timesteps 3163.
Path 338 | total_timesteps 3171.
Path 339 | total_timesteps 3179.
Path 340 | total_timesteps 3186.
Path 341 | total_timesteps 3195.
Path 342 | total_timesteps 3207.
Path 343 | total_timesteps 3220.
Path 344 | total_timesteps 3229.
Path 345 | total_timesteps 3237.
Path 346 | total_timesteps 3245.
Path 347 | total_timesteps 3253.
Path 348 | total_timesteps 3260.
Path 349 | total_timesteps 3274.
Path 350 | total_timesteps 3285.
Path 351 | total_timesteps 3298.
Path 352 | total_timesteps 3306.
Path 353 | total_timesteps 3312.
Path 354 | total_timesteps 3320.
Path 355 | total_timesteps 3328.
Path 356 | total_timesteps 3337.
Path 357 | total_timesteps 3344.
Path 358 | total_timesteps 3357.
Path 359 | total_timesteps 3368.
Path 360 | total_timesteps 3380.
Path 361 | total_timesteps 3389.
Path 362 | total_timesteps 3398.
Path 363 | total_timesteps 3406.
Path 364 | total_timesteps 3413.
Path 365 | total_timesteps 3421.
Path 366 | total_timesteps 3431.
Path 367 | total_timesteps 3440.
Path 368 | total_timesteps 3448.
Path 369 | total_timesteps 3454.
Path 370 | total_timesteps 3463.
Path 371 | total_timesteps 3472.
Path 372 | total_timesteps 3479.
Path 373 | total_timesteps 3488.
Path 374 | total_timesteps 3497.
Path 375 | total_timesteps 3508.
Path 376 | total_timesteps 3519.
Path 377 | total_timesteps 3534.
Path 378 | total_timesteps 3542.
Path 379 | total_timesteps 3549.
Path 380 | total_timesteps 3556.
Path 381 | total_timesteps 3563.
Path 382 | total_timesteps 3570.
Path 383 | total_timesteps 3579.
Path 384 | total_timesteps 3586.
Path 385 | total_timesteps 3594.
Path 386 | total_timesteps 3603.
Path 387 | total_timesteps 3614.
Path 388 | total_timesteps 3626.
Path 389 | total_timesteps 3632.
Path 390 | total_timesteps 3638.
Path 391 | total_timesteps 3647.
Path 392 | total_timesteps 3655.
Path 393 | total_timesteps 3669.
Path 394 | total_timesteps 3678.
Path 395 | total_timesteps 3686.
Path 396 | total_timesteps 3707.
Path 397 | total_timesteps 3716.
Path 398 | total_timesteps 3725.
Path 399 | total_timesteps 3747.
Path 400 | total_timesteps 3759.
Path 401 | total_timesteps 3771.
Path 402 | total_timesteps 3782.
Path 403 | total_timesteps 3794.
Path 404 | total_timesteps 3808.
Path 405 | total_timesteps 3815.
Path 406 | total_timesteps 3822.
Path 407 | total_timesteps 3829.
Path 408 | total_timesteps 3837.
Path 409 | total_timesteps 3846.
Path 410 | total_timesteps 3856.
Path 411 | total_timesteps 3863.
Path 412 | total_timesteps 3872.
Path 413 | total_timesteps 3882.
Path 414 | total_timesteps 3894.
Path 415 | total_timesteps 3911.
Path 416 | total_timesteps 3919.
Path 417 | total_timesteps 3925.
Path 418 | total_timesteps 3943.
Path 419 | total_timesteps 3951.
Path 420 | total_timesteps 3960.
Path 421 | total_timesteps 3967.
Path 422 | total_timesteps 3979.
Path 423 | total_timesteps 3986.
Path 424 | total_timesteps 3993.
Path 425 | total_timesteps 4008.
Path 426 | total_timesteps 4020.
Path 427 | total_timesteps 4030.
Path 428 | total_timesteps 4041.
Path 429 | total_timesteps 4048.
Path 430 | total_timesteps 4058.
Path 431 | total_timesteps 4066.
Path 432 | total_timesteps 4075.
Path 433 | total_timesteps 4084.
Path 434 | total_timesteps 4092.
Path 435 | total_timesteps 4100.
Path 436 | total_timesteps 4108.
Path 437 | total_timesteps 4115.
Path 438 | total_timesteps 4124.
Path 439 | total_timesteps 4134.
Path 440 | total_timesteps 4142.
Path 441 | total_timesteps 4153.
Path 442 | total_timesteps 4162.
Path 443 | total_timesteps 4170.
Path 444 | total_timesteps 4177.
Path 445 | total_timesteps 4189.
Path 446 | total_timesteps 4197.
Path 447 | total_timesteps 4207.
Path 448 | total_timesteps 4221.
Path 449 | total_timesteps 4231.
Path 450 | total_timesteps 4239.
Path 451 | total_timesteps 4248.
Path 452 | total_timesteps 4257.
Path 453 | total_timesteps 4267.
Path 454 | total_timesteps 4276.
Path 455 | total_timesteps 4283.
Path 456 | total_timesteps 4293.
Path 457 | total_timesteps 4303.
Path 458 | total_timesteps 4314.
Path 459 | total_timesteps 4320.
Path 460 | total_timesteps 4328.
Path 461 | total_timesteps 4337.
Path 462 | total_timesteps 4348.
Path 463 | total_timesteps 4358.
Path 464 | total_timesteps 4365.
Path 465 | total_timesteps 4374.
Path 466 | total_timesteps 4382.
Path 467 | total_timesteps 4395.
Path 468 | total_timesteps 4403.
Path 469 | total_timesteps 4419.
Path 470 | total_timesteps 4433.
Path 471 | total_timesteps 4442.
Path 472 | total_timesteps 4451.
Path 473 | total_timesteps 4458.
Path 474 | total_timesteps 4465.
Path 475 | total_timesteps 4479.
Path 476 | total_timesteps 4491.
Path 477 | total_timesteps 4501.
Path 478 | total_timesteps 4513.
Path 479 | total_timesteps 4522.
Path 480 | total_timesteps 4536.
Path 481 | total_timesteps 4544.
Path 482 | total_timesteps 4553.
Path 483 | total_timesteps 4560.
Path 484 | total_timesteps 4568.
Path 485 | total_timesteps 4580.
Path 486 | total_timesteps 4586.
Path 487 | total_timesteps 4600.
Path 488 | total_timesteps 4611.
Path 489 | total_timesteps 4618.
Path 490 | total_timesteps 4624.
Path 491 | total_timesteps 4631.
Path 492 | total_timesteps 4640.
Path 493 | total_timesteps 4649.
Path 494 | total_timesteps 4657.
Path 495 | total_timesteps 4665.
Path 496 | total_timesteps 4674.
Path 497 | total_timesteps 4685.
Path 498 | total_timesteps 4693.
Path 499 | total_timesteps 4701.
Path 500 | total_timesteps 4710.
Path 501 | total_timesteps 4716.
Path 502 | total_timesteps 4724.
Path 503 | total_timesteps 4734.
Path 504 | total_timesteps 4741.
Path 505 | total_timesteps 4750.
Path 506 | total_timesteps 4758.
Path 507 | total_timesteps 4766.
Path 508 | total_timesteps 4780.
Path 509 | total_timesteps 4792.
Path 510 | total_timesteps 4802.
Path 511 | total_timesteps 4809.
Path 512 | total_timesteps 4820.
Path 513 | total_timesteps 4828.
Path 514 | total_timesteps 4842.
Path 515 | total_timesteps 4849.
Path 516 | total_timesteps 4857.
Path 517 | total_timesteps 4872.
Path 518 | total_timesteps 4886.
Path 519 | total_timesteps 4894.
Path 520 | total_timesteps 4904.
Path 521 | total_timesteps 4911.
Path 522 | total_timesteps 4918.
Path 523 | total_timesteps 4929.
Path 524 | total_timesteps 4936.
Path 525 | total_timesteps 4945.
Path 526 | total_timesteps 4952.
Path 527 | total_timesteps 4963.
Path 528 | total_timesteps 4973.
Path 529 | total_timesteps 4984.
Path 530 | total_timesteps 4994.
Path 531 | total_timesteps 5002.
Path 532 | total_timesteps 5009.
Path 533 | total_timesteps 5021.
Path 534 | total_timesteps 5029.
Path 535 | total_timesteps 5037.
Path 536 | total_timesteps 5044.
Path 537 | total_timesteps 5053.
Path 538 | total_timesteps 5069.
Path 539 | total_timesteps 5077.
Path 540 | total_timesteps 5085.
Path 541 | total_timesteps 5092.
Path 542 | total_timesteps 5099.
Path 543 | total_timesteps 5111.
Path 544 | total_timesteps 5125.
Path 545 | total_timesteps 5134.
Path 546 | total_timesteps 5149.
Path 547 | total_timesteps 5156.
Path 548 | total_timesteps 5169.
Path 549 | total_timesteps 5178.
Path 550 | total_timesteps 5185.
Path 551 | total_timesteps 5196.
Path 552 | total_timesteps 5210.
Path 553 | total_timesteps 5219.
Path 554 | total_timesteps 5227.
Path 555 | total_timesteps 5234.
Path 556 | total_timesteps 5243.
Path 557 | total_timesteps 5252.
Path 558 | total_timesteps 5260.
Path 559 | total_timesteps 5268.
Path 560 | total_timesteps 5274.
Path 561 | total_timesteps 5284.
Path 562 | total_timesteps 5296.
Path 563 | total_timesteps 5304.
Path 564 | total_timesteps 5311.
Path 565 | total_timesteps 5329.
Path 566 | total_timesteps 5337.
Path 567 | total_timesteps 5345.
Path 568 | total_timesteps 5352.
Path 569 | total_timesteps 5361.
Path 570 | total_timesteps 5368.
Path 571 | total_timesteps 5377.
Path 572 | total_timesteps 5387.
Path 573 | total_timesteps 5395.
Path 574 | total_timesteps 5403.
Path 575 | total_timesteps 5410.
Path 576 | total_timesteps 5418.
Path 577 | total_timesteps 5425.
Path 578 | total_timesteps 5437.
Path 579 | total_timesteps 5448.
Path 580 | total_timesteps 5456.
Path 581 | total_timesteps 5465.
Path 582 | total_timesteps 5476.
Path 583 | total_timesteps 5483.
Path 584 | total_timesteps 5493.
Path 585 | total_timesteps 5504.
Path 586 | total_timesteps 5512.
Path 587 | total_timesteps 5522.
Path 588 | total_timesteps 5529.
Path 589 | total_timesteps 5537.
Path 590 | total_timesteps 5547.
Path 591 | total_timesteps 5556.
Path 592 | total_timesteps 5564.
Path 593 | total_timesteps 5576.
Path 594 | total_timesteps 5585.
Path 595 | total_timesteps 5596.
Path 596 | total_timesteps 5603.
Path 597 | total_timesteps 5609.
Path 598 | total_timesteps 5617.
Path 599 | total_timesteps 5624.
Path 600 | total_timesteps 5630.
Path 601 | total_timesteps 5645.
Path 602 | total_timesteps 5663.
Path 603 | total_timesteps 5670.
Path 604 | total_timesteps 5677.
Path 605 | total_timesteps 5686.
Path 606 | total_timesteps 5696.
Path 607 | total_timesteps 5704.
Path 608 | total_timesteps 5711.
Path 609 | total_timesteps 5718.
Path 610 | total_timesteps 5725.
Path 611 | total_timesteps 5731.
Path 612 | total_timesteps 5745.
Path 613 | total_timesteps 5756.
Path 614 | total_timesteps 5763.
Path 615 | total_timesteps 5773.
Path 616 | total_timesteps 5784.
Path 617 | total_timesteps 5795.
Path 618 | total_timesteps 5802.
Path 619 | total_timesteps 5809.
Path 620 | total_timesteps 5818.
Path 621 | total_timesteps 5826.
Path 622 | total_timesteps 5839.
Path 623 | total_timesteps 5847.
Path 624 | total_timesteps 5854.
Path 625 | total_timesteps 5865.
Path 626 | total_timesteps 5876.
Path 627 | total_timesteps 5882.
Path 628 | total_timesteps 5889.
Path 629 | total_timesteps 5895.
Path 630 | total_timesteps 5912.
Path 631 | total_timesteps 5920.
Path 632 | total_timesteps 5926.
Path 633 | total_timesteps 5936.
Path 634 | total_timesteps 5953.
Path 635 | total_timesteps 5961.
Path 636 | total_timesteps 5973.
Path 637 | total_timesteps 5982.
Path 638 | total_timesteps 5991.
Path 639 | total_timesteps 5999.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.17    |
| Iteration     | 30       |
| MaximumReturn | 1.23     |
| MinimumReturn | -18.3    |
| TotalSamples  | 128217   |
----------------------------
itr #31 | 
Fitting dynamics.
Validation loss = 0.0019484157674014568
Validation loss = 0.001939073670655489
Validation loss = 0.001965020317584276
Validation loss = 0.0018322090618312359
Validation loss = 0.0019825552590191364
Validation loss = 0.0019540435168892145
Validation loss = 0.0023776674643158913
Validation loss = 0.0018899617716670036
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 14.
Path 3 | total_timesteps 22.
Path 4 | total_timesteps 32.
Path 5 | total_timesteps 48.
Path 6 | total_timesteps 57.
Path 7 | total_timesteps 63.
Path 8 | total_timesteps 79.
Path 9 | total_timesteps 89.
Path 10 | total_timesteps 101.
Path 11 | total_timesteps 111.
Path 12 | total_timesteps 118.
Path 13 | total_timesteps 125.
Path 14 | total_timesteps 132.
Path 15 | total_timesteps 140.
Path 16 | total_timesteps 147.
Path 17 | total_timesteps 154.
Path 18 | total_timesteps 162.
Path 19 | total_timesteps 170.
Path 20 | total_timesteps 177.
Path 21 | total_timesteps 183.
Path 22 | total_timesteps 192.
Path 23 | total_timesteps 201.
Path 24 | total_timesteps 209.
Path 25 | total_timesteps 223.
Path 26 | total_timesteps 230.
Path 27 | total_timesteps 237.
Path 28 | total_timesteps 248.
Path 29 | total_timesteps 254.
Path 30 | total_timesteps 261.
Path 31 | total_timesteps 268.
Path 32 | total_timesteps 279.
Path 33 | total_timesteps 291.
Path 34 | total_timesteps 300.
Path 35 | total_timesteps 309.
Path 36 | total_timesteps 321.
Path 37 | total_timesteps 328.
Path 38 | total_timesteps 338.
Path 39 | total_timesteps 345.
Path 40 | total_timesteps 352.
Path 41 | total_timesteps 359.
Path 42 | total_timesteps 368.
Path 43 | total_timesteps 375.
Path 44 | total_timesteps 383.
Path 45 | total_timesteps 393.
Path 46 | total_timesteps 406.
Path 47 | total_timesteps 413.
Path 48 | total_timesteps 423.
Path 49 | total_timesteps 429.
Path 50 | total_timesteps 435.
Path 51 | total_timesteps 443.
Path 52 | total_timesteps 451.
Path 53 | total_timesteps 459.
Path 54 | total_timesteps 467.
Path 55 | total_timesteps 487.
Path 56 | total_timesteps 495.
Path 57 | total_timesteps 502.
Path 58 | total_timesteps 509.
Path 59 | total_timesteps 517.
Path 60 | total_timesteps 524.
Path 61 | total_timesteps 535.
Path 62 | total_timesteps 544.
Path 63 | total_timesteps 550.
Path 64 | total_timesteps 561.
Path 65 | total_timesteps 576.
Path 66 | total_timesteps 584.
Path 67 | total_timesteps 590.
Path 68 | total_timesteps 601.
Path 69 | total_timesteps 607.
Path 70 | total_timesteps 618.
Path 71 | total_timesteps 629.
Path 72 | total_timesteps 638.
Path 73 | total_timesteps 647.
Path 74 | total_timesteps 657.
Path 75 | total_timesteps 664.
Path 76 | total_timesteps 673.
Path 77 | total_timesteps 683.
Path 78 | total_timesteps 693.
Path 79 | total_timesteps 710.
Path 80 | total_timesteps 717.
Path 81 | total_timesteps 728.
Path 82 | total_timesteps 737.
Path 83 | total_timesteps 745.
Path 84 | total_timesteps 752.
Path 85 | total_timesteps 762.
Path 86 | total_timesteps 768.
Path 87 | total_timesteps 777.
Path 88 | total_timesteps 783.
Path 89 | total_timesteps 790.
Path 90 | total_timesteps 802.
Path 91 | total_timesteps 810.
Path 92 | total_timesteps 817.
Path 93 | total_timesteps 827.
Path 94 | total_timesteps 834.
Path 95 | total_timesteps 841.
Path 96 | total_timesteps 850.
Path 97 | total_timesteps 858.
Path 98 | total_timesteps 866.
Path 99 | total_timesteps 873.
Path 100 | total_timesteps 882.
Path 101 | total_timesteps 890.
Path 102 | total_timesteps 901.
Path 103 | total_timesteps 909.
Path 104 | total_timesteps 926.
Path 105 | total_timesteps 932.
Path 106 | total_timesteps 943.
Path 107 | total_timesteps 950.
Path 108 | total_timesteps 960.
Path 109 | total_timesteps 970.
Path 110 | total_timesteps 977.
Path 111 | total_timesteps 986.
Path 112 | total_timesteps 994.
Path 113 | total_timesteps 1010.
Path 114 | total_timesteps 1017.
Path 115 | total_timesteps 1027.
Path 116 | total_timesteps 1036.
Path 117 | total_timesteps 1045.
Path 118 | total_timesteps 1055.
Path 119 | total_timesteps 1063.
Path 120 | total_timesteps 1070.
Path 121 | total_timesteps 1077.
Path 122 | total_timesteps 1087.
Path 123 | total_timesteps 1094.
Path 124 | total_timesteps 1102.
Path 125 | total_timesteps 1111.
Path 126 | total_timesteps 1118.
Path 127 | total_timesteps 1127.
Path 128 | total_timesteps 1137.
Path 129 | total_timesteps 1145.
Path 130 | total_timesteps 1152.
Path 131 | total_timesteps 1159.
Path 132 | total_timesteps 1166.
Path 133 | total_timesteps 1174.
Path 134 | total_timesteps 1183.
Path 135 | total_timesteps 1192.
Path 136 | total_timesteps 1201.
Path 137 | total_timesteps 1209.
Path 138 | total_timesteps 1220.
Path 139 | total_timesteps 1237.
Path 140 | total_timesteps 1246.
Path 141 | total_timesteps 1258.
Path 142 | total_timesteps 1265.
Path 143 | total_timesteps 1273.
Path 144 | total_timesteps 1281.
Path 145 | total_timesteps 1294.
Path 146 | total_timesteps 1301.
Path 147 | total_timesteps 1312.
Path 148 | total_timesteps 1322.
Path 149 | total_timesteps 1329.
Path 150 | total_timesteps 1342.
Path 151 | total_timesteps 1354.
Path 152 | total_timesteps 1363.
Path 153 | total_timesteps 1370.
Path 154 | total_timesteps 1378.
Path 155 | total_timesteps 1385.
Path 156 | total_timesteps 1393.
Path 157 | total_timesteps 1400.
Path 158 | total_timesteps 1407.
Path 159 | total_timesteps 1414.
Path 160 | total_timesteps 1426.
Path 161 | total_timesteps 1434.
Path 162 | total_timesteps 1441.
Path 163 | total_timesteps 1449.
Path 164 | total_timesteps 1458.
Path 165 | total_timesteps 1467.
Path 166 | total_timesteps 1473.
Path 167 | total_timesteps 1480.
Path 168 | total_timesteps 1486.
Path 169 | total_timesteps 1495.
Path 170 | total_timesteps 1503.
Path 171 | total_timesteps 1511.
Path 172 | total_timesteps 1518.
Path 173 | total_timesteps 1526.
Path 174 | total_timesteps 1533.
Path 175 | total_timesteps 1539.
Path 176 | total_timesteps 1545.
Path 177 | total_timesteps 1553.
Path 178 | total_timesteps 1559.
Path 179 | total_timesteps 1568.
Path 180 | total_timesteps 1578.
Path 181 | total_timesteps 1585.
Path 182 | total_timesteps 1592.
Path 183 | total_timesteps 1600.
Path 184 | total_timesteps 1609.
Path 185 | total_timesteps 1616.
Path 186 | total_timesteps 1623.
Path 187 | total_timesteps 1630.
Path 188 | total_timesteps 1638.
Path 189 | total_timesteps 1645.
Path 190 | total_timesteps 1656.
Path 191 | total_timesteps 1664.
Path 192 | total_timesteps 1671.
Path 193 | total_timesteps 1678.
Path 194 | total_timesteps 1686.
Path 195 | total_timesteps 1692.
Path 196 | total_timesteps 1701.
Path 197 | total_timesteps 1710.
Path 198 | total_timesteps 1718.
Path 199 | total_timesteps 1725.
Path 200 | total_timesteps 1732.
Path 201 | total_timesteps 1743.
Path 202 | total_timesteps 1753.
Path 203 | total_timesteps 1762.
Path 204 | total_timesteps 1771.
Path 205 | total_timesteps 1780.
Path 206 | total_timesteps 1792.
Path 207 | total_timesteps 1799.
Path 208 | total_timesteps 1809.
Path 209 | total_timesteps 1816.
Path 210 | total_timesteps 1824.
Path 211 | total_timesteps 1830.
Path 212 | total_timesteps 1839.
Path 213 | total_timesteps 1848.
Path 214 | total_timesteps 1858.
Path 215 | total_timesteps 1866.
Path 216 | total_timesteps 1880.
Path 217 | total_timesteps 1887.
Path 218 | total_timesteps 1894.
Path 219 | total_timesteps 1900.
Path 220 | total_timesteps 1906.
Path 221 | total_timesteps 1921.
Path 222 | total_timesteps 1936.
Path 223 | total_timesteps 1944.
Path 224 | total_timesteps 1954.
Path 225 | total_timesteps 1964.
Path 226 | total_timesteps 1975.
Path 227 | total_timesteps 1982.
Path 228 | total_timesteps 1989.
Path 229 | total_timesteps 1995.
Path 230 | total_timesteps 2005.
Path 231 | total_timesteps 2015.
Path 232 | total_timesteps 2022.
Path 233 | total_timesteps 2028.
Path 234 | total_timesteps 2035.
Path 235 | total_timesteps 2044.
Path 236 | total_timesteps 2051.
Path 237 | total_timesteps 2059.
Path 238 | total_timesteps 2068.
Path 239 | total_timesteps 2083.
Path 240 | total_timesteps 2094.
Path 241 | total_timesteps 2104.
Path 242 | total_timesteps 2111.
Path 243 | total_timesteps 2120.
Path 244 | total_timesteps 2128.
Path 245 | total_timesteps 2138.
Path 246 | total_timesteps 2153.
Path 247 | total_timesteps 2162.
Path 248 | total_timesteps 2169.
Path 249 | total_timesteps 2176.
Path 250 | total_timesteps 2184.
Path 251 | total_timesteps 2195.
Path 252 | total_timesteps 2206.
Path 253 | total_timesteps 2214.
Path 254 | total_timesteps 2223.
Path 255 | total_timesteps 2231.
Path 256 | total_timesteps 2238.
Path 257 | total_timesteps 2246.
Path 258 | total_timesteps 2255.
Path 259 | total_timesteps 2263.
Path 260 | total_timesteps 2273.
Path 261 | total_timesteps 2279.
Path 262 | total_timesteps 2290.
Path 263 | total_timesteps 2300.
Path 264 | total_timesteps 2307.
Path 265 | total_timesteps 2318.
Path 266 | total_timesteps 2327.
Path 267 | total_timesteps 2335.
Path 268 | total_timesteps 2346.
Path 269 | total_timesteps 2356.
Path 270 | total_timesteps 2366.
Path 271 | total_timesteps 2373.
Path 272 | total_timesteps 2381.
Path 273 | total_timesteps 2390.
Path 274 | total_timesteps 2404.
Path 275 | total_timesteps 2415.
Path 276 | total_timesteps 2423.
Path 277 | total_timesteps 2429.
Path 278 | total_timesteps 2435.
Path 279 | total_timesteps 2442.
Path 280 | total_timesteps 2449.
Path 281 | total_timesteps 2457.
Path 282 | total_timesteps 2469.
Path 283 | total_timesteps 2478.
Path 284 | total_timesteps 2486.
Path 285 | total_timesteps 2494.
Path 286 | total_timesteps 2501.
Path 287 | total_timesteps 2508.
Path 288 | total_timesteps 2515.
Path 289 | total_timesteps 2526.
Path 290 | total_timesteps 2534.
Path 291 | total_timesteps 2546.
Path 292 | total_timesteps 2554.
Path 293 | total_timesteps 2567.
Path 294 | total_timesteps 2576.
Path 295 | total_timesteps 2583.
Path 296 | total_timesteps 2592.
Path 297 | total_timesteps 2599.
Path 298 | total_timesteps 2619.
Path 299 | total_timesteps 2626.
Path 300 | total_timesteps 2633.
Path 301 | total_timesteps 2640.
Path 302 | total_timesteps 2647.
Path 303 | total_timesteps 2656.
Path 304 | total_timesteps 2663.
Path 305 | total_timesteps 2670.
Path 306 | total_timesteps 2678.
Path 307 | total_timesteps 2684.
Path 308 | total_timesteps 2690.
Path 309 | total_timesteps 2698.
Path 310 | total_timesteps 2704.
Path 311 | total_timesteps 2712.
Path 312 | total_timesteps 2720.
Path 313 | total_timesteps 2729.
Path 314 | total_timesteps 2740.
Path 315 | total_timesteps 2750.
Path 316 | total_timesteps 2758.
Path 317 | total_timesteps 2767.
Path 318 | total_timesteps 2776.
Path 319 | total_timesteps 2787.
Path 320 | total_timesteps 2796.
Path 321 | total_timesteps 2802.
Path 322 | total_timesteps 2814.
Path 323 | total_timesteps 2823.
Path 324 | total_timesteps 2830.
Path 325 | total_timesteps 2836.
Path 326 | total_timesteps 2844.
Path 327 | total_timesteps 2850.
Path 328 | total_timesteps 2858.
Path 329 | total_timesteps 2865.
Path 330 | total_timesteps 2872.
Path 331 | total_timesteps 2879.
Path 332 | total_timesteps 2890.
Path 333 | total_timesteps 2898.
Path 334 | total_timesteps 2905.
Path 335 | total_timesteps 2917.
Path 336 | total_timesteps 2929.
Path 337 | total_timesteps 2942.
Path 338 | total_timesteps 2950.
Path 339 | total_timesteps 2958.
Path 340 | total_timesteps 2967.
Path 341 | total_timesteps 2977.
Path 342 | total_timesteps 2986.
Path 343 | total_timesteps 2995.
Path 344 | total_timesteps 3003.
Path 345 | total_timesteps 3014.
Path 346 | total_timesteps 3032.
Path 347 | total_timesteps 3041.
Path 348 | total_timesteps 3048.
Path 349 | total_timesteps 3055.
Path 350 | total_timesteps 3067.
Path 351 | total_timesteps 3076.
Path 352 | total_timesteps 3086.
Path 353 | total_timesteps 3095.
Path 354 | total_timesteps 3102.
Path 355 | total_timesteps 3108.
Path 356 | total_timesteps 3116.
Path 357 | total_timesteps 3128.
Path 358 | total_timesteps 3138.
Path 359 | total_timesteps 3147.
Path 360 | total_timesteps 3155.
Path 361 | total_timesteps 3166.
Path 362 | total_timesteps 3174.
Path 363 | total_timesteps 3181.
Path 364 | total_timesteps 3190.
Path 365 | total_timesteps 3203.
Path 366 | total_timesteps 3211.
Path 367 | total_timesteps 3219.
Path 368 | total_timesteps 3225.
Path 369 | total_timesteps 3236.
Path 370 | total_timesteps 3246.
Path 371 | total_timesteps 3259.
Path 372 | total_timesteps 3266.
Path 373 | total_timesteps 3273.
Path 374 | total_timesteps 3279.
Path 375 | total_timesteps 3286.
Path 376 | total_timesteps 3294.
Path 377 | total_timesteps 3301.
Path 378 | total_timesteps 3309.
Path 379 | total_timesteps 3319.
Path 380 | total_timesteps 3326.
Path 381 | total_timesteps 3335.
Path 382 | total_timesteps 3348.
Path 383 | total_timesteps 3356.
Path 384 | total_timesteps 3362.
Path 385 | total_timesteps 3370.
Path 386 | total_timesteps 3380.
Path 387 | total_timesteps 3388.
Path 388 | total_timesteps 3398.
Path 389 | total_timesteps 3405.
Path 390 | total_timesteps 3416.
Path 391 | total_timesteps 3423.
Path 392 | total_timesteps 3431.
Path 393 | total_timesteps 3445.
Path 394 | total_timesteps 3470.
Path 395 | total_timesteps 3476.
Path 396 | total_timesteps 3484.
Path 397 | total_timesteps 3491.
Path 398 | total_timesteps 3500.
Path 399 | total_timesteps 3520.
Path 400 | total_timesteps 3529.
Path 401 | total_timesteps 3538.
Path 402 | total_timesteps 3545.
Path 403 | total_timesteps 3551.
Path 404 | total_timesteps 3559.
Path 405 | total_timesteps 3567.
Path 406 | total_timesteps 3581.
Path 407 | total_timesteps 3588.
Path 408 | total_timesteps 3597.
Path 409 | total_timesteps 3604.
Path 410 | total_timesteps 3611.
Path 411 | total_timesteps 3622.
Path 412 | total_timesteps 3628.
Path 413 | total_timesteps 3637.
Path 414 | total_timesteps 3645.
Path 415 | total_timesteps 3655.
Path 416 | total_timesteps 3668.
Path 417 | total_timesteps 3674.
Path 418 | total_timesteps 3682.
Path 419 | total_timesteps 3692.
Path 420 | total_timesteps 3700.
Path 421 | total_timesteps 3710.
Path 422 | total_timesteps 3722.
Path 423 | total_timesteps 3730.
Path 424 | total_timesteps 3737.
Path 425 | total_timesteps 3748.
Path 426 | total_timesteps 3757.
Path 427 | total_timesteps 3765.
Path 428 | total_timesteps 3771.
Path 429 | total_timesteps 3779.
Path 430 | total_timesteps 3786.
Path 431 | total_timesteps 3793.
Path 432 | total_timesteps 3803.
Path 433 | total_timesteps 3811.
Path 434 | total_timesteps 3822.
Path 435 | total_timesteps 3833.
Path 436 | total_timesteps 3843.
Path 437 | total_timesteps 3850.
Path 438 | total_timesteps 3861.
Path 439 | total_timesteps 3868.
Path 440 | total_timesteps 3880.
Path 441 | total_timesteps 3889.
Path 442 | total_timesteps 3896.
Path 443 | total_timesteps 3902.
Path 444 | total_timesteps 3909.
Path 445 | total_timesteps 3917.
Path 446 | total_timesteps 3926.
Path 447 | total_timesteps 3937.
Path 448 | total_timesteps 3944.
Path 449 | total_timesteps 3952.
Path 450 | total_timesteps 3961.
Path 451 | total_timesteps 3968.
Path 452 | total_timesteps 3976.
Path 453 | total_timesteps 3983.
Path 454 | total_timesteps 3991.
Path 455 | total_timesteps 4013.
Path 456 | total_timesteps 4020.
Path 457 | total_timesteps 4027.
Path 458 | total_timesteps 4035.
Path 459 | total_timesteps 4042.
Path 460 | total_timesteps 4053.
Path 461 | total_timesteps 4063.
Path 462 | total_timesteps 4081.
Path 463 | total_timesteps 4091.
Path 464 | total_timesteps 4100.
Path 465 | total_timesteps 4108.
Path 466 | total_timesteps 4118.
Path 467 | total_timesteps 4128.
Path 468 | total_timesteps 4135.
Path 469 | total_timesteps 4145.
Path 470 | total_timesteps 4152.
Path 471 | total_timesteps 4159.
Path 472 | total_timesteps 4168.
Path 473 | total_timesteps 4175.
Path 474 | total_timesteps 4183.
Path 475 | total_timesteps 4191.
Path 476 | total_timesteps 4204.
Path 477 | total_timesteps 4213.
Path 478 | total_timesteps 4221.
Path 479 | total_timesteps 4231.
Path 480 | total_timesteps 4238.
Path 481 | total_timesteps 4247.
Path 482 | total_timesteps 4256.
Path 483 | total_timesteps 4264.
Path 484 | total_timesteps 4271.
Path 485 | total_timesteps 4279.
Path 486 | total_timesteps 4286.
Path 487 | total_timesteps 4316.
Path 488 | total_timesteps 4323.
Path 489 | total_timesteps 4334.
Path 490 | total_timesteps 4341.
Path 491 | total_timesteps 4348.
Path 492 | total_timesteps 4356.
Path 493 | total_timesteps 4364.
Path 494 | total_timesteps 4373.
Path 495 | total_timesteps 4384.
Path 496 | total_timesteps 4401.
Path 497 | total_timesteps 4412.
Path 498 | total_timesteps 4418.
Path 499 | total_timesteps 4426.
Path 500 | total_timesteps 4437.
Path 501 | total_timesteps 4451.
Path 502 | total_timesteps 4458.
Path 503 | total_timesteps 4467.
Path 504 | total_timesteps 4474.
Path 505 | total_timesteps 4482.
Path 506 | total_timesteps 4497.
Path 507 | total_timesteps 4503.
Path 508 | total_timesteps 4511.
Path 509 | total_timesteps 4517.
Path 510 | total_timesteps 4527.
Path 511 | total_timesteps 4538.
Path 512 | total_timesteps 4547.
Path 513 | total_timesteps 4555.
Path 514 | total_timesteps 4562.
Path 515 | total_timesteps 4578.
Path 516 | total_timesteps 4585.
Path 517 | total_timesteps 4595.
Path 518 | total_timesteps 4602.
Path 519 | total_timesteps 4615.
Path 520 | total_timesteps 4622.
Path 521 | total_timesteps 4634.
Path 522 | total_timesteps 4642.
Path 523 | total_timesteps 4652.
Path 524 | total_timesteps 4660.
Path 525 | total_timesteps 4667.
Path 526 | total_timesteps 4674.
Path 527 | total_timesteps 4682.
Path 528 | total_timesteps 4689.
Path 529 | total_timesteps 4696.
Path 530 | total_timesteps 4706.
Path 531 | total_timesteps 4722.
Path 532 | total_timesteps 4729.
Path 533 | total_timesteps 4736.
Path 534 | total_timesteps 4748.
Path 535 | total_timesteps 4755.
Path 536 | total_timesteps 4763.
Path 537 | total_timesteps 4773.
Path 538 | total_timesteps 4788.
Path 539 | total_timesteps 4794.
Path 540 | total_timesteps 4801.
Path 541 | total_timesteps 4807.
Path 542 | total_timesteps 4814.
Path 543 | total_timesteps 4824.
Path 544 | total_timesteps 4834.
Path 545 | total_timesteps 4845.
Path 546 | total_timesteps 4855.
Path 547 | total_timesteps 4865.
Path 548 | total_timesteps 4873.
Path 549 | total_timesteps 4880.
Path 550 | total_timesteps 4887.
Path 551 | total_timesteps 4895.
Path 552 | total_timesteps 4904.
Path 553 | total_timesteps 4910.
Path 554 | total_timesteps 4918.
Path 555 | total_timesteps 4925.
Path 556 | total_timesteps 4934.
Path 557 | total_timesteps 4942.
Path 558 | total_timesteps 4952.
Path 559 | total_timesteps 4960.
Path 560 | total_timesteps 4970.
Path 561 | total_timesteps 4977.
Path 562 | total_timesteps 4985.
Path 563 | total_timesteps 4993.
Path 564 | total_timesteps 5008.
Path 565 | total_timesteps 5017.
Path 566 | total_timesteps 5023.
Path 567 | total_timesteps 5036.
Path 568 | total_timesteps 5044.
Path 569 | total_timesteps 5051.
Path 570 | total_timesteps 5057.
Path 571 | total_timesteps 5064.
Path 572 | total_timesteps 5073.
Path 573 | total_timesteps 5082.
Path 574 | total_timesteps 5090.
Path 575 | total_timesteps 5097.
Path 576 | total_timesteps 5111.
Path 577 | total_timesteps 5119.
Path 578 | total_timesteps 5126.
Path 579 | total_timesteps 5134.
Path 580 | total_timesteps 5141.
Path 581 | total_timesteps 5151.
Path 582 | total_timesteps 5159.
Path 583 | total_timesteps 5166.
Path 584 | total_timesteps 5177.
Path 585 | total_timesteps 5185.
Path 586 | total_timesteps 5195.
Path 587 | total_timesteps 5202.
Path 588 | total_timesteps 5209.
Path 589 | total_timesteps 5216.
Path 590 | total_timesteps 5223.
Path 591 | total_timesteps 5231.
Path 592 | total_timesteps 5239.
Path 593 | total_timesteps 5246.
Path 594 | total_timesteps 5255.
Path 595 | total_timesteps 5265.
Path 596 | total_timesteps 5276.
Path 597 | total_timesteps 5285.
Path 598 | total_timesteps 5294.
Path 599 | total_timesteps 5309.
Path 600 | total_timesteps 5317.
Path 601 | total_timesteps 5323.
Path 602 | total_timesteps 5334.
Path 603 | total_timesteps 5342.
Path 604 | total_timesteps 5350.
Path 605 | total_timesteps 5356.
Path 606 | total_timesteps 5367.
Path 607 | total_timesteps 5376.
Path 608 | total_timesteps 5384.
Path 609 | total_timesteps 5392.
Path 610 | total_timesteps 5400.
Path 611 | total_timesteps 5407.
Path 612 | total_timesteps 5416.
Path 613 | total_timesteps 5426.
Path 614 | total_timesteps 5437.
Path 615 | total_timesteps 5445.
Path 616 | total_timesteps 5456.
Path 617 | total_timesteps 5462.
Path 618 | total_timesteps 5468.
Path 619 | total_timesteps 5475.
Path 620 | total_timesteps 5482.
Path 621 | total_timesteps 5490.
Path 622 | total_timesteps 5499.
Path 623 | total_timesteps 5511.
Path 624 | total_timesteps 5519.
Path 625 | total_timesteps 5527.
Path 626 | total_timesteps 5537.
Path 627 | total_timesteps 5544.
Path 628 | total_timesteps 5552.
Path 629 | total_timesteps 5560.
Path 630 | total_timesteps 5576.
Path 631 | total_timesteps 5583.
Path 632 | total_timesteps 5590.
Path 633 | total_timesteps 5601.
Path 634 | total_timesteps 5610.
Path 635 | total_timesteps 5619.
Path 636 | total_timesteps 5626.
Path 637 | total_timesteps 5636.
Path 638 | total_timesteps 5644.
Path 639 | total_timesteps 5651.
Path 640 | total_timesteps 5658.
Path 641 | total_timesteps 5665.
Path 642 | total_timesteps 5674.
Path 643 | total_timesteps 5682.
Path 644 | total_timesteps 5696.
Path 645 | total_timesteps 5703.
Path 646 | total_timesteps 5712.
Path 647 | total_timesteps 5724.
Path 648 | total_timesteps 5731.
Path 649 | total_timesteps 5738.
Path 650 | total_timesteps 5746.
Path 651 | total_timesteps 5753.
Path 652 | total_timesteps 5761.
Path 653 | total_timesteps 5768.
Path 654 | total_timesteps 5778.
Path 655 | total_timesteps 5786.
Path 656 | total_timesteps 5794.
Path 657 | total_timesteps 5801.
Path 658 | total_timesteps 5810.
Path 659 | total_timesteps 5818.
Path 660 | total_timesteps 5827.
Path 661 | total_timesteps 5837.
Path 662 | total_timesteps 5846.
Path 663 | total_timesteps 5860.
Path 664 | total_timesteps 5873.
Path 665 | total_timesteps 5882.
Path 666 | total_timesteps 5890.
Path 667 | total_timesteps 5899.
Path 668 | total_timesteps 5906.
Path 669 | total_timesteps 5912.
Path 670 | total_timesteps 5918.
Path 671 | total_timesteps 5925.
Path 672 | total_timesteps 5932.
Path 673 | total_timesteps 5942.
Path 674 | total_timesteps 5953.
Path 675 | total_timesteps 5960.
Path 676 | total_timesteps 5976.
Path 677 | total_timesteps 5982.
Path 678 | total_timesteps 5991.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.23    |
| Iteration     | 31       |
| MaximumReturn | -0.00603 |
| MinimumReturn | -19.4    |
| TotalSamples  | 132219   |
----------------------------
itr #32 | 
Fitting dynamics.
Validation loss = 0.0018127689836546779
Validation loss = 0.0017978039104491472
Validation loss = 0.0017800768837332726
Validation loss = 0.0018330382881686091
Validation loss = 0.0018025897443294525
Validation loss = 0.0018688645213842392
Validation loss = 0.0019442373886704445
Done fitting dynamics.
Updating randomness.
Done updating randomness.
Training policy using TRPO.
Re-initialize init_std.
Obtaining samples for iteration 0...
Obtaining samples for iteration 1...
Obtaining samples for iteration 2...
Obtaining samples for iteration 3...
Obtaining samples for iteration 4...
Obtaining samples for iteration 5...
Obtaining samples for iteration 6...
Obtaining samples for iteration 7...
Obtaining samples for iteration 8...
Obtaining samples for iteration 9...
Obtaining samples for iteration 10...
Obtaining samples for iteration 11...
Obtaining samples for iteration 12...
Obtaining samples for iteration 13...
Obtaining samples for iteration 14...
Obtaining samples for iteration 15...
Obtaining samples for iteration 16...
Obtaining samples for iteration 17...
Obtaining samples for iteration 18...
Obtaining samples for iteration 19...
Done training policy.
Generating on-policy rollouts.
Path 0 | total_timesteps 0.
Path 1 | total_timesteps 7.
Path 2 | total_timesteps 16.
Path 3 | total_timesteps 26.
Path 4 | total_timesteps 34.
Path 5 | total_timesteps 41.
Path 6 | total_timesteps 48.
Path 7 | total_timesteps 55.
Path 8 | total_timesteps 63.
Path 9 | total_timesteps 71.
Path 10 | total_timesteps 78.
Path 11 | total_timesteps 85.
Path 12 | total_timesteps 92.
Path 13 | total_timesteps 100.
Path 14 | total_timesteps 108.
Path 15 | total_timesteps 114.
Path 16 | total_timesteps 123.
Path 17 | total_timesteps 132.
Path 18 | total_timesteps 143.
Path 19 | total_timesteps 150.
Path 20 | total_timesteps 158.
Path 21 | total_timesteps 165.
Path 22 | total_timesteps 172.
Path 23 | total_timesteps 179.
Path 24 | total_timesteps 191.
Path 25 | total_timesteps 200.
Path 26 | total_timesteps 209.
Path 27 | total_timesteps 217.
Path 28 | total_timesteps 224.
Path 29 | total_timesteps 231.
Path 30 | total_timesteps 238.
Path 31 | total_timesteps 245.
Path 32 | total_timesteps 255.
Path 33 | total_timesteps 262.
Path 34 | total_timesteps 271.
Path 35 | total_timesteps 278.
Path 36 | total_timesteps 284.
Path 37 | total_timesteps 293.
Path 38 | total_timesteps 299.
Path 39 | total_timesteps 306.
Path 40 | total_timesteps 313.
Path 41 | total_timesteps 321.
Path 42 | total_timesteps 333.
Path 43 | total_timesteps 340.
Path 44 | total_timesteps 349.
Path 45 | total_timesteps 356.
Path 46 | total_timesteps 366.
Path 47 | total_timesteps 375.
Path 48 | total_timesteps 389.
Path 49 | total_timesteps 396.
Path 50 | total_timesteps 404.
Path 51 | total_timesteps 412.
Path 52 | total_timesteps 421.
Path 53 | total_timesteps 430.
Path 54 | total_timesteps 440.
Path 55 | total_timesteps 447.
Path 56 | total_timesteps 456.
Path 57 | total_timesteps 464.
Path 58 | total_timesteps 477.
Path 59 | total_timesteps 486.
Path 60 | total_timesteps 492.
Path 61 | total_timesteps 498.
Path 62 | total_timesteps 505.
Path 63 | total_timesteps 512.
Path 64 | total_timesteps 519.
Path 65 | total_timesteps 527.
Path 66 | total_timesteps 535.
Path 67 | total_timesteps 542.
Path 68 | total_timesteps 550.
Path 69 | total_timesteps 557.
Path 70 | total_timesteps 566.
Path 71 | total_timesteps 574.
Path 72 | total_timesteps 583.
Path 73 | total_timesteps 593.
Path 74 | total_timesteps 601.
Path 75 | total_timesteps 607.
Path 76 | total_timesteps 615.
Path 77 | total_timesteps 630.
Path 78 | total_timesteps 637.
Path 79 | total_timesteps 650.
Path 80 | total_timesteps 666.
Path 81 | total_timesteps 678.
Path 82 | total_timesteps 684.
Path 83 | total_timesteps 691.
Path 84 | total_timesteps 698.
Path 85 | total_timesteps 711.
Path 86 | total_timesteps 719.
Path 87 | total_timesteps 726.
Path 88 | total_timesteps 732.
Path 89 | total_timesteps 740.
Path 90 | total_timesteps 749.
Path 91 | total_timesteps 758.
Path 92 | total_timesteps 775.
Path 93 | total_timesteps 783.
Path 94 | total_timesteps 790.
Path 95 | total_timesteps 798.
Path 96 | total_timesteps 810.
Path 97 | total_timesteps 823.
Path 98 | total_timesteps 834.
Path 99 | total_timesteps 841.
Path 100 | total_timesteps 849.
Path 101 | total_timesteps 858.
Path 102 | total_timesteps 865.
Path 103 | total_timesteps 877.
Path 104 | total_timesteps 884.
Path 105 | total_timesteps 892.
Path 106 | total_timesteps 898.
Path 107 | total_timesteps 908.
Path 108 | total_timesteps 919.
Path 109 | total_timesteps 926.
Path 110 | total_timesteps 932.
Path 111 | total_timesteps 941.
Path 112 | total_timesteps 948.
Path 113 | total_timesteps 959.
Path 114 | total_timesteps 970.
Path 115 | total_timesteps 977.
Path 116 | total_timesteps 987.
Path 117 | total_timesteps 994.
Path 118 | total_timesteps 1003.
Path 119 | total_timesteps 1014.
Path 120 | total_timesteps 1026.
Path 121 | total_timesteps 1032.
Path 122 | total_timesteps 1041.
Path 123 | total_timesteps 1048.
Path 124 | total_timesteps 1056.
Path 125 | total_timesteps 1062.
Path 126 | total_timesteps 1068.
Path 127 | total_timesteps 1074.
Path 128 | total_timesteps 1084.
Path 129 | total_timesteps 1091.
Path 130 | total_timesteps 1099.
Path 131 | total_timesteps 1106.
Path 132 | total_timesteps 1113.
Path 133 | total_timesteps 1119.
Path 134 | total_timesteps 1126.
Path 135 | total_timesteps 1133.
Path 136 | total_timesteps 1140.
Path 137 | total_timesteps 1150.
Path 138 | total_timesteps 1158.
Path 139 | total_timesteps 1170.
Path 140 | total_timesteps 1178.
Path 141 | total_timesteps 1189.
Path 142 | total_timesteps 1196.
Path 143 | total_timesteps 1206.
Path 144 | total_timesteps 1213.
Path 145 | total_timesteps 1223.
Path 146 | total_timesteps 1231.
Path 147 | total_timesteps 1237.
Path 148 | total_timesteps 1243.
Path 149 | total_timesteps 1249.
Path 150 | total_timesteps 1255.
Path 151 | total_timesteps 1263.
Path 152 | total_timesteps 1274.
Path 153 | total_timesteps 1280.
Path 154 | total_timesteps 1290.
Path 155 | total_timesteps 1297.
Path 156 | total_timesteps 1305.
Path 157 | total_timesteps 1313.
Path 158 | total_timesteps 1320.
Path 159 | total_timesteps 1334.
Path 160 | total_timesteps 1341.
Path 161 | total_timesteps 1348.
Path 162 | total_timesteps 1357.
Path 163 | total_timesteps 1364.
Path 164 | total_timesteps 1374.
Path 165 | total_timesteps 1381.
Path 166 | total_timesteps 1389.
Path 167 | total_timesteps 1396.
Path 168 | total_timesteps 1407.
Path 169 | total_timesteps 1413.
Path 170 | total_timesteps 1421.
Path 171 | total_timesteps 1429.
Path 172 | total_timesteps 1438.
Path 173 | total_timesteps 1446.
Path 174 | total_timesteps 1453.
Path 175 | total_timesteps 1460.
Path 176 | total_timesteps 1467.
Path 177 | total_timesteps 1474.
Path 178 | total_timesteps 1482.
Path 179 | total_timesteps 1495.
Path 180 | total_timesteps 1506.
Path 181 | total_timesteps 1516.
Path 182 | total_timesteps 1525.
Path 183 | total_timesteps 1532.
Path 184 | total_timesteps 1540.
Path 185 | total_timesteps 1552.
Path 186 | total_timesteps 1560.
Path 187 | total_timesteps 1566.
Path 188 | total_timesteps 1575.
Path 189 | total_timesteps 1582.
Path 190 | total_timesteps 1588.
Path 191 | total_timesteps 1598.
Path 192 | total_timesteps 1605.
Path 193 | total_timesteps 1612.
Path 194 | total_timesteps 1618.
Path 195 | total_timesteps 1626.
Path 196 | total_timesteps 1633.
Path 197 | total_timesteps 1647.
Path 198 | total_timesteps 1653.
Path 199 | total_timesteps 1663.
Path 200 | total_timesteps 1671.
Path 201 | total_timesteps 1678.
Path 202 | total_timesteps 1685.
Path 203 | total_timesteps 1692.
Path 204 | total_timesteps 1700.
Path 205 | total_timesteps 1707.
Path 206 | total_timesteps 1713.
Path 207 | total_timesteps 1721.
Path 208 | total_timesteps 1729.
Path 209 | total_timesteps 1737.
Path 210 | total_timesteps 1748.
Path 211 | total_timesteps 1757.
Path 212 | total_timesteps 1764.
Path 213 | total_timesteps 1772.
Path 214 | total_timesteps 1780.
Path 215 | total_timesteps 1786.
Path 216 | total_timesteps 1802.
Path 217 | total_timesteps 1810.
Path 218 | total_timesteps 1819.
Path 219 | total_timesteps 1829.
Path 220 | total_timesteps 1836.
Path 221 | total_timesteps 1842.
Path 222 | total_timesteps 1854.
Path 223 | total_timesteps 1862.
Path 224 | total_timesteps 1871.
Path 225 | total_timesteps 1879.
Path 226 | total_timesteps 1886.
Path 227 | total_timesteps 1896.
Path 228 | total_timesteps 1903.
Path 229 | total_timesteps 1910.
Path 230 | total_timesteps 1917.
Path 231 | total_timesteps 1926.
Path 232 | total_timesteps 1933.
Path 233 | total_timesteps 1941.
Path 234 | total_timesteps 1948.
Path 235 | total_timesteps 1955.
Path 236 | total_timesteps 1961.
Path 237 | total_timesteps 1971.
Path 238 | total_timesteps 1979.
Path 239 | total_timesteps 1986.
Path 240 | total_timesteps 1996.
Path 241 | total_timesteps 2006.
Path 242 | total_timesteps 2012.
Path 243 | total_timesteps 2018.
Path 244 | total_timesteps 2026.
Path 245 | total_timesteps 2033.
Path 246 | total_timesteps 2041.
Path 247 | total_timesteps 2049.
Path 248 | total_timesteps 2058.
Path 249 | total_timesteps 2069.
Path 250 | total_timesteps 2075.
Path 251 | total_timesteps 2082.
Path 252 | total_timesteps 2089.
Path 253 | total_timesteps 2095.
Path 254 | total_timesteps 2104.
Path 255 | total_timesteps 2111.
Path 256 | total_timesteps 2118.
Path 257 | total_timesteps 2124.
Path 258 | total_timesteps 2132.
Path 259 | total_timesteps 2140.
Path 260 | total_timesteps 2146.
Path 261 | total_timesteps 2157.
Path 262 | total_timesteps 2164.
Path 263 | total_timesteps 2170.
Path 264 | total_timesteps 2180.
Path 265 | total_timesteps 2190.
Path 266 | total_timesteps 2196.
Path 267 | total_timesteps 2208.
Path 268 | total_timesteps 2214.
Path 269 | total_timesteps 2220.
Path 270 | total_timesteps 2228.
Path 271 | total_timesteps 2237.
Path 272 | total_timesteps 2247.
Path 273 | total_timesteps 2258.
Path 274 | total_timesteps 2272.
Path 275 | total_timesteps 2282.
Path 276 | total_timesteps 2291.
Path 277 | total_timesteps 2300.
Path 278 | total_timesteps 2306.
Path 279 | total_timesteps 2315.
Path 280 | total_timesteps 2322.
Path 281 | total_timesteps 2331.
Path 282 | total_timesteps 2338.
Path 283 | total_timesteps 2345.
Path 284 | total_timesteps 2353.
Path 285 | total_timesteps 2360.
Path 286 | total_timesteps 2370.
Path 287 | total_timesteps 2378.
Path 288 | total_timesteps 2384.
Path 289 | total_timesteps 2394.
Path 290 | total_timesteps 2405.
Path 291 | total_timesteps 2412.
Path 292 | total_timesteps 2421.
Path 293 | total_timesteps 2427.
Path 294 | total_timesteps 2435.
Path 295 | total_timesteps 2445.
Path 296 | total_timesteps 2454.
Path 297 | total_timesteps 2461.
Path 298 | total_timesteps 2472.
Path 299 | total_timesteps 2479.
Path 300 | total_timesteps 2488.
Path 301 | total_timesteps 2502.
Path 302 | total_timesteps 2509.
Path 303 | total_timesteps 2515.
Path 304 | total_timesteps 2525.
Path 305 | total_timesteps 2534.
Path 306 | total_timesteps 2549.
Path 307 | total_timesteps 2558.
Path 308 | total_timesteps 2567.
Path 309 | total_timesteps 2574.
Path 310 | total_timesteps 2583.
Path 311 | total_timesteps 2590.
Path 312 | total_timesteps 2599.
Path 313 | total_timesteps 2607.
Path 314 | total_timesteps 2614.
Path 315 | total_timesteps 2623.
Path 316 | total_timesteps 2630.
Path 317 | total_timesteps 2640.
Path 318 | total_timesteps 2646.
Path 319 | total_timesteps 2654.
Path 320 | total_timesteps 2661.
Path 321 | total_timesteps 2669.
Path 322 | total_timesteps 2675.
Path 323 | total_timesteps 2685.
Path 324 | total_timesteps 2694.
Path 325 | total_timesteps 2702.
Path 326 | total_timesteps 2709.
Path 327 | total_timesteps 2717.
Path 328 | total_timesteps 2726.
Path 329 | total_timesteps 2733.
Path 330 | total_timesteps 2742.
Path 331 | total_timesteps 2748.
Path 332 | total_timesteps 2755.
Path 333 | total_timesteps 2763.
Path 334 | total_timesteps 2769.
Path 335 | total_timesteps 2780.
Path 336 | total_timesteps 2789.
Path 337 | total_timesteps 2796.
Path 338 | total_timesteps 2805.
Path 339 | total_timesteps 2812.
Path 340 | total_timesteps 2819.
Path 341 | total_timesteps 2826.
Path 342 | total_timesteps 2837.
Path 343 | total_timesteps 2845.
Path 344 | total_timesteps 2852.
Path 345 | total_timesteps 2859.
Path 346 | total_timesteps 2868.
Path 347 | total_timesteps 2875.
Path 348 | total_timesteps 2881.
Path 349 | total_timesteps 2890.
Path 350 | total_timesteps 2899.
Path 351 | total_timesteps 2905.
Path 352 | total_timesteps 2913.
Path 353 | total_timesteps 2921.
Path 354 | total_timesteps 2928.
Path 355 | total_timesteps 2936.
Path 356 | total_timesteps 2944.
Path 357 | total_timesteps 2950.
Path 358 | total_timesteps 2957.
Path 359 | total_timesteps 2970.
Path 360 | total_timesteps 2976.
Path 361 | total_timesteps 2984.
Path 362 | total_timesteps 2992.
Path 363 | total_timesteps 2998.
Path 364 | total_timesteps 3009.
Path 365 | total_timesteps 3019.
Path 366 | total_timesteps 3026.
Path 367 | total_timesteps 3038.
Path 368 | total_timesteps 3050.
Path 369 | total_timesteps 3058.
Path 370 | total_timesteps 3065.
Path 371 | total_timesteps 3073.
Path 372 | total_timesteps 3083.
Path 373 | total_timesteps 3090.
Path 374 | total_timesteps 3097.
Path 375 | total_timesteps 3103.
Path 376 | total_timesteps 3110.
Path 377 | total_timesteps 3117.
Path 378 | total_timesteps 3127.
Path 379 | total_timesteps 3134.
Path 380 | total_timesteps 3143.
Path 381 | total_timesteps 3155.
Path 382 | total_timesteps 3165.
Path 383 | total_timesteps 3174.
Path 384 | total_timesteps 3192.
Path 385 | total_timesteps 3202.
Path 386 | total_timesteps 3208.
Path 387 | total_timesteps 3218.
Path 388 | total_timesteps 3226.
Path 389 | total_timesteps 3233.
Path 390 | total_timesteps 3241.
Path 391 | total_timesteps 3252.
Path 392 | total_timesteps 3259.
Path 393 | total_timesteps 3267.
Path 394 | total_timesteps 3280.
Path 395 | total_timesteps 3291.
Path 396 | total_timesteps 3304.
Path 397 | total_timesteps 3311.
Path 398 | total_timesteps 3320.
Path 399 | total_timesteps 3326.
Path 400 | total_timesteps 3337.
Path 401 | total_timesteps 3344.
Path 402 | total_timesteps 3351.
Path 403 | total_timesteps 3361.
Path 404 | total_timesteps 3367.
Path 405 | total_timesteps 3380.
Path 406 | total_timesteps 3390.
Path 407 | total_timesteps 3397.
Path 408 | total_timesteps 3403.
Path 409 | total_timesteps 3410.
Path 410 | total_timesteps 3418.
Path 411 | total_timesteps 3426.
Path 412 | total_timesteps 3433.
Path 413 | total_timesteps 3441.
Path 414 | total_timesteps 3448.
Path 415 | total_timesteps 3456.
Path 416 | total_timesteps 3470.
Path 417 | total_timesteps 3478.
Path 418 | total_timesteps 3485.
Path 419 | total_timesteps 3493.
Path 420 | total_timesteps 3499.
Path 421 | total_timesteps 3509.
Path 422 | total_timesteps 3519.
Path 423 | total_timesteps 3528.
Path 424 | total_timesteps 3542.
Path 425 | total_timesteps 3548.
Path 426 | total_timesteps 3557.
Path 427 | total_timesteps 3564.
Path 428 | total_timesteps 3575.
Path 429 | total_timesteps 3582.
Path 430 | total_timesteps 3589.
Path 431 | total_timesteps 3596.
Path 432 | total_timesteps 3604.
Path 433 | total_timesteps 3611.
Path 434 | total_timesteps 3621.
Path 435 | total_timesteps 3628.
Path 436 | total_timesteps 3635.
Path 437 | total_timesteps 3641.
Path 438 | total_timesteps 3649.
Path 439 | total_timesteps 3656.
Path 440 | total_timesteps 3662.
Path 441 | total_timesteps 3671.
Path 442 | total_timesteps 3680.
Path 443 | total_timesteps 3690.
Path 444 | total_timesteps 3696.
Path 445 | total_timesteps 3703.
Path 446 | total_timesteps 3715.
Path 447 | total_timesteps 3721.
Path 448 | total_timesteps 3729.
Path 449 | total_timesteps 3737.
Path 450 | total_timesteps 3744.
Path 451 | total_timesteps 3750.
Path 452 | total_timesteps 3759.
Path 453 | total_timesteps 3765.
Path 454 | total_timesteps 3780.
Path 455 | total_timesteps 3788.
Path 456 | total_timesteps 3795.
Path 457 | total_timesteps 3802.
Path 458 | total_timesteps 3813.
Path 459 | total_timesteps 3822.
Path 460 | total_timesteps 3828.
Path 461 | total_timesteps 3835.
Path 462 | total_timesteps 3841.
Path 463 | total_timesteps 3849.
Path 464 | total_timesteps 3856.
Path 465 | total_timesteps 3865.
Path 466 | total_timesteps 3872.
Path 467 | total_timesteps 3884.
Path 468 | total_timesteps 3899.
Path 469 | total_timesteps 3905.
Path 470 | total_timesteps 3912.
Path 471 | total_timesteps 3920.
Path 472 | total_timesteps 3928.
Path 473 | total_timesteps 3935.
Path 474 | total_timesteps 3941.
Path 475 | total_timesteps 3950.
Path 476 | total_timesteps 3956.
Path 477 | total_timesteps 3962.
Path 478 | total_timesteps 3968.
Path 479 | total_timesteps 3975.
Path 480 | total_timesteps 3982.
Path 481 | total_timesteps 3989.
Path 482 | total_timesteps 3997.
Path 483 | total_timesteps 4006.
Path 484 | total_timesteps 4015.
Path 485 | total_timesteps 4024.
Path 486 | total_timesteps 4032.
Path 487 | total_timesteps 4040.
Path 488 | total_timesteps 4046.
Path 489 | total_timesteps 4052.
Path 490 | total_timesteps 4060.
Path 491 | total_timesteps 4071.
Path 492 | total_timesteps 4080.
Path 493 | total_timesteps 4088.
Path 494 | total_timesteps 4100.
Path 495 | total_timesteps 4106.
Path 496 | total_timesteps 4119.
Path 497 | total_timesteps 4127.
Path 498 | total_timesteps 4138.
Path 499 | total_timesteps 4151.
Path 500 | total_timesteps 4161.
Path 501 | total_timesteps 4167.
Path 502 | total_timesteps 4173.
Path 503 | total_timesteps 4181.
Path 504 | total_timesteps 4188.
Path 505 | total_timesteps 4194.
Path 506 | total_timesteps 4205.
Path 507 | total_timesteps 4212.
Path 508 | total_timesteps 4223.
Path 509 | total_timesteps 4229.
Path 510 | total_timesteps 4241.
Path 511 | total_timesteps 4248.
Path 512 | total_timesteps 4254.
Path 513 | total_timesteps 4261.
Path 514 | total_timesteps 4269.
Path 515 | total_timesteps 4275.
Path 516 | total_timesteps 4285.
Path 517 | total_timesteps 4292.
Path 518 | total_timesteps 4302.
Path 519 | total_timesteps 4308.
Path 520 | total_timesteps 4314.
Path 521 | total_timesteps 4325.
Path 522 | total_timesteps 4331.
Path 523 | total_timesteps 4337.
Path 524 | total_timesteps 4347.
Path 525 | total_timesteps 4358.
Path 526 | total_timesteps 4365.
Path 527 | total_timesteps 4375.
Path 528 | total_timesteps 4383.
Path 529 | total_timesteps 4392.
Path 530 | total_timesteps 4400.
Path 531 | total_timesteps 4406.
Path 532 | total_timesteps 4416.
Path 533 | total_timesteps 4427.
Path 534 | total_timesteps 4435.
Path 535 | total_timesteps 4445.
Path 536 | total_timesteps 4454.
Path 537 | total_timesteps 4462.
Path 538 | total_timesteps 4471.
Path 539 | total_timesteps 4478.
Path 540 | total_timesteps 4485.
Path 541 | total_timesteps 4501.
Path 542 | total_timesteps 4508.
Path 543 | total_timesteps 4514.
Path 544 | total_timesteps 4521.
Path 545 | total_timesteps 4529.
Path 546 | total_timesteps 4537.
Path 547 | total_timesteps 4544.
Path 548 | total_timesteps 4553.
Path 549 | total_timesteps 4561.
Path 550 | total_timesteps 4568.
Path 551 | total_timesteps 4578.
Path 552 | total_timesteps 4590.
Path 553 | total_timesteps 4597.
Path 554 | total_timesteps 4604.
Path 555 | total_timesteps 4611.
Path 556 | total_timesteps 4618.
Path 557 | total_timesteps 4626.
Path 558 | total_timesteps 4635.
Path 559 | total_timesteps 4644.
Path 560 | total_timesteps 4652.
Path 561 | total_timesteps 4659.
Path 562 | total_timesteps 4668.
Path 563 | total_timesteps 4679.
Path 564 | total_timesteps 4690.
Path 565 | total_timesteps 4705.
Path 566 | total_timesteps 4713.
Path 567 | total_timesteps 4720.
Path 568 | total_timesteps 4727.
Path 569 | total_timesteps 4733.
Path 570 | total_timesteps 4739.
Path 571 | total_timesteps 4755.
Path 572 | total_timesteps 4762.
Path 573 | total_timesteps 4773.
Path 574 | total_timesteps 4783.
Path 575 | total_timesteps 4790.
Path 576 | total_timesteps 4802.
Path 577 | total_timesteps 4811.
Path 578 | total_timesteps 4817.
Path 579 | total_timesteps 4827.
Path 580 | total_timesteps 4834.
Path 581 | total_timesteps 4843.
Path 582 | total_timesteps 4851.
Path 583 | total_timesteps 4860.
Path 584 | total_timesteps 4866.
Path 585 | total_timesteps 4873.
Path 586 | total_timesteps 4881.
Path 587 | total_timesteps 4891.
Path 588 | total_timesteps 4908.
Path 589 | total_timesteps 4915.
Path 590 | total_timesteps 4930.
Path 591 | total_timesteps 4940.
Path 592 | total_timesteps 4947.
Path 593 | total_timesteps 4954.
Path 594 | total_timesteps 4960.
Path 595 | total_timesteps 4968.
Path 596 | total_timesteps 4978.
Path 597 | total_timesteps 4984.
Path 598 | total_timesteps 4990.
Path 599 | total_timesteps 4996.
Path 600 | total_timesteps 5004.
Path 601 | total_timesteps 5017.
Path 602 | total_timesteps 5027.
Path 603 | total_timesteps 5041.
Path 604 | total_timesteps 5052.
Path 605 | total_timesteps 5060.
Path 606 | total_timesteps 5067.
Path 607 | total_timesteps 5075.
Path 608 | total_timesteps 5082.
Path 609 | total_timesteps 5090.
Path 610 | total_timesteps 5098.
Path 611 | total_timesteps 5112.
Path 612 | total_timesteps 5119.
Path 613 | total_timesteps 5130.
Path 614 | total_timesteps 5136.
Path 615 | total_timesteps 5146.
Path 616 | total_timesteps 5154.
Path 617 | total_timesteps 5160.
Path 618 | total_timesteps 5167.
Path 619 | total_timesteps 5175.
Path 620 | total_timesteps 5182.
Path 621 | total_timesteps 5193.
Path 622 | total_timesteps 5202.
Path 623 | total_timesteps 5209.
Path 624 | total_timesteps 5223.
Path 625 | total_timesteps 5231.
Path 626 | total_timesteps 5240.
Path 627 | total_timesteps 5247.
Path 628 | total_timesteps 5255.
Path 629 | total_timesteps 5265.
Path 630 | total_timesteps 5274.
Path 631 | total_timesteps 5284.
Path 632 | total_timesteps 5291.
Path 633 | total_timesteps 5299.
Path 634 | total_timesteps 5306.
Path 635 | total_timesteps 5313.
Path 636 | total_timesteps 5322.
Path 637 | total_timesteps 5332.
Path 638 | total_timesteps 5342.
Path 639 | total_timesteps 5350.
Path 640 | total_timesteps 5356.
Path 641 | total_timesteps 5365.
Path 642 | total_timesteps 5372.
Path 643 | total_timesteps 5387.
Path 644 | total_timesteps 5394.
Path 645 | total_timesteps 5402.
Path 646 | total_timesteps 5408.
Path 647 | total_timesteps 5415.
Path 648 | total_timesteps 5423.
Path 649 | total_timesteps 5432.
Path 650 | total_timesteps 5439.
Path 651 | total_timesteps 5446.
Path 652 | total_timesteps 5454.
Path 653 | total_timesteps 5460.
Path 654 | total_timesteps 5468.
Path 655 | total_timesteps 5476.
Path 656 | total_timesteps 5485.
Path 657 | total_timesteps 5493.
Path 658 | total_timesteps 5500.
Path 659 | total_timesteps 5508.
Path 660 | total_timesteps 5521.
Path 661 | total_timesteps 5528.
Path 662 | total_timesteps 5535.
Path 663 | total_timesteps 5544.
Path 664 | total_timesteps 5551.
Path 665 | total_timesteps 5558.
Path 666 | total_timesteps 5565.
Path 667 | total_timesteps 5572.
Path 668 | total_timesteps 5579.
Path 669 | total_timesteps 5588.
Path 670 | total_timesteps 5596.
Path 671 | total_timesteps 5602.
Path 672 | total_timesteps 5608.
Path 673 | total_timesteps 5616.
Path 674 | total_timesteps 5632.
Path 675 | total_timesteps 5641.
Path 676 | total_timesteps 5650.
Path 677 | total_timesteps 5658.
Path 678 | total_timesteps 5665.
Path 679 | total_timesteps 5674.
Path 680 | total_timesteps 5681.
Path 681 | total_timesteps 5688.
Path 682 | total_timesteps 5698.
Path 683 | total_timesteps 5705.
Path 684 | total_timesteps 5712.
Path 685 | total_timesteps 5719.
Path 686 | total_timesteps 5726.
Path 687 | total_timesteps 5733.
Path 688 | total_timesteps 5741.
Path 689 | total_timesteps 5748.
Path 690 | total_timesteps 5755.
Path 691 | total_timesteps 5762.
Path 692 | total_timesteps 5773.
Path 693 | total_timesteps 5784.
Path 694 | total_timesteps 5794.
Path 695 | total_timesteps 5800.
Path 696 | total_timesteps 5811.
Path 697 | total_timesteps 5819.
Path 698 | total_timesteps 5826.
Path 699 | total_timesteps 5836.
Path 700 | total_timesteps 5846.
Path 701 | total_timesteps 5853.
Path 702 | total_timesteps 5860.
Path 703 | total_timesteps 5867.
Path 704 | total_timesteps 5874.
Path 705 | total_timesteps 5882.
Path 706 | total_timesteps 5888.
Path 707 | total_timesteps 5897.
Path 708 | total_timesteps 5904.
Path 709 | total_timesteps 5911.
Path 710 | total_timesteps 5921.
Path 711 | total_timesteps 5932.
Path 712 | total_timesteps 5941.
Path 713 | total_timesteps 5949.
Path 714 | total_timesteps 5956.
Path 715 | total_timesteps 5962.
Path 716 | total_timesteps 5969.
Path 717 | total_timesteps 5977.
Path 718 | total_timesteps 5991.
Path 719 | total_timesteps 5998.
Done generating on-policy rollouts.
Updating normalization.
Done updating normalization.
----------------------------
| AverageReturn | -6.18    |
| Iteration     | 32       |
| MaximumReturn | 1.18     |
| MinimumReturn | -13.6    |
| TotalSamples  | 136222   |
----------------------------
