/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
=== specification ====================================================
+: rlrd.training:Training
epochs: 10
rounds: 50
steps: 2000
stats_window: 10000
seed: 0
tag: ''
Env:
   +: rlrd.envs:RandomDelayEnv
   seed_val: 0
   id: Humanoid-v4
   frame_skip: 0
   min_observation_delay: 0
   sup_observation_delay: 1
   min_action_delay: 0
   sup_action_delay: 1
   real_world_sampler: 5
   action_noise: 0.05
Test:
   +: rlrd.testing:Test
   workers: 1
   number: 1
   device: cpu
Agent:
   +: rlrd.dcac:Agent
   batchsize: 128
   memory_size: 1000000
   lr: 0.0003
   discount: 0.99
   target_update: 0.005
   reward_scale: 5.0
   entropy_scale: 1.0
   start_training: 10000
   device: cpu
   training_steps: 1.0
   loss_alpha: 0.2
   rtac: false
   Model:
      +: rlrd.dcac_models:Mlp
      hidden_units: 256
      num_critics: 2
      act_delay: true
      obs_delay: true
   OutputNorm:
      +: rlrd.nn:PopArt
      beta: 0.0003
      zero_debias: true
      start_pop: 8
__format_version__: '3'
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>

<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 1/10 ===== round 1/50 ======================================
 85%|████████▌ | 1708/2000 [00:02<00:00, 775.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:02<00:00, 820.71it/s]
episodes                                   84
episode_length                       23.52381
returns                            115.615705
return_std                          38.853352
average_reward                        4.91743
round_time             0 days 00:00:02.448584
episodes_test                           372.0
episode_length_test                  26.77957
returns_test                        132.00569
return_std_test                     41.025803
average_reward_test                  4.928427
round_time_test        0 days 00:00:12.424868
round_time_total       0 days 00:00:14.208071 

=== epoch 1/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
 63%|██████▎   | 1258/2000 [00:01<00:00, 808.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:02<00:00, 813.29it/s]
episodes                                  169
episode_length                      23.526627
returns                            115.714301
return_std                          37.263726
average_reward                       4.917862
round_time             0 days 00:00:03.021143
episodes_test                           378.0
episode_length_test                 26.388889
returns_test                       130.264407
return_std_test                     38.247086
average_reward_test                   4.93668
round_time_test        0 days 00:00:12.485306
round_time_total       0 days 00:00:14.293849 

=== epoch 1/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
 64%|██████▎   | 1274/2000 [00:01<00:00, 795.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:02<00:00, 807.22it/s]
episodes                                  250
episode_length                         23.904
returns                            117.460633
return_std                          38.886393
average_reward                       4.914248
round_time             0 days 00:00:03.054283
episodes_test                           356.0
episode_length_test                 28.030899
returns_test                       138.829044
return_std_test                     50.696742
average_reward_test                  4.952954
round_time_test        0 days 00:00:12.697159
round_time_total       0 days 00:00:14.531122 

=== epoch 1/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
 71%|███████   | 1416/2000 [00:01<00:00, 840.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:02<00:00, 835.51it/s]
episodes                                  336
episode_length                      23.800595
returns                            116.831747
return_std                          38.412805
average_reward                       4.908778
round_time             0 days 00:00:02.872745
episodes_test                           371.0
episode_length_test                 26.927224
returns_test                       133.097219
return_std_test                     41.916808
average_reward_test                  4.942869
round_time_test        0 days 00:00:12.586177
round_time_total       0 days 00:00:14.362488 

=== epoch 1/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
 68%|██████▊   | 1370/2000 [00:01<00:00, 838.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:02<00:00, 829.54it/s]
episodes                                  418
episode_length                      23.839713
returns                            116.913893
return_std                          38.807819
average_reward                        4.90416
round_time             0 days 00:00:02.884504
episodes_test                           383.0
episode_length_test                 26.104439
returns_test                       129.508867
return_std_test                     38.589464
average_reward_test                   4.96117
round_time_test        0 days 00:00:12.592908
round_time_total       0 days 00:00:14.384048 

=== epoch 1/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 0/2000 [00:00<?, ?it/s]/<ANONYMIZED PATH>/rmst-rlrd/rlrd/nn.py:41: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  assert b.storage().data_ptr() == a.storage().data_ptr()
  0%|          | 8/2000 [00:01<06:17,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:15<00:00,  6.34it/s]
starting training
episodes                                  395
episode_length                      25.303797
returns                            124.643735
return_std                          44.536322
average_reward                         4.9259
round_time             0 days 00:05:16.221590
episodes_test                           370.0
episode_length_test                 26.967568
returns_test                       133.513806
return_std_test                     43.557012
average_reward_test                  4.950619
round_time_test        0 days 00:00:12.685727
round_time_total       0 days 00:05:16.222694
loss_total                       27815.935463
loss_critic                      34893.902714
loss_actor                        -495.935851
memory_size                         3377.4275 

=== epoch 1/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:37,  5.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:21<00:00,  6.22it/s]
episodes                                  381
episode_length                       26.19685
returns                            129.657652
return_std                          47.427847
average_reward                       4.949627
round_time             0 days 00:05:22.087262
episodes_test                           267.0
episode_length_test                 37.449438
returns_test                       196.477079
return_std_test                     52.210596
average_reward_test                   5.24643
round_time_test        0 days 00:00:12.773757
round_time_total       0 days 00:05:22.088375
loss_total                      967050.739752
loss_critic                     1209233.75652
loss_actor                       -1681.397398
memory_size                          4335.621 

=== epoch 1/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:40<00:00,  5.87it/s]
episodes                                  359
episode_length                      27.805014
returns                            137.879201
return_std                          52.306062
average_reward                       4.958924
round_time             0 days 00:05:41.048478
episodes_test                           483.0
episode_length_test                 20.668737
returns_test                        99.957008
return_std_test                     24.625989
average_reward_test                   4.83636
round_time_test        0 days 00:00:12.532452
round_time_total       0 days 00:05:41.049846
loss_total                     3840160.583547
loss_critic                    4801056.937625
loss_actor                       -3425.004483
memory_size                         5118.7375 

=== epoch 1/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                  330
episode_length                       30.09697
returns                            149.815861
return_std                          55.366868
average_reward                       4.977115
round_time             0 days 00:05:43.037724
episodes_test                           217.0
episode_length_test                   46.0553
returns_test                       230.318688
return_std_test                      72.46242
average_reward_test                  5.000826
round_time_test        0 days 00:00:12.825730
round_time_total       0 days 00:05:43.039074
loss_total                     1462771.681906
loss_critic                    1829256.654469
loss_actor                       -3168.336678
memory_size                          6203.398 

=== epoch 1/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:40<00:00,  5.87it/s]
episodes                                  306
episode_length                      32.594771
returns                            163.052431
return_std                          57.412656
average_reward                         5.0016
round_time             0 days 00:05:40.989626
episodes_test                           294.0
episode_length_test                 34.013605
returns_test                        172.69266
return_std_test                     53.861644
average_reward_test                  5.077164
round_time_test        0 days 00:00:12.988334
round_time_total       0 days 00:05:40.990728
loss_total                     1146802.768687
loss_critic                    1434303.880063
loss_actor                       -3201.781721
memory_size                          7177.326 

=== epoch 1/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:41<00:00,  5.85it/s]
episodes                                  314
episode_length                      31.792994
returns                            158.265281
return_std                          58.879986
average_reward                       4.978393
round_time             0 days 00:05:42.114913
episodes_test                           247.0
episode_length_test                 40.453441
returns_test                       201.245808
return_std_test                     74.803494
average_reward_test                  4.974674
round_time_test        0 days 00:00:12.998534
round_time_total       0 days 00:05:42.116004
loss_total                     1027732.092438
loss_critic                    1285474.499469
loss_actor                       -3237.629865
memory_size                          8196.533 

=== epoch 1/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                  301
episode_length                       33.13289
returns                            165.324046
return_std                           62.99511
average_reward                       4.989433
round_time             0 days 00:05:42.844247
episodes_test                           379.0
episode_length_test                 26.345646
returns_test                       129.147372
return_std_test                     37.564239
average_reward_test                  4.901887
round_time_test        0 days 00:00:12.549403
round_time_total       0 days 00:05:42.845559
loss_total                     1070923.114906
loss_critic                    1339510.892938
loss_actor                       -3428.100299
memory_size                          8999.256 

=== epoch 1/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.83it/s]
episodes                                  292
episode_length                      33.924658
returns                            169.470279
return_std                          65.892418
average_reward                       4.992199
round_time             0 days 00:05:43.844410
episodes_test                           238.0
episode_length_test                      42.0
returns_test                       212.039565
return_std_test                     78.531796
average_reward_test                  5.048501
round_time_test        0 days 00:00:12.754075
round_time_total       0 days 00:05:43.845881
loss_total                       1110005.1505
loss_critic                      1388397.5455
loss_actor                       -3564.527199
memory_size                        10106.5975 

=== epoch 1/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  287
episode_length                       34.71777
returns                            173.274615
return_std                          71.096716
average_reward                        4.98965
round_time             0 days 00:05:45.156476
episodes_test                           205.0
episode_length_test                  48.62439
returns_test                       244.684573
return_std_test                     82.531451
average_reward_test                  5.031545
round_time_test        0 days 00:00:12.799105
round_time_total       0 days 00:05:45.157584
loss_total                     1104353.375125
loss_critic                    1381368.171937
loss_actor                       -3705.910792
memory_size                        11278.6885 

=== epoch 1/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:24,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  287
episode_length                      34.651568
returns                            172.068929
return_std                           70.80663
average_reward                       4.965105
round_time             0 days 00:05:48.385557
episodes_test                           290.0
episode_length_test                 34.437931
returns_test                       171.863315
return_std_test                      65.19672
average_reward_test                  4.990345
round_time_test        0 days 00:00:12.629546
round_time_total       0 days 00:05:48.386645
loss_total                     1247921.774719
loss_critic                    1560832.277344
loss_actor                        -3720.33198
memory_size                         12312.447 

=== epoch 1/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  266
episode_length                       37.43985
returns                            186.551401
return_std                          74.547393
average_reward                       4.981448
round_time             0 days 00:05:50.374421
episodes_test                           227.0
episode_length_test                 43.898678
returns_test                       220.790744
return_std_test                     84.011206
average_reward_test                  5.028187
round_time_test        0 days 00:00:13.028978
round_time_total       0 days 00:05:50.375539
loss_total                     1287296.040688
loss_critic                    1610075.014063
loss_actor                       -3819.965246
memory_size                        13429.5235 

=== epoch 1/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  254
episode_length                      39.177165
returns                            194.862951
return_std                          80.696033
average_reward                         4.9753
round_time             0 days 00:05:51.267417
episodes_test                           208.0
episode_length_test                 47.995192
returns_test                       241.290181
return_std_test                     101.09769
average_reward_test                   5.02688
round_time_test        0 days 00:00:13.121671
round_time_total       0 days 00:05:51.268518
loss_total                     1302647.338844
loss_critic                    1629282.813187
loss_actor                       -3894.682971
memory_size                         14623.463 

=== epoch 1/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  260
episode_length                      38.315385
returns                            190.725427
return_std                          81.604102
average_reward                       4.978347
round_time             0 days 00:05:50.442840
episodes_test                           249.0
episode_length_test                  40.02008
returns_test                       201.125862
return_std_test                     78.106819
average_reward_test                  5.025016
round_time_test        0 days 00:00:12.528187
round_time_total       0 days 00:05:50.443952
loss_total                     1369874.419062
loss_critic                    1713338.153719
loss_actor                       -3980.641815
memory_size                         15774.002 

=== epoch 1/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  259
episode_length                      38.525097
returns                            192.384827
return_std                          85.077047
average_reward                       4.993772
round_time             0 days 00:05:52.092553
episodes_test                           266.0
episode_length_test                 37.552632
returns_test                       188.239357
return_std_test                     83.348135
average_reward_test                  5.012528
round_time_test        0 days 00:00:12.746719
round_time_total       0 days 00:05:52.093792
loss_total                     1554276.735469
loss_critic                    1943926.943719
loss_actor                       -4324.234625
memory_size                         16827.679 

=== epoch 1/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  257
episode_length                      38.747082
returns                            194.076501
return_std                          86.147785
average_reward                       5.008162
round_time             0 days 00:05:54.076014
episodes_test                           244.0
episode_length_test                  40.92623
returns_test                       208.110917
return_std_test                     88.764387
average_reward_test                  5.084663
round_time_test        0 days 00:00:12.886522
round_time_total       0 days 00:05:54.077407
loss_total                     1638032.575719
loss_critic                    2048651.664125
loss_actor                       -4443.921089
memory_size                        17957.8365 

=== epoch 1/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:36,  4.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  261
episode_length                      38.172414
returns                            191.079019
return_std                          85.883985
average_reward                       5.004662
round_time             0 days 00:05:53.964902
episodes_test                           298.0
episode_length_test                  33.47651
returns_test                       165.081247
return_std_test                     68.327988
average_reward_test                  4.931303
round_time_test        0 days 00:00:12.379108
round_time_total       0 days 00:05:53.966142
loss_total                     1726488.339406
loss_critic                     2159249.36675
loss_actor                       -4555.907231
memory_size                        19026.0445 

=== epoch 1/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  272
episode_length                      36.492647
returns                            182.393429
return_std                          80.805459
average_reward                       4.997484
round_time             0 days 00:05:51.110157
episodes_test                           265.0
episode_length_test                 37.532075
returns_test                       186.532499
return_std_test                     83.594875
average_reward_test                  4.968804
round_time_test        0 days 00:00:12.881446
round_time_total       0 days 00:05:51.111273
loss_total                     1843266.113469
loss_critic                     2305231.91775
loss_actor                       -4597.257681
memory_size                         20092.515 

=== epoch 1/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:48,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  268
episode_length                      37.078358
returns                            184.846908
return_std                          82.332326
average_reward                       4.987602
round_time             0 days 00:05:52.310034
episodes_test                           275.0
episode_length_test                 36.294545
returns_test                       179.757075
return_std_test                      86.45544
average_reward_test                  4.952559
round_time_test        0 days 00:00:12.722278
round_time_total       0 days 00:05:52.311238
loss_total                     2023119.735219
loss_critic                    2530064.427375
loss_actor                       -4659.203577
memory_size                         21165.684 

=== epoch 1/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  275
episode_length                      36.221818
returns                            179.856273
return_std                          78.840385
average_reward                       4.966326
round_time             0 days 00:05:53.679371
episodes_test                           300.0
episode_length_test                 33.293333
returns_test                       165.222243
return_std_test                     83.481843
average_reward_test                  4.962476
round_time_test        0 days 00:00:12.511065
round_time_total       0 days 00:05:53.680842
loss_total                     2169832.301687
loss_critic                    2713454.970312
loss_actor                       -4658.578917
memory_size                        22233.2485 

=== epoch 1/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  277
episode_length                      35.949458
returns                            178.119174
return_std                          80.752574
average_reward                       4.953411
round_time             0 days 00:05:53.879102
episodes_test                           260.0
episode_length_test                 38.380769
returns_test                        192.35199
return_std_test                     88.923989
average_reward_test                  5.011927
round_time_test        0 days 00:00:12.530833
round_time_total       0 days 00:05:53.880178
loss_total                     2148538.067813
loss_critic                    2686827.332813
loss_actor                       -4619.174213
memory_size                        23262.1365 

=== epoch 1/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                  279
episode_length                      35.713262
returns                            177.768662
return_std                          80.510801
average_reward                       4.977322
round_time             0 days 00:05:55.454181
episodes_test                           250.0
episode_length_test                    39.984
returns_test                       200.369178
return_std_test                     92.042178
average_reward_test                  5.011176
round_time_test        0 days 00:00:12.592904
round_time_total       0 days 00:05:55.455291
loss_total                     2208732.547937
loss_critic                    2762069.349063
loss_actor                       -4614.851802
memory_size                         24293.797 

=== epoch 1/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  279
episode_length                      35.741935
returns                            177.455408
return_std                          79.872612
average_reward                       4.965764
round_time             0 days 00:05:54.593371
episodes_test                           268.0
episode_length_test                 37.283582
returns_test                       185.452609
return_std_test                     87.938833
average_reward_test                  4.974006
round_time_test        0 days 00:00:12.785436
round_time_total       0 days 00:05:54.594715
loss_total                     2248696.069812
loss_critic                    2812015.188375
loss_actor                       -4580.582061
memory_size                         25341.276 

=== epoch 1/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  276
episode_length                      36.130435
returns                            179.732737
return_std                          82.677456
average_reward                       4.974541
round_time             0 days 00:05:55.725097
episodes_test                           251.0
episode_length_test                 39.701195
returns_test                       199.046733
return_std_test                       99.2441
average_reward_test                  5.013818
round_time_test        0 days 00:00:12.466641
round_time_total       0 days 00:05:55.726189
loss_total                       2284969.3585
loss_critic                    2857331.450375
loss_actor                       -4479.205172
memory_size                         26460.763 

=== epoch 1/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  279
episode_length                       35.74552
returns                            177.581059
return_std                          84.838494
average_reward                       4.968238
round_time             0 days 00:05:55.263677
episodes_test                           237.0
episode_length_test                 41.932489
returns_test                       206.678985
return_std_test                     98.792417
average_reward_test                  4.928235
round_time_test        0 days 00:00:12.916883
round_time_total       0 days 00:05:55.264917
loss_total                     2326791.923563
loss_critic                    2909567.471937
loss_actor                       -4310.480271
memory_size                        27507.7545 

=== epoch 1/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  280
episode_length                      35.446429
returns                            176.214874
return_std                          84.621484
average_reward                        4.97244
round_time             0 days 00:05:55.421482
episodes_test                           279.0
episode_length_test                 35.831541
returns_test                       176.477366
return_std_test                     72.804893
average_reward_test                  4.925178
round_time_test        0 days 00:00:12.821489
round_time_total       0 days 00:05:55.422869
loss_total                     2288776.678563
loss_critic                    2862025.844812
loss_actor                        -4220.20136
memory_size                        28467.2385 

=== epoch 1/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  282
episode_length                      35.340426
returns                            174.512385
return_std                           84.03953
average_reward                       4.938553
round_time             0 days 00:05:56.441250
episodes_test                           261.0
episode_length_test                 38.283525
returns_test                       189.208563
return_std_test                     94.921789
average_reward_test                  4.942185
round_time_test        0 days 00:00:12.722456
round_time_total       0 days 00:05:56.442400
loss_total                        2409217.904
loss_critic                     3012548.73675
loss_actor                       -4105.620626
memory_size                          29495.32 

=== epoch 1/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:03,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  289
episode_length                       34.32872
returns                            168.427308
return_std                          80.036705
average_reward                       4.904091
round_time             0 days 00:05:54.572776
episodes_test                           312.0
episode_length_test                 32.038462
returns_test                       156.817481
return_std_test                     65.929492
average_reward_test                  4.894657
round_time_test        0 days 00:00:12.716467
round_time_total       0 days 00:05:54.574056
loss_total                     2447534.578125
loss_critic                    3060430.566375
loss_actor                       -4049.594898
memory_size                         30472.508 

=== epoch 1/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  295
episode_length                      33.627119
returns                            164.250888
return_std                           73.69873
average_reward                       4.886052
round_time             0 days 00:05:56.234088
episodes_test                           288.0
episode_length_test                 34.670139
returns_test                        169.71475
return_std_test                     79.720786
average_reward_test                  4.895064
round_time_test        0 days 00:00:12.548930
round_time_total       0 days 00:05:56.235184
loss_total                     3264353.637562
loss_critic                    4081439.509375
loss_actor                       -3990.123005
memory_size                         31469.787 

=== epoch 1/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  279
episode_length                       35.62724
returns                            173.848835
return_std                          80.960646
average_reward                        4.88324
round_time             0 days 00:05:54.231795
episodes_test                           282.0
episode_length_test                 35.439716
returns_test                       173.879643
return_std_test                     79.659496
average_reward_test                  4.906306
round_time_test        0 days 00:00:12.617770
round_time_total       0 days 00:05:54.232884
loss_total                      2366171.27725
loss_critic                       2958657.216
loss_actor                       -3772.707659
memory_size                          32592.18 

=== epoch 1/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  271
episode_length                      36.800738
returns                            179.785386
return_std                          81.691683
average_reward                       4.885705
round_time             0 days 00:05:55.761316
episodes_test                           263.0
episode_length_test                 38.022814
returns_test                       186.525262
return_std_test                     86.680914
average_reward_test                  4.905614
round_time_test        0 days 00:00:12.759734
round_time_total       0 days 00:05:55.762399
loss_total                      2260865.73875
loss_critic                    2826945.245812
loss_actor                       -3452.479342
memory_size                         33819.156 

=== epoch 1/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:38,  4.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  267
episode_length                      37.404494
returns                            182.611961
return_std                          82.777837
average_reward                       4.882802
round_time             0 days 00:05:57.477160
episodes_test                           291.0
episode_length_test                 34.350515
returns_test                       167.907378
return_std_test                      71.70023
average_reward_test                  4.888047
round_time_test        0 days 00:00:12.696533
round_time_total       0 days 00:05:57.478257
loss_total                     1984436.608562
loss_critic                    2481328.588938
loss_actor                       -3131.480537
memory_size                         34920.068 

=== epoch 1/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  267
episode_length                       37.41573
returns                            183.709819
return_std                           84.81166
average_reward                       4.909375
round_time             0 days 00:05:57.218838
episodes_test                           286.0
episode_length_test                 34.940559
returns_test                       170.304507
return_std_test                     75.227964
average_reward_test                  4.874106
round_time_test        0 days 00:00:12.583026
round_time_total       0 days 00:05:57.219936
loss_total                     1752638.964687
loss_critic                    2191524.632688
loss_actor                       -2903.854304
memory_size                         35946.094 

=== epoch 1/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  263
episode_length                      37.570342
returns                             184.24274
return_std                          86.148515
average_reward                       4.908389
round_time             0 days 00:05:59.717164
episodes_test                           272.0
episode_length_test                 36.680147
returns_test                       178.700889
return_std_test                     83.307999
average_reward_test                  4.871899
round_time_test        0 days 00:00:12.914249
round_time_total       0 days 00:05:59.718282
loss_total                     1621164.548937
loss_critic                    2027151.560875
loss_actor                       -2783.634728
memory_size                         36927.985 

=== epoch 1/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  272
episode_length                      36.496324
returns                            178.736243
return_std                          81.381686
average_reward                       4.898374
round_time             0 days 00:05:59.096901
episodes_test                           290.0
episode_length_test                 34.372414
returns_test                       167.865224
return_std_test                     74.269507
average_reward_test                  4.883273
round_time_test        0 days 00:00:12.891758
round_time_total       0 days 00:05:59.098161
loss_total                     1472949.361406
loss_critic                     1841818.04075
loss_actor                       -2525.490736
memory_size                        38038.3835 

=== epoch 1/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:51,  4.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  272
episode_length                      36.672794
returns                            179.589241
return_std                          83.506418
average_reward                       4.897565
round_time             0 days 00:06:01.197806
episodes_test                           274.0
episode_length_test                  36.39781
returns_test                       178.780991
return_std_test                     81.890043
average_reward_test                  4.911737
round_time_test        0 days 00:00:12.908280
round_time_total       0 days 00:06:01.198892
loss_total                     1278530.311125
loss_critic                    1598737.176031
loss_actor                       -2297.265983
memory_size                        39196.4895 

=== epoch 1/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  274
episode_length                      36.419708
returns                             178.60207
return_std                          83.570854
average_reward                       4.904139
round_time             0 days 00:05:57.068227
episodes_test                           228.0
episode_length_test                 43.802632
returns_test                        215.89371
return_std_test                    110.191942
average_reward_test                  4.928632
round_time_test        0 days 00:00:12.921568
round_time_total       0 days 00:05:57.069322
loss_total                      1195314.34725
loss_critic                    1494683.829125
loss_actor                       -2163.686222
memory_size                        40311.6175 

=== epoch 1/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<06:07,  5.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  268
episode_length                       37.11194
returns                            181.716936
return_std                          82.840764
average_reward                       4.897439
round_time             0 days 00:05:58.264608
episodes_test                           278.0
episode_length_test                 35.841727
returns_test                       174.831333
return_std_test                     78.073456
average_reward_test                  4.877347
round_time_test        0 days 00:00:12.578863
round_time_total       0 days 00:05:58.265730
loss_total                     1184832.928094
loss_critic                    1481573.942875
loss_actor                       -2131.238058
memory_size                        41333.8245 

=== epoch 1/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  277
episode_length                      35.949458
returns                            176.228388
return_std                          80.097238
average_reward                       4.902954
round_time             0 days 00:06:01.296008
episodes_test                           288.0
episode_length_test                 34.510417
returns_test                       170.493138
return_std_test                     76.979264
average_reward_test                  4.941409
round_time_test        0 days 00:00:13.111379
round_time_total       0 days 00:06:01.297114
loss_total                     1176707.269875
loss_critic                    1471434.507375
loss_actor                       -2201.793666
memory_size                         42297.038 

=== epoch 1/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<08:13,  4.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  280
episode_length                      35.621429
returns                            175.018612
return_std                          79.462471
average_reward                       4.913527
round_time             0 days 00:05:59.714037
episodes_test                           258.0
episode_length_test                 38.647287
returns_test                       188.417975
return_std_test                     87.000768
average_reward_test                  4.874695
round_time_test        0 days 00:00:13.076905
round_time_total       0 days 00:05:59.715141
loss_total                     1235907.764969
loss_critic                    1545445.710344
loss_actor                       -2244.129905
memory_size                        43332.0215 

=== epoch 1/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:25,  4.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  287
episode_length                      34.592334
returns                            170.047913
return_std                          77.032974
average_reward                       4.917912
round_time             0 days 00:06:00.388127
episodes_test                           266.0
episode_length_test                 37.530075
returns_test                       185.188718
return_std_test                     90.868506
average_reward_test                  4.934267
round_time_test        0 days 00:00:13.027027
round_time_total       0 days 00:06:00.389454
loss_total                     1239219.859344
loss_critic                     1549579.38625
loss_actor                        -2218.35847
memory_size                        44321.6895 

=== epoch 1/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:34,  4.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  292
episode_length                      34.181507
returns                            167.931475
return_std                          73.363714
average_reward                       4.912564
round_time             0 days 00:06:01.303044
episodes_test                           268.0
episode_length_test                 37.216418
returns_test                       182.937638
return_std_test                     86.055075
average_reward_test                  4.915399
round_time_test        0 days 00:00:12.675672
round_time_total       0 days 00:06:01.304132
loss_total                     1259228.224125
loss_critic                     1574598.37075
loss_actor                       -2252.471441
memory_size                         45365.138 

=== epoch 1/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:49,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  289
episode_length                      34.567474
returns                            170.302638
return_std                          77.806754
average_reward                       4.926931
round_time             0 days 00:05:59.686035
episodes_test                           287.0
episode_length_test                 34.825784
returns_test                       172.453348
return_std_test                     78.198725
average_reward_test                  4.951834
round_time_test        0 days 00:00:12.665892
round_time_total       0 days 00:05:59.687132
loss_total                     1286053.187625
loss_critic                       1608134.974
loss_actor                       -2274.074264
memory_size                         46385.209 

=== epoch 1/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  284
episode_length                      35.066901
returns                             173.05142
return_std                          82.201177
average_reward                       4.934897
round_time             0 days 00:06:00.053105
episodes_test                           289.0
episode_length_test                 34.584775
returns_test                       170.277001
return_std_test                     76.234108
average_reward_test                  4.923429
round_time_test        0 days 00:00:12.583545
round_time_total       0 days 00:06:00.054195
loss_total                      1380048.54225
loss_critic                    1725635.038687
loss_actor                       -2297.579089
memory_size                         47430.904 

=== epoch 1/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:02<00:00,  5.52it/s]
episodes                                  284
episode_length                      35.066901
returns                            173.317188
return_std                          81.216952
average_reward                       4.943687
round_time             0 days 00:06:02.913590
episodes_test                           244.0
episode_length_test                 40.942623
returns_test                       204.434693
return_std_test                     87.922393
average_reward_test                  4.993044
round_time_test        0 days 00:00:12.672570
round_time_total       0 days 00:06:02.914677
loss_total                     1485801.690656
loss_critic                    1857830.559812
loss_actor                       -2313.909268
memory_size                        48512.8475 

=== epoch 1/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:38,  4.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  293
episode_length                      33.972696
returns                            167.866656
return_std                          79.368988
average_reward                       4.943763
round_time             0 days 00:05:58.704272
episodes_test                           315.0
episode_length_test                 31.634921
returns_test                       155.963214
return_std_test                     67.879542
average_reward_test                  4.930005
round_time_test        0 days 00:00:12.403988
round_time_total       0 days 00:05:58.705366
loss_total                     1613101.934188
loss_critic                    2016988.579188
loss_actor                       -2444.789128
memory_size                        49464.2235 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 2/10 ===== round 1/50 ======================================
  0%|          | 10/2000 [00:02<07:39,  4.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                   62
episode_length                      32.064516
returns                            157.423012
return_std                           70.88182
average_reward                       4.909468
round_time             0 days 00:05:43.546498
episodes_test                           324.0
episode_length_test                 30.805556
returns_test                       151.694691
return_std_test                     63.528571
average_reward_test                  4.924178
round_time_test        0 days 00:00:12.546616
round_time_total       0 days 00:05:43.547611
loss_total                     1804843.862125
loss_critic                    2256711.066375
loss_actor                       -2625.097328
memory_size                        50339.1455 

=== epoch 2/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  118
episode_length                      33.567797
returns                            164.598604
return_std                          71.113809
average_reward                       4.905146
round_time             0 days 00:05:45.188059
episodes_test                           283.0
episode_length_test                  35.30742
returns_test                       173.455322
return_std_test                     78.966336
average_reward_test                  4.912654
round_time_test        0 days 00:00:12.508283
round_time_total       0 days 00:05:45.189195
loss_total                     2020174.695563
loss_critic                    2525899.783687
loss_actor                       -2725.825214
memory_size                        51341.5265 

=== epoch 2/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  170
episode_length                      35.235294
returns                            173.872479
return_std                          72.620134
average_reward                       4.934429
round_time             0 days 00:05:47.682252
episodes_test                           310.0
episode_length_test                 32.177419
returns_test                       158.525127
return_std_test                     67.603853
average_reward_test                  4.926058
round_time_test        0 days 00:00:12.361200
round_time_total       0 days 00:05:47.683582
loss_total                     2320862.413188
loss_critic                    2901807.101875
loss_actor                        -2916.54328
memory_size                         52405.282 

=== epoch 2/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  223
episode_length                      35.780269
returns                             176.46451
return_std                          73.694969
average_reward                       4.932148
round_time             0 days 00:05:47.201418
episodes_test                           267.0
episode_length_test                 37.310861
returns_test                       184.814905
return_std_test                     76.956903
average_reward_test                  4.952693
round_time_test        0 days 00:00:13.057470
round_time_total       0 days 00:05:47.202500
loss_total                     2706433.594875
loss_critic                    3383820.662563
loss_actor                       -3114.923177
memory_size                        53509.0075 

=== epoch 2/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.80it/s]
episodes                                  285
episode_length                      35.087719
returns                             173.01666
return_std                          73.980354
average_reward                       4.930975
round_time             0 days 00:05:45.678957
episodes_test                           296.0
episode_length_test                 33.760135
returns_test                       166.814037
return_std_test                     78.135308
average_reward_test                  4.941056
round_time_test        0 days 00:00:12.639348
round_time_total       0 days 00:05:45.680163
loss_total                     3167065.882812
loss_critic                    3959657.485875
loss_actor                       -3300.781682
memory_size                         54557.382 

=== epoch 2/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  285
episode_length                      34.849123
returns                            171.781476
return_std                           71.63023
average_reward                       4.929955
round_time             0 days 00:05:49.143910
episodes_test                           337.0
episode_length_test                 29.578635
returns_test                       145.281596
return_std_test                     62.041611
average_reward_test                  4.910512
round_time_test        0 days 00:00:12.363670
round_time_total       0 days 00:05:49.145059
loss_total                     3848725.533125
loss_critic                    4811820.460125
loss_actor                       -3654.464137
memory_size                        55471.8085 

=== epoch 2/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  292
episode_length                      34.226027
returns                            168.947358
return_std                          73.011337
average_reward                       4.936364
round_time             0 days 00:05:49.930226
episodes_test                           314.0
episode_length_test                 31.834395
returns_test                       155.492111
return_std_test                      68.98705
average_reward_test                  4.884388
round_time_test        0 days 00:00:12.445677
round_time_total       0 days 00:05:49.931325
loss_total                     4561252.546625
loss_critic                      5702545.4395
loss_actor                        -3919.41046
memory_size                         56426.581 

=== epoch 2/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  296
episode_length                      33.577703
returns                            165.329772
return_std                          72.593684
average_reward                       4.924022
round_time             0 days 00:05:48.995038
episodes_test                           297.0
episode_length_test                 33.666667
returns_test                       164.678162
return_std_test                     74.716208
average_reward_test                  4.891437
round_time_test        0 days 00:00:12.645040
round_time_total       0 days 00:05:48.996504
loss_total                      5188778.15175
loss_critic                    6487014.543125
loss_actor                       -4167.894112
memory_size                        57402.6605 

=== epoch 2/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  304
episode_length                      32.815789
returns                             161.53742
return_std                          73.794874
average_reward                       4.922233
round_time             0 days 00:05:53.282636
episodes_test                           290.0
episode_length_test                 34.344828
returns_test                        167.59393
return_std_test                     71.484966
average_reward_test                   4.87968
round_time_test        0 days 00:00:12.627405
round_time_total       0 days 00:05:53.283745
loss_total                       6155313.5885
loss_critic                     7695269.79875
loss_actor                       -4511.778078
memory_size                        58437.8865 

=== epoch 2/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:07,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  305
episode_length                      32.770492
returns                            160.535317
return_std                          72.605236
average_reward                       4.898748
round_time             0 days 00:05:54.103579
episodes_test                           298.0
episode_length_test                 33.436242
returns_test                       164.001228
return_std_test                     69.290902
average_reward_test                  4.903534
round_time_test        0 days 00:00:12.701148
round_time_total       0 days 00:05:54.104847
loss_total                       6764476.9345
loss_critic                       8456738.549
loss_actor                       -4570.074682
memory_size                        59347.8305 

=== epoch 2/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:46,  4.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  305
episode_length                      32.737705
returns                            160.065694
return_std                          74.335142
average_reward                       4.889318
round_time             0 days 00:05:53.661466
episodes_test                           291.0
episode_length_test                 34.278351
returns_test                       166.999755
return_std_test                     71.539433
average_reward_test                  4.871793
round_time_test        0 days 00:00:12.873187
round_time_total       0 days 00:05:53.662574
loss_total                     7149776.456375
loss_critic                     8938402.63425
loss_actor                       -4728.817948
memory_size                         60318.176 

=== epoch 2/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:21,  4.51it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  305
episode_length                      32.659016
returns                            159.677059
return_std                          70.884318
average_reward                        4.88783
round_time             0 days 00:05:55.137217
episodes_test                           279.0
episode_length_test                 35.817204
returns_test                       175.551913
return_std_test                     65.806805
average_reward_test                  4.901264
round_time_test        0 days 00:00:12.751888
round_time_total       0 days 00:05:55.138322
loss_total                        7963698.134
loss_critic                       9955851.567
loss_actor                        -4916.25487
memory_size                        61223.3665 

=== epoch 2/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  311
episode_length                      32.041801
returns                            156.338361
return_std                          69.564073
average_reward                       4.880782
round_time             0 days 00:05:56.605181
episodes_test                           290.0
episode_length_test                 34.458621
returns_test                       168.839092
return_std_test                     64.799304
average_reward_test                  4.899724
round_time_test        0 days 00:00:12.587726
round_time_total       0 days 00:05:56.606295
loss_total                      9248752.26725
loss_critic                      11562252.213
loss_actor                       -5248.305646
memory_size                         62171.409 

=== epoch 2/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  309
episode_length                      32.087379
returns                            156.378553
return_std                          66.122678
average_reward                       4.869552
round_time             0 days 00:05:56.899751
episodes_test                           301.0
episode_length_test                 33.169435
returns_test                       160.649423
return_std_test                     62.453237
average_reward_test                  4.843059
round_time_test        0 days 00:00:12.773834
round_time_total       0 days 00:05:56.900953
loss_total                       10463242.858
loss_critic                    13080435.16775
loss_actor                       -5527.332399
memory_size                         63161.998 

=== epoch 2/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  314
episode_length                      31.802548
returns                             155.26015
return_std                          63.108019
average_reward                       4.882073
round_time             0 days 00:05:54.906250
episodes_test                           316.0
episode_length_test                 31.537975
returns_test                       152.771052
return_std_test                     60.280755
average_reward_test                  4.844337
round_time_test        0 days 00:00:12.500015
round_time_total       0 days 00:05:54.907360
loss_total                     11527266.96675
loss_critic                    14410552.20425
loss_actor                       -5875.011284
memory_size                         64081.149 

=== epoch 2/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  308
episode_length                      32.347403
returns                            158.015935
return_std                           60.34003
average_reward                       4.884554
round_time             0 days 00:05:52.791127
episodes_test                           313.0
episode_length_test                 31.840256
returns_test                       155.488697
return_std_test                     62.453842
average_reward_test                  4.882832
round_time_test        0 days 00:00:12.566050
round_time_total       0 days 00:05:52.792453
loss_total                       13179501.345
loss_critic                     16475949.9175
loss_actor                       -6294.014797
memory_size                        65003.2215 

=== epoch 2/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  312
episode_length                      32.022436
returns                            155.824138
return_std                          59.874793
average_reward                       4.865959
round_time             0 days 00:05:57.329017
episodes_test                           337.0
episode_length_test                 29.635015
returns_test                       144.146524
return_std_test                     51.532484
average_reward_test                  4.864164
round_time_test        0 days 00:00:12.750234
round_time_total       0 days 00:05:57.330141
loss_total                      14968230.5405
loss_critic                      18711955.359
loss_actor                       -6669.955704
memory_size                         65977.009 

=== epoch 2/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  312
episode_length                       32.00641
returns                            155.358789
return_std                          54.630157
average_reward                       4.853759
round_time             0 days 00:05:56.493814
episodes_test                           343.0
episode_length_test                 29.148688
returns_test                       142.219932
return_std_test                     53.125976
average_reward_test                  4.879127
round_time_test        0 days 00:00:12.429113
round_time_total       0 days 00:05:56.495221
loss_total                      16358423.1435
loss_critic                     20449779.7845
loss_actor                       -7004.818464
memory_size                         66872.736 

=== epoch 2/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:32,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  317
episode_length                       31.48265
returns                            152.404421
return_std                          51.541124
average_reward                       4.840808
round_time             0 days 00:05:53.848188
episodes_test                           298.0
episode_length_test                 33.439597
returns_test                       163.159502
return_std_test                     59.581613
average_reward_test                  4.879614
round_time_test        0 days 00:00:12.711548
round_time_total       0 days 00:05:53.849300
loss_total                       17966593.989
loss_critic                     22460050.1885
loss_actor                       -7232.476328
memory_size                        67811.6655 

=== epoch 2/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  309
episode_length                      32.184466
returns                            155.552329
return_std                          54.266388
average_reward                       4.833087
round_time             0 days 00:05:54.163337
episodes_test                           321.0
episode_length_test                 31.087227
returns_test                       150.413952
return_std_test                       56.9089
average_reward_test                  4.838368
round_time_test        0 days 00:00:12.617075
round_time_total       0 days 00:05:54.164445
loss_total                       21144879.293
loss_critic                     26433063.9235
loss_actor                       -7861.197418
memory_size                        68723.2665 

=== epoch 2/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  311
episode_length                      32.125402
returns                            154.944266
return_std                          56.082076
average_reward                       4.823271
round_time             0 days 00:05:57.166699
episodes_test                           327.0
episode_length_test                 30.550459
returns_test                       147.919122
return_std_test                     52.633393
average_reward_test                  4.841776
round_time_test        0 days 00:00:12.422354
round_time_total       0 days 00:05:57.167820
loss_total                        23809111.35
loss_critic                      29763463.481
loss_actor                       -8299.348598
memory_size                        69743.1035 

=== epoch 2/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  310
episode_length                      32.022581
returns                            154.467759
return_std                          55.444176
average_reward                       4.820917
round_time             0 days 00:05:56.070132
episodes_test                           338.0
episode_length_test                 29.556213
returns_test                       143.514688
return_std_test                     51.687097
average_reward_test                  4.855624
round_time_test        0 days 00:00:12.559630
round_time_total       0 days 00:05:56.071224
loss_total                        25946653.43
loss_critic                      32435505.352
loss_actor                       -8756.446141
memory_size                        70681.0665 

=== epoch 2/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:46,  4.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  314
episode_length                      31.726115
returns                            153.132433
return_std                          56.002063
average_reward                       4.826237
round_time             0 days 00:05:57.895924
episodes_test                           340.0
episode_length_test                 29.344118
returns_test                       141.857524
return_std_test                     51.942628
average_reward_test                  4.834228
round_time_test        0 days 00:00:12.294688
round_time_total       0 days 00:05:57.897012
loss_total                      28711141.1695
loss_critic                      35891199.974
loss_actor                       -9096.405633
memory_size                         71550.073 

=== epoch 2/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:50,  4.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  311
episode_length                      32.048232
returns                            154.870292
return_std                          58.041783
average_reward                       4.832295
round_time             0 days 00:05:58.023291
episodes_test                           331.0
episode_length_test                 30.169184
returns_test                       145.659196
return_std_test                     53.386381
average_reward_test                  4.828033
round_time_test        0 days 00:00:12.586115
round_time_total       0 days 00:05:58.024580
loss_total                      30108627.5485
loss_critic                       37638105.76
loss_actor                       -9287.635398
memory_size                        72468.4045 

=== epoch 2/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  316
episode_length                      31.632911
returns                            152.835068
return_std                          57.917713
average_reward                       4.831418
round_time             0 days 00:05:55.837331
episodes_test                           308.0
episode_length_test                 32.448052
returns_test                       155.541695
return_std_test                      66.62506
average_reward_test                  4.793585
round_time_test        0 days 00:00:12.565287
round_time_total       0 days 00:05:55.838418
loss_total                      30865257.6345
loss_critic                      38583884.872
loss_actor                       -9253.815124
memory_size                         73424.425 

=== epoch 2/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:38,  4.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  317
episode_length                      31.466877
returns                            151.928536
return_std                           59.07519
average_reward                       4.828692
round_time             0 days 00:05:58.789137
episodes_test                           317.0
episode_length_test                 31.454259
returns_test                       150.426013
return_std_test                     60.636882
average_reward_test                  4.783185
round_time_test        0 days 00:00:12.475705
round_time_total       0 days 00:05:58.790277
loss_total                       33350297.327
loss_critic                      41690230.471
loss_actor                       -9438.330838
memory_size                        74349.0035 

=== epoch 2/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  316
episode_length                      31.607595
returns                            152.461543
return_std                          59.698392
average_reward                        4.82343
round_time             0 days 00:05:56.810193
episodes_test                           329.0
episode_length_test                 30.361702
returns_test                       145.952429
return_std_test                     55.874363
average_reward_test                  4.807135
round_time_test        0 days 00:00:12.788775
round_time_total       0 days 00:05:56.811279
loss_total                       35044800.776
loss_critic                      43808363.997
loss_actor                       -9455.213376
memory_size                        75305.6365 

=== epoch 2/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:46,  4.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  314
episode_length                      31.812102
returns                            153.234203
return_std                          61.067502
average_reward                       4.816797
round_time             0 days 00:05:58.978722
episodes_test                           289.0
episode_length_test                 34.588235
returns_test                       167.633943
return_std_test                     65.081365
average_reward_test                  4.846553
round_time_test        0 days 00:00:12.895284
round_time_total       0 days 00:05:58.979821
loss_total                       37864681.369
loss_critic                      47333250.265
loss_actor                       -9597.390219
memory_size                        76219.0675 

=== epoch 2/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  316
episode_length                      31.575949
returns                            151.841916
return_std                          59.711086
average_reward                       4.808661
round_time             0 days 00:05:58.024459
episodes_test                           327.0
episode_length_test                 30.571865
returns_test                       147.212924
return_std_test                     58.518005
average_reward_test                  4.815326
round_time_test        0 days 00:00:12.513413
round_time_total       0 days 00:05:58.025551
loss_total                       38561670.651
loss_critic                      48204476.438
loss_actor                       -9556.143286
memory_size                        77114.8945 

=== epoch 2/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  315
episode_length                       31.67619
returns                            152.672608
return_std                          57.349354
average_reward                       4.819535
round_time             0 days 00:05:59.733087
episodes_test                           314.0
episode_length_test                 31.770701
returns_test                       153.529941
return_std_test                     59.748365
average_reward_test                  4.832539
round_time_test        0 days 00:00:12.671573
round_time_total       0 days 00:05:59.734579
loss_total                        37624440.37
loss_critic                      47032870.963
loss_actor                       -9285.309056
memory_size                        78022.0545 

=== epoch 2/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  318
episode_length                      31.396226
returns                            151.144531
return_std                          55.525001
average_reward                       4.814247
round_time             0 days 00:05:57.783722
episodes_test                           312.0
episode_length_test                 32.012821
returns_test                       154.491217
return_std_test                     59.249003
average_reward_test                  4.825851
round_time_test        0 days 00:00:12.250338
round_time_total       0 days 00:05:57.784840
loss_total                       38719621.206
loss_critic                      48401823.298
loss_actor                       -9190.439307
memory_size                         78993.234 

=== epoch 2/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:36,  4.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  324
episode_length                      30.845679
returns                            148.793456
return_std                          54.151891
average_reward                       4.823847
round_time             0 days 00:05:58.452943
episodes_test                           314.0
episode_length_test                 31.818471
returns_test                       152.567619
return_std_test                     59.643152
average_reward_test                   4.79495
round_time_test        0 days 00:00:12.640576
round_time_total       0 days 00:05:58.454391
loss_total                        40609385.64
loss_critic                      50764035.203
loss_actor                       -9216.147186
memory_size                         79858.553 

=== epoch 2/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:03,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  328
episode_length                      30.381098
returns                            146.599402
return_std                          53.777845
average_reward                       4.825267
round_time             0 days 00:05:56.426954
episodes_test                           343.0
episode_length_test                 29.075802
returns_test                       139.253331
return_std_test                      59.02124
average_reward_test                  4.789081
round_time_test        0 days 00:00:12.613831
round_time_total       0 days 00:05:56.428058
loss_total                       42231554.056
loss_critic                      52791783.751
loss_actor                       -9368.508229
memory_size                        80681.0235 

=== epoch 2/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  337
episode_length                      29.611276
returns                            142.841229
return_std                          53.380602
average_reward                       4.823222
round_time             0 days 00:05:58.878978
episodes_test                           354.0
episode_length_test                 28.183616
returns_test                       135.142216
return_std_test                     52.099292
average_reward_test                  4.795626
round_time_test        0 days 00:00:12.385757
round_time_total       0 days 00:05:58.880295
loss_total                       44830125.387
loss_critic                      56040063.628
loss_actor                       -9631.214439
memory_size                         81442.368 

=== epoch 2/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  335
episode_length                      29.722388
returns                            142.594644
return_std                          57.729261
average_reward                       4.796937
round_time             0 days 00:06:00.646082
episodes_test                           340.0
episode_length_test                 29.352941
returns_test                       141.436841
return_std_test                     58.204968
average_reward_test                   4.81814
round_time_test        0 days 00:00:12.545389
round_time_total       0 days 00:06:00.647179
loss_total                       45722615.117
loss_critic                      57155663.987
loss_actor                       -9584.455052
memory_size                         82342.045 

=== epoch 2/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  340
episode_length                          29.35
returns                            141.139943
return_std                          57.475924
average_reward                       4.809596
round_time             0 days 00:05:58.056860
episodes_test                           343.0
episode_length_test                 29.029155
returns_test                       139.915147
return_std_test                      58.75853
average_reward_test                  4.817894
round_time_test        0 days 00:00:12.544202
round_time_total       0 days 00:05:58.058045
loss_total                       45656904.155
loss_critic                      57073441.595
loss_actor                       -9249.370725
memory_size                        83257.2395 

=== epoch 2/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:24,  4.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  335
episode_length                      29.707463
returns                            142.588307
return_std                          60.785131
average_reward                       4.800361
round_time             0 days 00:05:59.583362
episodes_test                           322.0
episode_length_test                 31.046584
returns_test                       150.207953
return_std_test                     59.073488
average_reward_test                  4.838152
round_time_test        0 days 00:00:12.656453
round_time_total       0 days 00:05:59.584487
loss_total                       45686001.229
loss_critic                      57109781.026
loss_actor                       -9121.787819
memory_size                         84086.533 

=== epoch 2/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:09,  4.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  323
episode_length                      30.885449
returns                            147.932518
return_std                          63.192375
average_reward                       4.789265
round_time             0 days 00:05:57.212386
episodes_test                           325.0
episode_length_test                 30.670769
returns_test                       147.486972
return_std_test                      64.69026
average_reward_test                  4.808147
round_time_test        0 days 00:00:12.367257
round_time_total       0 days 00:05:57.213814
loss_total                       46120951.665
loss_critic                       57653392.12
loss_actor                        -8813.95184
memory_size                        85089.2485 

=== epoch 2/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  308
episode_length                      32.282468
returns                            154.640574
return_std                          67.205193
average_reward                       4.791493
round_time             0 days 00:05:58.056552
episodes_test                           326.0
episode_length_test                 30.668712
returns_test                       147.270256
return_std_test                     62.480386
average_reward_test                  4.801995
round_time_test        0 days 00:00:12.471015
round_time_total       0 days 00:05:58.057649
loss_total                       48303921.336
loss_critic                      60382126.811
loss_actor                       -8904.641906
memory_size                        86110.4975 

=== epoch 2/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  311
episode_length                      32.080386
returns                            153.838887
return_std                          66.104715
average_reward                       4.795137
round_time             0 days 00:06:00.919605
episodes_test                           343.0
episode_length_test                 29.139942
returns_test                       140.590336
return_std_test                     62.405552
average_reward_test                  4.824688
round_time_test        0 days 00:00:12.368044
round_time_total       0 days 00:06:00.920899
loss_total                        47412864.84
loss_critic                      59268236.076
loss_actor                       -8623.921776
memory_size                         87066.645 

=== epoch 2/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  307
episode_length                       32.42671
returns                            154.965547
return_std                          66.288606
average_reward                       4.778625
round_time             0 days 00:05:59.644275
episodes_test                           324.0
episode_length_test                 30.851852
returns_test                       148.656349
return_std_test                     64.039574
average_reward_test                  4.818403
round_time_test        0 days 00:00:12.564956
round_time_total       0 days 00:05:59.645388
loss_total                       47015704.316
loss_critic                      58771691.785
loss_actor                        -8249.22397
memory_size                        87991.2295 

=== epoch 2/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  310
episode_length                      32.177419
returns                            153.901194
return_std                           64.82444
average_reward                       4.783044
round_time             0 days 00:06:00.375570
episodes_test                           330.0
episode_length_test                 30.218182
returns_test                       144.995804
return_std_test                     60.423217
average_reward_test                  4.797919
round_time_test        0 days 00:00:12.747979
round_time_total       0 days 00:06:00.376990
loss_total                       48528415.803
loss_critic                      60662509.524
loss_actor                       -7963.448658
memory_size                        88859.4575 

=== epoch 2/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:28,  4.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  318
episode_length                       31.36478
returns                            149.882248
return_std                           62.73287
average_reward                       4.779331
round_time             0 days 00:06:01.383999
episodes_test                           315.0
episode_length_test                 31.711111
returns_test                        153.06446
return_std_test                     63.868825
average_reward_test                  4.826808
round_time_test        0 days 00:00:12.581904
round_time_total       0 days 00:06:01.385085
loss_total                       48806960.346
loss_critic                       61010680.41
loss_actor                       -7924.087123
memory_size                        89768.4185 

=== epoch 2/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:25,  4.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  328
episode_length                      30.463415
returns                            145.790748
return_std                          59.380641
average_reward                       4.786002
round_time             0 days 00:05:56.394571
episodes_test                           329.0
episode_length_test                 30.243161
returns_test                       145.289602
return_std_test                     57.579455
average_reward_test                  4.802874
round_time_test        0 days 00:00:12.408035
round_time_total       0 days 00:05:56.395690
loss_total                       48343157.852
loss_critic                      60430868.094
loss_actor                       -7687.658557
memory_size                         90662.232 

=== epoch 2/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  325
episode_length                          30.76
returns                            147.577521
return_std                          57.844715
average_reward                       4.797722
round_time             0 days 00:05:57.942112
episodes_test                           297.0
episode_length_test                 33.632997
returns_test                       161.357285
return_std_test                      71.40661
average_reward_test                  4.797618
round_time_test        0 days 00:00:12.470207
round_time_total       0 days 00:05:57.943375
loss_total                       51402910.293
loss_critic                       64255654.13
loss_actor                       -8069.440551
memory_size                         91541.048 

=== epoch 2/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  321
episode_length                      31.115265
returns                            149.664588
return_std                          61.342465
average_reward                       4.809812
round_time             0 days 00:06:00.319495
episodes_test                           300.0
episode_length_test                 33.333333
returns_test                       160.764483
return_std_test                     68.258958
average_reward_test                  4.822934
round_time_test        0 days 00:00:12.854591
round_time_total       0 days 00:06:00.320591
loss_total                       54234791.644
loss_critic                       67795568.54
loss_actor                       -8320.482961
memory_size                         92546.444 

=== epoch 2/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:31,  4.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  314
episode_length                      31.729299
returns                            153.065622
return_std                          64.276357
average_reward                       4.824198
round_time             0 days 00:05:57.488904
episodes_test                           332.0
episode_length_test                 30.039157
returns_test                        143.35712
return_std_test                     60.240991
average_reward_test                  4.772292
round_time_test        0 days 00:00:12.461256
round_time_total       0 days 00:05:57.490182
loss_total                       54072050.533
loss_critic                      67592065.485
loss_actor                       -8013.466369
memory_size                        93455.4385 

=== epoch 2/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  304
episode_length                      32.865132
returns                            159.291743
return_std                           69.87232
average_reward                        4.84681
round_time             0 days 00:05:59.930673
episodes_test                           262.0
episode_length_test                 38.045802
returns_test                        182.20401
return_std_test                     77.857653
average_reward_test                  4.788785
round_time_test        0 days 00:00:12.847253
round_time_total       0 days 00:05:59.931944
loss_total                       56447131.178
loss_critic                      70560925.512
loss_actor                       -8050.795244
memory_size                        94487.6265 

=== epoch 2/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  296
episode_length                      33.618243
returns                            162.716946
return_std                           70.97448
average_reward                         4.8393
round_time             0 days 00:06:02.066291
episodes_test                           316.0
episode_length_test                 31.591772
returns_test                       151.510006
return_std_test                     65.738657
average_reward_test                  4.796015
round_time_test        0 days 00:00:12.489767
round_time_total       0 days 00:06:02.067407
loss_total                       61470021.927
loss_critic                      76839632.142
loss_actor                       -8424.048854
memory_size                        95542.9955 

=== epoch 2/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  296
episode_length                      33.584459
returns                            162.649007
return_std                          72.487424
average_reward                       4.842505
round_time             0 days 00:06:00.179003
episodes_test                           323.0
episode_length_test                  30.93808
returns_test                       148.266374
return_std_test                     59.072285
average_reward_test                  4.792394
round_time_test        0 days 00:00:12.546054
round_time_total       0 days 00:06:00.180100
loss_total                       62474929.456
loss_critic                      78095789.836
loss_actor                       -8517.011844
memory_size                        96502.2315 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 3/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.83it/s]
episodes                                   62
episode_length                      32.064516
returns                            152.959335
return_std                          61.228562
average_reward                       4.770485
round_time             0 days 00:05:42.887884
episodes_test                           320.0
episode_length_test                  31.20625
returns_test                        148.03581
return_std_test                     63.029743
average_reward_test                  4.743875
round_time_test        0 days 00:00:12.659782
round_time_total       0 days 00:05:42.888992
loss_total                       69991904.831
loss_critic                      87492114.732
loss_actor                       -8941.175774
memory_size                         97489.019 

=== epoch 3/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:48,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.81it/s]
episodes                                  123
episode_length                      32.455285
returns                            154.963782
return_std                          71.306375
average_reward                       4.774741
round_time             0 days 00:05:44.864391
episodes_test                           298.0
episode_length_test                 33.540268
returns_test                       160.678266
return_std_test                     63.892757
average_reward_test                  4.790627
round_time_test        0 days 00:00:12.620404
round_time_total       0 days 00:05:44.865475
loss_total                       74312964.324
loss_critic                      92893531.906
loss_actor                       -9312.580909
memory_size                         98396.479 

=== epoch 3/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  183
episode_length                      32.639344
returns                            156.176281
return_std                          67.171303
average_reward                       4.784402
round_time             0 days 00:05:46.889578
episodes_test                           317.0
episode_length_test                 31.473186
returns_test                       152.207029
return_std_test                     65.247326
average_reward_test                  4.835933
round_time_test        0 days 00:00:12.542823
round_time_total       0 days 00:05:46.891076
loss_total                       76922441.402
loss_critic                      96155380.608
loss_actor                       -9322.082724
memory_size                        99384.7575 

=== epoch 3/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:09,  4.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  245
episode_length                      32.514286
returns                            155.463192
return_std                          66.776975
average_reward                       4.779851
round_time             0 days 00:05:51.441537
episodes_test                           297.0
episode_length_test                 33.670034
returns_test                       162.381305
return_std_test                     71.476726
average_reward_test                  4.822725
round_time_test        0 days 00:00:12.381699
round_time_total       0 days 00:05:51.442639
loss_total                         77759326.9
loss_critic                       97201457.44
loss_actor                       -9202.208096
memory_size                       100359.1905 

=== epoch 3/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:51,  3.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  310
episode_length                           32.2
returns                            153.732265
return_std                          64.283625
average_reward                       4.774345
round_time             0 days 00:05:50.643791
episodes_test                           319.0
episode_length_test                  31.31348
returns_test                       149.380682
return_std_test                     61.349268
average_reward_test                  4.770518
round_time_test        0 days 00:00:12.606394
round_time_total       0 days 00:05:50.644926
loss_total                        79196726.08
loss_critic                       98998204.68
loss_actor                       -9195.241011
memory_size                        101304.941 

=== epoch 3/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  312
episode_length                      31.929487
returns                            152.657477
return_std                          66.263257
average_reward                       4.779474
round_time             0 days 00:05:50.236958
episodes_test                           331.0
episode_length_test                 30.108761
returns_test                       143.787308
return_std_test                     60.509298
average_reward_test                  4.775006
round_time_test        0 days 00:00:12.507077
round_time_total       0 days 00:05:50.238050
loss_total                       81433051.298
loss_critic                     101793579.608
loss_actor                       -9069.184334
memory_size                       102192.2815 

=== epoch 3/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                  314
episode_length                      31.751592
returns                            151.636461
return_std                          61.463912
average_reward                       4.775895
round_time             0 days 00:05:48.689587
episodes_test                           315.0
episode_length_test                  31.67619
returns_test                       151.619851
return_std_test                      61.44457
average_reward_test                  4.786457
round_time_test        0 days 00:00:12.490294
round_time_total       0 days 00:05:48.691082
loss_total                       83804724.182
loss_critic                     104758216.202
loss_actor                       -9251.370026
memory_size                        103090.215 

=== epoch 3/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  321
episode_length                      31.084112
returns                            148.686647
return_std                          61.123208
average_reward                        4.78361
round_time             0 days 00:05:52.149003
episodes_test                           343.0
episode_length_test                 29.151603
returns_test                       140.088195
return_std_test                     55.488211
average_reward_test                  4.805516
round_time_test        0 days 00:00:12.603539
round_time_total       0 days 00:05:52.150113
loss_total                       90179672.437
loss_critic                     112726990.602
loss_actor                       -9608.212967
memory_size                        104005.302 

=== epoch 3/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  319
episode_length                      31.269592
returns                            149.524733
return_std                          60.525126
average_reward                       4.781316
round_time             0 days 00:05:51.557660
episodes_test                           302.0
episode_length_test                 33.109272
returns_test                       158.188981
return_std_test                     62.497331
average_reward_test                  4.777799
round_time_test        0 days 00:00:12.365248
round_time_total       0 days 00:05:51.558751
loss_total                       95943217.602
loss_critic                     119931546.056
loss_actor                      -10104.503309
memory_size                       104918.2485 

=== epoch 3/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  315
episode_length                      31.539683
returns                            151.243068
return_std                          62.300724
average_reward                       4.795125
round_time             0 days 00:05:54.403568
episodes_test                           298.0
episode_length_test                 33.506711
returns_test                       159.152448
return_std_test                     62.060396
average_reward_test                  4.749966
round_time_test        0 days 00:00:12.574539
round_time_total       0 days 00:05:54.405020
loss_total                       103962581.27
loss_critic                     129955832.904
loss_actor                      -10434.423359
memory_size                       105900.9855 

=== epoch 3/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  323
episode_length                      30.829721
returns                            147.813284
return_std                           58.82035
average_reward                       4.795356
round_time             0 days 00:05:54.299871
episodes_test                           321.0
episode_length_test                 31.121495
returns_test                       149.005619
return_std_test                     59.564951
average_reward_test                  4.787906
round_time_test        0 days 00:00:12.560414
round_time_total       0 days 00:05:54.301347
loss_total                      109297262.504
loss_critic                      136624307.83
loss_actor                      -10928.147847
memory_size                        106762.973 

=== epoch 3/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  326
episode_length                      30.659509
returns                            147.265259
return_std                          58.918658
average_reward                       4.803175
round_time             0 days 00:05:54.348873
episodes_test                           347.0
episode_length_test                 28.795389
returns_test                       139.793861
return_std_test                     55.592249
average_reward_test                   4.85473
round_time_test        0 days 00:00:12.477326
round_time_total       0 days 00:05:54.350138
loss_total                      118225546.466
loss_critic                     147784863.564
loss_actor                      -11731.693437
memory_size                        107604.738 

=== epoch 3/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  321
episode_length                      31.140187
returns                            149.380662
return_std                          60.257832
average_reward                       4.797138
round_time             0 days 00:05:53.141374
episodes_test                           328.0
episode_length_test                 30.484756
returns_test                       146.571707
return_std_test                     58.529214
average_reward_test                  4.808046
round_time_test        0 days 00:00:12.665611
round_time_total       0 days 00:05:53.142523
loss_total                      128787743.912
loss_critic                      160987731.94
loss_actor                      -12220.093392
memory_size                       108495.3005 

=== epoch 3/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  321
episode_length                      30.959502
returns                             148.85915
return_std                          60.871008
average_reward                       4.805947
round_time             0 days 00:05:53.400298
episodes_test                           290.0
episode_length_test                 34.417241
returns_test                       165.075291
return_std_test                     68.708602
average_reward_test                  4.796347
round_time_test        0 days 00:00:12.715975
round_time_total       0 days 00:05:53.401752
loss_total                      135042084.726
loss_critic                     168805733.232
loss_actor                      -12521.768577
memory_size                        109460.639 

=== epoch 3/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  318
episode_length                       31.40566
returns                            150.638808
return_std                          61.226746
average_reward                       4.796397
round_time             0 days 00:05:53.263311
episodes_test                           282.0
episode_length_test                 35.414894
returns_test                        169.92218
return_std_test                      73.56706
average_reward_test                   4.79808
round_time_test        0 days 00:00:12.494420
round_time_total       0 days 00:05:53.264767
loss_total                      141466409.282
loss_critic                      176836208.86
loss_actor                      -12800.348069
memory_size                        110486.456 

=== epoch 3/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  307
episode_length                      32.371336
returns                            155.407552
return_std                          62.899736
average_reward                       4.800583
round_time             0 days 00:05:54.195710
episodes_test                           322.0
episode_length_test                 30.913043
returns_test                       149.197117
return_std_test                     54.110529
average_reward_test                  4.825408
round_time_test        0 days 00:00:12.840659
round_time_total       0 days 00:05:54.196805
loss_total                      148451862.462
loss_critic                     185568023.856
loss_actor                       -12797.87264
memory_size                       111451.8545 

=== epoch 3/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  300
episode_length                      33.136667
returns                            158.631473
return_std                          63.302912
average_reward                       4.787618
round_time             0 days 00:05:55.178174
episodes_test                           299.0
episode_length_test                 33.384615
returns_test                       159.072094
return_std_test                     68.549621
average_reward_test                  4.765127
round_time_test        0 days 00:00:12.613588
round_time_total       0 days 00:05:55.179286
loss_total                       154043997.04
loss_critic                     192558271.912
loss_actor                      -13117.171378
memory_size                       112426.9865 

=== epoch 3/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:48,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  297
episode_length                       33.52862
returns                            160.560289
return_std                          61.826811
average_reward                       4.788815
round_time             0 days 00:05:53.865279
episodes_test                           296.0
episode_length_test                 33.756757
returns_test                       161.294744
return_std_test                     67.418394
average_reward_test                   4.77816
round_time_test        0 days 00:00:13.075050
round_time_total       0 days 00:05:53.866546
loss_total                      155087288.436
loss_critic                     193862340.492
loss_actor                      -12933.073164
memory_size                       113399.3965 

=== epoch 3/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  300
episode_length                      33.226667
returns                            159.149363
return_std                          62.878071
average_reward                       4.789165
round_time             0 days 00:05:54.922672
episodes_test                           296.0
episode_length_test                 33.689189
returns_test                       161.482948
return_std_test                     64.267723
average_reward_test                  4.794506
round_time_test        0 days 00:00:12.510515
round_time_total       0 days 00:05:54.924055
loss_total                      151318355.522
loss_critic                     189151148.408
loss_actor                      -12830.011488
memory_size                        114421.422 

=== epoch 3/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  304
episode_length                      32.815789
returns                            157.062562
return_std                          62.201101
average_reward                       4.785925
round_time             0 days 00:05:55.691966
episodes_test                           321.0
episode_length_test                 31.152648
returns_test                       148.768721
return_std_test                     56.718829
average_reward_test                  4.775476
round_time_test        0 days 00:00:12.786796
round_time_total       0 days 00:05:55.693074
loss_total                       150582685.52
loss_critic                     188231491.204
loss_actor                      -12549.878347
memory_size                       115349.3565 

=== epoch 3/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  305
episode_length                      32.704918
returns                            156.231631
return_std                          61.482129
average_reward                       4.777497
round_time             0 days 00:05:55.395021
episodes_test                           292.0
episode_length_test                 34.232877
returns_test                       163.555226
return_std_test                     61.593968
average_reward_test                   4.77777
round_time_test        0 days 00:00:12.385242
round_time_total       0 days 00:05:55.396147
loss_total                      153713109.106
loss_critic                     192144472.192
loss_actor                      -12356.779843
memory_size                       116294.2485 

=== epoch 3/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:22,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  310
episode_length                      32.177419
returns                            153.975804
return_std                          61.059898
average_reward                        4.78555
round_time             0 days 00:05:56.499559
episodes_test                           332.0
episode_length_test                 30.054217
returns_test                       143.953856
return_std_test                     56.367704
average_reward_test                  4.790144
round_time_test        0 days 00:00:12.454691
round_time_total       0 days 00:05:56.500965
loss_total                      159217814.184
loss_critic                     199025386.932
loss_actor                      -12489.716152
memory_size                        117200.995 

=== epoch 3/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  317
episode_length                      31.454259
returns                            150.704024
return_std                          60.165384
average_reward                       4.791564
round_time             0 days 00:05:52.657276
episodes_test                           308.0
episode_length_test                 32.383117
returns_test                       155.622416
return_std_test                     60.857004
average_reward_test                  4.805791
round_time_test        0 days 00:00:12.600984
round_time_total       0 days 00:05:52.658403
loss_total                      165904896.236
loss_critic                     207384269.204
loss_actor                      -12611.414785
memory_size                       118075.4205 

=== epoch 3/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:40,  4.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  319
episode_length                       31.23511
returns                            149.785684
return_std                           57.58116
average_reward                       4.794267
round_time             0 days 00:05:56.820122
episodes_test                           347.0
episode_length_test                 28.804035
returns_test                        137.97682
return_std_test                     55.673653
average_reward_test                  4.790209
round_time_test        0 days 00:00:12.480173
round_time_total       0 days 00:05:56.821213
loss_total                      175438899.172
loss_critic                     219301892.424
loss_actor                      -13089.002624
memory_size                        118986.872 

=== epoch 3/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:27,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  325
episode_length                      30.698462
returns                            147.334575
return_std                          56.455331
average_reward                       4.798701
round_time             0 days 00:05:54.852683
episodes_test                           328.0
episode_length_test                 30.466463
returns_test                         145.7672
return_std_test                     58.025774
average_reward_test                  4.784561
round_time_test        0 days 00:00:12.433147
round_time_total       0 days 00:05:54.854243
loss_total                      185054970.184
loss_critic                     231322137.652
loss_actor                       -13715.68711
memory_size                        119899.392 

=== epoch 3/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  324
episode_length                      30.691358
returns                            146.950072
return_std                           56.23345
average_reward                       4.790921
round_time             0 days 00:05:57.347568
episodes_test                           306.0
episode_length_test                 32.562092
returns_test                       154.610759
return_std_test                     65.460882
average_reward_test                   4.74872
round_time_test        0 days 00:00:12.397123
round_time_total       0 days 00:05:57.348677
loss_total                      195497329.216
loss_critic                     244375210.728
loss_actor                      -14212.867395
memory_size                        120772.165 

=== epoch 3/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  329
episode_length                      30.343465
returns                            145.455006
return_std                          55.666995
average_reward                       4.793733
round_time             0 days 00:05:56.863075
episodes_test                           316.0
episode_length_test                 31.556962
returns_test                       151.095503
return_std_test                     60.843994
average_reward_test                  4.788055
round_time_test        0 days 00:00:12.649290
round_time_total       0 days 00:05:56.864155
loss_total                       208449837.18
loss_critic                     260565979.896
loss_actor                      -14751.621132
memory_size                        121654.606 

=== epoch 3/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:48,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  330
episode_length                      30.245455
returns                             144.76564
return_std                          56.775887
average_reward                       4.786465
round_time             0 days 00:05:55.464015
episodes_test                           350.0
episode_length_test                 28.545714
returns_test                       136.600712
return_std_test                     52.281734
average_reward_test                  4.785342
round_time_test        0 days 00:00:12.687883
round_time_total       0 days 00:05:55.465232
loss_total                       215942350.28
loss_critic                     269931641.268
loss_actor                      -14832.316605
memory_size                       122518.2865 

=== epoch 3/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  328
episode_length                      30.387195
returns                            145.299859
return_std                          57.465103
average_reward                       4.782714
round_time             0 days 00:05:57.277300
episodes_test                           337.0
episode_length_test                 29.667656
returns_test                       141.806082
return_std_test                     56.510182
average_reward_test                  4.779851
round_time_test        0 days 00:00:12.555218
round_time_total       0 days 00:05:57.278405
loss_total                        218841235.8
loss_critic                     273555254.204
loss_actor                      -14857.817801
memory_size                        123432.248 

=== epoch 3/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:48,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  318
episode_length                      31.279874
returns                            149.670148
return_std                          57.778558
average_reward                       4.785464
round_time             0 days 00:05:56.700653
episodes_test                           305.0
episode_length_test                 32.665574
returns_test                       156.104463
return_std_test                     63.825008
average_reward_test                  4.778208
round_time_test        0 days 00:00:12.475343
round_time_total       0 days 00:05:56.701767
loss_total                      229240873.308
loss_critic                     286554884.168
loss_actor                      -15188.572531
memory_size                       124374.2715 

=== epoch 3/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  319
episode_length                      31.216301
returns                            149.550738
return_std                          60.466292
average_reward                       4.791707
round_time             0 days 00:05:56.146482
episodes_test                           314.0
episode_length_test                 31.799363
returns_test                       153.352927
return_std_test                     64.126728
average_reward_test                  4.822735
round_time_test        0 days 00:00:12.653516
round_time_total       0 days 00:05:56.147598
loss_total                       228668236.48
loss_critic                     285839003.484
loss_actor                      -14851.773298
memory_size                        125365.345 

=== epoch 3/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  313
episode_length                       31.70607
returns                            151.654714
return_std                          63.715098
average_reward                        4.78132
round_time             0 days 00:05:56.038475
episodes_test                           326.0
episode_length_test                 30.641104
returns_test                       145.268516
return_std_test                     57.652713
average_reward_test                  4.741053
round_time_test        0 days 00:00:12.493769
round_time_total       0 days 00:05:56.039565
loss_total                      230440141.632
loss_critic                     288053795.896
loss_actor                      -14495.045191
memory_size                       126289.8475 

=== epoch 3/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  307
episode_length                      32.319218
returns                            154.098887
return_std                          64.805531
average_reward                       4.770565
round_time             0 days 00:05:55.809953
episodes_test                           306.0
episode_length_test                 32.637255
returns_test                       156.229031
return_std_test                     63.579708
average_reward_test                  4.787041
round_time_test        0 days 00:00:12.634904
round_time_total       0 days 00:05:55.811159
loss_total                      225509607.964
loss_critic                     281890553.392
loss_actor                      -14193.436728
memory_size                        127241.653 

=== epoch 3/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  301
episode_length                      33.063123
returns                            158.026985
return_std                          68.199741
average_reward                       4.779996
round_time             0 days 00:05:59.908371
episodes_test                           292.0
episode_length_test                 34.130137
returns_test                       163.551038
return_std_test                     67.210103
average_reward_test                  4.792427
round_time_test        0 days 00:00:12.297188
round_time_total       0 days 00:05:59.909490
loss_total                      225103924.084
loss_critic                     281383397.412
loss_actor                      -13987.754391
memory_size                       128268.2185 

=== epoch 3/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:24,  4.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  304
episode_length                         32.625
returns                            155.713206
return_std                          68.473013
average_reward                       4.769078
round_time             0 days 00:05:56.864249
episodes_test                           310.0
episode_length_test                 32.245161
returns_test                       153.661233
return_std_test                     59.375006
average_reward_test                   4.76545
round_time_test        0 days 00:00:12.887156
round_time_total       0 days 00:05:56.865355
loss_total                      247662363.324
loss_critic                     309581612.528
loss_actor                      -14655.710049
memory_size                       129255.1405 

=== epoch 3/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  311
episode_length                      32.102894
returns                            153.395033
return_std                          66.619393
average_reward                       4.778321
round_time             0 days 00:05:59.655997
episodes_test                           327.0
episode_length_test                 30.529052
returns_test                       145.488611
return_std_test                     57.272049
average_reward_test                  4.765812
round_time_test        0 days 00:00:12.531352
round_time_total       0 days 00:05:59.657166
loss_total                      240452943.208
loss_critic                      300569675.68
loss_actor                      -14006.641666
memory_size                       130163.1505 

=== epoch 3/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  307
episode_length                      32.462541
returns                            155.217436
return_std                          64.824707
average_reward                       4.781221
round_time             0 days 00:05:57.066860
episodes_test                           307.0
episode_length_test                 32.456026
returns_test                       154.151788
return_std_test                     60.583176
average_reward_test                  4.751104
round_time_test        0 days 00:00:12.640777
round_time_total       0 days 00:05:57.067972
loss_total                      236409228.556
loss_critic                     295514835.208
loss_actor                      -13216.670339
memory_size                       131048.9495 

=== epoch 3/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  317
episode_length                      31.347003
returns                            150.077131
return_std                          61.710307
average_reward                       4.789772
round_time             0 days 00:05:58.461308
episodes_test                           321.0
episode_length_test                  31.11838
returns_test                       149.004608
return_std_test                     55.797968
average_reward_test                  4.788338
round_time_test        0 days 00:00:12.684016
round_time_total       0 days 00:05:58.462837
loss_total                      240395198.492
loss_critic                      300497153.14
loss_actor                      -12641.891845
memory_size                       131959.7785 

=== epoch 3/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  321
episode_length                      31.062305
returns                            148.018755
return_std                          58.296415
average_reward                        4.76675
round_time             0 days 00:05:59.244464
episodes_test                           301.0
episode_length_test                 33.146179
returns_test                       158.960553
return_std_test                     56.596006
average_reward_test                  4.795642
round_time_test        0 days 00:00:12.577783
round_time_total       0 days 00:05:59.245763
loss_total                      230869136.544
loss_critic                     288589553.192
loss_actor                      -12548.569572
memory_size                       132819.0015 

=== epoch 3/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:42,  4.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  321
episode_length                      30.987539
returns                            147.956945
return_std                          57.199389
average_reward                       4.775308
round_time             0 days 00:06:01.003815
episodes_test                           287.0
episode_length_test                 34.641115
returns_test                       165.173792
return_std_test                      63.91866
average_reward_test                  4.768181
round_time_test        0 days 00:00:12.588739
round_time_total       0 days 00:06:01.004928
loss_total                      235483324.912
loss_critic                     294357244.152
loss_actor                      -12371.618534
memory_size                        133777.718 

=== epoch 3/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  310
episode_length                      32.158065
returns                            152.837029
return_std                          57.527556
average_reward                       4.752626
round_time             0 days 00:06:00.277474
episodes_test                           286.0
episode_length_test                 34.874126
returns_test                       166.895793
return_std_test                      66.18352
average_reward_test                  4.785909
round_time_test        0 days 00:00:12.557985
round_time_total       0 days 00:06:00.278581
loss_total                      234858160.884
loss_critic                     293575762.708
loss_actor                      -12265.519497
memory_size                       134791.9485 

=== epoch 3/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:35,  5.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  309
episode_length                      32.249191
returns                            153.645689
return_std                          58.014887
average_reward                       4.763082
round_time             0 days 00:05:58.632535
episodes_test                           293.0
episode_length_test                  34.05802
returns_test                        162.65251
return_std_test                     65.937536
average_reward_test                  4.775923
round_time_test        0 days 00:00:12.733305
round_time_total       0 days 00:05:58.633947
loss_total                      222313294.076
loss_critic                      277894443.36
loss_actor                      -11323.799579
memory_size                       135801.2705 

=== epoch 3/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  294
episode_length                      33.942177
returns                            161.303811
return_std                          56.908655
average_reward                       4.752062
round_time             0 days 00:05:59.421898
episodes_test                           282.0
episode_length_test                 35.262411
returns_test                       167.483832
return_std_test                     60.192689
average_reward_test                   4.75025
round_time_test        0 days 00:00:12.703518
round_time_total       0 days 00:05:59.422989
loss_total                       236847042.37
loss_critic                     296061638.172
loss_actor                      -11360.474821
memory_size                        136813.536 

=== epoch 3/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  288
episode_length                      34.652778
returns                            164.948045
return_std                          58.203784
average_reward                        4.75983
round_time             0 days 00:05:59.754864
episodes_test                           278.0
episode_length_test                 35.877698
returns_test                       171.836199
return_std_test                     66.134929
average_reward_test                  4.788943
round_time_test        0 days 00:00:12.947005
round_time_total       0 days 00:05:59.756213
loss_total                       232782710.76
loss_critic                     290981142.264
loss_actor                      -11035.370906
memory_size                        137881.098 

=== epoch 3/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:56,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  286
episode_length                      34.944056
returns                            166.388825
return_std                          59.307239
average_reward                       4.761728
round_time             0 days 00:05:58.941763
episodes_test                           287.0
episode_length_test                 34.790941
returns_test                       164.540655
return_std_test                     62.444527
average_reward_test                  4.729487
round_time_test        0 days 00:00:12.578721
round_time_total       0 days 00:05:58.943058
loss_total                      234852775.276
loss_critic                     293568662.084
loss_actor                      -10791.451624
memory_size                        138912.588 

=== epoch 3/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  289
episode_length                      34.359862
returns                            164.123767
return_std                          59.079958
average_reward                       4.775092
round_time             0 days 00:05:59.618944
episodes_test                           293.0
episode_length_test                 34.010239
returns_test                       162.627758
return_std_test                     57.947181
average_reward_test                  4.779937
round_time_test        0 days 00:00:12.731262
round_time_total       0 days 00:05:59.620353
loss_total                      236044399.644
loss_critic                     295058149.304
loss_actor                      -10618.789536
memory_size                       139878.5135 

=== epoch 3/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  298
episode_length                      33.493289
returns                            159.765893
return_std                          56.939531
average_reward                       4.768519
round_time             0 days 00:06:02.176720
episodes_test                           311.0
episode_length_test                 32.073955
returns_test                       152.739509
return_std_test                     53.635369
average_reward_test                  4.762705
round_time_test        0 days 00:00:12.490467
round_time_total       0 days 00:06:02.178010
loss_total                       240159638.58
loss_critic                      300202324.92
loss_actor                      -11127.926465
memory_size                       140765.7805 

=== epoch 3/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  302
episode_length                      32.927152
returns                            157.310612
return_std                          58.298313
average_reward                       4.777478
round_time             0 days 00:06:01.878138
episodes_test                           315.0
episode_length_test                 31.736508
returns_test                       152.043118
return_std_test                     58.313081
average_reward_test                  4.790819
round_time_test        0 days 00:00:12.580493
round_time_total       0 days 00:06:01.879237
loss_total                      232964165.356
loss_critic                     291207889.564
loss_actor                      -10751.188451
memory_size                       141730.0875 

=== epoch 3/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  307
episode_length                      32.495114
returns                            155.638906
return_std                          56.769538
average_reward                       4.790334
round_time             0 days 00:06:02.072332
episodes_test                           304.0
episode_length_test                 32.888158
returns_test                       156.019863
return_std_test                     56.406926
average_reward_test                  4.743978
round_time_test        0 days 00:00:12.663588
round_time_total       0 days 00:06:02.073431
loss_total                      240561473.464
loss_critic                     300704418.956
loss_actor                      -10328.259161
memory_size                       142692.1315 

=== epoch 3/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  311
episode_length                      32.064309
returns                            153.134843
return_std                          55.084867
average_reward                        4.77675
round_time             0 days 00:05:59.747781
episodes_test                           332.0
episode_length_test                 30.105422
returns_test                       143.983923
return_std_test                     55.104849
average_reward_test                  4.782667
round_time_test        0 days 00:00:12.630219
round_time_total       0 days 00:05:59.749092
loss_total                        246233238.6
loss_critic                      307794080.48
loss_actor                      -10147.023925
memory_size                        143651.306 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 4/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:40<00:00,  5.87it/s]
episodes                                   64
episode_length                      30.859375
returns                            147.338417
return_std                          56.026119
average_reward                       4.773654
round_time             0 days 00:05:40.748776
episodes_test                           313.0
episode_length_test                 31.859425
returns_test                       151.812885
return_std_test                     53.777185
average_reward_test                  4.765686
round_time_test        0 days 00:00:12.609618
round_time_total       0 days 00:05:40.750153
loss_total                       233685573.94
loss_critic                     292109419.284
loss_actor                       -9827.143733
memory_size                       144587.5025 

=== epoch 4/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                  127
episode_length                      31.425197
returns                            149.928952
return_std                           62.58789
average_reward                       4.771066
round_time             0 days 00:05:43.901394
episodes_test                           292.0
episode_length_test                 34.092466
returns_test                       162.398046
return_std_test                     60.342324
average_reward_test                  4.764572
round_time_test        0 days 00:00:12.698404
round_time_total       0 days 00:05:43.902493
loss_total                       236911291.58
loss_critic                     296141412.888
loss_actor                       -9213.868386
memory_size                       145540.9155 

=== epoch 4/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                  199
episode_length                      30.090452
returns                            142.654769
return_std                          58.640516
average_reward                       4.740988
round_time             0 days 00:05:48.671153
episodes_test                           321.0
episode_length_test                 31.080997
returns_test                       149.038073
return_std_test                     57.371389
average_reward_test                  4.795692
round_time_test        0 days 00:00:12.371978
round_time_total       0 days 00:05:48.672248
loss_total                      248552110.076
loss_critic                     310692403.728
loss_actor                       -9082.837389
memory_size                       146384.2855 

=== epoch 4/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  269
episode_length                      29.654275
returns                            140.662989
return_std                          55.398919
average_reward                        4.74388
round_time             0 days 00:05:49.947656
episodes_test                           379.0
episode_length_test                 26.287599
returns_test                       124.589968
return_std_test                     40.112366
average_reward_test                   4.73995
round_time_test        0 days 00:00:12.853626
round_time_total       0 days 00:05:49.948794
loss_total                      254571972.874
loss_critic                      318217057.42
loss_actor                       -8386.335833
memory_size                         147153.59 

=== epoch 4/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  334
episode_length                       29.89521
returns                            142.275805
return_std                          54.926856
average_reward                       4.759395
round_time             0 days 00:05:48.891275
episodes_test                           306.0
episode_length_test                 32.656863
returns_test                       154.656904
return_std_test                     58.029611
average_reward_test                  4.735851
round_time_test        0 days 00:00:12.680044
round_time_total       0 days 00:05:48.892650
loss_total                       195700744.32
loss_critic                      244627580.38
loss_actor                       -6615.500841
memory_size                        148020.131 

=== epoch 4/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.78it/s]
episodes                                  333
episode_length                      29.891892
returns                            142.385968
return_std                          53.680201
average_reward                       4.762625
round_time             0 days 00:05:46.559528
episodes_test                           327.0
episode_length_test                 30.538226
returns_test                       145.608504
return_std_test                      56.63254
average_reward_test                  4.768054
round_time_test        0 days 00:00:12.851702
round_time_total       0 days 00:05:46.560644
loss_total                      171455150.186
loss_critic                     214320195.322
loss_actor                       -5045.547493
memory_size                        148933.137 

=== epoch 4/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  332
episode_length                      29.861446
returns                            142.429413
return_std                          53.166927
average_reward                       4.764502
round_time             0 days 00:05:49.606157
episodes_test                           311.0
episode_length_test                 32.073955
returns_test                       152.031215
return_std_test                     53.002062
average_reward_test                  4.740346
round_time_test        0 days 00:00:12.591071
round_time_total       0 days 00:05:49.607374
loss_total                       151212085.48
loss_critic                     189015959.546
loss_actor                         -3423.9086
memory_size                       149843.2305 

=== epoch 4/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:27,  4.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  317
episode_length                      31.318612
returns                             150.05211
return_std                          58.944094
average_reward                       4.790562
round_time             0 days 00:05:50.891632
episodes_test                           314.0
episode_length_test                 31.843949
returns_test                       151.941009
return_std_test                     55.642935
average_reward_test                  4.771438
round_time_test        0 days 00:00:12.655777
round_time_total       0 days 00:05:50.892733
loss_total                      132038421.499
loss_critic                     165048617.042
loss_actor                       -2371.617376
memory_size                       150867.4925 

=== epoch 4/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  314
episode_length                      31.789809
returns                             152.16694
return_std                          59.348755
average_reward                       4.786831
round_time             0 days 00:05:51.768925
episodes_test                           297.0
episode_length_test                 33.565657
returns_test                        159.85881
return_std_test                     58.956972
average_reward_test                  4.763021
round_time_test        0 days 00:00:12.813351
round_time_total       0 days 00:05:51.770073
loss_total                      117425382.684
loss_critic                     146782102.192
loss_actor                       -1504.599499
memory_size                       151812.0555 

=== epoch 4/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  326
episode_length                      30.588957
returns                            146.013156
return_std                          57.089745
average_reward                       4.773566
round_time             0 days 00:05:52.543342
episodes_test                           335.0
episode_length_test                 29.808955
returns_test                       141.904685
return_std_test                     43.758308
average_reward_test                  4.760655
round_time_test        0 days 00:00:12.575164
round_time_total       0 days 00:05:52.544475
loss_total                      103031724.088
loss_critic                     128789871.261
loss_actor                        -872.098348
memory_size                       152575.6055 

=== epoch 4/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  341
episode_length                      29.225806
returns                            139.193536
return_std                          54.734183
average_reward                       4.762796
round_time             0 days 00:05:54.470672
episodes_test                           377.0
episode_length_test                 26.464191
returns_test                       126.294413
return_std_test                     38.116193
average_reward_test                  4.771873
round_time_test        0 days 00:00:12.935448
round_time_total       0 days 00:05:54.471904
loss_total                      92799230.5275
loss_critic                     115999193.363
loss_actor                        -628.664351
memory_size                        153243.976 

=== epoch 4/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  362
episode_length                      27.582873
returns                            130.922301
return_std                          49.192129
average_reward                       4.746255
round_time             0 days 00:05:50.710423
episodes_test                           407.0
episode_length_test                 24.496314
returns_test                       115.451884
return_std_test                     36.602665
average_reward_test                  4.713313
round_time_test        0 days 00:00:12.695240
round_time_total       0 days 00:05:50.711805
loss_total                       77735258.474
loss_critic                      97169002.579
loss_actor                         276.086899
memory_size                        153920.062 

=== epoch 4/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:19,  4.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  387
episode_length                      25.775194
returns                            121.674897
return_std                          39.125412
average_reward                       4.720723
round_time             0 days 00:05:50.301433
episodes_test                           405.0
episode_length_test                 24.688889
returns_test                       115.831747
return_std_test                     38.790693
average_reward_test                  4.691677
round_time_test        0 days 00:00:12.752553
round_time_total       0 days 00:05:50.302540
loss_total                      59255906.1315
loss_critic                      74069527.257
loss_actor                        1417.027194
memory_size                        154505.139 

=== epoch 4/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  412
episode_length                      24.223301
returns                            114.216663
return_std                          36.008179
average_reward                       4.715529
round_time             0 days 00:05:53.477242
episodes_test                           404.0
episode_length_test                 24.715347
returns_test                       115.475526
return_std_test                     44.173363
average_reward_test                  4.672283
round_time_test        0 days 00:00:12.521745
round_time_total       0 days 00:05:53.478359
loss_total                      52481929.2515
loss_critic                     65601873.5455
loss_actor                        2147.325179
memory_size                       155068.2425 

=== epoch 4/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  444
episode_length                      22.448198
returns                            105.872391
return_std                          34.672781
average_reward                       4.717089
round_time             0 days 00:05:55.437104
episodes_test                           513.0
episode_length_test                 19.475634
returns_test                        92.175781
return_std_test                     23.885358
average_reward_test                  4.732941
round_time_test        0 days 00:00:12.660646
round_time_total       0 days 00:05:55.438515
loss_total                       32958866.568
loss_critic                    41197935.64725
loss_actor                         2587.22069
memory_size                       155409.6745 

=== epoch 4/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:44,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  481
episode_length                      20.760915
returns                             98.424344
return_std                          32.546278
average_reward                       4.741108
round_time             0 days 00:05:53.189777
episodes_test                           580.0
episode_length_test                  17.22069
returns_test                         82.64149
return_std_test                     15.272175
average_reward_test                  4.799145
round_time_test        0 days 00:00:12.839012
round_time_total       0 days 00:05:53.190883
loss_total                    18812041.136875
loss_critic                    23514396.64775
loss_actor                        2617.671852
memory_size                        155565.154 

=== epoch 4/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  495
episode_length                      20.117172
returns                             96.711663
return_std                          30.460054
average_reward                       4.807921
round_time             0 days 00:05:51.345506
episodes_test                           549.0
episode_length_test                 18.209472
returns_test                        89.942139
return_std_test                     25.744746
average_reward_test                  4.939287
round_time_test        0 days 00:00:12.802160
round_time_total       0 days 00:05:51.346825
loss_total                     6432582.705125
loss_critic                       8040208.101
loss_actor                        2080.729558
memory_size                       155832.1005 

=== epoch 4/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:59,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  509
episode_length                      19.599214
returns                             95.570763
return_std                          28.659839
average_reward                       4.876144
round_time             0 days 00:05:55.297630
episodes_test                           434.0
episode_length_test                 23.025346
returns_test                       117.330862
return_std_test                     32.045848
average_reward_test                  5.095551
round_time_test        0 days 00:00:13.034859
round_time_total       0 days 00:05:55.298733
loss_total                     3187993.514187
loss_critic                    3984671.510687
loss_actor                        1281.226704
memory_size                       156254.2555 

=== epoch 4/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:30,  4.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  489
episode_length                      20.398773
returns                            101.157502
return_std                          34.221102
average_reward                       4.958983
round_time             0 days 00:05:54.657405
episodes_test                           426.0
episode_length_test                 23.431925
returns_test                       119.731767
return_std_test                     31.042997
average_reward_test                  5.109609
round_time_test        0 days 00:00:13.074448
round_time_total       0 days 00:05:54.658691
loss_total                     2265399.516297
loss_critic                    2831593.965188
loss_actor                          621.61759
memory_size                        156835.623 

=== epoch 4/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  452
episode_length                      22.086283
returns                            110.993554
return_std                          38.680153
average_reward                       5.025458
round_time             0 days 00:05:56.129794
episodes_test                           314.0
episode_length_test                 31.789809
returns_test                       163.224655
return_std_test                     37.160885
average_reward_test                  5.134329
round_time_test        0 days 00:00:14.253244
round_time_total       0 days 00:05:56.130905
loss_total                     2094297.649594
loss_critic                       2617819.527
loss_actor                         209.699975
memory_size                        157634.886 

=== epoch 4/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  405
episode_length                      24.674074
returns                            124.652888
return_std                          42.349607
average_reward                       5.052127
round_time             0 days 00:05:57.980498
episodes_test                           330.0
episode_length_test                 30.281818
returns_test                       153.110369
return_std_test                     46.957954
average_reward_test                  5.056081
round_time_test        0 days 00:00:13.550935
round_time_total       0 days 00:05:57.981601
loss_total                     2806722.563711
loss_critic                    3508415.108164
loss_actor                         -47.793362
memory_size                       158474.2665 

=== epoch 4/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  395
episode_length                      25.303797
returns                            127.270676
return_std                          42.704078
average_reward                       5.029721
round_time             0 days 00:05:56.018595
episodes_test                           318.0
episode_length_test                 31.396226
returns_test                       159.994808
return_std_test                     49.693871
average_reward_test                   5.09589
round_time_test        0 days 00:00:13.377365
round_time_total       0 days 00:05:56.019799
loss_total                      490216.093434
loss_critic                     612810.807633
loss_actor                        -162.822756
memory_size                        159149.892 

=== epoch 4/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  384
episode_length                      25.960938
returns                            129.644228
return_std                          44.362217
average_reward                       4.993431
round_time             0 days 00:05:53.675158
episodes_test                           422.0
episode_length_test                 23.680095
returns_test                       116.460138
return_std_test                     44.377687
average_reward_test                  4.918044
round_time_test        0 days 00:00:12.830695
round_time_total       0 days 00:05:53.676264
loss_total                      215443.758125
loss_critic                     269383.363602
loss_actor                        -314.688526
memory_size                         159669.56 

=== epoch 4/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:23,  4.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  381
episode_length                      26.131234
returns                            129.146729
return_std                          50.194944
average_reward                       4.943202
round_time             0 days 00:05:55.966417
episodes_test                           374.0
episode_length_test                 26.724599
returns_test                       130.493447
return_std_test                     58.898573
average_reward_test                  4.882907
round_time_test        0 days 00:00:13.034792
round_time_total       0 days 00:05:55.967520
loss_total                     1405806.047867
loss_critic                    1757370.783082
loss_actor                        -453.083425
memory_size                       160352.8195 

=== epoch 4/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:33,  4.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:05<00:00,  5.48it/s]
episodes                                  345
episode_length                      28.791304
returns                            140.480023
return_std                          79.226362
average_reward                       4.878884
round_time             0 days 00:06:05.613936
episodes_test                           203.0
episode_length_test                 49.241379
returns_test                       246.142089
return_std_test                     144.21619
average_reward_test                  4.998636
round_time_test        0 days 00:00:14.028521
round_time_total       0 days 00:06:05.615302
loss_total                       78728.118262
loss_critic                      98549.510812
loss_actor                        -557.458987
memory_size                         161425.59 

=== epoch 4/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:35,  3.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:10<00:00,  5.40it/s]
episodes                                  316
episode_length                      31.484177
returns                            153.146938
return_std                          98.744491
average_reward                       4.862884
round_time             0 days 00:06:11.044420
episodes_test                           132.0
episode_length_test                 75.424242
returns_test                       360.278537
return_std_test                    122.429204
average_reward_test                  4.776147
round_time_test        0 days 00:00:14.767836
round_time_total       0 days 00:06:11.045549
loss_total                      554299.078664
loss_critic                     693043.469074
loss_actor                        -678.608853
memory_size                       162865.9415 

=== epoch 4/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:33<00:00,  5.08it/s]
episodes                                  261
episode_length                      38.045977
returns                            184.774852
return_std                         119.412873
average_reward                       4.855758
round_time             0 days 00:06:34.218921
episodes_test                           360.0
episode_length_test                 27.697222
returns_test                       137.939917
return_std_test                     48.535253
average_reward_test                  4.980342
round_time_test        0 days 00:00:13.664550
round_time_total       0 days 00:06:34.220462
loss_total                     1053141.969354
loss_critic                    1316620.102041
loss_actor                        -770.787617
memory_size                       164208.7555 

=== epoch 4/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  210
episode_length                      47.219048
returns                            230.199087
return_std                         128.625448
average_reward                       4.873123
round_time             0 days 00:06:00.717187
episodes_test                           125.0
episode_length_test                    79.856
returns_test                       386.955973
return_std_test                     86.673409
average_reward_test                  4.845756
round_time_test        0 days 00:00:14.519125
round_time_total       0 days 00:06:00.718326
loss_total                     1401346.755665
loss_critic                    1751890.046801
loss_actor                        -826.765131
memory_size                       165696.0865 

=== epoch 4/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  172
episode_length                      57.843023
returns                            283.213216
return_std                         126.186139
average_reward                       4.894167
round_time             0 days 00:05:58.138686
episodes_test                           131.0
episode_length_test                 76.175573
returns_test                       373.358914
return_std_test                    125.031427
average_reward_test                  4.900828
round_time_test        0 days 00:00:15.150593
round_time_total       0 days 00:05:58.140069
loss_total                       178453.59524
loss_critic                     223290.432319
loss_actor                        -893.775732
memory_size                       167119.1065 

=== epoch 4/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  168
episode_length                      58.702381
returns                            289.180033
return_std                         113.894656
average_reward                       4.926735
round_time             0 days 00:05:58.717234
episodes_test                           145.0
episode_length_test                 68.703448
returns_test                        330.47029
return_std_test                    106.489217
average_reward_test                  4.809501
round_time_test        0 days 00:00:14.680578
round_time_total       0 days 00:05:58.718356
loss_total                      332894.046062
loss_critic                      416355.44699
loss_actor                        -951.588965
memory_size                       168572.6245 

=== epoch 4/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  159
episode_length                      62.622642
returns                            309.182838
return_std                         108.959781
average_reward                       4.935951
round_time             0 days 00:05:57.582127
episodes_test                           132.0
episode_length_test                 75.537879
returns_test                        369.83143
return_std_test                    146.261333
average_reward_test                  4.895757
round_time_test        0 days 00:00:14.611293
round_time_total       0 days 00:05:57.583644
loss_total                       678002.48096
loss_critic                     847750.724332
loss_actor                        -990.640783
memory_size                       170048.6675 

=== epoch 4/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  157
episode_length                      63.273885
returns                            314.177902
return_std                         103.783339
average_reward                       4.961793
round_time             0 days 00:05:56.952575
episodes_test                           128.0
episode_length_test                 77.679688
returns_test                       372.513774
return_std_test                    121.352406
average_reward_test                   4.79516
round_time_test        0 days 00:00:14.626344
round_time_total       0 days 00:05:56.953826
loss_total                      460017.511448
loss_critic                     575281.797061
loss_actor                       -1039.804332
memory_size                       171537.6305 

=== epoch 4/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:46,  4.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  152
episode_length                      65.519737
returns                            324.944087
return_std                         103.645511
average_reward                       4.961394
round_time             0 days 00:05:59.860905
episodes_test                           124.0
episode_length_test                 80.564516
returns_test                       394.371619
return_std_test                    112.851266
average_reward_test                  4.895092
round_time_test        0 days 00:00:14.629083
round_time_total       0 days 00:05:59.862012
loss_total                      855113.177418
loss_critic                    1069162.528984
loss_actor                       -1084.288417
memory_size                       173025.3185 

=== epoch 4/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:24,  4.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  152
episode_length                      65.335526
returns                             325.18526
return_std                         104.765722
average_reward                       4.974188
round_time             0 days 00:05:59.233161
episodes_test                           117.0
episode_length_test                 85.068376
returns_test                       425.443136
return_std_test                     134.83774
average_reward_test                  5.001886
round_time_test        0 days 00:00:15.285422
round_time_total       0 days 00:05:59.234247
loss_total                     1268688.487896
loss_critic                    1586139.507301
loss_actor                       -1115.557134
memory_size                        174515.497 

=== epoch 4/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  152
episode_length                      65.565789
returns                            326.414659
return_std                         107.954527
average_reward                       4.976509
round_time             0 days 00:06:00.613940
episodes_test                           135.0
episode_length_test                 73.896296
returns_test                        376.11233
return_std_test                     92.556062
average_reward_test                  5.089436
round_time_test        0 days 00:00:14.842136
round_time_total       0 days 00:06:00.615231
loss_total                     3350713.670428
loss_critic                     4188663.70757
loss_actor                       -1087.131185
memory_size                        175989.269 

=== epoch 4/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  153
episode_length                      65.006536
returns                            322.246051
return_std                         125.374336
average_reward                       4.954919
round_time             0 days 00:06:01.944735
episodes_test                           137.0
episode_length_test                 72.927007
returns_test                       368.726713
return_std_test                    112.332417
average_reward_test                  5.055944
round_time_test        0 days 00:00:14.934048
round_time_total       0 days 00:06:01.945968
loss_total                     1936742.378064
loss_critic                    2421197.965855
loss_actor                       -1080.342454
memory_size                       177469.0135 

=== epoch 4/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  161
episode_length                       61.63354
returns                             305.87766
return_std                         125.313716
average_reward                       4.963894
round_time             0 days 00:05:56.056870
episodes_test                           228.0
episode_length_test                 43.745614
returns_test                       212.582979
return_std_test                    108.393647
average_reward_test                  4.859981
round_time_test        0 days 00:00:13.793643
round_time_total       0 days 00:05:56.058290
loss_total                      769691.510098
loss_critic                     962394.364975
loss_actor                       -1119.959312
memory_size                        178827.293 

=== epoch 4/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  165
episode_length                       59.90303
returns                            298.934024
return_std                         119.441307
average_reward                       4.986516
round_time             0 days 00:06:01.071640
episodes_test                           149.0
episode_length_test                 66.879195
returns_test                       341.792436
return_std_test                    100.575456
average_reward_test                  5.110892
round_time_test        0 days 00:00:14.513843
round_time_total       0 days 00:06:01.072776
loss_total                      245861.383977
loss_critic                      307608.03009
loss_actor                       -1125.220515
memory_size                        180219.678 

=== epoch 4/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  167
episode_length                      59.568862
returns                            297.227354
return_std                         113.303262
average_reward                       4.988359
round_time             0 days 00:05:58.979711
episodes_test                           160.0
episode_length_test                    62.275
returns_test                       320.778643
return_std_test                     96.685918
average_reward_test                  5.149328
round_time_test        0 days 00:00:14.757138
round_time_total       0 days 00:05:58.981091
loss_total                      541734.171906
loss_critic                     677452.504266
loss_actor                        -1139.22137
memory_size                        181682.155 

=== epoch 4/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  171
episode_length                      58.076023
returns                            292.193951
return_std                         108.647579
average_reward                       5.028117
round_time             0 days 00:05:58.713603
episodes_test                           153.0
episode_length_test                 65.019608
returns_test                       342.261164
return_std_test                     77.827219
average_reward_test                  5.261282
round_time_test        0 days 00:00:15.125708
round_time_total       0 days 00:05:58.714693
loss_total                      651736.187817
loss_critic                     814955.039016
loss_actor                       -1139.257931
memory_size                       183084.5455 

=== epoch 4/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  171
episode_length                      57.871345
returns                            294.642567
return_std                          85.895297
average_reward                       5.092316
round_time             0 days 00:06:02.312988
episodes_test                           147.0
episode_length_test                 67.843537
returns_test                       354.912036
return_std_test                    101.909716
average_reward_test                  5.230974
round_time_test        0 days 00:00:14.880977
round_time_total       0 days 00:06:02.314119
loss_total                     1155241.781072
loss_critic                    1444326.826141
loss_actor                       -1098.294712
memory_size                        184512.538 

=== epoch 4/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  164
episode_length                      60.457317
returns                              309.8835
return_std                          83.057411
average_reward                       5.123275
round_time             0 days 00:05:59.778462
episodes_test                           160.0
episode_length_test                     62.15
returns_test                       323.909985
return_std_test                     97.002446
average_reward_test                  5.210611
round_time_test        0 days 00:00:14.879772
round_time_total       0 days 00:05:59.779556
loss_total                      126047.156012
loss_critic                      157829.73373
loss_actor                       -1083.165311
memory_size                        185980.132 

=== epoch 4/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:31,  4.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  168
episode_length                      59.452381
returns                            306.982987
return_std                          81.705915
average_reward                       5.163628
round_time             0 days 00:05:59.349075
episodes_test                           172.0
episode_length_test                 57.924419
returns_test                        302.26764
return_std_test                     95.293212
average_reward_test                  5.217545
round_time_test        0 days 00:00:15.100099
round_time_total       0 days 00:05:59.350335
loss_total                     1825736.114334
loss_critic                    2282441.887066
loss_actor                       -1087.387371
memory_size                        187391.753 

=== epoch 4/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:19,  4.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  167
episode_length                      59.694611
returns                            310.270433
return_std                          81.070334
average_reward                        5.19618
round_time             0 days 00:05:58.305451
episodes_test                           164.0
episode_length_test                 60.786585
returns_test                       321.106553
return_std_test                     98.622105
average_reward_test                  5.281172
round_time_test        0 days 00:00:15.075498
round_time_total       0 days 00:05:58.306759
loss_total                     1328314.189076
loss_critic                    1660652.690262
loss_actor                       -1040.255832
memory_size                        188816.934 

=== epoch 4/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  161
episode_length                      61.515528
returns                            319.462624
return_std                          87.866908
average_reward                       5.192904
round_time             0 days 00:06:02.209742
episodes_test                           162.0
episode_length_test                 61.345679
returns_test                       321.994884
return_std_test                     92.799089
average_reward_test                  5.248364
round_time_test        0 days 00:00:15.236769
round_time_total       0 days 00:06:02.211064
loss_total                      469833.933304
loss_critic                     587544.281111
loss_actor                       -1007.481907
memory_size                       190280.2755 

=== epoch 4/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  159
episode_length                      62.559748
returns                            324.360941
return_std                          95.659669
average_reward                       5.184995
round_time             0 days 00:05:58.691417
episodes_test                           163.0
episode_length_test                 61.196319
returns_test                       319.566009
return_std_test                    102.306625
average_reward_test                  5.221645
round_time_test        0 days 00:00:14.751228
round_time_total       0 days 00:05:58.692876
loss_total                      282762.350592
loss_critic                     353703.423357
loss_actor                        -1001.93362
memory_size                       191757.7475 

=== epoch 4/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  159
episode_length                       62.72956
returns                            323.979301
return_std                          95.496516
average_reward                       5.164301
round_time             0 days 00:05:57.508468
episodes_test                           163.0
episode_length_test                 61.134969
returns_test                       323.698533
return_std_test                     86.171715
average_reward_test                  5.294295
round_time_test        0 days 00:00:14.830116
round_time_total       0 days 00:05:57.509620
loss_total                      373268.259162
loss_critic                      466835.87043
loss_actor                       -1002.148782
memory_size                        193273.781 

=== epoch 4/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:06<00:00,  5.45it/s]
episodes                                  152
episode_length                      64.881579
returns                            333.234351
return_std                          98.965699
average_reward                       5.138648
round_time             0 days 00:06:07.213772
episodes_test                           145.0
episode_length_test                 68.841379
returns_test                       357.263466
return_std_test                    102.596369
average_reward_test                  5.188799
round_time_test        0 days 00:00:14.666008
round_time_total       0 days 00:06:07.214946
loss_total                      902664.424194
loss_critic                    1128580.331312
loss_actor                        -999.214569
memory_size                        194731.565 

=== epoch 4/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  152
episode_length                      64.888158
returns                            332.652179
return_std                          99.804466
average_reward                       5.123879
round_time             0 days 00:05:56.065213
episodes_test                           151.0
episode_length_test                 66.013245
returns_test                       346.004246
return_std_test                     83.016116
average_reward_test                  5.241228
round_time_test        0 days 00:00:14.587035
round_time_total       0 days 00:05:56.066568
loss_total                       92207.944354
loss_critic                     115501.744807
loss_actor                        -967.265484
memory_size                       196203.1795 

=== epoch 4/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  156
episode_length                      63.564103
returns                            326.689059
return_std                           89.98151
average_reward                       5.141505
round_time             0 days 00:06:02.136724
episodes_test                           163.0
episode_length_test                 60.754601
returns_test                       319.544427
return_std_test                     87.946673
average_reward_test                  5.259285
round_time_test        0 days 00:00:14.729042
round_time_total       0 days 00:06:02.137831
loss_total                      193991.648384
loss_critic                     242730.470246
loss_actor                        -963.649631
memory_size                       197648.9625 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 5/10 ===== round 1/50 ======================================
  0%|          | 10/2000 [00:02<07:58,  4.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                   32
episode_length                           61.0
returns                             321.66409
return_std                          81.642654
average_reward                        5.27351
round_time             0 days 00:05:42.764301
episodes_test                           162.0
episode_length_test                 61.469136
returns_test                       318.070389
return_std_test                      88.26543
average_reward_test                  5.174287
round_time_test        0 days 00:00:14.414195
round_time_total       0 days 00:05:42.765786
loss_total                     2029931.383788
loss_critic                    2537651.935102
loss_actor                        -951.298332
memory_size                       199069.9935 

=== epoch 5/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   68
episode_length                      58.147059
returns                            303.672307
return_std                          79.514894
average_reward                        5.22123
round_time             0 days 00:05:45.994277
episodes_test                           155.0
episode_length_test                 64.329032
returns_test                       335.468762
return_std_test                      73.59938
average_reward_test                  5.214674
round_time_test        0 days 00:00:14.146270
round_time_total       0 days 00:05:45.995772
loss_total                     2188066.922416
loss_critic                    2735307.105006
loss_actor                        -894.061001
memory_size                       200476.1325 

=== epoch 5/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  102
episode_length                      58.205882
returns                            302.071645
return_std                           79.08407
average_reward                       5.190412
round_time             0 days 00:05:47.170356
episodes_test                           173.0
episode_length_test                 57.554913
returns_test                       303.371646
return_std_test                     59.981062
average_reward_test                   5.27122
round_time_test        0 days 00:00:14.665013
round_time_total       0 days 00:05:47.171601
loss_total                      2884824.16006
loss_critic                     3606254.02744
loss_actor                        -896.175456
memory_size                       201871.5955 

=== epoch 5/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  136
episode_length                      58.632353
returns                            305.532075
return_std                          72.902528
average_reward                       5.210371
round_time             0 days 00:05:52.955126
episodes_test                           167.0
episode_length_test                 59.796407
returns_test                       318.385515
return_std_test                     70.509576
average_reward_test                  5.324032
round_time_test        0 days 00:00:14.424621
round_time_total       0 days 00:05:52.956232
loss_total                      620449.745628
loss_critic                     775784.216156
loss_actor                        -888.160014
memory_size                       203318.0245 

=== epoch 5/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  170
episode_length                      58.588235
returns                            305.056811
return_std                           71.52137
average_reward                        5.20675
round_time             0 days 00:05:51.735587
episodes_test                           163.0
episode_length_test                 61.337423
returns_test                       318.781528
return_std_test                     76.534775
average_reward_test                  5.197119
round_time_test        0 days 00:00:14.146592
round_time_total       0 days 00:05:51.736699
loss_total                       75493.024514
loss_critic                      94590.161464
loss_actor                        -895.529763
memory_size                        204725.685 

=== epoch 5/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:41,  4.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  172
episode_length                      57.930233
returns                            301.107259
return_std                          65.006381
average_reward                       5.198375
round_time             0 days 00:05:53.333328
episodes_test                           153.0
episode_length_test                  65.24183
returns_test                       343.237394
return_std_test                    101.829001
average_reward_test                  5.260556
round_time_test        0 days 00:00:14.554818
round_time_total       0 days 00:05:53.334433
loss_total                      149156.405024
loss_critic                     186676.413296
loss_actor                        -923.678272
memory_size                       206157.1625 

=== epoch 5/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  170
episode_length                      58.752941
returns                            305.450892
return_std                          65.038469
average_reward                       5.199135
round_time             0 days 00:05:49.066279
episodes_test                           173.0
episode_length_test                 57.780347
returns_test                        306.44085
return_std_test                     72.935983
average_reward_test                  5.303384
round_time_test        0 days 00:00:14.775794
round_time_total       0 days 00:05:49.067364
loss_total                      2892226.55188
loss_critic                    3615516.060617
loss_actor                        -932.495669
memory_size                       207564.5155 

=== epoch 5/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:17,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  167
episode_length                      59.562874
returns                            310.066022
return_std                          74.038605
average_reward                       5.205806
round_time             0 days 00:05:53.309883
episodes_test                           154.0
episode_length_test                 64.493506
returns_test                       344.221071
return_std_test                     93.946669
average_reward_test                  5.335358
round_time_test        0 days 00:00:14.363263
round_time_total       0 days 00:05:53.311105
loss_total                      392121.168154
loss_critic                     490380.160162
loss_actor                        -914.888262
memory_size                        209028.534 

=== epoch 5/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:34,  4.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  164
episode_length                       60.47561
returns                            313.731437
return_std                          79.660634
average_reward                        5.18996
round_time             0 days 00:05:54.905207
episodes_test                           175.0
episode_length_test                      56.8
returns_test                       299.062685
return_std_test                     74.479744
average_reward_test                  5.265666
round_time_test        0 days 00:00:14.494585
round_time_total       0 days 00:05:54.906306
loss_total                       74958.327837
loss_critic                      93929.347451
loss_actor                        -925.756379
memory_size                       210476.4575 

=== epoch 5/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  164
episode_length                      60.628049
returns                            315.193048
return_std                          79.018118
average_reward                        5.19923
round_time             0 days 00:05:52.911106
episodes_test                           157.0
episode_length_test                 63.280255
returns_test                       337.679908
return_std_test                     81.931262
average_reward_test                  5.335882
round_time_test        0 days 00:00:14.353552
round_time_total       0 days 00:05:52.912516
loss_total                     2926201.951747
loss_critic                    3657982.323719
loss_actor                        -918.958976
memory_size                        211939.096 

=== epoch 5/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:35,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  162
episode_length                      61.333333
returns                            317.421277
return_std                           82.16265
average_reward                       5.175265
round_time             0 days 00:05:53.263665
episodes_test                           159.0
episode_length_test                 62.622642
returns_test                       331.045743
return_std_test                     83.295536
average_reward_test                  5.286145
round_time_test        0 days 00:00:14.483508
round_time_total       0 days 00:05:53.264767
loss_total                     3461584.547391
loss_critic                    4327197.668279
loss_actor                        -869.071086
memory_size                       213360.7645 

=== epoch 5/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  160
episode_length                          61.95
returns                            319.492836
return_std                          90.891551
average_reward                       5.158729
round_time             0 days 00:05:50.470119
episodes_test                           146.0
episode_length_test                 68.315068
returns_test                       353.592632
return_std_test                     95.851108
average_reward_test                  5.175311
round_time_test        0 days 00:00:14.386199
round_time_total       0 days 00:05:50.471547
loss_total                       68392.186868
loss_critic                      85710.916186
loss_actor                        -882.736368
memory_size                        214815.209 

=== epoch 5/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  161
episode_length                      61.776398
returns                            318.441035
return_std                          83.344173
average_reward                       5.156864
round_time             0 days 00:05:54.732608
episodes_test                           148.0
episode_length_test                 67.081081
returns_test                       350.369501
return_std_test                     87.025299
average_reward_test                  5.221941
round_time_test        0 days 00:00:14.393639
round_time_total       0 days 00:05:54.734165
loss_total                       70199.111871
loss_critic                      87973.663416
loss_actor                        -899.100371
memory_size                        216258.187 

=== epoch 5/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  165
episode_length                      60.509091
returns                            311.274212
return_std                          80.984726
average_reward                       5.144518
round_time             0 days 00:05:54.348314
episodes_test                           145.0
episode_length_test                 68.627586
returns_test                        356.70457
return_std_test                    102.952224
average_reward_test                  5.195757
round_time_test        0 days 00:00:14.635437
round_time_total       0 days 00:05:54.349497
loss_total                     1521210.510867
loss_critic                    1901733.066908
loss_actor                        -880.045933
memory_size                       217706.8495 

=== epoch 5/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  162
episode_length                      61.209877
returns                            313.446911
return_std                          85.220113
average_reward                       5.122466
round_time             0 days 00:05:51.097384
episodes_test                           146.0
episode_length_test                 68.006849
returns_test                       354.919859
return_std_test                     86.710555
average_reward_test                  5.218663
round_time_test        0 days 00:00:14.367867
round_time_total       0 days 00:05:51.098543
loss_total                     4948032.991325
loss_critic                    6185261.076928
loss_actor                        -880.329645
memory_size                        219136.206 

=== epoch 5/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  162
episode_length                      61.462963
returns                            313.975818
return_std                          88.888832
average_reward                       5.109502
round_time             0 days 00:05:54.989982
episodes_test                           158.0
episode_length_test                 63.031646
returns_test                       327.645625
return_std_test                     89.318086
average_reward_test                  5.198698
round_time_test        0 days 00:00:13.783902
round_time_total       0 days 00:05:54.991212
loss_total                     3336374.540025
loss_critic                    4170688.445212
loss_actor                        -882.313467
memory_size                       220585.8375 

=== epoch 5/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  163
episode_length                      60.944785
returns                              312.0621
return_std                          79.396682
average_reward                       5.121751
round_time             0 days 00:05:54.627525
episodes_test                           145.0
episode_length_test                 68.641379
returns_test                       353.944891
return_std_test                    106.936598
average_reward_test                  5.157651
round_time_test        0 days 00:00:14.724973
round_time_total       0 days 00:05:54.628638
loss_total                     1279549.652613
loss_critic                    1599658.408941
loss_actor                        -885.091487
memory_size                       222023.5965 

=== epoch 5/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  166
episode_length                      60.096386
returns                             307.10721
return_std                          87.171756
average_reward                        5.11179
round_time             0 days 00:05:53.682766
episodes_test                           150.0
episode_length_test                 66.586667
returns_test                       339.569816
return_std_test                     97.201478
average_reward_test                  5.099506
round_time_test        0 days 00:00:14.653607
round_time_total       0 days 00:05:53.683911
loss_total                     4677492.233284
loss_critic                    5847084.563342
loss_actor                        -877.867148
memory_size                        223462.128 

=== epoch 5/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  169
episode_length                      58.597633
returns                            298.330624
return_std                          91.372266
average_reward                       5.092052
round_time             0 days 00:05:55.770750
episodes_test                           174.0
episode_length_test                 57.275862
returns_test                       292.857165
return_std_test                     86.156392
average_reward_test                  5.112977
round_time_test        0 days 00:00:14.372706
round_time_total       0 days 00:05:55.771899
loss_total                     3082797.695662
loss_critic                    3853719.014517
loss_actor                        -887.399919
memory_size                        224840.338 

=== epoch 5/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  178
episode_length                      55.719101
returns                            283.545913
return_std                          89.766426
average_reward                       5.091916
round_time             0 days 00:05:54.231405
episodes_test                           208.0
episode_length_test                 48.033654
returns_test                       242.278727
return_std_test                     96.234667
average_reward_test                  5.043808
round_time_test        0 days 00:00:13.903628
round_time_total       0 days 00:05:54.232781
loss_total                      1584659.70045
loss_critic                    1981048.107158
loss_actor                        -893.995652
memory_size                       226181.8845 

=== epoch 5/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:46,  4.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  181
episode_length                      54.928177
returns                            280.059607
return_std                          97.985685
average_reward                       5.101007
round_time             0 days 00:05:56.387262
episodes_test                           198.0
episode_length_test                 50.449495
returns_test                       256.140439
return_std_test                     96.983351
average_reward_test                  5.077017
round_time_test        0 days 00:00:13.915191
round_time_total       0 days 00:05:56.388367
loss_total                     4107109.034341
loss_critic                     5134108.65109
loss_actor                        -890.531118
memory_size                          227540.5 

=== epoch 5/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:26,  4.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                  191
episode_length                      52.136126
returns                            265.458513
return_std                         101.815965
average_reward                       5.090482
round_time             0 days 00:05:55.534137
episodes_test                           202.0
episode_length_test                 49.356436
returns_test                        248.49894
return_std_test                     97.311289
average_reward_test                  5.034333
round_time_test        0 days 00:00:13.715819
round_time_total       0 days 00:05:55.535251
loss_total                     2196233.387205
loss_critic                    2745514.244151
loss_actor                        -889.630422
memory_size                        228848.413 

=== epoch 5/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  196
episode_length                      50.795918
returns                            259.211798
return_std                          94.889158
average_reward                       5.102924
round_time             0 days 00:05:54.849774
episodes_test                           166.0
episode_length_test                 59.963855
returns_test                       306.176118
return_std_test                     108.75801
average_reward_test                  5.104812
round_time_test        0 days 00:00:14.245331
round_time_total       0 days 00:05:54.851217
loss_total                     3063066.297504
loss_critic                    3829057.635997
loss_actor                        -899.533251
memory_size                       230163.4405 

=== epoch 5/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  201
episode_length                      49.507463
returns                            252.404993
return_std                          97.327744
average_reward                       5.097884
round_time             0 days 00:05:55.387890
episodes_test                           203.0
episode_length_test                 49.182266
returns_test                       246.293051
return_std_test                     85.945539
average_reward_test                  5.007644
round_time_test        0 days 00:00:13.677394
round_time_total       0 days 00:05:55.389000
loss_total                     1888760.235209
loss_critic                    2361173.634977
loss_actor                        -893.666611
memory_size                        231458.358 

=== epoch 5/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  202
episode_length                      49.480198
returns                            251.654186
return_std                          97.606203
average_reward                       5.086085
round_time             0 days 00:05:55.918339
episodes_test                           198.0
episode_length_test                 50.388889
returns_test                       250.184893
return_std_test                     98.645527
average_reward_test                  4.965098
round_time_test        0 days 00:00:13.564742
round_time_total       0 days 00:05:55.919841
loss_total                     1885790.411322
loss_critic                     2357455.97696
loss_actor                        -871.629537
memory_size                        232745.196 

=== epoch 5/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  205
episode_length                      48.643902
returns                            246.920567
return_std                          92.784477
average_reward                       5.076644
round_time             0 days 00:05:51.997363
episodes_test                           183.0
episode_length_test                 54.382514
returns_test                       275.332553
return_std_test                    106.273343
average_reward_test                  5.062589
round_time_test        0 days 00:00:13.745518
round_time_total       0 days 00:05:51.998465
loss_total                     3742991.012189
loss_critic                    4678955.903379
loss_actor                        -868.998344
memory_size                       234070.8645 

=== epoch 5/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:40,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  204
episode_length                      48.892157
returns                            247.363751
return_std                          91.965599
average_reward                       5.058637
round_time             0 days 00:05:53.119729
episodes_test                           221.0
episode_length_test                 45.230769
returns_test                       230.615956
return_std_test                    101.524519
average_reward_test                  5.098581
round_time_test        0 days 00:00:13.833045
round_time_total       0 days 00:05:53.121180
loss_total                        66790.22215
loss_critic                      83710.770034
loss_actor                        -891.975054
memory_size                       235361.5545 

=== epoch 5/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  201
episode_length                      49.651741
returns                            250.558617
return_std                          94.692823
average_reward                       5.045973
round_time             0 days 00:05:54.010965
episodes_test                           168.0
episode_length_test                 59.285714
returns_test                       301.526517
return_std_test                    100.609738
average_reward_test                  5.086809
round_time_test        0 days 00:00:14.373508
round_time_total       0 days 00:05:54.012073
loss_total                      980695.773939
loss_critic                    1226090.179673
loss_actor                        -881.847371
memory_size                        236691.464 

=== epoch 5/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  194
episode_length                      51.190722
returns                            258.565641
return_std                          93.576584
average_reward                       5.050323
round_time             0 days 00:05:52.098610
episodes_test                           149.0
episode_length_test                 66.865772
returns_test                       335.468706
return_std_test                    126.725123
average_reward_test                   5.01655
round_time_test        0 days 00:00:14.311307
round_time_total       0 days 00:05:52.099707
loss_total                     3973654.968704
loss_critic                    4967286.127955
loss_actor                        -868.932851
memory_size                         238050.58 

=== epoch 5/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  186
episode_length                      52.892473
returns                            267.032017
return_std                          97.217161
average_reward                       5.043826
round_time             0 days 00:05:52.460581
episodes_test                           146.0
episode_length_test                 68.219178
returns_test                       348.353596
return_std_test                    114.592056
average_reward_test                  5.104797
round_time_test        0 days 00:00:14.692259
round_time_total       0 days 00:05:52.461860
loss_total                        63687.92048
loss_critic                      79825.179412
loss_actor                        -861.120435
memory_size                         239479.49 

=== epoch 5/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.67it/s]
episodes                                  177
episode_length                      56.101695
returns                            282.292034
return_std                          99.487634
average_reward                       5.030995
round_time             0 days 00:05:53.547827
episodes_test                           136.0
episode_length_test                 73.323529
returns_test                       368.806526
return_std_test                    108.483775
average_reward_test                  5.030217
round_time_test        0 days 00:00:14.886337
round_time_total       0 days 00:05:53.548939
loss_total                      901480.750754
loss_critic                    1127066.301336
loss_actor                        -861.444835
memory_size                       240936.6375 

=== epoch 5/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  167
episode_length                      59.299401
returns                            298.702078
return_std                          97.405315
average_reward                       5.039308
round_time             0 days 00:05:54.241600
episodes_test                           133.0
episode_length_test                 75.135338
returns_test                        380.44842
return_std_test                    109.353289
average_reward_test                  5.063405
round_time_test        0 days 00:00:14.800840
round_time_total       0 days 00:05:54.242691
loss_total                     2229585.204437
loss_critic                    2787188.823257
loss_actor                        -829.584279
memory_size                       242399.6585 

=== epoch 5/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  161
episode_length                       61.57764
returns                            308.950822
return_std                          97.017098
average_reward                       5.020907
round_time             0 days 00:05:50.266444
episodes_test                           127.0
episode_length_test                 78.338583
returns_test                       392.726202
return_std_test                    119.561217
average_reward_test                  5.012074
round_time_test        0 days 00:00:15.153218
round_time_total       0 days 00:05:50.267551
loss_total                      469330.392783
loss_critic                     586876.490446
loss_actor                        -854.011292
memory_size                        243872.174 

=== epoch 5/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  154
episode_length                      64.324675
returns                            321.799774
return_std                            93.5946
average_reward                       5.005133
round_time             0 days 00:05:52.579784
episodes_test                           130.0
episode_length_test                      76.7
returns_test                       379.310316
return_std_test                    111.936198
average_reward_test                  4.946157
round_time_test        0 days 00:00:15.551659
round_time_total       0 days 00:05:52.581008
loss_total                        63564.82785
loss_critic                      79671.567126
loss_actor                        -862.135043
memory_size                        245384.767 

=== epoch 5/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:34,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  152
episode_length                      65.105263
returns                            326.666031
return_std                          82.604252
average_reward                       5.019577
round_time             0 days 00:05:52.766848
episodes_test                           145.0
episode_length_test                 68.944828
returns_test                        350.00747
return_std_test                     77.799733
average_reward_test                  5.076578
round_time_test        0 days 00:00:15.108780
round_time_total       0 days 00:05:52.767983
loss_total                       64897.342769
loss_critic                      81338.606598
loss_actor                        -867.718049
memory_size                        246861.592 

=== epoch 5/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  153
episode_length                      64.986928
returns                            327.978975
return_std                          81.725865
average_reward                       5.048745
round_time             0 days 00:05:55.750012
episodes_test                           147.0
episode_length_test                 67.632653
returns_test                       351.650082
return_std_test                      71.02944
average_reward_test                  5.199858
round_time_test        0 days 00:00:15.486079
round_time_total       0 days 00:05:55.751136
loss_total                       64428.539716
loss_critic                       80752.16255
loss_actor                        -865.957419
memory_size                        248337.907 

=== epoch 5/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                  152
episode_length                      65.414474
returns                            330.919342
return_std                            73.3051
average_reward                       5.059187
round_time             0 days 00:05:55.548767
episodes_test                           157.0
episode_length_test                 63.267516
returns_test                       330.918073
return_std_test                     57.586004
average_reward_test                  5.228113
round_time_test        0 days 00:00:15.588968
round_time_total       0 days 00:05:55.549868
loss_total                       64259.992274
loss_critic                      80541.528472
loss_actor                        -866.157328
memory_size                       249805.9515 

=== epoch 5/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  152
episode_length                           65.0
returns                             331.50293
return_std                          65.771988
average_reward                       5.096531
round_time             0 days 00:05:54.571262
episodes_test                           159.0
episode_length_test                 62.735849
returns_test                       327.147238
return_std_test                     42.070894
average_reward_test                  5.214109
round_time_test        0 days 00:00:14.998708
round_time_total       0 days 00:05:54.572394
loss_total                      631176.350916
loss_critic                     789186.000218
loss_actor                        -862.122619
memory_size                       251280.5835 

=== epoch 5/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  154
episode_length                      64.746753
returns                            332.683572
return_std                          72.050812
average_reward                       5.138271
round_time             0 days 00:05:56.328241
episodes_test                           141.0
episode_length_test                 70.503546
returns_test                       364.044443
return_std_test                     78.959551
average_reward_test                  5.164721
round_time_test        0 days 00:00:15.017996
round_time_total       0 days 00:05:56.329376
loss_total                       63569.853041
loss_critic                      79676.729549
loss_actor                        -857.658599
memory_size                        252758.062 

=== epoch 5/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:44,  4.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  152
episode_length                      65.467105
returns                            336.795875
return_std                          73.220691
average_reward                       5.144942
round_time             0 days 00:05:51.548618
episodes_test                           160.0
episode_length_test                      62.4
returns_test                       326.543958
return_std_test                     46.008607
average_reward_test                  5.232601
round_time_test        0 days 00:00:14.957421
round_time_total       0 days 00:05:51.549703
loss_total                        63962.01204
loss_critic                      80170.113726
loss_actor                        -870.400106
memory_size                       254249.7995 

=== epoch 5/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  153
episode_length                      65.137255
returns                            333.310393
return_std                          81.125623
average_reward                       5.118301
round_time             0 days 00:05:55.101962
episodes_test                           173.0
episode_length_test                 57.549133
returns_test                       296.800389
return_std_test                      41.71744
average_reward_test                  5.156337
round_time_test        0 days 00:00:14.929402
round_time_total       0 days 00:05:55.103072
loss_total                      835294.198013
loss_critic                    1044339.917938
loss_actor                        -888.749283
memory_size                         255741.39 

=== epoch 5/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  155
episode_length                      64.303226
returns                            328.712348
return_std                          81.867509
average_reward                        5.11085
round_time             0 days 00:05:55.274178
episodes_test                           203.0
episode_length_test                 49.049261
returns_test                       249.916455
return_std_test                     38.731028
average_reward_test                  5.095574
round_time_test        0 days 00:00:14.461086
round_time_total       0 days 00:05:55.275290
loss_total                     2852785.890845
loss_critic                    3566201.987875
loss_actor                        -877.978297
memory_size                        257185.733 

=== epoch 5/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  157
episode_length                      63.101911
returns                            320.126595
return_std                          82.284863
average_reward                       5.074641
round_time             0 days 00:05:52.944301
episodes_test                           160.0
episode_length_test                  62.14375
returns_test                       323.729056
return_std_test                     52.225759
average_reward_test                  5.209092
round_time_test        0 days 00:00:15.468765
round_time_total       0 days 00:05:52.945570
loss_total                      829671.138287
loss_critic                    1037302.495452
loss_actor                        -854.351832
memory_size                       258622.1955 

=== epoch 5/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  161
episode_length                      61.583851
returns                            311.142482
return_std                          82.586296
average_reward                       5.052325
round_time             0 days 00:05:58.800606
episodes_test                           173.0
episode_length_test                 57.653179
returns_test                       294.470956
return_std_test                     59.901227
average_reward_test                  5.107255
round_time_test        0 days 00:00:15.134316
round_time_total       0 days 00:05:58.802011
loss_total                     1064644.547813
loss_critic                    1331015.481718
loss_actor                        -839.163413
memory_size                       260029.9905 

=== epoch 5/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:29,  4.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  165
episode_length                           60.2
returns                            303.167175
return_std                           78.18989
average_reward                       5.035493
round_time             0 days 00:05:58.555082
episodes_test                           160.0
episode_length_test                   62.4375
returns_test                       314.789664
return_std_test                     77.127871
average_reward_test                  5.041525
round_time_test        0 days 00:00:15.006979
round_time_total       0 days 00:05:58.556366
loss_total                       64437.204335
loss_critic                      80753.930146
loss_actor                        -829.703918
memory_size                       261455.3735 

=== epoch 5/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  168
episode_length                      59.119048
returns                            297.936507
return_std                          69.137166
average_reward                       5.041154
round_time             0 days 00:05:59.199647
episodes_test                           153.0
episode_length_test                 65.045752
returns_test                       333.787655
return_std_test                     60.174024
average_reward_test                   5.13048
round_time_test        0 days 00:00:15.328277
round_time_total       0 days 00:05:59.201104
loss_total                        62506.92535
loss_critic                      78347.465515
loss_actor                        -855.240621
memory_size                       262878.3335 

=== epoch 5/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  169
episode_length                      58.846154
returns                            296.723536
return_std                          76.218096
average_reward                       5.043677
round_time             0 days 00:05:57.817582
episodes_test                           162.0
episode_length_test                 61.530864
returns_test                       315.290731
return_std_test                     54.965597
average_reward_test                  5.123854
round_time_test        0 days 00:00:15.185781
round_time_total       0 days 00:05:57.818700
loss_total                       65621.349852
loss_critic                       82246.68571
loss_actor                        -879.999336
memory_size                       264301.1415 

=== epoch 5/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  169
episode_length                      58.763314
returns                            295.633132
return_std                          75.551909
average_reward                       5.032123
round_time             0 days 00:05:57.502402
episodes_test                           158.0
episode_length_test                      63.0
returns_test                       322.332477
return_std_test                     48.586663
average_reward_test                  5.115621
round_time_test        0 days 00:00:15.262136
round_time_total       0 days 00:05:57.503523
loss_total                      834853.230961
loss_critic                    1043783.199758
loss_actor                        -866.631177
memory_size                       265730.3415 

=== epoch 5/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  167
episode_length                       59.57485
returns                            300.301409
return_std                          73.558069
average_reward                       5.038545
round_time             0 days 00:05:57.932276
episodes_test                           172.0
episode_length_test                 58.052326
returns_test                       298.001898
return_std_test                     50.005281
average_reward_test                  5.132983
round_time_test        0 days 00:00:15.191166
round_time_total       0 days 00:05:57.933433
loss_total                      1936867.80608
loss_critic                    2421295.268679
loss_actor                        -842.100958
memory_size                        267159.175 

=== epoch 5/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  170
episode_length                      58.523529
returns                            295.083292
return_std                          73.214956
average_reward                       5.041707
round_time             0 days 00:05:57.865653
episodes_test                           162.0
episode_length_test                 61.512346
returns_test                       315.939324
return_std_test                     62.509868
average_reward_test                  5.135427
round_time_test        0 days 00:00:15.002935
round_time_total       0 days 00:05:57.867054
loss_total                       61039.268089
loss_critic                      76515.137752
loss_actor                        -864.215803
memory_size                        268566.113 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 6/10 ===== round 1/50 ======================================
  0%|          | 10/2000 [00:02<07:09,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                   33
episode_length                       59.30303
returns                            298.155887
return_std                          77.184062
average_reward                       5.028373
round_time             0 days 00:05:47.277500
episodes_test                           176.0
episode_length_test                    56.625
returns_test                       289.551549
return_std_test                     38.803745
average_reward_test                  5.113631
round_time_test        0 days 00:00:15.001981
round_time_total       0 days 00:05:47.278604
loss_total                     4259499.337227
loss_critic                    5324584.309893
loss_actor                        -841.305745
memory_size                       269982.5965 

=== epoch 6/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                   68
episode_length                      58.367647
returns                            295.334797
return_std                          65.238894
average_reward                        5.05907
round_time             0 days 00:05:47.597841
episodes_test                           177.0
episode_length_test                 56.497175
returns_test                       290.656326
return_std_test                     49.588362
average_reward_test                  5.144617
round_time_test        0 days 00:00:14.777892
round_time_total       0 days 00:05:47.598971
loss_total                      250407.954072
loss_critic                     313218.296354
loss_actor                        -833.434444
memory_size                        271390.461 

=== epoch 6/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                  105
episode_length                      56.685714
returns                            285.205176
return_std                          58.532551
average_reward                       5.032493
round_time             0 days 00:05:49.361494
episodes_test                           178.0
episode_length_test                 55.921348
returns_test                       285.951125
return_std_test                     46.489992
average_reward_test                  5.113241
round_time_test        0 days 00:00:14.989462
round_time_total       0 days 00:05:49.362577
loss_total                     1112830.820497
loss_critic                    1391247.571363
loss_actor                        -836.363006
memory_size                        272782.541 

=== epoch 6/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  140
episode_length                      56.964286
returns                            287.566053
return_std                          56.874343
average_reward                       5.047705
round_time             0 days 00:05:50.603521
episodes_test                           185.0
episode_length_test                 53.745946
returns_test                       275.486078
return_std_test                     37.773429
average_reward_test                  5.124609
round_time_test        0 days 00:00:14.948716
round_time_total       0 days 00:05:50.604857
loss_total                      604435.818261
loss_critic                     755755.453314
loss_actor                        -842.808387
memory_size                       274176.5335 

=== epoch 6/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  175
episode_length                      56.834286
returns                            288.842214
return_std                          54.217112
average_reward                       5.082771
round_time             0 days 00:05:50.209995
episodes_test                           169.0
episode_length_test                 58.988166
returns_test                       302.906612
return_std_test                     57.391602
average_reward_test                   5.13437
round_time_test        0 days 00:00:14.741829
round_time_total       0 days 00:05:50.211113
loss_total                      210192.038702
loss_critic                     262947.661774
loss_actor                        -830.444123
memory_size                        275566.106 

=== epoch 6/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:35,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  174
episode_length                      57.005747
returns                              289.8434
return_std                          50.329287
average_reward                       5.085771
round_time             0 days 00:05:45.924525
episodes_test                           173.0
episode_length_test                 57.554913
returns_test                       298.810638
return_std_test                     38.734507
average_reward_test                  5.190743
round_time_test        0 days 00:00:15.353499
round_time_total       0 days 00:05:45.925611
loss_total                       57282.845521
loss_critic                      71814.105502
loss_actor                         -842.19896
memory_size                        276999.445 

=== epoch 6/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:18,  4.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  173
episode_length                      57.647399
returns                            294.063574
return_std                           54.58284
average_reward                       5.101524
round_time             0 days 00:05:50.653309
episodes_test                           168.0
episode_length_test                 59.244048
returns_test                       305.419975
return_std_test                     39.506276
average_reward_test                  5.155487
round_time_test        0 days 00:00:15.103281
round_time_total       0 days 00:05:50.654801
loss_total                      707738.375792
loss_critic                     884881.688902
loss_actor                        -835.157234
memory_size                       278427.6755 

=== epoch 6/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  165
episode_length                      60.254545
returns                            308.662738
return_std                          80.651084
average_reward                       5.123705
round_time             0 days 00:05:48.309085
episodes_test                           157.0
episode_length_test                 63.433121
returns_test                       328.019971
return_std_test                     55.447961
average_reward_test                  5.170562
round_time_test        0 days 00:00:15.006915
round_time_total       0 days 00:05:48.310586
loss_total                      188931.921705
loss_critic                     236376.879075
loss_actor                        -847.909405
memory_size                        279879.927 

=== epoch 6/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  165
episode_length                      60.460606
returns                            309.658943
return_std                          82.350557
average_reward                       5.122596
round_time             0 days 00:05:47.917993
episodes_test                           158.0
episode_length_test                 63.164557
returns_test                       327.794787
return_std_test                     50.312855
average_reward_test                  5.189126
round_time_test        0 days 00:00:15.095907
round_time_total       0 days 00:05:47.919423
loss_total                     2951509.789768
loss_critic                    3689593.511205
loss_actor                        -825.529291
memory_size                        281346.559 

=== epoch 6/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  163
episode_length                      60.883436
returns                            310.816855
return_std                           87.60348
average_reward                       5.104152
round_time             0 days 00:05:52.132264
episodes_test                           171.0
episode_length_test                 58.140351
returns_test                       302.088801
return_std_test                     37.636362
average_reward_test                  5.195271
round_time_test        0 days 00:00:14.960571
round_time_total       0 days 00:05:52.133767
loss_total                      237998.857521
loss_critic                     297702.027195
loss_actor                        -813.857386
memory_size                       282779.1275 

=== epoch 6/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  162
episode_length                      61.592593
returns                            315.266464
return_std                          90.375195
average_reward                       5.119565
round_time             0 days 00:05:53.797544
episodes_test                           162.0
episode_length_test                 61.425926
returns_test                       319.164477
return_std_test                     50.737958
average_reward_test                  5.195586
round_time_test        0 days 00:00:15.233173
round_time_total       0 days 00:05:53.798869
loss_total                      163439.499405
loss_critic                       204506.2472
loss_actor                        -827.484619
memory_size                       284225.2725 

=== epoch 6/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  161
episode_length                      61.658385
returns                            315.955165
return_std                          87.888474
average_reward                       5.123718
round_time             0 days 00:05:52.277426
episodes_test                           161.0
episode_length_test                 62.086957
returns_test                       325.835882
return_std_test                     44.516854
average_reward_test                  5.247919
round_time_test        0 days 00:00:14.972470
round_time_total       0 days 00:05:52.278542
loss_total                      224463.901906
loss_critic                     280789.258241
loss_actor                        -837.542691
memory_size                       285665.2795 

=== epoch 6/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:38,  4.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  165
episode_length                      60.460606
returns                            310.437365
return_std                          66.453432
average_reward                       5.134161
round_time             0 days 00:05:53.331714
episodes_test                           166.0
episode_length_test                 60.210843
returns_test                       313.890475
return_std_test                     38.635721
average_reward_test                  5.213023
round_time_test        0 days 00:00:15.039219
round_time_total       0 days 00:05:53.333006
loss_total                       58804.273584
loss_critic                      73715.583791
loss_actor                        -840.972363
memory_size                        287103.691 

=== epoch 6/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  163
episode_length                      60.865031
returns                            313.352346
return_std                          66.603664
average_reward                       5.146476
round_time             0 days 00:05:56.585290
episodes_test                           163.0
episode_length_test                 61.233129
returns_test                       319.807972
return_std_test                     33.576839
average_reward_test                  5.222282
round_time_test        0 days 00:00:15.008552
round_time_total       0 days 00:05:56.586416
loss_total                        58088.60829
loss_critic                      72823.452873
loss_actor                        -850.775141
memory_size                        288541.926 

=== epoch 6/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  161
episode_length                      61.745342
returns                            317.033493
return_std                          62.529675
average_reward                       5.134456
round_time             0 days 00:05:54.899326
episodes_test                           162.0
episode_length_test                 61.679012
returns_test                        318.09645
return_std_test                     39.955746
average_reward_test                  5.157073
round_time_test        0 days 00:00:15.570219
round_time_total       0 days 00:05:54.900643
loss_total                       57854.134268
loss_critic                      72531.848624
loss_actor                        -856.728373
memory_size                        290008.925 

=== epoch 6/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:43,  4.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  161
episode_length                      61.434783
returns                             315.60912
return_std                          57.624895
average_reward                       5.133389
round_time             0 days 00:05:56.361626
episodes_test                           161.0
episode_length_test                  61.78882
returns_test                       319.486365
return_std_test                     34.403132
average_reward_test                   5.16988
round_time_test        0 days 00:00:15.612517
round_time_total       0 days 00:05:56.362869
loss_total                       57731.702317
loss_critic                      72380.300469
loss_actor                        -862.695414
memory_size                       291466.3955 

=== epoch 6/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  162
episode_length                      61.283951
returns                            314.545601
return_std                          55.967537
average_reward                       5.129648
round_time             0 days 00:05:54.669018
episodes_test                           162.0
episode_length_test                 61.697531
returns_test                       320.431475
return_std_test                     41.299405
average_reward_test                  5.193437
round_time_test        0 days 00:00:15.076141
round_time_total       0 days 00:05:54.670127
loss_total                      186726.765554
loss_critic                     233623.938369
loss_actor                         -861.91133
memory_size                       292906.0925 

=== epoch 6/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  160
episode_length                         61.925
returns                            318.277639
return_std                          58.167845
average_reward                       5.141856
round_time             0 days 00:05:52.775467
episodes_test                           160.0
episode_length_test                  62.16875
returns_test                       320.914489
return_std_test                     38.261867
average_reward_test                   5.16164
round_time_test        0 days 00:00:15.255344
round_time_total       0 days 00:05:52.776708
loss_total                       57488.944224
loss_critic                        72079.2875
loss_actor                        -872.433553
memory_size                        294350.014 

=== epoch 6/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  159
episode_length                      62.477987
returns                            321.647626
return_std                          55.960491
average_reward                       5.146003
round_time             0 days 00:05:52.453215
episodes_test                           162.0
episode_length_test                 61.722222
returns_test                       320.195783
return_std_test                     39.996556
average_reward_test                  5.187663
round_time_test        0 days 00:00:15.167804
round_time_total       0 days 00:05:52.454450
loss_total                       88055.969837
loss_critic                     110287.720277
loss_actor                        -871.035643
memory_size                       295814.7265 

=== epoch 6/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  158
episode_length                      63.120253
returns                            324.881632
return_std                          54.644482
average_reward                       5.147215
round_time             0 days 00:05:53.743128
episodes_test                           164.0
episode_length_test                 60.621951
returns_test                       315.868308
return_std_test                      41.69044
average_reward_test                  5.210597
round_time_test        0 days 00:00:15.203069
round_time_total       0 days 00:05:53.744229
loss_total                       63514.188448
loss_critic                      79607.597107
loss_actor                        -859.450718
memory_size                       297287.4035 

=== epoch 6/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  159
episode_length                      62.842767
returns                            323.690482
return_std                          57.607674
average_reward                       5.150936
round_time             0 days 00:05:55.112877
episodes_test                           155.0
episode_length_test                 64.509677
returns_test                       334.998749
return_std_test                     43.947335
average_reward_test                   5.19297
round_time_test        0 days 00:00:14.923714
round_time_total       0 days 00:05:55.113996
loss_total                      301731.084321
loss_critic                     377375.296928
loss_actor                         -845.72758
memory_size                       298773.2465 

=== epoch 6/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  156
episode_length                      63.794872
returns                            328.472587
return_std                           64.03549
average_reward                       5.148867
round_time             0 days 00:05:52.302019
episodes_test                           159.0
episode_length_test                 62.660377
returns_test                       325.862219
return_std_test                     46.578714
average_reward_test                  5.200036
round_time_test        0 days 00:00:15.161207
round_time_total       0 days 00:05:52.303400
loss_total                      223044.159106
loss_critic                     279018.127554
loss_actor                         -851.74145
memory_size                        300214.705 

=== epoch 6/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  155
episode_length                      64.419355
returns                            330.153661
return_std                          64.190296
average_reward                        5.12571
round_time             0 days 00:05:53.623367
episodes_test                           160.0
episode_length_test                  62.36875
returns_test                       326.198989
return_std_test                     41.758236
average_reward_test                  5.229563
round_time_test        0 days 00:00:14.809835
round_time_total       0 days 00:05:53.624522
loss_total                       53547.473328
loss_critic                      67147.329656
loss_actor                        -851.956742
memory_size                       301694.3995 

=== epoch 6/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  156
episode_length                      63.833333
returns                            327.637807
return_std                          65.898443
average_reward                       5.133333
round_time             0 days 00:05:51.589174
episodes_test                           152.0
episode_length_test                 65.730263
returns_test                       341.485878
return_std_test                     52.887726
average_reward_test                  5.194984
round_time_test        0 days 00:00:15.183916
round_time_total       0 days 00:05:51.590273
loss_total                       53871.757169
loss_critic                      67553.661926
loss_actor                        -855.866513
memory_size                        303165.813 

=== epoch 6/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  158
episode_length                      62.867089
returns                            323.701657
return_std                          71.383012
average_reward                       5.147571
round_time             0 days 00:05:54.892954
episodes_test                           158.0
episode_length_test                 63.291139
returns_test                        332.36862
return_std_test                     54.453653
average_reward_test                  5.251424
round_time_test        0 days 00:00:15.076415
round_time_total       0 days 00:05:54.894069
loss_total                        114059.7639
loss_critic                       142790.5374
loss_actor                        -863.334446
memory_size                       304599.6525 

=== epoch 6/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  154
episode_length                      64.396104
returns                            333.022078
return_std                           80.34808
average_reward                       5.172104
round_time             0 days 00:05:51.816582
episodes_test                           144.0
episode_length_test                   69.4375
returns_test                        364.51051
return_std_test                      70.23342
average_reward_test                  5.249441
round_time_test        0 days 00:00:14.405416
round_time_total       0 days 00:05:51.817678
loss_total                      339002.671493
loss_critic                     423968.366964
loss_actor                        -860.198545
memory_size                       306078.3815 

=== epoch 6/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  154
episode_length                      64.285714
returns                             332.92986
return_std                          78.693824
average_reward                       5.177834
round_time             0 days 00:05:54.745048
episodes_test                           161.0
episode_length_test                  61.73913
returns_test                       322.178001
return_std_test                     42.657328
average_reward_test                  5.217304
round_time_test        0 days 00:00:14.954587
round_time_total       0 days 00:05:54.746184
loss_total                       88632.148568
loss_critic                     111008.628815
loss_actor                        -873.772445
memory_size                        307583.214 

=== epoch 6/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  155
episode_length                      64.206452
returns                            334.281084
return_std                          77.823426
average_reward                        5.20748
round_time             0 days 00:05:55.131578
episodes_test                           157.0
episode_length_test                 63.528662
returns_test                       333.485809
return_std_test                     42.117088
average_reward_test                  5.248615
round_time_test        0 days 00:00:14.949486
round_time_total       0 days 00:05:55.132728
loss_total                       54978.422082
loss_critic                      68940.760318
loss_actor                        -870.935625
memory_size                        309054.613 

=== epoch 6/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  152
episode_length                      65.236842
returns                            338.880579
return_std                          75.277586
average_reward                       5.194278
round_time             0 days 00:05:55.933249
episodes_test                           154.0
episode_length_test                 64.493506
returns_test                       336.321003
return_std_test                     38.853974
average_reward_test                  5.216426
round_time_test        0 days 00:00:15.090500
round_time_total       0 days 00:05:55.934596
loss_total                       53265.956443
loss_critic                       66799.28168
loss_actor                        -867.348728
memory_size                       310541.6355 

=== epoch 6/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  152
episode_length                      65.230263
returns                            340.098619
return_std                          72.089275
average_reward                       5.212658
round_time             0 days 00:05:56.760638
episodes_test                           155.0
episode_length_test                 64.496774
returns_test                       341.674439
return_std_test                     54.384799
average_reward_test                  5.297425
round_time_test        0 days 00:00:14.729258
round_time_total       0 days 00:05:56.761917
loss_total                        57507.93703
loss_critic                      72101.855755
loss_actor                        -867.741794
memory_size                        312013.927 

=== epoch 6/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:26,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  154
episode_length                      64.402597
returns                            334.472638
return_std                          62.474715
average_reward                         5.1935
round_time             0 days 00:05:57.694703
episodes_test                           155.0
episode_length_test                 64.290323
returns_test                       342.640741
return_std_test                     54.101342
average_reward_test                  5.327925
round_time_test        0 days 00:00:14.694877
round_time_total       0 days 00:05:57.695829
loss_total                       54896.641854
loss_critic                      68839.510854
loss_actor                        -874.837968
memory_size                        313461.478 

=== epoch 6/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  153
episode_length                      64.764706
returns                            337.167667
return_std                          71.340778
average_reward                       5.204764
round_time             0 days 00:05:59.358079
episodes_test                           153.0
episode_length_test                 65.163399
returns_test                       347.300463
return_std_test                     48.673879
average_reward_test                  5.328209
round_time_test        0 days 00:00:14.823346
round_time_total       0 days 00:05:59.359180
loss_total                       52711.081219
loss_critic                      66108.648518
loss_actor                        -879.192187
memory_size                       314935.1835 

=== epoch 6/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  152
episode_length                      65.105263
returns                            338.338029
return_std                          79.404146
average_reward                       5.196629
round_time             0 days 00:05:56.655693
episodes_test                           161.0
episode_length_test                 62.111801
returns_test                       332.740486
return_std_test                     55.274022
average_reward_test                  5.357122
round_time_test        0 days 00:00:14.532086
round_time_total       0 days 00:05:56.656804
loss_total                       76817.250382
loss_critic                      96242.089531
loss_actor                        -882.111949
memory_size                       316429.8925 

=== epoch 6/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:33,  4.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  152
episode_length                      65.210526
returns                            338.672067
return_std                          77.189595
average_reward                       5.194423
round_time             0 days 00:05:59.395881
episodes_test                           148.0
episode_length_test                  67.22973
returns_test                       358.687034
return_std_test                      53.34009
average_reward_test                   5.33495
round_time_test        0 days 00:00:14.468690
round_time_total       0 days 00:05:59.397032
loss_total                       52420.221588
loss_critic                      65743.488306
loss_actor                        -872.849582
memory_size                       317930.3285 

=== epoch 6/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  149
episode_length                      66.758389
returns                            346.349976
return_std                          78.853892
average_reward                       5.187575
round_time             0 days 00:05:59.549428
episodes_test                           143.0
episode_length_test                 69.566434
returns_test                        370.29488
return_std_test                     69.443372
average_reward_test                  5.323232
round_time_test        0 days 00:00:14.601315
round_time_total       0 days 00:05:59.550565
loss_total                       51357.664848
loss_critic                      64416.964906
loss_actor                        -879.540289
memory_size                       319430.0055 

=== epoch 6/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  147
episode_length                      67.176871
returns                            348.655624
return_std                          79.208584
average_reward                       5.190881
round_time             0 days 00:05:58.629691
episodes_test                           155.0
episode_length_test                  64.36129
returns_test                       343.554829
return_std_test                     53.751396
average_reward_test                     5.337
round_time_test        0 days 00:00:14.719444
round_time_total       0 days 00:05:58.630800
loss_total                       52027.243048
loss_critic                      65254.991078
loss_actor                        -883.753644
memory_size                        320929.549 

=== epoch 6/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  145
episode_length                       68.57931
returns                            355.100242
return_std                          76.494026
average_reward                       5.177604
round_time             0 days 00:05:58.473741
episodes_test                           141.0
episode_length_test                 70.737589
returns_test                        375.45069
return_std_test                     66.459056
average_reward_test                   5.30683
round_time_test        0 days 00:00:14.316699
round_time_total       0 days 00:05:58.474830
loss_total                       51569.661676
loss_critic                      64683.112579
loss_actor                          -884.1461
memory_size                        322459.077 

=== epoch 6/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:19,  4.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  144
episode_length                      68.958333
returns                            356.344934
return_std                          73.181624
average_reward                       5.169324
round_time             0 days 00:06:01.467182
episodes_test                           145.0
episode_length_test                 68.910345
returns_test                        371.21502
return_std_test                     61.628519
average_reward_test                  5.386541
round_time_test        0 days 00:00:14.246533
round_time_total       0 days 00:06:01.468295
loss_total                       52051.473816
loss_critic                      65286.830955
loss_actor                        -889.959281
memory_size                       323966.9295 

=== epoch 6/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:18,  4.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  146
episode_length                      67.958904
returns                             352.74992
return_std                          77.339115
average_reward                       5.188836
round_time             0 days 00:05:58.451562
episodes_test                           147.0
episode_length_test                 67.761905
returns_test                       367.475733
return_std_test                     66.303035
average_reward_test                  5.421421
round_time_test        0 days 00:00:14.091002
round_time_total       0 days 00:05:58.452665
loss_total                       53464.865199
loss_critic                      67056.183757
loss_actor                        -900.413471
memory_size                        325446.792 

=== epoch 6/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:21,  4.51it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  145
episode_length                      68.517241
returns                            355.468442
return_std                          77.313091
average_reward                       5.189671
round_time             0 days 00:06:00.922337
episodes_test                           141.0
episode_length_test                 70.829787
returns_test                       381.203866
return_std_test                     65.949764
average_reward_test                   5.38139
round_time_test        0 days 00:00:14.163403
round_time_total       0 days 00:06:00.923443
loss_total                       53094.842435
loss_critic                      66593.966681
loss_actor                        -901.658941
memory_size                        326936.503 

=== epoch 6/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  146
episode_length                      67.924658
returns                            353.814692
return_std                          75.596981
average_reward                       5.208229
round_time             0 days 00:05:58.256409
episodes_test                           132.0
episode_length_test                 75.492424
returns_test                       400.925694
return_std_test                      69.90376
average_reward_test                  5.309645
round_time_test        0 days 00:00:14.137059
round_time_total       0 days 00:05:58.257740
loss_total                      105117.553211
loss_critic                     131619.735586
loss_actor                        -891.188087
memory_size                       328427.6335 

=== epoch 6/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:31,  4.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  146
episode_length                           68.0
returns                            354.316124
return_std                          70.170379
average_reward                       5.210699
round_time             0 days 00:05:59.809891
episodes_test                           139.0
episode_length_test                 71.726619
returns_test                       386.056238
return_std_test                     76.995537
average_reward_test                  5.381554
round_time_test        0 days 00:00:14.357111
round_time_total       0 days 00:05:59.811271
loss_total                       50774.412546
loss_critic                       63689.59643
loss_actor                        -886.327076
memory_size                         329956.76 

=== epoch 6/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  147
episode_length                      67.564626
returns                            353.103328
return_std                          73.764096
average_reward                       5.226697
round_time             0 days 00:05:58.813314
episodes_test                           153.0
episode_length_test                 65.281046
returns_test                       355.704991
return_std_test                      64.04503
average_reward_test                  5.448208
round_time_test        0 days 00:00:14.109043
round_time_total       0 days 00:05:58.814417
loss_total                      232230.641484
loss_critic                     290510.830885
loss_actor                        -890.179061
memory_size                       331458.1395 

=== epoch 6/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  144
episode_length                      69.215278
returns                            362.013167
return_std                          73.443333
average_reward                       5.231631
round_time             0 days 00:05:58.408454
episodes_test                           142.0
episode_length_test                 70.239437
returns_test                       378.445694
return_std_test                     63.518858
average_reward_test                  5.387034
round_time_test        0 days 00:00:14.448247
round_time_total       0 days 00:05:58.409567
loss_total                       50852.405518
loss_critic                      63789.470119
loss_actor                         -895.85743
memory_size                       332964.3155 

=== epoch 6/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  144
episode_length                      69.152778
returns                            360.193552
return_std                          77.449515
average_reward                       5.209599
round_time             0 days 00:05:58.684557
episodes_test                           138.0
episode_length_test                 72.391304
returns_test                       385.969729
return_std_test                     67.549366
average_reward_test                  5.331276
round_time_test        0 days 00:00:14.676717
round_time_total       0 days 00:05:58.685836
loss_total                        51534.03864
loss_critic                       64641.71637
loss_actor                        -896.676885
memory_size                       334480.6835 

=== epoch 6/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  143
episode_length                      69.503497
returns                            359.618954
return_std                          78.480357
average_reward                       5.175934
round_time             0 days 00:05:57.599697
episodes_test                           140.0
episode_length_test                 71.028571
returns_test                        381.59972
return_std_test                     63.028614
average_reward_test                  5.372381
round_time_test        0 days 00:00:14.382700
round_time_total       0 days 00:05:57.601157
loss_total                       51017.959864
loss_critic                      63995.624307
loss_actor                        -892.702059
memory_size                        335991.101 

=== epoch 6/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:03,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  144
episode_length                      68.840278
returns                            356.060161
return_std                            84.1623
average_reward                       5.171255
round_time             0 days 00:05:55.244910
episodes_test                           133.0
episode_length_test                 75.172932
returns_test                       403.959947
return_std_test                     84.179017
average_reward_test                  5.373648
round_time_test        0 days 00:00:13.886121
round_time_total       0 days 00:05:55.246290
loss_total                       50469.602919
loss_critic                      63311.911271
loss_actor                        -899.634798
memory_size                        337488.324 

=== epoch 6/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  143
episode_length                      69.398601
returns                            356.415791
return_std                          91.866156
average_reward                        5.13785
round_time             0 days 00:05:59.185520
episodes_test                           132.0
episode_length_test                 75.469697
returns_test                       402.811696
return_std_test                     75.854865
average_reward_test                  5.336639
round_time_test        0 days 00:00:14.346927
round_time_total       0 days 00:05:59.187049
loss_total                       50452.756489
loss_critic                      63291.107109
loss_actor                        -900.650088
memory_size                       339007.2165 

=== epoch 6/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  144
episode_length                      68.715278
returns                            351.397622
return_std                           93.55283
average_reward                       5.116018
round_time             0 days 00:05:56.474397
episodes_test                           131.0
episode_length_test                 75.954198
returns_test                        404.90059
return_std_test                     74.162188
average_reward_test                   5.32996
round_time_test        0 days 00:00:14.474584
round_time_total       0 days 00:05:56.475496
loss_total                       50444.379998
loss_critic                      63280.525954
loss_actor                        -900.207823
memory_size                        340497.905 

=== epoch 6/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  141
episode_length                      70.347518
returns                            359.454526
return_std                          98.261112
average_reward                       5.111214
round_time             0 days 00:05:59.331489
episodes_test                           131.0
episode_length_test                 75.854962
returns_test                       403.417735
return_std_test                     62.132098
average_reward_test                  5.318229
round_time_test        0 days 00:00:14.514817
round_time_total       0 days 00:05:59.332571
loss_total                       51942.306024
loss_critic                      65154.702804
loss_actor                        -907.285575
memory_size                        342032.717 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 7/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<07:00,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                   28
episode_length                      71.214286
returns                              365.6148
return_std                          83.543637
average_reward                        5.13331
round_time             0 days 00:05:43.488586
episodes_test                           139.0
episode_length_test                 71.539568
returns_test                       383.407333
return_std_test                     64.834864
average_reward_test                  5.357674
round_time_test        0 days 00:00:14.387568
round_time_total       0 days 00:05:43.490043
loss_total                       52782.119697
loss_critic                       66203.28075
loss_actor                        -902.529536
memory_size                       343576.7695 

=== epoch 7/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                   55
episode_length                      71.672727
returns                            364.672533
return_std                          81.864896
average_reward                       5.088529
round_time             0 days 00:05:46.847488
episodes_test                           124.0
episode_length_test                 79.830645
returns_test                       420.333584
return_std_test                     89.719235
average_reward_test                  5.264212
round_time_test        0 days 00:00:14.608318
round_time_total       0 days 00:05:46.848587
loss_total                       51523.511263
loss_critic                      64630.644862
loss_actor                        -905.027478
memory_size                       345108.8935 

=== epoch 7/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                   84
episode_length                       70.72619
returns                            362.136961
return_std                          88.977831
average_reward                       5.120311
round_time             0 days 00:05:50.431862
episodes_test                           124.0
episode_length_test                 79.806452
returns_test                       414.675777
return_std_test                    104.935991
average_reward_test                  5.197279
round_time_test        0 days 00:00:14.392521
round_time_total       0 days 00:05:50.433137
loss_total                       51883.937544
loss_critic                      65081.774603
loss_actor                        -907.415368
memory_size                       346606.4085 

=== epoch 7/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  111
episode_length                      72.036036
returns                            367.992083
return_std                          92.709589
average_reward                       5.108342
round_time             0 days 00:05:50.849450
episodes_test                           124.0
episode_length_test                 80.379032
returns_test                       420.820585
return_std_test                    111.048505
average_reward_test                  5.234682
round_time_test        0 days 00:00:14.442261
round_time_total       0 days 00:05:50.850681
loss_total                       51589.549134
loss_critic                      64713.723861
loss_actor                        -907.154666
memory_size                        348160.368 

=== epoch 7/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  138
episode_length                      72.449275
returns                            371.120223
return_std                          91.924684
average_reward                       5.122438
round_time             0 days 00:05:52.283241
episodes_test                           124.0
episode_length_test                 80.556452
returns_test                       417.952854
return_std_test                    107.233722
average_reward_test                  5.188016
round_time_test        0 days 00:00:14.665946
round_time_total       0 days 00:05:52.284353
loss_total                       50685.987091
loss_critic                      63585.462866
loss_actor                        -911.920129
memory_size                       349687.3655 

=== epoch 7/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  134
episode_length                      73.932836
returns                            378.104134
return_std                          104.92947
average_reward                         5.1111
round_time             0 days 00:05:50.179864
episodes_test                           121.0
episode_length_test                 82.338843
returns_test                       430.673969
return_std_test                     100.59958
average_reward_test                   5.22968
round_time_test        0 days 00:00:14.436335
round_time_total       0 days 00:05:50.181076
loss_total                        50643.96284
loss_critic                      63532.491134
loss_actor                        -910.154469
memory_size                        351236.566 

=== epoch 7/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:43,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  134
episode_length                           74.5
returns                            382.966752
return_std                         106.696086
average_reward                        5.14031
round_time             0 days 00:05:51.716225
episodes_test                           114.0
episode_length_test                 87.473684
returns_test                       454.520633
return_std_test                    112.307522
average_reward_test                  5.195636
round_time_test        0 days 00:00:15.002666
round_time_total       0 days 00:05:51.717322
loss_total                       51276.623824
loss_critic                      64323.652958
loss_actor                        -911.496778
memory_size                       352813.5175 

=== epoch 7/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:22,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  132
episode_length                      75.371212
returns                            387.520551
return_std                         105.506506
average_reward                       5.143492
round_time             0 days 00:05:54.920984
episodes_test                           120.0
episode_length_test                 83.016667
returns_test                       432.803358
return_std_test                     98.622596
average_reward_test                  5.212317
round_time_test        0 days 00:00:14.730303
round_time_total       0 days 00:05:54.922102
loss_total                       54104.907918
loss_critic                      67860.327409
loss_actor                        -916.774769
memory_size                       354340.4985 

=== epoch 7/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  132
episode_length                      74.886364
returns                            386.255725
return_std                         101.151617
average_reward                       5.152868
round_time             0 days 00:05:51.539087
episodes_test                           133.0
episode_length_test                 74.864662
returns_test                       398.915713
return_std_test                     75.327095
average_reward_test                  5.326953
round_time_test        0 days 00:00:14.393040
round_time_total       0 days 00:05:51.540214
loss_total                       52470.270512
loss_critic                      65818.813787
loss_actor                         -923.90682
memory_size                        355892.249 

=== epoch 7/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:14,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  132
episode_length                      74.734848
returns                            383.666068
return_std                         102.806299
average_reward                       5.129211
round_time             0 days 00:05:52.919759
episodes_test                           120.0
episode_length_test                 82.958333
returns_test                       427.195014
return_std_test                    126.037739
average_reward_test                  5.149997
round_time_test        0 days 00:00:14.964713
round_time_total       0 days 00:05:52.921290
loss_total                       52459.911132
loss_critic                      65806.196126
loss_actor                        -925.233325
memory_size                       357417.7685 

=== epoch 7/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  133
episode_length                      74.458647
returns                            381.306753
return_std                          93.168166
average_reward                       5.121349
round_time             0 days 00:05:51.280926
episodes_test                           116.0
episode_length_test                 85.586207
returns_test                       428.940559
return_std_test                    119.334215
average_reward_test                  5.013436
round_time_test        0 days 00:00:15.039225
round_time_total       0 days 00:05:51.282024
loss_total                       52052.526896
loss_critic                      65297.579495
loss_actor                        -927.688473
memory_size                        358966.411 

=== epoch 7/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:10,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  133
episode_length                      74.609023
returns                            380.174181
return_std                          98.470768
average_reward                       5.097114
round_time             0 days 00:05:53.874384
episodes_test                           132.0
episode_length_test                      75.5
returns_test                       392.292094
return_std_test                     90.022286
average_reward_test                  5.195514
round_time_test        0 days 00:00:14.238373
round_time_total       0 days 00:05:53.875543
loss_total                       52433.011976
loss_critic                      65773.128643
loss_actor                         -927.45936
memory_size                       360527.7175 

=== epoch 7/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  131
episode_length                      75.496183
returns                            383.154054
return_std                         114.880232
average_reward                       5.080296
round_time             0 days 00:05:54.103529
episodes_test                           126.0
episode_length_test                 78.888889
returns_test                       399.034437
return_std_test                    112.106572
average_reward_test                  5.056973
round_time_test        0 days 00:00:14.772780
round_time_total       0 days 00:05:54.104646
loss_total                       52448.325503
loss_critic                      65791.477474
loss_actor                        -924.286387
memory_size                        362094.974 

=== epoch 7/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  133
episode_length                      74.278195
returns                            377.700211
return_std                         116.397665
average_reward                       5.086384
round_time             0 days 00:05:54.190631
episodes_test                           125.0
episode_length_test                    79.808
returns_test                       409.159046
return_std_test                    122.415966
average_reward_test                  5.126541
round_time_test        0 days 00:00:14.655684
round_time_total       0 days 00:05:54.191913
loss_total                      561320.440105
loss_critic                      701880.47254
loss_actor                        -919.598671
memory_size                        363621.609 

=== epoch 7/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  133
episode_length                      74.706767
returns                            380.881618
return_std                         116.161287
average_reward                       5.100322
round_time             0 days 00:05:54.517848
episodes_test                           131.0
episode_length_test                 75.885496
returns_test                       398.155991
return_std_test                    100.530801
average_reward_test                  5.246859
round_time_test        0 days 00:00:14.400946
round_time_total       0 days 00:05:54.518959
loss_total                       49937.186389
loss_critic                      62650.374696
loss_actor                        -915.571327
memory_size                       365147.3515 

=== epoch 7/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:30,  4.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  132
episode_length                      74.916667
returns                            382.216332
return_std                         139.787699
average_reward                       5.101821
round_time             0 days 00:05:53.991126
episodes_test                           131.0
episode_length_test                 76.328244
returns_test                       391.469229
return_std_test                     86.823214
average_reward_test                  5.128734
round_time_test        0 days 00:00:14.504629
round_time_total       0 days 00:05:53.992229
loss_total                       51852.683071
loss_critic                      65048.199058
loss_actor                        -929.385487
memory_size                       366697.9905 

=== epoch 7/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  132
episode_length                      75.015152
returns                            382.763726
return_std                         135.644927
average_reward                       5.103438
round_time             0 days 00:05:55.115641
episodes_test                           123.0
episode_length_test                 81.081301
returns_test                       420.105343
return_std_test                     79.275318
average_reward_test                  5.180704
round_time_test        0 days 00:00:15.384885
round_time_total       0 days 00:05:55.116761
loss_total                       58423.104699
loss_critic                      73260.678385
loss_actor                        -927.194275
memory_size                        368258.753 

=== epoch 7/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  132
episode_length                      75.613636
returns                            384.763039
return_std                         127.662495
average_reward                       5.088241
round_time             0 days 00:05:59.025070
episodes_test                           130.0
episode_length_test                 76.576923
returns_test                       395.984055
return_std_test                     86.200149
average_reward_test                  5.169948
round_time_test        0 days 00:00:14.361825
round_time_total       0 days 00:05:59.026179
loss_total                      555780.884437
loss_critic                     694957.541103
loss_actor                        -925.783554
memory_size                       369838.1815 

=== epoch 7/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  129
episode_length                      76.790698
returns                            390.384493
return_std                         132.482046
average_reward                       5.085667
round_time             0 days 00:05:59.032428
episodes_test                           133.0
episode_length_test                 74.691729
returns_test                       388.551362
return_std_test                     86.038051
average_reward_test                  5.199936
round_time_test        0 days 00:00:14.705678
round_time_total       0 days 00:05:59.033518
loss_total                       54202.345146
loss_critic                      67986.960734
loss_actor                        -936.122242
memory_size                       371388.5485 

=== epoch 7/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:35,  4.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  128
episode_length                      78.015625
returns                            395.498353
return_std                         133.449701
average_reward                       5.069239
round_time             0 days 00:05:59.049134
episodes_test                           133.0
episode_length_test                 75.120301
returns_test                       390.150283
return_std_test                     86.875196
average_reward_test                  5.193409
round_time_test        0 days 00:00:14.540680
round_time_total       0 days 00:05:59.050239
loss_total                       53240.120768
loss_critic                       66785.04344
loss_actor                        -939.574607
memory_size                       372950.4085 

=== epoch 7/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  129
episode_length                      77.100775
returns                            392.213499
return_std                         116.276769
average_reward                       5.088426
round_time             0 days 00:05:57.259698
episodes_test                           122.0
episode_length_test                 81.401639
returns_test                       416.853298
return_std_test                    101.294845
average_reward_test                  5.120979
round_time_test        0 days 00:00:14.823786
round_time_total       0 days 00:05:57.260787
loss_total                       57603.166503
loss_critic                      72241.179516
loss_actor                        -948.890284
memory_size                       374507.8815 

=== epoch 7/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:27,  4.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  129
episode_length                      76.891473
returns                             389.85019
return_std                         117.336992
average_reward                       5.072005
round_time             0 days 00:05:58.316827
episodes_test                           123.0
episode_length_test                  79.96748
returns_test                       414.574874
return_std_test                    107.220356
average_reward_test                  5.180765
round_time_test        0 days 00:00:14.979687
round_time_total       0 days 00:05:58.317924
loss_total                      577448.937228
loss_critic                     722046.601636
loss_actor                        -941.618158
memory_size                        376052.711 

=== epoch 7/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:28,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  131
episode_length                      75.908397
returns                            385.719318
return_std                         115.193315
average_reward                       5.077575
round_time             0 days 00:05:55.879912
episodes_test                           117.0
episode_length_test                 85.188034
returns_test                       435.011082
return_std_test                    115.514976
average_reward_test                  5.106303
round_time_test        0 days 00:00:14.928754
round_time_total       0 days 00:05:55.881293
loss_total                       62049.667427
loss_critic                      77795.947279
loss_actor                        -935.455832
memory_size                        377608.076 

=== epoch 7/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  131
episode_length                      75.465649
returns                            382.216614
return_std                         115.960682
average_reward                       5.062431
round_time             0 days 00:05:58.090534
episodes_test                           123.0
episode_length_test                 80.601626
returns_test                       409.399626
return_std_test                    110.231524
average_reward_test                  5.078001
round_time_test        0 days 00:00:14.867183
round_time_total       0 days 00:05:58.091622
loss_total                       57650.086595
loss_critic                      72297.100045
loss_actor                        -937.971786
memory_size                        379134.235 

=== epoch 7/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  131
episode_length                      75.938931
returns                            383.321782
return_std                         114.886401
average_reward                       5.049665
round_time             0 days 00:05:57.663715
episodes_test                           122.0
episode_length_test                 81.819672
returns_test                        407.54086
return_std_test                    123.327575
average_reward_test                  4.980893
round_time_test        0 days 00:00:15.420902
round_time_total       0 days 00:05:57.664830
loss_total                       61352.532729
loss_critic                      76927.218078
loss_actor                        -946.213496
memory_size                        380707.193 

=== epoch 7/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  132
episode_length                      75.318182
returns                            378.050429
return_std                         107.728821
average_reward                       5.021718
round_time             0 days 00:05:58.387220
episodes_test                           128.0
episode_length_test                 77.671875
returns_test                       389.578025
return_std_test                    106.569741
average_reward_test                  5.013064
round_time_test        0 days 00:00:14.990238
round_time_total       0 days 00:05:58.388709
loss_total                       60496.268737
loss_critic                      75858.082808
loss_actor                        -950.992683
memory_size                       382260.2075 

=== epoch 7/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:44,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  131
episode_length                      75.450382
returns                            375.930836
return_std                          106.25919
average_reward                       4.984721
round_time             0 days 00:05:56.702954
episodes_test                           120.0
episode_length_test                 82.916667
returns_test                       415.226657
return_std_test                    118.106107
average_reward_test                  5.008846
round_time_test        0 days 00:00:15.057684
round_time_total       0 days 00:05:56.704476
loss_total                      100217.503538
loss_critic                     125508.695547
loss_actor                        -947.283099
memory_size                       383812.2555 

=== epoch 7/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:40,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  131
episode_length                       75.51145
returns                            373.301417
return_std                         110.053245
average_reward                       4.945717
round_time             0 days 00:05:58.100715
episodes_test                           121.0
episode_length_test                 82.272727
returns_test                       401.686541
return_std_test                    136.913826
average_reward_test                   4.88185
round_time_test        0 days 00:00:14.898669
round_time_total       0 days 00:05:58.101827
loss_total                       57832.646012
loss_critic                      72526.499444
loss_actor                        -942.772498
memory_size                       385351.0795 

=== epoch 7/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  131
episode_length                       75.78626
returns                            371.190856
return_std                         109.748027
average_reward                       4.898905
round_time             0 days 00:05:58.097932
episodes_test                           124.0
episode_length_test                 80.629032
returns_test                       395.101233
return_std_test                    117.328654
average_reward_test                  4.900231
round_time_test        0 days 00:00:15.316819
round_time_total       0 days 00:05:58.099019
loss_total                       92621.889331
loss_critic                     116009.474352
loss_actor                        -928.447495
memory_size                       386894.1195 

=== epoch 7/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  133
episode_length                      74.368421
returns                            364.652979
return_std                         111.833493
average_reward                       4.904883
round_time             0 days 00:05:56.214577
episodes_test                           121.0
episode_length_test                 81.966942
returns_test                       417.222696
return_std_test                    113.244951
average_reward_test                  5.087379
round_time_test        0 days 00:00:15.129784
round_time_total       0 days 00:05:56.215819
loss_total                       53594.045753
loss_critic                       67224.20619
loss_actor                        -926.600942
memory_size                       388444.3165 

=== epoch 7/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  130
episode_length                      76.084615
returns                            372.309622
return_std                         117.412955
average_reward                       4.898696
round_time             0 days 00:05:57.439436
episodes_test                           130.0
episode_length_test                 76.576923
returns_test                       389.280908
return_std_test                     95.323473
average_reward_test                  5.083607
round_time_test        0 days 00:00:14.645965
round_time_total       0 days 00:05:57.440668
loss_total                       52006.140939
loss_critic                      65239.317623
loss_actor                        -926.570364
memory_size                       389991.6235 

=== epoch 7/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  130
episode_length                      76.030769
returns                            375.373518
return_std                         125.282612
average_reward                          4.936
round_time             0 days 00:05:57.388188
episodes_test                           123.0
episode_length_test                 80.918699
returns_test                       405.016819
return_std_test                    114.387259
average_reward_test                  5.005298
round_time_test        0 days 00:00:14.886893
round_time_total       0 days 00:05:57.389289
loss_total                      1247362.34226
loss_critic                    1559431.813842
loss_actor                           -915.773
memory_size                       391564.3965 

=== epoch 7/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:48,  4.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  131
episode_length                      75.916031
returns                            375.788548
return_std                         118.778116
average_reward                       4.950586
round_time             0 days 00:05:55.901583
episodes_test                           130.0
episode_length_test                 76.676923
returns_test                       399.405511
return_std_test                     80.659966
average_reward_test                  5.208259
round_time_test        0 days 00:00:14.657664
round_time_total       0 days 00:05:55.902675
loss_total                      293812.269135
loss_critic                     367491.375434
loss_actor                        -904.173199
memory_size                        393119.447 

=== epoch 7/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  131
episode_length                      75.778626
returns                            376.974957
return_std                         112.754392
average_reward                       4.977483
round_time             0 days 00:05:56.713832
episodes_test                           115.0
episode_length_test                 86.730435
returns_test                       429.378912
return_std_test                    111.033283
average_reward_test                  4.950643
round_time_test        0 days 00:00:14.850560
round_time_total       0 days 00:05:56.715050
loss_total                      213613.222907
loss_critic                     267245.306392
loss_actor                        -915.170315
memory_size                       394656.5795 

=== epoch 7/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  129
episode_length                      76.581395
returns                            382.031537
return_std                         108.307818
average_reward                       4.989629
round_time             0 days 00:05:58.227663
episodes_test                           121.0
episode_length_test                 82.132231
returns_test                       415.386376
return_std_test                    104.718785
average_reward_test                  5.057483
round_time_test        0 days 00:00:14.781870
round_time_total       0 days 00:05:58.228752
loss_total                       48789.247277
loss_critic                      61216.937713
loss_actor                        -921.518478
memory_size                        396216.254 

=== epoch 7/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  133
episode_length                      75.120301
returns                             377.37761
return_std                          99.811812
average_reward                       5.024048
round_time             0 days 00:05:58.772391
episodes_test                           127.0
episode_length_test                 78.086614
returns_test                       396.952935
return_std_test                     84.664301
average_reward_test                  5.081034
round_time_test        0 days 00:00:14.424993
round_time_total       0 days 00:05:58.773494
loss_total                      624246.506713
loss_critic                     780534.859576
loss_actor                        -906.882847
memory_size                       397758.8435 

=== epoch 7/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:22,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  132
episode_length                      75.409091
returns                            379.858497
return_std                          94.259113
average_reward                       5.036948
round_time             0 days 00:05:56.998490
episodes_test                           125.0
episode_length_test                    79.992
returns_test                       401.661739
return_std_test                     90.023228
average_reward_test                  5.021263
round_time_test        0 days 00:00:14.828674
round_time_total       0 days 00:05:56.999731
loss_total                     1016760.803317
loss_critic                    1271176.435005
loss_actor                        -901.573636
memory_size                        399301.331 

=== epoch 7/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:36,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  132
episode_length                      74.825758
returns                             377.35869
return_std                          92.993359
average_reward                        5.03856
round_time             0 days 00:05:59.233723
episodes_test                           130.0
episode_length_test                 76.746154
returns_test                       389.524069
return_std_test                     76.438395
average_reward_test                  5.075115
round_time_test        0 days 00:00:14.512041
round_time_total       0 days 00:05:59.235001
loss_total                       47882.508763
loss_critic                      60081.848203
loss_actor                        -914.853358
memory_size                        400846.224 

=== epoch 7/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 3/2000 [00:01<17:30,  1.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  131
episode_length                      75.419847
returns                            380.132234
return_std                          91.664115
average_reward                       5.043017
round_time             0 days 00:05:57.112602
episodes_test                           132.0
episode_length_test                 75.606061
returns_test                       389.577875
return_std_test                     97.852835
average_reward_test                  5.152084
round_time_test        0 days 00:00:14.408023
round_time_total       0 days 00:05:57.113939
loss_total                       45758.237593
loss_critic                      57426.847679
loss_actor                        -916.206889
memory_size                       402408.0405 

=== epoch 7/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:22,  5.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  130
episode_length                      76.161538
returns                            384.922657
return_std                          91.194181
average_reward                       5.052079
round_time             0 days 00:05:55.810412
episodes_test                           129.0
episode_length_test                 77.046512
returns_test                       400.053054
return_std_test                     77.950298
average_reward_test                  5.192119
round_time_test        0 days 00:00:14.397822
round_time_total       0 days 00:05:55.811729
loss_total                       47572.429979
loss_critic                      59694.623946
loss_actor                        -916.350041
memory_size                       403963.7625 

=== epoch 7/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  131
episode_length                      76.290076
returns                            384.241238
return_std                          97.497177
average_reward                        5.03651
round_time             0 days 00:05:59.990821
episodes_test                           122.0
episode_length_test                 81.942623
returns_test                       417.435654
return_std_test                    116.037428
average_reward_test                  5.094178
round_time_test        0 days 00:00:14.551419
round_time_total       0 days 00:05:59.991938
loss_total                       48710.629239
loss_critic                      61119.618152
loss_actor                        -925.330272
memory_size                       405538.8315 

=== epoch 7/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  130
episode_length                      76.346154
returns                            385.302039
return_std                          90.920689
average_reward                       5.045694
round_time             0 days 00:05:57.671424
episodes_test                           128.0
episode_length_test                 77.953125
returns_test                       406.839219
return_std_test                     91.594806
average_reward_test                  5.217887
round_time_test        0 days 00:00:14.254048
round_time_total       0 days 00:05:57.672525
loss_total                      141225.405215
loss_critic                     176763.256325
loss_actor                        -926.038088
memory_size                        407082.864 

=== epoch 7/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  130
episode_length                      76.276923
returns                            387.358178
return_std                          88.522593
average_reward                       5.074573
round_time             0 days 00:05:59.165882
episodes_test                           126.0
episode_length_test                 79.039683
returns_test                       413.003861
return_std_test                     86.843026
average_reward_test                  5.225013
round_time_test        0 days 00:00:14.504209
round_time_total       0 days 00:05:59.166974
loss_total                       48318.011648
loss_critic                      60626.379889
loss_actor                        -915.465957
memory_size                       408628.5655 

=== epoch 7/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  132
episode_length                      75.295455
returns                            383.460356
return_std                          90.092165
average_reward                       5.093825
round_time             0 days 00:05:59.030793
episodes_test                           119.0
episode_length_test                 83.840336
returns_test                       422.500369
return_std_test                    132.903141
average_reward_test                  5.039183
round_time_test        0 days 00:00:14.519264
round_time_total       0 days 00:05:59.031930
loss_total                        46730.92429
loss_critic                       58642.08591
loss_actor                        -913.726177
memory_size                         410160.87 

=== epoch 7/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:22,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.56it/s]
episodes                                  135
episode_length                      73.266667
returns                            374.097584
return_std                          85.930508
average_reward                       5.107163
round_time             0 days 00:06:00.561879
episodes_test                           122.0
episode_length_test                 81.418033
returns_test                       414.567979
return_std_test                    111.392483
average_reward_test                  5.089086
round_time_test        0 days 00:00:14.735313
round_time_total       0 days 00:06:00.562996
loss_total                      815761.607422
loss_critic                    1019930.784371
loss_actor                        -915.116149
memory_size                        411702.958 

=== epoch 7/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  135
episode_length                      73.577778
returns                            377.385976
return_std                          87.718786
average_reward                       5.129284
round_time             0 days 00:05:59.415422
episodes_test                           124.0
episode_length_test                 79.991935
returns_test                       410.668065
return_std_test                     98.954881
average_reward_test                  5.133024
round_time_test        0 days 00:00:14.491557
round_time_total       0 days 00:05:59.416780
loss_total                       47948.400698
loss_critic                      60166.804413
loss_actor                        -925.218115
memory_size                       413204.4275 

=== epoch 7/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  136
episode_length                      73.323529
returns                            375.367495
return_std                         100.751694
average_reward                       5.119956
round_time             0 days 00:05:58.207237
episodes_test                           129.0
episode_length_test                 77.302326
returns_test                       394.781749
return_std_test                     79.542245
average_reward_test                   5.10689
round_time_test        0 days 00:00:14.627956
round_time_total       0 days 00:05:58.208339
loss_total                       46444.947436
loss_critic                      58289.013924
loss_actor                        -931.322574
memory_size                        414765.337 

=== epoch 7/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:27,  4.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  135
episode_length                      73.540741
returns                            376.443811
return_std                         103.717018
average_reward                       5.119909
round_time             0 days 00:05:56.646033
episodes_test                           130.0
episode_length_test                 76.776923
returns_test                       398.272304
return_std_test                     86.325013
average_reward_test                  5.186806
round_time_test        0 days 00:00:14.289953
round_time_total       0 days 00:05:56.647147
loss_total                       45750.694682
loss_critic                      57420.905266
loss_actor                        -930.151513
memory_size                        416304.079 

=== epoch 7/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:22,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  134
episode_length                      74.037313
returns                            378.858392
return_std                         101.904121
average_reward                        5.11957
round_time             0 days 00:05:59.794090
episodes_test                           133.0
episode_length_test                 74.962406
returns_test                       393.309667
return_std_test                     71.242711
average_reward_test                  5.246028
round_time_test        0 days 00:00:14.551521
round_time_total       0 days 00:05:59.795186
loss_total                       46665.045665
loss_critic                      58563.776907
loss_actor                        -929.883477
memory_size                        417850.026 

=== epoch 7/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  134
episode_length                       73.91791
returns                             377.29105
return_std                          99.219691
average_reward                       5.106738
round_time             0 days 00:05:58.920559
episodes_test                           131.0
episode_length_test                 76.030534
returns_test                       397.513925
return_std_test                     91.066207
average_reward_test                  5.227296
round_time_test        0 days 00:00:14.509999
round_time_total       0 days 00:05:58.922113
loss_total                     1045351.741446
loss_critic                    1306920.073044
loss_actor                        -921.733529
memory_size                         419387.66 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 8/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:26,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   24
episode_length                         82.125
returns                            417.860358
return_std                         134.718954
average_reward                       5.086103
round_time             0 days 00:05:45.392313
episodes_test                           126.0
episode_length_test                 79.269841
returns_test                       404.124401
return_std_test                     93.963368
average_reward_test                  5.097794
round_time_test        0 days 00:00:14.638806
round_time_total       0 days 00:05:45.393575
loss_total                       46393.314869
loss_critic                      58223.042552
loss_actor                        -925.600006
memory_size                        420947.768 

=== epoch 8/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                   51
episode_length                       77.45098
returns                            399.298108
return_std                         117.624096
average_reward                       5.156797
round_time             0 days 00:05:49.344627
episodes_test                           131.0
episode_length_test                 75.916031
returns_test                        393.98099
return_std_test                     92.627584
average_reward_test                  5.189408
round_time_test        0 days 00:00:14.640221
round_time_total       0 days 00:05:49.346095
loss_total                       45111.339495
loss_critic                      56621.056701
loss_actor                        -927.533459
memory_size                        422485.929 

=== epoch 8/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                   77
episode_length                      77.155844
returns                            393.891183
return_std                         113.310425
average_reward                       5.105408
round_time             0 days 00:05:48.783402
episodes_test                           134.0
episode_length_test                 74.529851
returns_test                       388.404912
return_std_test                     74.519534
average_reward_test                  5.211057
round_time_test        0 days 00:00:14.568361
round_time_total       0 days 00:05:48.784504
loss_total                      703093.141332
loss_critic                      879098.03032
loss_actor                        -926.384915
memory_size                        424058.073 

=== epoch 8/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:51,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  104
episode_length                      76.413462
returns                            391.081954
return_std                         107.090869
average_reward                       5.115032
round_time             0 days 00:05:50.095861
episodes_test                           133.0
episode_length_test                 75.135338
returns_test                       393.824889
return_std_test                     81.475029
average_reward_test                  5.241292
round_time_test        0 days 00:00:14.247985
round_time_total       0 days 00:05:50.097329
loss_total                     1501306.163564
loss_critic                    1876864.519408
loss_actor                        -927.142964
memory_size                       425587.7935 

=== epoch 8/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:23,  4.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  129
episode_length                       77.20155
returns                            393.086789
return_std                         106.052493
average_reward                       5.091338
round_time             0 days 00:05:51.833155
episodes_test                           125.0
episode_length_test                    79.808
returns_test                       414.239416
return_std_test                     99.555914
average_reward_test                  5.189444
round_time_test        0 days 00:00:14.391134
round_time_total       0 days 00:05:51.834267
loss_total                        48139.80162
loss_critic                      60408.250582
loss_actor                        -933.998616
memory_size                         427160.66 

=== epoch 8/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:09,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  129
episode_length                      76.775194
returns                            389.832558
return_std                          96.487484
average_reward                       5.078521
round_time             0 days 00:05:52.139759
episodes_test                           131.0
episode_length_test                 75.969466
returns_test                       399.065509
return_std_test                     77.805457
average_reward_test                  5.252488
round_time_test        0 days 00:00:14.429038
round_time_total       0 days 00:05:52.140876
loss_total                       50000.937629
loss_critic                      62735.857282
loss_actor                        -938.744928
memory_size                       428751.7845 

=== epoch 8/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  131
episode_length                      76.236641
returns                            386.244934
return_std                          96.380793
average_reward                       5.066993
round_time             0 days 00:05:53.795495
episodes_test                           123.0
episode_length_test                 81.138211
returns_test                       416.639672
return_std_test                    123.521166
average_reward_test                  5.134415
round_time_test        0 days 00:00:14.432523
round_time_total       0 days 00:05:53.796594
loss_total                      1431523.48896
loss_critic                    1789634.613756
loss_actor                        -921.054389
memory_size                        430284.324 

=== epoch 8/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  131
episode_length                       75.48855
returns                            384.179121
return_std                          95.177346
average_reward                       5.090964
round_time             0 days 00:05:53.515299
episodes_test                           137.0
episode_length_test                 72.671533
returns_test                       380.407385
return_std_test                     73.145694
average_reward_test                  5.232172
round_time_test        0 days 00:00:14.323183
round_time_total       0 days 00:05:53.516396
loss_total                      1005401.18397
loss_critic                    1256979.826751
loss_actor                        -913.415834
memory_size                       431813.3365 

=== epoch 8/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  130
episode_length                      76.346154
returns                            388.341303
return_std                          99.879811
average_reward                       5.083495
round_time             0 days 00:05:56.769428
episodes_test                           132.0
episode_length_test                 75.643939
returns_test                       396.072454
return_std_test                     91.481477
average_reward_test                  5.235528
round_time_test        0 days 00:00:14.510923
round_time_total       0 days 00:05:56.770979
loss_total                       44626.135864
loss_critic                      56013.333534
loss_actor                        -922.658787
memory_size                       433375.0905 

=== epoch 8/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:19,  4.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  132
episode_length                      74.984848
returns                            382.367503
return_std                          99.253208
average_reward                       5.093603
round_time             0 days 00:05:58.183073
episodes_test                           127.0
episode_length_test                 78.685039
returns_test                       412.119198
return_std_test                    115.807215
average_reward_test                  5.237343
round_time_test        0 days 00:00:14.505915
round_time_total       0 days 00:05:58.184379
loss_total                       45007.858456
loss_critic                      56491.852769
loss_actor                        -928.122574
memory_size                       434912.8265 

=== epoch 8/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  132
episode_length                      75.159091
returns                            385.044739
return_std                         101.928639
average_reward                       5.124195
round_time             0 days 00:05:56.878073
episodes_test                           128.0
episode_length_test                  77.65625
returns_test                       401.184379
return_std_test                    102.337309
average_reward_test                  5.167626
round_time_test        0 days 00:00:14.502037
round_time_total       0 days 00:05:56.879412
loss_total                       66735.543712
loss_critic                      83653.024736
loss_actor                        -934.385453
memory_size                        436472.416 

=== epoch 8/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  129
episode_length                      76.472868
returns                            392.746909
return_std                         101.984108
average_reward                       5.131414
round_time             0 days 00:05:55.103956
episodes_test                           129.0
episode_length_test                  77.24031
returns_test                       407.994743
return_std_test                     84.900246
average_reward_test                  5.281212
round_time_test        0 days 00:00:14.460639
round_time_total       0 days 00:05:55.105079
loss_total                      430561.995134
loss_critic                     538434.493018
loss_actor                        -927.962047
memory_size                       438042.3245 

=== epoch 8/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  130
episode_length                           76.4
returns                            391.667188
return_std                          98.861682
average_reward                       5.123209
round_time             0 days 00:05:54.473716
episodes_test                           124.0
episode_length_test                 79.548387
returns_test                       417.632938
return_std_test                     82.440113
average_reward_test                  5.245385
round_time_test        0 days 00:00:14.240075
round_time_total       0 days 00:05:54.474826
loss_total                       379248.10018
loss_critic                     474292.325199
loss_actor                         -928.90446
memory_size                        439587.957 

=== epoch 8/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  128
episode_length                      78.007812
returns                            400.186873
return_std                          107.04571
average_reward                       5.130306
round_time             0 days 00:05:57.130115
episodes_test                           127.0
episode_length_test                 78.645669
returns_test                       417.421091
return_std_test                    102.720635
average_reward_test                  5.307068
round_time_test        0 days 00:00:14.355659
round_time_total       0 days 00:05:57.131247
loss_total                      326393.419254
loss_critic                     408221.895051
loss_actor                        -920.520385
memory_size                        441161.702 

=== epoch 8/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  125
episode_length                         79.192
returns                            405.020485
return_std                         109.974486
average_reward                       5.115877
round_time             0 days 00:05:58.153435
episodes_test                           127.0
episode_length_test                 78.338583
returns_test                        405.88327
return_std_test                     89.146788
average_reward_test                   5.18085
round_time_test        0 days 00:00:14.469885
round_time_total       0 days 00:05:58.154556
loss_total                       43832.487411
loss_critic                       55023.60694
loss_actor                        -931.994435
memory_size                       442748.7745 

=== epoch 8/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  128
episode_length                      77.320312
returns                            394.220695
return_std                         108.029988
average_reward                       5.100974
round_time             0 days 00:05:57.123470
episodes_test                           131.0
episode_length_test                 76.335878
returns_test                       400.156364
return_std_test                     81.717507
average_reward_test                  5.242048
round_time_test        0 days 00:00:14.106341
round_time_total       0 days 00:05:57.124631
loss_total                      197365.438536
loss_critic                     246938.217363
loss_actor                        -925.689206
memory_size                        444296.824 

=== epoch 8/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  127
episode_length                      78.220472
returns                            396.586569
return_std                         108.366421
average_reward                       5.071458
round_time             0 days 00:05:56.644629
episodes_test                           129.0
episode_length_test                 77.217054
returns_test                        398.05163
return_std_test                    104.389963
average_reward_test                  5.153856
round_time_test        0 days 00:00:14.340053
round_time_total       0 days 00:05:56.645720
loss_total                       44328.007482
loss_critic                      55638.663439
loss_actor                        -914.620035
memory_size                       445855.1555 

=== epoch 8/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  125
episode_length                          79.12
returns                            402.293849
return_std                           111.6071
average_reward                       5.078928
round_time             0 days 00:05:58.107355
episodes_test                           125.0
episode_length_test                    79.392
returns_test                       412.759652
return_std_test                    112.459722
average_reward_test                  5.199671
round_time_test        0 days 00:00:14.413953
round_time_total       0 days 00:05:58.108464
loss_total                       44956.346888
loss_critic                      56425.845983
loss_actor                        -921.652997
memory_size                        447427.155 

=== epoch 8/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:24,  4.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  128
episode_length                      77.273438
returns                            391.071851
return_std                         101.408178
average_reward                       5.060949
round_time             0 days 00:05:58.443847
episodes_test                           127.0
episode_length_test                 78.496063
returns_test                       409.405929
return_std_test                     110.00179
average_reward_test                  5.214894
round_time_test        0 days 00:00:14.524445
round_time_total       0 days 00:05:58.445202
loss_total                       47628.583494
loss_critic                      59769.850633
loss_actor                        -936.489178
memory_size                        448988.521 

=== epoch 8/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  133
episode_length                      74.744361
returns                            380.616684
return_std                           97.81307
average_reward                       5.093086
round_time             0 days 00:06:02.226411
episodes_test                           129.0
episode_length_test                 77.410853
returns_test                       399.948014
return_std_test                     76.421643
average_reward_test                  5.166213
round_time_test        0 days 00:00:14.702488
round_time_total       0 days 00:06:02.227520
loss_total                       48607.957281
loss_critic                      60994.990499
loss_actor                        -940.179583
memory_size                       450503.7405 

=== epoch 8/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  130
episode_length                      76.292308
returns                            388.381148
return_std                          100.43844
average_reward                       5.092813
round_time             0 days 00:05:58.009050
episodes_test                           126.0
episode_length_test                 78.912698
returns_test                       408.626045
return_std_test                    104.287133
average_reward_test                  5.177025
round_time_test        0 days 00:00:14.848423
round_time_total       0 days 00:05:58.010534
loss_total                       46595.487687
loss_critic                      58479.388861
loss_actor                        -940.121402
memory_size                        452055.391 

=== epoch 8/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  134
episode_length                      74.365672
returns                            380.781072
return_std                           103.0447
average_reward                       5.121345
round_time             0 days 00:05:56.009009
episodes_test                           132.0
episode_length_test                 75.409091
returns_test                       395.127239
return_std_test                     90.594835
average_reward_test                  5.237327
round_time_test        0 days 00:00:14.195978
round_time_total       0 days 00:05:56.010119
loss_total                       47914.670001
loss_critic                       60129.41702
loss_actor                        -944.321584
memory_size                       453612.5325 

=== epoch 8/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  135
episode_length                       73.62963
returns                            377.732795
return_std                          104.40819
average_reward                         5.1277
round_time             0 days 00:05:56.123803
episodes_test                           136.0
episode_length_test                 73.139706
returns_test                       386.190703
return_std_test                     73.734275
average_reward_test                  5.279421
round_time_test        0 days 00:00:14.026959
round_time_total       0 days 00:05:56.125220
loss_total                      110017.430746
loss_critic                     137758.638046
loss_actor                        -947.399026
memory_size                        455147.234 

=== epoch 8/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  134
episode_length                      74.179104
returns                            382.973912
return_std                         106.889758
average_reward                       5.162308
round_time             0 days 00:05:56.789316
episodes_test                           132.0
episode_length_test                 75.545455
returns_test                       397.442488
return_std_test                     66.233252
average_reward_test                  5.260283
round_time_test        0 days 00:00:14.200715
round_time_total       0 days 00:05:56.790518
loss_total                       49771.034608
loss_critic                      62448.994933
loss_actor                        -940.810911
memory_size                        456694.728 

=== epoch 8/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  132
episode_length                      75.083333
returns                            387.852485
return_std                         105.368384
average_reward                       5.162246
round_time             0 days 00:05:59.822256
episodes_test                           126.0
episode_length_test                  78.81746
returns_test                       411.192279
return_std_test                     82.836346
average_reward_test                  5.217335
round_time_test        0 days 00:00:14.252331
round_time_total       0 days 00:05:59.823624
loss_total                       50558.018327
loss_critic                      63434.595909
loss_actor                        -948.296751
memory_size                        458233.892 

=== epoch 8/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  131
episode_length                      75.618321
returns                            390.715546
return_std                         107.643899
average_reward                       5.168016
round_time             0 days 00:05:58.719950
episodes_test                           118.0
episode_length_test                 84.050847
returns_test                       433.955464
return_std_test                     98.609713
average_reward_test                  5.162909
round_time_test        0 days 00:00:14.551099
round_time_total       0 days 00:05:58.721209
loss_total                       50625.460349
loss_critic                      63518.314124
loss_actor                          -945.9589
memory_size                        459804.965 

=== epoch 8/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  126
episode_length                       78.68254
returns                            404.988646
return_std                         124.519857
average_reward                       5.147822
round_time             0 days 00:05:58.140928
episodes_test                           121.0
episode_length_test                 82.628099
returns_test                       425.481511
return_std_test                    110.647175
average_reward_test                  5.149308
round_time_test        0 days 00:00:14.551421
round_time_total       0 days 00:05:58.142047
loss_total                       43446.329364
loss_critic                      54542.764535
loss_actor                        -939.415068
memory_size                       461387.5385 

=== epoch 8/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:27,  4.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  126
episode_length                      79.071429
returns                            406.466976
return_std                         131.154376
average_reward                       5.141519
round_time             0 days 00:06:00.516811
episodes_test                           120.0
episode_length_test                      83.1
returns_test                       427.475137
return_std_test                    105.903979
average_reward_test                  5.143704
round_time_test        0 days 00:00:14.358542
round_time_total       0 days 00:06:00.518205
loss_total                       44135.248499
loss_critic                      55403.925815
loss_actor                        -939.464808
memory_size                       462992.6425 

=== epoch 8/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  124
episode_length                      79.895161
returns                            409.528306
return_std                         136.202018
average_reward                       5.128048
round_time             0 days 00:05:56.261485
episodes_test                           115.0
episode_length_test                 85.956522
returns_test                       445.276668
return_std_test                    100.284806
average_reward_test                   5.17994
round_time_test        0 days 00:00:14.383404
round_time_total       0 days 00:05:56.263014
loss_total                       43201.083751
loss_critic                      54235.812821
loss_actor                        -937.836352
memory_size                       464563.5855 

=== epoch 8/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.54it/s]
episodes                                  123
episode_length                      80.658537
returns                            412.630682
return_std                         140.909954
average_reward                       5.117623
round_time             0 days 00:06:01.761894
episodes_test                           123.0
episode_length_test                 80.829268
returns_test                       416.167497
return_std_test                     118.04308
average_reward_test                  5.148887
round_time_test        0 days 00:00:14.571181
round_time_total       0 days 00:06:01.763128
loss_total                      127994.836625
loss_critic                     160228.218087
loss_actor                        -938.710941
memory_size                        466113.023 

=== epoch 8/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  126
episode_length                      79.269841
returns                            406.550035
return_std                         139.269888
average_reward                       5.129333
round_time             0 days 00:05:57.059449
episodes_test                           127.0
episode_length_test                  78.23622
returns_test                       409.549757
return_std_test                    100.013225
average_reward_test                  5.234145
round_time_test        0 days 00:00:13.939567
round_time_total       0 days 00:05:57.060547
loss_total                       42415.461194
loss_critic                      53254.978811
loss_actor                        -942.612924
memory_size                       467665.8805 

=== epoch 8/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  127
episode_length                      78.338583
returns                            401.509562
return_std                         118.354068
average_reward                       5.125287
round_time             0 days 00:05:58.233420
episodes_test                           132.0
episode_length_test                 75.659091
returns_test                       394.122224
return_std_test                     93.133064
average_reward_test                  5.208825
round_time_test        0 days 00:00:14.304114
round_time_total       0 days 00:05:58.234529
loss_total                       41577.026629
loss_critic                      52207.619551
loss_actor                         -945.34871
memory_size                         469239.92 

=== epoch 8/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  127
episode_length                      77.677165
returns                            399.554345
return_std                         103.290053
average_reward                       5.136935
round_time             0 days 00:06:00.038056
episodes_test                           136.0
episode_length_test                 73.183824
returns_test                       384.871381
return_std_test                     83.667672
average_reward_test                   5.25829
round_time_test        0 days 00:00:14.375680
round_time_total       0 days 00:06:00.039150
loss_total                      326762.831016
loss_critic                     408688.905471
loss_actor                        -941.525356
memory_size                        470802.408 

=== epoch 8/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  128
episode_length                        77.5625
returns                             398.11042
return_std                          96.632104
average_reward                       5.133223
round_time             0 days 00:05:57.362302
episodes_test                           133.0
episode_length_test                 75.165414
returns_test                       398.245007
return_std_test                     84.031758
average_reward_test                  5.298127
round_time_test        0 days 00:00:14.711087
round_time_total       0 days 00:05:57.363386
loss_total                     1365454.355153
loss_critic                    1707048.842249
loss_actor                        -923.767452
memory_size                       472362.1675 

=== epoch 8/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  129
episode_length                      77.093023
returns                            395.620687
return_std                          90.916731
average_reward                       5.130519
round_time             0 days 00:05:58.922902
episodes_test                           125.0
episode_length_test                    79.712
returns_test                       411.332582
return_std_test                     90.823973
average_reward_test                  5.159269
round_time_test        0 days 00:00:14.206876
round_time_total       0 days 00:05:58.924011
loss_total                       40055.847509
loss_critic                       50302.24449
loss_actor                        -929.744153
memory_size                         473913.92 

=== epoch 8/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:08,  4.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  129
episode_length                       76.55814
returns                            391.780743
return_std                          91.461171
average_reward                       5.116231
round_time             0 days 00:05:56.402386
episodes_test                           115.0
episode_length_test                 86.704348
returns_test                       442.567706
return_std_test                    144.068055
average_reward_test                  5.103961
round_time_test        0 days 00:00:14.752690
round_time_total       0 days 00:05:56.403514
loss_total                        92657.30764
loss_critic                     116058.276715
loss_actor                         -946.58364
memory_size                        475450.211 

=== epoch 8/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  132
episode_length                      75.272727
returns                            385.898598
return_std                         101.281426
average_reward                       5.127505
round_time             0 days 00:05:57.869465
episodes_test                           129.0
episode_length_test                 77.457364
returns_test                        401.95718
return_std_test                     94.946656
average_reward_test                  5.189189
round_time_test        0 days 00:00:14.420308
round_time_total       0 days 00:05:57.870730
loss_total                      577285.294778
loss_critic                     721843.244729
loss_actor                        -946.502297
memory_size                       477004.5955 

=== epoch 8/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  128
episode_length                      77.023438
returns                            393.788291
return_std                          111.39815
average_reward                       5.113348
round_time             0 days 00:05:58.118144
episodes_test                           114.0
episode_length_test                 87.631579
returns_test                       445.112446
return_std_test                    106.571585
average_reward_test                  5.079177
round_time_test        0 days 00:00:14.684499
round_time_total       0 days 00:05:58.119235
loss_total                       93070.876216
loss_critic                     116575.600191
loss_actor                        -948.030872
memory_size                       478586.9285 

=== epoch 8/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  130
episode_length                      76.338462
returns                            390.805875
return_std                         115.799636
average_reward                       5.119359
round_time             0 days 00:05:56.391073
episodes_test                           115.0
episode_length_test                 86.530435
returns_test                       441.877296
return_std_test                    109.745687
average_reward_test                  5.105796
round_time_test        0 days 00:00:14.692270
round_time_total       0 days 00:05:56.392264
loss_total                      483979.370213
loss_critic                     605210.980705
loss_actor                        -947.208454
memory_size                       480151.7765 

=== epoch 8/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  129
episode_length                      77.054264
returns                            392.913975
return_std                         117.448982
average_reward                       5.098619
round_time             0 days 00:05:57.662607
episodes_test                           109.0
episode_length_test                 91.422018
returns_test                       457.880989
return_std_test                    133.422111
average_reward_test                  5.008082
round_time_test        0 days 00:00:14.665854
round_time_total       0 days 00:05:57.664040
loss_total                      259133.543717
loss_critic                     324151.399901
loss_actor                        -937.871028
memory_size                         481709.19 

=== epoch 8/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  129
episode_length                       77.03876
returns                            392.296425
return_std                         119.020678
average_reward                       5.089164
round_time             0 days 00:06:01.290996
episodes_test                           122.0
episode_length_test                 81.409836
returns_test                       416.325367
return_std_test                    106.397587
average_reward_test                  5.112269
round_time_test        0 days 00:00:14.922199
round_time_total       0 days 00:06:01.292114
loss_total                      155938.151379
loss_critic                     195160.133019
loss_actor                        -949.776842
memory_size                        483259.608 

=== epoch 8/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:03,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  127
episode_length                      77.992126
returns                            394.893994
return_std                         114.539046
average_reward                       5.064072
round_time             0 days 00:05:57.013335
episodes_test                           119.0
episode_length_test                 83.655462
returns_test                        433.30202
return_std_test                      86.17684
average_reward_test                  5.177563
round_time_test        0 days 00:00:14.798304
round_time_total       0 days 00:05:57.014796
loss_total                       43186.179607
loss_critic                       54217.64812
loss_actor                        -939.697962
memory_size                        484805.368 

=== epoch 8/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:04,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  129
episode_length                      76.751938
returns                            388.390346
return_std                         105.374187
average_reward                       5.061966
round_time             0 days 00:05:58.947209
episodes_test                           122.0
episode_length_test                 81.795082
returns_test                       425.686475
return_std_test                    105.864067
average_reward_test                  5.203557
round_time_test        0 days 00:00:14.458498
round_time_total       0 days 00:05:58.948352
loss_total                       43546.233815
loss_critic                      54667.976628
loss_actor                        -940.740934
memory_size                       486386.0375 

=== epoch 8/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:25,  4.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  129
episode_length                      76.674419
returns                             386.50034
return_std                           99.91833
average_reward                       5.043447
round_time             0 days 00:05:58.065659
episodes_test                           122.0
episode_length_test                 81.770492
returns_test                       428.106583
return_std_test                    113.487054
average_reward_test                  5.234746
round_time_test        0 days 00:00:14.339205
round_time_total       0 days 00:05:58.066757
loss_total                       45013.437385
loss_critic                      56502.799521
loss_actor                        -944.015832
memory_size                        487932.197 

=== epoch 8/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:45,  4.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  131
episode_length                      75.763359
returns                            382.750803
return_std                          96.283108
average_reward                       5.054248
round_time             0 days 00:05:58.734576
episodes_test                           122.0
episode_length_test                 81.606557
returns_test                       429.686529
return_std_test                    102.806452
average_reward_test                  5.263794
round_time_test        0 days 00:00:14.387473
round_time_total       0 days 00:05:58.735701
loss_total                       722554.37718
loss_critic                     903428.932632
loss_actor                        -943.955496
memory_size                        489468.505 

=== epoch 8/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  128
episode_length                      77.609375
returns                            392.133994
return_std                         104.367114
average_reward                       5.054047
round_time             0 days 00:06:00.250817
episodes_test                           126.0
episode_length_test                 78.730159
returns_test                       408.001834
return_std_test                     91.581829
average_reward_test                  5.180933
round_time_test        0 days 00:00:14.588822
round_time_total       0 days 00:06:00.251923
loss_total                       56577.456521
loss_critic                      70955.426527
loss_actor                        -934.428313
memory_size                       491028.5535 

=== epoch 8/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  129
episode_length                      77.310078
returns                            389.869805
return_std                         103.591813
average_reward                       5.043946
round_time             0 days 00:05:56.899764
episodes_test                           129.0
episode_length_test                 76.976744
returns_test                       400.285823
return_std_test                     85.161098
average_reward_test                  5.200252
round_time_test        0 days 00:00:14.376899
round_time_total       0 days 00:05:56.900851
loss_total                       46992.894973
loss_critic                      58978.437859
loss_actor                        -949.281251
memory_size                       492614.6715 

=== epoch 8/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  130
episode_length                      76.353846
returns                            384.510259
return_std                          98.949925
average_reward                       5.035758
round_time             0 days 00:05:58.264814
episodes_test                           129.0
episode_length_test                 77.217054
returns_test                       397.636453
return_std_test                     95.797983
average_reward_test                  5.148858
round_time_test        0 days 00:00:14.208379
round_time_total       0 days 00:05:58.266206
loss_total                       45793.460762
loss_critic                      57481.333615
loss_actor                        -958.034395
memory_size                        494173.071 

=== epoch 8/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  130
episode_length                      76.753846
returns                            386.677365
return_std                         102.854532
average_reward                       5.038644
round_time             0 days 00:05:57.464259
episodes_test                           109.0
episode_length_test                 91.385321
returns_test                       465.276016
return_std_test                    125.656792
average_reward_test                  5.091435
round_time_test        0 days 00:00:14.968772
round_time_total       0 days 00:05:57.465383
loss_total                       45597.832187
loss_critic                       57234.76772
loss_actor                         -949.91399
memory_size                        495709.495 

=== epoch 8/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:05,  4.68it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  126
episode_length                      78.825397
returns                            397.507399
return_std                          115.06239
average_reward                       5.045546
round_time             0 days 00:05:57.847974
episodes_test                           115.0
episode_length_test                 86.286957
returns_test                       439.261592
return_std_test                     88.418948
average_reward_test                  5.092144
round_time_test        0 days 00:00:14.856099
round_time_total       0 days 00:05:57.849319
loss_total                      165481.461832
loss_critic                      207089.07212
loss_actor                        -949.011117
memory_size                        497269.283 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 9/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:35,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                   24
episode_length                      80.791667
returns                            413.628839
return_std                         148.455907
average_reward                       5.108218
round_time             0 days 00:05:44.637168
episodes_test                           118.0
episode_length_test                 84.694915
returns_test                        443.91545
return_std_test                     85.897983
average_reward_test                  5.241143
round_time_test        0 days 00:00:14.246125
round_time_total       0 days 00:05:44.638280
loss_total                      504129.943771
loss_critic                     630400.620489
loss_actor                        -952.891441
memory_size                       498879.4215 

=== epoch 9/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                   51
episode_length                      77.352941
returns                            394.458986
return_std                         120.494395
average_reward                         5.1022
round_time             0 days 00:05:47.394521
episodes_test                           126.0
episode_length_test                 79.071429
returns_test                       412.834547
return_std_test                    107.608964
average_reward_test                  5.220148
round_time_test        0 days 00:00:14.488255
round_time_total       0 days 00:05:47.395824
loss_total                      112968.772243
loss_critic                     141446.797549
loss_actor                        -943.343811
memory_size                       500420.6765 

=== epoch 9/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                   79
episode_length                      75.708861
returns                            389.298797
return_std                         113.835276
average_reward                       5.141222
round_time             0 days 00:05:52.342356
episodes_test                           117.0
episode_length_test                 84.777778
returns_test                       434.907732
return_std_test                    108.529552
average_reward_test                   5.13072
round_time_test        0 days 00:00:14.662194
round_time_total       0 days 00:05:52.343577
loss_total                      181772.642669
loss_critic                     227451.021473
loss_actor                        -940.882146
memory_size                        501957.843 

=== epoch 9/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  104
episode_length                      76.711538
returns                            389.616978
return_std                         113.685064
average_reward                       5.078498
round_time             0 days 00:05:51.769030
episodes_test                           111.0
episode_length_test                 89.855856
returns_test                       452.565738
return_std_test                    132.533599
average_reward_test                  5.035941
round_time_test        0 days 00:00:14.733874
round_time_total       0 days 00:05:51.770131
loss_total                      226984.603901
loss_critic                     283969.252758
loss_actor                        -954.042013
memory_size                       503506.9775 

=== epoch 9/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  130
episode_length                      76.915385
returns                            388.257898
return_std                         107.229782
average_reward                       5.047842
round_time             0 days 00:05:54.964310
episodes_test                           122.0
episode_length_test                 81.680328
returns_test                        417.59302
return_std_test                     84.818819
average_reward_test                  5.112111
round_time_test        0 days 00:00:14.621338
round_time_total       0 days 00:05:54.965588
loss_total                      742070.889851
loss_critic                     927825.020034
loss_actor                        -945.809865
memory_size                        505083.635 

=== epoch 9/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  128
episode_length                           77.5
returns                            390.429168
return_std                          94.954688
average_reward                       5.034537
round_time             0 days 00:05:53.193810
episodes_test                           113.0
episode_length_test                 88.168142
returns_test                       450.867929
return_std_test                     95.613646
average_reward_test                  5.113384
round_time_test        0 days 00:00:14.985994
round_time_total       0 days 00:05:53.194936
loss_total                       45697.200553
loss_critic                      57355.114072
loss_actor                        -934.457041
memory_size                       506664.6955 

=== epoch 9/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  124
episode_length                      80.185484
returns                            403.574002
return_std                         109.941888
average_reward                         5.0322
round_time             0 days 00:05:50.792950
episodes_test                           121.0
episode_length_test                 82.280992
returns_test                       427.924096
return_std_test                    102.624671
average_reward_test                  5.200007
round_time_test        0 days 00:00:14.959614
round_time_total       0 days 00:05:50.794061
loss_total                       47636.356883
loss_critic                      59782.602353
loss_actor                        -948.629341
memory_size                       508268.2385 

=== epoch 9/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  121
episode_length                      82.264463
returns                            410.483739
return_std                          105.54227
average_reward                       4.990254
round_time             0 days 00:05:54.317564
episodes_test                           117.0
episode_length_test                 85.239316
returns_test                       438.177595
return_std_test                     87.655969
average_reward_test                  5.140082
round_time_test        0 days 00:00:14.906674
round_time_total       0 days 00:05:54.319046
loss_total                       398282.15059
loss_critic                     498087.184178
loss_actor                        -938.105911
memory_size                        509862.741 

=== epoch 9/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  120
episode_length                         82.425
returns                            412.745519
return_std                         107.126654
average_reward                       5.009189
round_time             0 days 00:05:57.957976
episodes_test                           114.0
episode_length_test                 87.614035
returns_test                       438.327566
return_std_test                    114.163251
average_reward_test                  5.002778
round_time_test        0 days 00:00:14.668039
round_time_total       0 days 00:05:57.959296
loss_total                       49572.598777
loss_critic                      62202.262514
loss_actor                        -946.060167
memory_size                       511442.8595 

=== epoch 9/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:27,  4.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  119
episode_length                      82.991597
returns                            419.707621
return_std                         112.476046
average_reward                       5.052561
round_time             0 days 00:05:54.324122
episodes_test                           116.0
episode_length_test                 85.534483
returns_test                       435.167591
return_std_test                     110.23709
average_reward_test                  5.086381
round_time_test        0 days 00:00:14.719196
round_time_total       0 days 00:05:54.325249
loss_total                       60067.032736
loss_critic                      75322.890392
loss_actor                         -956.40245
memory_size                       513027.0345 

=== epoch 9/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  123
episode_length                      80.869919
returns                            410.486705
return_std                         112.244141
average_reward                       5.074207
round_time             0 days 00:05:58.017034
episodes_test                           119.0
episode_length_test                 83.705882
returns_test                       431.439552
return_std_test                     99.239073
average_reward_test                   5.15296
round_time_test        0 days 00:00:14.597273
round_time_total       0 days 00:05:58.018435
loss_total                       53460.641995
loss_critic                      67066.156819
loss_actor                        -961.421474
memory_size                       514580.0025 

=== epoch 9/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  126
episode_length                       78.34127
returns                            398.529696
return_std                          98.645308
average_reward                       5.083739
round_time             0 days 00:05:55.446750
episodes_test                           112.0
episode_length_test                 89.214286
returns_test                       450.681168
return_std_test                    106.116482
average_reward_test                  5.051559
round_time_test        0 days 00:00:14.923412
round_time_total       0 days 00:05:55.447956
loss_total                       46333.068877
loss_critic                      58156.621388
loss_actor                        -961.145521
memory_size                        516132.707 

=== epoch 9/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  127
episode_length                      77.952756
returns                            399.250803
return_std                          98.828244
average_reward                       5.119852
round_time             0 days 00:05:56.319833
episodes_test                           123.0
episode_length_test                  80.95935
returns_test                       413.633801
return_std_test                      96.70049
average_reward_test                  5.109316
round_time_test        0 days 00:00:14.852990
round_time_total       0 days 00:05:56.320967
loss_total                      365371.708439
loss_critic                     456954.199444
loss_actor                        -958.273999
memory_size                       517685.8025 

=== epoch 9/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:19,  4.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  129
episode_length                      77.387597
returns                             397.47189
return_std                          91.326802
average_reward                       5.137042
round_time             0 days 00:05:53.374483
episodes_test                           106.0
episode_length_test                 93.660377
returns_test                       466.514896
return_std_test                    151.388694
average_reward_test                  4.980038
round_time_test        0 days 00:00:14.774927
round_time_total       0 days 00:05:53.375579
loss_total                      936621.744324
loss_critic                    1171008.104338
loss_actor                        -923.863885
memory_size                        519253.635 

=== epoch 9/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:23,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  130
episode_length                      76.792308
returns                            394.566673
return_std                          88.683398
average_reward                       5.139456
round_time             0 days 00:05:56.142062
episodes_test                           124.0
episode_length_test                 80.532258
returns_test                       407.850184
return_std_test                     78.355317
average_reward_test                  5.064249
round_time_test        0 days 00:00:14.875783
round_time_total       0 days 00:05:56.143185
loss_total                       56866.756731
loss_critic                      71319.666762
loss_actor                        -944.887509
memory_size                       520810.2195 

=== epoch 9/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:48,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  125
episode_length                         78.736
returns                            403.221793
return_std                         106.746147
average_reward                       5.117307
round_time             0 days 00:05:56.664670
episodes_test                           121.0
episode_length_test                  82.14876
returns_test                         417.6251
return_std_test                     98.230716
average_reward_test                  5.082964
round_time_test        0 days 00:00:14.794595
round_time_total       0 days 00:05:56.665771
loss_total                       66723.314874
loss_critic                      83643.462529
loss_actor                        -957.280665
memory_size                       522399.0725 

=== epoch 9/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:26,  4.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  125
episode_length                         79.272
returns                            405.480492
return_std                         105.979303
average_reward                       5.112761
round_time             0 days 00:05:58.567025
episodes_test                           119.0
episode_length_test                  83.97479
returns_test                       431.701798
return_std_test                     94.345672
average_reward_test                  5.140677
round_time_test        0 days 00:00:14.949464
round_time_total       0 days 00:05:58.568173
loss_total                       71451.384381
loss_critic                      89555.410551
loss_actor                        -964.723725
memory_size                       523985.6155 

=== epoch 9/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  124
episode_length                      80.129032
returns                            409.729266
return_std                         116.757482
average_reward                       5.108736
round_time             0 days 00:05:57.705683
episodes_test                           120.0
episode_length_test                 83.041667
returns_test                       427.075323
return_std_test                    110.950727
average_reward_test                  5.142782
round_time_test        0 days 00:00:15.020622
round_time_total       0 days 00:05:57.706957
loss_total                      189033.393409
loss_critic                       236532.5395
loss_actor                        -963.201254
memory_size                        525576.502 

=== epoch 9/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  122
episode_length                      80.827869
returns                             413.32043
return_std                         114.684082
average_reward                       5.115264
round_time             0 days 00:05:56.527250
episodes_test                           132.0
episode_length_test                 75.416667
returns_test                       393.478328
return_std_test                      92.98182
average_reward_test                  5.217524
round_time_test        0 days 00:00:14.706343
round_time_total       0 days 00:05:56.528367
loss_total                      160298.386451
loss_critic                     200611.543343
loss_actor                        -954.258156
memory_size                       527143.6395 

=== epoch 9/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  123
episode_length                      81.243902
returns                            413.400148
return_std                         126.640197
average_reward                       5.088225
round_time             0 days 00:05:55.000540
episodes_test                           118.0
episode_length_test                 84.033898
returns_test                        424.14356
return_std_test                    103.337287
average_reward_test                   5.04679
round_time_test        0 days 00:00:14.713562
round_time_total       0 days 00:05:55.001642
loss_total                       83133.234175
loss_critic                     104155.959136
loss_actor                        -957.673204
memory_size                       528713.6735 

=== epoch 9/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  125
episode_length                         79.792
returns                            407.090277
return_std                         123.171479
average_reward                       5.101365
round_time             0 days 00:05:58.358095
episodes_test                           116.0
episode_length_test                 85.603448
returns_test                       439.403979
return_std_test                    124.386959
average_reward_test                  5.131622
round_time_test        0 days 00:00:14.615852
round_time_total       0 days 00:05:58.359192
loss_total                      987415.297569
loss_critic                    1234508.542525
loss_actor                        -957.925341
memory_size                       530293.3015 

=== epoch 9/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:34,  4.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  126
episode_length                      78.873016
returns                            402.317134
return_std                         127.213364
average_reward                       5.096945
round_time             0 days 00:05:57.172087
episodes_test                           129.0
episode_length_test                 76.914729
returns_test                       402.142302
return_std_test                     82.523232
average_reward_test                  5.225057
round_time_test        0 days 00:00:14.203701
round_time_total       0 days 00:05:57.173193
loss_total                      317544.775955
loss_critic                     397172.049356
loss_actor                        -964.276805
memory_size                         531849.37 

=== epoch 9/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  126
episode_length                      78.515873
returns                             396.37776
return_std                         119.550889
average_reward                       5.049627
round_time             0 days 00:06:00.166835
episodes_test                           113.0
episode_length_test                 88.362832
returns_test                       449.299383
return_std_test                    101.712477
average_reward_test                  5.084413
round_time_test        0 days 00:00:14.691232
round_time_total       0 days 00:06:00.167920
loss_total                       57750.605659
loss_critic                      72427.253056
loss_actor                        -955.986144
memory_size                        533404.987 

=== epoch 9/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  128
episode_length                      77.429688
returns                            388.928286
return_std                         120.789919
average_reward                       5.025536
round_time             0 days 00:06:02.397775
episodes_test                           116.0
episode_length_test                 85.689655
returns_test                       436.893391
return_std_test                    104.582194
average_reward_test                  5.098758
round_time_test        0 days 00:00:14.382670
round_time_total       0 days 00:06:02.399079
loss_total                       45518.078542
loss_critic                      57134.911463
loss_actor                        -949.257145
memory_size                       534966.0275 

=== epoch 9/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.56it/s]
episodes                                  126
episode_length                      77.753968
returns                            391.430414
return_std                          116.00284
average_reward                       5.026893
round_time             0 days 00:06:00.632541
episodes_test                           118.0
episode_length_test                      84.0
returns_test                       426.803183
return_std_test                     86.799714
average_reward_test                  5.078155
round_time_test        0 days 00:00:15.038037
round_time_total       0 days 00:06:00.633664
loss_total                       80553.784043
loss_critic                     100931.500977
loss_actor                        -957.094854
memory_size                       536533.0685 

=== epoch 9/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  126
episode_length                      78.595238
returns                            395.104736
return_std                         107.809278
average_reward                       5.028836
round_time             0 days 00:05:59.628047
episodes_test                           121.0
episode_length_test                 82.636364
returns_test                       420.627223
return_std_test                      91.95395
average_reward_test                  5.090081
round_time_test        0 days 00:00:14.973697
round_time_total       0 days 00:05:59.629158
loss_total                      454383.335717
loss_critic                     568216.834778
loss_actor                        -950.761054
memory_size                        538123.717 

=== epoch 9/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  124
episode_length                      79.862903
returns                            401.544255
return_std                         107.850449
average_reward                       5.029191
round_time             0 days 00:05:57.605151
episodes_test                           111.0
episode_length_test                 89.441441
returns_test                       454.543996
return_std_test                    109.659876
average_reward_test                  5.081168
round_time_test        0 days 00:00:14.823656
round_time_total       0 days 00:05:57.606254
loss_total                       43947.067621
loss_critic                       55171.70382
loss_actor                        -951.480769
memory_size                        539681.088 

=== epoch 9/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  124
episode_length                          79.75
returns                            404.219768
return_std                         107.947732
average_reward                       5.063877
round_time             0 days 00:05:58.635606
episodes_test                           120.0
episode_length_test                 83.016667
returns_test                       426.488476
return_std_test                    100.051166
average_reward_test                  5.137019
round_time_test        0 days 00:00:14.619056
round_time_total       0 days 00:05:58.636917
loss_total                      129091.748085
loss_critic                     161601.851937
loss_actor                        -948.667084
memory_size                       541274.7455 

=== epoch 9/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  125
episode_length                         79.696
returns                              405.5437
return_std                         111.566977
average_reward                       5.088476
round_time             0 days 00:05:58.542106
episodes_test                           117.0
episode_length_test                 84.709402
returns_test                       439.031892
return_std_test                     97.049289
average_reward_test                  5.183817
round_time_test        0 days 00:00:14.827881
round_time_total       0 days 00:05:58.543477
loss_total                      656086.128497
loss_critic                     820342.793679
loss_actor                           -940.787
memory_size                        542833.706 

=== epoch 9/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:02<00:00,  5.52it/s]
episodes                                  128
episode_length                      77.742188
returns                            396.913058
return_std                         109.891267
average_reward                        5.10509
round_time             0 days 00:06:03.077081
episodes_test                           115.0
episode_length_test                 86.904348
returns_test                       448.960995
return_std_test                     95.860779
average_reward_test                  5.165989
round_time_test        0 days 00:00:14.634525
round_time_total       0 days 00:06:03.078305
loss_total                       43109.162593
loss_critic                      54118.476173
loss_actor                        -928.094923
memory_size                       544368.3225 

=== epoch 9/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  129
episode_length                      77.108527
returns                            392.073473
return_std                          106.86659
average_reward                       5.085894
round_time             0 days 00:06:01.341433
episodes_test                           106.0
episode_length_test                 92.915094
returns_test                       472.349602
return_std_test                    121.600826
average_reward_test                  5.079766
round_time_test        0 days 00:00:14.897846
round_time_total       0 days 00:06:01.342879
loss_total                      484998.869438
loss_critic                     606479.320828
loss_actor                        -922.926819
memory_size                       545929.1105 

=== epoch 9/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  127
episode_length                      78.259843
returns                            397.650129
return_std                         110.522389
average_reward                        5.08242
round_time             0 days 00:06:01.517349
episodes_test                           113.0
episode_length_test                 88.159292
returns_test                       446.649161
return_std_test                    106.018179
average_reward_test                  5.065872
round_time_test        0 days 00:00:14.676546
round_time_total       0 days 00:06:01.518438
loss_total                      119681.748569
loss_critic                     149839.472913
loss_actor                         -949.17713
memory_size                        547506.825 

=== epoch 9/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  130
episode_length                      76.553846
returns                            389.775994
return_std                          113.15423
average_reward                       5.093281
round_time             0 days 00:05:59.333481
episodes_test                           128.0
episode_length_test                   77.8125
returns_test                       405.848415
return_std_test                     83.986178
average_reward_test                  5.215449
round_time_test        0 days 00:00:14.164173
round_time_total       0 days 00:05:59.334594
loss_total                      354137.175415
loss_critic                     442912.225922
loss_actor                        -963.098641
memory_size                       549069.4635 

=== epoch 9/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  128
episode_length                      77.757812
returns                            395.159461
return_std                         116.338805
average_reward                       5.083707
round_time             0 days 00:05:58.377174
episodes_test                           120.0
episode_length_test                      82.9
returns_test                       431.374545
return_std_test                        108.74
average_reward_test                  5.202313
round_time_test        0 days 00:00:14.234557
round_time_total       0 days 00:05:58.378270
loss_total                       46552.436201
loss_critic                      58429.241418
loss_actor                         -954.78838
memory_size                        550624.663 

=== epoch 9/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  124
episode_length                      80.016129
returns                            405.928711
return_std                          115.08232
average_reward                       5.073533
round_time             0 days 00:06:00.773851
episodes_test                           125.0
episode_length_test                    79.336
returns_test                       412.198807
return_std_test                     90.821497
average_reward_test                  5.195837
round_time_test        0 days 00:00:14.303162
round_time_total       0 days 00:06:00.775078
loss_total                      305519.215162
loss_critic                     382135.760143
loss_actor                        -947.022281
memory_size                       552208.4555 

=== epoch 9/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:03,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  121
episode_length                      81.471074
returns                            414.272765
return_std                         128.820276
average_reward                       5.080689
round_time             0 days 00:05:59.663932
episodes_test                           111.0
episode_length_test                 89.522523
returns_test                       455.760023
return_std_test                    115.162373
average_reward_test                  5.092518
round_time_test        0 days 00:00:14.639723
round_time_total       0 days 00:05:59.665126
loss_total                      849033.964196
loss_critic                    1061527.748604
loss_actor                        -941.216339
memory_size                        553824.927 

=== epoch 9/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  122
episode_length                      81.368852
returns                            414.784308
return_std                         122.747278
average_reward                       5.097823
round_time             0 days 00:05:58.669503
episodes_test                           112.0
episode_length_test                   88.6875
returns_test                       448.501546
return_std_test                    110.513577
average_reward_test                  5.057002
round_time_test        0 days 00:00:14.871639
round_time_total       0 days 00:05:58.670611
loss_total                      215588.103763
loss_critic                     269721.453261
loss_actor                        -945.326777
memory_size                        555411.657 

=== epoch 9/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  119
episode_length                      82.756303
returns                            421.822042
return_std                         122.525195
average_reward                        5.09115
round_time             0 days 00:05:58.812595
episodes_test                           110.0
episode_length_test                 90.527273
returns_test                       461.889227
return_std_test                     93.884989
average_reward_test                  5.101802
round_time_test        0 days 00:00:15.090664
round_time_total       0 days 00:05:58.813701
loss_total                       45529.532957
loss_critic                      57148.621762
loss_actor                        -946.826047
memory_size                       557014.0185 

=== epoch 9/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  120
episode_length                      82.591667
returns                            424.048532
return_std                         114.509346
average_reward                       5.136282
round_time             0 days 00:06:01.331375
episodes_test                           109.0
episode_length_test                 90.880734
returns_test                       457.239403
return_std_test                    128.677322
average_reward_test                  5.032425
round_time_test        0 days 00:00:14.637167
round_time_total       0 days 00:06:01.332465
loss_total                      133510.170545
loss_critic                     167121.489949
loss_actor                        -935.102749
memory_size                        558576.617 

=== epoch 9/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  121
episode_length                      82.099174
returns                             423.14759
return_std                         107.665819
average_reward                       5.155327
round_time             0 days 00:05:58.373517
episodes_test                           114.0
episode_length_test                 86.710526
returns_test                       443.442438
return_std_test                     94.402191
average_reward_test                  5.109638
round_time_test        0 days 00:00:14.647115
round_time_total       0 days 00:05:58.375022
loss_total                       44309.737474
loss_critic                      55623.467857
loss_actor                        -945.188529
memory_size                        560156.853 

=== epoch 9/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  123
episode_length                      81.065041
returns                            419.889086
return_std                         106.176251
average_reward                       5.179858
round_time             0 days 00:05:55.406993
episodes_test                           109.0
episode_length_test                 91.348624
returns_test                       468.360584
return_std_test                     108.05404
average_reward_test                  5.128681
round_time_test        0 days 00:00:14.649289
round_time_total       0 days 00:05:55.408276
loss_total                       43049.476108
loss_critic                       54050.93987
loss_actor                        -956.382463
memory_size                       561724.9505 

=== epoch 9/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  123
episode_length                      80.723577
returns                            418.311568
return_std                         111.862921
average_reward                       5.183982
round_time             0 days 00:06:00.357630
episodes_test                           116.0
episode_length_test                 85.362069
returns_test                       443.805443
return_std_test                    116.303974
average_reward_test                  5.192563
round_time_test        0 days 00:00:14.659319
round_time_total       0 days 00:06:00.359073
loss_total                       39755.550084
loss_critic                      49933.418712
loss_actor                        -955.927718
memory_size                        563306.811 

=== epoch 9/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  125
episode_length                         79.432
returns                            410.883153
return_std                         113.209945
average_reward                        5.17303
round_time             0 days 00:05:57.599751
episodes_test                           108.0
episode_length_test                  92.37037
returns_test                       469.901099
return_std_test                    119.452972
average_reward_test                  5.086891
round_time_test        0 days 00:00:14.704039
round_time_total       0 days 00:05:57.601111
loss_total                      977175.104784
loss_critic                    1221703.479793
loss_actor                        -938.494799
memory_size                        564884.658 

=== epoch 9/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  125
episode_length                          79.36
returns                            406.240662
return_std                          119.02734
average_reward                       5.120596
round_time             0 days 00:05:57.115568
episodes_test                           109.0
episode_length_test                 91.541284
returns_test                       470.674942
return_std_test                     90.716444
average_reward_test                  5.141142
round_time_test        0 days 00:00:14.497929
round_time_total       0 days 00:05:57.116682
loss_total                       37529.042818
loss_critic                      47148.680647
loss_actor                        -949.511692
memory_size                        566429.024 

=== epoch 9/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.54it/s]
episodes                                  127
episode_length                      78.188976
returns                            398.778313
return_std                         124.157344
average_reward                       5.101077
round_time             0 days 00:06:01.512771
episodes_test                           110.0
episode_length_test                 90.690909
returns_test                       472.655517
return_std_test                    153.773202
average_reward_test                    5.2111
round_time_test        0 days 00:00:14.393701
round_time_total       0 days 00:06:01.513863
loss_total                      221080.435813
loss_critic                     276589.407216
loss_actor                        -955.459554
memory_size                        567998.429 

=== epoch 9/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  124
episode_length                      79.766129
returns                            404.428224
return_std                          123.62036
average_reward                       5.071563
round_time             0 days 00:05:58.750214
episodes_test                           110.0
episode_length_test                 90.854545
returns_test                       458.834332
return_std_test                    115.000682
average_reward_test                  5.050095
round_time_test        0 days 00:00:14.557592
round_time_total       0 days 00:05:58.751314
loss_total                       75283.690276
loss_critic                      94344.626285
loss_actor                        -960.069644
memory_size                        569587.577 

=== epoch 9/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  127
episode_length                      78.086614
returns                            394.682445
return_std                         113.835546
average_reward                       5.055633
round_time             0 days 00:05:57.463867
episodes_test                           111.0
episode_length_test                 89.099099
returns_test                       454.290517
return_std_test                      92.32904
average_reward_test                  5.092187
round_time_test        0 days 00:00:14.440074
round_time_total       0 days 00:05:57.464979
loss_total                       95399.863873
loss_critic                     119492.499265
loss_actor                        -970.681112
memory_size                        571161.385 

=== epoch 9/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  123
episode_length                      80.601626
returns                            406.952683
return_std                         128.024386
average_reward                        5.05129
round_time             0 days 00:06:01.303409
episodes_test                           101.0
episode_length_test                 97.910891
returns_test                       491.397841
return_std_test                    156.460054
average_reward_test                  5.015511
round_time_test        0 days 00:00:14.814627
round_time_total       0 days 00:06:01.304495
loss_total                      180125.054563
loss_critic                     225400.373907
loss_actor                        -976.230593
memory_size                       572751.0135 

=== epoch 9/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:08,  4.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  121
episode_length                      81.859504
returns                            414.344853
return_std                         130.343913
average_reward                       5.060399
round_time             0 days 00:05:59.579275
episodes_test                           110.0
episode_length_test                 90.672727
returns_test                       460.347795
return_std_test                     122.53925
average_reward_test                  5.077006
round_time_test        0 days 00:00:14.751215
round_time_total       0 days 00:05:59.580382
loss_total                      238666.990613
loss_critic                     298575.833027
loss_actor                        -968.381146
memory_size                       574345.2235 

=== epoch 9/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.54it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  120
episode_length                      82.633333
returns                             418.03191
return_std                          126.00695
average_reward                       5.060437
round_time             0 days 00:06:01.743959
episodes_test                           112.0
episode_length_test                      89.0
returns_test                        456.72531
return_std_test                     91.047256
average_reward_test                  5.131275
round_time_test        0 days 00:00:14.713690
round_time_total       0 days 00:06:01.745064
loss_total                      157681.819627
loss_critic                     197343.028227
loss_actor                        -963.034832
memory_size                        575922.017 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
=== epoch 10/10 ==== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:43,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   23
episode_length                      84.608696
returns                            418.193242
return_std                         109.791975
average_reward                       4.943181
round_time             0 days 00:05:45.558344
episodes_test                           116.0
episode_length_test                 85.853448
returns_test                        445.92003
return_std_test                    105.642859
average_reward_test                  5.193265
round_time_test        0 days 00:00:14.391850
round_time_total       0 days 00:05:45.559479
loss_total                       319480.73069
loss_critic                     399587.414104
loss_actor                        -945.965698
memory_size                        577505.296 

=== epoch 10/10 ==== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                   46
episode_length                      86.065217
returns                            429.524551
return_std                         111.440051
average_reward                       4.989585
round_time             0 days 00:05:48.572285
episodes_test                           111.0
episode_length_test                 89.387387
returns_test                       457.172951
return_std_test                    114.541404
average_reward_test                  5.117377
round_time_test        0 days 00:00:14.696406
round_time_total       0 days 00:05:48.573372
loss_total                       37121.358124
loss_critic                      46639.976814
loss_actor                        -953.120055
memory_size                       579111.3295 

=== epoch 10/10 ==== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   68
episode_length                      87.235294
returns                            438.034523
return_std                         122.041925
average_reward                       5.016569
round_time             0 days 00:05:52.189440
episodes_test                           110.0
episode_length_test                 90.572727
returns_test                       463.497082
return_std_test                    128.159993
average_reward_test                  5.116981
round_time_test        0 days 00:00:14.484047
round_time_total       0 days 00:05:52.190709
loss_total                       91458.061328
loss_critic                     114563.248203
loss_actor                        -962.698904
memory_size                        580719.896 

=== epoch 10/10 ==== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   91
episode_length                      87.384615
returns                            439.101549
return_std                          119.63955
average_reward                        5.02537
round_time             0 days 00:05:53.265097
episodes_test                           108.0
episode_length_test                 91.527778
returns_test                       470.999723
return_std_test                     107.91329
average_reward_test                  5.143772
round_time_test        0 days 00:00:14.571056
round_time_total       0 days 00:05:53.266411
loss_total                       37215.186417
loss_critic                      46761.365709
loss_actor                        -969.533974
memory_size                         582354.09 

=== epoch 10/10 ==== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  117
episode_length                      85.128205
returns                             431.52966
return_std                         112.042962
average_reward                       5.068914
round_time             0 days 00:05:51.784421
episodes_test                           111.0
episode_length_test                 89.477477
returns_test                       463.042861
return_std_test                    117.677057
average_reward_test                  5.173497
round_time_test        0 days 00:00:14.550782
round_time_total       0 days 00:05:51.785507
loss_total                       37456.483486
loss_critic                       47063.58161
loss_actor                        -971.912146
memory_size                       583924.1175 

=== epoch 10/10 ==== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  116
episode_length                      85.560345
returns                            435.776386
return_std                         109.275733
average_reward                       5.091112
round_time             0 days 00:05:52.294654
episodes_test                           112.0
episode_length_test                    89.125
returns_test                       459.533331
return_std_test                     80.730618
average_reward_test                   5.15568
round_time_test        0 days 00:00:14.669896
round_time_total       0 days 00:05:52.296031
loss_total                      314627.790278
loss_critic                     393525.770804
loss_actor                        -964.137031
memory_size                       585496.3975 

=== epoch 10/10 ==== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  119
episode_length                      83.277311
returns                             425.59426
return_std                          103.72838
average_reward                       5.111749
round_time             0 days 00:05:52.339866
episodes_test                           112.0
episode_length_test                 88.857143
returns_test                       461.614384
return_std_test                    118.933128
average_reward_test                  5.194924
round_time_test        0 days 00:00:14.927155
round_time_total       0 days 00:05:52.341033
loss_total                       36945.488132
loss_critic                      46421.597634
loss_actor                        -958.953048
memory_size                       587095.0445 

=== epoch 10/10 ==== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  122
episode_length                      81.327869
returns                            416.725337
return_std                          87.820494
average_reward                       5.124027
round_time             0 days 00:05:53.629421
episodes_test                           115.0
episode_length_test                 86.956522
returns_test                       453.204572
return_std_test                     87.698884
average_reward_test                  5.211853
round_time_test        0 days 00:00:15.064010
round_time_total       0 days 00:05:53.630522
loss_total                      356257.296363
loss_critic                     445563.152356
loss_actor                        -966.056742
memory_size                        588654.192 

=== epoch 10/10 ==== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:26,  4.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  123
episode_length                      80.796748
returns                            413.361054
return_std                          80.433089
average_reward                       5.117172
round_time             0 days 00:05:56.066364
episodes_test                           107.0
episode_length_test                 92.766355
returns_test                       484.881511
return_std_test                    105.890088
average_reward_test                  5.225527
round_time_test        0 days 00:00:15.330866
round_time_total       0 days 00:05:56.067466
loss_total                      465147.528392
loss_critic                     581673.579081
loss_actor                        -956.740661
memory_size                        590246.991 

=== epoch 10/10 ==== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:11,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  120
episode_length                         82.675
returns                            420.098479
return_std                          87.111219
average_reward                       5.082901
round_time             0 days 00:05:54.951704
episodes_test                           102.0
episode_length_test                  97.45098
returns_test                        495.39492
return_std_test                    115.729405
average_reward_test                  5.084251
round_time_test        0 days 00:00:15.023622
round_time_total       0 days 00:05:54.953208
loss_total                        35317.58855
loss_critic                       44389.27693
loss_actor                        -969.168028
memory_size                       591847.5675 

=== epoch 10/10 ==== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  122
episode_length                      81.565574
returns                            414.432902
return_std                          80.883982
average_reward                       5.080193
round_time             0 days 00:05:52.557542
episodes_test                           112.0
episode_length_test                 89.142857
returns_test                       465.489792
return_std_test                    104.112761
average_reward_test                  5.221404
round_time_test        0 days 00:00:14.779315
round_time_total       0 days 00:05:52.558654
loss_total                      113698.843063
loss_critic                     142365.436214
loss_actor                        -967.543305
memory_size                        593433.711 

=== epoch 10/10 ==== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  121
episode_length                      81.801653
returns                            415.249142
return_std                          85.912825
average_reward                       5.077246
round_time             0 days 00:05:59.989650
episodes_test                           102.0
episode_length_test                 97.509804
returns_test                       503.589828
return_std_test                     126.60714
average_reward_test                  5.163349
round_time_test        0 days 00:00:14.719111
round_time_total       0 days 00:05:59.990935
loss_total                       35963.455465
loss_critic                      45195.489688
loss_actor                        -964.684572
memory_size                       595009.3145 

=== epoch 10/10 ==== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  120
episode_length                          83.05
returns                            420.856338
return_std                         103.637621
average_reward                       5.069937
round_time             0 days 00:05:54.981261
episodes_test                           102.0
episode_length_test                 97.529412
returns_test                       498.078738
return_std_test                    126.392278
average_reward_test                  5.106958
round_time_test        0 days 00:00:14.715425
round_time_total       0 days 00:05:54.982411
loss_total                      790571.142696
loss_critic                     988452.638868
loss_actor                        -955.016172
memory_size                       596604.4965 

=== epoch 10/10 ==== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  120
episode_length                          82.75
returns                            420.333252
return_std                          116.23839
average_reward                         5.0745
round_time             0 days 00:05:58.203110
episodes_test                           114.0
episode_length_test                  87.54386
returns_test                       448.489775
return_std_test                    116.649774
average_reward_test                  5.122683
round_time_test        0 days 00:00:14.553575
round_time_total       0 days 00:05:58.204224
loss_total                       34439.307625
loss_critic                       43288.82351
loss_actor                        -958.759014
memory_size                       598192.8135 

=== epoch 10/10 ==== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:11,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  120
episode_length                      83.033333
returns                              420.1112
return_std                         116.069661
average_reward                        5.06049
round_time             0 days 00:05:59.117581
episodes_test                           103.0
episode_length_test                 96.514563
returns_test                       494.131217
return_std_test                    113.956659
average_reward_test                  5.118851
round_time_test        0 days 00:00:14.730089
round_time_total       0 days 00:05:59.118673
loss_total                      138159.533199
loss_critic                     172942.608685
loss_actor                         -972.76531
memory_size                         599791.36 

=== epoch 10/10 ==== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  116
episode_length                      85.474138
returns                            434.044587
return_std                         137.211541
average_reward                       5.078337
round_time             0 days 00:05:56.013409
episodes_test                           109.0
episode_length_test                 91.633028
returns_test                       477.284164
return_std_test                    113.140449
average_reward_test                  5.208293
round_time_test        0 days 00:00:14.641960
round_time_total       0 days 00:05:56.014521
loss_total                      303707.805509
loss_critic                     379878.148473
loss_actor                        -973.594351
memory_size                         601424.79 

=== epoch 10/10 ==== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  117
episode_length                      84.846154
returns                            430.468291
return_std                         140.558033
average_reward                       5.075372
round_time             0 days 00:05:57.275152
episodes_test                           119.0
episode_length_test                 83.857143
returns_test                       434.803166
return_std_test                     96.346996
average_reward_test                  5.184628
round_time_test        0 days 00:00:14.798016
round_time_total       0 days 00:05:57.276249
loss_total                      360712.033414
loss_critic                     451131.201251
loss_actor                        -964.643187
memory_size                        603026.766 

=== epoch 10/10 ==== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  117
episode_length                      85.008547
returns                            429.865776
return_std                         128.797689
average_reward                       5.056406
round_time             0 days 00:05:58.240919
episodes_test                           107.0
episode_length_test                 93.317757
returns_test                       483.684578
return_std_test                    105.083265
average_reward_test                  5.182792
round_time_test        0 days 00:00:14.398812
round_time_total       0 days 00:05:58.242040
loss_total                       85287.207345
loss_critic                     106851.311827
loss_actor                        -969.210448
memory_size                        604579.292 

=== epoch 10/10 ==== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  119
episode_length                      83.008403
returns                            419.992106
return_std                         133.992941
average_reward                       5.061409
round_time             0 days 00:05:58.817027
episodes_test                           114.0
episode_length_test                 86.640351
returns_test                       449.055698
return_std_test                    103.257094
average_reward_test                  5.182119
round_time_test        0 days 00:00:14.724244
round_time_total       0 days 00:05:58.818187
loss_total                      180823.207125
loss_critic                     226274.804923
loss_actor                        -983.198049
memory_size                        606171.938 

=== epoch 10/10 ==== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  121
episode_length                       80.77686
returns                            411.540147
return_std                         129.088652
average_reward                       5.092546
round_time             0 days 00:05:57.775008
episodes_test                           114.0
episode_length_test                 87.666667
returns_test                       455.706443
return_std_test                     94.750013
average_reward_test                  5.197977
round_time_test        0 days 00:00:14.385059
round_time_total       0 days 00:05:57.776111
loss_total                      405979.795687
loss_critic                     507722.127796
loss_actor                        -989.561003
memory_size                        607733.577 

=== epoch 10/10 ==== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  122
episode_length                      81.360656
returns                            414.599697
return_std                         129.654053
average_reward                       5.095611
round_time             0 days 00:05:57.508707
episodes_test                           113.0
episode_length_test                 88.123894
returns_test                       466.682955
return_std_test                     107.42586
average_reward_test                  5.294623
round_time_test        0 days 00:00:14.236817
round_time_total       0 days 00:05:57.509804
loss_total                       37305.143165
loss_critic                      46878.434574
loss_actor                        -988.025541
memory_size                        609327.666 

=== epoch 10/10 ==== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:28,  4.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  119
episode_length                      83.302521
returns                            423.832618
return_std                         131.847551
average_reward                       5.088989
round_time             0 days 00:06:00.126064
episodes_test                           104.0
episode_length_test                 95.759615
returns_test                       493.983821
return_std_test                    144.481957
average_reward_test                  5.158119
round_time_test        0 days 00:00:14.943948
round_time_total       0 days 00:06:00.127152
loss_total                       403372.82279
loss_critic                      504464.00838
loss_actor                        -991.976439
memory_size                       610948.1165 

=== epoch 10/10 ==== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:39,  4.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  118
episode_length                      83.711864
returns                            426.017193
return_std                         130.516306
average_reward                       5.092497
round_time             0 days 00:05:55.789829
episodes_test                           110.0
episode_length_test                 90.236364
returns_test                       473.810952
return_std_test                    125.325454
average_reward_test                   5.24914
round_time_test        0 days 00:00:14.734411
round_time_total       0 days 00:05:55.791286
loss_total                       37426.815763
loss_critic                      47028.652327
loss_actor                        -980.533383
memory_size                       612548.4975 

=== epoch 10/10 ==== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  117
episode_length                      84.957265
returns                            432.639972
return_std                         120.401137
average_reward                       5.093432
round_time             0 days 00:05:49.722796
episodes_test                           106.0
episode_length_test                 94.122642
returns_test                       490.957384
return_std_test                    141.655885
average_reward_test                  5.215563
round_time_test        0 days 00:00:14.546780
round_time_total       0 days 00:05:49.724071
loss_total                       167382.45005
loss_critic                     209476.264856
loss_actor                         -992.82559
memory_size                        614139.095 

=== epoch 10/10 ==== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  117
episode_length                      84.726496
returns                            429.799576
return_std                         103.264196
average_reward                       5.073373
round_time             0 days 00:05:53.982193
episodes_test                           111.0
episode_length_test                 89.945946
returns_test                       468.131089
return_std_test                    103.243722
average_reward_test                  5.204099
round_time_test        0 days 00:00:14.231825
round_time_total       0 days 00:05:53.983531
loss_total                      362756.226413
loss_critic                     453692.804524
loss_actor                        -990.164653
memory_size                        615729.905 

=== epoch 10/10 ==== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:25,  4.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  120
episode_length                      82.533333
returns                            418.482952
return_std                          97.905831
average_reward                       5.070949
round_time             0 days 00:05:52.867158
episodes_test                           110.0
episode_length_test                 90.645455
returns_test                       470.266152
return_std_test                    128.862213
average_reward_test                  5.187306
round_time_test        0 days 00:00:14.500767
round_time_total       0 days 00:05:52.868416
loss_total                      240009.848689
loss_critic                     300257.212346
loss_actor                        -979.642057
memory_size                        617304.548 

=== epoch 10/10 ==== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  121
episode_length                      82.421488
returns                            417.518235
return_std                         106.568016
average_reward                       5.063487
round_time             0 days 00:05:51.863392
episodes_test                           106.0
episode_length_test                 94.150943
returns_test                       487.038999
return_std_test                    137.223794
average_reward_test                  5.172509
round_time_test        0 days 00:00:14.478106
round_time_total       0 days 00:05:51.864509
loss_total                      277927.719375
loss_critic                     347653.493023
loss_actor                        -975.359811
memory_size                        618885.923 

=== epoch 10/10 ==== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:17,  4.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  121
episode_length                      82.570248
returns                            418.874147
return_std                         112.637917
average_reward                       5.073914
round_time             0 days 00:05:52.632218
episodes_test                           108.0
episode_length_test                 92.425926
returns_test                       478.635621
return_std_test                    123.105437
average_reward_test                  5.178181
round_time_test        0 days 00:00:14.656582
round_time_total       0 days 00:05:52.633306
loss_total                      535377.414604
loss_critic                      669467.68491
loss_actor                        -983.678834
memory_size                        620494.005 

=== epoch 10/10 ==== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  118
episode_length                      84.254237
returns                            428.058335
return_std                         117.615035
average_reward                       5.082543
round_time             0 days 00:05:50.207241
episodes_test                           100.0
episode_length_test                     99.32
returns_test                       500.123307
return_std_test                    141.224617
average_reward_test                  5.033766
round_time_test        0 days 00:00:14.763997
round_time_total       0 days 00:05:50.208335
loss_total                      113532.133242
loss_critic                      142159.32893
loss_actor                        -976.662612
memory_size                        622115.721 

=== epoch 10/10 ==== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:25,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  115
episode_length                      85.843478
returns                            435.685498
return_std                         128.189955
average_reward                       5.078632
round_time             0 days 00:05:51.147847
episodes_test                           106.0
episode_length_test                 93.660377
returns_test                       482.967905
return_std_test                     125.56082
average_reward_test                  5.154872
round_time_test        0 days 00:00:14.431729
round_time_total       0 days 00:05:51.148949
loss_total                       37891.744574
loss_critic                      47608.607285
loss_actor                        -975.709875
memory_size                       623716.8415 

=== epoch 10/10 ==== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:07,  4.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  113
episode_length                      87.486726
returns                            443.931213
return_std                         136.153477
average_reward                       5.073509
round_time             0 days 00:05:52.218769
episodes_test                           110.0
episode_length_test                 90.681818
returns_test                       470.549266
return_std_test                     126.84525
average_reward_test                  5.188403
round_time_test        0 days 00:00:14.540305
round_time_total       0 days 00:05:52.219860
loss_total                       36641.227354
loss_critic                      46048.586438
loss_actor                        -988.212249
memory_size                       625345.7655 

=== epoch 10/10 ==== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  113
episode_length                      87.884956
returns                            448.588253
return_std                         130.654192
average_reward                        5.10525
round_time             0 days 00:05:53.738916
episodes_test                           100.0
episode_length_test                     99.27
returns_test                       501.164301
return_std_test                    138.818217
average_reward_test                  5.047234
round_time_test        0 days 00:00:14.414061
round_time_total       0 days 00:05:53.740019
loss_total                       169447.67691
loss_critic                      212059.06619
loss_actor                        -997.893886
memory_size                       626948.1305 

=== epoch 10/10 ==== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  114
episode_length                      87.070175
returns                            446.026736
return_std                         134.923107
average_reward                       5.122886
round_time             0 days 00:05:50.384345
episodes_test                           115.0
episode_length_test                 86.426087
returns_test                       456.369275
return_std_test                     99.822393
average_reward_test                  5.280193
round_time_test        0 days 00:00:14.367714
round_time_total       0 days 00:05:50.385612
loss_total                       36683.025472
loss_critic                      46103.392408
loss_actor                        -998.445488
memory_size                        628551.403 

=== epoch 10/10 ==== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  114
episode_length                      86.885965
returns                            444.245506
return_std                         136.300619
average_reward                       5.114437
round_time             0 days 00:05:51.526786
episodes_test                           108.0
episode_length_test                 92.444444
returns_test                       482.003955
return_std_test                    103.486533
average_reward_test                  5.213313
round_time_test        0 days 00:00:14.157094
round_time_total       0 days 00:05:51.527874
loss_total                      182358.920688
loss_critic                     228199.129365
loss_actor                       -1001.909143
memory_size                        630151.541 

=== epoch 10/10 ==== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:30,  4.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  117
episode_length                      84.495726
returns                            433.935713
return_std                         126.796344
average_reward                       5.134891
round_time             0 days 00:05:54.418603
episodes_test                           116.0
episode_length_test                 86.094828
returns_test                       446.233913
return_std_test                     93.492153
average_reward_test                  5.182659
round_time_test        0 days 00:00:13.906647
round_time_total       0 days 00:05:54.419716
loss_total                      388018.067385
loss_critic                     485270.658168
loss_actor                        -992.380357
memory_size                        631748.199 

=== epoch 10/10 ==== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:21,  4.51it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  117
episode_length                      85.042735
returns                            436.232917
return_std                          129.14854
average_reward                       5.130145
round_time             0 days 00:05:50.674943
episodes_test                           110.0
episode_length_test                 90.218182
returns_test                       468.134189
return_std_test                    116.245574
average_reward_test                  5.188052
round_time_test        0 days 00:00:14.450171
round_time_total       0 days 00:05:50.676017
loss_total                      274424.849232
loss_critic                     343277.077576
loss_actor                         -984.07852
memory_size                       633344.2535 

=== epoch 10/10 ==== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  117
episode_length                      84.923077
returns                            433.345593
return_std                         128.771216
average_reward                       5.103887
round_time             0 days 00:05:52.353858
episodes_test                           123.0
episode_length_test                 80.577236
returns_test                       429.390486
return_std_test                     88.267949
average_reward_test                  5.328091
round_time_test        0 days 00:00:14.063434
round_time_total       0 days 00:05:52.354959
loss_total                      326302.894257
loss_critic                     408124.865578
loss_actor                        -985.024321
memory_size                       634950.6435 

=== epoch 10/10 ==== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:03,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  116
episode_length                      85.724138
returns                            436.738108
return_std                         131.410964
average_reward                        5.09323
round_time             0 days 00:05:53.454341
episodes_test                           105.0
episode_length_test                 94.857143
returns_test                       480.182939
return_std_test                    115.979294
average_reward_test                  5.061843
round_time_test        0 days 00:00:14.818663
round_time_total       0 days 00:05:53.455710
loss_total                      174753.316741
loss_critic                     218688.635558
loss_actor                        -987.936746
memory_size                       636563.8775 

=== epoch 10/10 ==== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:10,  4.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  115
episode_length                      85.573913
returns                            436.812153
return_std                         133.142885
average_reward                       5.105524
round_time             0 days 00:05:53.926439
episodes_test                           107.0
episode_length_test                 93.345794
returns_test                       479.559782
return_std_test                    131.117267
average_reward_test                  5.137149
round_time_test        0 days 00:00:15.079576
round_time_total       0 days 00:05:53.927531
loss_total                      192517.467655
loss_critic                      240895.53227
loss_actor                        -994.805353
memory_size                        638168.338 

=== epoch 10/10 ==== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:35,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  114
episode_length                      86.649123
returns                             443.34648
return_std                         139.803613
average_reward                        5.11478
round_time             0 days 00:05:52.111387
episodes_test                           114.0
episode_length_test                 87.447368
returns_test                       452.442337
return_std_test                    105.758885
average_reward_test                  5.173564
round_time_test        0 days 00:00:14.648129
round_time_total       0 days 00:05:52.112615
loss_total                       82046.898718
loss_critic                     102804.387107
loss_actor                        -983.060067
memory_size                       639771.4815 

=== epoch 10/10 ==== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  116
episode_length                      85.112069
returns                            434.986494
return_std                         139.067911
average_reward                       5.107246
round_time             0 days 00:05:52.285310
episodes_test                           111.0
episode_length_test                 89.531532
returns_test                       469.003895
return_std_test                    121.549965
average_reward_test                  5.237941
round_time_test        0 days 00:00:14.150993
round_time_total       0 days 00:05:52.286410
loss_total                      120444.702217
loss_critic                     150803.473337
loss_actor                        -990.368099
memory_size                       641356.3075 

=== epoch 10/10 ==== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:32,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  115
episode_length                      84.921739
returns                            432.901269
return_std                           130.2006
average_reward                       5.096425
round_time             0 days 00:05:54.554063
episodes_test                            97.0
episode_length_test                103.030928
returns_test                        525.85164
return_std_test                    135.362524
average_reward_test                  5.103683
round_time_test        0 days 00:00:14.840289
round_time_total       0 days 00:05:54.555150
loss_total                       35880.118287
loss_critic                       45100.56494
loss_actor                        -1001.67141
memory_size                       642947.8485 

=== epoch 10/10 ==== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:14,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  115
episode_length                      86.521739
returns                            440.207447
return_std                         138.030899
average_reward                       5.088373
round_time             0 days 00:05:50.782652
episodes_test                            99.0
episode_length_test                 100.20202
returns_test                       511.660455
return_std_test                    138.345211
average_reward_test                  5.105851
round_time_test        0 days 00:00:14.602273
round_time_total       0 days 00:05:50.783764
loss_total                      386459.506059
loss_critic                     483325.515631
loss_actor                       -1004.558554
memory_size                       644577.2655 

=== epoch 10/10 ==== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  113
episode_length                      87.477876
returns                             443.75719
return_std                         137.181245
average_reward                       5.077278
round_time             0 days 00:05:50.687987
episodes_test                           117.0
episode_length_test                  85.34188
returns_test                       445.429194
return_std_test                     96.324451
average_reward_test                  5.218886
round_time_test        0 days 00:00:14.334250
round_time_total       0 days 00:05:50.689271
loss_total                      179904.727382
loss_critic                     225128.664963
loss_actor                        -991.036638
memory_size                       646218.7355 

=== epoch 10/10 ==== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  114
episode_length                      87.447368
returns                            441.001587
return_std                         131.765445
average_reward                        5.04132
round_time             0 days 00:05:54.040770
episodes_test                            96.0
episode_length_test                103.885417
returns_test                       525.636966
return_std_test                    107.628806
average_reward_test                  5.059457
round_time_test        0 days 00:00:14.708306
round_time_total       0 days 00:05:54.041866
loss_total                       36030.710573
loss_critic                      45287.585827
loss_actor                        -996.793591
memory_size                       647827.3615 

=== epoch 10/10 ==== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:23,  4.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  111
episode_length                      89.279279
returns                            449.591554
return_std                          134.76576
average_reward                       5.038179
round_time             0 days 00:05:52.276245
episodes_test                           109.0
episode_length_test                 90.770642
returns_test                       468.959406
return_std_test                    129.507607
average_reward_test                  5.163882
round_time_test        0 days 00:00:14.365716
round_time_total       0 days 00:05:52.277683
loss_total                      324211.479419
loss_critic                     405511.910678
loss_actor                        -990.277809
memory_size                        649446.519 

=== epoch 10/10 ==== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:06,  4.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  111
episode_length                      89.621622
returns                            452.832916
return_std                         134.801432
average_reward                       5.050531
round_time             0 days 00:05:52.376511
episodes_test                           104.0
episode_length_test                 95.701923
returns_test                       486.576235
return_std_test                    128.793522
average_reward_test                  5.084875
round_time_test        0 days 00:00:15.053164
round_time_total       0 days 00:05:52.377612
loss_total                       178939.51725
loss_critic                     223921.940494
loss_actor                        -990.194065
memory_size                        651054.516 

=== epoch 10/10 ==== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:16,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  112
episode_length                      88.267857
returns                            444.434051
return_std                         130.832513
average_reward                       5.035773
round_time             0 days 00:05:53.729343
episodes_test                           109.0
episode_length_test                 91.229358
returns_test                       475.270963
return_std_test                    122.089763
average_reward_test                  5.208103
round_time_test        0 days 00:00:14.357001
round_time_total       0 days 00:05:53.730441
loss_total                       36599.529062
loss_critic                      45999.853404
loss_actor                       -1001.771562
memory_size                        652660.932 

=== epoch 10/10 ==== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:38,  4.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  116
episode_length                      85.810345
returns                            432.061661
return_std                         121.689481
average_reward                       5.034699
round_time             0 days 00:05:52.305618
episodes_test                           115.0
episode_length_test                 86.373913
returns_test                       447.949409
return_std_test                     98.281179
average_reward_test                  5.185531
round_time_test        0 days 00:00:14.080881
round_time_total       0 days 00:05:52.306714
loss_total                       57351.356201
loss_critic                      71941.746757
loss_actor                       -1010.209073
memory_size                       654258.7035 

=== epoch 10/10 ==== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Humanoid-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HumanoidEnv<Humanoid-v4>>>>>>>>>>
episodes                                  110
episode_length                      88.309091
returns                            445.165601
return_std                         129.122904
average_reward                       5.034961
round_time             0 days 00:05:49.469603
episodes_test                           114.0
episode_length_test                 87.368421
returns_test                       452.445718
return_std_test                      98.42421
average_reward_test                  5.178123
round_time_test        0 days 00:00:13.963546
round_time_total       0 days 00:05:49.470733
loss_total                       37297.274459
loss_critic                      46875.208333
loss_actor                       -1014.464429
memory_size                        655890.628 


