/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
=== specification ====================================================
+: rlrd.training:Training
epochs: 10
rounds: 50
steps: 2000
stats_window: 10000
seed: 0
tag: ''
Env:
   +: rlrd.envs:RandomDelayEnv
   seed_val: 0
   id: Hopper-v4
   frame_skip: 0
   min_observation_delay: 0
   sup_observation_delay: 1
   min_action_delay: 0
   sup_action_delay: 1
   real_world_sampler: 5
   action_noise: 0.05
Test:
   +: rlrd.testing:Test
   workers: 1
   number: 1
   device: cpu
Agent:
   +: rlrd.dcac:Agent
   batchsize: 128
   memory_size: 1000000
   lr: 0.0003
   discount: 0.99
   target_update: 0.005
   reward_scale: 5.0
   entropy_scale: 1.0
   start_training: 10000
   device: cpu
   training_steps: 1.0
   loss_alpha: 0.2
   rtac: false
   Model:
      +: rlrd.dcac_models:Mlp
      hidden_units: 256
      num_critics: 2
      act_delay: true
      obs_delay: true
   OutputNorm:
      +: rlrd.nn:PopArt
      beta: 0.0003
      zero_debias: true
      start_pop: 8
__format_version__: '3'
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>

<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 1/10 ===== round 1/50 ======================================
100%|██████████| 2000/2000 [00:01<00:00, 1204.91it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  112
episode_length                      17.776786
returns                              13.47995
return_std                          12.178966
average_reward                       0.758643
round_time             0 days 00:00:01.700241
episodes_test                           507.0
episode_length_test                 19.715976
returns_test                        15.623297
return_std_test                      8.365622
average_reward_test                  0.792466
round_time_test        0 days 00:00:08.608773
round_time_total       0 days 00:00:10.422613 

=== epoch 1/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
100%|██████████| 2000/2000 [00:01<00:00, 1176.65it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  232
episode_length                      17.219828
returns                             12.954027
return_std                          10.844766
average_reward                       0.752652
round_time             0 days 00:00:02.191648
episodes_test                           483.0
episode_length_test                 20.701863
returns_test                        16.742667
return_std_test                      9.678444
average_reward_test                   0.80877
round_time_test        0 days 00:00:08.654875
round_time_total       0 days 00:00:10.406174 

=== epoch 1/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
100%|██████████| 2000/2000 [00:01<00:00, 1113.28it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  343
episode_length                       17.48688
returns                             13.565152
return_std                          12.513227
average_reward                       0.775806
round_time             0 days 00:00:02.311066
episodes_test                           503.0
episode_length_test                 19.864811
returns_test                        15.827316
return_std_test                      8.974833
average_reward_test                  0.796713
round_time_test        0 days 00:00:08.671056
round_time_total       0 days 00:00:10.564301 

=== epoch 1/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
100%|██████████| 2000/2000 [00:01<00:00, 1166.97it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  460
episode_length                      17.382609
returns                             13.503325
return_std                          13.456741
average_reward                       0.776863
round_time             0 days 00:00:02.244084
episodes_test                           492.0
episode_length_test                 20.292683
returns_test                        16.441759
return_std_test                      8.924475
average_reward_test                  0.810379
round_time_test        0 days 00:00:08.692619
round_time_total       0 days 00:00:10.469061 

=== epoch 1/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
100%|██████████| 2000/2000 [00:01<00:00, 1177.61it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  578
episode_length                      17.280277
returns                             13.313105
return_std                          12.651052
average_reward                       0.770137
round_time             0 days 00:00:02.229572
episodes_test                           489.0
episode_length_test                 20.447853
returns_test                        16.725626
return_std_test                      9.351472
average_reward_test                  0.817977
round_time_test        0 days 00:00:08.860894
round_time_total       0 days 00:00:10.681167 

=== epoch 1/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 0/2000 [00:00<?, ?it/s]/<ANONYMIZED PATH>/rmst-rlrd/rlrd/nn.py:41: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  assert b.storage().data_ptr() == a.storage().data_ptr()
  0%|          | 9/2000 [00:01<05:48,  5.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
starting training
episodes                                  538
episode_length                      18.546468
returns                             17.015458
return_std                          18.313652
average_reward                       0.917754
round_time             0 days 00:05:42.955033
episodes_test                           504.0
episode_length_test                 19.835317
returns_test                        15.563619
return_std_test                      8.484282
average_reward_test                  0.784675
round_time_test        0 days 00:00:08.750158
round_time_total       0 days 00:05:42.956329
loss_total                        2527.133969
loss_critic                       3192.233604
loss_actor                        -133.264762
memory_size                          2505.759 

=== epoch 1/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                  521
episode_length                      19.099808
returns                             19.195921
return_std                          20.719132
average_reward                       1.006797
round_time             0 days 00:05:43.020611
episodes_test                           465.0
episode_length_test                 21.492473
returns_test                        30.833289
return_std_test                     28.390285
average_reward_test                  1.434304
round_time_test        0 days 00:00:08.816429
round_time_total       0 days 00:05:43.021699
loss_total                       22018.278393
loss_critic                       27643.39506
loss_actor                        -482.190225
memory_size                          3175.011 

=== epoch 1/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.81it/s]
episodes                                  528
episode_length                      18.869318
returns                             20.038095
return_std                           21.16082
average_reward                       1.059896
round_time             0 days 00:05:44.822224
episodes_test                           460.0
episode_length_test                 21.730435
returns_test                         26.82961
return_std_test                     25.438998
average_reward_test                  1.234555
round_time_test        0 days 00:00:08.751276
round_time_total       0 days 00:05:44.823356
loss_total                      118527.007733
loss_critic                     148461.785287
loss_actor                       -1212.113143
memory_size                          3788.929 

=== epoch 1/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:51,  5.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                  529
episode_length                      18.827977
returns                             20.993963
return_std                          20.868844
average_reward                       1.114082
round_time             0 days 00:05:49.476204
episodes_test                           588.0
episode_length_test                 17.003401
returns_test                        17.795015
return_std_test                     16.589338
average_reward_test                  1.046544
round_time_test        0 days 00:00:08.721320
round_time_total       0 days 00:05:49.477329
loss_total                       539133.50775
loss_critic                     674579.236648
loss_actor                       -2649.457396
memory_size                         4181.6215 

=== epoch 1/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  538
episode_length                      18.578067
returns                             21.534987
return_std                          20.684402
average_reward                       1.158529
round_time             0 days 00:05:51.798721
episodes_test                           641.0
episode_length_test                 15.585023
returns_test                        15.158326
return_std_test                     13.377432
average_reward_test                  0.972766
round_time_test        0 days 00:00:08.884107
round_time_total       0 days 00:05:51.800065
loss_total                     2039995.828531
loss_critic                    2551253.957656
loss_actor                       -5036.870106
memory_size                          4561.902 

=== epoch 1/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  579
episode_length                      17.212435
returns                              17.97586
return_std                          16.654067
average_reward                       1.047169
round_time             0 days 00:05:51.821168
episodes_test                           678.0
episode_length_test                 14.727139
returns_test                        13.245267
return_std_test                     10.537424
average_reward_test                  0.899248
round_time_test        0 days 00:00:09.056988
round_time_total       0 days 00:05:51.822390
loss_total                     6233805.380688
loss_critic                     7794393.13525
loss_actor                       -8546.146017
memory_size                          4888.157 

=== epoch 1/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  585
episode_length                      17.046154
returns                             16.869245
return_std                           14.96283
average_reward                       0.990394
round_time             0 days 00:05:53.440666
episodes_test                           597.0
episode_length_test                 16.740369
returns_test                        15.629998
return_std_test                     12.782066
average_reward_test                  0.933638
round_time_test        0 days 00:00:08.876971
round_time_total       0 days 00:05:53.442004
loss_total                     13983135.32525
loss_critic                      17482014.235
loss_actor                      -12381.488646
memory_size                          5373.007 

=== epoch 1/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  576
episode_length                      17.314236
returns                             16.655596
return_std                          14.445071
average_reward                       0.961766
round_time             0 days 00:05:52.870658
episodes_test                           568.0
episode_length_test                 17.602113
returns_test                        15.724419
return_std_test                     13.565292
average_reward_test                   0.89333
round_time_test        0 days 00:00:08.752305
round_time_total       0 days 00:05:52.871757
loss_total                       24614213.859
loss_critic                     30771782.8675
loss_actor                      -16064.254565
memory_size                          5839.624 

=== epoch 1/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  577
episode_length                      17.305026
returns                             16.376054
return_std                          14.555194
average_reward                       0.944609
round_time             0 days 00:05:52.417482
episodes_test                           587.0
episode_length_test                 17.022147
returns_test                        16.058499
return_std_test                     15.907898
average_reward_test                  0.943369
round_time_test        0 days 00:00:08.779290
round_time_total       0 days 00:05:52.418598
loss_total                      34670175.8565
loss_critic                     43342454.5835
loss_actor                      -18942.234425
memory_size                         6329.7105 

=== epoch 1/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  634
episode_length                      15.742902
returns                             14.577897
return_std                          13.703174
average_reward                       0.925921
round_time             0 days 00:05:54.108447
episodes_test                           671.0
episode_length_test                 14.897168
returns_test                        14.925109
return_std_test                     13.633895
average_reward_test                  1.001821
round_time_test        0 days 00:00:08.740583
round_time_total       0 days 00:05:54.109762
loss_total                       49312217.108
loss_critic                      61646106.097
loss_actor                      -23343.014686
memory_size                          6550.341 

=== epoch 1/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  711
episode_length                      14.040788
returns                              12.96744
return_std                          12.038351
average_reward                       0.923085
round_time             0 days 00:05:54.641646
episodes_test                           985.0
episode_length_test                 10.149239
returns_test                         9.291221
return_std_test                      4.265323
average_reward_test                  0.915474
round_time_test        0 days 00:00:09.139901
round_time_total       0 days 00:05:54.642752
loss_total                       87117100.128
loss_critic                      108904227.99
loss_actor                      -31418.382741
memory_size                         6560.4425 

=== epoch 1/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  791
episode_length                      12.619469
returns                             11.690531
return_std                           9.936031
average_reward                       0.926681
round_time             0 days 00:05:53.171930
episodes_test                           957.0
episode_length_test                 10.446186
returns_test                         9.684886
return_std_test                      4.934889
average_reward_test                  0.927153
round_time_test        0 days 00:00:09.060850
round_time_total       0 days 00:05:53.173043
loss_total                      129607108.138
loss_critic                     162018422.944
loss_actor                      -38161.555745
memory_size                          6574.871 

=== epoch 1/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  873
episode_length                      11.435281
returns                             10.511394
return_std                           7.564474
average_reward                       0.920124
round_time             0 days 00:05:53.302584
episodes_test                           914.0
episode_length_test                 10.939825
returns_test                        10.114727
return_std_test                      6.434534
average_reward_test                  0.924583
round_time_test        0 days 00:00:09.138016
round_time_total       0 days 00:05:53.303709
loss_total                      175318432.628
loss_critic                      219159346.36
loss_actor                      -45237.315463
memory_size                          6594.795 

=== epoch 1/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  931
episode_length                      10.721805
returns                              9.759097
return_std                           5.078151
average_reward                       0.909687
round_time             0 days 00:05:54.059185
episodes_test                           944.0
episode_length_test                 10.588983
returns_test                          9.87652
return_std_test                      6.689219
average_reward_test                  0.932726
round_time_test        0 days 00:00:09.046576
round_time_total       0 days 00:05:54.060469
loss_total                      237106123.104
loss_critic                     296395832.352
loss_actor                        -52735.9566
memory_size                         6627.6305 

=== epoch 1/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  917
episode_length                      10.878953
returns                             10.030723
return_std                           5.516578
average_reward                         0.9209
round_time             0 days 00:05:53.834861
episodes_test                           904.0
episode_length_test                 11.056416
returns_test                        10.299341
return_std_test                      6.659368
average_reward_test                  0.931414
round_time_test        0 days 00:00:09.046576
round_time_total       0 days 00:05:53.836224
loss_total                      290506572.496
loss_critic                     363147880.044
loss_actor                      -58685.072486
memory_size                           6696.04 

=== epoch 1/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  875
episode_length                      11.414857
returns                             10.310081
return_std                           6.734677
average_reward                       0.903199
round_time             0 days 00:05:55.941169
episodes_test                           866.0
episode_length_test                 11.535797
returns_test                        10.718446
return_std_test                      8.802975
average_reward_test                  0.929295
round_time_test        0 days 00:00:08.987035
round_time_total       0 days 00:05:55.942302
loss_total                      308294771.852
loss_critic                     385383392.072
loss_actor                      -59733.068301
memory_size                         6834.4555 

=== epoch 1/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:06,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  833
episode_length                      11.991597
returns                             10.126183
return_std                           7.070419
average_reward                       0.844687
round_time             0 days 00:05:53.336805
episodes_test                           752.0
episode_length_test                 13.292553
returns_test                         7.598004
return_std_test                      5.733277
average_reward_test                  0.571697
round_time_test        0 days 00:00:08.878690
round_time_total       0 days 00:05:53.338020
loss_total                       319230073.84
loss_critic                     399052697.576
loss_actor                      -60446.879082
memory_size                         7048.1935 

=== epoch 1/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  790
episode_length                      12.643038
returns                              9.835105
return_std                           7.254799
average_reward                       0.778111
round_time             0 days 00:05:53.483201
episodes_test                           718.0
episode_length_test                 13.926184
returns_test                         8.196451
return_std_test                      6.598461
average_reward_test                  0.588602
round_time_test        0 days 00:00:09.011283
round_time_total       0 days 00:05:53.484306
loss_total                       349109928.28
loss_critic                     436403048.168
loss_actor                      -62580.836619
memory_size                         7233.2225 

=== epoch 1/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  751
episode_length                      13.283622
returns                              9.727399
return_std                           7.694013
average_reward                       0.732759
round_time             0 days 00:05:52.832108
episodes_test                           681.0
episode_length_test                  14.67254
returns_test                         9.096725
return_std_test                      8.154286
average_reward_test                  0.620293
round_time_test        0 days 00:00:09.013236
round_time_total       0 days 00:05:52.833245
loss_total                      345085732.672
loss_critic                     431372766.144
loss_actor                      -62429.311318
memory_size                          7399.108 

=== epoch 1/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:51,  5.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  699
episode_length                      14.277539
returns                             10.629997
return_std                          10.829891
average_reward                       0.744669
round_time             0 days 00:05:56.026594
episodes_test                           613.0
episode_length_test                  16.30832
returns_test                        14.587159
return_std_test                     17.030344
average_reward_test                  0.894466
round_time_test        0 days 00:00:08.773186
round_time_total       0 days 00:05:56.027801
loss_total                      310075666.096
loss_critic                     387609321.432
loss_actor                      -58982.349307
memory_size                         7781.4975 

=== epoch 1/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  658
episode_length                      15.161094
returns                             11.974549
return_std                           13.68582
average_reward                       0.789242
round_time             0 days 00:05:56.516793
episodes_test                           584.0
episode_length_test                 17.113014
returns_test                         16.52331
return_std_test                     18.622065
average_reward_test                  0.965558
round_time_test        0 days 00:00:08.941950
round_time_total       0 days 00:05:56.517934
loss_total                        263917118.2
loss_critic                      329909981.84
loss_actor                      -54360.421289
memory_size                          8306.313 

=== epoch 1/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  637
episode_length                      15.684458
returns                             13.355932
return_std                          15.357666
average_reward                       0.851409
round_time             0 days 00:05:54.398123
episodes_test                           612.0
episode_length_test                  16.27451
returns_test                        15.759851
return_std_test                     17.741908
average_reward_test                  0.970954
round_time_test        0 days 00:00:08.842199
round_time_total       0 days 00:05:54.399300
loss_total                      199052144.432
loss_critic                     248826826.896
loss_actor                      -46603.628555
memory_size                         8844.6035 

=== epoch 1/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  677
episode_length                      14.745938
returns                             12.918685
return_std                          15.012724
average_reward                       0.876119
round_time             0 days 00:05:54.826150
episodes_test                           831.0
episode_length_test                 12.019254
returns_test                          9.23321
return_std_test                      7.490243
average_reward_test                  0.768601
round_time_test        0 days 00:00:09.119113
round_time_total       0 days 00:05:54.827249
loss_total                      142296040.364
loss_critic                     177880150.864
loss_actor                      -40413.890023
memory_size                           9065.36 

=== epoch 1/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  726
episode_length                       13.72865
returns                             12.073938
return_std                          14.192726
average_reward                       0.881013
round_time             0 days 00:05:55.335953
episodes_test                           971.0
episode_length_test                 10.292482
returns_test                         7.300694
return_std_test                      4.022736
average_reward_test                  0.709493
round_time_test        0 days 00:00:09.018764
round_time_total       0 days 00:05:55.337121
loss_total                      116567287.336
loss_critic                      145718765.65
loss_actor                      -38636.320603
memory_size                         9110.9065 

=== epoch 1/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:42,  5.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  784
episode_length                      12.733418
returns                             10.530954
return_std                          12.153776
average_reward                        0.82796
round_time             0 days 00:05:55.270896
episodes_test                           960.0
episode_length_test                 10.413542
returns_test                         7.361319
return_std_test                      3.960721
average_reward_test                  0.706984
round_time_test        0 days 00:00:09.122143
round_time_total       0 days 00:05:55.272169
loss_total                       99778581.972
loss_critic                      124732414.43
loss_actor                      -36756.072552
memory_size                         9194.6945 

=== epoch 1/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  865
episode_length                      11.553757
returns                              8.694213
return_std                           8.351287
average_reward                       0.752564
round_time             0 days 00:05:53.901250
episodes_test                           923.0
episode_length_test                 10.826652
returns_test                         7.684233
return_std_test                      3.673601
average_reward_test                  0.709827
round_time_test        0 days 00:00:09.174984
round_time_total       0 days 00:05:53.902388
loss_total                       76369561.171
loss_critic                      95470220.736
loss_actor                      -33084.040445
memory_size                          9289.347 

=== epoch 1/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:59,  5.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  920
episode_length                      10.859783
returns                              7.631817
return_std                           4.977688
average_reward                       0.702694
round_time             0 days 00:05:55.848333
episodes_test                           938.0
episode_length_test                 10.656716
returns_test                         7.649433
return_std_test                      3.930618
average_reward_test                  0.717881
round_time_test        0 days 00:00:09.118013
round_time_total       0 days 00:05:55.849447
loss_total                       56444856.021
loss_critic                      70563418.755
loss_actor                      -29399.715479
memory_size                         9337.9625 

=== epoch 1/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  908
episode_length                      11.006608
returns                              7.766943
return_std                           5.198784
average_reward                       0.705664
round_time             0 days 00:05:55.264646
episodes_test                           899.0
episode_length_test                 11.120133
returns_test                          7.78395
return_std_test                      4.308856
average_reward_test                   0.70002
round_time_test        0 days 00:00:08.894935
round_time_total       0 days 00:05:55.265738
loss_total                      41192246.8925
loss_critic                      51496789.058
loss_actor                      -25925.394107
memory_size                          9412.569 

=== epoch 1/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  898
episode_length                      11.111359
returns                              7.856445
return_std                           5.265335
average_reward                       0.707918
round_time             0 days 00:05:56.669585
episodes_test                           913.0
episode_length_test                 10.946331
returns_test                          7.85258
return_std_test                      4.595631
average_reward_test                  0.717422
round_time_test        0 days 00:00:09.179657
round_time_total       0 days 00:05:56.670719
loss_total                      31453665.2335
loss_critic                      39322768.291
loss_actor                      -22749.980771
memory_size                          9492.883 

=== epoch 1/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  897
episode_length                      11.134894
returns                              7.959019
return_std                            4.59157
average_reward                        0.71452
round_time             0 days 00:05:55.880380
episodes_test                           896.0
episode_length_test                 11.154018
returns_test                         8.212413
return_std_test                      5.204769
average_reward_test                  0.736414
round_time_test        0 days 00:00:08.926218
round_time_total       0 days 00:05:55.881924
loss_total                      22774105.6035
loss_critic                     28472499.4535
loss_actor                      -19471.709487
memory_size                          9557.331 

=== epoch 1/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  876
episode_length                      11.400685
returns                              8.372338
return_std                           5.206926
average_reward                       0.734485
round_time             0 days 00:05:52.759471
episodes_test                           837.0
episode_length_test                 11.941458
returns_test                          8.82326
return_std_test                       5.41217
average_reward_test                  0.739013
round_time_test        0 days 00:00:09.126657
round_time_total       0 days 00:05:52.760577
loss_total                     16768694.52275
loss_critic                     20965058.8175
loss_actor                      -16764.173862
memory_size                         9635.3375 

=== epoch 1/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  845
episode_length                      11.811834
returns                              8.882541
return_std                           5.661238
average_reward                       0.752391
round_time             0 days 00:05:55.774325
episodes_test                           823.0
episode_length_test                 12.139733
returns_test                         8.982954
return_std_test                      5.785453
average_reward_test                  0.740276
round_time_test        0 days 00:00:09.176545
round_time_total       0 days 00:05:55.775817
loss_total                     12367036.23425
loss_critic                    15462378.25675
loss_actor                        -14332.9221
memory_size                         9804.9375 

=== epoch 1/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  820
episode_length                      12.171951
returns                              9.161597
return_std                           5.602622
average_reward                       0.752901
round_time             0 days 00:05:53.649576
episodes_test                           770.0
episode_length_test                 12.976623
returns_test                         9.750969
return_std_test                      6.183363
average_reward_test                  0.751378
round_time_test        0 days 00:00:09.038482
round_time_total       0 days 00:05:53.650696
loss_total                     9474949.887375
loss_critic                    11846801.61325
loss_actor                      -12457.845668
memory_size                          9990.298 

=== epoch 1/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  784
episode_length                      12.707908
returns                              9.677274
return_std                           6.177646
average_reward                       0.762289
round_time             0 days 00:05:56.023247
episodes_test                           743.0
episode_length_test                 13.449529
returns_test                         9.888173
return_std_test                      7.161724
average_reward_test                  0.735218
round_time_test        0 days 00:00:08.836972
round_time_total       0 days 00:05:56.024714
loss_total                     7337845.219125
loss_critic                    9174999.527125
loss_actor                      -10772.684039
memory_size                        10196.4725 

=== epoch 1/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  750
episode_length                         13.312
returns                             10.084863
return_std                           6.688057
average_reward                       0.757485
round_time             0 days 00:05:55.906457
episodes_test                           720.0
episode_length_test                 13.858333
returns_test                        10.550193
return_std_test                      7.690034
average_reward_test                  0.761518
round_time_test        0 days 00:00:08.948472
round_time_total       0 days 00:05:55.907590
loss_total                     5843677.295125
loss_critic                     7306946.62275
loss_actor                       -9400.511131
memory_size                         10465.931 

=== epoch 1/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  727
episode_length                      13.718019
returns                             10.443282
return_std                           7.198275
average_reward                       0.760726
round_time             0 days 00:05:55.357763
episodes_test                           707.0
episode_length_test                 14.121641
returns_test                        10.769073
return_std_test                      8.101741
average_reward_test                  0.762253
round_time_test        0 days 00:00:09.045259
round_time_total       0 days 00:05:55.358850
loss_total                     4471160.330063
loss_critic                       5590985.177
loss_actor                       -8139.433269
memory_size                         10651.457 

=== epoch 1/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  707
episode_length                      14.082037
returns                             10.781393
return_std                           7.863881
average_reward                       0.764385
round_time             0 days 00:05:55.925952
episodes_test                           691.0
episode_length_test                 14.463097
returns_test                        10.967224
return_std_test                      8.737427
average_reward_test                  0.758352
round_time_test        0 days 00:00:08.805824
round_time_total       0 days 00:05:55.927254
loss_total                     3506490.512938
loss_critic                    4384850.635375
loss_actor                       -6950.273098
memory_size                         10954.582 

=== epoch 1/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:45,  5.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  688
episode_length                      14.510174
returns                             11.350755
return_std                           9.022073
average_reward                       0.782044
round_time             0 days 00:05:54.930186
episodes_test                           658.0
episode_length_test                 15.182371
returns_test                        12.364257
return_std_test                     11.089904
average_reward_test                  0.814411
round_time_test        0 days 00:00:08.993275
round_time_total       0 days 00:05:54.931308
loss_total                     2558915.309563
loss_critic                    3200062.186875
loss_actor                         -5672.4304
memory_size                         11267.707 

=== epoch 1/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  686
episode_length                      14.523324
returns                             11.374996
return_std                           8.896748
average_reward                       0.784994
round_time             0 days 00:05:55.762543
episodes_test                           672.0
episode_length_test                 14.869048
returns_test                        11.877023
return_std_test                      9.440215
average_reward_test                  0.798842
round_time_test        0 days 00:00:08.899785
round_time_total       0 days 00:05:55.763643
loss_total                     1836758.210656
loss_critic                    2297107.528313
loss_actor                       -4639.227544
memory_size                         11533.093 

=== epoch 1/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:15,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  671
episode_length                       14.85544
returns                             11.764152
return_std                           9.845741
average_reward                       0.792638
round_time             0 days 00:05:54.381894
episodes_test                           661.0
episode_length_test                 15.125567
returns_test                         12.72542
return_std_test                     11.745353
average_reward_test                  0.841346
round_time_test        0 days 00:00:08.868609
round_time_total       0 days 00:05:54.383038
loss_total                     1302781.136125
loss_critic                    1629432.370406
loss_actor                       -3823.917498
memory_size                         11791.431 

=== epoch 1/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:57,  5.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  645
episode_length                      15.485271
returns                              12.51553
return_std                          11.254785
average_reward                       0.807381
round_time             0 days 00:05:57.528502
episodes_test                           588.0
episode_length_test                      17.0
returns_test                        14.623977
return_std_test                     12.376196
average_reward_test                  0.860212
round_time_test        0 days 00:00:08.871255
round_time_total       0 days 00:05:57.529609
loss_total                      946567.687703
loss_critic                    1184009.428094
loss_actor                       -3199.356301
memory_size                          12219.26 

=== epoch 1/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:01,  5.51it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  631
episode_length                      15.800317
returns                             13.032621
return_std                          12.018036
average_reward                       0.825214
round_time             0 days 00:05:55.858747
episodes_test                           626.0
episode_length_test                 15.958466
returns_test                        14.137495
return_std_test                      12.93075
average_reward_test                  0.885811
round_time_test        0 days 00:00:09.053086
round_time_total       0 days 00:05:55.859859
loss_total                      664367.828344
loss_critic                     831131.804313
loss_actor                       -2688.130207
memory_size                         12616.492 

=== epoch 1/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  617
episode_length                      16.147488
returns                             13.601913
return_std                          12.253005
average_reward                       0.843827
round_time             0 days 00:05:57.273907
episodes_test                           589.0
episode_length_test                 16.967742
returns_test                         16.19186
return_std_test                     15.902376
average_reward_test                  0.954339
round_time_test        0 days 00:00:08.777014
round_time_total       0 days 00:05:57.275126
loss_total                      481516.608047
loss_critic                     602474.206219
loss_actor                       -2313.825335
memory_size                         13090.754 

=== epoch 1/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  593
episode_length                      16.849916
returns                             15.084844
return_std                           14.60035
average_reward                       0.895282
round_time             0 days 00:05:57.032925
episodes_test                           587.0
episode_length_test                 17.013629
returns_test                        16.263351
return_std_test                     14.947733
average_reward_test                  0.955806
round_time_test        0 days 00:00:08.857082
round_time_total       0 days 00:05:57.034062
loss_total                      357842.025102
loss_critic                      447808.13725
loss_actor                       -2022.455165
memory_size                        13459.4575 

=== epoch 1/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  576
episode_length                      17.269097
returns                              16.50831
return_std                          16.094708
average_reward                       0.959298
round_time             0 days 00:05:56.956510
episodes_test                           594.0
episode_length_test                 16.819865
returns_test                        16.955228
return_std_test                     14.977798
average_reward_test                  1.007949
round_time_test        0 days 00:00:08.850824
round_time_total       0 days 00:05:56.957834
loss_total                      267487.674145
loss_critic                     334808.661914
loss_actor                       -1796.300831
memory_size                        13961.2265 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 2/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:41<00:00,  5.86it/s]
episodes                                  107
episode_length                      18.626168
returns                             20.078825
return_std                           18.26226
average_reward                       1.077489
round_time             0 days 00:05:41.426989
episodes_test                           560.0
episode_length_test                   17.8125
returns_test                         18.24202
return_std_test                     16.492088
average_reward_test                  1.023977
round_time_test        0 days 00:00:08.914755
round_time_total       0 days 00:05:41.428350
loss_total                      205981.033129
loss_critic                     257874.400133
loss_actor                       -1592.452093
memory_size                         14482.976 

=== epoch 2/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.85it/s]
episodes                                  222
episode_length                           18.0
returns                             19.245951
return_std                          18.805803
average_reward                       1.069044
round_time             0 days 00:05:42.539844
episodes_test                           563.0
episode_length_test                  17.73357
returns_test                        17.961152
return_std_test                      15.98544
average_reward_test                  1.012639
round_time_test        0 days 00:00:08.854835
round_time_total       0 days 00:05:42.540949
loss_total                      149082.629857
loss_critic                     186695.963262
loss_actor                       -1370.717734
memory_size                         15016.155 

=== epoch 2/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.83it/s]
episodes                                  327
episode_length                      18.281346
returns                             19.372979
return_std                          18.937103
average_reward                       1.061343
round_time             0 days 00:05:43.641364
episodes_test                           549.0
episode_length_test                 18.163934
returns_test                        19.301423
return_std_test                     16.375581
average_reward_test                  1.064073
round_time_test        0 days 00:00:08.815306
round_time_total       0 days 00:05:43.642690
loss_total                      105408.016193
loss_critic                     132047.748279
loss_actor                       -1150.921522
memory_size                        15529.9015 

=== epoch 2/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.83it/s]
episodes                                  434
episode_length                      18.407834
returns                              19.34943
return_std                          18.594309
average_reward                       1.051141
round_time             0 days 00:05:43.595738
episodes_test                           525.0
episode_length_test                 19.020952
returns_test                        20.315179
return_std_test                     17.602702
average_reward_test                  1.068228
round_time_test        0 days 00:00:08.752479
round_time_total       0 days 00:05:43.596851
loss_total                       74217.291723
loss_critic                      93010.754373
loss_actor                        -956.564669
memory_size                        16092.4335 

=== epoch 2/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:48,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  526
episode_length                      19.001901
returns                             20.737871
return_std                          19.286542
average_reward                       1.091268
round_time             0 days 00:05:45.457830
episodes_test                           468.0
episode_length_test                 21.363248
returns_test                        25.799932
return_std_test                     22.542691
average_reward_test                  1.207644
round_time_test        0 days 00:00:08.823737
round_time_total       0 days 00:05:45.459061
loss_total                       52318.169874
loss_critic                      65595.382359
loss_actor                        -790.684832
memory_size                         16687.823 

=== epoch 2/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:54,  5.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  516
episode_length                      19.313953
returns                             21.353467
return_std                          20.063768
average_reward                       1.106659
round_time             0 days 00:05:45.560244
episodes_test                           448.0
episode_length_test                 22.316964
returns_test                        27.373931
return_std_test                     23.171888
average_reward_test                  1.226528
round_time_test        0 days 00:00:08.690608
round_time_total       0 days 00:05:45.561558
loss_total                       34722.761921
loss_critic                      43564.152677
loss_actor                         -642.80422
memory_size                         17260.965 

=== epoch 2/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  495
episode_length                      20.187879
returns                             23.234489
return_std                          21.081922
average_reward                       1.150714
round_time             0 days 00:05:46.995942
episodes_test                           469.0
episode_length_test                 21.292111
returns_test                        27.081608
return_std_test                     24.860677
average_reward_test                  1.271855
round_time_test        0 days 00:00:08.744132
round_time_total       0 days 00:05:46.997045
loss_total                       23122.447863
loss_critic                      29035.631284
loss_actor                        -530.287688
memory_size                        17919.4745 

=== epoch 2/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  481
episode_length                      20.765073
returns                             24.820419
return_std                          22.038855
average_reward                       1.195056
round_time             0 days 00:05:47.780516
episodes_test                           490.0
episode_length_test                 20.385714
returns_test                        26.243508
return_std_test                     22.603816
average_reward_test                  1.287147
round_time_test        0 days 00:00:08.729819
round_time_total       0 days 00:05:47.781651
loss_total                       16941.745036
loss_critic                      21289.115269
loss_actor                        -447.737466
memory_size                         18657.705 

=== epoch 2/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  465
episode_length                      21.382796
returns                             26.798074
return_std                          23.658463
average_reward                       1.255692
round_time             0 days 00:05:49.670321
episodes_test                           481.0
episode_length_test                 20.765073
returns_test                        26.744443
return_std_test                     23.963181
average_reward_test                  1.287539
round_time_test        0 days 00:00:08.707575
round_time_total       0 days 00:05:49.671468
loss_total                       12781.680367
loss_critic                      16074.040987
loss_actor                        -387.763207
memory_size                         19323.128 

=== epoch 2/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  467
episode_length                      21.340471
returns                              26.99719
return_std                          23.946491
average_reward                       1.264751
round_time             0 days 00:05:51.645125
episodes_test                           488.0
episode_length_test                 20.456967
returns_test                        26.197613
return_std_test                     23.780165
average_reward_test                  1.280574
round_time_test        0 days 00:00:08.697770
round_time_total       0 days 00:05:51.646224
loss_total                       10475.632943
loss_critic                      13183.725119
loss_actor                        -356.736734
memory_size                         19967.485 

=== epoch 2/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  452
episode_length                      22.077434
returns                             28.978514
return_std                          25.779527
average_reward                       1.312851
round_time             0 days 00:05:50.586109
episodes_test                           421.0
episode_length_test                 23.743468
returns_test                        33.417128
return_std_test                     30.131928
average_reward_test                  1.407217
round_time_test        0 days 00:00:08.857199
round_time_total       0 days 00:05:50.587210
loss_total                        9180.862864
loss_critic                      11560.358895
loss_actor                        -337.122038
memory_size                        20708.3635 

=== epoch 2/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:53,  5.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  440
episode_length                      22.661364
returns                             30.030452
return_std                          26.177362
average_reward                       1.322807
round_time             0 days 00:05:51.378283
episodes_test                           434.0
episode_length_test                 23.011521
returns_test                        32.425527
return_std_test                     28.317552
average_reward_test                   1.40883
round_time_test        0 days 00:00:08.875752
round_time_total       0 days 00:05:51.379614
loss_total                        8548.475442
loss_critic                      10766.676387
loss_actor                        -324.329131
memory_size                         21485.546 

=== epoch 2/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  439
episode_length                      22.708428
returns                             30.431004
return_std                          26.159301
average_reward                       1.338229
round_time             0 days 00:05:51.866035
episodes_test                           374.0
episode_length_test                 26.679144
returns_test                        40.113851
return_std_test                     30.928904
average_reward_test                  1.503348
round_time_test        0 days 00:00:08.825133
round_time_total       0 days 00:05:51.867283
loss_total                        8133.466846
loss_critic                      10247.412995
loss_actor                        -322.318487
memory_size                         22252.748 

=== epoch 2/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  425
episode_length                      23.456471
returns                             31.811249
return_std                          26.444636
average_reward                       1.356461
round_time             0 days 00:05:51.707343
episodes_test                           377.0
episode_length_test                 26.509284
returns_test                        39.658506
return_std_test                     28.578637
average_reward_test                  1.495717
round_time_test        0 days 00:00:08.608414
round_time_total       0 days 00:05:51.708442
loss_total                        8150.782604
loss_critic                      10269.238091
loss_actor                        -323.040081
memory_size                         23020.059 

=== epoch 2/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<05:58,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  410
episode_length                      24.207317
returns                             33.305331
return_std                          26.928671
average_reward                       1.377899
round_time             0 days 00:05:50.423822
episodes_test                           416.0
episode_length_test                 24.024038
returns_test                        35.109342
return_std_test                     27.165988
average_reward_test                   1.46109
round_time_test        0 days 00:00:08.896274
round_time_total       0 days 00:05:50.424934
loss_total                        8084.931561
loss_critic                      10187.439002
loss_actor                        -325.098878
memory_size                        23818.2805 

=== epoch 2/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:35,  5.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  411
episode_length                      24.216545
returns                             33.622377
return_std                          28.459534
average_reward                       1.389078
round_time             0 days 00:05:49.241266
episodes_test                           365.0
episode_length_test                 27.347945
returns_test                        41.423553
return_std_test                     29.873374
average_reward_test                  1.514425
round_time_test        0 days 00:00:08.830735
round_time_total       0 days 00:05:49.242355
loss_total                        8197.189405
loss_critic                      10328.221991
loss_actor                        -326.941655
memory_size                        24658.6805 

=== epoch 2/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  401
episode_length                      24.882793
returns                             35.364691
return_std                          29.893668
average_reward                       1.421095
round_time             0 days 00:05:53.531543
episodes_test                           369.0
episode_length_test                 27.094851
returns_test                        39.855078
return_std_test                     33.402365
average_reward_test                  1.470835
round_time_test        0 days 00:00:08.677338
round_time_total       0 days 00:05:53.532682
loss_total                        8324.411255
loss_critic                      10488.192117
loss_actor                         -330.71287
memory_size                        25524.9475 

=== epoch 2/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:48,  5.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  399
episode_length                      24.974937
returns                             35.699992
return_std                          30.204796
average_reward                       1.426751
round_time             0 days 00:05:52.540619
episodes_test                           386.0
episode_length_test                 25.896373
returns_test                        39.031917
return_std_test                     30.473054
average_reward_test                  1.506977
round_time_test        0 days 00:00:08.618767
round_time_total       0 days 00:05:52.541822
loss_total                         8124.97244
loss_critic                      10239.888001
loss_actor                        -334.690558
memory_size                        26312.9635 

=== epoch 2/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  395
episode_length                       25.23038
returns                             36.122911
return_std                          30.829617
average_reward                       1.432393
round_time             0 days 00:05:54.407319
episodes_test                           392.0
episode_length_test                 25.451531
returns_test                        37.680825
return_std_test                     35.715703
average_reward_test                  1.479708
round_time_test        0 days 00:00:08.712086
round_time_total       0 days 00:05:54.408425
loss_total                        8465.903971
loss_critic                       10666.67489
loss_actor                        -337.180432
memory_size                         27075.125 

=== epoch 2/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  398
episode_length                      25.067839
returns                             35.875025
return_std                          31.532313
average_reward                       1.431228
round_time             0 days 00:05:54.995377
episodes_test                           396.0
episode_length_test                 25.222222
returns_test                        37.919371
return_std_test                     27.597396
average_reward_test                  1.502952
round_time_test        0 days 00:00:08.659559
round_time_total       0 days 00:05:54.996494
loss_total                        8588.788175
loss_critic                      10821.470113
loss_actor                        -341.940311
memory_size                         27969.854 

=== epoch 2/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:22,  5.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  384
episode_length                      25.914062
returns                             37.166607
return_std                          31.066121
average_reward                       1.435626
round_time             0 days 00:05:55.993024
episodes_test                           413.0
episode_length_test                 24.213075
returns_test                        34.976308
return_std_test                     33.107913
average_reward_test                  1.444522
round_time_test        0 days 00:00:08.594762
round_time_total       0 days 00:05:55.994162
loss_total                        8789.126612
loss_critic                      11072.660765
loss_actor                        -345.010791
memory_size                         28809.327 

=== epoch 2/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  380
episode_length                      26.213158
returns                             37.737456
return_std                          31.833973
average_reward                       1.439773
round_time             0 days 00:05:54.319768
episodes_test                           382.0
episode_length_test                 26.099476
returns_test                        38.901473
return_std_test                     35.479763
average_reward_test                  1.490731
round_time_test        0 days 00:00:08.705774
round_time_total       0 days 00:05:54.321280
loss_total                        8711.948017
loss_critic                      10976.940077
loss_actor                        -348.020947
memory_size                         29783.156 

=== epoch 2/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  377
episode_length                      26.461538
returns                               38.1778
return_std                          32.677784
average_reward                       1.442992
round_time             0 days 00:05:55.890907
episodes_test                           360.0
episode_length_test                 27.772222
returns_test                        42.589965
return_std_test                     34.306888
average_reward_test                  1.533431
round_time_test        0 days 00:00:08.673319
round_time_total       0 days 00:05:55.892526
loss_total                        8792.083181
loss_critic                      11077.728241
loss_actor                        -350.497759
memory_size                         30688.331 

=== epoch 2/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  383
episode_length                      26.052219
returns                             37.532108
return_std                          32.132619
average_reward                       1.441087
round_time             0 days 00:05:56.379081
episodes_test                           342.0
episode_length_test                 29.087719
returns_test                        45.744261
return_std_test                     36.834302
average_reward_test                  1.571573
round_time_test        0 days 00:00:08.508104
round_time_total       0 days 00:05:56.380238
loss_total                        8947.546612
loss_critic                      11273.094448
loss_actor                        -354.645559
memory_size                         31459.809 

=== epoch 2/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  380
episode_length                      26.234211
returns                              38.16233
return_std                           33.53579
average_reward                       1.454131
round_time             0 days 00:05:57.007301
episodes_test                           341.0
episode_length_test                 29.284457
returns_test                        45.887522
return_std_test                     43.667415
average_reward_test                  1.566138
round_time_test        0 days 00:00:08.567266
round_time_total       0 days 00:05:57.008400
loss_total                        9071.501925
loss_critic                      11428.519101
loss_actor                        -356.567631
memory_size                        32305.0745 

=== epoch 2/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  388
episode_length                      25.654639
returns                             37.111207
return_std                          32.530353
average_reward                       1.448747
round_time             0 days 00:05:54.811043
episodes_test                           302.0
episode_length_test                 33.059603
returns_test                        53.349177
return_std_test                     45.284691
average_reward_test                  1.612949
round_time_test        0 days 00:00:08.904149
round_time_total       0 days 00:05:54.812137
loss_total                        9065.164002
loss_critic                      11420.767672
loss_actor                        -357.251469
memory_size                        33176.3925 

=== epoch 2/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  396
episode_length                      25.065657
returns                             36.018636
return_std                          33.088193
average_reward                       1.441257
round_time             0 days 00:05:56.791379
episodes_test                           330.0
episode_length_test                 30.275758
returns_test                        48.019052
return_std_test                     42.120279
average_reward_test                  1.585585
round_time_test        0 days 00:00:08.709601
round_time_total       0 days 00:05:56.792496
loss_total                        8890.256424
loss_critic                      11202.980062
loss_actor                        -360.638892
memory_size                         34091.375 

=== epoch 2/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  403
episode_length                      24.754342
returns                             35.284461
return_std                          32.739572
average_reward                       1.425768
round_time             0 days 00:05:57.980865
episodes_test                           282.0
episode_length_test                 35.446809
returns_test                        57.695911
return_std_test                     44.103483
average_reward_test                  1.627434
round_time_test        0 days 00:00:08.445684
round_time_total       0 days 00:05:57.981973
loss_total                        9110.866096
loss_critic                      11478.698799
loss_actor                        -360.465415
memory_size                         34905.579 

=== epoch 2/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:28,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  396
episode_length                      25.207071
returns                             36.290448
return_std                          35.562566
average_reward                       1.439893
round_time             0 days 00:05:56.034863
episodes_test                           312.0
episode_length_test                 32.044872
returns_test                        50.300093
return_std_test                     41.753823
average_reward_test                  1.569551
round_time_test        0 days 00:00:08.663241
round_time_total       0 days 00:05:56.035958
loss_total                         9129.17007
loss_critic                      11502.452034
loss_actor                        -363.958615
memory_size                         35786.101 

=== epoch 2/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  378
episode_length                      26.298942
returns                             38.195418
return_std                          35.530495
average_reward                       1.454131
round_time             0 days 00:05:57.966320
episodes_test                           307.0
episode_length_test                 32.364821
returns_test                        50.440504
return_std_test                     41.646734
average_reward_test                  1.559458
round_time_test        0 days 00:00:08.628282
round_time_total       0 days 00:05:57.967598
loss_total                        9137.954566
loss_critic                      11513.612103
loss_actor                        -364.676311
memory_size                        36792.0245 

=== epoch 2/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  371
episode_length                      26.908356
returns                             39.390956
return_std                          37.796037
average_reward                       1.463569
round_time             0 days 00:05:58.113547
episodes_test                           296.0
episode_length_test                 33.773649
returns_test                        53.260104
return_std_test                     40.042745
average_reward_test                   1.57678
round_time_test        0 days 00:00:08.650586
round_time_total       0 days 00:05:58.114652
loss_total                        9171.520026
loss_critic                      11555.759978
loss_actor                        -365.440558
memory_size                        37780.7695 

=== epoch 2/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:42,  5.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  373
episode_length                      26.697051
returns                             38.864014
return_std                          37.628505
average_reward                       1.456944
round_time             0 days 00:05:56.341491
episodes_test                           324.0
episode_length_test                 30.808642
returns_test                        49.007805
return_std_test                       31.1888
average_reward_test                  1.590279
round_time_test        0 days 00:00:08.587125
round_time_total       0 days 00:05:56.342564
loss_total                        9057.028051
loss_critic                      11413.208109
loss_actor                         -367.69298
memory_size                         38784.685 

=== epoch 2/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  360
episode_length                      27.580556
returns                             40.677746
return_std                          37.955317
average_reward                       1.476791
round_time             0 days 00:05:57.463082
episodes_test                           330.0
episode_length_test                 30.236364
returns_test                        47.955454
return_std_test                     36.363623
average_reward_test                  1.585962
round_time_test        0 days 00:00:08.569126
round_time_total       0 days 00:05:57.464183
loss_total                          9218.9718
loss_critic                      11615.501258
loss_actor                        -367.146804
memory_size                         39616.738 

=== epoch 2/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:06,  5.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  370
episode_length                      26.954054
returns                             39.129877
return_std                          35.292697
average_reward                       1.451666
round_time             0 days 00:05:58.562729
episodes_test                           336.0
episode_length_test                 29.672619
returns_test                        46.730623
return_std_test                     35.428339
average_reward_test                  1.574965
round_time_test        0 days 00:00:08.701709
round_time_total       0 days 00:05:58.563828
loss_total                        9129.561909
loss_critic                      11504.021885
loss_actor                         -368.27875
memory_size                         40469.262 

=== epoch 2/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  381
episode_length                      26.133858
returns                             37.654271
return_std                          35.337753
average_reward                        1.44024
round_time             0 days 00:05:58.637694
episodes_test                           300.0
episode_length_test                 33.266667
returns_test                        51.287958
return_std_test                     35.488663
average_reward_test                  1.541497
round_time_test        0 days 00:00:08.656712
round_time_total       0 days 00:05:58.638809
loss_total                        9109.068992
loss_critic                      11478.779444
loss_actor                        -369.773659
memory_size                         41388.734 

=== epoch 2/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  395
episode_length                      25.260759
returns                             35.571118
return_std                          32.756444
average_reward                       1.408312
round_time             0 days 00:06:00.541527
episodes_test                           325.0
episode_length_test                 30.716923
returns_test                         48.91553
return_std_test                     31.929763
average_reward_test                  1.591289
round_time_test        0 days 00:00:08.633437
round_time_total       0 days 00:06:00.542622
loss_total                        9134.342145
loss_critic                      11510.686067
loss_actor                        -371.034342
memory_size                         42311.129 

=== epoch 2/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  394
episode_length                      25.352792
returns                             35.742148
return_std                          32.121899
average_reward                       1.410013
round_time             0 days 00:05:58.574652
episodes_test                           353.0
episode_length_test                 28.300283
returns_test                        43.457372
return_std_test                     30.640223
average_reward_test                  1.535077
round_time_test        0 days 00:00:08.742047
round_time_total       0 days 00:05:58.575979
loss_total                        9209.955451
loss_critic                      11604.959013
loss_actor                        -370.059562
memory_size                        43144.6115 

=== epoch 2/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  387
episode_length                      25.744186
returns                             36.604272
return_std                          33.692198
average_reward                       1.422056
round_time             0 days 00:05:59.828877
episodes_test                           319.0
episode_length_test                 31.316614
returns_test                        49.822667
return_std_test                     31.956712
average_reward_test                   1.59028
round_time_test        0 days 00:00:08.719103
round_time_total       0 days 00:05:59.829982
loss_total                        8888.086605
loss_critic                      11201.775034
loss_actor                        -366.667829
memory_size                         44103.106 

=== epoch 2/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:17,  5.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  371
episode_length                      26.800539
returns                              38.85923
return_std                          35.779222
average_reward                       1.450358
round_time             0 days 00:05:58.331544
episodes_test                           320.0
episode_length_test                 31.109375
returns_test                        50.194855
return_std_test                     33.265126
average_reward_test                  1.613802
round_time_test        0 days 00:00:08.535673
round_time_total       0 days 00:05:58.332629
loss_total                        8646.691061
loss_critic                      10899.804839
loss_actor                        -365.764858
memory_size                         45106.204 

=== epoch 2/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:46,  4.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  373
episode_length                      26.605898
returns                             38.460192
return_std                          36.037353
average_reward                       1.448847
round_time             0 days 00:06:00.084005
episodes_test                           347.0
episode_length_test                 28.806916
returns_test                        44.861508
return_std_test                     28.528103
average_reward_test                  1.557088
round_time_test        0 days 00:00:08.683251
round_time_total       0 days 00:06:00.085186
loss_total                        8878.046002
loss_critic                       11189.26429
loss_actor                         -366.82791
memory_size                        46080.3895 

=== epoch 2/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  369
episode_length                       27.02981
returns                             39.872362
return_std                          37.033195
average_reward                       1.475261
round_time             0 days 00:05:58.262902
episodes_test                           349.0
episode_length_test                 28.610315
returns_test                        45.132371
return_std_test                     35.115897
average_reward_test                  1.577198
round_time_test        0 days 00:00:08.719359
round_time_total       0 days 00:05:58.264104
loss_total                        8952.592811
loss_critic                      11282.569799
loss_actor                         -367.31593
memory_size                        47000.0665 

=== epoch 2/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  362
episode_length                      27.356354
returns                             40.588806
return_std                          37.256238
average_reward                       1.487873
round_time             0 days 00:05:58.640977
episodes_test                           336.0
episode_length_test                 29.633929
returns_test                        46.849763
return_std_test                     33.969003
average_reward_test                  1.581008
round_time_test        0 days 00:00:08.682321
round_time_total       0 days 00:05:58.642214
loss_total                        8737.685482
loss_critic                      11014.184099
loss_actor                        -368.309703
memory_size                        47943.0945 

=== epoch 2/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:46,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  371
episode_length                      26.857143
returns                             39.631563
return_std                          36.618118
average_reward                       1.476098
round_time             0 days 00:05:57.128282
episodes_test                           362.0
episode_length_test                 27.582873
returns_test                        43.971738
return_std_test                     31.822038
average_reward_test                  1.593612
round_time_test        0 days 00:00:08.723118
round_time_total       0 days 00:05:57.129469
loss_total                        8741.819554
loss_critic                      11019.780596
loss_actor                        -370.025432
memory_size                        48871.4515 

=== epoch 2/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:43,  4.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  395
episode_length                      25.283544
returns                             36.696975
return_std                          34.507602
average_reward                       1.451225
round_time             0 days 00:06:00.624286
episodes_test                           357.0
episode_length_test                 28.005602
returns_test                         43.95614
return_std_test                     24.956165
average_reward_test                  1.569408
round_time_test        0 days 00:00:09.264277
round_time_total       0 days 00:06:00.625392
loss_total                        8809.020226
loss_critic                      11103.200931
loss_actor                        -367.703467
memory_size                         49640.888 

=== epoch 2/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:01,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.54it/s]
episodes                                  402
episode_length                      24.823383
returns                             35.807028
return_std                          33.311451
average_reward                         1.4437
round_time             0 days 00:06:01.313873
episodes_test                           356.0
episode_length_test                 28.042135
returns_test                        43.881611
return_std_test                     24.162686
average_reward_test                  1.564453
round_time_test        0 days 00:00:08.795555
round_time_total       0 days 00:06:01.314969
loss_total                        8713.427312
loss_critic                      10983.579733
loss_actor                        -367.183174
memory_size                        50491.2515 

=== epoch 2/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  403
episode_length                       24.73201
returns                             35.758183
return_std                          32.722889
average_reward                       1.447931
round_time             0 days 00:06:02.130086
episodes_test                           367.0
episode_length_test                 27.144414
returns_test                        41.645167
return_std_test                     32.397296
average_reward_test                  1.534282
round_time_test        0 days 00:00:08.742298
round_time_total       0 days 00:06:02.131208
loss_total                        8606.914493
loss_critic                      10850.201448
loss_actor                        -366.234051
memory_size                         51340.778 

=== epoch 2/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  416
episode_length                      23.995192
returns                             34.405907
return_std                           30.95542
average_reward                       1.433459
round_time             0 days 00:05:59.787219
episodes_test                           335.0
episode_length_test                 29.773134
returns_test                        46.892321
return_std_test                     33.733848
average_reward_test                  1.575128
round_time_test        0 days 00:00:08.592995
round_time_total       0 days 00:05:59.788317
loss_total                        8626.556317
loss_critic                      10875.058484
loss_actor                        -367.453088
memory_size                        52193.6595 

=== epoch 2/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  404
episode_length                      24.613861
returns                             35.775741
return_std                          32.327971
average_reward                       1.453862
round_time             0 days 00:05:59.160209
episodes_test                           350.0
episode_length_test                 28.462857
returns_test                        43.154015
return_std_test                      29.89094
average_reward_test                  1.516479
round_time_test        0 days 00:00:08.644495
round_time_total       0 days 00:05:59.161474
loss_total                        8774.660055
loss_critic                      11060.725448
loss_actor                        -369.602296
memory_size                         53058.659 

=== epoch 2/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  387
episode_length                      25.751938
returns                              38.17552
return_std                          34.445802
average_reward                        1.48228
round_time             0 days 00:06:02.001696
episodes_test                           351.0
episode_length_test                 28.481481
returns_test                        43.621414
return_std_test                     25.749542
average_reward_test                  1.531395
round_time_test        0 days 00:00:08.703455
round_time_total       0 days 00:06:02.003029
loss_total                        8572.932445
loss_critic                      10808.353157
loss_actor                        -368.751133
memory_size                        54049.0365 

=== epoch 2/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  369
episode_length                      27.081301
returns                             40.724672
return_std                          35.318862
average_reward                       1.503238
round_time             0 days 00:06:01.197249
episodes_test                           387.0
episode_length_test                 25.806202
returns_test                        38.763497
return_std_test                      29.71942
average_reward_test                  1.501484
round_time_test        0 days 00:00:08.884403
round_time_total       0 days 00:06:01.198369
loss_total                        8814.189828
loss_critic                      11109.939001
loss_actor                        -368.807567
memory_size                         55097.636 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 3/10 ===== round 1/50 ======================================
  1%|          | 12/2000 [00:02<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:38<00:00,  5.90it/s]
episodes                                   60
episode_length                      32.333333
returns                             51.599758
return_std                          40.852064
average_reward                       1.592079
round_time             0 days 00:05:39.023222
episodes_test                           361.0
episode_length_test                 27.567867
returns_test                        42.094979
return_std_test                     29.176404
average_reward_test                  1.527428
round_time_test        0 days 00:00:08.830822
round_time_total       0 days 00:05:39.024352
loss_total                        8670.456703
loss_critic                      10930.635645
loss_actor                        -370.259838
memory_size                         56195.632 

=== epoch 3/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.81it/s]
episodes                                  128
episode_length                          31.25
returns                             49.188277
return_std                          42.975328
average_reward                       1.574025
round_time             0 days 00:05:44.539508
episodes_test                           322.0
episode_length_test                 31.052795
returns_test                        47.904494
return_std_test                     26.488461
average_reward_test                  1.542624
round_time_test        0 days 00:00:08.683309
round_time_total       0 days 00:05:44.540583
loss_total                        8707.908652
loss_critic                      10977.604452
loss_actor                        -370.875288
memory_size                        57273.6405 

=== epoch 3/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.83it/s]
episodes                                  202
episode_length                      29.658416
returns                             46.090304
return_std                          39.837735
average_reward                       1.553329
round_time             0 days 00:05:43.620469
episodes_test                           292.0
episode_length_test                 34.116438
returns_test                        51.810394
return_std_test                     30.889068
average_reward_test                  1.518731
round_time_test        0 days 00:00:08.802249
round_time_total       0 days 00:05:43.621925
loss_total                        8707.617937
loss_critic                      10976.728751
loss_actor                        -368.826061
memory_size                        58219.3825 

=== epoch 3/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:10,  4.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.83it/s]
episodes                                  274
episode_length                      29.178832
returns                             45.138449
return_std                          38.765298
average_reward                       1.546593
round_time             0 days 00:05:43.502285
episodes_test                           368.0
episode_length_test                 27.105978
returns_test                        42.546317
return_std_test                     22.641536
average_reward_test                  1.569777
round_time_test        0 days 00:00:08.827086
round_time_total       0 days 00:05:43.503425
loss_total                        8362.679999
loss_critic                      10545.609548
loss_actor                        -369.038922
memory_size                        59200.5395 

=== epoch 3/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:59,  5.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                  341
episode_length                      29.237537
returns                             45.276929
return_std                          38.886782
average_reward                       1.548795
round_time             0 days 00:05:44.070386
episodes_test                           355.0
episode_length_test                 28.126761
returns_test                        43.473842
return_std_test                     22.832272
average_reward_test                  1.545242
round_time_test        0 days 00:00:08.620774
round_time_total       0 days 00:05:44.071932
loss_total                        8743.538715
loss_critic                      11022.240938
loss_actor                        -371.270907
memory_size                         60171.451 

=== epoch 3/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:25,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                  358
episode_length                       27.77933
returns                             42.218991
return_std                          36.597554
average_reward                       1.521634
round_time             0 days 00:05:46.369313
episodes_test                           333.0
episode_length_test                 29.921922
returns_test                        47.453462
return_std_test                     26.721178
average_reward_test                  1.585804
round_time_test        0 days 00:00:08.813686
round_time_total       0 days 00:05:46.370603
loss_total                        8661.169299
loss_critic                      10919.226543
loss_actor                         -371.06044
memory_size                        61169.7405 

=== epoch 3/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:51,  5.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.78it/s]
episodes                                  368
episode_length                      27.171196
returns                             40.942716
return_std                          35.107297
average_reward                       1.506791
round_time             0 days 00:05:46.764473
episodes_test                           340.0
episode_length_test                 29.355882
returns_test                        45.934657
return_std_test                     28.225232
average_reward_test                  1.564125
round_time_test        0 days 00:00:08.696277
round_time_total       0 days 00:05:46.765752
loss_total                        8583.910555
loss_critic                      10822.370941
loss_actor                        -369.931706
memory_size                        62062.1155 

=== epoch 3/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                  355
episode_length                      27.974648
returns                             42.570534
return_std                          37.007721
average_reward                       1.522419
round_time             0 days 00:05:48.572488
episodes_test                           288.0
episode_length_test                 34.618056
returns_test                        54.177838
return_std_test                     26.650545
average_reward_test                  1.565168
round_time_test        0 days 00:00:08.750714
round_time_total       0 days 00:05:48.573711
loss_total                        8373.756603
loss_critic                      10559.667935
loss_actor                        -369.889454
memory_size                         63066.911 

=== epoch 3/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  350
episode_length                      28.414286
returns                             43.452013
return_std                          37.861701
average_reward                       1.529602
round_time             0 days 00:05:47.814525
episodes_test                           343.0
episode_length_test                 29.148688
returns_test                        44.803794
return_std_test                     27.324781
average_reward_test                  1.536963
round_time_test        0 days 00:00:08.611202
round_time_total       0 days 00:05:47.815646
loss_total                        8527.721195
loss_critic                      10751.963876
loss_actor                        -369.250308
memory_size                         64088.999 

=== epoch 3/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:52,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  364
episode_length                       27.35989
returns                             41.183261
return_std                          36.358982
average_reward                       1.505818
round_time             0 days 00:05:48.199243
episodes_test                           318.0
episode_length_test                 31.396226
returns_test                        49.893692
return_std_test                     31.594683
average_reward_test                  1.588754
round_time_test        0 days 00:00:08.660149
round_time_total       0 days 00:05:48.200347
loss_total                        8475.081575
loss_critic                      10685.848006
loss_actor                        -367.984848
memory_size                        64999.5515 

=== epoch 3/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:48,  5.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                  350
episode_length                      28.517143
returns                             43.528638
return_std                          37.212279
average_reward                       1.526901
round_time             0 days 00:05:49.485707
episodes_test                           316.0
episode_length_test                 31.629747
returns_test                        50.067352
return_std_test                     26.168078
average_reward_test                  1.582577
round_time_test        0 days 00:00:08.645193
round_time_total       0 days 00:05:49.486796
loss_total                        8457.670697
loss_critic                      10664.119114
loss_actor                         -368.12372
memory_size                        65986.0275 

=== epoch 3/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  342
episode_length                      28.994152
returns                             44.158807
return_std                          36.603525
average_reward                       1.524286
round_time             0 days 00:05:47.952356
episodes_test                           363.0
episode_length_test                 27.509642
returns_test                        42.550429
return_std_test                     26.785297
average_reward_test                  1.546206
round_time_test        0 days 00:00:08.815783
round_time_total       0 days 00:05:47.953465
loss_total                        8480.007306
loss_critic                       10692.28262
loss_actor                        -369.094669
memory_size                        66998.2265 

=== epoch 3/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  348
episode_length                      28.683908
returns                             43.165964
return_std                          35.578245
average_reward                       1.506322
round_time             0 days 00:05:49.199213
episodes_test                           344.0
episode_length_test                 29.069767
returns_test                        45.841099
return_std_test                     22.327715
average_reward_test                  1.576934
round_time_test        0 days 00:00:08.865186
round_time_total       0 days 00:05:49.200325
loss_total                        8543.496901
loss_critic                      10771.820971
loss_actor                        -369.800119
memory_size                        68004.8675 

=== epoch 3/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:17,  5.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  356
episode_length                      28.078652
returns                             41.953257
return_std                          36.391999
average_reward                       1.494046
round_time             0 days 00:05:49.184005
episodes_test                           308.0
episode_length_test                 32.457792
returns_test                        51.057916
return_std_test                     27.579424
average_reward_test                  1.572898
round_time_test        0 days 00:00:08.668241
round_time_total       0 days 00:05:49.185140
loss_total                          8433.5566
loss_critic                       10634.06659
loss_actor                        -368.484096
memory_size                         68985.801 

=== epoch 3/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  345
episode_length                      28.849275
returns                             43.213773
return_std                          36.769404
average_reward                       1.498958
round_time             0 days 00:05:51.382982
episodes_test                           372.0
episode_length_test                 26.860215
returns_test                        41.454505
return_std_test                     29.276759
average_reward_test                  1.543012
round_time_test        0 days 00:00:08.680216
round_time_total       0 days 00:05:51.384075
loss_total                        8328.954629
loss_critic                      10502.612578
loss_actor                         -365.67788
memory_size                        69948.1685 

=== epoch 3/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:41,  5.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  353
episode_length                       28.17847
returns                             41.734337
return_std                          37.477113
average_reward                       1.482139
round_time             0 days 00:05:49.775578
episodes_test                           333.0
episode_length_test                 29.987988
returns_test                        47.348369
return_std_test                     34.180426
average_reward_test                  1.578112
round_time_test        0 days 00:00:08.790273
round_time_total       0 days 00:05:49.776758
loss_total                        8162.901266
loss_critic                      10294.425619
loss_actor                         -363.19687
memory_size                        70876.1535 

=== epoch 3/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  358
episode_length                      27.888268
returns                             41.624033
return_std                          38.285509
average_reward                       1.491811
round_time             0 days 00:05:53.249935
episodes_test                           301.0
episode_length_test                 33.126246
returns_test                        51.926967
return_std_test                     27.089258
average_reward_test                  1.567477
round_time_test        0 days 00:00:08.520497
round_time_total       0 days 00:05:53.251419
loss_total                        8240.085167
loss_critic                      10391.274981
loss_actor                        -364.674794
memory_size                        71866.5455 

=== epoch 3/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:57,  5.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  364
episode_length                      27.052198
returns                             40.272915
return_std                          36.341906
average_reward                        1.49395
round_time             0 days 00:05:52.001914
episodes_test                           356.0
episode_length_test                 28.081461
returns_test                        43.401125
return_std_test                     31.062701
average_reward_test                  1.545385
round_time_test        0 days 00:00:08.749304
round_time_total       0 days 00:05:52.003020
loss_total                        8207.051994
loss_critic                      10350.348505
loss_actor                         -366.13475
memory_size                        72796.4975 

=== epoch 3/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  374
episode_length                       26.59893
returns                             39.059097
return_std                           35.13718
average_reward                       1.469337
round_time             0 days 00:05:50.798937
episodes_test                           379.0
episode_length_test                 26.364116
returns_test                        41.587188
return_std_test                     28.648201
average_reward_test                  1.576942
round_time_test        0 days 00:00:08.661723
round_time_total       0 days 00:05:50.800471
loss_total                        8389.894857
loss_critic                       10579.13345
loss_actor                         -367.06029
memory_size                        73668.1225 

=== epoch 3/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:36,  5.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  378
episode_length                      26.351852
returns                             38.515022
return_std                          35.246438
average_reward                       1.462317
round_time             0 days 00:05:53.046223
episodes_test                           322.0
episode_length_test                 31.024845
returns_test                        48.919548
return_std_test                     31.520352
average_reward_test                  1.576257
round_time_test        0 days 00:00:08.703489
round_time_total       0 days 00:05:53.047708
loss_total                        8342.707971
loss_critic                      10519.282761
loss_actor                        -363.591988
memory_size                        74533.8265 

=== epoch 3/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  372
episode_length                       26.77957
returns                             39.387301
return_std                          34.882681
average_reward                       1.470755
round_time             0 days 00:05:53.192574
episodes_test                           323.0
episode_length_test                 30.959752
returns_test                         48.83148
return_std_test                     28.219565
average_reward_test                  1.577257
round_time_test        0 days 00:00:08.919319
round_time_total       0 days 00:05:53.193757
loss_total                        8173.020059
loss_critic                      10307.356686
loss_actor                        -364.327141
memory_size                        75558.6825 

=== epoch 3/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:43,  5.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  368
episode_length                      27.057065
returns                             39.435654
return_std                          34.994985
average_reward                       1.459677
round_time             0 days 00:05:53.381107
episodes_test                           330.0
episode_length_test                 30.284848
returns_test                        47.841402
return_std_test                     26.744704
average_reward_test                  1.579387
round_time_test        0 days 00:00:08.601882
round_time_total       0 days 00:05:53.382275
loss_total                        8173.390363
loss_critic                      10308.036848
loss_actor                        -365.196323
memory_size                         76519.904 

=== epoch 3/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  363
episode_length                      27.487603
returns                             40.088508
return_std                          36.726671
average_reward                       1.459561
round_time             0 days 00:05:52.568683
episodes_test                           346.0
episode_length_test                 28.872832
returns_test                        44.649348
return_std_test                     27.930699
average_reward_test                  1.545875
round_time_test        0 days 00:00:08.652131
round_time_total       0 days 00:05:52.569796
loss_total                        8094.702915
loss_critic                      10209.446348
loss_actor                        -364.271541
memory_size                          77520.25 

=== epoch 3/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  335
episode_length                      29.713433
returns                             44.597546
return_std                          38.236685
average_reward                       1.501934
round_time             0 days 00:05:54.207142
episodes_test                           313.0
episode_length_test                 31.827476
returns_test                        50.305225
return_std_test                     24.957159
average_reward_test                  1.580786
round_time_test        0 days 00:00:08.786858
round_time_total       0 days 00:05:54.208462
loss_total                        8035.797104
loss_critic                      10136.527595
loss_actor                        -367.125605
memory_size                         78564.396 

=== epoch 3/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  324
episode_length                      30.753086
returns                             46.436365
return_std                          38.155322
average_reward                       1.510026
round_time             0 days 00:05:54.100796
episodes_test                           322.0
episode_length_test                 31.040373
returns_test                        48.814364
return_std_test                     24.225878
average_reward_test                  1.572293
round_time_test        0 days 00:00:08.567775
round_time_total       0 days 00:05:54.101944
loss_total                        8327.751495
loss_critic                      10500.972275
loss_actor                        -365.132355
memory_size                        79651.8375 

=== epoch 3/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:48,  5.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  334
episode_length                      29.835329
returns                             44.308503
return_std                          37.329258
average_reward                       1.485949
round_time             0 days 00:05:53.874894
episodes_test                           319.0
episode_length_test                  31.31348
returns_test                        49.484831
return_std_test                     24.537872
average_reward_test                  1.579696
round_time_test        0 days 00:00:08.819171
round_time_total       0 days 00:05:53.876423
loss_total                        7994.437175
loss_critic                      10084.498344
loss_actor                        -365.808173
memory_size                         80608.235 

=== epoch 3/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  340
episode_length                      29.391176
returns                             43.778072
return_std                          36.586736
average_reward                       1.489042
round_time             0 days 00:05:54.591250
episodes_test                           318.0
episode_length_test                 31.421384
returns_test                        50.216575
return_std_test                     33.396173
average_reward_test                  1.597656
round_time_test        0 days 00:00:08.882128
round_time_total       0 days 00:05:54.592340
loss_total                        8118.389283
loss_critic                      10239.147652
loss_actor                        -364.644885
memory_size                        81564.4145 

=== epoch 3/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:48,  4.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  336
episode_length                      29.642857
returns                             44.230497
return_std                          35.166754
average_reward                       1.492651
round_time             0 days 00:05:55.923234
episodes_test                           290.0
episode_length_test                 34.441379
returns_test                        55.378135
return_std_test                     35.635652
average_reward_test                  1.607281
round_time_test        0 days 00:00:08.604123
round_time_total       0 days 00:05:55.924397
loss_total                        8083.332129
loss_critic                      10194.984118
loss_actor                        -363.276535
memory_size                        82530.2025 

=== epoch 3/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  338
episode_length                      29.423077
returns                             43.965176
return_std                          35.842072
average_reward                       1.494219
round_time             0 days 00:05:56.182021
episodes_test                           315.0
episode_length_test                 31.746032
returns_test                        50.281666
return_std_test                     21.440613
average_reward_test                  1.583872
round_time_test        0 days 00:00:08.787976
round_time_total       0 days 00:05:56.183127
loss_total                        8012.012067
loss_critic                      10106.447677
loss_actor                        -365.731064
memory_size                         83605.071 

=== epoch 3/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  335
episode_length                      29.710448
returns                             45.246423
return_std                          36.297279
average_reward                       1.523329
round_time             0 days 00:05:55.318808
episodes_test                           310.0
episode_length_test                 32.167742
returns_test                        50.555375
return_std_test                      30.61026
average_reward_test                  1.571594
round_time_test        0 days 00:00:08.509164
round_time_total       0 days 00:05:55.319931
loss_total                        8160.498787
loss_critic                      10291.902512
loss_actor                        -365.116845
memory_size                        84726.5325 

=== epoch 3/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  319
episode_length                      31.250784
returns                             48.627929
return_std                          38.884318
average_reward                       1.556513
round_time             0 days 00:05:57.560654
episodes_test                           331.0
episode_length_test                 30.172205
returns_test                         48.08353
return_std_test                     22.083318
average_reward_test                  1.593365
round_time_test        0 days 00:00:08.649987
round_time_total       0 days 00:05:57.561757
loss_total                        7975.600051
loss_critic                      10059.845085
loss_actor                        -361.380756
memory_size                        85781.9545 

=== epoch 3/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  314
episode_length                      31.732484
returns                              49.56406
return_std                          38.896053
average_reward                       1.560998
round_time             0 days 00:05:54.542219
episodes_test                           322.0
episode_length_test                 31.043478
returns_test                        48.725815
return_std_test                     26.452793
average_reward_test                  1.569353
round_time_test        0 days 00:00:08.685328
round_time_total       0 days 00:05:54.543315
loss_total                        7927.018622
loss_critic                       9999.440252
loss_actor                        -362.668583
memory_size                         86851.469 

=== epoch 3/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:20,  4.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  309
episode_length                      32.190939
returns                             50.408001
return_std                          39.484255
average_reward                       1.566899
round_time             0 days 00:05:58.356009
episodes_test                           330.0
episode_length_test                 30.169697
returns_test                        48.178038
return_std_test                     25.200742
average_reward_test                  1.597295
round_time_test        0 days 00:00:08.950315
round_time_total       0 days 00:05:58.357103
loss_total                        7915.811365
loss_critic                       9985.272719
loss_actor                        -362.034676
memory_size                         87816.928 

=== epoch 3/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  309
episode_length                       32.31068
returns                             50.861057
return_std                          39.616807
average_reward                       1.572997
round_time             0 days 00:05:55.972119
episodes_test                           307.0
episode_length_test                 32.566775
returns_test                        51.059807
return_std_test                     26.571499
average_reward_test                  1.567742
round_time_test        0 days 00:00:08.743177
round_time_total       0 days 00:05:55.973395
loss_total                        7857.138998
loss_critic                       9911.831198
loss_actor                        -361.630471
memory_size                         88940.621 

=== epoch 3/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:56,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  304
episode_length                      32.582237
returns                             51.144048
return_std                          39.739797
average_reward                       1.571476
round_time             0 days 00:05:55.043388
episodes_test                           315.0
episode_length_test                 31.669841
returns_test                        49.705426
return_std_test                      22.67202
average_reward_test                   1.56924
round_time_test        0 days 00:00:08.784181
round_time_total       0 days 00:05:55.044476
loss_total                        7845.111817
loss_critic                       9896.854055
loss_actor                        -361.857813
memory_size                        90040.4335 

=== epoch 3/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  307
episode_length                      32.429967
returns                              50.74362
return_std                          38.310585
average_reward                       1.564915
round_time             0 days 00:05:57.638391
episodes_test                           292.0
episode_length_test                 34.174658
returns_test                        54.238195
return_std_test                     25.216326
average_reward_test                  1.587014
round_time_test        0 days 00:00:08.647032
round_time_total       0 days 00:05:57.639643
loss_total                        7784.849693
loss_critic                       9821.460558
loss_actor                        -361.594481
memory_size                        91154.3405 

=== epoch 3/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:36,  5.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  295
episode_length                      33.816949
returns                             53.889409
return_std                          41.371205
average_reward                       1.591752
round_time             0 days 00:05:56.860264
episodes_test                           311.0
episode_length_test                 32.115756
returns_test                        50.995198
return_std_test                     24.613873
average_reward_test                  1.587187
round_time_test        0 days 00:00:08.695701
round_time_total       0 days 00:05:56.861404
loss_total                        7955.398602
loss_critic                      10035.285311
loss_actor                        -364.148931
memory_size                         92235.598 

=== epoch 3/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  299
episode_length                      33.431438
returns                             53.361407
return_std                          41.506876
average_reward                       1.596094
round_time             0 days 00:05:58.661735
episodes_test                           313.0
episode_length_test                 31.913738
returns_test                        50.574807
return_std_test                     22.558993
average_reward_test                   1.58421
round_time_test        0 days 00:00:08.682779
round_time_total       0 days 00:05:58.662838
loss_total                        7921.368819
loss_critic                       9993.324124
loss_actor                        -366.453026
memory_size                        93309.2005 

=== epoch 3/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:40,  5.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  297
episode_length                      33.444444
returns                             53.144534
return_std                          40.012232
average_reward                       1.590058
round_time             0 days 00:05:58.197496
episodes_test                           295.0
episode_length_test                 33.783051
returns_test                        53.743033
return_std_test                     21.529773
average_reward_test                  1.591133
round_time_test        0 days 00:00:08.783435
round_time_total       0 days 00:05:58.198590
loss_total                        8050.566497
loss_critic                      10154.541744
loss_actor                         -365.33519
memory_size                         94398.389 

=== epoch 3/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  316
episode_length                      31.550633
returns                             49.667705
return_std                          39.142896
average_reward                       1.574177
round_time             0 days 00:05:56.906423
episodes_test                           312.0
episode_length_test                 31.955128
returns_test                        50.960625
return_std_test                     20.786994
average_reward_test                  1.594859
round_time_test        0 days 00:00:08.750606
round_time_total       0 days 00:05:56.907532
loss_total                        7834.242552
loss_critic                       9884.519593
loss_actor                        -366.866301
memory_size                         95412.847 

=== epoch 3/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:09,  4.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  323
episode_length                       30.77709
returns                             48.125055
return_std                           38.35368
average_reward                        1.56512
round_time             0 days 00:05:57.206594
episodes_test                           356.0
episode_length_test                 28.025281
returns_test                        43.594506
return_std_test                       22.1669
average_reward_test                   1.55534
round_time_test        0 days 00:00:08.878416
round_time_total       0 days 00:05:57.207748
loss_total                        8028.856086
loss_critic                      10126.916656
loss_actor                          -363.3869
memory_size                         96349.786 

=== epoch 3/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  328
episode_length                      30.347561
returns                             46.953318
return_std                          35.200941
average_reward                       1.548576
round_time             0 days 00:05:56.397559
episodes_test                           309.0
episode_length_test                 32.300971
returns_test                        51.427239
return_std_test                     25.376775
average_reward_test                  1.591775
round_time_test        0 days 00:00:08.586279
round_time_total       0 days 00:05:56.398702
loss_total                        7840.649962
loss_critic                       9891.751513
loss_actor                        -363.756912
memory_size                        97366.5835 

=== epoch 3/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  329
episode_length                      30.206687
returns                             46.531362
return_std                          35.272157
average_reward                       1.540098
round_time             0 days 00:05:59.143766
episodes_test                           319.0
episode_length_test                 31.322884
returns_test                        49.639864
return_std_test                      24.12682
average_reward_test                  1.584243
round_time_test        0 days 00:00:08.750190
round_time_total       0 days 00:05:59.144869
loss_total                        7968.074655
loss_critic                      10050.533338
loss_actor                        -361.760752
memory_size                        98436.8855 

=== epoch 3/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:26,  5.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  337
episode_length                      29.557864
returns                             45.215331
return_std                          34.552671
average_reward                       1.530011
round_time             0 days 00:05:59.177802
episodes_test                           317.0
episode_length_test                 31.523659
returns_test                        50.292572
return_std_test                     24.796484
average_reward_test                  1.594905
round_time_test        0 days 00:00:08.746844
round_time_total       0 days 00:05:59.178924
loss_total                        7782.670712
loss_critic                       9818.671305
loss_actor                        -361.332357
memory_size                         99408.557 

=== epoch 3/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:13,  5.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  317
episode_length                      31.425868
returns                             49.002108
return_std                          38.416998
average_reward                       1.559911
round_time             0 days 00:05:57.400506
episodes_test                           329.0
episode_length_test                 30.379939
returns_test                        47.580804
return_std_test                      26.06861
average_reward_test                  1.565882
round_time_test        0 days 00:00:08.831049
round_time_total       0 days 00:05:57.401836
loss_total                        7839.015218
loss_critic                       9888.781268
loss_actor                        -360.049683
memory_size                       100499.3465 

=== epoch 3/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:25,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  299
episode_length                      33.284281
returns                             52.870232
return_std                          39.329144
average_reward                       1.589318
round_time             0 days 00:05:57.779765
episodes_test                           310.0
episode_length_test                 32.216129
returns_test                         51.52922
return_std_test                     23.770023
average_reward_test                  1.598909
round_time_test        0 days 00:00:08.599308
round_time_total       0 days 00:05:57.780981
loss_total                        7804.384081
loss_critic                       9845.410165
loss_actor                        -359.720913
memory_size                        101705.798 

=== epoch 3/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  294
episode_length                      33.867347
returns                             54.012444
return_std                          39.236674
average_reward                       1.594711
round_time             0 days 00:05:59.969794
episodes_test                           311.0
episode_length_test                  32.14791
returns_test                        51.781643
return_std_test                     21.686829
average_reward_test                  1.610601
round_time_test        0 days 00:00:08.564968
round_time_total       0 days 00:05:59.970887
loss_total                        7691.214312
loss_critic                       9703.983211
loss_actor                        -359.861924
memory_size                         102802.99 

=== epoch 3/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.55it/s]
episodes                                  290
episode_length                      34.244828
returns                             54.997859
return_std                          39.012404
average_reward                        1.60771
round_time             0 days 00:06:00.655167
episodes_test                           310.0
episode_length_test                 32.167742
returns_test                        51.133004
return_std_test                      23.47134
average_reward_test                  1.589188
round_time_test        0 days 00:00:08.808056
round_time_total       0 days 00:06:00.656416
loss_total                        7727.356846
loss_critic                       9749.244921
loss_actor                        -360.196077
memory_size                        103909.991 

=== epoch 3/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:25,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  287
episode_length                      34.752613
returns                             55.967982
return_std                          39.405699
average_reward                       1.610437
round_time             0 days 00:05:58.283801
episodes_test                           326.0
episode_length_test                 30.650307
returns_test                         49.06914
return_std_test                     24.263909
average_reward_test                  1.600547
round_time_test        0 days 00:00:08.734126
round_time_total       0 days 00:05:58.284972
loss_total                        7508.339224
loss_critic                       9475.105862
loss_actor                        -358.727938
memory_size                       104982.8155 

=== epoch 3/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:25,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  293
episode_length                      33.962457
returns                             54.307311
return_std                          37.452178
average_reward                       1.598596
round_time             0 days 00:06:00.035555
episodes_test                           312.0
episode_length_test                 32.022436
returns_test                        51.219711
return_std_test                      20.01923
average_reward_test                  1.598911
round_time_test        0 days 00:00:08.745038
round_time_total       0 days 00:06:00.036829
loss_total                        7631.064464
loss_critic                       9628.685835
loss_actor                        -359.421683
memory_size                       106045.1755 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 4/10 ===== round 1/50 ======================================
  1%|          | 13/2000 [00:02<05:51,  5.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:38<00:00,  5.91it/s]
episodes                                   65
episode_length                      30.569231
returns                             47.004424
return_std                          33.405817
average_reward                       1.533395
round_time             0 days 00:05:38.699638
episodes_test                           308.0
episode_length_test                 32.418831
returns_test                        51.617959
return_std_test                     20.219298
average_reward_test                  1.591865
round_time_test        0 days 00:00:08.773840
round_time_total       0 days 00:05:38.701137
loss_total                        7725.907493
loss_critic                       9747.232511
loss_actor                         -359.39323
memory_size                        107071.067 

=== epoch 4/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:46,  5.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                  126
episode_length                       31.52381
returns                             48.833748
return_std                          32.472721
average_reward                       1.548685
round_time             0 days 00:05:44.043969
episodes_test                           292.0
episode_length_test                 34.239726
returns_test                        54.425328
return_std_test                     19.333678
average_reward_test                  1.589385
round_time_test        0 days 00:00:08.754330
round_time_total       0 days 00:05:44.045064
loss_total                        7702.234196
loss_critic                       9717.856264
loss_actor                        -360.254769
memory_size                        108103.719 

=== epoch 4/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:35,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                  185
episode_length                      32.291892
returns                             50.441337
return_std                          34.848383
average_reward                       1.561192
round_time             0 days 00:05:43.229326
episodes_test                           320.0
episode_length_test                 31.234375
returns_test                        50.173022
return_std_test                     21.743484
average_reward_test                  1.605968
round_time_test        0 days 00:00:08.690591
round_time_total       0 days 00:05:43.230402
loss_total                        7683.735812
loss_critic                       9694.361082
loss_actor                        -358.765956
memory_size                       109187.8955 

=== epoch 4/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:35,  4.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  244
episode_length                      32.627049
returns                              51.11329
return_std                          35.040056
average_reward                       1.566757
round_time             0 days 00:05:45.241329
episodes_test                           307.0
episode_length_test                 32.534202
returns_test                        52.202796
return_std_test                     26.354684
average_reward_test                   1.60398
round_time_test        0 days 00:00:08.860139
round_time_total       0 days 00:05:45.242463
loss_total                        7678.032389
loss_critic                       9687.328208
loss_actor                        -359.151539
memory_size                        110229.431 

=== epoch 4/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:47,  5.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  303
episode_length                      32.993399
returns                             51.689665
return_std                          35.922558
average_reward                       1.566493
round_time             0 days 00:05:45.973477
episodes_test                           335.0
episode_length_test                 29.791045
returns_test                        46.691226
return_std_test                      22.12137
average_reward_test                  1.566703
round_time_test        0 days 00:00:08.566107
round_time_total       0 days 00:05:45.974964
loss_total                         7512.72313
loss_critic                       9480.498445
loss_actor                        -358.378815
memory_size                        111353.762 

=== epoch 4/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                  302
episode_length                      33.003311
returns                             52.030272
return_std                          36.605381
average_reward                       1.576795
round_time             0 days 00:05:46.352441
episodes_test                           341.0
episode_length_test                 29.304985
returns_test                        47.045585
return_std_test                     21.056306
average_reward_test                  1.604933
round_time_test        0 days 00:00:08.606185
round_time_total       0 days 00:05:46.353543
loss_total                        7623.102287
loss_critic                       9618.878223
loss_actor                        -360.002115
memory_size                       112398.6425 

=== epoch 4/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.78it/s]
episodes                                  297
episode_length                      33.521886
returns                             53.088814
return_std                          37.896645
average_reward                       1.584249
round_time             0 days 00:05:46.788007
episodes_test                           305.0
episode_length_test                 32.688525
returns_test                         52.69751
return_std_test                     25.941135
average_reward_test                  1.611936
round_time_test        0 days 00:00:08.711344
round_time_total       0 days 00:05:46.789104
loss_total                        7610.568373
loss_critic                       9602.605594
loss_actor                        -357.581259
memory_size                        113460.426 

=== epoch 4/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.78it/s]
episodes                                  296
episode_length                      33.692568
returns                               53.5113
return_std                          40.328843
average_reward                       1.587308
round_time             0 days 00:05:46.682982
episodes_test                           322.0
episode_length_test                 30.953416
returns_test                        49.513381
return_std_test                     23.975338
average_reward_test                  1.599744
round_time_test        0 days 00:00:08.746779
round_time_total       0 days 00:05:46.684067
loss_total                        7468.090621
loss_critic                       9424.428544
loss_actor                        -357.261731
memory_size                       114612.8515 

=== epoch 4/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  291
episode_length                      34.274914
returns                             54.556792
return_std                          40.939449
average_reward                       1.591817
round_time             0 days 00:05:48.122845
episodes_test                           328.0
episode_length_test                 30.478659
returns_test                        48.891154
return_std_test                     21.822954
average_reward_test                  1.603914
round_time_test        0 days 00:00:08.778872
round_time_total       0 days 00:05:48.123970
loss_total                        7440.461399
loss_critic                       9390.106803
loss_actor                        -358.120827
memory_size                       115749.9505 

=== epoch 4/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:42,  5.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.78it/s]
episodes                                  290
episode_length                      34.303448
returns                             54.755515
return_std                          40.338982
average_reward                       1.596182
round_time             0 days 00:05:46.801039
episodes_test                           306.0
episode_length_test                 32.477124
returns_test                        52.856781
return_std_test                     25.192276
average_reward_test                  1.629168
round_time_test        0 days 00:00:08.719232
round_time_total       0 days 00:05:46.802224
loss_total                         7507.78898
loss_critic                        9475.02128
loss_actor                        -361.140868
memory_size                       116884.4335 

=== epoch 4/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  281
episode_length                      35.327402
returns                             56.709553
return_std                          41.730991
average_reward                       1.605201
round_time             0 days 00:05:49.272462
episodes_test                           318.0
episode_length_test                 31.386792
returns_test                        50.190731
return_std_test                     22.140277
average_reward_test                   1.59875
round_time_test        0 days 00:00:08.619915
round_time_total       0 days 00:05:49.273598
loss_total                        7643.349631
loss_critic                       9644.324547
loss_actor                        -360.550661
memory_size                       118002.9155 

=== epoch 4/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:14,  5.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  280
episode_length                      35.546429
returns                             56.580737
return_std                          41.137039
average_reward                       1.595288
round_time             0 days 00:05:48.281818
episodes_test                           294.0
episode_length_test                 33.928571
returns_test                        54.085002
return_std_test                     23.358467
average_reward_test                  1.593662
round_time_test        0 days 00:00:08.723225
round_time_total       0 days 00:05:48.282943
loss_total                        7475.579493
loss_critic                       9433.550026
loss_actor                         -356.30331
memory_size                       119158.4235 

=== epoch 4/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:06,  5.43it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                  282
episode_length                      35.397163
returns                             56.476486
return_std                          37.568405
average_reward                       1.596108
round_time             0 days 00:05:49.568452
episodes_test                           294.0
episode_length_test                 33.965986
returns_test                        54.078057
return_std_test                     22.186551
average_reward_test                  1.591352
round_time_test        0 days 00:00:08.653674
round_time_total       0 days 00:05:49.569746
loss_total                        7519.997911
loss_critic                        9489.63055
loss_actor                        -358.533269
memory_size                       120284.8175 

=== epoch 4/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  285
episode_length                      34.936842
returns                             55.910701
return_std                          37.533269
average_reward                       1.601588
round_time             0 days 00:05:49.073097
episodes_test                           308.0
episode_length_test                 32.409091
returns_test                        51.737773
return_std_test                     17.597706
average_reward_test                  1.596197
round_time_test        0 days 00:00:08.659335
round_time_total       0 days 00:05:49.074629
loss_total                        7532.956435
loss_critic                       9505.742016
loss_actor                        -358.186584
memory_size                        121361.438 

=== epoch 4/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  283
episode_length                      35.208481
returns                             56.353767
return_std                          37.302244
average_reward                        1.60073
round_time             0 days 00:05:49.824719
episodes_test                           307.0
episode_length_test                 32.517915
returns_test                          51.5332
return_std_test                     21.253433
average_reward_test                  1.584215
round_time_test        0 days 00:00:08.651718
round_time_total       0 days 00:05:49.825872
loss_total                        7494.303788
loss_critic                       9457.053303
loss_actor                        -356.694959
memory_size                        122487.672 

=== epoch 4/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:59,  5.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  283
episode_length                      35.113074
returns                             55.783638
return_std                          35.351731
average_reward                       1.588031
round_time             0 days 00:05:49.136985
episodes_test                           309.0
episode_length_test                 32.333333
returns_test                        51.629186
return_std_test                     23.510826
average_reward_test                  1.596291
round_time_test        0 days 00:00:08.619728
round_time_total       0 days 00:05:49.138292
loss_total                        7414.592559
loss_critic                       9356.882486
loss_actor                        -354.567754
memory_size                       123618.0665 

=== epoch 4/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:22,  5.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  289
episode_length                      34.429066
returns                             54.609302
return_std                          34.913091
average_reward                        1.58655
round_time             0 days 00:05:51.431938
episodes_test                           328.0
episode_length_test                 30.396341
returns_test                        47.375033
return_std_test                     23.002637
average_reward_test                  1.558407
round_time_test        0 days 00:00:08.684293
round_time_total       0 days 00:05:51.433156
loss_total                        7355.482486
loss_critic                       9283.465958
loss_actor                        -356.452045
memory_size                       124685.8855 

=== epoch 4/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                  293
episode_length                      34.044369
returns                              53.56271
return_std                          35.760818
average_reward                       1.571629
round_time             0 days 00:05:51.744379
episodes_test                           300.0
episode_length_test                     33.32
returns_test                         53.45666
return_std_test                     20.782787
average_reward_test                  1.604044
round_time_test        0 days 00:00:08.840460
round_time_total       0 days 00:05:51.745460
loss_total                        7427.937762
loss_critic                       9374.388603
loss_actor                         -357.86624
memory_size                        125759.775 

=== epoch 4/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  300
episode_length                      33.296667
returns                             51.947283
return_std                          35.875185
average_reward                       1.559417
round_time             0 days 00:05:51.498261
episodes_test                           318.0
episode_length_test                  31.40566
returns_test                        49.497657
return_std_test                      19.84178
average_reward_test                  1.575376
round_time_test        0 days 00:00:08.785268
round_time_total       0 days 00:05:51.499407
loss_total                        7416.067316
loss_critic                       9359.959812
loss_actor                        -359.503344
memory_size                        126795.229 

=== epoch 4/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  300
episode_length                      33.246667
returns                             52.041479
return_std                          37.050409
average_reward                       1.565703
round_time             0 days 00:05:52.051962
episodes_test                           300.0
episode_length_test                 33.316667
returns_test                        53.125927
return_std_test                     22.162452
average_reward_test                  1.594264
round_time_test        0 days 00:00:08.607937
round_time_total       0 days 00:05:52.053080
loss_total                        7414.633012
loss_critic                       9357.511425
loss_actor                        -356.881374
memory_size                       127882.9255 

=== epoch 4/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  306
episode_length                      32.552288
returns                             50.634173
return_std                           37.10476
average_reward                       1.554931
round_time             0 days 00:05:52.234112
episodes_test                           291.0
episode_length_test                 34.233677
returns_test                        54.779224
return_std_test                     17.441219
average_reward_test                  1.600044
round_time_test        0 days 00:00:08.755630
round_time_total       0 days 00:05:52.235695
loss_total                        7370.961871
loss_critic                        9302.49964
loss_actor                        -355.189859
memory_size                       129007.8505 

=== epoch 4/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  310
episode_length                      32.048387
returns                             49.645829
return_std                          36.848822
average_reward                       1.550382
round_time             0 days 00:05:53.655657
episodes_test                           306.0
episode_length_test                 32.604575
returns_test                        51.865549
return_std_test                     19.558491
average_reward_test                  1.590717
round_time_test        0 days 00:00:08.705315
round_time_total       0 days 00:05:53.656910
loss_total                        7289.327158
loss_critic                       9200.139308
loss_actor                        -353.922075
memory_size                       130065.4665 

=== epoch 4/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  310
episode_length                      32.029032
returns                             49.637037
return_std                          36.863502
average_reward                       1.550792
round_time             0 days 00:05:51.371273
episodes_test                           291.0
episode_length_test                 34.353952
returns_test                        54.600037
return_std_test                     20.424342
average_reward_test                  1.589112
round_time_test        0 days 00:00:08.597553
round_time_total       0 days 00:05:51.372382
loss_total                        7219.350444
loss_critic                       9112.565549
loss_actor                        -353.510623
memory_size                       131069.3075 

=== epoch 4/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:01<00:00,  5.53it/s]
episodes                                  306
episode_length                      32.434641
returns                             50.077436
return_std                          35.325401
average_reward                       1.544749
round_time             0 days 00:06:02.122851
episodes_test                           290.0
episode_length_test                 34.424138
returns_test                        55.273796
return_std_test                     15.590384
average_reward_test                  1.605253
round_time_test        0 days 00:00:08.658972
round_time_total       0 days 00:06:02.124118
loss_total                        7195.852758
loss_critic                       9083.199358
loss_actor                        -353.534254
memory_size                       132133.8205 

=== epoch 4/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:14,  5.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  310
episode_length                      32.148387
returns                             49.368501
return_std                          35.261321
average_reward                       1.536017
round_time             0 days 00:05:56.111706
episodes_test                           285.0
episode_length_test                 34.961404
returns_test                        55.588965
return_std_test                     26.989923
average_reward_test                   1.59007
round_time_test        0 days 00:00:08.750578
round_time_total       0 days 00:05:56.112817
loss_total                        7360.984827
loss_critic                       9289.849132
loss_actor                        -354.473057
memory_size                       133254.3755 

=== epoch 4/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:14<00:00,  5.34it/s]
episodes                                  297
episode_length                      33.555556
returns                             51.702871
return_std                          35.215062
average_reward                       1.540107
round_time             0 days 00:06:14.783345
episodes_test                           299.0
episode_length_test                 33.384615
returns_test                        52.998451
return_std_test                     19.426761
average_reward_test                  1.586827
round_time_test        0 days 00:00:08.700781
round_time_total       0 days 00:06:14.784452
loss_total                         7193.46434
loss_critic                       9080.041398
loss_actor                        -352.844565
memory_size                        134395.519 

=== epoch 4/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:27,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  296
episode_length                      33.658784
returns                              52.02557
return_std                          36.141319
average_reward                       1.545104
round_time             0 days 00:05:54.586461
episodes_test                           296.0
episode_length_test                 33.726351
returns_test                        53.877391
return_std_test                     22.779357
average_reward_test                  1.596566
round_time_test        0 days 00:00:08.629732
round_time_total       0 days 00:05:54.587799
loss_total                        7263.821418
loss_critic                       9168.222841
loss_actor                        -353.784917
memory_size                       135548.6925 

=== epoch 4/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:14,  5.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  290
episode_length                       34.32069
returns                             53.469567
return_std                          37.077912
average_reward                       1.559216
round_time             0 days 00:05:53.882568
episodes_test                           288.0
episode_length_test                 34.697917
returns_test                        55.477819
return_std_test                     24.775304
average_reward_test                   1.59836
round_time_test        0 days 00:00:08.535725
round_time_total       0 days 00:05:53.883705
loss_total                        7353.621284
loss_critic                       9280.528017
loss_actor                        -354.006297
memory_size                       136604.2145 

=== epoch 4/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  284
episode_length                      35.056338
returns                              54.74332
return_std                          37.957816
average_reward                       1.561759
round_time             0 days 00:05:55.743873
episodes_test                           248.0
episode_length_test                 40.294355
returns_test                        64.811733
return_std_test                      20.62877
average_reward_test                  1.608068
round_time_test        0 days 00:00:08.725994
round_time_total       0 days 00:05:55.744988
loss_total                        7183.084667
loss_critic                       9067.383138
loss_actor                        -354.109835
memory_size                       137770.2245 

=== epoch 4/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:26,  5.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  283
episode_length                      35.201413
returns                             54.967529
return_std                          38.002575
average_reward                       1.561512
round_time             0 days 00:05:53.311394
episodes_test                           304.0
episode_length_test                 32.855263
returns_test                        52.313327
return_std_test                     19.666068
average_reward_test                  1.591547
round_time_test        0 days 00:00:08.721122
round_time_total       0 days 00:05:53.312846
loss_total                         7271.10312
loss_critic                       9176.721162
loss_actor                        -351.369732
memory_size                       138900.5075 

=== epoch 4/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  286
episode_length                      34.748252
returns                              54.48387
return_std                          37.951803
average_reward                       1.569947
round_time             0 days 00:05:54.516684
episodes_test                           306.0
episode_length_test                 32.633987
returns_test                        52.254964
return_std_test                     22.426812
average_reward_test                  1.600571
round_time_test        0 days 00:00:08.683904
round_time_total       0 days 00:05:54.517793
loss_total                         7123.00875
loss_critic                       8991.756784
loss_actor                        -351.983979
memory_size                       140024.1865 

=== epoch 4/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:45,  5.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  280
episode_length                      35.646429
returns                             55.782089
return_std                          36.562497
average_reward                       1.565365
round_time             0 days 00:05:54.555282
episodes_test                           230.0
episode_length_test                 43.352174
returns_test                        68.507862
return_std_test                     22.872318
average_reward_test                  1.579589
round_time_test        0 days 00:00:08.689697
round_time_total       0 days 00:05:54.556378
loss_total                        7260.451373
loss_critic                       9163.712651
loss_actor                        -352.594371
memory_size                        141170.865 

=== epoch 4/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:46,  5.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  275
episode_length                      36.265455
returns                             56.540562
return_std                          36.565204
average_reward                       1.558615
round_time             0 days 00:05:54.932983
episodes_test                           291.0
episode_length_test                 34.292096
returns_test                         55.02679
return_std_test                     19.765247
average_reward_test                  1.604162
round_time_test        0 days 00:00:08.556312
round_time_total       0 days 00:05:54.934351
loss_total                        7095.759598
loss_critic                       8957.983735
loss_actor                        -353.137482
memory_size                        142335.691 

=== epoch 4/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  280
episode_length                      35.503571
returns                              55.11365
return_std                          36.311209
average_reward                       1.553135
round_time             0 days 00:05:55.117812
episodes_test                           293.0
episode_length_test                 34.010239
returns_test                         53.75648
return_std_test                     21.199979
average_reward_test                  1.580729
round_time_test        0 days 00:00:08.734138
round_time_total       0 days 00:05:55.119083
loss_total                         7140.82552
loss_critic                       9014.104439
loss_actor                        -352.290766
memory_size                        143424.883 

=== epoch 4/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:12,  4.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  272
episode_length                         36.625
returns                             56.664186
return_std                          34.190831
average_reward                       1.546798
round_time             0 days 00:05:54.283357
episodes_test                           271.0
episode_length_test                 36.867159
returns_test                         59.58546
return_std_test                     20.798791
average_reward_test                  1.615603
round_time_test        0 days 00:00:08.801128
round_time_total       0 days 00:05:54.284502
loss_total                        7137.027408
loss_critic                       9009.067678
loss_actor                        -351.134281
memory_size                       144586.5265 

=== epoch 4/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:45,  5.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  276
episode_length                      35.956522
returns                             55.411566
return_std                          33.005131
average_reward                       1.540673
round_time             0 days 00:05:55.014376
episodes_test                           271.0
episode_length_test                 36.833948
returns_test                        58.714921
return_std_test                     24.017227
average_reward_test                  1.593245
round_time_test        0 days 00:00:08.605544
round_time_total       0 days 00:05:55.015493
loss_total                        6994.855882
loss_critic                       8831.261853
loss_actor                        -350.768633
memory_size                        145706.241 

=== epoch 4/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:47,  5.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  276
episode_length                      35.960145
returns                             56.088076
return_std                          36.140237
average_reward                       1.557254
round_time             0 days 00:05:57.954478
episodes_test                           286.0
episode_length_test                 34.867133
returns_test                        55.232962
return_std_test                     21.252127
average_reward_test                  1.583738
round_time_test        0 days 00:00:08.869885
round_time_total       0 days 00:05:57.955551
loss_total                        7020.460703
loss_critic                        8863.70843
loss_actor                        -352.530807
memory_size                       146848.1455 

=== epoch 4/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  283
episode_length                      35.286219
returns                             54.518992
return_std                          34.935303
average_reward                       1.544569
round_time             0 days 00:05:56.231922
episodes_test                           293.0
episode_length_test                 34.085324
returns_test                        54.685702
return_std_test                     15.685576
average_reward_test                  1.603839
round_time_test        0 days 00:00:08.912383
round_time_total       0 days 00:05:56.233026
loss_total                        7051.639099
loss_critic                       8903.059974
loss_actor                        -354.044987
memory_size                        147961.921 

=== epoch 4/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  273
episode_length                       36.29304
returns                             56.413818
return_std                          36.982922
average_reward                       1.555238
round_time             0 days 00:05:56.429469
episodes_test                           273.0
episode_length_test                 36.549451
returns_test                        57.871036
return_std_test                     29.217306
average_reward_test                  1.582972
round_time_test        0 days 00:00:08.574094
round_time_total       0 days 00:05:56.431003
loss_total                         7221.15559
loss_critic                       9114.706142
loss_actor                          -353.0472
memory_size                        149138.729 

=== epoch 4/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:22,  5.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  274
episode_length                      36.452555
returns                             57.292367
return_std                          38.799031
average_reward                       1.571181
round_time             0 days 00:05:55.370859
episodes_test                           251.0
episode_length_test                 39.784861
returns_test                        63.551491
return_std_test                     22.806655
average_reward_test                  1.596688
round_time_test        0 days 00:00:08.598371
round_time_total       0 days 00:05:55.372003
loss_total                        6994.263095
loss_critic                       8830.696527
loss_actor                        -351.471252
memory_size                       150322.8125 

=== epoch 4/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  269
episode_length                      37.063197
returns                             58.611107
return_std                           39.89307
average_reward                        1.58103
round_time             0 days 00:05:57.077342
episodes_test                           285.0
episode_length_test                 35.038596
returns_test                        55.462936
return_std_test                     21.556613
average_reward_test                  1.582517
round_time_test        0 days 00:00:08.644055
round_time_total       0 days 00:05:57.078448
loss_total                        7062.720728
loss_critic                       8916.869912
loss_actor                        -353.876599
memory_size                        151518.887 

=== epoch 4/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  262
episode_length                      37.908397
returns                             59.759413
return_std                          38.218585
average_reward                       1.577715
round_time             0 days 00:05:59.072379
episodes_test                           273.0
episode_length_test                 36.487179
returns_test                        58.397464
return_std_test                     18.338738
average_reward_test                   1.60035
round_time_test        0 days 00:00:08.547079
round_time_total       0 days 00:05:59.073467
loss_total                        7259.125123
loss_critic                       9162.614476
loss_actor                        -354.832854
memory_size                       152725.5665 

=== epoch 4/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  254
episode_length                      39.149606
returns                              62.15881
return_std                          39.140087
average_reward                       1.587565
round_time             0 days 00:05:56.212033
episodes_test                           269.0
episode_length_test                 37.048327
returns_test                        59.323813
return_std_test                     18.465215
average_reward_test                  1.601124
round_time_test        0 days 00:00:08.585000
round_time_total       0 days 00:05:56.213159
loss_total                        7246.756591
loss_critic                       9146.969205
loss_actor                        -354.094474
memory_size                        153909.276 

=== epoch 4/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  260
episode_length                      38.307692
returns                               60.9045
return_std                          38.363517
average_reward                       1.590397
round_time             0 days 00:05:56.934576
episodes_test                           315.0
episode_length_test                 31.736508
returns_test                        50.931541
return_std_test                     14.732083
average_reward_test                  1.604613
round_time_test        0 days 00:00:08.793019
round_time_total       0 days 00:05:56.935690
loss_total                        7038.156937
loss_critic                       8886.593156
loss_actor                        -355.588558
memory_size                        155052.808 

=== epoch 4/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  271
episode_length                      36.778598
returns                             57.390364
return_std                           37.64994
average_reward                        1.56007
round_time             0 days 00:05:58.180522
episodes_test                           286.0
episode_length_test                 34.828671
returns_test                        55.626096
return_std_test                     19.207124
average_reward_test                  1.597169
round_time_test        0 days 00:00:08.680485
round_time_total       0 days 00:05:58.181685
loss_total                        7211.428907
loss_critic                        9102.51054
loss_actor                         -352.89824
memory_size                       156143.5365 

=== epoch 4/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  281
episode_length                      35.469751
returns                             54.516821
return_std                          37.765371
average_reward                       1.538331
round_time             0 days 00:05:57.752090
episodes_test                           289.0
episode_length_test                 34.529412
returns_test                        54.729285
return_std_test                     21.332363
average_reward_test                  1.584608
round_time_test        0 days 00:00:08.779246
round_time_total       0 days 00:05:57.753187
loss_total                        7043.833195
loss_critic                       8892.988941
loss_actor                        -352.790431
memory_size                       157242.8085 

=== epoch 4/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [06:00<00:00,  5.56it/s]
episodes                                  295
episode_length                      33.813559
returns                             51.336322
return_std                          37.431437
average_reward                       1.519775
round_time             0 days 00:06:00.624912
episodes_test                           268.0
episode_length_test                 37.291045
returns_test                        59.271843
return_std_test                     21.182674
average_reward_test                  1.588962
round_time_test        0 days 00:00:08.677524
round_time_total       0 days 00:06:00.626453
loss_total                        7143.642233
loss_critic                       9018.443416
loss_actor                        -355.563124
memory_size                       158316.9585 

=== epoch 4/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  311
episode_length                      31.903537
returns                             47.971419
return_std                          36.428353
average_reward                       1.508419
round_time             0 days 00:05:58.058324
episodes_test                           314.0
episode_length_test                 31.796178
returns_test                        50.946415
return_std_test                     17.162856
average_reward_test                  1.601666
round_time_test        0 days 00:00:08.678137
round_time_total       0 days 00:05:58.059639
loss_total                        7164.830026
loss_critic                       9044.565422
loss_actor                        -354.112227
memory_size                        159337.784 

=== epoch 4/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  310
episode_length                      32.148387
returns                             48.245633
return_std                          36.136774
average_reward                        1.50006
round_time             0 days 00:05:58.595516
episodes_test                           263.0
episode_length_test                      38.0
returns_test                        60.511199
return_std_test                     19.602111
average_reward_test                  1.592019
round_time_test        0 days 00:00:08.653115
round_time_total       0 days 00:05:58.596640
loss_total                        7160.896631
loss_critic                       9039.534542
loss_actor                        -353.655633
memory_size                       160411.8705 

=== epoch 4/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  303
episode_length                      32.768977
returns                             49.476658
return_std                          36.641923
average_reward                       1.512327
round_time             0 days 00:05:56.782720
episodes_test                           289.0
episode_length_test                 34.567474
returns_test                        54.808437
return_std_test                     21.244412
average_reward_test                  1.585009
round_time_test        0 days 00:00:08.750850
round_time_total       0 days 00:05:56.783853
loss_total                        7126.886723
loss_critic                       8997.703885
loss_actor                        -356.382531
memory_size                        161535.255 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 5/10 ===== round 1/50 ======================================
  1%|          | 13/2000 [00:02<05:40,  5.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:39<00:00,  5.89it/s]
episodes                                   56
episode_length                      35.321429
returns                             54.332576
return_std                          32.361715
average_reward                       1.537891
round_time             0 days 00:05:39.477098
episodes_test                           256.0
episode_length_test                 39.042969
returns_test                        61.413043
return_std_test                     26.207263
average_reward_test                  1.572674
round_time_test        0 days 00:00:08.634635
round_time_total       0 days 00:05:39.478217
loss_total                        7240.975943
loss_critic                       9139.963024
loss_actor                        -354.973027
memory_size                       162647.5535 

=== epoch 5/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:26,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.85it/s]
episodes                                  111
episode_length                      36.009009
returns                             55.504854
return_std                           34.03276
average_reward                       1.540977
round_time             0 days 00:05:42.623917
episodes_test                           257.0
episode_length_test                 38.673152
returns_test                        61.532372
return_std_test                     20.070395
average_reward_test                  1.592518
round_time_test        0 days 00:00:08.683393
round_time_total       0 days 00:05:42.625040
loss_total                         7057.75936
loss_critic                       8910.548476
loss_actor                        -353.397698
memory_size                         163801.17 

=== epoch 5/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                  166
episode_length                      36.072289
returns                             55.633002
return_std                          34.721025
average_reward                       1.541451
round_time             0 days 00:05:42.896051
episodes_test                           285.0
episode_length_test                  34.97193
returns_test                        56.221338
return_std_test                     17.902031
average_reward_test                  1.607438
round_time_test        0 days 00:00:08.769803
round_time_total       0 days 00:05:42.897165
loss_total                        7033.695493
loss_critic                       8881.064602
loss_actor                        -355.781594
memory_size                       164940.1885 

=== epoch 5/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  220
episode_length                      36.031818
returns                             55.816049
return_std                          34.794696
average_reward                        1.55156
round_time             0 days 00:05:46.198695
episodes_test                           263.0
episode_length_test                 37.996198
returns_test                        60.261351
return_std_test                     20.392937
average_reward_test                  1.585419
round_time_test        0 days 00:00:08.669035
round_time_total       0 days 00:05:46.199982
loss_total                        7089.221056
loss_critic                       8950.392465
loss_actor                        -355.465134
memory_size                        166081.135 

=== epoch 5/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                  270
episode_length                           36.8
returns                              57.43404
return_std                          35.528206
average_reward                       1.558373
round_time             0 days 00:05:46.452739
episodes_test                           295.0
episode_length_test                 33.823729
returns_test                        53.782181
return_std_test                     22.650584
average_reward_test                  1.589059
round_time_test        0 days 00:00:08.832423
round_time_total       0 days 00:05:46.453849
loss_total                        7162.049397
loss_critic                       9040.774765
loss_actor                        -352.852705
memory_size                       167267.0695 

=== epoch 5/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:43,  5.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.83it/s]
episodes                                  264
episode_length                      37.609848
returns                             58.861574
return_std                          36.883139
average_reward                       1.566958
round_time             0 days 00:05:43.634539
episodes_test                           287.0
episode_length_test                 34.825784
returns_test                        55.311368
return_std_test                     24.490121
average_reward_test                  1.587843
round_time_test        0 days 00:00:08.763303
round_time_total       0 days 00:05:43.635637
loss_total                        7042.367251
loss_critic                       8891.253305
loss_actor                        -353.177596
memory_size                        168465.783 

=== epoch 5/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:59,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.80it/s]
episodes                                  274
episode_length                      36.259124
returns                             56.785371
return_std                          36.698045
average_reward                       1.565518
round_time             0 days 00:05:45.613522
episodes_test                           284.0
episode_length_test                 35.112676
returns_test                        55.434216
return_std_test                     23.675235
average_reward_test                  1.578437
round_time_test        0 days 00:00:08.657913
round_time_total       0 days 00:05:45.614813
loss_total                        7092.710997
loss_critic                       8954.002766
loss_actor                        -352.456828
memory_size                       169578.0865 

=== epoch 5/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:57,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  277
episode_length                      35.866426
returns                             55.958677
return_std                          35.955251
average_reward                       1.561187
round_time             0 days 00:05:47.301132
episodes_test                           319.0
episode_length_test                 31.341693
returns_test                        50.276957
return_std_test                     16.796951
average_reward_test                  1.604025
round_time_test        0 days 00:00:08.552851
round_time_total       0 days 00:05:47.302343
loss_total                        6988.506397
loss_critic                       8823.729108
loss_actor                        -352.385078
memory_size                       170607.7725 

=== epoch 5/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:48,  5.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.80it/s]
episodes                                  288
episode_length                      34.642361
returns                             53.345686
return_std                          37.653855
average_reward                       1.540315
round_time             0 days 00:05:45.668268
episodes_test                           259.0
episode_length_test                 38.505792
returns_test                        61.207918
return_std_test                      23.40603
average_reward_test                  1.589222
round_time_test        0 days 00:00:08.657077
round_time_total       0 days 00:05:45.669595
loss_total                        6998.477862
loss_critic                       8835.848476
loss_actor                        -351.005208
memory_size                       171709.4445 

=== epoch 5/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  300
episode_length                      33.156667
returns                             50.552248
return_std                          36.845297
average_reward                       1.525397
round_time             0 days 00:05:48.712279
episodes_test                           310.0
episode_length_test                 32.229032
returns_test                        50.268416
return_std_test                     24.204072
average_reward_test                  1.559222
round_time_test        0 days 00:00:08.710152
round_time_total       0 days 00:05:48.713395
loss_total                        6913.896311
loss_critic                       8729.820316
loss_actor                        -349.800301
memory_size                       172757.8165 

=== epoch 5/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  314
episode_length                      31.694268
returns                             47.660278
return_std                           36.93616
average_reward                       1.504856
round_time             0 days 00:05:49.840634
episodes_test                           273.0
episode_length_test                 36.578755
returns_test                        58.837565
return_std_test                     26.387497
average_reward_test                  1.607894
round_time_test        0 days 00:00:08.724068
round_time_total       0 days 00:05:49.841866
loss_total                        6965.932569
loss_critic                       8794.782182
loss_actor                         -349.46649
memory_size                        173808.892 

=== epoch 5/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:58,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  308
episode_length                      32.305195
returns                             48.602956
return_std                          38.086356
average_reward                       1.504785
round_time             0 days 00:05:51.547663
episodes_test                           260.0
episode_length_test                 38.446154
returns_test                        61.563415
return_std_test                     19.886693
average_reward_test                  1.601055
round_time_test        0 days 00:00:08.712826
round_time_total       0 days 00:05:51.548756
loss_total                         6947.50584
loss_critic                       8772.048364
loss_actor                        -350.664851
memory_size                       174880.6545 

=== epoch 5/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  301
episode_length                      32.877076
returns                             49.366313
return_std                          38.484061
average_reward                       1.506274
round_time             0 days 00:05:47.733716
episodes_test                           277.0
episode_length_test                  36.01083
returns_test                        57.447302
return_std_test                     20.347298
average_reward_test                  1.594611
round_time_test        0 days 00:00:09.081064
round_time_total       0 days 00:05:47.734833
loss_total                        6825.273038
loss_critic                       8619.958257
loss_actor                        -353.468467
memory_size                        175986.479 

=== epoch 5/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:54,  5.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  292
episode_length                      34.130137
returns                             51.689387
return_std                          38.083853
average_reward                       1.514082
round_time             0 days 00:05:49.827160
episodes_test                           296.0
episode_length_test                 33.682432
returns_test                        52.984622
return_std_test                     25.360074
average_reward_test                  1.572845
round_time_test        0 days 00:00:08.675568
round_time_total       0 days 00:05:49.828472
loss_total                        7066.968562
loss_critic                       8922.717349
loss_actor                        -356.027145
memory_size                        177187.593 

=== epoch 5/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  286
episode_length                      34.909091
returns                             53.184811
return_std                          37.748969
average_reward                       1.524014
round_time             0 days 00:05:50.028192
episodes_test                           259.0
episode_length_test                 38.590734
returns_test                        61.418258
return_std_test                     29.787926
average_reward_test                  1.591141
round_time_test        0 days 00:00:08.662935
round_time_total       0 days 00:05:50.029447
loss_total                        7115.357753
loss_critic                       8983.578429
loss_actor                        -357.525576
memory_size                       178321.8425 

=== epoch 5/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  278
episode_length                      35.802158
returns                             54.567237
return_std                          36.376298
average_reward                       1.524938
round_time             0 days 00:05:51.115032
episodes_test                           267.0
episode_length_test                 37.419476
returns_test                        58.972756
return_std_test                     19.982617
average_reward_test                  1.575574
round_time_test        0 days 00:00:08.633056
round_time_total       0 days 00:05:51.116502
loss_total                        7024.248745
loss_critic                       8869.633054
loss_actor                        -357.289159
memory_size                       179447.5645 

=== epoch 5/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  270
episode_length                           36.8
returns                             56.157088
return_std                          36.818551
average_reward                       1.527958
round_time             0 days 00:05:49.653072
episodes_test                           249.0
episode_length_test                 39.995984
returns_test                        63.561914
return_std_test                     25.430892
average_reward_test                  1.589206
round_time_test        0 days 00:00:08.705625
round_time_total       0 days 00:05:49.654182
loss_total                        7113.771793
loss_critic                       8981.257706
loss_actor                        -356.172428
memory_size                         180591.89 

=== epoch 5/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  270
episode_length                      36.907407
returns                             56.409245
return_std                          37.372404
average_reward                       1.526779
round_time             0 days 00:05:52.435856
episodes_test                           263.0
episode_length_test                 37.775665
returns_test                        61.145695
return_std_test                     27.892563
average_reward_test                  1.618234
round_time_test        0 days 00:00:08.500229
round_time_total       0 days 00:05:52.437017
loss_total                        7159.045291
loss_critic                       9037.807447
loss_actor                        -356.003941
memory_size                        181805.951 

=== epoch 5/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  271
episode_length                      36.741697
returns                             56.354129
return_std                          36.666376
average_reward                       1.534064
round_time             0 days 00:05:50.350151
episodes_test                           231.0
episode_length_test                 43.077922
returns_test                        68.291159
return_std_test                     23.730043
average_reward_test                  1.583383
round_time_test        0 days 00:00:08.693270
round_time_total       0 days 00:05:50.351260
loss_total                        7073.262452
loss_critic                       8930.153378
loss_actor                        -354.301831
memory_size                        182987.238 

=== epoch 5/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  270
episode_length                      36.822222
returns                             56.011164
return_std                          37.944652
average_reward                       1.520376
round_time             0 days 00:05:50.883750
episodes_test                           260.0
episode_length_test                 38.411538
returns_test                        60.761995
return_std_test                     28.307518
average_reward_test                  1.581561
round_time_test        0 days 00:00:08.696754
round_time_total       0 days 00:05:50.884923
loss_total                        7003.120049
loss_critic                       8842.390546
loss_actor                        -353.962538
memory_size                       184101.4485 

=== epoch 5/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  271
episode_length                      36.752768
returns                             56.149581
return_std                           38.44365
average_reward                       1.527911
round_time             0 days 00:05:52.718991
episodes_test                           240.0
episode_length_test                      41.5
returns_test                        66.474275
return_std_test                     19.180915
average_reward_test                  1.601846
round_time_test        0 days 00:00:08.549141
round_time_total       0 days 00:05:52.720384
loss_total                        7013.816294
loss_critic                       8855.498687
loss_actor                        -352.913871
memory_size                        185269.253 

=== epoch 5/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:52,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  279
episode_length                      35.795699
returns                             54.549575
return_std                          38.343307
average_reward                       1.523913
round_time             0 days 00:05:49.958423
episodes_test                           285.0
episode_length_test                 34.982456
returns_test                        56.067738
return_std_test                      15.93759
average_reward_test                  1.602192
round_time_test        0 days 00:00:08.744272
round_time_total       0 days 00:05:49.959521
loss_total                        6921.215916
loss_critic                       8740.041519
loss_actor                         -354.08714
memory_size                        186387.887 

=== epoch 5/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  280
episode_length                      35.485714
returns                             54.450769
return_std                          39.246703
average_reward                       1.532343
round_time             0 days 00:05:53.719862
episodes_test                           271.0
episode_length_test                 36.771218
returns_test                        58.688784
return_std_test                     20.071161
average_reward_test                  1.596013
round_time_test        0 days 00:00:08.685456
round_time_total       0 days 00:05:53.721126
loss_total                        7002.565975
loss_critic                       8842.076789
loss_actor                        -355.477872
memory_size                        187525.363 

=== epoch 5/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  269
episode_length                      37.104089
returns                             56.522986
return_std                          39.351294
average_reward                        1.52352
round_time             0 days 00:05:52.903541
episodes_test                           280.0
episode_length_test                 35.592857
returns_test                        56.580893
return_std_test                     28.083182
average_reward_test                  1.589505
round_time_test        0 days 00:00:08.672129
round_time_total       0 days 00:05:52.904665
loss_total                        6944.334326
loss_critic                       8768.529441
loss_actor                        -352.446751
memory_size                       188755.3625 

=== epoch 5/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  266
episode_length                      37.571429
returns                             57.534016
return_std                          38.142967
average_reward                       1.531142
round_time             0 days 00:05:51.476264
episodes_test                           254.0
episode_length_test                 39.212598
returns_test                        62.329767
return_std_test                     21.684822
average_reward_test                  1.589232
round_time_test        0 days 00:00:08.637643
round_time_total       0 days 00:05:51.477374
loss_total                        6833.119893
loss_critic                       8629.483148
loss_actor                        -352.333728
memory_size                        189980.676 

=== epoch 5/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:43,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  258
episode_length                      38.593023
returns                              59.45169
return_std                          37.261495
average_reward                       1.540482
round_time             0 days 00:05:53.443191
episodes_test                           254.0
episode_length_test                  39.34252
returns_test                        63.621902
return_std_test                     27.723357
average_reward_test                  1.616645
round_time_test        0 days 00:00:08.656576
round_time_total       0 days 00:05:53.444279
loss_total                        6814.436527
loss_critic                       8606.097149
loss_actor                         -352.20658
memory_size                       191144.1105 

=== epoch 5/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:14,  5.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  257
episode_length                      38.743191
returns                              59.22408
return_std                          35.998296
average_reward                       1.527548
round_time             0 days 00:05:53.879272
episodes_test                           209.0
episode_length_test                 47.655502
returns_test                        71.497595
return_std_test                     29.134921
average_reward_test                  1.500327
round_time_test        0 days 00:00:08.736241
round_time_total       0 days 00:05:53.880393
loss_total                        6902.475243
loss_critic                       8716.233348
loss_actor                        -352.557817
memory_size                       192278.0195 

=== epoch 5/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  256
episode_length                      38.800781
returns                             58.692741
return_std                          34.451686
average_reward                       1.512458
round_time             0 days 00:05:56.437524
episodes_test                           232.0
episode_length_test                 42.991379
returns_test                        67.682312
return_std_test                      20.31978
average_reward_test                  1.574234
round_time_test        0 days 00:00:08.697464
round_time_total       0 days 00:05:56.438838
loss_total                         6839.78511
loss_critic                       8637.870428
loss_actor                        -352.556708
memory_size                        193485.167 

=== epoch 5/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  263
episode_length                      37.813688
returns                             56.974998
return_std                          34.665576
average_reward                        1.50713
round_time             0 days 00:05:55.679436
episodes_test                           256.0
episode_length_test                 39.046875
returns_test                        62.002529
return_std_test                     22.134049
average_reward_test                  1.587602
round_time_test        0 days 00:00:08.638120
round_time_total       0 days 00:05:55.680524
loss_total                        6802.980033
loss_critic                       8591.690701
loss_actor                        -351.863275
memory_size                       194631.8245 

=== epoch 5/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  265
episode_length                      37.433962
returns                             56.159031
return_std                          35.500415
average_reward                       1.501832
round_time             0 days 00:05:54.748163
episodes_test                           256.0
episode_length_test                 38.964844
returns_test                        61.813468
return_std_test                     19.555445
average_reward_test                   1.58613
round_time_test        0 days 00:00:08.787298
round_time_total       0 days 00:05:54.749687
loss_total                        6779.570917
loss_critic                        8561.70662
loss_actor                        -348.972493
memory_size                        195796.498 

=== epoch 5/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  263
episode_length                      37.768061
returns                             56.399738
return_std                          35.314925
average_reward                       1.493484
round_time             0 days 00:05:57.175350
episodes_test                           239.0
episode_length_test                 41.753138
returns_test                        64.415286
return_std_test                     26.860504
average_reward_test                  1.542324
round_time_test        0 days 00:00:08.762060
round_time_total       0 days 00:05:57.176474
loss_total                        6572.359556
loss_critic                       8302.441485
loss_actor                        -347.968741
memory_size                        196992.739 

=== epoch 5/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  264
episode_length                      37.795455
returns                             56.593231
return_std                          35.133049
average_reward                       1.497745
round_time             0 days 00:05:53.700659
episodes_test                           258.0
episode_length_test                 38.755814
returns_test                        61.403794
return_std_test                     19.675182
average_reward_test                  1.584316
round_time_test        0 days 00:00:08.731061
round_time_total       0 days 00:05:53.701800
loss_total                        6729.569604
loss_critic                       8499.327605
loss_actor                        -349.462972
memory_size                        198162.826 

=== epoch 5/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  266
episode_length                      37.413534
returns                             56.258807
return_std                          35.481489
average_reward                       1.504401
round_time             0 days 00:05:56.757489
episodes_test                           297.0
episode_length_test                 33.609428
returns_test                        53.488417
return_std_test                     20.402875
average_reward_test                  1.591032
round_time_test        0 days 00:00:08.774070
round_time_total       0 days 00:05:56.758579
loss_total                        6723.746022
loss_critic                       8491.721016
loss_actor                        -348.154564
memory_size                        199279.576 

=== epoch 5/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  264
episode_length                      37.844697
returns                             56.874188
return_std                          36.309341
average_reward                       1.502667
round_time             0 days 00:05:54.580049
episodes_test                           271.0
episode_length_test                 36.785978
returns_test                        58.125884
return_std_test                      22.64884
average_reward_test                  1.578153
round_time_test        0 days 00:00:08.752053
round_time_total       0 days 00:05:54.581149
loss_total                        6644.467802
loss_critic                       8393.169771
loss_actor                        -350.340625
memory_size                       200465.8895 

=== epoch 5/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  266
episode_length                      37.488722
returns                             56.541998
return_std                          35.619964
average_reward                       1.507994
round_time             0 days 00:05:56.636564
episodes_test                           282.0
episode_length_test                 35.333333
returns_test                        55.929048
return_std_test                     20.841748
average_reward_test                  1.582803
round_time_test        0 days 00:00:08.496320
round_time_total       0 days 00:05:56.637770
loss_total                        6675.499839
loss_critic                        8431.65169
loss_actor                        -349.108181
memory_size                        201624.524 

=== epoch 5/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  271
episode_length                      36.715867
returns                             54.943662
return_std                          37.135696
average_reward                       1.497177
round_time             0 days 00:05:58.464468
episodes_test                           276.0
episode_length_test                 36.134058
returns_test                        57.086774
return_std_test                      19.88604
average_reward_test                  1.579466
round_time_test        0 days 00:00:08.624397
round_time_total       0 days 00:05:58.465973
loss_total                        6678.068533
loss_critic                       8434.753997
loss_actor                        -348.673894
memory_size                        202772.764 

=== epoch 5/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  257
episode_length                      38.661479
returns                             57.921845
return_std                          39.375151
average_reward                        1.49936
round_time             0 days 00:05:57.546874
episodes_test                           215.0
episode_length_test                 46.283721
returns_test                        71.472942
return_std_test                     23.386183
average_reward_test                  1.542784
round_time_test        0 days 00:00:08.719476
round_time_total       0 days 00:05:57.548111
loss_total                        6644.002263
loss_critic                       8392.731591
loss_actor                        -350.915625
memory_size                       203980.7965 

=== epoch 5/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:26,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  266
episode_length                      37.466165
returns                             55.546292
return_std                          39.834648
average_reward                       1.482704
round_time             0 days 00:05:58.368423
episodes_test                           217.0
episode_length_test                 46.050691
returns_test                         71.68409
return_std_test                     20.957357
average_reward_test                  1.556285
round_time_test        0 days 00:00:08.724105
round_time_total       0 days 00:05:58.369532
loss_total                        6647.168374
loss_critic                        8396.75015
loss_actor                        -351.159291
memory_size                       205175.4015 

=== epoch 5/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  274
episode_length                      36.259124
returns                             53.839087
return_std                          39.380177
average_reward                       1.483956
round_time             0 days 00:05:56.682576
episodes_test                           225.0
episode_length_test                 44.346667
returns_test                        69.571468
return_std_test                     25.178094
average_reward_test                  1.568217
round_time_test        0 days 00:00:08.747435
round_time_total       0 days 00:05:56.684044
loss_total                        6789.013496
loss_critic                       8573.936678
loss_actor                        -350.679765
memory_size                       206256.2695 

=== epoch 5/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  271
episode_length                      36.719557
returns                             54.312725
return_std                          40.094275
average_reward                       1.479788
round_time             0 days 00:05:55.974023
episodes_test                           227.0
episode_length_test                  43.92511
returns_test                        68.774539
return_std_test                     26.030945
average_reward_test                  1.565311
round_time_test        0 days 00:00:08.712996
round_time_total       0 days 00:05:55.975123
loss_total                         6761.39276
loss_critic                       8539.765122
loss_actor                        -352.097229
memory_size                        207390.855 

=== epoch 5/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  270
episode_length                      36.966667
returns                              55.34148
return_std                          39.691426
average_reward                       1.497168
round_time             0 days 00:05:57.462854
episodes_test                           236.0
episode_length_test                 42.305085
returns_test                        68.334082
return_std_test                     20.119957
average_reward_test                  1.614221
round_time_test        0 days 00:00:08.689008
round_time_total       0 days 00:05:57.463958
loss_total                        6701.372675
loss_critic                       8464.882428
loss_actor                          -352.6669
memory_size                       208593.0585 

=== epoch 5/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  273
episode_length                       36.43956
returns                             54.862099
return_std                          38.331765
average_reward                       1.505521
round_time             0 days 00:05:57.882656
episodes_test                           237.0
episode_length_test                 42.151899
returns_test                        66.306842
return_std_test                     22.098873
average_reward_test                  1.572593
round_time_test        0 days 00:00:08.628295
round_time_total       0 days 00:05:57.883989
loss_total                        6739.933806
loss_critic                       8512.831305
loss_actor                        -351.656741
memory_size                       209776.2805 

=== epoch 5/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:43,  4.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  264
episode_length                      37.848485
returns                             57.401428
return_std                          38.334135
average_reward                       1.517398
round_time             0 days 00:05:58.858499
episodes_test                           244.0
episode_length_test                 40.864754
returns_test                        64.770584
return_std_test                     24.083875
average_reward_test                  1.583725
round_time_test        0 days 00:00:08.558446
round_time_total       0 days 00:05:58.859634
loss_total                        6774.250904
loss_critic                       8556.086022
loss_actor                        -353.090183
memory_size                       210975.6935 

=== epoch 5/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:24,  4.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  255
episode_length                      38.929412
returns                             58.822622
return_std                          37.600805
average_reward                       1.511628
round_time             0 days 00:05:58.736286
episodes_test                           238.0
episode_length_test                 41.869748
returns_test                        66.288218
return_std_test                      26.05309
average_reward_test                   1.58302
round_time_test        0 days 00:00:08.500478
round_time_total       0 days 00:05:58.737400
loss_total                        6726.325392
loss_critic                       8496.242479
loss_actor                        -353.343562
memory_size                        212168.637 

=== epoch 5/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  252
episode_length                      39.257937
returns                             59.210684
return_std                          37.647311
average_reward                       1.507867
round_time             0 days 00:05:58.595189
episodes_test                           281.0
episode_length_test                 35.562278
returns_test                        56.347545
return_std_test                     19.387409
average_reward_test                  1.584046
round_time_test        0 days 00:00:08.628699
round_time_total       0 days 00:05:58.596508
loss_total                        6682.371251
loss_critic                       8440.920139
loss_actor                        -351.824922
memory_size                        213366.037 

=== epoch 5/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  252
episode_length                      39.559524
returns                             59.504855
return_std                          37.845133
average_reward                       1.504911
round_time             0 days 00:05:59.030880
episodes_test                           226.0
episode_length_test                 44.168142
returns_test                        70.852818
return_std_test                     30.871317
average_reward_test                  1.603715
round_time_test        0 days 00:00:08.611143
round_time_total       0 days 00:05:59.031990
loss_total                        6683.444196
loss_critic                       8442.058349
loss_actor                        -351.013079
memory_size                       214579.6275 

=== epoch 5/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  255
episode_length                       39.05098
returns                             58.710868
return_std                          36.865718
average_reward                       1.502864
round_time             0 days 00:05:58.663871
episodes_test                           242.0
episode_length_test                 41.260331
returns_test                        66.037901
return_std_test                     24.368524
average_reward_test                   1.59983
round_time_test        0 days 00:00:08.762659
round_time_total       0 days 00:05:58.664989
loss_total                        6591.583835
loss_critic                        8326.38877
loss_actor                        -347.636471
memory_size                       215746.6195 

=== epoch 5/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:51,  5.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  257
episode_length                       38.88716
returns                             58.977939
return_std                          36.278603
average_reward                       1.516255
round_time             0 days 00:05:59.132637
episodes_test                           238.0
episode_length_test                 42.012605
returns_test                        66.695793
return_std_test                     31.953873
average_reward_test                  1.587456
round_time_test        0 days 00:00:08.692326
round_time_total       0 days 00:05:59.133747
loss_total                        6606.858408
loss_critic                       8345.455068
loss_actor                        -347.528837
memory_size                        216931.739 

=== epoch 5/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:32,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                  259
episode_length                      38.227799
returns                             57.711052
return_std                          36.889126
average_reward                       1.513623
round_time             0 days 00:05:59.773550
episodes_test                           276.0
episode_length_test                 36.152174
returns_test                        57.822329
return_std_test                     18.804466
average_reward_test                  1.598951
round_time_test        0 days 00:00:08.737898
round_time_total       0 days 00:05:59.774789
loss_total                        6632.576221
loss_critic                       8378.265216
loss_actor                        -350.180339
memory_size                       218085.8155 

=== epoch 5/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  262
episode_length                      38.076336
returns                             57.892849
return_std                          36.463786
average_reward                       1.520379
round_time             0 days 00:05:58.614846
episodes_test                           280.0
episode_length_test                 35.710714
returns_test                        56.613826
return_std_test                     24.026933
average_reward_test                  1.585288
round_time_test        0 days 00:00:08.735770
round_time_total       0 days 00:05:58.615967
loss_total                        6682.821171
loss_critic                       8441.446124
loss_actor                        -351.679216
memory_size                        219240.541 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 6/10 ===== round 1/50 ======================================
  1%|          | 13/2000 [00:02<05:48,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:41<00:00,  5.86it/s]
episodes                                   54
episode_length                      37.037037
returns                             55.908229
return_std                          38.623837
average_reward                       1.509522
round_time             0 days 00:05:41.654105
episodes_test                           281.0
episode_length_test                 35.537367
returns_test                        56.701913
return_std_test                     18.637765
average_reward_test                  1.594899
round_time_test        0 days 00:00:08.694794
round_time_total       0 days 00:05:41.655362
loss_total                        6737.744629
loss_critic                       8510.406643
loss_actor                         -352.90395
memory_size                       220414.2955 

=== epoch 6/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:40,  4.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.83it/s]
episodes                                  116
episode_length                      34.293103
returns                             51.647317
return_std                          35.798905
average_reward                       1.504309
round_time             0 days 00:05:43.562788
episodes_test                           287.0
episode_length_test                 34.839721
returns_test                        55.667544
return_std_test                     19.326082
average_reward_test                  1.597755
round_time_test        0 days 00:00:08.812657
round_time_total       0 days 00:05:43.563889
loss_total                        6691.802526
loss_critic                       8452.347819
loss_actor                        -350.379241
memory_size                       221501.7215 

=== epoch 6/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:06,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  164
episode_length                      36.585366
returns                             55.139732
return_std                          35.625761
average_reward                       1.507153
round_time             0 days 00:05:45.718847
episodes_test                           245.0
episode_length_test                 40.677551
returns_test                        63.916455
return_std_test                     25.890876
average_reward_test                  1.570921
round_time_test        0 days 00:00:08.592234
round_time_total       0 days 00:05:45.719928
loss_total                        6593.357724
loss_critic                       8329.139255
loss_actor                        -349.769018
memory_size                        222660.786 

=== epoch 6/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  215
episode_length                           37.2
returns                             56.246024
return_std                          36.108346
average_reward                       1.511856
round_time             0 days 00:05:45.154296
episodes_test                           261.0
episode_length_test                 38.298851
returns_test                        60.725054
return_std_test                      24.18363
average_reward_test                  1.585316
round_time_test        0 days 00:00:08.764181
round_time_total       0 days 00:05:45.155778
loss_total                        6486.221062
loss_critic                       8195.046175
loss_actor                        -349.079942
memory_size                       223867.4095 

=== epoch 6/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:45,  5.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                  262
episode_length                      38.137405
returns                             57.607811
return_std                          36.189859
average_reward                       1.510146
round_time             0 days 00:05:46.460594
episodes_test                           228.0
episode_length_test                 43.859649
returns_test                        68.603428
return_std_test                     24.065178
average_reward_test                  1.564158
round_time_test        0 days 00:00:08.646684
round_time_total       0 days 00:05:46.461740
loss_total                        6668.814675
loss_critic                        8423.37195
loss_actor                        -349.415032
memory_size                        225042.954 

=== epoch 6/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:44,  5.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  258
episode_length                      38.658915
returns                             58.468239
return_std                          35.514156
average_reward                        1.51225
round_time             0 days 00:05:47.899826
episodes_test                           238.0
episode_length_test                  41.89916
returns_test                        66.038639
return_std_test                     21.966291
average_reward_test                  1.575513
round_time_test        0 days 00:00:08.640899
round_time_total       0 days 00:05:47.901132
loss_total                         6649.19129
loss_critic                       8399.440688
loss_actor                        -351.806846
memory_size                       226271.2365 

=== epoch 6/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  245
episode_length                      40.632653
returns                             61.400344
return_std                          37.518357
average_reward                       1.512554
round_time             0 days 00:05:48.130016
episodes_test                           269.0
episode_length_test                 37.126394
returns_test                         59.25016
return_std_test                     21.603985
average_reward_test                  1.595508
round_time_test        0 days 00:00:08.657244
round_time_total       0 days 00:05:48.131149
loss_total                        6576.918863
loss_critic                       8309.025549
loss_actor                        -351.508409
memory_size                        227485.432 

=== epoch 6/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  253
episode_length                      39.521739
returns                             59.890968
return_std                          37.928507
average_reward                       1.515333
round_time             0 days 00:05:48.816661
episodes_test                           211.0
episode_length_test                 47.388626
returns_test                        73.707446
return_std_test                     23.423347
average_reward_test                  1.555326
round_time_test        0 days 00:00:08.655927
round_time_total       0 days 00:05:48.817747
loss_total                        6655.702097
loss_critic                        8407.32015
loss_actor                        -350.770737
memory_size                        228684.308 

=== epoch 6/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.76it/s]
episodes                                  246
episode_length                      40.317073
returns                             61.144321
return_std                          39.824511
average_reward                       1.516689
round_time             0 days 00:05:47.562589
episodes_test                           244.0
episode_length_test                 40.934426
returns_test                        65.194844
return_std_test                     23.538071
average_reward_test                  1.592157
round_time_test        0 days 00:00:08.763468
round_time_total       0 days 00:05:47.563701
loss_total                        6584.968082
loss_critic                       8318.909697
loss_actor                        -350.799003
memory_size                       229878.3785 

=== epoch 6/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:51,  4.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  247
episode_length                      40.364372
returns                             60.677886
return_std                           40.13702
average_reward                       1.504102
round_time             0 days 00:05:48.770155
episodes_test                           215.0
episode_length_test                 46.506977
returns_test                        72.385628
return_std_test                     25.673808
average_reward_test                  1.556385
round_time_test        0 days 00:00:08.683324
round_time_total       0 days 00:05:48.771315
loss_total                        6663.067053
loss_critic                       8416.754703
loss_actor                        -351.684186
memory_size                       231152.4845 

=== epoch 6/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  244
episode_length                       40.72541
returns                             60.680887
return_std                          40.898107
average_reward                       1.491647
round_time             0 days 00:05:48.700670
episodes_test                           197.0
episode_length_test                 50.736041
returns_test                        74.000479
return_std_test                     32.279398
average_reward_test                  1.458192
round_time_test        0 days 00:00:08.684820
round_time_total       0 days 00:05:48.702202
loss_total                        6571.158058
loss_critic                       8302.061077
loss_actor                        -352.454594
memory_size                       232411.8055 

=== epoch 6/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                  246
episode_length                      40.471545
returns                             59.731225
return_std                          39.577762
average_reward                       1.477008
round_time             0 days 00:05:49.503645
episodes_test                           214.0
episode_length_test                 46.602804
returns_test                        71.824165
return_std_test                     21.770149
average_reward_test                  1.540607
round_time_test        0 days 00:00:08.853416
round_time_total       0 days 00:05:49.504762
loss_total                        6505.027154
loss_critic                       8219.566865
loss_actor                         -353.13226
memory_size                        233663.802 

=== epoch 6/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  248
episode_length                      39.794355
returns                             58.027595
return_std                          38.640106
average_reward                       1.457584
round_time             0 days 00:05:49.312164
episodes_test                           216.0
episode_length_test                    46.125
returns_test                        71.796493
return_std_test                      21.30435
average_reward_test                  1.556331
round_time_test        0 days 00:00:08.702729
round_time_total       0 days 00:05:49.313313
loss_total                        6616.031402
loss_critic                       8357.795594
loss_actor                        -351.025933
memory_size                        234807.518 

=== epoch 6/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  256
episode_length                      38.957031
returns                             56.491625
return_std                          37.047169
average_reward                       1.449995
round_time             0 days 00:05:49.917595
episodes_test                           187.0
episode_length_test                 53.160428
returns_test                        79.927631
return_std_test                     19.820965
average_reward_test                  1.502994
round_time_test        0 days 00:00:08.670185
round_time_total       0 days 00:05:49.918690
loss_total                        6441.168774
loss_critic                       8138.250967
loss_actor                        -347.160567
memory_size                        235987.665 

=== epoch 6/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  261
episode_length                      37.796935
returns                             54.693115
return_std                          35.964206
average_reward                        1.44827
round_time             0 days 00:05:49.682605
episodes_test                           225.0
episode_length_test                 44.346667
returns_test                        68.842055
return_std_test                     24.976869
average_reward_test                  1.552286
round_time_test        0 days 00:00:08.635609
round_time_total       0 days 00:05:49.683950
loss_total                        6412.912284
loss_critic                       8102.760508
loss_actor                        -346.481206
memory_size                       237117.1695 

=== epoch 6/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:42,  4.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  263
episode_length                      38.015209
returns                             55.037721
return_std                          37.132027
average_reward                        1.44796
round_time             0 days 00:05:51.795652
episodes_test                           163.0
episode_length_test                  61.08589
returns_test                        87.142599
return_std_test                     26.750819
average_reward_test                  1.426469
round_time_test        0 days 00:00:08.710952
round_time_total       0 days 00:05:51.796748
loss_total                        6459.933906
loss_critic                       8162.560071
loss_actor                        -350.571295
memory_size                        238321.431 

=== epoch 6/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:22,  5.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  264
episode_length                      37.723485
returns                             54.464692
return_std                          36.148988
average_reward                       1.445297
round_time             0 days 00:05:50.655645
episodes_test                           197.0
episode_length_test                 50.492386
returns_test                        74.823527
return_std_test                     29.179751
average_reward_test                  1.482111
round_time_test        0 days 00:00:08.448897
round_time_total       0 days 00:05:50.656877
loss_total                        6528.869256
loss_critic                       8249.329051
loss_actor                        -352.970494
memory_size                       239532.0475 

=== epoch 6/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:49,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                  250
episode_length                         39.744
returns                             57.804915
return_std                          37.162019
average_reward                       1.454096
round_time             0 days 00:05:51.388837
episodes_test                           218.0
episode_length_test                 45.688073
returns_test                        71.548657
return_std_test                     19.787562
average_reward_test                  1.565786
round_time_test        0 days 00:00:08.714376
round_time_total       0 days 00:05:51.389937
loss_total                        6715.396751
loss_critic                       8481.791719
loss_actor                        -350.183683
memory_size                        240761.875 

=== epoch 6/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  244
episode_length                       40.67623
returns                             58.875685
return_std                          38.677241
average_reward                       1.445639
round_time             0 days 00:05:50.987813
episodes_test                           163.0
episode_length_test                 61.079755
returns_test                        86.743707
return_std_test                     29.277972
average_reward_test                  1.418187
round_time_test        0 days 00:00:08.752848
round_time_total       0 days 00:05:50.989086
loss_total                        6414.660033
loss_critic                       8105.506661
loss_actor                        -348.727047
memory_size                        242029.647 

=== epoch 6/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  239
episode_length                      41.757322
returns                             59.773253
return_std                          37.557615
average_reward                       1.430799
round_time             0 days 00:05:52.897200
episodes_test                            23.0
episode_length_test                429.130435
returns_test                        409.98821
return_std_test                    195.249702
average_reward_test                   0.95739
round_time_test        0 days 00:00:08.472944
round_time_total       0 days 00:05:52.898392
loss_total                        6469.481292
loss_critic                       8174.505509
loss_actor                        -350.616132
memory_size                        243315.187 

=== epoch 6/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  238
episode_length                      42.004202
returns                             59.494455
return_std                           38.17301
average_reward                        1.41625
round_time             0 days 00:05:54.497867
episodes_test                           222.0
episode_length_test                 44.774775
returns_test                        69.992321
return_std_test                     19.377536
average_reward_test                   1.56218
round_time_test        0 days 00:00:08.661646
round_time_total       0 days 00:05:54.498969
loss_total                        6398.579877
loss_critic                       8085.789811
loss_actor                        -350.260417
memory_size                        244555.232 

=== epoch 6/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:47,  5.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  232
episode_length                      42.918103
returns                             60.832151
return_std                          38.858883
average_reward                       1.416795
round_time             0 days 00:05:52.629895
episodes_test                           160.0
episode_length_test                    62.225
returns_test                        88.997037
return_std_test                      26.20581
average_reward_test                   1.43058
round_time_test        0 days 00:00:08.788657
round_time_total       0 days 00:05:52.631001
loss_total                        6502.408884
loss_critic                       8215.500322
loss_actor                        -349.957416
memory_size                       245814.4495 

=== epoch 6/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  227
episode_length                      43.920705
returns                             62.202078
return_std                          42.520234
average_reward                       1.416398
round_time             0 days 00:05:53.171905
episodes_test                           136.0
episode_length_test                 72.992647
returns_test                        97.871076
return_std_test                     37.183988
average_reward_test                  1.339769
round_time_test        0 days 00:00:08.756140
round_time_total       0 days 00:05:53.173015
loss_total                        6454.252434
loss_critic                       8155.259698
loss_actor                        -349.777156
memory_size                        247154.076 

=== epoch 6/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:33,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  231
episode_length                       42.78355
returns                              60.09617
return_std                           41.84234
average_reward                       1.399951
round_time             0 days 00:05:52.750230
episodes_test                           212.0
episode_length_test                 47.070755
returns_test                        73.171796
return_std_test                     18.977188
average_reward_test                  1.553862
round_time_test        0 days 00:00:08.567085
round_time_total       0 days 00:05:52.751324
loss_total                        6349.115929
loss_critic                       8024.088948
loss_actor                        -350.776733
memory_size                       248426.4545 

=== epoch 6/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  228
episode_length                      43.219298
returns                             60.589262
return_std                          43.430492
average_reward                       1.401908
round_time             0 days 00:05:55.971414
episodes_test                           178.0
episode_length_test                 55.910112
returns_test                        81.937584
return_std_test                     27.283241
average_reward_test                  1.466073
round_time_test        0 days 00:00:08.783075
round_time_total       0 days 00:05:55.972759
loss_total                        6366.538939
loss_critic                       8046.197628
loss_actor                        -352.096419
memory_size                        249675.641 

=== epoch 6/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:26,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  228
episode_length                       43.52193
returns                             61.863868
return_std                          42.698603
average_reward                       1.420766
round_time             0 days 00:05:56.495519
episodes_test                           213.0
episode_length_test                 46.896714
returns_test                        71.713128
return_std_test                     24.246395
average_reward_test                  1.528309
round_time_test        0 days 00:00:08.844207
round_time_total       0 days 00:05:56.496647
loss_total                        6393.998927
loss_critic                       8081.136212
loss_actor                        -354.550812
memory_size                       250949.1195 

=== epoch 6/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  225
episode_length                      44.333333
returns                             63.057109
return_std                          43.533451
average_reward                        1.42134
round_time             0 days 00:05:53.773668
episodes_test                           226.0
episode_length_test                 44.159292
returns_test                        70.635495
return_std_test                     25.351177
average_reward_test                  1.599113
round_time_test        0 days 00:00:08.599712
round_time_total       0 days 00:05:53.774919
loss_total                        6570.039776
loss_critic                       8300.794865
loss_actor                        -352.981185
memory_size                       252226.6465 

=== epoch 6/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:59,  5.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  240
episode_length                           41.3
returns                             57.970092
return_std                          40.886128
average_reward                       1.404013
round_time             0 days 00:05:55.486665
episodes_test                           112.0
episode_length_test                     88.75
returns_test                       111.606843
return_std_test                     63.024516
average_reward_test                  1.258651
round_time_test        0 days 00:00:08.736718
round_time_total       0 days 00:05:55.487780
loss_total                        6455.785033
loss_critic                       8157.554647
loss_actor                        -351.293964
memory_size                        253501.569 

=== epoch 6/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  240
episode_length                          41.45
returns                             58.587673
return_std                          41.129143
average_reward                       1.411868
round_time             0 days 00:05:54.738789
episodes_test                           179.0
episode_length_test                 55.631285
returns_test                        80.386251
return_std_test                     31.814319
average_reward_test                  1.444994
round_time_test        0 days 00:00:08.696280
round_time_total       0 days 00:05:54.740390
loss_total                        6422.462504
loss_critic                       8116.125973
loss_actor                        -352.191951
memory_size                       254691.3985 

=== epoch 6/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  240
episode_length                      41.583333
returns                              59.75564
return_std                          40.937131
average_reward                       1.437001
round_time             0 days 00:05:56.527651
episodes_test                           136.0
episode_length_test                 73.132353
returns_test                        98.214705
return_std_test                      42.38443
average_reward_test                  1.343662
round_time_test        0 days 00:00:08.764426
round_time_total       0 days 00:05:56.528758
loss_total                        6424.238543
loss_critic                       8118.105807
loss_actor                        -351.231111
memory_size                        255945.132 

=== epoch 6/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  237
episode_length                      41.970464
returns                             59.824981
return_std                          40.991881
average_reward                       1.425479
round_time             0 days 00:05:56.206380
episodes_test                           228.0
episode_length_test                 43.688596
returns_test                        68.574012
return_std_test                     21.565957
average_reward_test                  1.569633
round_time_test        0 days 00:00:08.541112
round_time_total       0 days 00:05:56.207474
loss_total                        6284.130196
loss_critic                       7942.719525
loss_actor                        -350.227706
memory_size                       257229.7575 

=== epoch 6/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:41,  5.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  242
episode_length                       41.07438
returns                             59.277119
return_std                          40.489063
average_reward                        1.44403
round_time             0 days 00:05:55.707334
episodes_test                           174.0
episode_length_test                 57.454023
returns_test                        83.798672
return_std_test                     24.727815
average_reward_test                  1.458388
round_time_test        0 days 00:00:09.011313
round_time_total       0 days 00:05:55.708573
loss_total                        6289.472613
loss_critic                       7949.480302
loss_actor                        -350.558708
memory_size                        258489.366 

=== epoch 6/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:28,  5.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  239
episode_length                      41.527197
returns                             60.199347
return_std                          40.261491
average_reward                       1.450775
round_time             0 days 00:05:55.007252
episodes_test                           205.0
episode_length_test                 48.653659
returns_test                        75.014468
return_std_test                     20.505074
average_reward_test                  1.541642
round_time_test        0 days 00:00:08.674583
round_time_total       0 days 00:05:55.008394
loss_total                        6304.588299
loss_critic                       7968.114338
loss_actor                        -349.516458
memory_size                        259724.457 

=== epoch 6/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:10,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                  233
episode_length                      42.652361
returns                             62.263976
return_std                          39.998519
average_reward                       1.459859
round_time             0 days 00:05:54.488640
episodes_test                           196.0
episode_length_test                 50.959184
returns_test                        75.705927
return_std_test                     27.831494
average_reward_test                  1.485235
round_time_test        0 days 00:00:09.159740
round_time_total       0 days 00:05:54.489739
loss_total                        6204.393618
loss_critic                       7843.053077
loss_actor                        -350.244765
memory_size                       260959.5015 

=== epoch 6/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  234
episode_length                      42.512821
returns                             61.100302
return_std                          40.217941
average_reward                       1.438793
round_time             0 days 00:05:56.678723
episodes_test                           128.0
episode_length_test                  77.40625
returns_test                       102.682118
return_std_test                     39.219042
average_reward_test                  1.325447
round_time_test        0 days 00:00:08.660516
round_time_total       0 days 00:05:56.680218
loss_total                        6367.972485
loss_critic                       8047.461484
loss_actor                        -349.984047
memory_size                        262279.564 

=== epoch 6/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  238
episode_length                      41.743697
returns                             60.029158
return_std                          40.806918
average_reward                       1.437581
round_time             0 days 00:05:57.631253
episodes_test                           169.0
episode_length_test                 59.136095
returns_test                        83.670948
return_std_test                     32.077592
average_reward_test                  1.414532
round_time_test        0 days 00:00:08.720626
round_time_total       0 days 00:05:57.632357
loss_total                        6258.652614
loss_critic                       7911.182154
loss_actor                        -351.466085
memory_size                        263494.838 

=== epoch 6/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  235
episode_length                      42.306383
returns                             58.939691
return_std                          42.191275
average_reward                       1.393923
round_time             0 days 00:05:55.135903
episodes_test                           155.0
episode_length_test                 64.380645
returns_test                        90.478581
return_std_test                     28.801001
average_reward_test                  1.405322
round_time_test        0 days 00:00:08.565116
round_time_total       0 days 00:05:55.137276
loss_total                         6291.78016
loss_critic                       7952.711694
loss_actor                        -351.946494
memory_size                       264789.4945 

=== epoch 6/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:00,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  230
episode_length                      43.204348
returns                             59.557127
return_std                          40.597407
average_reward                       1.380144
round_time             0 days 00:05:57.835806
episodes_test                            43.0
episode_length_test                230.139535
returns_test                       253.886109
return_std_test                     135.91661
average_reward_test                  1.104301
round_time_test        0 days 00:00:08.798074
round_time_total       0 days 00:05:57.836928
loss_total                        6347.143289
loss_critic                       8021.955929
loss_actor                        -352.107791
memory_size                        266042.576 

=== epoch 6/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  235
episode_length                      42.446809
returns                             57.491646
return_std                          39.602671
average_reward                       1.354892
round_time             0 days 00:05:58.109931
episodes_test                            24.0
episode_length_test                411.916667
returns_test                       416.819662
return_std_test                    238.256875
average_reward_test                  1.012582
round_time_test        0 days 00:00:08.654655
round_time_total       0 days 00:05:58.111039
loss_total                        6252.100713
loss_critic                       7902.751643
loss_actor                        -350.503529
memory_size                        267291.941 

=== epoch 6/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:16,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  228
episode_length                      43.820175
returns                             59.507038
return_std                          41.498303
average_reward                       1.357769
round_time             0 days 00:05:56.613552
episodes_test                           113.0
episode_length_test                 88.433628
returns_test                       111.898305
return_std_test                     52.216838
average_reward_test                    1.2652
round_time_test        0 days 00:00:08.869794
round_time_total       0 days 00:05:56.614839
loss_total                         6239.10664
loss_critic                       7886.921065
loss_actor                         -352.15155
memory_size                       268611.7975 

=== epoch 6/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  217
episode_length                      45.824885
returns                             62.173982
return_std                          41.675011
average_reward                       1.357281
round_time             0 days 00:05:58.798211
episodes_test                           121.0
episode_length_test                 82.066116
returns_test                       108.755964
return_std_test                     41.597588
average_reward_test                   1.32569
round_time_test        0 days 00:00:08.746171
round_time_total       0 days 00:05:58.799534
loss_total                        6290.893481
loss_critic                       7951.891957
loss_actor                        -353.101006
memory_size                       269961.9455 

=== epoch 6/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:47,  5.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  212
episode_length                      47.080189
returns                             64.797544
return_std                          40.937297
average_reward                       1.375652
round_time             0 days 00:05:57.850615
episodes_test                           118.0
episode_length_test                 84.161017
returns_test                       109.778931
return_std_test                     40.207187
average_reward_test                  1.305074
round_time_test        0 days 00:00:08.680647
round_time_total       0 days 00:05:57.851887
loss_total                        6355.067551
loss_critic                       8032.080334
loss_actor                        -352.984172
memory_size                       271297.3225 

=== epoch 6/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
episodes                                  211
episode_length                      47.194313
returns                             65.283044
return_std                          41.000396
average_reward                        1.38132
round_time             0 days 00:06:00.126108
episodes_test                           234.0
episode_length_test                 42.666667
returns_test                        67.275348
return_std_test                     21.389218
average_reward_test                  1.576217
round_time_test        0 days 00:00:08.797026
round_time_total       0 days 00:06:00.127399
loss_total                        6246.884314
loss_critic                       7896.669409
loss_actor                        -352.256626
memory_size                        272619.854 

=== epoch 6/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  195
episode_length                      50.953846
returns                             70.919229
return_std                          43.722447
average_reward                       1.394041
round_time             0 days 00:05:56.249358
episodes_test                           169.0
episode_length_test                 59.130178
returns_test                        85.090499
return_std_test                     27.023302
average_reward_test                  1.438642
round_time_test        0 days 00:00:08.648802
round_time_total       0 days 00:05:56.250608
loss_total                          6296.2801
loss_critic                       7958.909407
loss_actor                        -354.237672
memory_size                       273994.5875 

=== epoch 6/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                  195
episode_length                      50.651282
returns                             69.693746
return_std                          42.191887
average_reward                       1.374616
round_time             0 days 00:05:59.443750
episodes_test                           123.0
episode_length_test                 80.691057
returns_test                        104.26094
return_std_test                     48.958049
average_reward_test                  1.292763
round_time_test        0 days 00:00:08.526607
round_time_total       0 days 00:05:59.444975
loss_total                        6277.615023
loss_critic                       7934.968552
loss_actor                        -351.799656
memory_size                       275428.3575 

=== epoch 6/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:49,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  199
episode_length                      49.979899
returns                             66.268581
return_std                          42.418639
average_reward                       1.328989
round_time             0 days 00:05:59.152534
episodes_test                            36.0
episode_length_test                272.444444
returns_test                       259.100016
return_std_test                    205.313448
average_reward_test                   0.94959
round_time_test        0 days 00:00:08.707737
round_time_total       0 days 00:05:59.154059
loss_total                        6215.443733
loss_critic                       7857.597019
loss_actor                         -353.16996
memory_size                        276826.309 

=== epoch 6/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:54,  4.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  196
episode_length                       50.80102
returns                             66.117252
return_std                          43.448155
average_reward                       1.301144
round_time             0 days 00:05:58.599841
episodes_test                            67.0
episode_length_test                 146.19403
returns_test                        171.19669
return_std_test                     85.493635
average_reward_test                  1.169499
round_time_test        0 days 00:00:08.546336
round_time_total       0 days 00:05:58.600954
loss_total                        6284.432572
loss_critic                       7944.221392
loss_actor                        -354.723255
memory_size                       278138.2185 

=== epoch 6/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.59it/s]
episodes                                  190
episode_length                      52.415789
returns                             67.762523
return_std                          45.631379
average_reward                       1.294126
round_time             0 days 00:05:58.690238
episodes_test                           142.0
episode_length_test                  70.28169
returns_test                        94.473197
return_std_test                     38.053597
average_reward_test                  1.343866
round_time_test        0 days 00:00:08.747499
round_time_total       0 days 00:05:58.691344
loss_total                        6333.382235
loss_critic                       8004.965771
loss_actor                        -352.952455
memory_size                        279539.563 

=== epoch 6/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  203
episode_length                      48.975369
returns                             61.943729
return_std                          42.130908
average_reward                       1.266773
round_time             0 days 00:05:58.044322
episodes_test                           109.0
episode_length_test                 91.155963
returns_test                       116.154824
return_std_test                        49.921
average_reward_test                  1.274689
round_time_test        0 days 00:00:08.807271
round_time_total       0 days 00:05:58.045461
loss_total                        6208.759334
loss_critic                        7849.49476
loss_actor                        -354.182895
memory_size                       280832.5115 

=== epoch 6/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.56it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  202
episode_length                      48.980198
returns                             62.483808
return_std                          43.013352
average_reward                       1.270466
round_time             0 days 00:05:59.904036
episodes_test                            41.0
episode_length_test                241.146341
returns_test                        243.44694
return_std_test                    122.768668
average_reward_test                  1.010192
round_time_test        0 days 00:00:08.558538
round_time_total       0 days 00:05:59.905306
loss_total                        6292.334042
loss_critic                       7954.364891
loss_actor                        -355.789922
memory_size                       282171.4635 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 7/10 ===== round 1/50 ======================================
  1%|          | 11/2000 [00:02<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:41<00:00,  5.85it/s]
episodes                                   41
episode_length                      47.926829
returns                             63.439146
return_std                          44.514491
average_reward                       1.321677
round_time             0 days 00:05:41.858424
episodes_test                            42.0
episode_length_test                237.619048
returns_test                        216.24702
return_std_test                    110.040646
average_reward_test                  0.910675
round_time_test        0 days 00:00:08.712449
round_time_total       0 days 00:05:41.859531
loss_total                        6325.461063
loss_critic                       7995.556426
loss_actor                        -354.920927
memory_size                       283506.8805 

=== epoch 7/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:31,  6.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                   75
episode_length                          53.12
returns                             68.716226
return_std                           50.37489
average_reward                       1.293552
round_time             0 days 00:05:43.064991
episodes_test                           138.0
episode_length_test                 72.050725
returns_test                        99.016122
return_std_test                     28.321411
average_reward_test                  1.373966
round_time_test        0 days 00:00:08.716860
round_time_total       0 days 00:05:43.066136
loss_total                        6237.998154
loss_critic                       7886.324189
loss_actor                        -355.306495
memory_size                        284874.405 

=== epoch 7/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  1%|          | 11/2000 [00:01<05:27,  6.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                  116
episode_length                       51.37931
returns                             66.504426
return_std                          46.579048
average_reward                       1.294758
round_time             0 days 00:05:46.334152
episodes_test                            62.0
episode_length_test                159.064516
returns_test                       182.513514
return_std_test                    130.183837
average_reward_test                  1.146841
round_time_test        0 days 00:00:08.658690
round_time_total       0 days 00:05:46.335407
loss_total                        6258.218069
loss_critic                       7911.565312
loss_actor                        -355.171468
memory_size                        286313.089 

=== epoch 7/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  155
episode_length                      51.180645
returns                             65.163278
return_std                          46.755485
average_reward                       1.271283
round_time             0 days 00:05:45.933504
episodes_test                            50.0
episode_length_test                     199.1
returns_test                       223.715034
return_std_test                    103.110391
average_reward_test                  1.124082
round_time_test        0 days 00:00:08.529959
round_time_total       0 days 00:05:45.934841
loss_total                        6220.928252
loss_critic                       7865.229283
loss_actor                        -356.276451
memory_size                       287663.5005 

=== epoch 7/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:48,  5.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  194
episode_length                      51.402062
returns                             64.153343
return_std                           46.73328
average_reward                       1.246914
round_time             0 days 00:05:46.920555
episodes_test                            48.0
episode_length_test                208.020833
returns_test                       229.289607
return_std_test                    160.642899
average_reward_test                  1.102157
round_time_test        0 days 00:00:08.573689
round_time_total       0 days 00:05:46.921780
loss_total                        6234.773094
loss_critic                       7882.762301
loss_actor                        -357.184289
memory_size                        289042.551 

=== epoch 7/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  188
episode_length                      53.058511
returns                             66.496915
return_std                          46.566711
average_reward                       1.251744
round_time             0 days 00:05:45.889603
episodes_test                           146.0
episode_length_test                 68.356164
returns_test                        93.982853
return_std_test                     35.174607
average_reward_test                  1.374696
round_time_test        0 days 00:00:08.714094
round_time_total       0 days 00:05:45.890717
loss_total                        6233.212444
loss_critic                       7880.721685
loss_actor                        -356.825096
memory_size                        290436.528 

=== epoch 7/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:40,  5.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  195
episode_length                      50.825641
returns                              63.44471
return_std                          43.058276
average_reward                        1.25051
round_time             0 days 00:05:48.435127
episodes_test                            88.0
episode_length_test                113.068182
returns_test                       137.939401
return_std_test                      78.17987
average_reward_test                  1.220883
round_time_test        0 days 00:00:08.912135
round_time_total       0 days 00:05:48.436242
loss_total                        6102.273174
loss_critic                       7716.975527
loss_actor                        -356.536775
memory_size                       291807.2665 

=== epoch 7/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                  199
episode_length                      50.105528
returns                             61.950433
return_std                          42.964862
average_reward                       1.237262
round_time             0 days 00:05:48.669977
episodes_test                            38.0
episode_length_test                255.552632
returns_test                       233.966065
return_std_test                     155.27849
average_reward_test                  0.916261
round_time_test        0 days 00:00:08.772510
round_time_total       0 days 00:05:48.671094
loss_total                        6322.204988
loss_critic                       7992.062747
loss_actor                        -357.226656
memory_size                         293129.09 

=== epoch 7/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:13,  5.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                  192
episode_length                      51.484375
returns                             64.196257
return_std                          44.097044
average_reward                       1.248769
round_time             0 days 00:05:45.738370
episodes_test                            47.0
episode_length_test                212.595745
returns_test                        205.56712
return_std_test                    159.009013
average_reward_test                  0.966937
round_time_test        0 days 00:00:08.585759
round_time_total       0 days 00:05:45.739463
loss_total                        6273.010188
loss_critic                       7930.610126
loss_actor                        -357.390151
memory_size                        294525.403 

=== epoch 7/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  196
episode_length                      50.596939
returns                             63.370829
return_std                          43.286537
average_reward                        1.25352
round_time             0 days 00:05:49.792619
episodes_test                            39.0
episode_length_test                255.025641
returns_test                       252.585052
return_std_test                    141.693518
average_reward_test                  0.990927
round_time_test        0 days 00:00:08.690364
round_time_total       0 days 00:05:49.794093
loss_total                        6289.870439
loss_critic                       7951.712845
loss_actor                        -357.499733
memory_size                        295945.489 

=== epoch 7/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:31,  6.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                  206
episode_length                      48.242718
returns                             59.449917
return_std                          42.302292
average_reward                        1.23138
round_time             0 days 00:05:48.348024
episodes_test                            48.0
episode_length_test                202.833333
returns_test                       181.539716
return_std_test                    114.928502
average_reward_test                  0.896364
round_time_test        0 days 00:00:08.886064
round_time_total       0 days 00:05:48.349116
loss_total                         6125.38415
loss_critic                       7745.835412
loss_actor                        -356.421458
memory_size                        297222.914 

=== epoch 7/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:45,  5.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  204
episode_length                      48.887255
returns                             59.121254
return_std                          42.773098
average_reward                       1.210207
round_time             0 days 00:05:50.275615
episodes_test                           125.0
episode_length_test                    79.504
returns_test                       103.980592
return_std_test                     64.602032
average_reward_test                  1.308263
round_time_test        0 days 00:00:08.708106
round_time_total       0 days 00:05:50.276717
loss_total                        6137.636259
loss_critic                       7761.141786
loss_actor                        -356.386413
memory_size                        298540.277 

=== epoch 7/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:49,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  206
episode_length                       47.61165
returns                             58.027602
return_std                          42.094242
average_reward                       1.216824
round_time             0 days 00:05:48.882610
episodes_test                           182.0
episode_length_test                  54.71978
returns_test                        80.145803
return_std_test                     27.295062
average_reward_test                   1.46488
round_time_test        0 days 00:00:08.565981
round_time_total       0 days 00:05:48.883739
loss_total                        6096.384509
loss_critic                       7709.438956
loss_actor                        -355.833826
memory_size                       299833.1445 

=== epoch 7/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  213
episode_length                      46.676056
returns                             56.713537
return_std                          40.426714
average_reward                        1.21478
round_time             0 days 00:05:52.258355
episodes_test                            83.0
episode_length_test                118.698795
returns_test                        97.686081
return_std_test                     62.414594
average_reward_test                  0.824435
round_time_test        0 days 00:00:08.711863
round_time_total       0 days 00:05:52.259529
loss_total                        6126.343022
loss_critic                        7747.05502
loss_actor                         -356.50551
memory_size                        301208.653 

=== epoch 7/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  214
episode_length                      46.686916
returns                             57.211112
return_std                          41.131985
average_reward                       1.225114
round_time             0 days 00:05:49.120507
episodes_test                            93.0
episode_length_test                105.849462
returns_test                        79.723928
return_std_test                     30.202882
average_reward_test                  0.755233
round_time_test        0 days 00:00:08.626161
round_time_total       0 days 00:05:49.121645
loss_total                        6248.903011
loss_critic                       7899.493866
loss_actor                        -353.460943
memory_size                       302523.6075 

=== epoch 7/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  220
episode_length                      45.209091
returns                             55.698056
return_std                          40.804719
average_reward                       1.231835
round_time             0 days 00:05:52.980790
episodes_test                            53.0
episode_length_test                186.264151
returns_test                       164.419762
return_std_test                    125.419822
average_reward_test                  0.883802
round_time_test        0 days 00:00:08.803369
round_time_total       0 days 00:05:52.981890
loss_total                        5956.407118
loss_critic                       7533.868592
loss_actor                         -353.43925
memory_size                        303817.478 

=== epoch 7/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:25,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  218
episode_length                       45.62844
returns                               56.0451
return_std                          41.350901
average_reward                       1.228034
round_time             0 days 00:05:50.462405
episodes_test                            40.0
episode_length_test                   248.875
returns_test                       246.792825
return_std_test                    136.702139
average_reward_test                  0.992531
round_time_test        0 days 00:00:08.691418
round_time_total       0 days 00:05:50.463513
loss_total                        6035.832437
loss_critic                       7633.685963
loss_actor                        -355.582143
memory_size                       305120.8155 

=== epoch 7/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:49,  4.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  214
episode_length                      46.593458
returns                             57.289977
return_std                          42.696138
average_reward                       1.229308
round_time             0 days 00:05:51.829284
episodes_test                            62.0
episode_length_test                160.032258
returns_test                       134.237689
return_std_test                     59.497962
average_reward_test                  0.839264
round_time_test        0 days 00:00:08.566666
round_time_total       0 days 00:05:51.830374
loss_total                        6103.711222
loss_critic                       7718.279282
loss_actor                        -354.561557
memory_size                        306476.552 

=== epoch 7/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                  223
episode_length                      44.672646
returns                             54.303192
return_std                          40.187222
average_reward                       1.217234
round_time             0 days 00:05:50.835382
episodes_test                            64.0
episode_length_test                  154.3125
returns_test                       171.816937
return_std_test                    129.569482
average_reward_test                  1.115037
round_time_test        0 days 00:00:08.700166
round_time_total       0 days 00:05:50.836478
loss_total                        6025.158434
loss_critic                       7619.992136
loss_actor                        -354.176905
memory_size                       307764.6615 

=== epoch 7/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:47,  4.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  219
episode_length                        45.3379
returns                             54.507675
return_std                           39.68903
average_reward                       1.202108
round_time             0 days 00:05:51.860939
episodes_test                            76.0
episode_length_test                131.052632
returns_test                       147.014148
return_std_test                     81.613561
average_reward_test                  1.123086
round_time_test        0 days 00:00:08.493388
round_time_total       0 days 00:05:51.862095
loss_total                        6007.512085
loss_critic                       7598.016242
loss_actor                        -354.505051
memory_size                       309075.8125 

=== epoch 7/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  217
episode_length                      45.571429
returns                             53.177634
return_std                          39.268937
average_reward                       1.167252
round_time             0 days 00:05:52.712053
episodes_test                           176.0
episode_length_test                 56.659091
returns_test                        54.402908
return_std_test                     51.164105
average_reward_test                  0.960894
round_time_test        0 days 00:00:08.640554
round_time_total       0 days 00:05:52.713297
loss_total                        6211.368972
loss_critic                       7853.451379
loss_actor                        -356.961184
memory_size                        310319.724 

=== epoch 7/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:32,  5.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  219
episode_length                      45.296804
returns                             51.773707
return_std                           40.05436
average_reward                       1.144535
round_time             0 days 00:05:53.184920
episodes_test                           154.0
episode_length_test                 64.448052
returns_test                        60.154961
return_std_test                     52.132274
average_reward_test                  0.934909
round_time_test        0 days 00:00:08.688530
round_time_total       0 days 00:05:53.186097
loss_total                        6115.092495
loss_critic                       7733.456917
loss_actor                        -358.365736
memory_size                        311627.364 

=== epoch 7/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  210
episode_length                      47.071429
returns                             53.139348
return_std                          43.231142
average_reward                       1.132171
round_time             0 days 00:05:52.515350
episodes_test                            51.0
episode_length_test                194.980392
returns_test                       190.999554
return_std_test                    110.407661
average_reward_test                   0.97984
round_time_test        0 days 00:00:08.587481
round_time_total       0 days 00:05:52.516625
loss_total                        6229.113423
loss_critic                        7875.93664
loss_actor                        -358.179982
memory_size                       313083.2745 

=== epoch 7/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:36,  5.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  203
episode_length                      48.724138
returns                             55.436591
return_std                          45.561001
average_reward                       1.140831
round_time             0 days 00:05:54.672067
episodes_test                            56.0
episode_length_test                    176.75
returns_test                       158.881493
return_std_test                    116.654873
average_reward_test                  0.901651
round_time_test        0 days 00:00:08.782054
round_time_total       0 days 00:05:54.673237
loss_total                        6089.412913
loss_critic                       7700.778371
loss_actor                        -356.049425
memory_size                        314497.167 

=== epoch 7/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  201
episode_length                      49.462687
returns                             57.906748
return_std                          46.035619
average_reward                        1.17284
round_time             0 days 00:05:52.894966
episodes_test                           124.0
episode_length_test                 80.548387
returns_test                       107.168101
return_std_test                     36.524419
average_reward_test                  1.330056
round_time_test        0 days 00:00:08.804700
round_time_total       0 days 00:05:52.896065
loss_total                        6069.986206
loss_critic                       7676.630894
loss_actor                        -356.593057
memory_size                       315854.7375 

=== epoch 7/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  193
episode_length                      51.694301
returns                             62.980063
return_std                          46.121879
average_reward                       1.217457
round_time             0 days 00:05:53.111746
episodes_test                            98.0
episode_length_test                 98.897959
returns_test                       123.380236
return_std_test                     58.830075
average_reward_test                  1.237573
round_time_test        0 days 00:00:08.777153
round_time_total       0 days 00:05:53.112856
loss_total                        6044.469085
loss_critic                         7644.7594
loss_actor                        -356.692683
memory_size                        317223.542 

=== epoch 7/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:38,  5.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  197
episode_length                      50.324873
returns                             63.239108
return_std                          43.648583
average_reward                       1.257552
round_time             0 days 00:05:52.725500
episodes_test                           107.0
episode_length_test                 91.663551
returns_test                       116.721382
return_std_test                     49.705183
average_reward_test                  1.269925
round_time_test        0 days 00:00:08.472138
round_time_total       0 days 00:05:52.726607
loss_total                        6011.973773
loss_critic                       7604.299229
loss_actor                        -357.328569
memory_size                       318524.3895 

=== epoch 7/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:35,  5.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  207
episode_length                       47.42029
returns                              59.16819
return_std                          40.966576
average_reward                       1.241989
round_time             0 days 00:05:55.873425
episodes_test                            65.0
episode_length_test                153.615385
returns_test                       141.923372
return_std_test                     83.603577
average_reward_test                  0.924342
round_time_test        0 days 00:00:08.764761
round_time_total       0 days 00:05:55.874724
loss_total                        6087.638553
loss_critic                       7699.131255
loss_actor                        -358.332735
memory_size                        319783.864 

=== epoch 7/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:26,  5.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                  198
episode_length                      50.217172
returns                              61.38378
return_std                          43.829183
average_reward                       1.224017
round_time             0 days 00:05:52.625058
episodes_test                            84.0
episode_length_test                118.642857
returns_test                        93.619954
return_std_test                     46.471896
average_reward_test                  0.789635
round_time_test        0 days 00:00:08.666112
round_time_total       0 days 00:05:52.626399
loss_total                        6140.270353
loss_critic                       7765.408585
loss_actor                        -360.283119
memory_size                        321281.433 

=== epoch 7/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  193
episode_length                       51.46114
returns                             61.904093
return_std                          45.113354
average_reward                       1.204678
round_time             0 days 00:05:56.252371
episodes_test                            54.0
episode_length_test                183.425926
returns_test                        162.93674
return_std_test                     98.230324
average_reward_test                  0.888755
round_time_test        0 days 00:00:08.763577
round_time_total       0 days 00:05:56.253665
loss_total                        6155.172381
loss_critic                       7783.407628
loss_actor                        -357.769116
memory_size                        322698.229 

=== epoch 7/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                  184
episode_length                      53.907609
returns                              63.88385
return_std                          46.722646
average_reward                       1.185988
round_time             0 days 00:05:54.618922
episodes_test                            78.0
episode_length_test                 126.25641
returns_test                        150.52847
return_std_test                     67.717134
average_reward_test                  1.190054
round_time_test        0 days 00:00:08.682589
round_time_total       0 days 00:05:54.620071
loss_total                        5992.155717
loss_critic                       7579.389653
loss_actor                        -356.780574
memory_size                       324179.0585 

=== epoch 7/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  173
episode_length                      57.508671
returns                             67.758006
return_std                          49.033121
average_reward                        1.17734
round_time             0 days 00:05:57.152320
episodes_test                            64.0
episode_length_test                153.953125
returns_test                       136.036107
return_std_test                     73.146411
average_reward_test                  0.883495
round_time_test        0 days 00:00:08.754819
round_time_total       0 days 00:05:57.153706
loss_total                        6041.614206
loss_critic                       7641.908041
loss_actor                        -359.561688
memory_size                        325684.839 

=== epoch 7/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:04,  4.69it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  168
episode_length                      58.672619
returns                             69.735937
return_std                          46.162708
average_reward                       1.185864
round_time             0 days 00:05:54.163476
episodes_test                            52.0
episode_length_test                191.442308
returns_test                       184.104342
return_std_test                     95.051614
average_reward_test                  0.961231
round_time_test        0 days 00:00:08.625397
round_time_total       0 days 00:05:54.164655
loss_total                         6059.68901
loss_critic                        7664.18188
loss_actor                        -358.282986
memory_size                       327095.0085 

=== epoch 7/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  172
episode_length                      57.930233
returns                              68.89082
return_std                          43.274265
average_reward                       1.187644
round_time             0 days 00:05:56.361694
episodes_test                            58.0
episode_length_test                171.672414
returns_test                       169.388757
return_std_test                     93.994612
average_reward_test                  0.987722
round_time_test        0 days 00:00:08.736629
round_time_total       0 days 00:05:56.362789
loss_total                        5968.711515
loss_critic                       7550.155312
loss_actor                         -357.06418
memory_size                        328473.322 

=== epoch 7/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                  175
episode_length                      56.897143
returns                             66.846451
return_std                          43.958065
average_reward                       1.176089
round_time             0 days 00:05:56.438854
episodes_test                            66.0
episode_length_test                     150.0
returns_test                       122.923911
return_std_test                     58.130296
average_reward_test                  0.821299
round_time_test        0 days 00:00:08.626178
round_time_total       0 days 00:05:56.439974
loss_total                         6041.36409
loss_critic                        7641.68117
loss_actor                        -359.904776
memory_size                        329904.865 

=== epoch 7/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  168
episode_length                      58.547619
returns                             68.069297
return_std                           46.99903
average_reward                       1.159471
round_time             0 days 00:05:56.005513
episodes_test                            81.0
episode_length_test                122.987654
returns_test                        95.605953
return_std_test                     35.612393
average_reward_test                   0.77868
round_time_test        0 days 00:00:08.528891
round_time_total       0 days 00:05:56.006625
loss_total                        6140.813711
loss_critic                       7766.630091
loss_actor                        -362.452336
memory_size                        331375.387 

=== epoch 7/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:17,  5.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  170
episode_length                      58.305882
returns                             66.196751
return_std                           49.21909
average_reward                       1.138326
round_time             0 days 00:05:57.677478
episodes_test                            72.0
episode_length_test                138.472222
returns_test                       111.856198
return_std_test                     50.512884
average_reward_test                  0.808575
round_time_test        0 days 00:00:08.813431
round_time_total       0 days 00:05:57.678583
loss_total                        6183.544037
loss_critic                       7820.529091
loss_actor                        -364.396673
memory_size                        332918.696 

=== epoch 7/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:27,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  163
episode_length                      61.233129
returns                              70.26084
return_std                          52.136342
average_reward                       1.146868
round_time             0 days 00:05:55.038300
episodes_test                           110.0
episode_length_test                 90.690909
returns_test                       115.443091
return_std_test                     65.171333
average_reward_test                  1.272906
round_time_test        0 days 00:00:08.600362
round_time_total       0 days 00:05:55.039682
loss_total                        6125.098718
loss_critic                       7746.964467
loss_actor                        -362.364807
memory_size                       334382.1365 

=== epoch 7/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  163
episode_length                      61.018405
returns                             70.858155
return_std                          53.099412
average_reward                       1.159497
round_time             0 days 00:05:57.756351
episodes_test                            79.0
episode_length_test                124.936709
returns_test                       148.555004
return_std_test                     76.676562
average_reward_test                  1.188715
round_time_test        0 days 00:00:08.629234
round_time_total       0 days 00:05:57.757620
loss_total                        6147.413669
loss_critic                       7774.808856
loss_actor                        -362.167623
memory_size                         335882.81 

=== epoch 7/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:44,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  163
episode_length                      60.705521
returns                             70.088916
return_std                          55.887696
average_reward                       1.151585
round_time             0 days 00:05:54.282001
episodes_test                            62.0
episode_length_test                161.129032
returns_test                       137.958499
return_std_test                     78.790021
average_reward_test                  0.856309
round_time_test        0 days 00:00:08.674719
round_time_total       0 days 00:05:54.283115
loss_total                        6247.426935
loss_critic                       7900.445766
loss_actor                        -364.648946
memory_size                       337325.2415 

=== epoch 7/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  175
episode_length                      56.617143
returns                             65.418447
return_std                          51.679066
average_reward                       1.153532
round_time             0 days 00:05:56.747616
episodes_test                            44.0
episode_length_test                226.409091
returns_test                       214.847882
return_std_test                    106.640379
average_reward_test                  0.949808
round_time_test        0 days 00:00:08.714767
round_time_total       0 days 00:05:56.748728
loss_total                        6207.240898
loss_critic                       7850.291312
loss_actor                        -364.961301
memory_size                        338736.138 

=== epoch 7/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:17,  5.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  171
episode_length                      58.128655
returns                             67.535262
return_std                          51.332631
average_reward                       1.164308
round_time             0 days 00:05:58.077665
episodes_test                            62.0
episode_length_test                161.112903
returns_test                       137.865134
return_std_test                     78.294691
average_reward_test                  0.855989
round_time_test        0 days 00:00:08.721513
round_time_total       0 days 00:05:58.078762
loss_total                        6065.750459
loss_critic                       7673.153225
loss_actor                        -363.861155
memory_size                         340176.64 

=== epoch 7/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:46,  5.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                  182
episode_length                      54.532967
returns                             62.387383
return_std                          49.828954
average_reward                        1.14396
round_time             0 days 00:05:56.580372
episodes_test                            35.0
episode_length_test                268.457143
returns_test                       271.530899
return_std_test                     119.07049
average_reward_test                  1.008827
round_time_test        0 days 00:00:08.662988
round_time_total       0 days 00:05:56.581487
loss_total                          6076.5177
loss_critic                       7686.057779
loss_actor                        -361.643118
memory_size                        341576.327 

=== epoch 7/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:12,  5.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                  180
episode_length                      54.861111
returns                             62.442105
return_std                           49.18892
average_reward                       1.139407
round_time             0 days 00:05:58.144231
episodes_test                            39.0
episode_length_test                247.641026
returns_test                       239.957795
return_std_test                    139.744231
average_reward_test                  0.970206
round_time_test        0 days 00:00:08.727656
round_time_total       0 days 00:05:58.145333
loss_total                        6015.462082
loss_critic                       7609.551735
loss_actor                        -360.897015
memory_size                       343020.7165 

=== epoch 7/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:13,  5.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  174
episode_length                       56.62069
returns                             65.240655
return_std                          47.988712
average_reward                       1.148842
round_time             0 days 00:05:55.987246
episodes_test                            51.0
episode_length_test                194.137255
returns_test                       207.408891
return_std_test                     92.184572
average_reward_test                  1.067655
round_time_test        0 days 00:00:08.574351
round_time_total       0 days 00:05:55.988526
loss_total                         5909.74688
loss_critic                       7477.837674
loss_actor                        -362.616785
memory_size                       344471.3245 

=== epoch 7/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:49,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.60it/s]
episodes                                  165
episode_length                      60.424242
returns                             67.717205
return_std                          50.954051
average_reward                       1.120006
round_time             0 days 00:05:57.395722
episodes_test                            90.0
episode_length_test                110.755556
returns_test                        83.150014
return_std_test                     38.872663
average_reward_test                  0.752592
round_time_test        0 days 00:00:08.725942
round_time_total       0 days 00:05:57.396834
loss_total                        6193.179622
loss_critic                        7833.48546
loss_actor                        -368.044269
memory_size                       345972.5215 

=== epoch 7/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:47,  5.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  160
episode_length                       62.03125
returns                             69.167449
return_std                          50.058348
average_reward                       1.114323
round_time             0 days 00:05:59.161393
episodes_test                            49.0
episode_length_test                200.061224
returns_test                       175.976079
return_std_test                      84.40414
average_reward_test                  0.880354
round_time_test        0 days 00:00:08.573544
round_time_total       0 days 00:05:59.162488
loss_total                        6181.130165
loss_critic                       7818.324248
loss_actor                        -367.646684
memory_size                       347559.5025 

=== epoch 7/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                  147
episode_length                      67.632653
returns                             74.686211
return_std                          49.763175
average_reward                       1.105776
round_time             0 days 00:05:57.732163
episodes_test                            77.0
episode_length_test                 128.38961
returns_test                       101.756688
return_std_test                     37.023653
average_reward_test                  0.793522
round_time_test        0 days 00:00:08.696044
round_time_total       0 days 00:05:57.733443
loss_total                        6027.810102
loss_critic                        7625.97197
loss_actor                         -364.83794
memory_size                        349100.435 

=== epoch 7/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:15,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                  142
episode_length                      70.140845
returns                             75.557026
return_std                          49.540221
average_reward                       1.076768
round_time             0 days 00:05:58.808940
episodes_test                            73.0
episode_length_test                136.520548
returns_test                        161.50853
return_std_test                     62.729073
average_reward_test                   1.18327
round_time_test        0 days 00:00:08.546717
round_time_total       0 days 00:05:58.810094
loss_total                         5946.97528
loss_critic                        7524.51372
loss_actor                        -363.178992
memory_size                       350610.2085 

=== epoch 7/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  145
episode_length                      68.910345
returns                             73.328569
return_std                          47.136447
average_reward                        1.06386
round_time             0 days 00:05:57.595310
episodes_test                            69.0
episode_length_test                143.057971
returns_test                       116.654744
return_std_test                     48.800367
average_reward_test                  0.815553
round_time_test        0 days 00:00:08.707603
round_time_total       0 days 00:05:57.596703
loss_total                        5911.874062
loss_critic                       7480.586251
loss_actor                        -362.975241
memory_size                        352079.277 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 8/10 ===== round 1/50 ======================================
  1%|          | 12/2000 [00:02<05:52,  5.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:43<00:00,  5.82it/s]
episodes                                   28
episode_length                      70.571429
returns                             83.088301
return_std                          60.883509
average_reward                       1.178351
round_time             0 days 00:05:43.862267
episodes_test                           108.0
episode_length_test                 92.074074
returns_test                       115.910135
return_std_test                     53.869278
average_reward_test                  1.258523
round_time_test        0 days 00:00:08.821397
round_time_total       0 days 00:05:43.863416
loss_total                        5909.731966
loss_critic                       7478.359018
loss_actor                        -364.776779
memory_size                        353594.848 

=== epoch 8/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.83it/s]
episodes                                   62
episode_length                      63.741935
returns                             76.291567
return_std                          57.605255
average_reward                       1.196581
round_time             0 days 00:05:43.346640
episodes_test                           120.0
episode_length_test                    82.825
returns_test                       106.279658
return_std_test                     48.802194
average_reward_test                  1.284111
round_time_test        0 days 00:00:08.628535
round_time_total       0 days 00:05:43.348044
loss_total                         5985.65017
loss_critic                       7573.187041
loss_actor                        -364.497831
memory_size                       355076.5315 

=== epoch 8/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                   92
episode_length                      64.565217
returns                             75.410397
return_std                          57.201585
average_reward                       1.168242
round_time             0 days 00:05:45.337667
episodes_test                           144.0
episode_length_test                   69.3125
returns_test                        95.691594
return_std_test                     28.644322
average_reward_test                  1.380272
round_time_test        0 days 00:00:08.829587
round_time_total       0 days 00:05:45.339013
loss_total                        6107.939445
loss_critic                       7725.602297
loss_actor                         -362.71247
memory_size                        356585.238 

=== epoch 8/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:57,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:44<00:00,  5.80it/s]
episodes                                  119
episode_length                      66.495798
returns                             78.197986
return_std                          59.160368
average_reward                       1.176863
round_time             0 days 00:05:45.165008
episodes_test                           161.0
episode_length_test                 61.826087
returns_test                        87.601735
return_std_test                     28.662688
average_reward_test                  1.416762
round_time_test        0 days 00:00:08.736094
round_time_total       0 days 00:05:45.166333
loss_total                        5831.311492
loss_critic                       7379.602746
loss_actor                        -361.854031
memory_size                       358118.5405 

=== epoch 8/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.77it/s]
episodes                                  149
episode_length                      66.919463
returns                             78.550171
return_std                          56.843495
average_reward                       1.173269
round_time             0 days 00:05:47.336445
episodes_test                            40.0
episode_length_test                   246.375
returns_test                       248.148029
return_std_test                     81.355246
average_reward_test                  1.007567
round_time_test        0 days 00:00:08.764520
round_time_total       0 days 00:05:47.337916
loss_total                        5893.816264
loss_critic                        7458.07043
loss_actor                        -363.200899
memory_size                       359674.2755 

=== epoch 8/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  151
episode_length                      65.801325
returns                             76.053025
return_std                          55.853907
average_reward                       1.156095
round_time             0 days 00:05:47.637875
episodes_test                           103.0
episode_length_test                 96.815534
returns_test                       123.213804
return_std_test                     47.431751
average_reward_test                  1.272849
round_time_test        0 days 00:00:08.637752
round_time_total       0 days 00:05:47.638976
loss_total                         5920.15571
loss_critic                       7491.177699
loss_actor                        -363.932764
memory_size                       361195.3255 

=== epoch 8/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:05,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:46<00:00,  5.76it/s]
episodes                                  147
episode_length                      67.421769
returns                             77.372729
return_std                          56.769518
average_reward                       1.149394
round_time             0 days 00:05:47.516743
episodes_test                           100.0
episode_length_test                     99.72
returns_test                       125.617189
return_std_test                     44.623745
average_reward_test                  1.259876
round_time_test        0 days 00:00:08.709966
round_time_total       0 days 00:05:47.517858
loss_total                        5897.138588
loss_critic                       7462.555157
loss_actor                        -364.528197
memory_size                        362739.447 

=== epoch 8/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:14,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  146
episode_length                      68.171233
returns                             78.286032
return_std                            56.7694
average_reward                       1.149517
round_time             0 days 00:05:49.840586
episodes_test                           170.0
episode_length_test                 58.617647
returns_test                        84.044152
return_std_test                     28.384005
average_reward_test                  1.433637
round_time_test        0 days 00:00:08.904745
round_time_total       0 days 00:05:49.841695
loss_total                        5971.796415
loss_critic                       7556.020624
loss_actor                        -365.100902
memory_size                       364281.6615 

=== epoch 8/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:36,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                  156
episode_length                      63.769231
returns                             72.530111
return_std                          53.818194
average_reward                       1.137949
round_time             0 days 00:05:47.762175
episodes_test                            62.0
episode_length_test                160.693548
returns_test                       186.864412
return_std_test                     89.215986
average_reward_test                  1.163326
round_time_test        0 days 00:00:08.583004
round_time_total       0 days 00:05:47.763283
loss_total                        5883.579855
loss_critic                       7445.645016
loss_actor                        -364.681313
memory_size                       365703.1315 

=== epoch 8/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  161
episode_length                      61.161491
returns                             69.649869
return_std                          53.484048
average_reward                       1.135896
round_time             0 days 00:05:49.758336
episodes_test                            37.0
episode_length_test                265.162162
returns_test                        258.53373
return_std_test                    124.241423
average_reward_test                  0.975086
round_time_test        0 days 00:00:08.627656
round_time_total       0 days 00:05:49.759472
loss_total                        5837.022055
loss_critic                        7387.54809
loss_actor                        -365.082567
memory_size                       367201.6665 

=== epoch 8/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                  161
episode_length                      60.975155
returns                             69.847195
return_std                          51.418673
average_reward                       1.143126
round_time             0 days 00:05:48.677146
episodes_test                           125.0
episode_length_test                    79.648
returns_test                        105.89451
return_std_test                     33.604566
average_reward_test                  1.330051
round_time_test        0 days 00:00:08.601260
round_time_total       0 days 00:05:48.678269
loss_total                        5937.931814
loss_critic                       7514.194168
loss_actor                        -367.118112
memory_size                       368643.0125 

=== epoch 8/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                  158
episode_length                      61.113924
returns                             70.349384
return_std                          50.222206
average_reward                       1.146748
round_time             0 days 00:05:49.712081
episodes_test                            53.0
episode_length_test                186.811321
returns_test                       213.048221
return_std_test                    132.843521
average_reward_test                  1.140883
round_time_test        0 days 00:00:08.835081
round_time_total       0 days 00:05:49.713190
loss_total                        6053.633463
loss_critic                       7659.612554
loss_actor                         -370.28347
memory_size                        370129.671 

=== epoch 8/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:46,  5.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                  160
episode_length                       62.03125
returns                              71.98212
return_std                          52.159202
average_reward                       1.159686
round_time             0 days 00:05:49.001984
episodes_test                           139.0
episode_length_test                 71.741007
returns_test                          97.5717
return_std_test                     30.928862
average_reward_test                   1.35987
round_time_test        0 days 00:00:08.855669
round_time_total       0 days 00:05:49.003212
loss_total                        6070.164245
loss_critic                       7679.531166
loss_actor                        -367.303959
memory_size                        371679.365 

=== epoch 8/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  152
episode_length                      65.282895
returns                             75.164609
return_std                          54.330025
average_reward                       1.153594
round_time             0 days 00:05:52.075054
episodes_test                           106.0
episode_length_test                 94.018868
returns_test                       119.867451
return_std_test                      41.81284
average_reward_test                   1.27525
round_time_test        0 days 00:00:08.758709
round_time_total       0 days 00:05:52.076141
loss_total                        5967.541734
loss_critic                       7551.082383
loss_actor                        -366.621377
memory_size                       373178.4405 

=== epoch 8/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  146
episode_length                       67.30137
returns                             77.853726
return_std                          53.636305
average_reward                       1.156445
round_time             0 days 00:05:50.012806
episodes_test                            75.0
episode_length_test                132.266667
returns_test                       157.099202
return_std_test                      91.03045
average_reward_test                  1.188015
round_time_test        0 days 00:00:08.731298
round_time_total       0 days 00:05:50.014100
loss_total                        5844.692208
loss_critic                       7397.250723
loss_actor                        -365.542332
memory_size                       374734.1935 

=== epoch 8/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  142
episode_length                      69.830986
returns                             78.764033
return_std                          57.536521
average_reward                       1.130549
round_time             0 days 00:05:50.054771
episodes_test                           121.0
episode_length_test                 82.504132
returns_test                       109.661518
return_std_test                     30.348497
average_reward_test                   1.32885
round_time_test        0 days 00:00:08.641692
round_time_total       0 days 00:05:50.056083
loss_total                        5857.576971
loss_critic                       7414.208091
loss_actor                        -368.947997
memory_size                       376241.3295 

=== epoch 8/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:10,  5.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  138
episode_length                      72.188406
returns                             80.013629
return_std                          58.418701
average_reward                       1.106784
round_time             0 days 00:05:53.865743
episodes_test                           122.0
episode_length_test                 81.311475
returns_test                        108.62176
return_std_test                     29.642606
average_reward_test                  1.335751
round_time_test        0 days 00:00:08.820209
round_time_total       0 days 00:05:53.866891
loss_total                        6032.455049
loss_critic                       7632.966003
loss_actor                        -369.589303
memory_size                        377855.693 

=== epoch 8/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                  143
episode_length                      69.608392
returns                             78.268755
return_std                          54.712001
average_reward                       1.126925
round_time             0 days 00:05:50.387984
episodes_test                           173.0
episode_length_test                 57.763006
returns_test                        84.732771
return_std_test                     20.414368
average_reward_test                  1.466511
round_time_test        0 days 00:00:08.715757
round_time_total       0 days 00:05:50.389100
loss_total                        5934.284956
loss_critic                       7511.039636
loss_actor                        -372.734336
memory_size                       379402.9175 

=== epoch 8/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  142
episode_length                      69.823944
returns                             80.618011
return_std                          55.002696
average_reward                       1.155089
round_time             0 days 00:05:53.611673
episodes_test                           187.0
episode_length_test                 53.475936
returns_test                        79.402029
return_std_test                     23.850338
average_reward_test                  1.484818
round_time_test        0 days 00:00:08.692564
round_time_total       0 days 00:05:53.612776
loss_total                         6028.80121
loss_critic                       7629.486891
loss_actor                        -373.942009
memory_size                        380888.817 

=== epoch 8/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                  141
episode_length                      70.723404
returns                             81.537315
return_std                          56.766134
average_reward                       1.153443
round_time             0 days 00:05:52.077903
episodes_test                            98.0
episode_length_test                 101.55102
returns_test                       128.944452
return_std_test                     34.816851
average_reward_test                  1.268328
round_time_test        0 days 00:00:08.702366
round_time_total       0 days 00:05:52.079020
loss_total                        5990.722608
loss_critic                       7581.902107
loss_actor                        -373.995911
memory_size                        382437.954 

=== epoch 8/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                  137
episode_length                      72.029197
returns                             83.863371
return_std                          57.524825
average_reward                       1.162427
round_time             0 days 00:05:53.288865
episodes_test                            27.0
episode_length_test                359.259259
returns_test                       346.966038
return_std_test                     135.55599
average_reward_test                  0.964007
round_time_test        0 days 00:00:08.711593
round_time_total       0 days 00:05:53.289971
loss_total                        6033.027938
loss_critic                       7634.856831
loss_actor                         -374.28815
memory_size                       384014.7195 

=== epoch 8/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                  130
episode_length                      75.523077
returns                             85.892876
return_std                          57.804114
average_reward                       1.140024
round_time             0 days 00:05:52.491992
episodes_test                            52.0
episode_length_test                191.076923
returns_test                       218.095736
return_std_test                        76.636
average_reward_test                  1.142048
round_time_test        0 days 00:00:08.494847
round_time_total       0 days 00:05:52.493109
loss_total                          5978.2836
loss_critic                        7566.34365
loss_actor                        -373.957167
memory_size                        385675.402 

=== epoch 8/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                  119
episode_length                      83.571429
returns                             91.902391
return_std                          61.189772
average_reward                       1.102693
round_time             0 days 00:05:55.575276
episodes_test                            60.0
episode_length_test                166.633333
returns_test                       139.285325
return_std_test                      53.76575
average_reward_test                  0.835898
round_time_test        0 days 00:00:08.595818
round_time_total       0 days 00:05:55.576619
loss_total                        5980.774135
loss_critic                       7569.324438
loss_actor                        -373.427592
memory_size                       387334.2835 

=== epoch 8/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<07:28,  4.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                  107
episode_length                      90.074766
returns                             93.627594
return_std                          61.958431
average_reward                       1.041994
round_time             0 days 00:05:53.838339
episodes_test                            54.0
episode_length_test                184.240741
returns_test                       157.030437
return_std_test                     57.980305
average_reward_test                  0.853251
round_time_test        0 days 00:00:08.715237
round_time_total       0 days 00:05:53.839584
loss_total                         5993.19913
loss_critic                       7585.359776
loss_actor                        -375.444029
memory_size                       389007.0665 

=== epoch 8/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   95
episode_length                     102.873684
returns                             103.79965
return_std                          72.976376
average_reward                       1.009759
round_time             0 days 00:05:52.070756
episodes_test                            35.0
episode_length_test                272.771429
returns_test                       245.998729
return_std_test                     95.619286
average_reward_test                  0.905624
round_time_test        0 days 00:00:08.600162
round_time_total       0 days 00:05:52.071886
loss_total                        6111.690726
loss_critic                       7733.889091
loss_actor                         -377.10326
memory_size                       390725.3965 

=== epoch 8/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   88
episode_length                     110.613636
returns                            111.223804
return_std                          81.517708
average_reward                       1.003842
round_time             0 days 00:05:55.034796
episodes_test                            38.0
episode_length_test                259.263158
returns_test                       275.929998
return_std_test                    153.015673
average_reward_test                  1.065342
round_time_test        0 days 00:00:08.793787
round_time_total       0 days 00:05:55.036185
loss_total                         6086.13517
loss_critic                       7702.215603
loss_actor                        -378.187123
memory_size                        392457.892 

=== epoch 8/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:04,  5.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   91
episode_length                     108.461538
returns                            110.242634
return_std                          87.758036
average_reward                       1.018805
round_time             0 days 00:05:54.378984
episodes_test                            64.0
episode_length_test                 155.84375
returns_test                       182.173291
return_std_test                     71.053498
average_reward_test                  1.169079
round_time_test        0 days 00:00:08.675632
round_time_total       0 days 00:05:54.380272
loss_total                        5990.980828
loss_critic                       7583.283314
loss_actor                        -378.229606
memory_size                        394201.721 

=== epoch 8/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   89
episode_length                      109.94382
returns                            110.326285
return_std                          93.698847
average_reward                        1.00409
round_time             0 days 00:05:57.053791
episodes_test                            76.0
episode_length_test                130.342105
returns_test                       157.041712
return_std_test                     46.419579
average_reward_test                  1.203645
round_time_test        0 days 00:00:08.619919
round_time_total       0 days 00:05:57.054925
loss_total                         6013.16919
loss_critic                       7611.172204
loss_actor                         -378.84335
memory_size                       395848.9545 

=== epoch 8/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:18,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   92
episode_length                     106.967391
returns                            111.729392
return_std                         103.742418
average_reward                       1.046561
round_time             0 days 00:05:53.413889
episodes_test                            55.0
episode_length_test                177.581818
returns_test                       204.421912
return_std_test                      64.81226
average_reward_test                  1.149338
round_time_test        0 days 00:00:08.577634
round_time_total       0 days 00:05:53.415361
loss_total                        5967.128122
loss_critic                       7554.259859
loss_actor                         -381.39934
memory_size                        397518.786 

=== epoch 8/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:01,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   99
episode_length                      99.484848
returns                            104.574609
return_std                         102.050718
average_reward                       1.052758
round_time             0 days 00:05:54.716250
episodes_test                            82.0
episode_length_test                120.317073
returns_test                       146.446346
return_std_test                     48.197865
average_reward_test                  1.216882
round_time_test        0 days 00:00:08.755491
round_time_total       0 days 00:05:54.717360
loss_total                        6090.653588
loss_critic                       7709.295603
loss_actor                         -383.91498
memory_size                        399241.599 

=== epoch 8/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:26,  5.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  101
episode_length                       97.09901
returns                            102.979367
return_std                          91.064184
average_reward                       1.058476
round_time             0 days 00:05:56.041811
episodes_test                            45.0
episode_length_test                221.533333
returns_test                       193.923042
return_std_test                      66.55894
average_reward_test                  0.876474
round_time_test        0 days 00:00:08.708378
round_time_total       0 days 00:05:56.042922
loss_total                        6097.710532
loss_critic                       7717.788551
loss_actor                        -382.602063
memory_size                        400907.789 

=== epoch 8/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<05:56,  5.58it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                  105
episode_length                      94.457143
returns                             101.75895
return_std                          86.151883
average_reward                       1.078392
round_time             0 days 00:05:55.525025
episodes_test                            93.0
episode_length_test                106.569892
returns_test                       132.719061
return_std_test                     41.774567
average_reward_test                  1.243669
round_time_test        0 days 00:00:08.560724
round_time_total       0 days 00:05:55.526332
loss_total                        6201.679228
loss_critic                       7847.334837
loss_actor                        -380.943787
memory_size                        402552.439 

=== epoch 8/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:13,  5.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                  103
episode_length                      96.601942
returns                            103.251123
return_std                          83.402938
average_reward                       1.068594
round_time             0 days 00:05:56.696510
episodes_test                            80.0
episode_length_test                  123.6125
returns_test                       151.715882
return_std_test                     45.202909
average_reward_test                  1.226996
round_time_test        0 days 00:00:08.724902
round_time_total       0 days 00:05:56.697850
loss_total                        6105.226227
loss_critic                       7727.710667
loss_actor                        -384.712065
memory_size                       404200.6225 

=== epoch 8/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:25,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                  102
episode_length                      96.764706
returns                             98.774274
return_std                          75.735485
average_reward                       1.023009
round_time             0 days 00:05:55.047550
episodes_test                            24.0
episode_length_test                412.458333
returns_test                       390.543695
return_std_test                    195.760233
average_reward_test                  0.946691
round_time_test        0 days 00:00:08.638191
round_time_total       0 days 00:05:55.048644
loss_total                        6248.105041
loss_critic                       7906.285126
loss_actor                        -384.615862
memory_size                        405864.853 

=== epoch 8/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   99
episode_length                      98.777778
returns                            101.588279
return_std                          73.940768
average_reward                       1.027033
round_time             0 days 00:05:57.566375
episodes_test                            37.0
episode_length_test                267.486486
returns_test                       289.921006
return_std_test                    129.755323
average_reward_test                  1.083439
round_time_test        0 days 00:00:08.610780
round_time_total       0 days 00:05:57.567711
loss_total                        6039.664015
loss_critic                       7645.616901
loss_actor                        -384.148088
memory_size                        407577.882 

=== epoch 8/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                  100
episode_length                          99.04
returns                            101.976071
return_std                          79.374176
average_reward                       1.032097
round_time             0 days 00:05:56.088285
episodes_test                            26.0
episode_length_test                374.653846
returns_test                       348.448666
return_std_test                    111.548331
average_reward_test                    0.9311
round_time_test        0 days 00:00:08.791632
round_time_total       0 days 00:05:56.089399
loss_total                        6110.233078
loss_critic                       7734.418996
loss_actor                        -386.511154
memory_size                        409253.139 

=== epoch 8/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:55,  5.61it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   90
episode_length                          109.5
returns                            110.691931
return_std                          84.444859
average_reward                       1.009599
round_time             0 days 00:05:56.993892
episodes_test                           126.0
episode_length_test                 79.206349
returns_test                       105.699732
return_std_test                     26.776599
average_reward_test                  1.334328
round_time_test        0 days 00:00:08.686502
round_time_total       0 days 00:05:56.995183
loss_total                        6188.417376
loss_critic                       7832.499705
loss_actor                        -387.912508
memory_size                       410992.6915 

=== epoch 8/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   87
episode_length                     113.402299
returns                            118.521518
return_std                          86.167182
average_reward                       1.044592
round_time             0 days 00:05:57.121867
episodes_test                            69.0
episode_length_test                144.666667
returns_test                       171.840576
return_std_test                     45.971994
average_reward_test                  1.187205
round_time_test        0 days 00:00:08.621958
round_time_total       0 days 00:05:57.123041
loss_total                        6226.352008
loss_critic                       7880.711635
loss_actor                        -391.087023
memory_size                        412739.974 

=== epoch 8/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                   86
episode_length                     114.430233
returns                            123.343768
return_std                          79.758965
average_reward                        1.07845
round_time             0 days 00:05:58.142857
episodes_test                           108.0
episode_length_test                 92.435185
returns_test                        120.03705
return_std_test                     23.763818
average_reward_test                  1.298418
round_time_test        0 days 00:00:08.811407
round_time_total       0 days 00:05:58.143978
loss_total                        6253.587218
loss_critic                       7915.226569
loss_actor                        -392.970773
memory_size                         414447.15 

=== epoch 8/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   80
episode_length                        123.075
returns                            132.283963
return_std                           81.55625
average_reward                       1.076691
round_time             0 days 00:05:57.903637
episodes_test                           127.0
episode_length_test                 78.464567
returns_test                       105.730297
return_std_test                     22.044014
average_reward_test                  1.347609
round_time_test        0 days 00:00:08.637903
round_time_total       0 days 00:05:57.904753
loss_total                        6233.587305
loss_critic                       7890.081275
loss_actor                        -392.389148
memory_size                       416195.7115 

=== epoch 8/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   72
episode_length                     136.847222
returns                            144.035256
return_std                          86.584293
average_reward                       1.050671
round_time             0 days 00:05:54.681557
episodes_test                            52.0
episode_length_test                188.807692
returns_test                       214.679897
return_std_test                     71.506145
average_reward_test                  1.135649
round_time_test        0 days 00:00:08.716455
round_time_total       0 days 00:05:54.682668
loss_total                        6248.833487
loss_critic                        7909.22195
loss_actor                        -392.720913
memory_size                        417983.203 

=== epoch 8/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<07:02,  4.71it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   75
episode_length                     131.733333
returns                            135.307821
return_std                         102.498805
average_reward                       1.025413
round_time             0 days 00:05:57.664433
episodes_test                            44.0
episode_length_test                224.818182
returns_test                       253.104874
return_std_test                     31.598332
average_reward_test                  1.126001
round_time_test        0 days 00:00:08.825918
round_time_total       0 days 00:05:57.665539
loss_total                        6360.492053
loss_critic                       8049.120732
loss_actor                        -394.023165
memory_size                        419718.499 

=== epoch 8/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:58,  5.56it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                   67
episode_length                     148.731343
returns                            149.414482
return_std                         118.195498
average_reward                       1.005885
round_time             0 days 00:05:58.095115
episodes_test                            17.0
episode_length_test                561.294118
returns_test                       585.661469
return_std_test                     180.94865
average_reward_test                  1.042644
round_time_test        0 days 00:00:08.607720
round_time_total       0 days 00:05:58.096231
loss_total                        6278.004773
loss_critic                       7945.821756
loss_actor                        -393.263698
memory_size                        421554.153 

=== epoch 8/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:50,  4.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                   62
episode_length                     153.177419
returns                            151.257608
return_std                         119.251847
average_reward                       0.993401
round_time             0 days 00:05:58.851537
episodes_test                            46.0
episode_length_test                212.956522
returns_test                       238.899356
return_std_test                     75.780656
average_reward_test                  1.121986
round_time_test        0 days 00:00:08.547388
round_time_total       0 days 00:05:58.852687
loss_total                        6217.717308
loss_critic                       7870.436796
loss_actor                        -393.161201
memory_size                        423342.126 

=== epoch 8/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:13,  5.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   61
episode_length                     160.393443
returns                            156.240981
return_std                         124.519759
average_reward                       0.972449
round_time             0 days 00:05:57.359089
episodes_test                            89.0
episode_length_test                112.247191
returns_test                       140.154049
return_std_test                     23.257147
average_reward_test                  1.248344
round_time_test        0 days 00:00:08.664722
round_time_total       0 days 00:05:57.360313
loss_total                        6158.157612
loss_critic                       7797.283321
loss_actor                        -398.345774
memory_size                        425149.749 

=== epoch 8/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:01,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                   64
episode_length                       154.8125
returns                            152.498147
return_std                         122.727057
average_reward                       0.987534
round_time             0 days 00:05:58.219251
episodes_test                            25.0
episode_length_test                    385.44
returns_test                       356.885644
return_std_test                    108.796824
average_reward_test                  0.926156
round_time_test        0 days 00:00:08.521885
round_time_total       0 days 00:05:58.220576
loss_total                        6399.451982
loss_critic                        8099.54697
loss_actor                        -400.928539
memory_size                       426924.4955 

=== epoch 8/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   57
episode_length                      164.77193
returns                            168.569162
return_std                         114.919326
average_reward                       1.022834
round_time             0 days 00:05:57.712127
episodes_test                            76.0
episode_length_test                131.302632
returns_test                       158.402002
return_std_test                     37.875619
average_reward_test                  1.206552
round_time_test        0 days 00:00:08.738711
round_time_total       0 days 00:05:57.713328
loss_total                        6333.767443
loss_critic                       8017.273573
loss_actor                        -400.257621
memory_size                         428735.75 

=== epoch 8/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                   57
episode_length                     168.157895
returns                            172.655665
return_std                         110.695102
average_reward                        1.02149
round_time             0 days 00:05:58.951601
episodes_test                            93.0
episode_length_test                106.688172
returns_test                       132.822008
return_std_test                     36.468249
average_reward_test                  1.244773
round_time_test        0 days 00:00:08.658974
round_time_total       0 days 00:05:58.953164
loss_total                        6360.271386
loss_critic                       8050.909906
loss_actor                        -402.283242
memory_size                       430558.3645 

=== epoch 8/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.58it/s]
episodes                                   52
episode_length                     189.519231
returns                            194.291311
return_std                         127.524466
average_reward                       1.026845
round_time             0 days 00:05:59.014324
episodes_test                            78.0
episode_length_test                126.461538
returns_test                       153.815275
return_std_test                     27.135056
average_reward_test                   1.21596
round_time_test        0 days 00:00:08.778751
round_time_total       0 days 00:05:59.015786
loss_total                        6377.053971
loss_critic                       8072.048437
loss_actor                        -402.924407
memory_size                        432429.207 

=== epoch 8/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:36,  5.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                   54
episode_length                     182.888889
returns                            194.891329
return_std                         128.959689
average_reward                       1.066293
round_time             0 days 00:05:58.468416
episodes_test                            84.0
episode_length_test                118.416667
returns_test                       146.063429
return_std_test                     22.596595
average_reward_test                  1.233439
round_time_test        0 days 00:00:08.628566
round_time_total       0 days 00:05:58.469574
loss_total                        6512.769323
loss_critic                       8242.706992
loss_actor                        -406.981916
memory_size                        434262.352 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 9/10 ===== round 1/50 ======================================
  1%|          | 12/2000 [00:02<05:59,  5.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.85it/s]
episodes                                    9
episode_length                     215.333333
returns                            248.738191
return_std                           98.76099
average_reward                        1.15374
round_time             0 days 00:05:42.053938
episodes_test                           111.0
episode_length_test                  89.27027
returns_test                        117.45544
return_std_test                     10.218138
average_reward_test                  1.315205
round_time_test        0 days 00:00:08.769033
round_time_total       0 days 00:05:42.055060
loss_total                        6513.942677
loss_critic                       8244.474667
loss_actor                        -408.185884
memory_size                       436039.7795 

=== epoch 9/10 ===== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:44,  5.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   20
episode_length                          199.7
returns                            220.872266
return_std                         114.479814
average_reward                       1.105855
round_time             0 days 00:05:46.137688
episodes_test                            91.0
episode_length_test                108.956044
returns_test                       136.032493
return_std_test                     29.046209
average_reward_test                  1.248566
round_time_test        0 days 00:00:08.495042
round_time_total       0 days 00:05:46.138814
loss_total                        6544.393883
loss_critic                       8282.862314
loss_actor                        -409.480447
memory_size                        437874.611 

=== epoch 9/10 ===== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:10,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   34
episode_length                          176.0
returns                            199.541884
return_std                         110.098018
average_reward                       1.133451
round_time             0 days 00:05:45.818927
episodes_test                            37.0
episode_length_test                268.621622
returns_test                        296.31457
return_std_test                     68.745979
average_reward_test                  1.103368
round_time_test        0 days 00:00:08.668000
round_time_total       0 days 00:05:45.820030
loss_total                        6671.130874
loss_critic                       8441.537754
loss_actor                         -410.49722
memory_size                       439692.0495 

=== epoch 9/10 ===== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                   48
episode_length                       164.0625
returns                            186.867962
return_std                         108.903816
average_reward                       1.137494
round_time             0 days 00:05:48.901644
episodes_test                            79.0
episode_length_test                126.544304
returns_test                       154.017267
return_std_test                     26.132879
average_reward_test                   1.21704
round_time_test        0 days 00:00:08.695059
round_time_total       0 days 00:05:48.902881
loss_total                        6518.211349
loss_critic                       8250.349622
loss_actor                        -410.342347
memory_size                        441438.319 

=== epoch 9/10 ===== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:16,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.78it/s]
episodes                                   61
episode_length                     162.622951
returns                            185.077221
return_std                         100.567801
average_reward                       1.137973
round_time             0 days 00:05:46.406428
episodes_test                            46.0
episode_length_test                216.304348
returns_test                       243.862166
return_std_test                     44.385267
average_reward_test                  1.127599
round_time_test        0 days 00:00:08.606169
round_time_total       0 days 00:05:46.407976
loss_total                         6668.20107
loss_critic                       8437.866369
loss_actor                        -410.460703
memory_size                        443227.706 

=== epoch 9/10 ===== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:29,  5.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                   64
episode_length                     154.328125
returns                            174.669274
return_std                          95.534688
average_reward                       1.132753
round_time             0 days 00:05:48.660830
episodes_test                            20.0
episode_length_test                    479.15
returns_test                       506.982676
return_std_test                    124.713145
average_reward_test                  1.057794
round_time_test        0 days 00:00:08.563603
round_time_total       0 days 00:05:48.662092
loss_total                        6540.382414
loss_critic                       8279.604829
loss_actor                        -416.507794
memory_size                       445011.6105 

=== epoch 9/10 ===== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:47,  4.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.74it/s]
episodes                                   67
episode_length                     143.402985
returns                            165.059127
return_std                          85.823983
average_reward                       1.149391
round_time             0 days 00:05:48.905351
episodes_test                            76.0
episode_length_test                131.407895
returns_test                        159.44141
return_std_test                     27.857864
average_reward_test                   1.21311
round_time_test        0 days 00:00:08.736483
round_time_total       0 days 00:05:48.906600
loss_total                        6608.998769
loss_critic                       8364.766579
loss_actor                        -414.073027
memory_size                       446790.5105 

=== epoch 9/10 ===== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.64it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.75it/s]
episodes                                   63
episode_length                     156.190476
returns                            177.123266
return_std                          92.772468
average_reward                       1.136153
round_time             0 days 00:05:48.717227
episodes_test                            34.0
episode_length_test                289.411765
returns_test                       317.008545
return_std_test                     74.496093
average_reward_test                  1.095753
round_time_test        0 days 00:00:08.655916
round_time_total       0 days 00:05:48.718486
loss_total                        6515.686429
loss_critic                       8248.432326
loss_actor                        -415.297694
memory_size                        448599.806 

=== epoch 9/10 ===== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   58
episode_length                     168.206897
returns                            190.550881
return_std                         103.219247
average_reward                        1.13064
round_time             0 days 00:05:51.850373
episodes_test                            65.0
episode_length_test                153.046154
returns_test                       179.345516
return_std_test                     48.299545
average_reward_test                  1.172135
round_time_test        0 days 00:00:08.581537
round_time_total       0 days 00:05:51.851483
loss_total                        6534.456382
loss_critic                       8272.043886
loss_actor                        -415.894214
memory_size                       450411.5905 

=== epoch 9/10 ===== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:44,  5.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                   51
episode_length                     189.392157
returns                            212.042594
return_std                         130.367095
average_reward                       1.118024
round_time             0 days 00:05:50.449122
episodes_test                            25.0
episode_length_test                     388.8
returns_test                       417.118638
return_std_test                     59.704633
average_reward_test                   1.07301
round_time_test        0 days 00:00:08.924261
round_time_total       0 days 00:05:50.450708
loss_total                        6656.063457
loss_critic                       8425.334594
loss_actor                        -421.021712
memory_size                       452290.5335 

=== epoch 9/10 ===== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:38,  5.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                   52
episode_length                     188.057692
returns                             211.58592
return_std                         132.200817
average_reward                        1.12695
round_time             0 days 00:05:50.650289
episodes_test                            97.0
episode_length_test                102.453608
returns_test                       129.574624
return_std_test                      27.39706
average_reward_test                   1.26479
round_time_test        0 days 00:00:08.627418
round_time_total       0 days 00:05:50.651439
loss_total                        6691.395137
loss_critic                        8470.65622
loss_actor                        -425.649808
memory_size                        454118.451 

=== epoch 9/10 ===== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   52
episode_length                     183.365385
returns                            208.457327
return_std                         129.726209
average_reward                       1.128983
round_time             0 days 00:05:51.137764
episodes_test                            68.0
episode_length_test                146.441176
returns_test                       173.141512
return_std_test                     44.205459
average_reward_test                  1.182543
round_time_test        0 days 00:00:08.723344
round_time_total       0 days 00:05:51.138876
loss_total                        6753.456998
loss_critic                       8548.478605
loss_actor                        -426.630001
memory_size                        455891.942 

=== epoch 9/10 ===== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:03,  5.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                   48
episode_length                     199.041667
returns                            225.652555
return_std                         159.509373
average_reward                       1.130458
round_time             0 days 00:05:50.236713
episodes_test                            93.0
episode_length_test                106.817204
returns_test                       134.149821
return_std_test                     26.600946
average_reward_test                  1.255637
round_time_test        0 days 00:00:08.745615
round_time_total       0 days 00:05:50.237852
loss_total                        6950.662854
loss_critic                       8795.352309
loss_actor                        -428.095609
memory_size                       457730.4505 

=== epoch 9/10 ===== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:48<00:00,  5.73it/s]
episodes                                   47
episode_length                     202.808511
returns                              230.1052
return_std                         155.239749
average_reward                       1.132583
round_time             0 days 00:05:49.425261
episodes_test                            71.0
episode_length_test                138.915493
returns_test                       166.439606
return_std_test                     31.542508
average_reward_test                  1.197934
round_time_test        0 days 00:00:08.618531
round_time_total       0 days 00:05:49.426381
loss_total                        6991.021413
loss_critic                       8846.194829
loss_actor                        -429.672868
memory_size                       459587.2595 

=== epoch 9/10 ===== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:25,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.70it/s]
episodes                                   54
episode_length                      181.87037
returns                            209.867129
return_std                         137.193698
average_reward                       1.154045
round_time             0 days 00:05:51.672271
episodes_test                            55.0
episode_length_test                181.581818
returns_test                       209.155806
return_std_test                     36.359846
average_reward_test                  1.151742
round_time_test        0 days 00:00:08.597876
round_time_total       0 days 00:05:51.673505
loss_total                        6955.521811
loss_critic                       8802.940329
loss_actor                        -434.152818
memory_size                       461438.9175 

=== epoch 9/10 ===== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:14,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   51
episode_length                     191.352941
returns                            220.376181
return_std                         133.582708
average_reward                        1.15317
round_time             0 days 00:05:51.138589
episodes_test                            94.0
episode_length_test                105.638298
returns_test                       132.971881
return_std_test                     25.547408
average_reward_test                  1.258901
round_time_test        0 days 00:00:08.642394
round_time_total       0 days 00:05:51.139902
loss_total                        6915.310969
loss_critic                       8752.899473
loss_actor                         -435.04365
memory_size                        463223.077 

=== epoch 9/10 ===== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:51,  5.67it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   49
episode_length                     190.714286
returns                            221.659282
return_std                         107.231382
average_reward                       1.157932
round_time             0 days 00:05:51.465066
episodes_test                            86.0
episode_length_test                116.069767
returns_test                       143.573629
return_std_test                     26.289522
average_reward_test                   1.23691
round_time_test        0 days 00:00:08.722157
round_time_total       0 days 00:05:51.466179
loss_total                        6900.443546
loss_critic                       8734.037229
loss_actor                        -433.931809
memory_size                        465035.312 

=== epoch 9/10 ===== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                   58
episode_length                     170.844828
returns                            199.314682
return_std                           97.75977
average_reward                       1.167531
round_time             0 days 00:05:50.606820
episodes_test                            88.0
episode_length_test                112.590909
returns_test                        141.11504
return_std_test                     15.845217
average_reward_test                  1.253307
round_time_test        0 days 00:00:08.483256
round_time_total       0 days 00:05:50.607917
loss_total                          6708.3776
loss_critic                       8493.129675
loss_actor                        -430.631276
memory_size                        466803.772 

=== epoch 9/10 ===== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:28,  5.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   60
episode_length                     162.183333
returns                            189.538155
return_std                          95.333272
average_reward                       1.168961
round_time             0 days 00:05:54.341474
episodes_test                            70.0
episode_length_test                     141.7
returns_test                       169.566446
return_std_test                     31.323536
average_reward_test                  1.196419
round_time_test        0 days 00:00:08.856910
round_time_total       0 days 00:05:54.342717
loss_total                        6785.969587
loss_critic                       8590.101134
loss_actor                        -430.557212
memory_size                       468595.1795 

=== epoch 9/10 ===== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:11,  5.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                   60
episode_length                     159.983333
returns                            186.574102
return_std                         101.368507
average_reward                       1.163414
round_time             0 days 00:05:52.543504
episodes_test                            57.0
episode_length_test                172.807018
returns_test                       201.077287
return_std_test                     20.549556
average_reward_test                  1.163944
round_time_test        0 days 00:00:08.617253
round_time_total       0 days 00:05:52.544752
loss_total                        6914.368344
loss_critic                       8752.088728
loss_actor                        -436.513795
memory_size                        470381.801 

=== epoch 9/10 ===== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:46,  5.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                   63
episode_length                     157.650794
returns                            183.394116
return_std                         109.738312
average_reward                       1.164628
round_time             0 days 00:05:52.795875
episodes_test                            78.0
episode_length_test                126.974359
returns_test                       154.296434
return_std_test                     32.150259
average_reward_test                  1.215034
round_time_test        0 days 00:00:08.610659
round_time_total       0 days 00:05:52.797105
loss_total                        6975.290526
loss_critic                       8828.873751
loss_actor                         -439.04302
memory_size                       472192.9185 

=== epoch 9/10 ===== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   65
episode_length                     150.446154
returns                            175.767098
return_std                         102.613382
average_reward                       1.166156
round_time             0 days 00:05:54.522784
episodes_test                            75.0
episode_length_test                133.106667
returns_test                       161.031267
return_std_test                     22.822956
average_reward_test                  1.209692
round_time_test        0 days 00:00:08.510834
round_time_total       0 days 00:05:54.524279
loss_total                        6799.761742
loss_critic                       8610.077479
loss_actor                        -441.501791
memory_size                       473949.2925 

=== epoch 9/10 ===== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:01,  4.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   61
episode_length                     157.508197
returns                            182.340832
return_std                          113.57563
average_reward                       1.155662
round_time             0 days 00:05:52.291853
episodes_test                            47.0
episode_length_test                210.319149
returns_test                       238.270087
return_std_test                     44.338018
average_reward_test                  1.133215
round_time_test        0 days 00:00:08.583607
round_time_total       0 days 00:05:52.292988
loss_total                        6962.704681
loss_critic                       8813.062536
loss_actor                         -438.72738
memory_size                       475767.5565 

=== epoch 9/10 ===== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:23,  5.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                   63
episode_length                      157.68254
returns                            183.330544
return_std                         123.134507
average_reward                        1.16323
round_time             0 days 00:05:52.720951
episodes_test                            69.0
episode_length_test                144.130435
returns_test                       171.141169
return_std_test                     40.338244
average_reward_test                  1.184303
round_time_test        0 days 00:00:08.573598
round_time_total       0 days 00:05:52.722083
loss_total                        6959.496298
loss_critic                       8810.337837
loss_actor                        -443.870468
memory_size                       477602.1655 

=== epoch 9/10 ===== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:01,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   63
episode_length                     157.587302
returns                            183.353492
return_std                         119.007348
average_reward                       1.163946
round_time             0 days 00:05:53.188004
episodes_test                            65.0
episode_length_test                153.030769
returns_test                       181.600939
return_std_test                     18.287073
average_reward_test                  1.187175
round_time_test        0 days 00:00:08.701858
round_time_total       0 days 00:05:53.189283
loss_total                        7111.364981
loss_critic                       9001.216596
loss_actor                        -448.042153
memory_size                        479375.339 

=== epoch 9/10 ===== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   59
episode_length                     165.915254
returns                            192.031508
return_std                         125.663371
average_reward                       1.158107
round_time             0 days 00:05:52.218682
episodes_test                            31.0
episode_length_test                317.612903
returns_test                       346.268647
return_std_test                     56.453495
average_reward_test                  1.090656
round_time_test        0 days 00:00:08.666454
round_time_total       0 days 00:05:52.219790
loss_total                        7193.863893
loss_critic                       9105.091964
loss_actor                        -451.048976
memory_size                        481196.502 

=== epoch 9/10 ===== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:55,  4.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   60
episode_length                          162.3
returns                            188.772144
return_std                         121.236331
average_reward                       1.160369
round_time             0 days 00:05:54.349231
episodes_test                            67.0
episode_length_test                148.910448
returns_test                        176.67346
return_std_test                     29.299374
average_reward_test                  1.186555
round_time_test        0 days 00:00:08.632255
round_time_total       0 days 00:05:54.350329
loss_total                        7198.622179
loss_critic                       9111.821407
loss_actor                        -454.175325
memory_size                       482996.1285 

=== epoch 9/10 ===== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   63
episode_length                     151.269841
returns                              176.4842
return_std                         100.267606
average_reward                       1.162489
round_time             0 days 00:05:53.829971
episodes_test                            73.0
episode_length_test                135.383562
returns_test                        163.05476
return_std_test                     29.281685
average_reward_test                  1.204337
round_time_test        0 days 00:00:08.799032
round_time_total       0 days 00:05:53.831067
loss_total                        7062.365003
loss_critic                        8941.04887
loss_actor                        -452.371057
memory_size                       484737.7245 

=== epoch 9/10 ===== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:19,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   64
episode_length                     153.890625
returns                            179.524334
return_std                         104.345682
average_reward                       1.167107
round_time             0 days 00:05:55.037793
episodes_test                            81.0
episode_length_test                 122.08642
returns_test                       149.753218
return_std_test                     38.538483
average_reward_test                  1.227023
round_time_test        0 days 00:00:08.576178
round_time_total       0 days 00:05:55.039325
loss_total                        7227.453291
loss_critic                       9147.913056
loss_actor                        -454.386423
memory_size                       486545.0285 

=== epoch 9/10 ===== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:14,  5.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   58
episode_length                          164.0
returns                            190.402819
return_std                         116.538447
average_reward                       1.158483
round_time             0 days 00:05:54.910892
episodes_test                            80.0
episode_length_test                  124.4375
returns_test                       152.317307
return_std_test                     25.586696
average_reward_test                   1.22424
round_time_test        0 days 00:00:08.796084
round_time_total       0 days 00:05:54.912020
loss_total                        7056.757118
loss_critic                       8933.803893
loss_actor                        -451.430621
memory_size                        488355.823 

=== epoch 9/10 ===== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   55
episode_length                     180.109091
returns                             206.74397
return_std                         160.262415
average_reward                       1.148802
round_time             0 days 00:05:55.757707
episodes_test                            40.0
episode_length_test                    249.45
returns_test                       277.799518
return_std_test                     37.155104
average_reward_test                  1.114032
round_time_test        0 days 00:00:08.660593
round_time_total       0 days 00:05:55.759049
loss_total                        7008.791465
loss_critic                       8873.849943
loss_actor                        -451.443044
memory_size                       490252.2415 

=== epoch 9/10 ===== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   48
episode_length                     206.166667
returns                            233.465011
return_std                         179.699971
average_reward                       1.134425
round_time             0 days 00:05:54.457389
episodes_test                            16.0
episode_length_test                  623.5625
returns_test                       651.594844
return_std_test                    213.740324
average_reward_test                  1.045339
round_time_test        0 days 00:00:08.748448
round_time_total       0 days 00:05:54.458638
loss_total                        7116.926182
loss_critic                       9009.452321
loss_actor                        -453.179004
memory_size                       492124.4915 

=== epoch 9/10 ===== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:10,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   43
episode_length                     225.674419
returns                            254.285276
return_std                         188.182199
average_reward                       1.129962
round_time             0 days 00:05:56.028702
episodes_test                            60.0
episode_length_test                    165.15
returns_test                       192.729923
return_std_test                     29.202972
average_reward_test                  1.167656
round_time_test        0 days 00:00:08.631409
round_time_total       0 days 00:05:56.030198
loss_total                        7018.593221
loss_critic                       8886.461022
loss_actor                        -452.878598
memory_size                         493986.67 

=== epoch 9/10 ===== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:57,  5.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   40
episode_length                         248.45
returns                            277.332487
return_std                         188.901078
average_reward                       1.117815
round_time             0 days 00:05:53.057843
episodes_test                            66.0
episode_length_test                 150.80303
returns_test                       178.259502
return_std_test                     33.834308
average_reward_test                  1.182375
round_time_test        0 days 00:00:08.704573
round_time_total       0 days 00:05:53.059014
loss_total                        7007.952254
loss_critic                       8873.128647
loss_actor                         -452.75391
memory_size                       495823.1405 

=== epoch 9/10 ===== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   42
episode_length                     233.166667
returns                            261.308864
return_std                         177.644641
average_reward                       1.122047
round_time             0 days 00:05:55.463143
episodes_test                            70.0
episode_length_test                142.457143
returns_test                       171.102461
return_std_test                     14.483351
average_reward_test                  1.201792
round_time_test        0 days 00:00:08.687495
round_time_total       0 days 00:05:55.464260
loss_total                        6975.635406
loss_critic                       8833.583202
loss_actor                        -456.156399
memory_size                       497666.7425 

=== epoch 9/10 ===== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:53,  5.62it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   42
episode_length                     225.880952
returns                            253.646388
return_std                         154.815014
average_reward                       1.121183
round_time             0 days 00:05:57.621353
episodes_test                            69.0
episode_length_test                142.855072
returns_test                       170.377945
return_std_test                     35.367355
average_reward_test                  1.192412
round_time_test        0 days 00:00:08.666130
round_time_total       0 days 00:05:57.622667
loss_total                        7281.708783
loss_critic                        9216.79312
loss_actor                        -458.629217
memory_size                        499512.748 

=== epoch 9/10 ===== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.67it/s]
episodes                                   48
episode_length                       206.3125
returns                            231.514599
return_std                         144.367923
average_reward                       1.122907
round_time             0 days 00:05:53.562386
episodes_test                            40.0
episode_length_test                   246.925
returns_test                       273.821135
return_std_test                     75.782905
average_reward_test                  1.109783
round_time_test        0 days 00:00:08.692644
round_time_total       0 days 00:05:53.563638
loss_total                        7144.093831
loss_critic                       9044.815582
loss_actor                         -458.79374
memory_size                       501337.2845 

=== epoch 9/10 ===== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:03,  4.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   45
episode_length                     216.688889
returns                            243.545893
return_std                         140.066743
average_reward                       1.124175
round_time             0 days 00:05:56.412956
episodes_test                            60.0
episode_length_test                165.816667
returns_test                       191.988031
return_std_test                     54.588072
average_reward_test                  1.158034
round_time_test        0 days 00:00:08.638076
round_time_total       0 days 00:05:56.414076
loss_total                        7089.093426
loss_critic                       8975.868654
loss_actor                        -458.008148
memory_size                        503189.314 

=== epoch 9/10 ===== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   43
episode_length                     218.767442
returns                            245.187359
return_std                         146.180514
average_reward                       1.118081
round_time             0 days 00:05:55.847510
episodes_test                            10.0
episode_length_test                     900.8
returns_test                       921.025867
return_std_test                    305.291811
average_reward_test                    1.0224
round_time_test        0 days 00:00:08.674035
round_time_total       0 days 00:05:55.848791
loss_total                        7038.418463
loss_critic                       8912.481688
loss_actor                        -457.835032
memory_size                        505061.097 

=== epoch 9/10 ===== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:34,  5.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   41
episode_length                     232.780488
returns                             258.82354
return_std                         195.284561
average_reward                       1.109892
round_time             0 days 00:05:55.100919
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1022.124435
return_std_test                      2.053138
average_reward_test                  1.022124
round_time_test        0 days 00:00:08.900731
round_time_total       0 days 00:05:55.102022
loss_total                        7211.018894
loss_critic                       9128.454826
loss_actor                        -458.725526
memory_size                       506952.0665 

=== epoch 9/10 ===== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:43,  4.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   40
episode_length                        245.475
returns                            272.938384
return_std                         207.546404
average_reward                       1.112256
round_time             0 days 00:05:57.511019
episodes_test                            25.0
episode_length_test                    387.76
returns_test                       415.995709
return_std_test                     85.919429
average_reward_test                   1.07322
round_time_test        0 days 00:00:08.518149
round_time_total       0 days 00:05:57.512115
loss_total                        7062.532342
loss_critic                       8943.147366
loss_actor                        -459.928393
memory_size                       508847.2945 

=== epoch 9/10 ===== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.60it/s]
episodes                                   33
episode_length                     292.424242
returns                            322.725896
return_std                         218.205582
average_reward                       1.104675
round_time             0 days 00:05:57.887247
episodes_test                            34.0
episode_length_test                287.294118
returns_test                       314.513677
return_std_test                     69.674884
average_reward_test                  1.094905
round_time_test        0 days 00:00:08.643654
round_time_total       0 days 00:05:57.888833
loss_total                        7301.097719
loss_critic                       9242.652202
loss_actor                        -465.120862
memory_size                       510738.7895 

=== epoch 9/10 ===== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:37,  5.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   39
episode_length                     253.923077
returns                             281.57654
return_std                         220.079047
average_reward                       1.110405
round_time             0 days 00:05:56.482408
episodes_test                            11.0
episode_length_test                868.272727
returns_test                       894.956721
return_std_test                    220.355209
average_reward_test                  1.031804
round_time_test        0 days 00:00:08.654985
round_time_total       0 days 00:05:56.483511
loss_total                        7204.359829
loss_critic                       9122.236168
loss_actor                        -467.146163
memory_size                       512615.1275 

=== epoch 9/10 ===== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                   37
episode_length                     254.918919
returns                            282.777571
return_std                         197.876355
average_reward                       1.106892
round_time             0 days 00:05:59.397882
episodes_test                            52.0
episode_length_test                188.134615
returns_test                       216.108338
return_std_test                     36.382645
average_reward_test                  1.147993
round_time_test        0 days 00:00:08.692598
round_time_total       0 days 00:05:59.398989
loss_total                        7334.837291
loss_critic                       9286.017789
loss_actor                        -469.885324
memory_size                       514458.9215 

=== epoch 9/10 ===== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:49,  5.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   40
episode_length                          242.2
returns                            270.808855
return_std                         192.137632
average_reward                       1.118263
round_time             0 days 00:05:53.980031
episodes_test                            57.0
episode_length_test                173.385965
returns_test                       201.686421
return_std_test                     21.921036
average_reward_test                  1.163753
round_time_test        0 days 00:00:08.713254
round_time_total       0 days 00:05:53.981154
loss_total                        7385.954075
loss_critic                       9350.525583
loss_actor                        -472.332655
memory_size                       516303.6435 

=== epoch 9/10 ===== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:53,  4.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   43
episode_length                     227.418605
returns                            255.183101
return_std                         164.991011
average_reward                       1.120716
round_time             0 days 00:05:57.032819
episodes_test                            55.0
episode_length_test                178.072727
returns_test                       206.513769
return_std_test                     27.573178
average_reward_test                  1.159262
round_time_test        0 days 00:00:08.678575
round_time_total       0 days 00:05:57.033955
loss_total                        7219.287071
loss_critic                       9142.186828
loss_actor                        -472.312609
memory_size                       518160.1755 

=== epoch 9/10 ===== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:01,  5.51it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                   43
episode_length                     216.302326
returns                            243.954694
return_std                         156.001204
average_reward                       1.126682
round_time             0 days 00:05:59.459768
episodes_test                            28.0
episode_length_test                348.464286
returns_test                       376.793822
return_std_test                     80.114214
average_reward_test                  1.081881
round_time_test        0 days 00:00:08.472610
round_time_total       0 days 00:05:59.460886
loss_total                        7247.200677
loss_critic                       9176.912873
loss_actor                        -471.648698
memory_size                        520026.471 

=== epoch 9/10 ===== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.62it/s]
episodes                                   38
episode_length                     239.921053
returns                             270.36588
return_std                         153.229801
average_reward                       1.119349
round_time             0 days 00:05:56.566327
episodes_test                            42.0
episode_length_test                237.809524
returns_test                       264.540142
return_std_test                     76.834065
average_reward_test                  1.112329
round_time_test        0 days 00:00:08.633128
round_time_total       0 days 00:05:56.567463
loss_total                        7205.567388
loss_critic                       9125.258961
loss_actor                         -473.19946
memory_size                       521902.7025 

=== epoch 9/10 ===== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:59,  5.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:59<00:00,  5.57it/s]
episodes                                   40
episode_length                         245.75
returns                            275.761759
return_std                         166.579806
average_reward                       1.122151
round_time             0 days 00:05:59.620835
episodes_test                            58.0
episode_length_test                168.827586
returns_test                       196.634028
return_std_test                     36.905726
average_reward_test                  1.164101
round_time_test        0 days 00:00:08.626827
round_time_total       0 days 00:05:59.622322
loss_total                        7488.290093
loss_critic                       9478.419243
loss_actor                         -472.22718
memory_size                        523789.362 

=== epoch 9/10 ===== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:15,  4.57it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                   36
episode_length                     269.527778
returns                            297.188221
return_std                          176.57225
average_reward                       1.103113
round_time             0 days 00:05:55.714423
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1024.196062
return_std_test                      1.592344
average_reward_test                  1.024196
round_time_test        0 days 00:00:08.726230
round_time_total       0 days 00:05:55.715708
loss_total                        7369.102277
loss_critic                       9329.697747
loss_actor                        -473.280206
memory_size                        525676.465 


<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 10/10 ==== round 1/50 ======================================
  1%|          | 12/2000 [00:02<07:12,  4.60it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:42<00:00,  5.84it/s]
episodes                                    8
episode_length                        244.375
returns                            269.404724
return_std                         110.037772
average_reward                       1.103837
round_time             0 days 00:05:42.395051
episodes_test                            10.0
episode_length_test                     972.3
returns_test                       997.430043
return_std_test                     82.203876
average_reward_test                  1.027652
round_time_test        0 days 00:00:08.638588
round_time_total       0 days 00:05:42.396377
loss_total                        7155.259216
loss_critic                       9062.378147
loss_actor                         -473.21713
memory_size                       527545.2675 

=== epoch 10/10 ==== round 2/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:31,  5.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:45<00:00,  5.79it/s]
episodes                                   18
episode_length                     221.666667
returns                            248.943183
return_std                         113.970989
average_reward                       1.122863
round_time             0 days 00:05:45.824598
episodes_test                            10.0
episode_length_test                     999.9
returns_test                      1026.446568
return_std_test                      2.962178
average_reward_test                  1.026549
round_time_test        0 days 00:00:08.662776
round_time_total       0 days 00:05:45.825735
loss_total                        7340.391332
loss_critic                       9293.476357
loss_actor                        -471.949426
memory_size                         529379.75 

=== epoch 10/10 ==== round 3/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:02,  5.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.76it/s]
episodes                                   24
episode_length                     248.416667
returns                            276.375472
return_std                         121.868177
average_reward                       1.113573
round_time             0 days 00:05:47.789035
episodes_test                            11.0
episode_length_test                843.272727
returns_test                       868.588021
return_std_test                     200.54029
average_reward_test                  1.030412
round_time_test        0 days 00:00:08.602613
round_time_total       0 days 00:05:47.790152
loss_total                        7307.765645
loss_critic                       9253.129597
loss_actor                        -473.690768
memory_size                       531246.7105 

=== epoch 10/10 ==== round 4/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:21,  5.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                   31
episode_length                     254.870968
returns                            283.002092
return_std                         120.638087
average_reward                       1.109656
round_time             0 days 00:05:48.504419
episodes_test                            11.0
episode_length_test                845.909091
returns_test                        872.36225
return_std_test                    237.348912
average_reward_test                   1.03165
round_time_test        0 days 00:00:08.550677
round_time_total       0 days 00:05:48.505561
loss_total                        7347.499841
loss_critic                       9303.351234
loss_actor                        -475.906399
memory_size                        533124.917 

=== epoch 10/10 ==== round 5/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:00,  5.53it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.73it/s]
episodes                                   34
episode_length                     288.882353
returns                            315.509098
return_std                         174.007367
average_reward                       1.091959
round_time             0 days 00:05:49.865101
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.276851
return_std_test                       1.17366
average_reward_test                  1.021277
round_time_test        0 days 00:00:08.617821
round_time_total       0 days 00:05:49.866355
loss_total                        7382.971587
loss_critic                       9348.163159
loss_actor                        -477.795325
memory_size                       535045.3955 

=== epoch 10/10 ==== round 6/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:01,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                   30
episode_length                     317.133333
returns                            344.673834
return_std                         221.859597
average_reward                       1.087067
round_time             0 days 00:05:50.273563
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1023.901602
return_std_test                      1.341207
average_reward_test                  1.023902
round_time_test        0 days 00:00:08.564099
round_time_total       0 days 00:05:50.274906
loss_total                        7212.187095
loss_critic                       9134.896674
loss_actor                        -478.651839
memory_size                       536984.1235 

=== epoch 10/10 ==== round 7/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:15,  5.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                   28
episode_length                     346.892857
returns                            373.978872
return_std                         262.406909
average_reward                       1.078975
round_time             0 days 00:05:48.289722
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.837879
return_std_test                      1.831203
average_reward_test                  1.021838
round_time_test        0 days 00:00:08.598916
round_time_total       0 days 00:05:48.290847
loss_total                        7343.986907
loss_critic                       9299.928157
loss_actor                        -479.778707
memory_size                        538880.019 

=== epoch 10/10 ==== round 8/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:47<00:00,  5.75it/s]
episodes                                   27
episode_length                      363.62963
returns                            392.338302
return_std                         287.569364
average_reward                       1.080324
round_time             0 days 00:05:48.487641
episodes_test                            26.0
episode_length_test                380.115385
returns_test                       408.429659
return_std_test                     96.751777
average_reward_test                  1.075681
round_time_test        0 days 00:00:08.687586
round_time_total       0 days 00:05:48.488728
loss_total                         7384.06681
loss_critic                       9350.736998
loss_actor                        -482.614545
memory_size                       540759.3385 

=== epoch 10/10 ==== round 9/50 ======================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:30,  5.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   24
episode_length                     395.791667
returns                            426.819998
return_std                         303.523341
average_reward                       1.072514
round_time             0 days 00:05:51.940997
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                       1022.93361
return_std_test                      1.392472
average_reward_test                  1.022934
round_time_test        0 days 00:00:08.677958
round_time_total       0 days 00:05:51.942125
loss_total                        7235.945006
loss_critic                       9165.826898
loss_actor                        -483.583137
memory_size                        542695.785 

=== epoch 10/10 ==== round 10/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:24,  5.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   29
episode_length                     312.137931
returns                            340.031244
return_std                          237.71892
average_reward                        1.08647
round_time             0 days 00:05:51.219265
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.751219
return_std_test                      2.111253
average_reward_test                  1.021751
round_time_test        0 days 00:00:08.606922
round_time_total       0 days 00:05:51.220517
loss_total                        7170.063875
loss_critic                       9083.386544
loss_actor                        -483.227429
memory_size                       544584.8775 

=== epoch 10/10 ==== round 11/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:39,  4.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   29
episode_length                     316.793103
returns                              345.8818
return_std                         218.317704
average_reward                       1.087619
round_time             0 days 00:05:51.420104
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1025.664579
return_std_test                      2.071208
average_reward_test                  1.025665
round_time_test        0 days 00:00:08.683990
round_time_total       0 days 00:05:51.421237
loss_total                        7245.162804
loss_critic                       9177.368655
loss_actor                         -483.66121
memory_size                        546482.323 

=== epoch 10/10 ==== round 12/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:47,  5.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                   23
episode_length                     390.869565
returns                            421.565612
return_std                          245.64878
average_reward                       1.079138
round_time             0 days 00:05:52.798901
episodes_test                            23.0
episode_length_test                431.826087
returns_test                       459.967099
return_std_test                    188.781368
average_reward_test                  1.066376
round_time_test        0 days 00:00:08.701866
round_time_total       0 days 00:05:52.800001
loss_total                        7391.464495
loss_critic                       9361.379481
loss_actor                        -488.196128
memory_size                       548420.6385 

=== epoch 10/10 ==== round 13/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:00,  5.52it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   25
episode_length                          360.8
returns                            390.447516
return_std                         259.036153
average_reward                        1.07861
round_time             0 days 00:05:51.150143
episodes_test                            13.0
episode_length_test                738.076923
returns_test                       765.236788
return_std_test                    212.372252
average_reward_test                  1.037874
round_time_test        0 days 00:00:08.823777
round_time_total       0 days 00:05:51.151512
loss_total                         7390.64171
loss_critic                       9360.676433
loss_actor                        -489.497833
memory_size                       550333.6995 

=== epoch 10/10 ==== round 14/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:22,  5.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:49<00:00,  5.72it/s]
episodes                                   26
episode_length                     372.653846
returns                            404.564594
return_std                         287.645937
average_reward                       1.083116
round_time             0 days 00:05:50.458698
episodes_test                            10.0
episode_length_test                     901.0
returns_test                       920.007065
return_std_test                    305.620302
average_reward_test                  1.021187
round_time_test        0 days 00:00:08.787966
round_time_total       0 days 00:05:50.459805
loss_total                        7405.928702
loss_critic                       9379.357606
loss_actor                        -487.787561
memory_size                        552245.567 

=== epoch 10/10 ==== round 15/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:52,  5.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   22
episode_length                     427.818182
returns                            461.586748
return_std                         310.876649
average_reward                       1.078878
round_time             0 days 00:05:53.363606
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1023.090966
return_std_test                      1.648608
average_reward_test                  1.023091
round_time_test        0 days 00:00:08.718127
round_time_total       0 days 00:05:53.364968
loss_total                        7479.824148
loss_critic                       9471.517669
loss_actor                        -486.950647
memory_size                        554169.152 

=== epoch 10/10 ==== round 16/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:51,  5.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                   20
episode_length                         450.35
returns                            484.759074
return_std                         321.998998
average_reward                       1.072676
round_time             0 days 00:05:52.735427
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1020.593929
return_std_test                      1.950027
average_reward_test                  1.020594
round_time_test        0 days 00:00:08.598787
round_time_total       0 days 00:05:52.736632
loss_total                        7342.891847
loss_critic                       9300.458897
loss_actor                        -487.376946
memory_size                       556105.7165 

=== epoch 10/10 ==== round 17/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:53,  5.63it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.68it/s]
episodes                                   22
episode_length                     437.772727
returns                            468.478575
return_std                         306.063066
average_reward                       1.069384
round_time             0 days 00:05:52.919970
episodes_test                            13.0
episode_length_test                     767.0
returns_test                       794.482004
return_std_test                    228.987541
average_reward_test                  1.036468
round_time_test        0 days 00:00:08.586726
round_time_total       0 days 00:05:52.921076
loss_total                        7332.388822
loss_critic                       9287.433417
loss_actor                        -487.790221
memory_size                       558052.3765 

=== epoch 10/10 ==== round 18/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:45,  5.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   19
episode_length                     494.631579
returns                            525.519712
return_std                         269.349093
average_reward                       1.064317
round_time             0 days 00:05:56.054896
episodes_test                            10.0
episode_length_test                     900.7
returns_test                       919.212899
return_std_test                    305.236546
average_reward_test                  1.020997
round_time_test        0 days 00:00:08.664706
round_time_total       0 days 00:05:56.055999
loss_total                        7170.851339
loss_critic                        9086.16089
loss_actor                        -490.387422
memory_size                        559984.416 

=== epoch 10/10 ==== round 19/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:59,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   16
episode_length                       530.1875
returns                            562.722971
return_std                         255.459609
average_reward                       1.057435
round_time             0 days 00:05:51.394356
episodes_test                            11.0
episode_length_test                878.363636
returns_test                       903.084049
return_std_test                    260.057998
average_reward_test                  1.029707
round_time_test        0 days 00:00:08.591249
round_time_total       0 days 00:05:51.395479
loss_total                        7423.351997
loss_critic                       9402.317674
loss_actor                        -492.511365
memory_size                        561928.503 

=== epoch 10/10 ==== round 20/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:02,  5.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   15
episode_length                     595.266667
returns                             628.28143
return_std                         265.399742
average_reward                       1.054157
round_time             0 days 00:05:53.848034
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1020.700572
return_std_test                      0.493579
average_reward_test                  1.020701
round_time_test        0 days 00:00:08.503808
round_time_total       0 days 00:05:53.849521
loss_total                        7350.647931
loss_critic                       9311.243625
loss_actor                        -491.735473
memory_size                        563879.806 

=== epoch 10/10 ==== round 21/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:08,  5.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   16
episode_length                         569.75
returns                            604.553912
return_std                         325.819603
average_reward                        1.05673
round_time             0 days 00:05:54.814449
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                       1019.85475
return_std_test                      1.812014
average_reward_test                  1.019855
round_time_test        0 days 00:00:08.752033
round_time_total       0 days 00:05:54.815569
loss_total                        7400.068112
loss_critic                       9373.532925
loss_actor                        -493.791739
memory_size                        565821.089 

=== epoch 10/10 ==== round 22/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<05:56,  5.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   15
episode_length                     588.466667
returns                            624.649898
return_std                         325.091095
average_reward                       1.061075
round_time             0 days 00:05:56.351493
episodes_test                            12.0
episode_length_test                     759.0
returns_test                        782.34631
return_std_test                    360.886432
average_reward_test                  1.030612
round_time_test        0 days 00:00:08.734955
round_time_total       0 days 00:05:56.352841
loss_total                        7359.887824
loss_critic                       9323.264525
loss_actor                        -493.619596
memory_size                       567762.4485 

=== epoch 10/10 ==== round 23/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:58,  4.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   14
episode_length                     620.214286
returns                            655.673327
return_std                         333.338007
average_reward                       1.054672
round_time             0 days 00:05:53.214173
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1017.497062
return_std_test                      1.211572
average_reward_test                  1.017497
round_time_test        0 days 00:00:08.709588
round_time_total       0 days 00:05:53.215276
loss_total                        7331.052443
loss_critic                       9287.621527
loss_actor                        -495.224555
memory_size                       569719.2115 

=== epoch 10/10 ==== round 24/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:52,  4.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.63it/s]
episodes                                   14
episode_length                     672.642857
returns                            708.304473
return_std                         351.116944
average_reward                       1.052804
round_time             0 days 00:05:55.564409
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1016.959667
return_std_test                      1.215519
average_reward_test                   1.01696
round_time_test        0 days 00:00:08.673896
round_time_total       0 days 00:05:55.565523
loss_total                        7472.164565
loss_critic                       9464.413598
loss_actor                        -496.832207
memory_size                        571678.923 

=== epoch 10/10 ==== round 25/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:06,  5.44it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   16
episode_length                       606.4375
returns                            640.828209
return_std                         340.593613
average_reward                       1.055632
round_time             0 days 00:05:52.237472
episodes_test                            11.0
episode_length_test                825.636364
returns_test                       852.560777
return_std_test                    270.363897
average_reward_test                  1.031844
round_time_test        0 days 00:00:08.622820
round_time_total       0 days 00:05:52.238611
loss_total                        7406.032631
loss_critic                        9382.28318
loss_actor                        -498.970224
memory_size                       573621.1705 

=== epoch 10/10 ==== round 26/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:33,  5.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.71it/s]
episodes                                   18
episode_length                     550.666667
returns                            580.648754
return_std                         348.501699
average_reward                       1.055382
round_time             0 days 00:05:51.015847
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.797779
return_std_test                      1.583756
average_reward_test                  1.021798
round_time_test        0 days 00:00:08.547019
round_time_total       0 days 00:05:51.017313
loss_total                        7300.748685
loss_critic                       9250.945168
loss_actor                        -500.037876
memory_size                       575534.8555 

=== epoch 10/10 ==== round 27/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   21
episode_length                     446.428571
returns                            475.096228
return_std                         315.069755
average_reward                       1.065166
round_time             0 days 00:05:54.261917
episodes_test                            10.0
episode_length_test                     951.1
returns_test                       975.312252
return_std_test                    145.157934
average_reward_test                  1.026671
round_time_test        0 days 00:00:08.391877
round_time_total       0 days 00:05:54.263174
loss_total                        7467.024654
loss_critic                       9459.256539
loss_actor                        -501.903474
memory_size                       577425.1195 

=== epoch 10/10 ==== round 28/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<05:38,  5.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.69it/s]
episodes                                   21
episode_length                     444.952381
returns                            475.455446
return_std                         316.212009
average_reward                       1.066824
round_time             0 days 00:05:51.754273
episodes_test                            10.0
episode_length_test                     946.2
returns_test                       967.956241
return_std_test                    159.609931
average_reward_test                  1.023953
round_time_test        0 days 00:00:08.799393
round_time_total       0 days 00:05:51.755373
loss_total                         7658.38345
loss_critic                       9698.913339
loss_actor                         -503.73677
memory_size                        579353.921 

=== epoch 10/10 ==== round 29/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:07,  5.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:50<00:00,  5.70it/s]
episodes                                   23
episode_length                      423.26087
returns                            453.501996
return_std                         302.552807
average_reward                       1.071946
round_time             0 days 00:05:51.499655
episodes_test                            10.0
episode_length_test                     900.9
returns_test                       916.777263
return_std_test                    303.471049
average_reward_test                  1.017867
round_time_test        0 days 00:00:08.683976
round_time_total       0 days 00:05:51.500749
loss_total                        7488.309061
loss_critic                       9485.798563
loss_actor                        -501.649615
memory_size                        581289.017 

=== epoch 10/10 ==== round 30/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:19,  5.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:52<00:00,  5.67it/s]
episodes                                   22
episode_length                     439.909091
returns                            474.650611
return_std                         289.808462
average_reward                       1.076175
round_time             0 days 00:05:53.407738
episodes_test                            11.0
episode_length_test                821.636364
returns_test                       837.781345
return_std_test                    382.052014
average_reward_test                  1.019681
round_time_test        0 days 00:00:08.691252
round_time_total       0 days 00:05:53.408836
loss_total                        7372.424155
loss_critic                       9341.070784
loss_actor                        -502.163021
memory_size                        583236.995 

=== epoch 10/10 ==== round 31/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:41,  4.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   18
episode_length                     533.944444
returns                            570.625899
return_std                         315.523322
average_reward                       1.069988
round_time             0 days 00:05:55.708201
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1018.693725
return_std_test                      1.160006
average_reward_test                  1.018694
round_time_test        0 days 00:00:08.692809
round_time_total       0 days 00:05:55.709332
loss_total                        7435.392845
loss_critic                       9420.080852
loss_actor                        -503.359818
memory_size                        585177.315 

=== epoch 10/10 ==== round 32/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 10/2000 [00:01<06:03,  5.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:51<00:00,  5.68it/s]
episodes                                   16
episode_length                       557.8125
returns                            591.594545
return_std                         318.468783
average_reward                       1.058994
round_time             0 days 00:05:52.322364
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                       1020.22307
return_std_test                      1.655456
average_reward_test                  1.020223
round_time_test        0 days 00:00:08.578823
round_time_total       0 days 00:05:52.323461
loss_total                        7425.572267
loss_critic                       9407.664365
loss_actor                        -502.796799
memory_size                        587113.853 

=== epoch 10/10 ==== round 33/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:09,  5.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   17
episode_length                     526.823529
returns                             561.23096
return_std                         347.927883
average_reward                       1.063465
round_time             0 days 00:05:55.167204
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1020.810026
return_std_test                      1.282196
average_reward_test                   1.02081
round_time_test        0 days 00:00:08.815894
round_time_total       0 days 00:05:55.168301
loss_total                        7510.327753
loss_critic                       9513.879225
loss_actor                        -503.878765
memory_size                       589042.9055 

=== epoch 10/10 ==== round 34/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   19
episode_length                     483.894737
returns                            517.356886
return_std                         365.627307
average_reward                       1.067435
round_time             0 days 00:05:54.967506
episodes_test                            10.0
episode_length_test                     900.8
returns_test                        921.11723
return_std_test                    305.345029
average_reward_test                  1.022639
round_time_test        0 days 00:00:08.433174
round_time_total       0 days 00:05:54.968609
loss_total                        7368.498374
loss_critic                       9336.258799
loss_actor                        -502.543918
memory_size                       590990.7185 

=== epoch 10/10 ==== round 35/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:05,  5.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   19
episode_length                     461.526316
returns                            494.493822
return_std                         374.967139
average_reward                       1.067078
round_time             0 days 00:05:53.639531
episodes_test                            10.0
episode_length_test                     900.8
returns_test                       916.971108
return_std_test                     304.20105
average_reward_test                  1.018166
round_time_test        0 days 00:00:08.579515
round_time_total       0 days 00:05:53.640621
loss_total                        7449.252821
loss_critic                       9437.633616
loss_actor                        -504.271005
memory_size                        592875.211 

=== epoch 10/10 ==== round 36/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<07:13,  4.59it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   21
episode_length                     459.857143
returns                            493.599536
return_std                         362.512117
average_reward                       1.075112
round_time             0 days 00:05:56.144145
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1018.737891
return_std_test                      1.283677
average_reward_test                  1.018738
round_time_test        0 days 00:00:08.525336
round_time_total       0 days 00:05:56.145250
loss_total                        7512.438615
loss_critic                       9516.613199
loss_actor                        -504.260371
memory_size                        594818.149 

=== epoch 10/10 ==== round 37/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 7/2000 [00:01<06:59,  4.75it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.65it/s]
episodes                                   19
episode_length                     467.684211
returns                            502.501223
return_std                          377.68695
average_reward                       1.070923
round_time             0 days 00:05:54.338967
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.230314
return_std_test                      1.119019
average_reward_test                   1.02123
round_time_test        0 days 00:00:08.390550
round_time_total       0 days 00:05:54.340084
loss_total                        7390.780127
loss_critic                       9364.814977
loss_actor                        -505.359969
memory_size                       596766.2825 

=== epoch 10/10 ==== round 38/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:57,  4.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.63it/s]
episodes                                   18
episode_length                     513.277778
returns                            546.934688
return_std                         380.277688
average_reward                       1.063454
round_time             0 days 00:05:55.942068
episodes_test                            10.0
episode_length_test                     900.8
returns_test                       915.899136
return_std_test                    303.408186
average_reward_test                  1.016791
round_time_test        0 days 00:00:08.611302
round_time_total       0 days 00:05:55.943152
loss_total                        7337.618548
loss_critic                       9297.890518
loss_actor                        -503.469978
memory_size                        598717.859 

=== epoch 10/10 ==== round 39/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:35,  5.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   14
episode_length                          647.5
returns                            682.441065
return_std                         369.021023
average_reward                       1.054588
round_time             0 days 00:05:56.971602
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.797498
return_std_test                      1.260405
average_reward_test                  1.021797
round_time_test        0 days 00:00:08.425079
round_time_total       0 days 00:05:56.973048
loss_total                        7261.523514
loss_critic                       9203.451805
loss_actor                        -506.190289
memory_size                        600675.077 

=== epoch 10/10 ==== round 40/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:09,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   16
episode_length                          602.5
returns                            634.124877
return_std                         351.249495
average_reward                        1.05482
round_time             0 days 00:05:53.880192
episodes_test                            10.0
episode_length_test                     977.6
returns_test                       997.621684
return_std_test                     65.243058
average_reward_test                  1.022044
round_time_test        0 days 00:00:08.494993
round_time_total       0 days 00:05:53.881332
loss_total                        7402.068662
loss_critic                       9379.308199
loss_actor                        -506.890072
memory_size                        602610.826 

=== epoch 10/10 ==== round 41/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:08,  5.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.65it/s]
episodes                                   14
episode_length                     632.642857
returns                            664.149316
return_std                         362.011036
average_reward                       1.049316
round_time             0 days 00:05:54.801253
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                       1018.40061
return_std_test                      1.469423
average_reward_test                  1.018401
round_time_test        0 days 00:00:08.697685
round_time_total       0 days 00:05:54.802357
loss_total                         7319.96009
loss_critic                       9277.449648
loss_actor                        -509.998762
memory_size                        604546.233 

=== epoch 10/10 ==== round 42/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:20,  5.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   15
episode_length                     658.533333
returns                            692.721185
return_std                         361.519808
average_reward                       1.052029
round_time             0 days 00:05:56.204178
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1018.979293
return_std_test                      0.936506
average_reward_test                  1.018979
round_time_test        0 days 00:00:08.611632
round_time_total       0 days 00:05:56.205271
loss_total                        7270.644302
loss_critic                       9216.367249
loss_actor                        -512.248148
memory_size                        606502.522 

=== epoch 10/10 ==== round 43/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:26,  5.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   14
episode_length                     681.142857
returns                            720.732733
return_std                         363.434869
average_reward                       1.059113
round_time             0 days 00:05:56.894898
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1020.851355
return_std_test                      2.082937
average_reward_test                  1.020851
round_time_test        0 days 00:00:08.766703
round_time_total       0 days 00:05:56.895990
loss_total                        7330.473286
loss_critic                       9291.098262
loss_actor                        -512.027217
memory_size                        608459.627 

=== epoch 10/10 ==== round 44/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:26,  5.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:53<00:00,  5.66it/s]
episodes                                   14
episode_length                     685.785714
returns                            726.606245
return_std                         368.416324
average_reward                       1.059847
round_time             0 days 00:05:54.110749
episodes_test                            10.0
episode_length_test                     976.8
returns_test                       999.782264
return_std_test                     68.280693
average_reward_test                   1.02516
round_time_test        0 days 00:00:08.496778
round_time_total       0 days 00:05:54.111850
loss_total                        7316.281643
loss_critic                       9273.980165
loss_actor                         -514.51303
memory_size                       610419.4495 

=== epoch 10/10 ==== round 45/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:59,  4.74it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:57<00:00,  5.59it/s]
episodes                                   11
episode_length                     798.363636
returns                            838.874123
return_std                         366.834104
average_reward                        1.04962
round_time             0 days 00:05:58.453241
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1017.690086
return_std_test                      2.024544
average_reward_test                   1.01769
round_time_test        0 days 00:00:08.473707
round_time_total       0 days 00:05:58.454525
loss_total                        7636.611924
loss_critic                       9675.140152
loss_actor                        -517.501614
memory_size                       612382.6145 

=== epoch 10/10 ==== round 46/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:39,  4.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:54<00:00,  5.64it/s]
episodes                                   12
episode_length                     761.333333
returns                            798.500812
return_std                         371.360338
average_reward                       1.047189
round_time             0 days 00:05:55.304221
episodes_test                            10.0
episode_length_test                     940.8
returns_test                       963.793474
return_std_test                    176.354699
average_reward_test                  1.025265
round_time_test        0 days 00:00:08.542329
round_time_total       0 days 00:05:55.305308
loss_total                        7570.785117
loss_critic                       9593.224874
loss_actor                        -518.974595
memory_size                       614340.9265 

=== epoch 10/10 ==== round 47/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:27,  5.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
episodes                                   13
episode_length                     690.076923
returns                             725.88974
return_std                         363.259375
average_reward                       1.052285
round_time             0 days 00:05:56.201259
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1021.154389
return_std_test                      1.805805
average_reward_test                  1.021154
round_time_test        0 days 00:00:08.635906
round_time_total       0 days 00:05:56.202366
loss_total                        7487.971273
loss_critic                       9489.402009
loss_actor                        -517.752339
memory_size                       616284.0235 

=== epoch 10/10 ==== round 48/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:17,  5.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:58<00:00,  5.57it/s]
episodes                                   13
episode_length                     710.538462
returns                            743.204088
return_std                           370.2807
average_reward                       1.045274
round_time             0 days 00:05:59.307610
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1022.055669
return_std_test                      0.830442
average_reward_test                  1.022056
round_time_test        0 days 00:00:08.597443
round_time_total       0 days 00:05:59.309071
loss_total                        7397.430637
loss_critic                       9375.902981
loss_actor                        -516.459396
memory_size                        618225.105 

=== epoch 10/10 ==== round 49/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 8/2000 [00:01<06:45,  4.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:56<00:00,  5.61it/s]
episodes                                   13
episode_length                     682.076923
returns                            712.045221
return_std                         361.701455
average_reward                       1.042502
round_time             0 days 00:05:57.009761
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1020.893336
return_std_test                      0.962861
average_reward_test                  1.020893
round_time_test        0 days 00:00:08.590107
round_time_total       0 days 00:05:57.010882
loss_total                        7516.839121
loss_critic                       9525.104526
loss_actor                          -516.2232
memory_size                       620185.7905 

=== epoch 10/10 ==== round 50/50 =====================================
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 9/2000 [00:01<06:16,  5.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [05:55<00:00,  5.62it/s]
<MM1_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
episodes                                   14
episode_length                     649.214286
returns                            682.872361
return_std                         296.490246
average_reward                       1.050851
round_time             0 days 00:05:56.472822
episodes_test                            10.0
episode_length_test                    1000.0
returns_test                      1023.452165
return_std_test                      2.836113
average_reward_test                  1.023452
round_time_test        0 days 00:00:08.798800
round_time_total       0 days 00:05:56.473907
loss_total                        7239.314711
loss_critic                       9178.516455
loss_actor                         -517.49292
memory_size                        622127.457 


