/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
=== specification ====================================================
+: rlrd.training:Training
epochs: 10
rounds: 50
steps: 2000
stats_window: 10000
seed: 0
tag: ''
Env:
   +: rlrd.envs:RandomDelayEnv
   seed_val: 0
   id: Hopper-v4
   frame_skip: 0
   min_observation_delay: 0
   sup_observation_delay: 1
   min_action_delay: 0
   sup_action_delay: 1
   real_world_sampler: 3
   action_noise: 0.05
Test:
   +: rlrd.testing:Test
   workers: 1
   number: 1
   device: cpu
Agent:
   +: rlrd.dcac:Agent
   batchsize: 128
   memory_size: 1000000
   lr: 0.0003
   discount: 0.99
   target_update: 0.005
   reward_scale: 5.0
   entropy_scale: 1.0
   start_training: 10000
   device: cpu
   training_steps: 1.0
   loss_alpha: 0.2
   rtac: false
   Model:
      +: rlrd.dcac_models:Mlp
      hidden_units: 256
      num_critics: 2
      act_delay: true
      obs_delay: true
   OutputNorm:
      +: rlrd.nn:PopArt
      beta: 0.0003
      zero_debias: true
      start_pop: 8
__format_version__: '3'
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>

<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 1/10 ===== round 1/50 ======================================
100%|██████████| 2000/2000 [00:01<00:00, 1137.71it/s]
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  117
episode_length                      17.094017
returns                             14.310515
return_std                           9.574812
average_reward                       0.837165
round_time             0 days 00:00:01.774508
episodes_test                           543.0
episode_length_test                  18.39779
returns_test                        16.208822
return_std_test                      7.141692
average_reward_test                  0.881071
round_time_test        0 days 00:00:09.076027
round_time_total       0 days 00:00:10.825469 

=== epoch 1/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
 95%|█████████▌| 1902/2000 [00:01<00:00, 1151.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:01<00:00, 1161.47it/s]
episodes                                  228
episode_length                      17.482456
returns                             14.141401
return_std                           9.123539
average_reward                       0.810039
round_time             0 days 00:00:02.254541
episodes_test                           554.0
episode_length_test                 18.036101
returns_test                        15.888273
return_std_test                      7.021297
average_reward_test                  0.880933
round_time_test        0 days 00:00:09.012715
round_time_total       0 days 00:00:10.753256 

=== epoch 1/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
 95%|█████████▍| 1899/2000 [00:01<00:00, 1081.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:01<00:00, 1134.70it/s]
episodes                                  333
episode_length                      17.996997
returns                             15.305022
return_std                          14.872394
average_reward                        0.85038
round_time             0 days 00:00:02.274317
episodes_test                           544.0
episode_length_test                 18.351103
returns_test                        16.043945
return_std_test                      7.436389
average_reward_test                  0.874297
round_time_test        0 days 00:00:09.105385
round_time_total       0 days 00:00:10.920017 

=== epoch 1/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
 93%|█████████▎| 1861/2000 [00:01<00:00, 1114.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:01<00:00, 1134.17it/s]
episodes                                  443
episode_length                      18.058691
returns                              15.19315
return_std                          13.888303
average_reward                       0.841321
round_time             0 days 00:00:02.288190
episodes_test                           557.0
episode_length_test                 17.931777
returns_test                        15.654733
return_std_test                      6.754711
average_reward_test                  0.872851
round_time_test        0 days 00:00:09.093362
round_time_total       0 days 00:00:10.861945 

=== epoch 1/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
 93%|█████████▎| 1866/2000 [00:01<00:00, 1141.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [00:01<00:00, 1143.32it/s]
episodes                                  548
episode_length                      18.220803
returns                             15.415268
return_std                          14.574614
average_reward                       0.846062
round_time             0 days 00:00:02.255259
episodes_test                           553.0
episode_length_test                 18.037975
returns_test                        15.826682
return_std_test                      7.371679
average_reward_test                   0.87833
round_time_test        0 days 00:00:08.805748
round_time_total       0 days 00:00:10.601620 

=== epoch 1/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 0/2000 [00:00<?, ?it/s]/<ANONYMIZED PATH>/rmst-rlrd/rlrd/nn.py:41: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
  assert b.storage().data_ptr() == a.storage().data_ptr()
  0%|          | 4/2000 [00:01<11:11,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:15<00:00,  3.25it/s]
starting training
episodes                                  504
episode_length                      19.839286
returns                             18.150858
return_std                          20.900631
average_reward                       0.914902
round_time             0 days 00:10:15.994746
episodes_test                           539.0
episode_length_test                 18.541744
returns_test                        16.506837
return_std_test                      7.463801
average_reward_test                  0.890309
round_time_test        0 days 00:00:08.989424
round_time_total       0 days 00:10:15.995867
loss_total                        1239.735324
loss_critic                       1584.845178
loss_actor                        -140.704199
memory_size                          1321.991 

=== epoch 1/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:08,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  518
episode_length                      19.274131
returns                             17.670237
return_std                          20.728262
average_reward                       0.918285
round_time             0 days 00:10:18.390515
episodes_test                           588.0
episode_length_test                 16.982993
returns_test                          16.9902
return_std_test                      15.94402
average_reward_test                  1.000277
round_time_test        0 days 00:00:09.092875
round_time_total       0 days 00:10:18.391638
loss_total                       30433.428066
loss_critic                      38456.890237
loss_actor                       -1660.423206
memory_size                         1642.8135 

=== epoch 1/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:38,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  537
episode_length                      18.605214
returns                             16.444011
return_std                          18.447579
average_reward                       0.883862
round_time             0 days 00:10:20.830638
episodes_test                           631.0
episode_length_test                 15.847861
returns_test                        11.602696
return_std_test                       7.72403
average_reward_test                   0.73213
round_time_test        0 days 00:00:09.049102
round_time_total       0 days 00:10:20.831756
loss_total                     2555623.718527
loss_critic                    3197679.335758
loss_actor                      -12598.963172
memory_size                         1736.6205 

=== epoch 1/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:27,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  543
episode_length                      18.395948
returns                             15.339454
return_std                          18.143448
average_reward                       0.833833
round_time             0 days 00:10:18.163575
episodes_test                           591.0
episode_length_test                 16.913706
returns_test                        11.532129
return_std_test                      7.640194
average_reward_test                   0.68193
round_time_test        0 days 00:00:09.072898
round_time_total       0 days 00:10:18.164695
loss_total                      51557048.4695
loss_critic                    64458297.01075
loss_actor                       -47950.18864
memory_size                          1841.685 

=== epoch 1/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:25,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:14<00:00,  3.25it/s]
episodes                                  567
episode_length                      17.594356
returns                             13.607077
return_std                          16.587515
average_reward                       0.774076
round_time             0 days 00:10:15.424521
episodes_test                           593.0
episode_length_test                 16.827993
returns_test                        10.203704
return_std_test                      6.844964
average_reward_test                  0.606725
round_time_test        0 days 00:00:09.131995
round_time_total       0 days 00:10:15.425648
loss_total                      260407028.074
loss_critic                     325536481.196
loss_actor                     -110805.802051
memory_size                         1889.5215 

=== epoch 1/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:59,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  650
episode_length                      15.356923
returns                             10.081419
return_std                           7.112178
average_reward                       0.656861
round_time             0 days 00:10:17.678551
episodes_test                           719.0
episode_length_test                 13.873435
returns_test                         8.381947
return_std_test                      4.670944
average_reward_test                  0.603629
round_time_test        0 days 00:00:09.239677
round_time_total       0 days 00:10:17.679838
loss_total                     1115337435.264
loss_critic                    1394228777.152
loss_actor                     -228037.174391
memory_size                         1901.9885 

=== epoch 1/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  682
episode_length                      14.636364
returns                              9.077375
return_std                            6.47284
average_reward                       0.620074
round_time             0 days 00:10:17.845908
episodes_test                           791.0
episode_length_test                 12.632111
returns_test                         7.855765
return_std_test                      4.865476
average_reward_test                  0.622042
round_time_test        0 days 00:00:09.401243
round_time_total       0 days 00:10:17.847121
loss_total                     4932828417.216
loss_critic                    6166143194.176
loss_actor                     -431141.600859
memory_size                          1914.925 

=== epoch 1/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  753
episode_length                      13.260292
returns                               7.87183
return_std                           4.874625
average_reward                       0.593818
round_time             0 days 00:10:28.540407
episodes_test                           917.0
episode_length_test                 10.897492
returns_test                         6.846223
return_std_test                      2.862436
average_reward_test                  0.628484
round_time_test        0 days 00:00:09.277186
round_time_total       0 days 00:10:28.541539
loss_total                 17345155817.728001
loss_critic                21681636376.832001
loss_actor                     -768038.838062
memory_size                            1946.0 

=== epoch 1/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  845
episode_length                      11.821302
returns                               7.27291
return_std                           4.390493
average_reward                       0.614929
round_time             0 days 00:10:24.439883
episodes_test                          1042.0
episode_length_test                  9.590211
returns_test                         6.251023
return_std_test                      2.108542
average_reward_test                  0.651815
round_time_test        0 days 00:00:09.458382
round_time_total       0 days 00:10:24.441186
loss_total                 54224891017.216003
loss_critic                67781438215.167999
loss_actor                    -1302670.960406
memory_size                            1946.0 

=== epoch 1/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:01,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  916
episode_length                      10.905022
returns                               6.90041
return_std                           3.661542
average_reward                       0.632636
round_time             0 days 00:10:24.359519
episodes_test                          1017.0
episode_length_test                  9.829892
returns_test                         6.467025
return_std_test                      2.523618
average_reward_test                  0.657971
round_time_test        0 days 00:00:09.412784
round_time_total       0 days 00:10:24.360827
loss_total                143324161662.976013
loss_critic               179155705165.824005
loss_actor                    -2025395.336625
memory_size                            1946.0 

=== epoch 1/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:32,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  951
episode_length                      10.515247
returns                                6.7693
return_std                           3.328978
average_reward                        0.64376
round_time             0 days 00:10:25.063517
episodes_test                          1018.0
episode_length_test                  9.819253
returns_test                         6.391506
return_std_test                      2.034445
average_reward_test                  0.651061
round_time_test        0 days 00:00:09.423146
round_time_total       0 days 00:10:25.064726
loss_total                342859284393.984009
loss_critic               428574839619.583984
loss_actor                     -2965736.09875
memory_size                          1948.406 

=== epoch 1/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:01,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  975
episode_length                      10.252308
returns                              6.710507
return_std                           2.479715
average_reward                       0.654421
round_time             0 days 00:10:24.126464
episodes_test                           946.0
episode_length_test                 10.562368
returns_test                         7.040102
return_std_test                      2.924197
average_reward_test                  0.666605
round_time_test        0 days 00:00:09.430220
round_time_total       0 days 00:10:24.127821
loss_total                739143967686.656006
loss_critic               923930988707.839966
loss_actor                    -4173991.367875
memory_size                          1951.167 

=== epoch 1/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:54,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  960
episode_length                      10.404167
returns                              6.933304
return_std                           2.735408
average_reward                       0.666524
round_time             0 days 00:10:25.335124
episodes_test                           905.0
episode_length_test                 11.045304
returns_test                         7.666531
return_std_test                      4.077287
average_reward_test                  0.694131
round_time_test        0 days 00:00:09.436634
round_time_total       0 days 00:10:25.336602
loss_total                    1492027987968.0
loss_critic               1865036378603.52002
loss_actor                     -5677187.11475
memory_size                          1953.711 

=== epoch 1/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:57,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  930
episode_length                      10.746237
returns                              7.228519
return_std                           2.551864
average_reward                       0.672698
round_time             0 days 00:10:26.617793
episodes_test                           889.0
episode_length_test                  11.23622
returns_test                         7.655945
return_std_test                      2.668828
average_reward_test                  0.681642
round_time_test        0 days 00:00:09.535876
round_time_total       0 days 00:10:26.618887
loss_total               2775767209246.720215
loss_critic              3469710803795.967773
loss_actor                      -7418622.9005
memory_size                            1957.0 

=== epoch 1/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:21,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  888
episode_length                      11.253378
returns                              7.767575
return_std                           3.628879
average_reward                       0.690082
round_time             0 days 00:10:24.779264
episodes_test                           819.0
episode_length_test                 12.192918
returns_test                         8.761537
return_std_test                      4.020953
average_reward_test                  0.718399
round_time_test        0 days 00:00:09.334301
round_time_total       0 days 00:10:24.780363
loss_total               4820986441826.303711
loss_critic              6026235330101.248047
loss_actor                        -9414764.07
memory_size                         1976.1965 

=== epoch 1/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:02,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  851
episode_length                      11.740306
returns                              8.265799
return_std                           4.583201
average_reward                       0.704091
round_time             0 days 00:10:24.781958
episodes_test                           769.0
episode_length_test                 12.996099
returns_test                         9.556235
return_std_test                      7.235085
average_reward_test                  0.735377
round_time_test        0 days 00:00:09.397578
round_time_total       0 days 00:10:24.783118
loss_total               7783723915411.456055
loss_critic              9729657510559.744141
loss_actor                      -11494927.392
memory_size                         2015.4775 

=== epoch 1/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  836
episode_length                      11.947368
returns                              8.498362
return_std                           4.716749
average_reward                       0.711423
round_time             0 days 00:10:23.634004
episodes_test                           864.0
episode_length_test                 11.569444
returns_test                         8.267674
return_std_test                      3.496888
average_reward_test                  0.714695
round_time_test        0 days 00:00:09.427648
round_time_total       0 days 00:10:23.635328
loss_total              10575401632661.503906
loss_critic             13219255305633.792969
loss_actor                      -13617402.222
memory_size                            2031.0 

=== epoch 1/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:18,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  827
episode_length                      12.083434
returns                              8.644131
return_std                           4.709821
average_reward                       0.715383
round_time             0 days 00:10:25.588421
episodes_test                           845.0
episode_length_test                 11.830769
returns_test                         8.407634
return_std_test                      3.851929
average_reward_test                  0.710759
round_time_test        0 days 00:00:09.380207
round_time_total       0 days 00:10:25.589549
loss_total              15423165554229.248047
loss_critic              19278960806854.65625
loss_actor                     -16567703.4245
memory_size                            2031.0 

=== epoch 1/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:46,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  823
episode_length                      12.137303
returns                              8.704005
return_std                           4.832559
average_reward                       0.717113
round_time             0 days 00:10:22.980722
episodes_test                           849.0
episode_length_test                 11.778563
returns_test                          8.39586
return_std_test                      3.744485
average_reward_test                  0.712808
round_time_test        0 days 00:00:09.381900
round_time_total       0 days 00:10:22.981847
loss_total              23775126175350.785156
loss_critic               29718912419495.9375
loss_actor                      -20396866.485
memory_size                            2031.0 

=== epoch 1/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:58,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  845
episode_length                      11.831953
returns                              8.339624
return_std                           4.053455
average_reward                        0.70489
round_time             0 days 00:10:23.460318
episodes_test                           887.0
episode_length_test                  11.26832
returns_test                         7.828547
return_std_test                       2.70305
average_reward_test                  0.694853
round_time_test        0 days 00:00:09.512470
round_time_total       0 days 00:10:23.461608
loss_total              35718814129192.960938
loss_critic             44648522337746.945312
loss_actor                      -24542959.524
memory_size                            2031.0 

=== epoch 1/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  867
episode_length                      11.520185
returns                              8.056449
return_std                           2.948546
average_reward                       0.699306
round_time             0 days 00:10:25.444648
episodes_test                           869.0
episode_length_test                 11.505178
returns_test                         8.169654
return_std_test                      4.049232
average_reward_test                  0.710139
round_time_test        0 days 00:00:09.343628
round_time_total       0 days 00:10:25.445766
loss_total              51818594297184.257812
loss_critic             64773248064159.742188
loss_actor                      -29125951.182
memory_size                            2031.0 

=== epoch 1/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  870
episode_length                      11.477011
returns                              7.991093
return_std                           2.601051
average_reward                       0.696226
round_time             0 days 00:10:24.793173
episodes_test                           885.0
episode_length_test                 11.285876
returns_test                         7.812903
return_std_test                      2.703998
average_reward_test                  0.692368
round_time_test        0 days 00:00:09.340217
round_time_total       0 days 00:10:24.794674
loss_total                70442214416711.6875
loss_critic                88052776727216.125
loss_actor                      -33807799.124
memory_size                            2031.0 

=== epoch 1/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:25,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  869
episode_length                      11.491369
returns                              8.023434
return_std                           2.903014
average_reward                       0.698053
round_time             0 days 00:10:25.445872
episodes_test                           842.0
episode_length_test                 11.870546
returns_test                         8.394907
return_std_test                      4.164044
average_reward_test                  0.707347
round_time_test        0 days 00:00:09.431647
round_time_total       0 days 00:10:25.447014
loss_total              94572467327074.296875
loss_critic              118215592160264.1875
loss_actor                      -38879601.835
memory_size                            2031.0 

=== epoch 1/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:59,  2.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  864
episode_length                      11.565972
returns                              8.073559
return_std                           2.758611
average_reward                       0.697937
round_time             0 days 00:10:26.173634
episodes_test                           860.0
episode_length_test                 11.617442
returns_test                         8.144974
return_std_test                      2.537728
average_reward_test                  0.701241
round_time_test        0 days 00:00:09.292985
round_time_total       0 days 00:10:26.174747
loss_total                  123795544687509.5
loss_critic              154744440483741.6875
loss_actor                      -44447819.614
memory_size                            2031.0 

=== epoch 1/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:52,  2.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  852
episode_length                      11.717136
returns                              8.256492
return_std                           3.159726
average_reward                       0.704778
round_time             0 days 00:10:23.069933
episodes_test                           853.0
episode_length_test                 11.712778
returns_test                         8.321837
return_std_test                      3.265441
average_reward_test                  0.710525
round_time_test        0 days 00:00:09.388228
round_time_total       0 days 00:10:23.071376
loss_total                 160562677295349.75
loss_critic              200703360372834.3125
loss_actor                      -50210746.404
memory_size                            2031.0 

=== epoch 1/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:37,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  849
episode_length                       11.76914
returns                              8.270152
return_std                           3.142455
average_reward                       0.702704
round_time             0 days 00:10:23.932957
episodes_test                           862.0
episode_length_test                 11.593968
returns_test                         8.042019
return_std_test                      2.115669
average_reward_test                  0.693788
round_time_test        0 days 00:00:09.565166
round_time_total       0 days 00:10:23.934095
loss_total                201900650437541.875
loss_critic                 252375827528810.5
loss_actor                      -56144891.418
memory_size                            2031.0 

=== epoch 1/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  849
episode_length                      11.776207
returns                              8.283624
return_std                           3.299475
average_reward                       0.703399
round_time             0 days 00:10:25.521227
episodes_test                           848.0
episode_length_test                 11.784198
returns_test                         8.319406
return_std_test                      2.965147
average_reward_test                  0.706074
round_time_test        0 days 00:00:09.293298
round_time_total       0 days 00:10:25.522382
loss_total              249297009888985.09375
loss_critic                 311621272586944.5
loss_actor                      -62516210.782
memory_size                            2031.0 

=== epoch 1/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  839
episode_length                      11.908224
returns                               8.40117
return_std                           3.063576
average_reward                       0.705427
round_time             0 days 00:10:25.459191
episodes_test                           831.0
episode_length_test                 12.026474
returns_test                         8.388966
return_std_test                      2.876646
average_reward_test                  0.697658
round_time_test        0 days 00:00:09.431746
round_time_total       0 days 00:10:25.460281
loss_total               308579120810295.3125
loss_critic                 385723902866751.5
loss_actor                      -69349958.608
memory_size                            2031.0 

=== epoch 1/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:26,  2.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  835
episode_length                      11.960479
returns                              8.457832
return_std                            3.14596
average_reward                       0.707057
round_time             0 days 00:10:26.350924
episodes_test                           816.0
episode_length_test                 12.242647
returns_test                         8.763648
return_std_test                      3.546173
average_reward_test                  0.715718
round_time_test        0 days 00:00:09.359917
round_time_total       0 days 00:10:26.352049
loss_total                386955605396422.625
loss_critic              483694501714460.6875
loss_actor                      -77002774.048
memory_size                            2031.0 

=== epoch 1/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  831
episode_length                      12.013237
returns                              8.493828
return_std                           2.977841
average_reward                       0.707119
round_time             0 days 00:10:25.685957
episodes_test                           821.0
episode_length_test                 12.171742
returns_test                         8.654328
return_std_test                      3.024728
average_reward_test                  0.711158
round_time_test        0 days 00:00:09.355718
round_time_total       0 days 00:10:25.687255
loss_total                470032785868849.125
loss_critic               587540994245787.625
loss_actor                       -85069164.36
memory_size                            2031.0 

=== epoch 1/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:26,  2.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  825
episode_length                      12.104242
returns                              8.619506
return_std                           3.008015
average_reward                       0.712038
round_time             0 days 00:10:24.909824
episodes_test                           826.0
episode_length_test                 12.096852
returns_test                         8.585949
return_std_test                       3.13665
average_reward_test                  0.709935
round_time_test        0 days 00:00:09.304344
round_time_total       0 days 00:10:24.911109
loss_total                 571457683790495.75
loss_critic                714322114231926.75
loss_actor                      -94156203.484
memory_size                            2031.0 

=== epoch 1/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  819
episode_length                      12.180708
returns                              8.758581
return_std                           3.324785
average_reward                        0.71917
round_time             0 days 00:10:24.544091
episodes_test                           817.0
episode_length_test                  12.23623
returns_test                         8.737801
return_std_test                      3.456362
average_reward_test                   0.71417
round_time_test        0 days 00:00:09.396808
round_time_total       0 days 00:10:24.545398
loss_total                  702306756121854.0
loss_critic                877883438651146.25
loss_actor                     -103735280.508
memory_size                         2031.4745 

=== epoch 1/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  829
episode_length                       12.04222
returns                              8.634259
return_std                           3.103178
average_reward                       0.717125
round_time             0 days 00:10:28.007119
episodes_test                           819.0
episode_length_test                 12.202686
returns_test                         8.835707
return_std_test                      3.916321
average_reward_test                  0.724258
round_time_test        0 days 00:00:09.535965
round_time_total       0 days 00:10:28.008431
loss_total                848963104430620.625
loss_critic                1061203865694634.0
loss_actor                     -113370561.736
memory_size                            2033.0 

=== epoch 1/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:18,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  827
episode_length                      12.081016
returns                              8.650809
return_std                           3.154976
average_reward                       0.715994
round_time             0 days 00:10:26.472974
episodes_test                           797.0
episode_length_test                 12.545797
returns_test                         9.163054
return_std_test                       3.32653
average_reward_test                   0.73039
round_time_test        0 days 00:00:09.125599
round_time_total       0 days 00:10:26.474077
loss_total                 1008872568431575.0
loss_critic               1261090690083848.25
loss_actor                     -123228004.856
memory_size                            2033.0 

=== epoch 1/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:22,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.21it/s]
episodes                                  823
episode_length                      12.132442
returns                              8.682085
return_std                           3.187301
average_reward                       0.715632
round_time             0 days 00:10:24.546846
episodes_test                           795.0
episode_length_test                 12.561006
returns_test                         8.934348
return_std_test                      3.862544
average_reward_test                  0.711514
round_time_test        0 days 00:00:09.352122
round_time_total       0 days 00:10:24.547950
loss_total                 1192269568500826.0
loss_critic               1490336934357106.75
loss_actor                      -134334530.58
memory_size                            2033.0 

=== epoch 1/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:36,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  820
episode_length                      12.190244
returns                              8.735763
return_std                           3.316197
average_reward                       0.716531
round_time             0 days 00:10:26.850387
episodes_test                           800.0
episode_length_test                    12.495
returns_test                         8.680647
return_std_test                       2.95081
average_reward_test                   0.69483
round_time_test        0 days 00:00:09.261631
round_time_total       0 days 00:10:26.851620
loss_total                 1396146829422231.5
loss_critic                1745183505949655.0
loss_actor                     -145830078.436
memory_size                         2033.3125 

=== epoch 1/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:10,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  816
episode_length                      12.238971
returns                               8.71206
return_std                           3.087052
average_reward                       0.711849
round_time             0 days 00:10:27.723655
episodes_test                           805.0
episode_length_test                 12.416149
returns_test                         8.882294
return_std_test                       4.33642
average_reward_test                  0.715506
round_time_test        0 days 00:00:09.306750
round_time_total       0 days 00:10:27.724842
loss_total                1665674466245476.25
loss_critic                2082093049453740.0
loss_actor                     -157565106.224
memory_size                            2034.0 

=== epoch 1/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  803
episode_length                       12.43462
returns                               8.85859
return_std                           3.377821
average_reward                       0.712848
round_time             0 days 00:10:25.545907
episodes_test                           798.0
episode_length_test                 12.525063
returns_test                         8.962771
return_std_test                      4.182551
average_reward_test                  0.715726
round_time_test        0 days 00:00:09.182354
round_time_total       0 days 00:10:25.546997
loss_total                 1949743175283245.0
loss_critic                2437178930591957.0
loss_actor                     -170723005.664
memory_size                            2034.0 

=== epoch 1/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  801
episode_length                      12.458177
returns                              8.846546
return_std                           3.237809
average_reward                       0.709965
round_time             0 days 00:10:24.221190
episodes_test                           792.0
episode_length_test                 12.613636
returns_test                         9.065103
return_std_test                      4.628765
average_reward_test                  0.718728
round_time_test        0 days 00:00:09.429101
round_time_total       0 days 00:10:24.222288
loss_total                 2312188345007997.0
loss_critic                2890235377883283.5
loss_actor                     -184566986.144
memory_size                            2034.0 

=== epoch 1/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:57,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  800
episode_length                       12.49125
returns                              8.877948
return_std                           3.528772
average_reward                        0.71082
round_time             0 days 00:10:26.271233
episodes_test                           792.0
episode_length_test                 12.621212
returns_test                         9.121004
return_std_test                      4.026872
average_reward_test                  0.722774
round_time_test        0 days 00:00:09.304673
round_time_total       0 days 00:10:26.272581
loss_total                 2765193904553722.0
loss_critic                3456492318683562.0
loss_actor                     -199197276.728
memory_size                          2039.472 

=== epoch 1/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  800
episode_length                        12.4975
returns                              8.847494
return_std                           3.416062
average_reward                       0.708001
round_time             0 days 00:10:24.384423
episodes_test                           786.0
episode_length_test                 12.708651
returns_test                         9.291518
return_std_test                      4.454716
average_reward_test                  0.731235
round_time_test        0 days 00:00:09.373094
round_time_total       0 days 00:10:24.385501
loss_total                 3236650017376698.5
loss_critic                4045812455400538.0
loss_actor                     -213788715.848
memory_size                            2043.0 

=== epoch 1/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:14,  3.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  798
episode_length                      12.525063
returns                              8.862653
return_std                           3.191605
average_reward                       0.707687
round_time             0 days 00:10:26.747907
episodes_test                           788.0
episode_length_test                 12.677665
returns_test                         9.215039
return_std_test                      4.064512
average_reward_test                  0.726801
round_time_test        0 days 00:00:09.280087
round_time_total       0 days 00:10:26.749084
loss_total                 3741271605934817.5
loss_critic                4676589426250351.0
loss_actor                     -229752567.208
memory_size                            2043.0 

=== epoch 1/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:57,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  798
episode_length                      12.507519
returns                              8.838532
return_std                           3.116172
average_reward                        0.70656
round_time             0 days 00:10:23.651126
episodes_test                           791.0
episode_length_test                 12.642225
returns_test                         9.172133
return_std_test                      4.021309
average_reward_test                  0.725516
round_time_test        0 days 00:00:09.299338
round_time_total       0 days 00:10:23.652525
loss_total                 4315353122584133.5
loss_critic                5394191308807995.0
loss_actor                     -245784292.952
memory_size                            2043.0 

=== epoch 1/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  799
episode_length                      12.510638
returns                              8.887782
return_std                           3.263881
average_reward                        0.71044
round_time             0 days 00:10:24.789479
episodes_test                           783.0
episode_length_test                 12.768838
returns_test                         9.302541
return_std_test                      4.149276
average_reward_test                  0.728562
round_time_test        0 days 00:00:09.475757
round_time_total       0 days 00:10:24.790625
loss_total                 4920672202665230.0
loss_critic                6150840142434140.0
loss_actor                      -262951847.56
memory_size                          2044.553 

=== epoch 1/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:14,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  797
episode_length                       12.53074
returns                              8.924061
return_std                           3.032871
average_reward                        0.71221
round_time             0 days 00:10:27.232982
episodes_test                           796.0
episode_length_test                 12.562814
returns_test                         8.951603
return_std_test                      4.211748
average_reward_test                  0.712548
round_time_test        0 days 00:00:09.458183
round_time_total       0 days 00:10:27.234100
loss_total                 5611944481382203.0
loss_critic                7014930482273976.0
loss_actor                     -279775826.752
memory_size                            2047.0 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 2/10 ===== round 1/50 ======================================
  0%|          | 7/2000 [00:02<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:18<00:00,  3.23it/s]
episodes                                  150
episode_length                      13.293333
returns                              9.707252
return_std                           4.352624
average_reward                       0.730359
round_time             0 days 00:10:18.463542
episodes_test                           774.0
episode_length_test                 12.906977
returns_test                         9.237987
return_std_test                      3.385673
average_reward_test                  0.715667
round_time_test        0 days 00:00:09.263790
round_time_total       0 days 00:10:18.464680
loss_total                 6378742663239172.0
loss_critic                7973428201089139.0
loss_actor                      -297226675.44
memory_size                            2047.0 

=== epoch 2/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  302
episode_length                      13.225166
returns                              9.438056
return_std                             3.7182
average_reward                       0.714011
round_time             0 days 00:10:17.887788
episodes_test                           770.0
episode_length_test                 12.987013
returns_test                         9.311826
return_std_test                       3.08696
average_reward_test                  0.717011
round_time_test        0 days 00:00:09.264001
round_time_total       0 days 00:10:17.889050
loss_total                 7238773512532394.0
loss_critic                9048466739200786.0
loss_actor                     -315445746.688
memory_size                         2047.3285 

=== epoch 2/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:57,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:16<00:00,  3.24it/s]
episodes                                  453
episode_length                      13.231788
returns                               9.38895
return_std                           3.838881
average_reward                       0.709913
round_time             0 days 00:10:17.296976
episodes_test                           733.0
episode_length_test                 13.635744
returns_test                        10.097248
return_std_test                      4.868207
average_reward_test                   0.74062
round_time_test        0 days 00:00:09.224534
round_time_total       0 days 00:10:17.298093
loss_total                 8168099106951004.0
loss_critic               10210123706521354.0
loss_actor                      -334016580.96
memory_size                            2048.0 

=== epoch 2/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:18<00:00,  3.23it/s]
episodes                                  597
episode_length                      13.393635
returns                              9.731635
return_std                           4.793174
average_reward                       0.726699
round_time             0 days 00:10:19.164717
episodes_test                           741.0
episode_length_test                 13.477733
returns_test                         9.572823
return_std_test                      5.307375
average_reward_test                  0.710189
round_time_test        0 days 00:00:09.280855
round_time_total       0 days 00:10:19.166022
loss_total                 9608638140839362.0
loss_critic               12010797468984804.0
loss_actor                     -352871674.032
memory_size                          2054.639 

=== epoch 2/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:14,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:18<00:00,  3.23it/s]
episodes                                  746
episode_length                      13.395442
returns                              9.751672
return_std                            4.93265
average_reward                       0.728211
round_time             0 days 00:10:18.989550
episodes_test                           747.0
episode_length_test                 13.376171
returns_test                         9.735413
return_std_test                      4.372684
average_reward_test                  0.727999
round_time_test        0 days 00:00:09.218319
round_time_total       0 days 00:10:18.990647
loss_total                11099756507049230.0
loss_critic               13874695404634768.0
loss_actor                     -373639330.784
memory_size                          2058.633 

=== epoch 2/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:17<00:00,  3.24it/s]
episodes                                  739
episode_length                      13.511502
returns                              9.804128
return_std                           5.041559
average_reward                       0.725311
round_time             0 days 00:10:18.496510
episodes_test                           718.0
episode_length_test                 13.915042
returns_test                         9.608528
return_std_test                      4.443666
average_reward_test                  0.690901
round_time_test        0 days 00:00:09.488167
round_time_total       0 days 00:10:18.497609
loss_total                12619175028966032.0
loss_critic               15773968523006574.0
loss_actor                     -394393949.264
memory_size                          2065.786 

=== epoch 2/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  727
episode_length                      13.727648
returns                               9.99975
return_std                            5.40038
average_reward                       0.728658
round_time             0 days 00:10:19.604862
episodes_test                           744.0
episode_length_test                  13.43414
returns_test                         9.287154
return_std_test                      3.857055
average_reward_test                  0.691458
round_time_test        0 days 00:00:09.297350
round_time_total       0 days 00:10:19.606180
loss_total                14170874396230550.0
loss_critic               17713592700210512.0
loss_actor                     -412951920.272
memory_size                          2071.553 

=== epoch 2/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:33,  2.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  713
episode_length                      13.990182
returns                             10.077934
return_std                           5.415704
average_reward                       0.722546
round_time             0 days 00:10:19.926557
episodes_test                           696.0
episode_length_test                 14.362069
returns_test                         9.599609
return_std_test                      5.086145
average_reward_test                  0.668509
round_time_test        0 days 00:00:09.245574
round_time_total       0 days 00:10:19.927978
loss_total                15903516486073844.0
loss_critic               19879395247553248.0
loss_actor                     -430575170.704
memory_size                          2078.065 

=== epoch 2/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:42,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  702
episode_length                      14.220798
returns                              9.908508
return_std                            5.23157
average_reward                       0.696841
round_time             0 days 00:10:20.253219
episodes_test                           691.0
episode_length_test                 14.465991
returns_test                        10.014004
return_std_test                      6.152173
average_reward_test                  0.692343
round_time_test        0 days 00:00:09.283786
round_time_total       0 days 00:10:20.254338
loss_total                18186343427241672.0
loss_critic               22732928897907688.0
loss_actor                     -447998775.616
memory_size                          2105.376 

=== epoch 2/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  684
episode_length                      14.589181
returns                              9.899046
return_std                           5.197445
average_reward                       0.679095
round_time             0 days 00:10:21.464545
episodes_test                           689.0
episode_length_test                 14.510885
returns_test                         9.584062
return_std_test                      4.917691
average_reward_test                  0.660537
round_time_test        0 days 00:00:09.192767
round_time_total       0 days 00:10:21.465923
loss_total                20062264389665816.0
loss_critic               25077830048458736.0
loss_actor                     -463774279.888
memory_size                         2112.8365 

=== epoch 2/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  671
episode_length                      14.880775
returns                              9.980165
return_std                           5.725293
average_reward                       0.670574
round_time             0 days 00:10:23.007140
episodes_test                           659.0
episode_length_test                 15.159332
returns_test                         9.839682
return_std_test                         5.748
average_reward_test                  0.649136
round_time_test        0 days 00:00:09.320235
round_time_total       0 days 00:10:23.008268
loss_total                22106562478366260.0
loss_critic               27633202619001864.0
loss_actor                       -478838972.4
memory_size                         2147.7695 

=== epoch 2/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:20,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  656
episode_length                      15.233232
returns                               9.88132
return_std                           6.115001
average_reward                       0.648793
round_time             0 days 00:10:22.751449
episodes_test                           604.0
episode_length_test                 16.546358
returns_test                         9.462449
return_std_test                      7.171657
average_reward_test                  0.572038
round_time_test        0 days 00:00:09.301365
round_time_total       0 days 00:10:22.752760
loss_total                25085971659600428.0
loss_critic               31357464052729120.0
loss_actor                     -488730959.632
memory_size                         2199.0915 

=== epoch 2/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:11,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.22it/s]
episodes                                  651
episode_length                      15.351767
returns                              9.716955
return_std                            6.03159
average_reward                       0.632904
round_time             0 days 00:10:22.613743
episodes_test                           611.0
episode_length_test                 16.366612
returns_test                         9.543793
return_std_test                      7.136085
average_reward_test                  0.583126
round_time_test        0 days 00:00:09.472738
round_time_total       0 days 00:10:22.615192
loss_total                25174693783386720.0
loss_critic               31468366674779964.0
loss_actor                     -489202224.768
memory_size                         2219.5725 

=== epoch 2/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<12:15,  2.72it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  660
episode_length                      15.136364
returns                              9.474147
return_std                           5.859506
average_reward                       0.625444
round_time             0 days 00:10:22.144341
episodes_test                           668.0
episode_length_test                 14.947605
returns_test                         8.780253
return_std_test                      5.807791
average_reward_test                  0.587229
round_time_test        0 days 00:00:09.621028
round_time_total       0 days 00:10:22.145890
loss_total                25875063062279488.0
loss_critic               32343828288428312.0
loss_actor                     -491449153.296
memory_size                         2222.4345 

=== epoch 2/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:34,  2.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  670
episode_length                      14.886567
returns                               9.13119
return_std                           5.820572
average_reward                       0.612171
round_time             0 days 00:10:22.223452
episodes_test                           669.0
episode_length_test                 14.920777
returns_test                         8.669679
return_std_test                      6.205262
average_reward_test                  0.580999
round_time_test        0 days 00:00:09.614244
round_time_total       0 days 00:10:22.224553
loss_total                26742632694996272.0
loss_critic               33428290300333260.0
loss_actor                      -497880856.72
memory_size                         2229.7565 

=== epoch 2/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:09,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  682
episode_length                      14.642229
returns                              8.772715
return_std                           5.483982
average_reward                       0.599208
round_time             0 days 00:10:22.438094
episodes_test                           717.0
episode_length_test                 13.942817
returns_test                         8.494024
return_std_test                      5.402197
average_reward_test                  0.609318
round_time_test        0 days 00:00:09.111518
round_time_total       0 days 00:10:22.439196
loss_total                27727439955121668.0
loss_critic               34659299358578572.0
loss_actor                     -507130817.744
memory_size                           2242.96 

=== epoch 2/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  701
episode_length                      14.248217
returns                              8.509161
return_std                           5.094886
average_reward                       0.596847
round_time             0 days 00:10:21.306086
episodes_test                           712.0
episode_length_test                 14.032303
returns_test                         8.635836
return_std_test                      5.946263
average_reward_test                  0.615536
round_time_test        0 days 00:00:09.565824
round_time_total       0 days 00:10:21.307627
loss_total                28800686735285552.0
loss_critic               36000857803584440.0
loss_actor                     -517386344.768
memory_size                         2256.2305 

=== epoch 2/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:11,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  724
episode_length                       13.79558
returns                              8.457809
return_std                           5.613583
average_reward                        0.61325
round_time             0 days 00:10:22.662005
episodes_test                           767.0
episode_length_test                 13.022164
returns_test                         8.316533
return_std_test                      4.462046
average_reward_test                  0.638637
round_time_test        0 days 00:00:09.340382
round_time_total       0 days 00:10:22.663252
loss_total                31468477211132036.0
loss_critic               39335595838665656.0
loss_actor                     -533880100.272
memory_size                         2297.0465 

=== epoch 2/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:29,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  735
episode_length                      13.582313
returns                               8.48183
return_std                           5.378732
average_reward                       0.624444
round_time             0 days 00:10:23.615661
episodes_test                           756.0
episode_length_test                 13.224868
returns_test                         8.446192
return_std_test                      5.448893
average_reward_test                  0.638716
round_time_test        0 days 00:00:09.452155
round_time_total       0 days 00:10:23.616758
loss_total                32865429143213308.0
loss_critic               41081785751082896.0
loss_actor                     -547686025.824
memory_size                         2311.1825 

=== epoch 2/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  753
episode_length                      13.262948
returns                              8.515951
return_std                           5.020148
average_reward                       0.641396
round_time             0 days 00:10:20.548616
episodes_test                           770.0
episode_length_test                 12.983117
returns_test                         8.589401
return_std_test                      4.900431
average_reward_test                  0.661668
round_time_test        0 days 00:00:09.210500
round_time_total       0 days 00:10:20.549689
loss_total                34711501189958076.0
loss_critic               43389375711937560.0
loss_actor                     -562521083.856
memory_size                         2320.3535 

=== epoch 2/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  767
episode_length                      13.022164
returns                              8.483436
return_std                           4.899286
average_reward                       0.651513
round_time             0 days 00:10:24.041912
episodes_test                           786.0
episode_length_test                 12.721374
returns_test                         8.474528
return_std_test                      4.604051
average_reward_test                  0.666192
round_time_test        0 days 00:00:09.254381
round_time_total       0 days 00:10:24.043022
loss_total                37822532119489936.0
loss_critic               47278164327949928.0
loss_actor                     -580998933.424
memory_size                          2346.332 

=== epoch 2/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:13,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  792
episode_length                      12.617424
returns                                8.3224
return_std                            4.49546
average_reward                       0.659576
round_time             0 days 00:10:23.151021
episodes_test                           785.0
episode_length_test                 12.735032
returns_test                         7.733568
return_std_test                       4.23238
average_reward_test                  0.607369
round_time_test        0 days 00:00:09.081334
round_time_total       0 days 00:10:23.152111
loss_total                39289346285960688.0
loss_critic               49111681989330600.0
loss_actor                     -599869744.848
memory_size                            2360.0 

=== epoch 2/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  822
episode_length                      12.143552
returns                              8.128819
return_std                           3.536712
average_reward                       0.669839
round_time             0 days 00:10:21.495607
episodes_test                           889.0
episode_length_test                 11.235096
returns_test                         8.196093
return_std_test                      3.554837
average_reward_test                  0.729675
round_time_test        0 days 00:00:09.304181
round_time_total       0 days 00:10:21.496704
loss_total                42174050301762336.0
loss_critic               52717561968548904.0
loss_actor                     -630311463.392
memory_size                            2360.0 

=== epoch 2/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:52,  2.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  857
episode_length                      11.656943
returns                              8.055837
return_std                           3.608119
average_reward                       0.690859
round_time             0 days 00:10:23.150833
episodes_test                           900.0
episode_length_test                 11.101111
returns_test                          8.36113
return_std_test                      4.202577
average_reward_test                  0.753382
round_time_test        0 days 00:00:09.678927
round_time_total       0 days 00:10:23.151917
loss_total                46130612534264200.0
loss_critic               57663264637574968.0
loss_actor                     -665732248.512
memory_size                            2360.0 

=== epoch 2/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:13,  3.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  880
episode_length                      11.345455
returns                              7.989279
return_std                           3.505272
average_reward                       0.702704
round_time             0 days 00:10:23.001164
episodes_test                           907.0
episode_length_test                 11.017641
returns_test                         8.226776
return_std_test                      3.404373
average_reward_test                   0.74681
round_time_test        0 days 00:00:09.451989
round_time_total       0 days 00:10:23.002268
loss_total                50908109474931672.0
loss_critic               63635135713819752.0
loss_actor                     -705420519.968
memory_size                            2360.0 

=== epoch 2/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  905
episode_length                      11.036464
returns                              7.991042
return_std                           3.281648
average_reward                       0.723893
round_time             0 days 00:10:23.393575
episodes_test                           891.0
episode_length_test                 11.215488
returns_test                         8.347718
return_std_test                      3.069468
average_reward_test                  0.744401
round_time_test        0 days 00:00:09.595212
round_time_total       0 days 00:10:23.394655
loss_total                56492880949141304.0
loss_critic               70616099957797552.0
loss_actor                     -739832880.928
memory_size                          2365.467 

=== epoch 2/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  918
episode_length                      10.875817
returns                              8.189123
return_std                           3.323116
average_reward                       0.752897
round_time             0 days 00:10:24.199038
episodes_test                           921.0
episode_length_test                 10.850163
returns_test                          8.34393
return_std_test                      3.752058
average_reward_test                  0.769064
round_time_test        0 days 00:00:09.500491
round_time_total       0 days 00:10:24.200155
loss_total                61656610711609016.0
loss_critic               77070762076861888.0
loss_actor                     -776791788.864
memory_size                            2367.0 

=== epoch 2/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  909
episode_length                        10.9978
returns                              8.460102
return_std                           3.504734
average_reward                        0.76925
round_time             0 days 00:10:24.947845
episodes_test                           884.0
episode_length_test                 11.294118
returns_test                         9.121308
return_std_test                      4.553813
average_reward_test                   0.80789
round_time_test        0 days 00:00:09.350541
round_time_total       0 days 00:10:24.948916
loss_total                67899464369913728.0
loss_critic               84874328944121216.0
loss_actor                     -814271188.576
memory_size                            2367.0 

=== epoch 2/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  896
episode_length                       11.15625
returns                              8.794819
return_std                           3.539789
average_reward                       0.788363
round_time             0 days 00:10:22.878546
episodes_test                           862.0
episode_length_test                 11.584687
returns_test                         9.969089
return_std_test                      4.445196
average_reward_test                  0.860427
round_time_test        0 days 00:00:09.383185
round_time_total       0 days 00:10:22.879663
loss_total                74940616450552896.0
loss_critic               93675768969758240.0
loss_actor                      -857374842.24
memory_size                            2367.0 

=== epoch 2/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:32,  2.65it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  888
episode_length                          11.25
returns                              9.153061
return_std                           3.912961
average_reward                       0.813817
round_time             0 days 00:10:22.787532
episodes_test                           868.0
episode_length_test                 11.517281
returns_test                         9.977978
return_std_test                      5.341033
average_reward_test                  0.866377
round_time_test        0 days 00:00:09.360206
round_time_total       0 days 00:10:22.788925
loss_total                82665655439294720.0
loss_critic              103332067392152928.0
loss_actor                      -899006682.56
memory_size                          2368.654 

=== epoch 2/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:46,  2.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  881
episode_length                      11.347333
returns                              9.407961
return_std                           3.786097
average_reward                       0.829025
round_time             0 days 00:10:24.284260
episodes_test                           872.0
episode_length_test                 11.463303
returns_test                           9.6603
return_std_test                      3.758209
average_reward_test                  0.842801
round_time_test        0 days 00:00:09.490327
round_time_total       0 days 00:10:24.285371
loss_total                90892150584828224.0
loss_critic              113615186230922704.0
loss_actor                     -934989596.288
memory_size                            2375.0 

=== epoch 2/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:50,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  864
episode_length                      11.559028
returns                              9.937564
return_std                           4.755721
average_reward                       0.859732
round_time             0 days 00:10:22.132928
episodes_test                           885.0
episode_length_test                 11.298305
returns_test                         9.864293
return_std_test                      4.419909
average_reward_test                  0.873087
round_time_test        0 days 00:00:09.298023
round_time_total       0 days 00:10:22.134006
loss_total                99740776510601040.0
loss_critic              124675968512779360.0
loss_actor                     -976936478.944
memory_size                         2383.4775 

=== epoch 2/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  861
episode_length                      11.603949
returns                             10.172967
return_std                           5.119945
average_reward                       0.876616
round_time             0 days 00:10:25.363054
episodes_test                           888.0
episode_length_test                 11.256757
returns_test                         9.689172
return_std_test                      3.986223
average_reward_test                  0.860806
round_time_test        0 days 00:00:09.237098
round_time_total       0 days 00:10:25.364189
loss_total               113545523798411840.0
loss_critic              141931902628448432.0
loss_actor                     -1016057582.24
memory_size                          2412.271 

=== epoch 2/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  860
episode_length                      11.617442
returns                             10.239774
return_std                           5.747326
average_reward                       0.881156
round_time             0 days 00:10:25.594851
episodes_test                           888.0
episode_length_test                 11.246622
returns_test                         9.546062
return_std_test                      3.579255
average_reward_test                  0.848862
round_time_test        0 days 00:00:09.399637
round_time_total       0 days 00:10:25.596141
loss_total               124849083762419440.0
loss_critic              156061352040144576.0
loss_actor                    -1054467238.272
memory_size                          2433.127 

=== epoch 2/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:00,  2.77it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  858
episode_length                       11.65035
returns                             10.291308
return_std                           5.579791
average_reward                       0.883379
round_time             0 days 00:10:23.794676
episodes_test                           848.0
episode_length_test                 11.785377
returns_test                        10.240692
return_std_test                      4.610141
average_reward_test                  0.868993
round_time_test        0 days 00:00:09.381406
round_time_total       0 days 00:10:23.795764
loss_total               130764237965864464.0
loss_critic              163455294686539808.0
loss_actor                     -1091226720.96
memory_size                            2445.0 

=== epoch 2/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:02,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  852
episode_length                      11.719484
returns                             10.368486
return_std                           5.607067
average_reward                        0.88476
round_time             0 days 00:10:25.170246
episodes_test                           846.0
episode_length_test                 11.816785
returns_test                        10.476243
return_std_test                       5.04989
average_reward_test                  0.886561
round_time_test        0 days 00:00:09.453593
round_time_total       0 days 00:10:25.171323
loss_total               140702460579894256.0
loss_critic              175878072720001344.0
loss_actor                    -1125794047.968
memory_size                            2445.0 

=== epoch 2/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:17,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  851
episode_length                       11.73443
returns                              10.24786
return_std                           5.331751
average_reward                       0.873666
round_time             0 days 00:10:23.266761
episodes_test                           848.0
episode_length_test                 11.787736
returns_test                         10.51411
return_std_test                      5.386035
average_reward_test                  0.891963
round_time_test        0 days 00:00:09.441261
round_time_total       0 days 00:10:23.268066
loss_total               149847094325636224.0
loss_critic              187308864626227136.0
loss_actor                    -1162204435.904
memory_size                         2453.2745 

=== epoch 2/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:58,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  846
episode_length                      11.795508
returns                             10.260334
return_std                           5.213845
average_reward                       0.871066
round_time             0 days 00:10:27.259099
episodes_test                           829.0
episode_length_test                 12.044632
returns_test                        10.653297
return_std_test                      5.276577
average_reward_test                  0.885181
round_time_test        0 days 00:00:09.377742
round_time_total       0 days 00:10:27.260448
loss_total               163105998858363072.0
loss_critic              203882495010278464.0
loss_actor                    -1203114337.728
memory_size                         2463.9235 

=== epoch 2/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:16,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  840
episode_length                      11.892857
returns                             10.307119
return_std                           4.734432
average_reward                       0.866595
round_time             0 days 00:10:26.925879
episodes_test                           833.0
episode_length_test                 12.002401
returns_test                        10.232269
return_std_test                      4.971694
average_reward_test                  0.852544
round_time_test        0 days 00:00:09.177021
round_time_total       0 days 00:10:26.927419
loss_total               175376047363005088.0
loss_critic              219220055345801984.0
loss_actor                    -1242703724.864
memory_size                            2469.0 

=== epoch 2/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:47,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  831
episode_length                       12.01444
returns                             10.396933
return_std                           4.977718
average_reward                       0.865259
round_time             0 days 00:10:26.673593
episodes_test                           840.0
episode_length_test                  11.89881
returns_test                        10.147486
return_std_test                      5.024107
average_reward_test                  0.852872
round_time_test        0 days 00:00:09.316169
round_time_total       0 days 00:10:26.674728
loss_total               187313134878133856.0
loss_critic              234141414667235360.0
loss_actor                    -1288362090.048
memory_size                            2469.0 

=== epoch 2/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:01,  2.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  826
episode_length                      12.084746
returns                             10.471699
return_std                           5.137544
average_reward                       0.866336
round_time             0 days 00:10:24.568539
episodes_test                           817.0
episode_length_test                 12.233782
returns_test                        10.237095
return_std_test                      4.982564
average_reward_test                  0.836843
round_time_test        0 days 00:00:09.410832
round_time_total       0 days 00:10:24.569683
loss_total               203131383574706528.0
loss_critic              253914225108991360.0
loss_actor                    -1336428744.448
memory_size                            2469.0 

=== epoch 2/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  831
episode_length                       12.01444
returns                             10.241936
return_std                           4.575461
average_reward                       0.852509
round_time             0 days 00:10:26.481080
episodes_test                           823.0
episode_length_test                 12.147023
returns_test                        10.357026
return_std_test                      5.922678
average_reward_test                  0.852646
round_time_test        0 days 00:00:09.190500
round_time_total       0 days 00:10:26.482168
loss_total               217827257989273216.0
loss_critic              272284067703071712.0
loss_actor                     -1383595625.92
memory_size                            2469.0 

=== epoch 2/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:39,  2.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  826
episode_length                      12.089588
returns                             10.099497
return_std                            4.54849
average_reward                       0.835928
round_time             0 days 00:10:25.268337
episodes_test                           839.0
episode_length_test                 11.914184
returns_test                         9.636481
return_std_test                       3.74026
average_reward_test                  0.808884
round_time_test        0 days 00:00:09.332945
round_time_total       0 days 00:10:25.269429
loss_total               235291885502157824.0
loss_critic              294114851938484864.0
loss_actor                    -1440252086.848
memory_size                         2472.5245 

=== epoch 2/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.19it/s]
episodes                                  823
episode_length                      12.132442
returns                              9.928739
return_std                           4.412238
average_reward                       0.818537
round_time             0 days 00:10:26.528308
episodes_test                           832.0
episode_length_test                 12.009615
returns_test                         9.754212
return_std_test                      4.861406
average_reward_test                  0.812183
round_time_test        0 days 00:00:09.213031
round_time_total       0 days 00:10:26.529599
loss_total               254097913498667328.0
loss_critic              317622386681792448.0
loss_actor                    -1492345506.112
memory_size                            2476.0 

=== epoch 2/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:05,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  827
episode_length                      12.085852
returns                              9.744238
return_std                           4.332505
average_reward                       0.806213
round_time             0 days 00:10:24.301400
episodes_test                           808.0
episode_length_test                 12.361386
returns_test                         9.767857
return_std_test                      5.094715
average_reward_test                  0.790678
round_time_test        0 days 00:00:09.574031
round_time_total       0 days 00:10:24.302474
loss_total               273919240586783616.0
loss_critic              342399044794613504.0
loss_actor                      -1545605979.2
memory_size                            2476.0 

=== epoch 2/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  826
episode_length                      12.098063
returns                              9.476756
return_std                           4.094245
average_reward                        0.78334
round_time             0 days 00:10:26.404721
episodes_test                           835.0
episode_length_test                 11.968862
returns_test                         9.209385
return_std_test                      4.010872
average_reward_test                  0.769626
round_time_test        0 days 00:00:09.124873
round_time_total       0 days 00:10:26.405968
loss_total               295955263858820544.0
loss_critic              369944073092238144.0
loss_actor                    -1609507622.272
memory_size                            2476.0 

=== epoch 2/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:34,  2.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  821
episode_length                       12.16687
returns                              9.428131
return_std                           4.466822
average_reward                       0.775032
round_time             0 days 00:10:27.054962
episodes_test                           817.0
episode_length_test                 12.239902
returns_test                         9.336953
return_std_test                      4.735802
average_reward_test                  0.762829
round_time_test        0 days 00:00:09.455616
round_time_total       0 days 00:10:27.056073
loss_total               326484858994790784.0
loss_critic              408106066533312128.0
loss_actor                    -1662870637.568
memory_size                           2482.58 

=== epoch 2/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:19,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  823
episode_length                      12.144593
returns                              9.375237
return_std                            4.60248
average_reward                       0.771898
round_time             0 days 00:10:27.390431
episodes_test                           811.0
episode_length_test                 12.316893
returns_test                          9.47384
return_std_test                      3.562237
average_reward_test                  0.769264
round_time_test        0 days 00:00:09.196409
round_time_total       0 days 00:10:27.391521
loss_total               352118651636246656.0
loss_critic              440148306639347264.0
loss_actor                    -1726082803.648
memory_size                         2485.8275 

=== epoch 2/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:44,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  824
episode_length                      12.123786
returns                              9.244879
return_std                           4.471645
average_reward                       0.762876
round_time             0 days 00:10:25.594164
episodes_test                           832.0
episode_length_test                 12.018029
returns_test                         9.002209
return_std_test                       3.84558
average_reward_test                  0.749085
round_time_test        0 days 00:00:09.489559
round_time_total       0 days 00:10:25.595689
loss_total               383824453162846144.0
loss_critic              479780558525048000.0
loss_actor                     -1781199136.64
memory_size                            2496.0 

=== epoch 2/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  815
episode_length                      12.268712
returns                              9.324885
return_std                           5.038448
average_reward                       0.760075
round_time             0 days 00:10:28.319179
episodes_test                           819.0
episode_length_test                 12.206349
returns_test                         9.272382
return_std_test                      3.956589
average_reward_test                  0.759683
round_time_test        0 days 00:00:09.306435
round_time_total       0 days 00:10:28.320487
loss_total               406853765558559552.0
loss_critic              508567198319610176.0
loss_actor                    -1843480133.504
memory_size                          2506.077 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 3/10 ===== round 1/50 ======================================
  0%|          | 7/2000 [00:02<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  157
episode_length                      12.719745
returns                              9.824809
return_std                            4.27928
average_reward                       0.772581
round_time             0 days 00:10:27.787130
episodes_test                           802.0
episode_length_test                 12.466334
returns_test                         9.411719
return_std_test                       4.18598
average_reward_test                  0.754992
round_time_test        0 days 00:00:09.298125
round_time_total       0 days 00:10:27.788260
loss_total               444787471429037696.0
loss_critic              555984330363502592.0
loss_actor                    -1904033706.496
memory_size                            2532.0 

=== epoch 3/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:32,  2.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:44<00:00,  3.11it/s]
episodes                                  312
episode_length                      12.814103
returns                              9.757237
return_std                           4.134225
average_reward                       0.761522
round_time             0 days 00:10:44.633403
episodes_test                           788.0
episode_length_test                 12.676396
returns_test                         9.658387
return_std_test                      3.939433
average_reward_test                  0.762034
round_time_test        0 days 00:00:09.433179
round_time_total       0 days 00:10:44.634546
loss_total               464812815399358848.0
loss_critic              581016009701486208.0
loss_actor                     -1960696365.12
memory_size                         2538.3155 

=== epoch 3/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  464
episode_length                      12.924569
returns                              9.920878
return_std                           5.423569
average_reward                        0.76773
round_time             0 days 00:10:23.287677
episodes_test                           783.0
episode_length_test                 12.767561
returns_test                         9.926067
return_std_test                      5.291428
average_reward_test                  0.777516
round_time_test        0 days 00:00:09.314683
round_time_total       0 days 00:10:23.288776
loss_total               498517960911331776.0
loss_critic              623147440358796800.0
loss_actor                     -2018883503.68
memory_size                         2568.4475 

=== epoch 3/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:58,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  620
episode_length                      12.891935
returns                              9.755557
return_std                           4.918455
average_reward                       0.756764
round_time             0 days 00:10:22.275032
episodes_test                           790.0
episode_length_test                 12.650633
returns_test                         9.512257
return_std_test                      6.023358
average_reward_test                   0.75209
round_time_test        0 days 00:00:09.243854
round_time_total       0 days 00:10:22.276184
loss_total               523427964848021952.0
loss_critic              654284945200202624.0
loss_actor                    -2076707441.792
memory_size                            2586.0 

=== epoch 3/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  782
episode_length                      12.778772
returns                              9.606936
return_std                           4.588573
average_reward                       0.751781
round_time             0 days 00:10:21.440214
episodes_test                           788.0
episode_length_test                 12.678934
returns_test                         9.454406
return_std_test                      3.449107
average_reward_test                  0.745677
round_time_test        0 days 00:00:09.306918
round_time_total       0 days 00:10:21.441615
loss_total               554870580051556928.0
loss_critic              693588213165239296.0
loss_actor                     -2132021376.96
memory_size                            2586.0 

=== epoch 3/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  786
episode_length                      12.711196
returns                              9.450466
return_std                           4.440037
average_reward                       0.743304
round_time             0 days 00:10:22.225844
episodes_test                           813.0
episode_length_test                 12.291513
returns_test                         8.996268
return_std_test                       2.86117
average_reward_test                  0.731997
round_time_test        0 days 00:00:09.372649
round_time_total       0 days 00:10:22.227011
loss_total               592061218955913600.0
loss_critic              740076510668256128.0
loss_actor                    -2199876180.096
memory_size                            2586.0 

=== epoch 3/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:47,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  788
episode_length                      12.664975
returns                              9.418741
return_std                           4.570812
average_reward                        0.74361
round_time             0 days 00:10:20.907902
episodes_test                           797.0
episode_length_test                 12.537014
returns_test                         9.260115
return_std_test                      4.060889
average_reward_test                  0.738914
round_time_test        0 days 00:00:09.370866
round_time_total       0 days 00:10:20.909305
loss_total               636542774551303424.0
loss_critic              795678454453823872.0
loss_actor                    -2268760173.824
memory_size                         2587.2375 

=== epoch 3/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  798
episode_length                      12.512531
returns                              9.175388
return_std                            3.56947
average_reward                       0.733256
round_time             0 days 00:10:21.812260
episodes_test                           801.0
episode_length_test                 12.480649
returns_test                          9.23277
return_std_test                      3.767651
average_reward_test                  0.739829
round_time_test        0 days 00:00:09.315428
round_time_total       0 days 00:10:21.813357
loss_total               680900207739322496.0
loss_critic              851125244951005184.0
loss_actor                    -2347517825.408
memory_size                            2596.0 

=== epoch 3/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  806
episode_length                      12.389578
returns                              9.147183
return_std                           3.468271
average_reward                       0.738201
round_time             0 days 00:10:21.452724
episodes_test                           801.0
episode_length_test                 12.475655
returns_test                         9.178375
return_std_test                       3.63375
average_reward_test                  0.735789
round_time_test        0 days 00:00:09.520626
round_time_total       0 days 00:10:21.453894
loss_total               726322185396704768.0
loss_critic              907902715753570176.0
loss_actor                     -2421428434.24
memory_size                            2596.0 

=== epoch 3/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.22it/s]
episodes                                  810
episode_length                      12.335802
returns                              9.048704
return_std                           3.507511
average_reward                       0.733295
round_time             0 days 00:10:22.566159
episodes_test                           818.0
episode_length_test                 12.222494
returns_test                         9.156858
return_std_test                      4.632756
average_reward_test                  0.749227
round_time_test        0 days 00:00:09.469611
round_time_total       0 days 00:10:22.567252
loss_total               789130853020971520.0
loss_critic              986413548662553600.0
loss_actor                     -2500419575.68
memory_size                            2596.0 

=== epoch 3/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  810
episode_length                      12.320988
returns                              8.962222
return_std                           3.312593
average_reward                       0.727305
round_time             0 days 00:10:21.703084
episodes_test                           813.0
episode_length_test                 12.291513
returns_test                         8.694403
return_std_test                      2.492814
average_reward_test                  0.707482
round_time_test        0 days 00:00:09.311966
round_time_total       0 days 00:10:21.704670
loss_total               857277289671241472.0
loss_critic             1071596593646462336.0
loss_actor                    -2588769089.792
memory_size                            2596.0 

=== epoch 3/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:17,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  815
episode_length                      12.263804
returns                              8.825738
return_std                           3.273409
average_reward                       0.719565
round_time             0 days 00:10:20.179408
episodes_test                           810.0
episode_length_test                 12.338272
returns_test                         8.809374
return_std_test                      3.681387
average_reward_test                  0.714083
round_time_test        0 days 00:00:09.172375
round_time_total       0 days 00:10:20.180491
loss_total               922311451611326592.0
loss_critic             1152889295272704768.0
loss_actor                    -2676569775.872
memory_size                         2597.4535 

=== epoch 3/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:13,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  814
episode_length                      12.262899
returns                              8.826303
return_std                           4.398262
average_reward                       0.719785
round_time             0 days 00:10:21.406091
episodes_test                           803.0
episode_length_test                 12.447073
returns_test                         8.973886
return_std_test                      3.660377
average_reward_test                  0.721112
round_time_test        0 days 00:00:09.336428
round_time_total       0 days 00:10:21.407173
loss_total               991232633446204416.0
loss_critic             1239040771475380480.0
loss_actor                    -2766667496.704
memory_size                         2630.9325 

=== epoch 3/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:08,  2.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  809
episode_length                      12.349815
returns                               8.88683
return_std                           4.898802
average_reward                       0.719566
round_time             0 days 00:10:23.624584
episodes_test                           804.0
episode_length_test                  12.43408
returns_test                         8.891232
return_std_test                      3.980519
average_reward_test                  0.715139
round_time_test        0 days 00:00:09.320658
round_time_total       0 days 00:10:23.625697
loss_total              1031111527213312256.0
loss_critic             1288889388521056256.0
loss_actor                    -2857119109.376
memory_size                            2643.0 

=== epoch 3/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:06,  2.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  805
episode_length                      12.409938
returns                              8.945696
return_std                           4.823227
average_reward                       0.720702
round_time             0 days 00:10:20.038043
episodes_test                           811.0
episode_length_test                 12.319359
returns_test                         8.686834
return_std_test                      3.604573
average_reward_test                  0.705163
round_time_test        0 days 00:00:09.411939
round_time_total       0 days 00:10:20.039136
loss_total              1080112279833146240.0
loss_critic             1350140326078918400.0
loss_actor                    -2927636315.392
memory_size                            2643.0 

=== epoch 3/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:08,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  804
episode_length                      12.421642
returns                              8.978556
return_std                           4.994933
average_reward                       0.722714
round_time             0 days 00:10:23.857907
episodes_test                           797.0
episode_length_test                 12.544542
returns_test                         9.096692
return_std_test                      3.978355
average_reward_test                  0.725201
round_time_test        0 days 00:00:09.282882
round_time_total       0 days 00:10:23.859166
loss_total              1143349669812999552.0
loss_critic             1429187063463540736.0
loss_actor                    -3000655861.632
memory_size                         2643.1445 

=== epoch 3/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  805
episode_length                      12.391304
returns                              8.924808
return_std                           4.798179
average_reward                       0.719973
round_time             0 days 00:10:24.731069
episodes_test                           809.0
episode_length_test                 12.352287
returns_test                         8.704968
return_std_test                      3.485098
average_reward_test                  0.704803
round_time_test        0 days 00:00:09.323636
round_time_total       0 days 00:10:24.732177
loss_total              1200395473679544320.0
loss_critic             1500494314929467136.0
loss_actor                    -3069259727.232
memory_size                            2644.0 

=== epoch 3/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  809
episode_length                      12.349815
returns                              8.874182
return_std                           3.725237
average_reward                       0.718528
round_time             0 days 00:10:24.880805
episodes_test                           808.0
episode_length_test                 12.376238
returns_test                         8.806096
return_std_test                      3.758667
average_reward_test                  0.711533
round_time_test        0 days 00:00:09.387158
round_time_total       0 days 00:10:24.881912
loss_total              1244526879746845184.0
loss_critic             1555658573063349248.0
loss_actor                    -3146777673.728
memory_size                            2644.0 

=== epoch 3/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:58,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  815
episode_length                      12.256442
returns                              8.689839
return_std                           3.106217
average_reward                       0.708904
round_time             0 days 00:10:24.371552
episodes_test                           815.0
episode_length_test                 12.255215
returns_test                         8.701236
return_std_test                      3.089331
average_reward_test                  0.710098
round_time_test        0 days 00:00:09.323240
round_time_total       0 days 00:10:24.372833
loss_total              1316065571229233408.0
loss_critic             1645081935182951424.0
loss_actor                     -3220741746.56
memory_size                         2644.3975 

=== epoch 3/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:15,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  821
episode_length                      12.174178
returns                              8.597434
return_std                           3.062942
average_reward                       0.706227
round_time             0 days 00:10:23.093506
episodes_test                           817.0
episode_length_test                 12.227662
returns_test                         8.620255
return_std_test                      4.009654
average_reward_test                  0.705031
round_time_test        0 days 00:00:09.230206
round_time_total       0 days 00:10:23.094715
loss_total              1366443621901280512.0
loss_critic             1708054497633952000.0
loss_actor                    -3293981332.224
memory_size                            2647.0 

=== epoch 3/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  826
episode_length                       12.09322
returns                              8.489332
return_std                           2.778096
average_reward                       0.701802
round_time             0 days 00:10:22.204989
episodes_test                           824.0
episode_length_test                 12.127427
returns_test                         8.498831
return_std_test                      2.996061
average_reward_test                  0.701014
round_time_test        0 days 00:00:09.303282
round_time_total       0 days 00:10:22.206134
loss_total              1448549893309521152.0
loss_critic             1810687335687367168.0
loss_actor                    -3376528523.008
memory_size                            2647.0 

=== epoch 3/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  823
episode_length                      12.137303
returns                              8.519221
return_std                           2.588203
average_reward                       0.701785
round_time             0 days 00:10:25.125195
episodes_test                           811.0
episode_length_test                 12.321825
returns_test                          8.80816
return_std_test                      5.327759
average_reward_test                  0.715045
round_time_test        0 days 00:00:09.413624
round_time_total       0 days 00:10:25.126295
loss_total              1510607997128020992.0
loss_critic             1888259964738937600.0
loss_actor                    -3451584840.064
memory_size                           2648.61 

=== epoch 3/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<13:23,  2.48it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  817
episode_length                      12.226438
returns                              8.606935
return_std                           3.275793
average_reward                       0.703873
round_time             0 days 00:10:26.437700
episodes_test                           792.0
episode_length_test                 12.612374
returns_test                         9.016359
return_std_test                      4.600328
average_reward_test                  0.714893
round_time_test        0 days 00:00:09.274604
round_time_total       0 days 00:10:26.439032
loss_total              1592867804563302656.0
loss_critic             1991084723921370368.0
loss_actor                    -3547839730.816
memory_size                         2667.1765 

=== epoch 3/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:18,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  812
episode_length                      12.295567
returns                              8.695941
return_std                           3.578374
average_reward                       0.707236
round_time             0 days 00:10:22.070992
episodes_test                           797.0
episode_length_test                 12.542033
returns_test                         8.879953
return_std_test                      4.265203
average_reward_test                  0.708152
round_time_test        0 days 00:00:09.327286
round_time_total       0 days 00:10:22.072254
loss_total              1672710277410802688.0
loss_critic             2090887813048009984.0
loss_actor                    -3629916003.712
memory_size                         2669.1155 

=== epoch 3/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:24,  2.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  807
episode_length                      12.377943
returns                              8.727225
return_std                           3.667137
average_reward                       0.704913
round_time             0 days 00:10:22.841834
episodes_test                           806.0
episode_length_test                 12.404467
returns_test                         8.671825
return_std_test                      3.207444
average_reward_test                   0.69915
round_time_test        0 days 00:00:09.637431
round_time_total       0 days 00:10:22.842944
loss_total              1783013873762919424.0
loss_critic             2228767304579735808.0
loss_actor                     -3715066197.12
memory_size                            2676.0 

=== epoch 3/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  804
episode_length                      12.415423
returns                              8.760623
return_std                           3.861183
average_reward                       0.705534
round_time             0 days 00:10:22.879462
episodes_test                           803.0
episode_length_test                 12.448319
returns_test                         8.742065
return_std_test                      2.809309
average_reward_test                  0.702381
round_time_test        0 days 00:00:09.525160
round_time_total       0 days 00:10:22.880564
loss_total              1858921917870180864.0
loss_critic             2323652359456114688.0
loss_actor                    -3815021935.616
memory_size                         2677.7125 

=== epoch 3/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:04,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  806
episode_length                      12.400744
returns                              8.754403
return_std                           4.261071
average_reward                       0.705827
round_time             0 days 00:10:22.094559
episodes_test                           826.0
episode_length_test                 12.095642
returns_test                          8.17985
return_std_test                      2.300527
average_reward_test                  0.676358
round_time_test        0 days 00:00:09.436362
round_time_total       0 days 00:10:22.095834
loss_total              1981781569671536896.0
loss_critic             2477226919981561856.0
loss_actor                     -3904939403.52
memory_size                          2699.251 

=== epoch 3/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:14,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  811
episode_length                      12.316893
returns                              8.568749
return_std                            3.61845
average_reward                       0.695405
round_time             0 days 00:10:23.873441
episodes_test                           818.0
episode_length_test                 12.223716
returns_test                          8.49283
return_std_test                      3.491904
average_reward_test                  0.694807
round_time_test        0 days 00:00:09.445817
round_time_total       0 days 00:10:23.874524
loss_total              2093386349112123648.0
loss_critic             2616732894222165504.0
loss_actor                    -3995932972.672
memory_size                            2703.0 

=== epoch 3/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:39,  2.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  813
episode_length                      12.289053
returns                               8.40361
return_std                           3.466756
average_reward                       0.683899
round_time             0 days 00:10:23.806069
episodes_test                           812.0
episode_length_test                 12.307882
returns_test                         8.614878
return_std_test                      4.070732
average_reward_test                  0.700107
round_time_test        0 days 00:00:09.426305
round_time_total       0 days 00:10:23.807354
loss_total              2196077731521153280.0
loss_critic             2745097118702989312.0
loss_actor                    -4085215102.848
memory_size                          2704.106 

=== epoch 3/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:21,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  815
episode_length                      12.250307
returns                              8.342703
return_std                           3.373153
average_reward                        0.68103
round_time             0 days 00:10:24.414045
episodes_test                           801.0
episode_length_test                 12.475655
returns_test                         8.906658
return_std_test                      4.967211
average_reward_test                  0.714091
round_time_test        0 days 00:00:09.280350
round_time_total       0 days 00:10:24.415155
loss_total              2312213068209041920.0
loss_critic             2890266283232068608.0
loss_actor                      -4178165484.8
memory_size                            2710.0 

=== epoch 3/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:17,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  815
episode_length                      12.267485
returns                              8.348577
return_std                           3.395717
average_reward                       0.680517
round_time             0 days 00:10:23.262890
episodes_test                           820.0
episode_length_test                 12.190244
returns_test                         8.469639
return_std_test                      4.147394
average_reward_test                   0.69486
round_time_test        0 days 00:00:09.380904
round_time_total       0 days 00:10:23.263993
loss_total              2400875269799175680.0
loss_critic             3001094032067229696.0
loss_actor                    -4267900202.112
memory_size                          2711.542 

=== epoch 3/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  814
episode_length                      12.260442
returns                               8.30473
return_std                           2.915256
average_reward                       0.677539
round_time             0 days 00:10:24.134842
episodes_test                           799.0
episode_length_test                 12.446809
returns_test                         8.706448
return_std_test                      4.674838
average_reward_test                  0.705018
round_time_test        0 days 00:00:09.334863
round_time_total       0 days 00:10:24.136102
loss_total              2534953697338158080.0
loss_critic             3168692064197445632.0
loss_actor                    -4375814818.944
memory_size                            2713.0 

=== epoch 3/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  809
episode_length                      12.353523
returns                              8.385925
return_std                            3.14742
average_reward                       0.678985
round_time             0 days 00:10:24.545542
episodes_test                           795.0
episode_length_test                 12.578616
returns_test                         8.784171
return_std_test                      4.414315
average_reward_test                  0.698342
round_time_test        0 days 00:00:09.467201
round_time_total       0 days 00:10:24.546759
loss_total              2675920848692998656.0
loss_critic             3344901001630059520.0
loss_actor                    -4473316676.096
memory_size                         2713.0665 

=== epoch 3/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  808
episode_length                      12.360149
returns                              8.454457
return_std                           2.986397
average_reward                       0.683992
round_time             0 days 00:10:22.451769
episodes_test                           816.0
episode_length_test                 12.253676
returns_test                         8.220119
return_std_test                      3.910053
average_reward_test                  0.670861
round_time_test        0 days 00:00:09.285218
round_time_total       0 days 00:10:22.452880
loss_total              2813143793484995584.0
loss_critic             3516429678153289216.0
loss_actor                    -4581238920.704
memory_size                            2720.0 

=== epoch 3/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:40,  2.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  805
episode_length                      12.397516
returns                              8.524341
return_std                           3.353501
average_reward                       0.687467
round_time             0 days 00:10:23.667083
episodes_test                           813.0
episode_length_test                 12.289053
returns_test                         8.416552
return_std_test                      3.136055
average_reward_test                  0.684932
round_time_test        0 days 00:00:09.198345
round_time_total       0 days 00:10:23.668187
loss_total              2952286873225657344.0
loss_critic             3690358525475474432.0
loss_actor                    -4683654906.496
memory_size                          2729.979 

=== epoch 3/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:15,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  802
episode_length                      12.446384
returns                              8.571076
return_std                           3.090039
average_reward                       0.688499
round_time             0 days 00:10:23.920064
episodes_test                           818.0
episode_length_test                 12.212714
returns_test                         8.286628
return_std_test                      2.561204
average_reward_test                  0.678471
round_time_test        0 days 00:00:09.251067
round_time_total       0 days 00:10:23.921152
loss_total              3094444767395314688.0
loss_critic             3868055894338597376.0
loss_actor                    -4787651006.464
memory_size                            2737.0 

=== epoch 3/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  795
episode_length                      12.566038
returns                              8.858097
return_std                           6.031671
average_reward                       0.704743
round_time             0 days 00:10:23.759314
episodes_test                           812.0
episode_length_test                 12.310345
returns_test                         8.325514
return_std_test                      2.319484
average_reward_test                  0.676406
round_time_test        0 days 00:00:09.491281
round_time_total       0 days 00:10:23.760556
loss_total              3254088765078272512.0
loss_critic             4067610886683471360.0
loss_actor                    -4861945434.752
memory_size                         2774.3375 

=== epoch 3/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:36,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  801
episode_length                      12.464419
returns                              8.725988
return_std                           5.900141
average_reward                       0.700059
round_time             0 days 00:10:26.022966
episodes_test                           789.0
episode_length_test                 12.673004
returns_test                         8.771491
return_std_test                      4.423331
average_reward_test                  0.692172
round_time_test        0 days 00:00:09.442383
round_time_total       0 days 00:10:26.024297
loss_total              3350664771050355712.0
loss_critic             4188330893323941376.0
loss_actor                    -4888047347.712
memory_size                            2787.0 

=== epoch 3/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:18,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  805
episode_length                      12.406211
returns                              8.685013
return_std                           5.939227
average_reward                       0.699919
round_time             0 days 00:10:23.396126
episodes_test                           831.0
episode_length_test                 12.021661
returns_test                         8.109611
return_std_test                      2.750997
average_reward_test                   0.67474
round_time_test        0 days 00:00:09.350286
round_time_total       0 days 00:10:23.397232
loss_total              3413529144958797312.0
loss_critic             4266911362771077632.0
loss_actor                    -4943707361.664
memory_size                          2787.539 

=== epoch 3/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:34,  2.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  808
episode_length                      12.357673
returns                              8.610419
return_std                           5.879395
average_reward                       0.697227
round_time             0 days 00:10:25.391872
episodes_test                           834.0
episode_length_test                 11.978417
returns_test                         8.099544
return_std_test                       2.65879
average_reward_test                  0.676276
round_time_test        0 days 00:00:09.343798
round_time_total       0 days 00:10:25.392972
loss_total              3437756801282188800.0
loss_critic             4297195932814539776.0
loss_actor                    -4986033911.808
memory_size                         2788.7975 

=== epoch 3/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:56,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  811
episode_length                      12.316893
returns                              8.544373
return_std                           5.875687
average_reward                       0.693727
round_time             0 days 00:10:23.920907
episodes_test                           807.0
episode_length_test                 12.365551
returns_test                         8.559996
return_std_test                      2.815747
average_reward_test                  0.694109
round_time_test        0 days 00:00:09.271278
round_time_total       0 days 00:10:23.922184
loss_total              3519728620062077952.0
loss_critic             4399660700620044800.0
loss_actor                    -5042761278.464
memory_size                            2790.0 

=== epoch 3/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:24,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  823
episode_length                      12.148238
returns                              8.185272
return_std                            2.70369
average_reward                       0.673715
round_time             0 days 00:10:24.240236
episodes_test                           801.0
episode_length_test                 12.470662
returns_test                         8.674091
return_std_test                      4.249129
average_reward_test                  0.695477
round_time_test        0 days 00:00:09.364642
round_time_total       0 days 00:10:24.241339
loss_total              3543941566706596352.0
loss_critic             4429926882843360768.0
loss_actor                    -5096587033.344
memory_size                            2790.0 

=== epoch 3/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:21,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  818
episode_length                      12.205379
returns                              8.317947
return_std                           2.929957
average_reward                       0.681508
round_time             0 days 00:10:26.546699
episodes_test                           813.0
episode_length_test                 12.293973
returns_test                          8.42264
return_std_test                       2.85064
average_reward_test                  0.685231
round_time_test        0 days 00:00:09.296043
round_time_total       0 days 00:10:26.547806
loss_total              3681492468879123968.0
loss_critic             4601865510146702848.0
loss_actor                     -5145595338.24
memory_size                            2790.0 

=== epoch 3/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  817
episode_length                      12.227662
returns                              8.304312
return_std                           2.735652
average_reward                       0.679017
round_time             0 days 00:10:22.980857
episodes_test                           822.0
episode_length_test                 12.152068
returns_test                         8.370708
return_std_test                      3.134728
average_reward_test                  0.688881
round_time_test        0 days 00:00:09.251100
round_time_total       0 days 00:10:22.982034
loss_total              3768798691810067968.0
loss_critic             4710998281543299072.0
loss_actor                    -5219181286.144
memory_size                            2790.0 

=== epoch 3/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  818
episode_length                      12.215159
returns                              8.279329
return_std                           2.427023
average_reward                       0.677923
round_time             0 days 00:10:24.386940
episodes_test                           835.0
episode_length_test                 11.965269
returns_test                         7.956165
return_std_test                      3.688853
average_reward_test                  0.665011
round_time_test        0 days 00:00:09.403732
round_time_total       0 days 00:10:24.388053
loss_total              3891118690660127744.0
loss_critic             4863898284400840704.0
loss_actor                    -5284280673.792
memory_size                            2790.0 

=== epoch 3/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  820
episode_length                      12.185366
returns                              8.281671
return_std                           2.458428
average_reward                       0.679615
round_time             0 days 00:10:25.471704
episodes_test                           831.0
episode_length_test                 12.020457
returns_test                         8.250399
return_std_test                      3.491936
average_reward_test                  0.686555
round_time_test        0 days 00:00:09.362616
round_time_total       0 days 00:10:25.472801
loss_total              3991093551849809920.0
loss_critic             4988866857159912448.0
loss_actor                    -5348151922.944
memory_size                            2790.0 

=== epoch 3/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  819
episode_length                      12.208791
returns                              8.305183
return_std                           2.473265
average_reward                       0.680291
round_time             0 days 00:10:27.514842
episodes_test                           812.0
episode_length_test                 12.305419
returns_test                         8.349152
return_std_test                      3.600081
average_reward_test                  0.678497
round_time_test        0 days 00:00:08.996633
round_time_total       0 days 00:10:27.516283
loss_total              4100190029467084800.0
loss_critic             5125237452978914304.0
loss_actor                    -5409779597.056
memory_size                            2790.0 

=== epoch 3/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  828
episode_length                      12.062802
returns                               8.13677
return_std                            2.18799
average_reward                        0.67439
round_time             0 days 00:10:26.513058
episodes_test                           815.0
episode_length_test                 12.257669
returns_test                         8.455035
return_std_test                      3.764859
average_reward_test                  0.689818
round_time_test        0 days 00:00:09.374633
round_time_total       0 days 00:10:26.514142
loss_total              4207991794072062976.0
loss_critic             5259989653100139520.0
loss_actor                    -5506323330.304
memory_size                            2790.0 

=== epoch 3/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  827
episode_length                      12.081016
returns                              8.150969
return_std                           2.280317
average_reward                       0.674683
round_time             0 days 00:10:25.147420
episodes_test                           834.0
episode_length_test                 11.986811
returns_test                         7.924051
return_std_test                        2.4156
average_reward_test                  0.661182
round_time_test        0 days 00:00:09.504268
round_time_total       0 days 00:10:25.148717
loss_total              4348791166102898176.0
loss_critic             5435988868447921152.0
loss_actor                    -5580812568.064
memory_size                            2790.0 

=== epoch 3/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:42,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  824
episode_length                      12.114078
returns                              8.211014
return_std                           2.525477
average_reward                       0.677579
round_time             0 days 00:10:23.045638
episodes_test                           823.0
episode_length_test                 12.149453
returns_test                          8.11291
return_std_test                      2.764397
average_reward_test                  0.667797
round_time_test        0 days 00:00:09.498925
round_time_total       0 days 00:10:23.046742
loss_total              4530418588371535360.0
loss_critic             5663023135494760448.0
loss_actor                     -5672065002.24
memory_size                          2791.415 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 4/10 ===== round 1/50 ======================================
  0%|          | 6/2000 [00:01<10:24,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  169
episode_length                      11.822485
returns                              7.845659
return_std                           2.190641
average_reward                       0.663927
round_time             0 days 00:10:19.591268
episodes_test                           828.0
episode_length_test                 12.076087
returns_test                          8.00049
return_std_test                      2.841846
average_reward_test                  0.662542
round_time_test        0 days 00:00:09.373897
round_time_total       0 days 00:10:19.592429
loss_total              4638976695686546432.0
loss_critic             5798720759966258176.0
loss_actor                      -5759307468.8
memory_size                            2795.0 

=== epoch 4/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  329
episode_length                      12.158055
returns                              8.093806
return_std                           3.025232
average_reward                       0.665716
round_time             0 days 00:10:21.309825
episodes_test                           799.0
episode_length_test                 12.503129
returns_test                         8.249462
return_std_test                       4.96168
average_reward_test                  0.659803
round_time_test        0 days 00:00:09.460670
round_time_total       0 days 00:10:21.310901
loss_total              4817559736009250816.0
loss_critic             6021949560232199168.0
loss_actor                     -5873238536.96
memory_size                          2798.507 

=== epoch 4/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  491
episode_length                      12.211813
returns                              8.070862
return_std                           2.780338
average_reward                       0.661117
round_time             0 days 00:10:22.000036
episodes_test                           815.0
episode_length_test                 12.257669
returns_test                         8.183778
return_std_test                      3.047211
average_reward_test                  0.667714
round_time_test        0 days 00:00:09.281650
round_time_total       0 days 00:10:22.001169
loss_total              5002724586457763840.0
loss_critic             6253405619805327360.0
loss_actor                    -5958804873.472
memory_size                            2799.0 

=== epoch 4/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  658
episode_length                      12.141337
returns                              7.988394
return_std                           2.617743
average_reward                       0.657921
round_time             0 days 00:10:20.007225
episodes_test                           846.0
episode_length_test                 11.809693
returns_test                         7.522696
return_std_test                      3.293941
average_reward_test                  0.637257
round_time_test        0 days 00:00:09.367585
round_time_total       0 days 00:10:20.008326
loss_total              5198395803085622272.0
loss_critic             6497994633151266816.0
loss_actor                    -6084352708.608
memory_size                            2799.0 

=== epoch 4/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  826
episode_length                      12.095642
returns                              7.986233
return_std                           2.698451
average_reward                       0.660415
round_time             0 days 00:10:19.835185
episodes_test                           835.0
episode_length_test                 11.971257
returns_test                         7.774465
return_std_test                      2.274432
average_reward_test                  0.649567
round_time_test        0 days 00:00:09.396882
round_time_total       0 days 00:10:19.836284
loss_total              5428567601054232576.0
loss_critic             6785709375595508736.0
loss_actor                    -6176917496.832
memory_size                         2799.0195 

=== epoch 4/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  823
episode_length                      12.110571
returns                              8.009003
return_std                           2.711191
average_reward                       0.662799
round_time             0 days 00:10:21.288114
episodes_test                           847.0
episode_length_test                 11.793388
returns_test                          7.58865
return_std_test                      2.285212
average_reward_test                  0.643431
round_time_test        0 days 00:00:09.407706
round_time_total       0 days 00:10:21.289358
loss_total              5570529681480113152.0
loss_critic             6963161972004689920.0
loss_actor                    -6285637768.192
memory_size                            2801.0 

=== epoch 4/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:41,  2.85it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  825
episode_length                      12.117576
returns                              7.995315
return_std                           3.070724
average_reward                       0.659877
round_time             0 days 00:10:21.603504
episodes_test                           843.0
episode_length_test                  11.86121
returns_test                         7.727938
return_std_test                      3.201632
average_reward_test                  0.651571
round_time_test        0 days 00:00:09.466237
round_time_total       0 days 00:10:21.604777
loss_total              5918721309488149504.0
loss_critic             7398401507323973632.0
loss_actor                    -6420879298.816
memory_size                         2818.8965 

=== epoch 4/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:23,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  829
episode_length                      12.047045
returns                              7.970064
return_std                           3.491766
average_reward                       0.661471
round_time             0 days 00:10:22.909724
episodes_test                           828.0
episode_length_test                 12.077295
returns_test                         7.919809
return_std_test                      3.849221
average_reward_test                   0.65576
round_time_test        0 days 00:00:09.334847
round_time_total       0 days 00:10:22.910839
loss_total              6197102243435813888.0
loss_critic             7746377667611727872.0
loss_actor                     -6551239668.48
memory_size                          2823.084 

=== epoch 4/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:28,  2.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  829
episode_length                      12.050663
returns                              7.940793
return_std                           3.481291
average_reward                       0.658898
round_time             0 days 00:10:22.125559
episodes_test                           846.0
episode_length_test                 11.814421
returns_test                          7.71999
return_std_test                      2.593147
average_reward_test                  0.653502
round_time_test        0 days 00:00:09.278716
round_time_total       0 days 00:10:22.126927
loss_total              6479421438368415744.0
loss_critic             8099276658528701440.0
loss_actor                    -6715636496.384
memory_size                            2835.0 

=== epoch 4/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  823
episode_length                      12.148238
returns                              7.979122
return_std                           3.436633
average_reward                       0.656742
round_time             0 days 00:10:19.835442
episodes_test                           828.0
episode_length_test                 12.071256
returns_test                         7.881533
return_std_test                      2.617608
average_reward_test                  0.652956
round_time_test        0 days 00:00:09.339656
round_time_total       0 days 00:10:19.836531
loss_total              6819363346245732352.0
loss_critic             8524204047601594368.0
loss_actor                    -6856388049.152
memory_size                            2835.0 

=== epoch 4/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  815
episode_length                      12.235583
returns                              8.023536
return_std                           3.701462
average_reward                       0.656864
round_time             0 days 00:10:23.685965
episodes_test                           804.0
episode_length_test                 12.427861
returns_test                         8.308484
return_std_test                      4.417651
average_reward_test                  0.668533
round_time_test        0 days 00:00:09.260341
round_time_total       0 days 00:10:23.687074
loss_total              7011097654356674560.0
loss_critic             8763871929201219584.0
loss_actor                    -6991052919.808
memory_size                          2845.932 

=== epoch 4/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  813
episode_length                      12.279213
returns                              8.018403
return_std                           3.549831
average_reward                       0.653242
round_time             0 days 00:10:22.049732
episodes_test                           823.0
episode_length_test                 12.147023
returns_test                         7.964041
return_std_test                      2.608332
average_reward_test                  0.655702
round_time_test        0 days 00:00:09.377710
round_time_total       0 days 00:10:22.051084
loss_total              7379309102672081920.0
loss_critic             9224136220388384768.0
loss_actor                     -7102445396.48
memory_size                            2851.0 

=== epoch 4/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  814
episode_length                      12.264128
returns                              7.952702
return_std                           3.089251
average_reward                       0.648759
round_time             0 days 00:10:21.201278
episodes_test                           813.0
episode_length_test                 12.287823
returns_test                         8.074964
return_std_test                      3.303109
average_reward_test                  0.657187
round_time_test        0 days 00:00:09.336454
round_time_total       0 days 00:10:21.202373
loss_total              7602823478955665408.0
loss_critic             9503529195415789568.0
loss_actor                    -7232490429.696
memory_size                            2851.0 

=== epoch 4/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:24,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  811
episode_length                      12.320592
returns                              8.047556
return_std                           3.596789
average_reward                       0.653255
round_time             0 days 00:10:21.877246
episodes_test                           821.0
episode_length_test                  12.17905
returns_test                         7.942302
return_std_test                      2.961601
average_reward_test                  0.652163
round_time_test        0 days 00:00:09.311273
round_time_total       0 days 00:10:21.878342
loss_total              7925670634142622720.0
loss_critic             9907088124796596224.0
loss_actor                    -7391468026.112
memory_size                          2855.511 

=== epoch 4/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:20<00:00,  3.22it/s]
episodes                                  815
episode_length                      12.263804
returns                              7.950924
return_std                           3.825922
average_reward                       0.648403
round_time             0 days 00:10:20.961714
episodes_test                           830.0
episode_length_test                 12.037349
returns_test                         7.654856
return_std_test                      2.871516
average_reward_test                  0.635967
round_time_test        0 days 00:00:09.336428
round_time_total       0 days 00:10:20.962953
loss_total              8383559677573217280.0
loss_critic            10479449418227163136.0
loss_actor                    -7528865912.576
memory_size                          2866.224 

=== epoch 4/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:15,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  823
episode_length                      12.131227
returns                              7.829093
return_std                            3.41573
average_reward                        0.64539
round_time             0 days 00:10:23.738394
episodes_test                           824.0
episode_length_test                 12.129854
returns_test                         7.619982
return_std_test                      2.347048
average_reward_test                  0.628331
round_time_test        0 days 00:00:09.455809
round_time_total       0 days 00:10:23.739762
loss_total              8759536722253543424.0
loss_critic            10949420710299316224.0
loss_actor                    -7664098789.376
memory_size                            2873.0 

=== epoch 4/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  833
episode_length                      11.992797
returns                              7.696791
return_std                           3.345713
average_reward                       0.641366
round_time             0 days 00:10:22.175042
episodes_test                           835.0
episode_length_test                 11.976048
returns_test                         7.571102
return_std_test                      2.891348
average_reward_test                  0.632187
round_time_test        0 days 00:00:09.406613
round_time_total       0 days 00:10:22.176142
loss_total              9108985014399120384.0
loss_critic            11386231061943547904.0
loss_actor                    -7811968391.424
memory_size                            2873.0 

=== epoch 4/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:07,  2.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  829
episode_length                      12.049457
returns                              7.694934
return_std                           3.382386
average_reward                       0.638712
round_time             0 days 00:10:23.581521
episodes_test                           825.0
episode_length_test                 12.110303
returns_test                         7.648291
return_std_test                      2.599221
average_reward_test                  0.631523
round_time_test        0 days 00:00:09.375111
round_time_total       0 days 00:10:23.582612
loss_total              9493892956075509760.0
loss_critic            11867365986805653504.0
loss_actor                     -7965357071.36
memory_size                            2873.0 

=== epoch 4/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:45,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  830
episode_length                       12.03494
returns                              7.564772
return_std                           2.895996
average_reward                        0.62845
round_time             0 days 00:10:22.183274
episodes_test                           824.0
episode_length_test                 12.134709
returns_test                         7.776567
return_std_test                      3.554111
average_reward_test                  0.640894
round_time_test        0 days 00:00:09.294122
round_time_total       0 days 00:10:22.184763
loss_total              9780495324530075648.0
loss_critic            12225618938543407104.0
loss_actor                    -8120630907.136
memory_size                            2873.0 

=== epoch 4/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  827
episode_length                      12.082225
returns                              7.542086
return_std                            3.07462
average_reward                       0.624147
round_time             0 days 00:10:24.435114
episodes_test                           805.0
episode_length_test                 12.407453
returns_test                         7.694689
return_std_test                      5.334926
average_reward_test                  0.620251
round_time_test        0 days 00:00:09.576510
round_time_total       0 days 00:10:24.436216
loss_total             10390265827913900032.0
loss_critic            12987832054682128384.0
loss_actor                    -8296628466.944
memory_size                         2880.7715 

=== epoch 4/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  824
episode_length                      12.127427
returns                              7.432327
return_std                            2.75914
average_reward                       0.612894
round_time             0 days 00:10:26.095307
episodes_test                           808.0
episode_length_test                 12.365099
returns_test                         7.532697
return_std_test                       3.33334
average_reward_test                  0.609274
round_time_test        0 days 00:00:09.151480
round_time_total       0 days 00:10:26.096421
loss_total             10803333359277920256.0
loss_critic            13504166476515016704.0
loss_actor                    -8462492431.616
memory_size                            2884.0 

=== epoch 4/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:47,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  822
episode_length                      12.145985
returns                              7.388136
return_std                           2.766531
average_reward                       0.608265
round_time             0 days 00:10:24.938013
episodes_test                           811.0
episode_length_test                 12.323058
returns_test                         7.563798
return_std_test                      3.380041
average_reward_test                  0.613969
round_time_test        0 days 00:00:09.439296
round_time_total       0 days 00:10:24.939106
loss_total             11390050357969385472.0
loss_critic            14237562702957832192.0
loss_actor                    -8633835315.712
memory_size                            2884.0 

=== epoch 4/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  820
episode_length                      12.173171
returns                              7.356935
return_std                           2.794703
average_reward                       0.604528
round_time             0 days 00:10:24.131338
episodes_test                           821.0
episode_length_test                 12.175396
returns_test                          7.39142
return_std_test                      2.626015
average_reward_test                  0.607229
round_time_test        0 days 00:00:09.373827
round_time_total       0 days 00:10:24.132417
loss_total             11921282351193933824.0
loss_critic            14901602677377370112.0
loss_actor                    -8802138620.416
memory_size                            2884.0 

=== epoch 4/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  819
episode_length                       12.19536
returns                              7.312243
return_std                           2.771787
average_reward                       0.599368
round_time             0 days 00:10:25.042167
episodes_test                           814.0
episode_length_test                 12.276413
returns_test                         7.422323
return_std_test                       2.13754
average_reward_test                  0.604759
round_time_test        0 days 00:00:09.446919
round_time_total       0 days 00:10:25.043277
loss_total             12370547402934736896.0
loss_critic            15463183969444665344.0
loss_actor                 -8996409388.799999
memory_size                            2884.0 

=== epoch 4/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:44,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  817
episode_length                      12.227662
returns                              7.280224
return_std                           2.549688
average_reward                       0.595485
round_time             0 days 00:10:26.773350
episodes_test                           802.0
episode_length_test                 12.462594
returns_test                         7.558275
return_std_test                      3.911621
average_reward_test                  0.606624
round_time_test        0 days 00:00:09.399736
round_time_total       0 days 00:10:26.774478
loss_total             12842249424028104704.0
loss_critic            16052811508180879360.0
loss_actor                 -9192517203.200001
memory_size                          2887.079 

=== epoch 4/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:14,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  817
episode_length                       12.22399
returns                              7.262958
return_std                           2.866918
average_reward                       0.594028
round_time             0 days 00:10:23.765427
episodes_test                           812.0
episode_length_test                 12.305419
returns_test                         7.304667
return_std_test                      3.199034
average_reward_test                  0.593843
round_time_test        0 days 00:00:09.483216
round_time_total       0 days 00:10:23.766510
loss_total             13354102594082334720.0
loss_critic            16692627951988250624.0
loss_actor                 -9397009502.719999
memory_size                            2888.0 

=== epoch 4/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:56,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  809
episode_length                      12.331273
returns                              7.324839
return_std                           2.953021
average_reward                       0.594002
round_time             0 days 00:10:23.234181
episodes_test                           792.0
episode_length_test                 12.623737
returns_test                         7.818053
return_std_test                      4.745868
average_reward_test                  0.619371
round_time_test        0 days 00:00:09.468906
round_time_total       0 days 00:10:23.235491
loss_total             14003596558256609280.0
loss_critic            17504495412360056832.0
loss_actor                     -9604562282.24
memory_size                          2888.143 

=== epoch 4/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<12:19,  2.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  802
episode_length                      12.462594
returns                              7.418019
return_std                             3.4036
average_reward                         0.5952
round_time             0 days 00:10:27.139892
episodes_test                           802.0
episode_length_test                 12.453865
returns_test                         7.511776
return_std_test                      3.827664
average_reward_test                  0.603502
round_time_test        0 days 00:00:09.268483
round_time_total       0 days 00:10:27.140973
loss_total             14591623750122016768.0
loss_critic            18239529383637555200.0
loss_actor                      -9807469017.6
memory_size                         2901.5255 

=== epoch 4/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:17,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  789
episode_length                       12.65526
returns                              7.529357
return_std                           3.926569
average_reward                       0.594945
round_time             0 days 00:10:24.816002
episodes_test                           797.0
episode_length_test                  12.53325
returns_test                         7.333426
return_std_test                      3.934194
average_reward_test                  0.585534
round_time_test        0 days 00:00:09.287333
round_time_total       0 days 00:10:24.817121
loss_total             15498527406781175808.0
loss_critic            19373158939411939328.0
loss_actor                -10016235960.832001
memory_size                         2915.0115 

=== epoch 4/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:32,  2.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  780
episode_length                      12.817949
returns                              7.425415
return_std                           3.770396
average_reward                       0.579383
round_time             0 days 00:10:26.162749
episodes_test                           761.0
episode_length_test                 13.134034
returns_test                         7.366642
return_std_test                      3.461019
average_reward_test                  0.560981
round_time_test        0 days 00:00:09.383273
round_time_total       0 days 00:10:26.163858
loss_total             16234734376370726912.0
loss_critic            20293417629065048064.0
loss_actor                   -10243326214.656
memory_size                         2926.6875 

=== epoch 4/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:36,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  767
episode_length                      13.023468
returns                              7.502551
return_std                           4.012944
average_reward                        0.57584
round_time             0 days 00:10:25.976056
episodes_test                           771.0
episode_length_test                 12.968872
returns_test                         7.074034
return_std_test                      3.299347
average_reward_test                  0.545511
round_time_test        0 days 00:00:09.260350
round_time_total       0 days 00:10:25.977160
loss_total             16939331219816652800.0
loss_critic            21174163658908323840.0
loss_actor                -10465785571.327999
memory_size                         2929.9055 

=== epoch 4/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:48,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  768
episode_length                      13.009115
returns                              7.584858
return_std                           4.304534
average_reward                       0.582967
round_time             0 days 00:10:24.456305
episodes_test                           792.0
episode_length_test                 12.611111
returns_test                         6.860443
return_std_test                      2.331409
average_reward_test                   0.54415
round_time_test        0 days 00:00:09.274419
round_time_total       0 days 00:10:24.457828
loss_total             17905320227820077056.0
loss_critic            22381649922004979712.0
loss_actor                   -10652223747.584
memory_size                            2933.0 

=== epoch 4/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:23,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  778
episode_length                      12.835476
returns                              7.380479
return_std                           3.898459
average_reward                       0.575084
round_time             0 days 00:10:26.874130
episodes_test                           800.0
episode_length_test                   12.4975
returns_test                         7.202125
return_std_test                      3.032429
average_reward_test                  0.576359
round_time_test        0 days 00:00:09.341414
round_time_total       0 days 00:10:26.875215
loss_total             18562030657617743872.0
loss_critic            23202537905238552576.0
loss_actor                -10867418212.351999
memory_size                            2933.0 

=== epoch 4/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:32,  2.88it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  783
episode_length                      12.763729
returns                              7.378425
return_std                           4.213393
average_reward                       0.578161
round_time             0 days 00:10:25.618688
episodes_test                           803.0
episode_length_test                 12.442092
returns_test                         7.118254
return_std_test                      2.307387
average_reward_test                   0.57216
round_time_test        0 days 00:00:09.420542
round_time_total       0 days 00:10:25.619899
loss_total             19460137493377994752.0
loss_critic            24325171447258808320.0
loss_actor                   -11118337456.128
memory_size                          2943.098 

=== epoch 4/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  789
episode_length                      12.657795
returns                              7.546648
return_std                           4.567525
average_reward                       0.596001
round_time             0 days 00:10:27.991353
episodes_test                           804.0
episode_length_test                  12.43408
returns_test                         7.165151
return_std_test                      3.713528
average_reward_test                   0.57638
round_time_test        0 days 00:00:09.232613
round_time_total       0 days 00:10:27.992436
loss_total             20957106960705273856.0
loss_critic            26196383249463349248.0
loss_actor                   -11377786273.792
memory_size                         2982.7075 

=== epoch 4/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:02,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  787
episode_length                      12.691233
returns                              7.619116
return_std                           4.892504
average_reward                       0.600349
round_time             0 days 00:10:26.014786
episodes_test                           797.0
episode_length_test                 12.542033
returns_test                         7.170279
return_std_test                      3.408879
average_reward_test                  0.571885
round_time_test        0 days 00:00:09.493963
round_time_total       0 days 00:10:26.016031
loss_total             20853879919561453568.0
loss_critic            26067349429616754688.0
loss_actor                   -11445405750.784
memory_size                          3001.479 

=== epoch 4/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  785
episode_length                      12.732484
returns                              7.629095
return_std                           4.736899
average_reward                       0.599112
round_time             0 days 00:10:26.995087
episodes_test                           773.0
episode_length_test                 12.928849
returns_test                         7.747137
return_std_test                      3.660517
average_reward_test                  0.599446
round_time_test        0 days 00:00:09.259342
round_time_total       0 days 00:10:26.996345
loss_total             20544848467039399936.0
loss_critic            25681060143582281728.0
loss_actor                   -11445887969.792
memory_size                         3026.3025 

=== epoch 4/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:07,  2.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  767
episode_length                      13.019557
returns                              7.915755
return_std                            5.02103
average_reward                       0.608164
round_time             0 days 00:10:28.283445
episodes_test                           759.0
episode_length_test                 13.172596
returns_test                         8.034935
return_std_test                      4.479671
average_reward_test                  0.610027
round_time_test        0 days 00:00:09.413898
round_time_total       0 days 00:10:28.284539
loss_total             20014842637912150016.0
loss_critic            25018552866381631488.0
loss_actor                -11449787938.304001
memory_size                          3033.178 

=== epoch 4/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:01,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  755
episode_length                      13.227815
returns                              8.035614
return_std                           4.637028
average_reward                       0.607792
round_time             0 days 00:10:27.475298
episodes_test                           737.0
episode_length_test                  13.56038
returns_test                         8.342451
return_std_test                      3.695032
average_reward_test                  0.615317
round_time_test        0 days 00:00:09.183113
round_time_total       0 days 00:10:27.476408
loss_total             19714324064557957120.0
loss_critic            24642904647283703808.0
loss_actor                -11364702109.695999
memory_size                          3042.793 

=== epoch 4/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  750
episode_length                         13.316
returns                              8.150101
return_std                            4.58473
average_reward                       0.612278
round_time             0 days 00:10:28.190175
episodes_test                           725.0
episode_length_test                 13.777931
returns_test                         8.634307
return_std_test                      5.129036
average_reward_test                  0.626936
round_time_test        0 days 00:00:09.368291
round_time_total       0 days 00:10:28.191294
loss_total             19155343399515082752.0
loss_critic            23944178819347369984.0
loss_actor                   -11243187674.112
memory_size                          3047.657 

=== epoch 4/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  748
episode_length                       13.34492
returns                              8.231252
return_std                           3.997107
average_reward                       0.616939
round_time             0 days 00:10:26.971340
episodes_test                           754.0
episode_length_test                 13.251989
returns_test                         8.170627
return_std_test                      5.411877
average_reward_test                  0.616832
round_time_test        0 days 00:00:09.409418
round_time_total       0 days 00:10:26.972430
loss_total             18468897372910440448.0
loss_critic            23086121315022467072.0
loss_actor                -11108520011.775999
memory_size                         3056.4245 

=== epoch 4/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:04,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  748
episode_length                      13.354278
returns                              8.184365
return_std                           3.929292
average_reward                       0.612407
round_time             0 days 00:10:28.716147
episodes_test                           792.0
episode_length_test                 12.623737
returns_test                         7.772288
return_std_test                      3.700062
average_reward_test                  0.615771
round_time_test        0 days 00:00:09.384581
round_time_total       0 days 00:10:28.717350
loss_total             17876297877334091776.0
loss_critic            22345371963006775296.0
loss_actor                   -10907825694.208
memory_size                         3062.2375 

=== epoch 4/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  761
episode_length                      13.130092
returns                              7.992637
return_std                           3.606621
average_reward                       0.608737
round_time             0 days 00:10:27.115070
episodes_test                           790.0
episode_length_test                 12.646835
returns_test                         7.694916
return_std_test                      3.350329
average_reward_test                  0.608499
round_time_test        0 days 00:00:09.221665
round_time_total       0 days 00:10:27.116168
loss_total             16941048433228623872.0
loss_critic            21176310171962433536.0
loss_actor                -10700536296.959999
memory_size                         3072.7375 

=== epoch 4/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:16,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  778
episode_length                      12.843188
returns                              7.893363
return_std                           3.700744
average_reward                       0.614296
round_time             0 days 00:10:28.024512
episodes_test                           806.0
episode_length_test                 12.401985
returns_test                         7.298605
return_std_test                      3.063186
average_reward_test                  0.588672
round_time_test        0 days 00:00:09.338653
round_time_total       0 days 00:10:28.025770
loss_total             16067906165932308480.0
loss_critic            20084882357083492352.0
loss_actor                    -10512606289.92
memory_size                         3081.4305 

=== epoch 4/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:49,  2.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  798
episode_length                       12.52381
returns                              7.610833
return_std                           3.386204
average_reward                       0.607933
round_time             0 days 00:10:28.274530
episodes_test                           801.0
episode_length_test                 12.476904
returns_test                         7.623783
return_std_test                      3.873267
average_reward_test                  0.611216
round_time_test        0 days 00:00:09.306758
round_time_total       0 days 00:10:28.275620
loss_total             15169121821002625024.0
loss_critic            18961401947568021504.0
loss_actor                   -10276396382.208
memory_size                          3086.811 

=== epoch 4/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:42,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  824
episode_length                      12.116505
returns                              7.253828
return_std                            3.31623
average_reward                       0.598888
round_time             0 days 00:10:28.678896
episodes_test                           870.0
episode_length_test                 11.494253
returns_test                         6.833599
return_std_test                      3.187426
average_reward_test                  0.594523
round_time_test        0 days 00:00:09.339267
round_time_total       0 days 00:10:28.680008
loss_total             14387906493838032896.0
loss_critic            17984882819673487360.0
loss_actor                    -10076100846.08
memory_size                         3098.6035 

=== epoch 4/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:01,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  847
episode_length                      11.804014
returns                              7.053748
return_std                           3.169873
average_reward                       0.597658
round_time             0 days 00:10:27.086049
episodes_test                           855.0
episode_length_test                 11.688889
returns_test                         7.081228
return_std_test                      2.686822
average_reward_test                   0.60596
round_time_test        0 days 00:00:09.349528
round_time_total       0 days 00:10:27.087281
loss_total             13764434016049401856.0
loss_critic            17205542219757639680.0
loss_actor                 -9892988708.351999
memory_size                            3110.0 

=== epoch 4/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:50,  2.81it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  870
episode_length                      11.475862
returns                              6.859026
return_std                           3.124611
average_reward                       0.597729
round_time             0 days 00:10:26.965608
episodes_test                           893.0
episode_length_test                 11.198208
returns_test                         6.719925
return_std_test                      2.530221
average_reward_test                  0.600089
round_time_test        0 days 00:00:09.373991
round_time_total       0 days 00:10:26.966714
loss_total             13016745017505044480.0
loss_critic            16270930997621874688.0
loss_actor                    -9661858479.872
memory_size                            3110.0 

=== epoch 4/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:29<00:00,  3.18it/s]
episodes                                  891
episode_length                      11.205387
returns                              6.622891
return_std                           2.553325
average_reward                       0.590941
round_time             0 days 00:10:30.296367
episodes_test                           886.0
episode_length_test                 11.278781
returns_test                         6.756699
return_std_test                      3.299711
average_reward_test                  0.599165
round_time_test        0 days 00:00:09.587773
round_time_total       0 days 00:10:30.297830
loss_total             12495416923096442880.0
loss_critic            15619270875968989184.0
loss_actor                    -9510227501.312
memory_size                         3110.2515 

=== epoch 4/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:36,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  908
episode_length                      10.997797
returns                              6.531282
return_std                           2.269011
average_reward                       0.594039
round_time             0 days 00:10:27.520952
episodes_test                           930.0
episode_length_test                 10.743011
returns_test                         6.525273
return_std_test                      3.101716
average_reward_test                  0.607516
round_time_test        0 days 00:00:09.421462
round_time_total       0 days 00:10:27.522259
loss_total             12032357453998901248.0
loss_critic            15040446546675167232.0
loss_actor                    -9364292580.608
memory_size                            3112.0 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 5/10 ===== round 1/50 ======================================
  0%|          | 6/2000 [00:02<11:43,  2.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  196
episode_length                      10.183673
returns                              5.960804
return_std                           1.884426
average_reward                        0.58605
round_time             0 days 00:10:23.646288
episodes_test                           971.0
episode_length_test                 10.294542
returns_test                         6.345248
return_std_test                      2.837649
average_reward_test                  0.616536
round_time_test        0 days 00:00:09.405285
round_time_total       0 days 00:10:23.647393
loss_total             11639682173918857216.0
loss_critic            14549602465472968704.0
loss_actor                    -9354507720.448
memory_size                            3112.0 

=== epoch 5/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  393
episode_length                      10.170483
returns                              6.191241
return_std                           1.951006
average_reward                       0.609055
round_time             0 days 00:10:25.897270
episodes_test                           952.0
episode_length_test                 10.494748
returns_test                         6.449458
return_std_test                      3.498355
average_reward_test                  0.614527
round_time_test        0 days 00:00:09.504213
round_time_total       0 days 00:10:25.898357
loss_total             11496219701025226752.0
loss_critic            14370274367209105408.0
loss_actor                    -9331699405.056
memory_size                            3112.0 

=== epoch 5/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:23,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  586
episode_length                      10.232082
returns                              6.203414
return_std                           2.108365
average_reward                       0.606545
round_time             0 days 00:10:24.700579
episodes_test                           983.0
episode_length_test                 10.170905
returns_test                         6.188611
return_std_test                       2.63571
average_reward_test                  0.608521
round_time_test        0 days 00:00:09.497793
round_time_total       0 days 00:10:24.701714
loss_total             11398436851574165504.0
loss_critic            14248045825598803968.0
loss_actor                 -9343222051.072001
memory_size                            3112.0 

=== epoch 5/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:54,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  780
episode_length                       10.25641
returns                              6.290897
return_std                           2.386076
average_reward                       0.613362
round_time             0 days 00:10:24.890161
episodes_test                           953.0
episode_length_test                 10.493179
returns_test                         6.640211
return_std_test                       3.18075
average_reward_test                  0.632812
round_time_test        0 days 00:00:09.573875
round_time_total       0 days 00:10:24.891289
loss_total             11278752101033224192.0
loss_critic            14098439874915684352.0
loss_actor                    -9380125192.704
memory_size                            3112.0 

=== epoch 5/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  969
episode_length                      10.307534
returns                              6.324655
return_std                           2.498538
average_reward                       0.613671
round_time             0 days 00:10:26.706114
episodes_test                           946.0
episode_length_test                 10.569767
returns_test                         6.684732
return_std_test                      2.879635
average_reward_test                   0.63247
round_time_test        0 days 00:00:09.328673
round_time_total       0 days 00:10:26.707248
loss_total             11432979619595526144.0
loss_critic            14291224268170760192.0
loss_actor                 -9385745450.752001
memory_size                         3128.9625 

=== epoch 5/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  978
episode_length                      10.215746
returns                              6.348729
return_std                           2.517274
average_reward                       0.621596
round_time             0 days 00:10:25.740468
episodes_test                          1022.0
episode_length_test                  9.782779
returns_test                         5.821553
return_std_test                      1.834059
average_reward_test                  0.595158
round_time_test        0 days 00:00:09.418340
round_time_total       0 days 00:10:25.741588
loss_total             11270611919473543168.0
loss_critic            14088264654288273408.0
loss_actor                 -9391758628.351999
memory_size                            3135.0 

=== epoch 5/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:55,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  980
episode_length                      10.191837
returns                              6.390014
return_std                           2.812206
average_reward                       0.626804
round_time             0 days 00:10:26.886626
episodes_test                           995.0
episode_length_test                 10.045226
returns_test                         6.356126
return_std_test                      2.857465
average_reward_test                  0.632934
round_time_test        0 days 00:00:09.435506
round_time_total       0 days 00:10:26.887764
loss_total             11393622636888291328.0
loss_critic            14242028037931288576.0
loss_actor                    -9361965115.136
memory_size                          3138.599 

=== epoch 5/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:27,  2.90it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  983
episode_length                      10.159715
returns                              6.528536
return_std                           2.814177
average_reward                       0.642801
round_time             0 days 00:10:23.994554
episodes_test                           982.0
episode_length_test                 10.177189
returns_test                         6.924837
return_std_test                      3.763558
average_reward_test                  0.680524
round_time_test        0 days 00:00:09.466963
round_time_total       0 days 00:10:23.995664
loss_total             11213875772028801024.0
loss_critic            14017344462698082304.0
loss_actor                    -9161859824.896
memory_size                            3139.0 

=== epoch 5/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  979
episode_length                      10.206333
returns                              6.622917
return_std                           2.784477
average_reward                       0.649047
round_time             0 days 00:10:24.621667
episodes_test                           966.0
episode_length_test                  10.34472
returns_test                         7.026087
return_std_test                      3.322625
average_reward_test                  0.679366
round_time_test        0 days 00:00:09.701809
round_time_total       0 days 00:10:24.622775
loss_total             11058187054739986432.0
loss_critic            13822733576944742400.0
loss_actor                    -9258706872.576
memory_size                            3139.0 

=== epoch 5/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:57,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  990
episode_length                      10.093939
returns                              6.665239
return_std                           2.639881
average_reward                       0.660319
round_time             0 days 00:10:26.880426
episodes_test                           991.0
episode_length_test                 10.084763
returns_test                         6.785793
return_std_test                       2.78935
average_reward_test                  0.673041
round_time_test        0 days 00:00:09.405372
round_time_total       0 days 00:10:26.881557
loss_total             11017330708158126080.0
loss_critic            13771663144473331712.0
loss_actor                    -9373449643.264
memory_size                            3139.0 

=== epoch 5/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:15,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  982
episode_length                      10.170061
returns                              6.764102
return_std                           2.681985
average_reward                       0.664811
round_time             0 days 00:10:27.207922
episodes_test                           992.0
episode_length_test                 10.077621
returns_test                         6.361922
return_std_test                      2.513123
average_reward_test                  0.631373
round_time_test        0 days 00:00:09.674571
round_time_total       0 days 00:10:27.209311
loss_total             11078033073320210432.0
loss_critic            13847541105736298496.0
loss_actor                 -9527021726.464001
memory_size                            3139.0 

=== epoch 5/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  974
episode_length                      10.256674
returns                              6.873571
return_std                           2.703244
average_reward                        0.67022
round_time             0 days 00:10:25.554711
episodes_test                           992.0
episode_length_test                 10.069556
returns_test                         6.671601
return_std_test                      2.533585
average_reward_test                  0.663112
round_time_test        0 days 00:00:09.584084
round_time_total       0 days 00:10:25.555803
loss_total             11153734835370924032.0
loss_critic            13942168285622265856.0
loss_actor                 -9570913526.271999
memory_size                            3139.0 

=== epoch 5/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  970
episode_length                      10.296907
returns                              6.852361
return_std                           2.842918
average_reward                        0.66539
round_time             0 days 00:10:24.912125
episodes_test                           988.0
episode_length_test                 10.116397
returns_test                           6.9349
return_std_test                       3.17618
average_reward_test                  0.685562
round_time_test        0 days 00:00:09.475799
round_time_total       0 days 00:10:24.913226
loss_total             11096716354230890496.0
loss_critic            13870895196704167936.0
loss_actor                    -9667920105.216
memory_size                            3139.0 

=== epoch 5/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:50,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  982
episode_length                      10.174134
returns                              6.802783
return_std                           2.700209
average_reward                       0.668651
round_time             0 days 00:10:24.035417
episodes_test                           999.0
episode_length_test                 10.003003
returns_test                         6.783356
return_std_test                      2.838404
average_reward_test                  0.678199
round_time_test        0 days 00:00:09.553860
round_time_total       0 days 00:10:24.036748
loss_total             11138890294249705472.0
loss_critic            13923612619528660992.0
loss_actor                    -9797874306.048
memory_size                         3139.4055 

=== epoch 5/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  976
episode_length                      10.232582
returns                              6.934563
return_std                           2.949397
average_reward                       0.677845
round_time             0 days 00:10:24.928148
episodes_test                           955.0
episode_length_test                 10.463874
returns_test                         7.470875
return_std_test                      3.697107
average_reward_test                  0.714041
round_time_test        0 days 00:00:09.203048
round_time_total       0 days 00:10:24.929232
loss_total             11280792774548289536.0
loss_critic            14100990719627014144.0
loss_actor                    -9828810243.584
memory_size                          3140.697 

=== epoch 5/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:36,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  966
episode_length                      10.338509
returns                              7.214216
return_std                           3.244792
average_reward                       0.697757
round_time             0 days 00:10:28.639814
episodes_test                           966.0
episode_length_test                  10.34058
returns_test                         7.867064
return_std_test                      4.088096
average_reward_test                   0.76088
round_time_test        0 days 00:00:09.409804
round_time_total       0 days 00:10:28.640942
loss_total             11405007199085096960.0
loss_critic            14256258764591675392.0
loss_actor                     -9912026723.84
memory_size                         3142.9235 

=== epoch 5/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:47,  2.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  973
episode_length                      10.261048
returns                              7.283161
return_std                           3.095686
average_reward                       0.709968
round_time             0 days 00:10:24.849874
episodes_test                           945.0
episode_length_test                  10.57672
returns_test                         8.104222
return_std_test                      3.913344
average_reward_test                  0.766295
round_time_test        0 days 00:00:09.432155
round_time_total       0 days 00:10:24.851004
loss_total             11492870283492542464.0
loss_critic            14366087600378492928.0
loss_actor                    -9989693634.048
memory_size                            3143.0 

=== epoch 5/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  980
episode_length                       10.20102
returns                              7.364814
return_std                           3.056443
average_reward                       0.721795
round_time             0 days 00:10:26.646450
episodes_test                           972.0
episode_length_test                 10.287037
returns_test                         7.854288
return_std_test                      3.811301
average_reward_test                  0.763536
round_time_test        0 days 00:00:09.483714
round_time_total       0 days 00:10:26.647580
loss_total             11597180261950984192.0
loss_critic            14496475073520263168.0
loss_actor                -10049761760.256001
memory_size                            3143.0 

=== epoch 5/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:58,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  967
episode_length                      10.332989
returns                              7.633833
return_std                           3.523234
average_reward                       0.738754
round_time             0 days 00:10:26.904582
episodes_test                           957.0
episode_length_test                 10.446186
returns_test                         7.824542
return_std_test                      4.151615
average_reward_test                  0.749098
round_time_test        0 days 00:00:09.682739
round_time_total       0 days 00:10:26.905675
loss_total             11650236908164462592.0
loss_critic            14562795888674455552.0
loss_actor                   -10095977623.552
memory_size                         3144.0035 

=== epoch 5/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:56,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  968
episode_length                      10.320248
returns                              7.679122
return_std                           3.622993
average_reward                       0.743899
round_time             0 days 00:10:26.079248
episodes_test                           970.0
episode_length_test                 10.304124
returns_test                          7.82376
return_std_test                      4.120751
average_reward_test                  0.759344
round_time_test        0 days 00:00:09.462081
round_time_total       0 days 00:10:26.080529
loss_total             11785798239236165632.0
loss_critic            14732247541278451712.0
loss_actor                -10135186989.568001
memory_size                            3152.0 

=== epoch 5/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:56,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  969
episode_length                      10.296182
returns                               7.64423
return_std                           3.759987
average_reward                       0.742931
round_time             0 days 00:10:27.581087
episodes_test                           955.0
episode_length_test                 10.452356
returns_test                         7.797143
return_std_test                      3.844778
average_reward_test                  0.745577
round_time_test        0 days 00:00:09.363266
round_time_total       0 days 00:10:27.582204
loss_total             12021670557356124160.0
loss_critic            15027087930922577920.0
loss_actor                   -10185306369.024
memory_size                         3161.7765 

=== epoch 5/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:23,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  967
episode_length                       10.33092
returns                              7.666939
return_std                           3.890091
average_reward                       0.742356
round_time             0 days 00:10:26.090877
episodes_test                           961.0
episode_length_test                 10.394381
returns_test                         7.799345
return_std_test                      3.500498
average_reward_test                  0.750533
round_time_test        0 days 00:00:09.494550
round_time_total       0 days 00:10:26.092380
loss_total             12095698663010564096.0
loss_critic            15119623074157543424.0
loss_actor                -10276599435.775999
memory_size                            3164.0 

=== epoch 5/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:23,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  967
episode_length                      10.334023
returns                              7.713213
return_std                           4.044894
average_reward                       0.746424
round_time             0 days 00:10:26.636689
episodes_test                           946.0
episode_length_test                 10.561311
returns_test                         7.836157
return_std_test                      3.160533
average_reward_test                  0.742125
round_time_test        0 days 00:00:09.419773
round_time_total       0 days 00:10:26.637768
loss_total             12313481498483808256.0
loss_critic            15391851600632033280.0
loss_actor                   -10359262682.112
memory_size                         3165.5405 

=== epoch 5/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:48,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  964
episode_length                      10.364108
returns                              7.739171
return_std                           4.061636
average_reward                       0.746815
round_time             0 days 00:10:26.314343
episodes_test                           928.0
episode_length_test                 10.772629
returns_test                         8.210529
return_std_test                      4.356245
average_reward_test                  0.762222
round_time_test        0 days 00:00:09.433033
round_time_total       0 days 00:10:26.315452
loss_total             12435730460773814272.0
loss_critic            15544662808030027776.0
loss_actor                -10430651489.280001
memory_size                         3166.6795 

=== epoch 5/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  954
episode_length                      10.473795
returns                              7.861353
return_std                           4.054738
average_reward                       0.750612
round_time             0 days 00:10:28.025866
episodes_test                           919.0
episode_length_test                   10.8716
returns_test                         8.549357
return_std_test                      4.224922
average_reward_test                  0.786353
round_time_test        0 days 00:00:09.630424
round_time_total       0 days 00:10:28.027156
loss_total             12489245644705202176.0
loss_critic            15611556785126762496.0
loss_actor                -10464712500.224001
memory_size                         3175.1455 

=== epoch 5/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:19,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  953
episode_length                      10.485834
returns                              7.934036
return_std                           4.087069
average_reward                         0.7566
round_time             0 days 00:10:26.820195
episodes_test                           886.0
episode_length_test                 11.278781
returns_test                         8.697577
return_std_test                       4.13578
average_reward_test                  0.771275
round_time_test        0 days 00:00:09.372160
round_time_total       0 days 00:10:26.821310
loss_total             12711363364969213952.0
loss_critic            15889203941297934336.0
loss_actor                   -10500055474.688
memory_size                          3192.459 

=== epoch 5/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:05,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  945
episode_length                      10.572487
returns                              8.046242
return_std                           4.172191
average_reward                       0.761066
round_time             0 days 00:10:25.801713
episodes_test                           928.0
episode_length_test                 10.773707
returns_test                          8.26138
return_std_test                      3.595946
average_reward_test                  0.766861
round_time_test        0 days 00:00:09.468980
round_time_total       0 days 00:10:25.802817
loss_total             12799921479263879168.0
loss_critic            15999901582585718784.0
loss_actor                -10617937091.072001
memory_size                            3197.0 

=== epoch 5/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  929
episode_length                      10.755651
returns                              8.230773
return_std                            4.08303
average_reward                       0.765286
round_time             0 days 00:10:27.273217
episodes_test                           919.0
episode_length_test                 10.881393
returns_test                         8.418096
return_std_test                       4.41164
average_reward_test                  0.773623
round_time_test        0 days 00:00:09.353087
round_time_total       0 days 00:10:27.274459
loss_total             13036591782928541696.0
loss_critic            16295739459349047296.0
loss_actor                -10697882926.591999
memory_size                            3197.0 

=== epoch 5/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  926
episode_length                      10.789417
returns                              8.274723
return_std                           3.879447
average_reward                       0.766942
round_time             0 days 00:10:27.315347
episodes_test                           928.0
episode_length_test                 10.773707
returns_test                         8.344809
return_std_test                      4.535288
average_reward_test                  0.774599
round_time_test        0 days 00:00:09.511355
round_time_total       0 days 00:10:27.316491
loss_total             13394590839014621184.0
loss_critic            16743238263857326080.0
loss_actor                -10829504219.648001
memory_size                          3201.627 

=== epoch 5/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:09,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  921
episode_length                      10.834962
returns                              8.327477
return_std                           3.916844
average_reward                       0.768706
round_time             0 days 00:10:25.333011
episodes_test                           902.0
episode_length_test                 11.080931
returns_test                         8.427761
return_std_test                      4.066881
average_reward_test                  0.760554
round_time_test        0 days 00:00:09.131766
round_time_total       0 days 00:10:25.334101
loss_total             13713610603708577792.0
loss_critic            17142012993432991744.0
loss_actor                -10857899358.719999
memory_size                         3207.8865 

=== epoch 5/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:09,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  913
episode_length                      10.943045
returns                              8.387174
return_std                           3.799268
average_reward                       0.766399
round_time             0 days 00:10:28.114186
episodes_test                           895.0
episode_length_test                 11.168715
returns_test                         8.353639
return_std_test                      3.476753
average_reward_test                  0.748004
round_time_test        0 days 00:00:09.454849
round_time_total       0 days 00:10:28.115522
loss_total             13814931609010055168.0
loss_critic            17268664222434609152.0
loss_actor                -10925134536.191999
memory_size                            3213.0 

=== epoch 5/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:47,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  903
episode_length                      11.066445
returns                              8.398027
return_std                           3.652498
average_reward                        0.75869
round_time             0 days 00:10:28.341637
episodes_test                           895.0
episode_length_test                 11.167598
returns_test                         8.570636
return_std_test                      4.903152
average_reward_test                  0.767477
round_time_test        0 days 00:00:09.530451
round_time_total       0 days 00:10:28.342884
loss_total             13923818056129484800.0
loss_critic            17404772280577982464.0
loss_actor                    -11027457236.48
memory_size                            3213.0 

=== epoch 5/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:29<00:00,  3.18it/s]
episodes                                  903
episode_length                      11.058693
returns                              8.434522
return_std                            4.05915
average_reward                       0.762832
round_time             0 days 00:10:30.191835
episodes_test                           902.0
episode_length_test                 11.079823
returns_test                         8.364255
return_std_test                      4.182652
average_reward_test                  0.755044
round_time_test        0 days 00:00:09.444187
round_time_total       0 days 00:10:30.193354
loss_total             14117317763612045312.0
loss_critic            17646646929293553664.0
loss_actor                    -11080209927.68
memory_size                          3221.091 

=== epoch 5/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:58,  2.78it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  904
episode_length                       11.05531
returns                              8.421965
return_std                            4.27262
average_reward                       0.761817
round_time             0 days 00:10:27.873273
episodes_test                           931.0
episode_length_test                 10.737916
returns_test                         8.185508
return_std_test                      3.911163
average_reward_test                  0.762375
round_time_test        0 days 00:00:09.431307
round_time_total       0 days 00:10:27.874408
loss_total             14322845589816655872.0
loss_critic            17903556695831519232.0
loss_actor                -11156255948.799999
memory_size                         3226.6865 

=== epoch 5/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<10:14,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  908
episode_length                      11.003304
returns                              8.309927
return_std                           4.077168
average_reward                       0.755119
round_time             0 days 00:10:29.039486
episodes_test                           915.0
episode_length_test                 10.925683
returns_test                         8.382004
return_std_test                      4.701912
average_reward_test                  0.767249
round_time_test        0 days 00:00:09.407959
round_time_total       0 days 00:10:29.040617
loss_total             14249045610422429696.0
loss_critic            17811306714510630912.0
loss_actor                -11232001014.271999
memory_size                            3227.0 

=== epoch 5/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  910
episode_length                      10.986813
returns                              8.277511
return_std                           3.911321
average_reward                        0.75345
round_time             0 days 00:10:26.584058
episodes_test                           924.0
episode_length_test                 10.818182
returns_test                         8.056286
return_std_test                      4.428937
average_reward_test                  0.744773
round_time_test        0 days 00:00:09.347527
round_time_total       0 days 00:10:26.585139
loss_total             14499361005120397312.0
loss_critic            18124200955615346688.0
loss_actor                -11296145115.648001
memory_size                         3227.8395 

=== epoch 5/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  910
episode_length                      10.976923
returns                              8.313898
return_std                           4.024628
average_reward                       0.757305
round_time             0 days 00:10:28.203260
episodes_test                           930.0
episode_length_test                 10.748387
returns_test                         7.962526
return_std_test                      3.406411
average_reward_test                  0.740908
round_time_test        0 days 00:00:09.201859
round_time_total       0 days 00:10:28.204362
loss_total             14850925323061667840.0
loss_critic            18563656350980349952.0
loss_actor                    -11389906158.08
memory_size                         3229.9805 

=== epoch 5/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  911
episode_length                      10.970362
returns                              8.223575
return_std                           3.638964
average_reward                       0.749609
round_time             0 days 00:10:27.688529
episodes_test                           939.0
episode_length_test                 10.644302
returns_test                         7.823245
return_std_test                      2.953153
average_reward_test                  0.735034
round_time_test        0 days 00:00:09.327214
round_time_total       0 days 00:10:27.689620
loss_total             14999542294646562816.0
loss_critic            18749427573089333248.0
loss_actor                -11488434004.992001
memory_size                            3230.0 

=== epoch 5/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  907
episode_length                      11.017641
returns                              8.206883
return_std                           3.362393
average_reward                       0.744887
round_time             0 days 00:10:28.639711
episodes_test                           914.0
episode_length_test                  10.93326
returns_test                         8.204232
return_std_test                      4.100299
average_reward_test                  0.750455
round_time_test        0 days 00:00:09.526383
round_time_total       0 days 00:10:28.640861
loss_total             15268556871667251200.0
loss_critic            19085695774711418880.0
loss_actor                    -11592653816.32
memory_size                         3232.2575 

=== epoch 5/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:45,  2.83it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  897
episode_length                      11.128205
returns                              8.354006
return_std                           3.718258
average_reward                        0.75047
round_time             0 days 00:10:27.836550
episodes_test                           906.0
episode_length_test                 11.034216
returns_test                         8.358422
return_std_test                       4.36189
average_reward_test                  0.757569
round_time_test        0 days 00:00:09.206419
round_time_total       0 days 00:10:27.837645
loss_total             15535002760002400256.0
loss_critic            19418753117332013056.0
loss_actor                -11684028989.440001
memory_size                            3237.0 

=== epoch 5/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.18it/s]
episodes                                  898
episode_length                      11.110245
returns                               8.33649
return_std                           3.739252
average_reward                       0.750041
round_time             0 days 00:10:28.538369
episodes_test                           885.0
episode_length_test                 11.298305
returns_test                         8.794293
return_std_test                      4.834562
average_reward_test                  0.778393
round_time_test        0 days 00:00:09.454342
round_time_total       0 days 00:10:28.539477
loss_total             15961869715074480128.0
loss_critic            19952336827115032576.0
loss_actor                -11794561490.431999
memory_size                            3237.0 

=== epoch 5/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:36,  2.87it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  893
episode_length                       11.18701
returns                              8.476583
return_std                           3.964017
average_reward                       0.757679
round_time             0 days 00:10:29.527879
episodes_test                           872.0
episode_length_test                 11.441514
returns_test                         9.039294
return_std_test                      5.205476
average_reward_test                  0.790074
round_time_test        0 days 00:00:09.321864
round_time_total       0 days 00:10:29.528995
loss_total             16018629542976026624.0
loss_critic            20023286601615327232.0
loss_actor                   -11854740431.872
memory_size                            3237.0 

=== epoch 5/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:56,  3.34it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  883
episode_length                      11.306908
returns                              8.704789
return_std                           4.761089
average_reward                       0.769698
round_time             0 days 00:10:28.638013
episodes_test                           897.0
episode_length_test                 11.142698
returns_test                         8.452982
return_std_test                      4.476772
average_reward_test                   0.75875
round_time_test        0 days 00:00:09.367418
round_time_total       0 days 00:10:28.639352
loss_total             16206151524046985216.0
loss_critic            20257689070463598592.0
loss_actor                   -11974637437.952
memory_size                          3257.118 

=== epoch 5/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:37,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  879
episode_length                      11.367463
returns                              8.830777
return_std                           5.097809
average_reward                       0.776809
round_time             0 days 00:10:27.495137
episodes_test                           896.0
episode_length_test                 11.151786
returns_test                          8.60665
return_std_test                      4.964885
average_reward_test                  0.771729
round_time_test        0 days 00:00:09.516478
round_time_total       0 days 00:10:27.496216
loss_total             16763358809050527744.0
loss_critic            20954198155140112384.0
loss_actor                -12031383450.624001
memory_size                          3264.089 

=== epoch 5/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  880
episode_length                      11.351136
returns                              8.862445
return_std                           5.200954
average_reward                       0.780545
round_time             0 days 00:10:29.159423
episodes_test                           867.0
episode_length_test                 11.534025
returns_test                          8.93139
return_std_test                      4.910261
average_reward_test                  0.774352
round_time_test        0 days 00:00:09.270489
round_time_total       0 days 00:10:29.160598
loss_total             17067125278455400448.0
loss_critic            21333906224716333056.0
loss_actor                   -12071022450.688
memory_size                          3271.572 

=== epoch 5/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:09,  2.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  869
episode_length                      11.497123
returns                              8.980394
return_std                           5.483461
average_reward                       0.781149
round_time             0 days 00:10:28.681571
episodes_test                           878.0
episode_length_test                  11.38041
returns_test                         8.905397
return_std_test                      4.488562
average_reward_test                  0.782738
round_time_test        0 days 00:00:09.462466
round_time_total       0 days 00:10:28.682665
loss_total             17051709108110848000.0
loss_critic            21314636007456317440.0
loss_actor                   -12160437630.976
memory_size                          3276.743 

=== epoch 5/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  870
episode_length                      11.487356
returns                              8.953862
return_std                           5.304256
average_reward                       0.779258
round_time             0 days 00:10:27.661573
episodes_test                           868.0
episode_length_test                 11.519585
returns_test                         8.985357
return_std_test                      5.282452
average_reward_test                  0.780028
round_time_test        0 days 00:00:09.346608
round_time_total       0 days 00:10:27.663021
loss_total             17361939684165863424.0
loss_critic            21702424239550992384.0
loss_actor                   -12208625953.792
memory_size                          3285.265 

=== epoch 5/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:22,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:30<00:00,  3.17it/s]
episodes                                  873
episode_length                      11.446735
returns                              8.867822
return_std                           4.700146
average_reward                       0.774585
round_time             0 days 00:10:30.865826
episodes_test                           883.0
episode_length_test                 11.325028
returns_test                         8.805726
return_std_test                      5.217256
average_reward_test                  0.777546
round_time_test        0 days 00:00:09.207634
round_time_total       0 days 00:10:30.867111
loss_total             17688914126590099456.0
loss_critic            22111142258221551616.0
loss_actor                   -12232766767.104
memory_size                          3291.817 

=== epoch 5/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:03,  2.76it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  875
episode_length                      11.420571
returns                              8.775795
return_std                           4.401826
average_reward                        0.76829
round_time             0 days 00:10:29.307156
episodes_test                           895.0
episode_length_test                  11.17095
returns_test                         8.465871
return_std_test                      4.196727
average_reward_test                  0.757894
round_time_test        0 days 00:00:09.533808
round_time_total       0 days 00:10:29.308426
loss_total             17705669331947859968.0
loss_critic            22132086281273987072.0
loss_actor                   -12340762929.664
memory_size                            3292.0 

=== epoch 5/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:24,  2.92it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:29<00:00,  3.18it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  876
episode_length                      11.406393
returns                              8.773717
return_std                           4.126009
average_reward                       0.769065
round_time             0 days 00:10:30.143514
episodes_test                           868.0
episode_length_test                 11.518433
returns_test                         9.188189
return_std_test                       4.95247
average_reward_test                   0.79775
round_time_test        0 days 00:00:09.412654
round_time_total       0 days 00:10:30.144977
loss_total             17827882039216330752.0
loss_critic            22284852158556344320.0
loss_actor                -12414977356.799999
memory_size                            3292.0 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 6/10 ===== round 1/50 ======================================
  0%|          | 7/2000 [00:02<10:32,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  175
episode_length                      11.354286
returns                              8.762681
return_std                           4.192231
average_reward                       0.771799
round_time             0 days 00:10:26.139192
episodes_test                           871.0
episode_length_test                 11.475316
returns_test                         8.973107
return_std_test                      4.675116
average_reward_test                  0.782003
round_time_test        0 days 00:00:09.406800
round_time_total       0 days 00:10:26.140291
loss_total             18090513841836369920.0
loss_critic            22613141890116042752.0
loss_actor                   -12513346100.736
memory_size                         3292.7055 

=== epoch 6/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:23,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  347
episode_length                      11.495677
returns                                9.1308
return_std                           5.292705
average_reward                        0.79553
round_time             0 days 00:10:25.783243
episodes_test                           869.0
episode_length_test                 11.504028
returns_test                          9.03051
return_std_test                      5.320649
average_reward_test                  0.785053
round_time_test        0 days 00:00:09.372769
round_time_total       0 days 00:10:25.784328
loss_total             18696943305553801216.0
loss_critic            23371178720518742016.0
loss_actor                   -12550275401.216
memory_size                         3298.2345 

=== epoch 6/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:18,  2.70it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  523
episode_length                      11.458891
returns                              9.146367
return_std                           5.740598
average_reward                       0.798199
round_time             0 days 00:10:27.614086
episodes_test                           863.0
episode_length_test                 11.587486
returns_test                         9.187827
return_std_test                      4.778771
average_reward_test                  0.792909
round_time_test        0 days 00:00:09.447551
round_time_total       0 days 00:10:27.615176
loss_total             18889649425125851136.0
loss_critic            23612061350811070464.0
loss_actor                -12598847882.752001
memory_size                          3315.171 

=== epoch 6/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:11,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  693
episode_length                      11.518038
returns                              9.151559
return_std                           5.438412
average_reward                        0.79535
round_time             0 days 00:10:27.339899
episodes_test                           855.0
episode_length_test                 11.694737
returns_test                         9.171912
return_std_test                      4.594256
average_reward_test                  0.784297
round_time_test        0 days 00:00:09.345805
round_time_total       0 days 00:10:27.341201
loss_total             18910924686226636800.0
loss_critic            23638655412274925568.0
loss_actor                -12675547560.959999
memory_size                            3323.0 

=== epoch 6/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  868
episode_length                      11.506912
returns                              9.215171
return_std                           5.510907
average_reward                       0.800838
round_time             0 days 00:10:28.681493
episodes_test                           863.0
episode_length_test                 11.580533
returns_test                         9.172029
return_std_test                      4.687581
average_reward_test                  0.792045
round_time_test        0 days 00:00:09.242790
round_time_total       0 days 00:10:28.682969
loss_total             19043725541211885568.0
loss_critic            23804656509593792512.0
loss_actor                -12717225547.775999
memory_size                          3324.109 

=== epoch 6/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  866
episode_length                      11.540416
returns                              9.227613
return_std                           5.515592
average_reward                       0.799674
round_time             0 days 00:10:23.955978
episodes_test                           858.0
episode_length_test                 11.642191
returns_test                         9.297743
return_std_test                      5.322994
average_reward_test                  0.798718
round_time_test        0 days 00:00:09.293585
round_time_total       0 days 00:10:23.957097
loss_total             19251442346908205056.0
loss_critic            24064302497678897152.0
loss_actor                   -12804354850.816
memory_size                            3327.0 

=== epoch 6/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:53,  2.79it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  863
episode_length                      11.578216
returns                              9.128577
return_std                           5.114294
average_reward                       0.788604
round_time             0 days 00:10:26.354572
episodes_test                           852.0
episode_length_test                 11.712441
returns_test                         9.307759
return_std_test                      4.786325
average_reward_test                  0.794898
round_time_test        0 days 00:00:09.302773
round_time_total       0 days 00:10:26.355680
loss_total             19588600618586140672.0
loss_critic            24485750335970643968.0
loss_actor                -12860582254.591999
memory_size                         3327.7285 

=== epoch 6/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  863
episode_length                      11.573581
returns                              9.109013
return_std                           4.566135
average_reward                       0.787006
round_time             0 days 00:10:27.974295
episodes_test                           853.0
episode_length_test                 11.723329
returns_test                         9.304553
return_std_test                      4.541341
average_reward_test                  0.793678
round_time_test        0 days 00:00:09.397961
round_time_total       0 days 00:10:27.975424
loss_total             19387494713688121344.0
loss_critic            24234367954573242368.0
loss_actor                -12952066671.615999
memory_size                            3328.0 

=== epoch 6/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:20,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  855
episode_length                      11.676023
returns                                9.2949
return_std                           5.029685
average_reward                       0.796291
round_time             0 days 00:10:27.840129
episodes_test                           858.0
episode_length_test                 11.644522
returns_test                         9.230484
return_std_test                       4.60298
average_reward_test                  0.792594
round_time_test        0 days 00:00:09.334565
round_time_total       0 days 00:10:27.841225
loss_total             19820724067011362816.0
loss_critic            24775904656260337664.0
loss_actor                    -12875630791.68
memory_size                         3339.8065 

=== epoch 6/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:20,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  849
episode_length                      11.765607
returns                              9.310312
return_std                           4.868122
average_reward                       0.791474
round_time             0 days 00:10:27.997585
episodes_test                           874.0
episode_length_test                 11.439359
returns_test                         8.879677
return_std_test                       3.81429
average_reward_test                  0.776287
round_time_test        0 days 00:00:09.408673
round_time_total       0 days 00:10:27.998810
loss_total             19407579382916419584.0
loss_critic            24259473805058670592.0
loss_actor                   -12864228375.552
memory_size                            3343.0 

=== epoch 6/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:11,  2.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  849
episode_length                      11.751472
returns                              9.300432
return_std                           4.756032
average_reward                       0.791064
round_time             0 days 00:10:29.156448
episodes_test                           867.0
episode_length_test                 11.527105
returns_test                         8.833093
return_std_test                      4.732676
average_reward_test                  0.766459
round_time_test        0 days 00:00:09.501491
round_time_total       0 days 00:10:29.157564
loss_total             19886573286235238400.0
loss_critic            24858216164622143488.0
loss_actor                -12987977759.743999
memory_size                            3343.0 

=== epoch 6/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  847
episode_length                      11.782763
returns                              9.510027
return_std                           5.189623
average_reward                       0.806805
round_time             0 days 00:10:27.891332
episodes_test                           853.0
episode_length_test                 11.717468
returns_test                          9.34315
return_std_test                      4.915879
average_reward_test                  0.797463
round_time_test        0 days 00:00:09.363675
round_time_total       0 days 00:10:27.892775
loss_total             19913396869651881984.0
loss_critic            24891745625201246208.0
loss_actor                   -13038743655.424
memory_size                          3377.083 

=== epoch 6/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:10,  2.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:28<00:00,  3.18it/s]
episodes                                  837
episode_length                      11.934289
returns                               9.67943
return_std                           5.451509
average_reward                        0.81118
round_time             0 days 00:10:29.575175
episodes_test                           858.0
episode_length_test                 11.645688
returns_test                         9.210728
return_std_test                      4.372615
average_reward_test                  0.790969
round_time_test        0 days 00:00:09.344000
round_time_total       0 days 00:10:29.576295
loss_total             20079245296207814656.0
loss_critic            25099056175369879552.0
loss_actor                   -13125808167.424
memory_size                          3384.848 

=== epoch 6/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:54,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  840
episode_length                           11.9
returns                              9.620013
return_std                           5.458866
average_reward                       0.808447
round_time             0 days 00:10:27.963959
episodes_test                           860.0
episode_length_test                  11.62093
returns_test                         9.386266
return_std_test                      5.395618
average_reward_test                  0.807657
round_time_test        0 days 00:00:09.296526
round_time_total       0 days 00:10:27.965060
loss_total             20302077417554907136.0
loss_critic            25377596324648558592.0
loss_actor                -13136110459.903999
memory_size                         3406.3665 

=== epoch 6/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:03,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  842
episode_length                      11.866983
returns                              9.639496
return_std                           5.747427
average_reward                       0.812272
round_time             0 days 00:10:28.432548
episodes_test                           856.0
episode_length_test                 11.681075
returns_test                         9.188209
return_std_test                       4.65189
average_reward_test                  0.786612
round_time_test        0 days 00:00:09.400879
round_time_total       0 days 00:10:28.433894
loss_total             20446901725484613632.0
loss_critic            25558626706880630784.0
loss_actor                -13184458514.431999
memory_size                         3424.8925 

=== epoch 6/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:10,  2.73it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:29<00:00,  3.18it/s]
episodes                                  835
episode_length                       11.97485
returns                              9.855544
return_std                           6.077948
average_reward                       0.822974
round_time             0 days 00:10:29.819631
episodes_test                           850.0
episode_length_test                 11.761176
returns_test                         9.263312
return_std_test                      4.958397
average_reward_test                  0.787664
round_time_test        0 days 00:00:09.442789
round_time_total       0 days 00:10:29.820743
loss_total             20606038833944797184.0
loss_critic            25757548081735622656.0
loss_actor                   -13220110533.632
memory_size                         3433.1565 

=== epoch 6/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  839
episode_length                       11.90584
returns                              9.650911
return_std                            5.76917
average_reward                       0.810455
round_time             0 days 00:10:27.320877
episodes_test                           825.0
episode_length_test                 12.118788
returns_test                         9.677801
return_std_test                      6.187016
average_reward_test                  0.798588
round_time_test        0 days 00:00:09.266231
round_time_total       0 days 00:10:27.321984
loss_total             20870219327279861760.0
loss_critic            26087773695380795392.0
loss_actor                -13239934141.440001
memory_size                          3436.523 

=== epoch 6/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:36,  2.86it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  840
episode_length                      11.864286
returns                              9.449625
return_std                           5.371434
average_reward                       0.797639
round_time             0 days 00:10:27.623530
episodes_test                           857.0
episode_length_test                 11.668611
returns_test                          9.24171
return_std_test                      5.350755
average_reward_test                  0.792015
round_time_test        0 days 00:00:09.355089
round_time_total       0 days 00:10:27.624627
loss_total             21008855230550335488.0
loss_critic            26261068601544364032.0
loss_actor                -13323850988.544001
memory_size                            3437.0 

=== epoch 6/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:02,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:26<00:00,  3.19it/s]
episodes                                  852
episode_length                      11.717136
returns                              9.183033
return_std                           4.778342
average_reward                         0.7841
round_time             0 days 00:10:27.047509
episodes_test                           850.0
episode_length_test                 11.756471
returns_test                          9.21821
return_std_test                      5.460008
average_reward_test                  0.784195
round_time_test        0 days 00:00:09.306384
round_time_total       0 days 00:10:27.048612
loss_total             21149261147430412288.0
loss_critic            26436575955450699776.0
loss_actor                   -13393877473.792
memory_size                           3441.99 

=== epoch 6/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:10,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:27<00:00,  3.19it/s]
episodes                                  855
episode_length                      11.671345
returns                              9.046593
return_std                           4.593002
average_reward                       0.774982
round_time             0 days 00:10:28.415780
episodes_test                           845.0
episode_length_test                 11.827219
returns_test                         9.129881
return_std_test                      4.949194
average_reward_test                  0.772002
round_time_test        0 days 00:00:09.350337
round_time_total       0 days 00:10:28.416878
loss_total             21321802855693357056.0
loss_critic            26652253114556321792.0
loss_actor                -13412579089.408001
memory_size                         3452.4475 

=== epoch 6/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:10,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  858
episode_length                      11.651515
returns                              8.958828
return_std                           4.258913
average_reward                       0.768968
round_time             0 days 00:10:25.769183
episodes_test                           857.0
episode_length_test                 11.668611
returns_test                         8.831942
return_std_test                      4.563921
average_reward_test                  0.756897
round_time_test        0 days 00:00:09.396373
round_time_total       0 days 00:10:25.770262
loss_total             21565805542644035584.0
loss_critic            26957256437785419776.0
loss_actor                -13442380998.143999
memory_size                            3456.0 

=== epoch 6/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:16,  2.95it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  849
episode_length                      11.762073
returns                              9.107194
return_std                           4.993313
average_reward                       0.774372
round_time             0 days 00:10:22.744937
episodes_test                           853.0
episode_length_test                  11.71864
returns_test                         9.239487
return_std_test                       5.24919
average_reward_test                   0.78848
round_time_test        0 days 00:00:09.049336
round_time_total       0 days 00:10:22.746029
loss_total             21844735235148541952.0
loss_critic            27305918556302270464.0
loss_actor                     -13488679360.0
memory_size                          3461.685 

=== epoch 6/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:24,  2.91it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  850
episode_length                      11.742353
returns                              9.104823
return_std                           5.033664
average_reward                       0.775399
round_time             0 days 00:10:23.150299
episodes_test                           852.0
episode_length_test                 11.731221
returns_test                         9.205695
return_std_test                      4.472792
average_reward_test                  0.784701
round_time_test        0 days 00:00:09.373995
round_time_total       0 days 00:10:23.151390
loss_total             22018373076514910208.0
loss_critic            27522965836638474240.0
loss_actor                   -13530749058.048
memory_size                            3495.0 

=== epoch 6/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:36,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  844
episode_length                      11.838863
returns                              9.225372
return_std                           5.150528
average_reward                       0.779272
round_time             0 days 00:10:23.991913
episodes_test                           877.0
episode_length_test                 11.400228
returns_test                         8.661904
return_std_test                      4.437487
average_reward_test                  0.759825
round_time_test        0 days 00:00:09.277158
round_time_total       0 days 00:10:23.992985
loss_total             22026394380251787264.0
loss_critic            27532992473387671552.0
loss_actor                   -13522276343.296
memory_size                         3495.4925 

=== epoch 6/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:46,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:19<00:00,  3.23it/s]
episodes                                  841
episode_length                      11.882283
returns                              9.208318
return_std                           4.917721
average_reward                       0.774946
round_time             0 days 00:10:20.235561
episodes_test                           850.0
episode_length_test                 11.757647
returns_test                         9.192894
return_std_test                      5.189129
average_reward_test                  0.781989
round_time_test        0 days 00:00:09.333729
round_time_total       0 days 00:10:20.236649
loss_total             21814021212355944448.0
loss_critic            27267526021489332224.0
loss_actor                   -13535038321.664
memory_size                         3497.0455 

=== epoch 6/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:52,  2.80it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  839
episode_length                       11.90584
returns                                9.2939
return_std                           5.905574
average_reward                       0.780501
round_time             0 days 00:10:24.027804
episodes_test                           857.0
episode_length_test                 11.662777
returns_test                         8.980627
return_std_test                      4.899719
average_reward_test                  0.770032
round_time_test        0 days 00:00:09.309468
round_time_total       0 days 00:10:24.029263
loss_total             21786525203911639040.0
loss_critic            27233156020279799808.0
loss_actor                -13491494470.143999
memory_size                          3524.348 

=== epoch 6/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:47,  2.82it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  847
episode_length                      11.787485
returns                              9.079108
return_std                           5.362973
average_reward                        0.77075
round_time             0 days 00:10:23.835065
episodes_test                           830.0
episode_length_test                 12.036145
returns_test                          9.01906
return_std_test                      4.792917
average_reward_test                  0.749472
round_time_test        0 days 00:00:09.047385
round_time_total       0 days 00:10:23.836536
loss_total             21965119364694945792.0
loss_critic            27456398726481612800.0
loss_actor                   -13500952846.336
memory_size                         3528.9295 

=== epoch 6/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:47,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.21it/s]
episodes                                  841
episode_length                      11.889417
returns                              9.239591
return_std                           5.804134
average_reward                       0.777046
round_time             0 days 00:10:24.563813
episodes_test                           837.0
episode_length_test                 11.940263
returns_test                         9.099942
return_std_test                      5.413286
average_reward_test                  0.762143
round_time_test        0 days 00:00:09.398061
round_time_total       0 days 00:10:24.564876
loss_total             21850111922874667008.0
loss_critic            27312639414035783680.0
loss_actor                -13527354248.191999
memory_size                          3535.388 

=== epoch 6/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:57,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:22<00:00,  3.21it/s]
episodes                                  837
episode_length                      11.939068
returns                               9.30886
return_std                           6.164747
average_reward                        0.77957
round_time             0 days 00:10:23.217231
episodes_test                           832.0
episode_length_test                 12.010817
returns_test                         9.408794
return_std_test                      5.386073
average_reward_test                  0.783452
round_time_test        0 days 00:00:09.026891
round_time_total       0 days 00:10:23.218442
loss_total             22023321039643475968.0
loss_critic            27529150839133851648.0
loss_actor                -13543443254.271999
memory_size                          3557.488 

=== epoch 6/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:18,  2.94it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:21<00:00,  3.22it/s]
episodes                                  838
episode_length                      11.916468
returns                              9.397356
return_std                           6.157024
average_reward                       0.788679
round_time             0 days 00:10:22.506562
episodes_test                           839.0
episode_length_test                 11.889154
returns_test                         8.859699
return_std_test                      4.457407
average_reward_test                  0.746591
round_time_test        0 days 00:00:09.337211
round_time_total       0 days 00:10:22.507680
loss_total             21907004695750787072.0
loss_critic            27383755384116662272.0
loss_actor                -13536687472.639999
memory_size                            3573.0 

=== epoch 6/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:01,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:24<00:00,  3.20it/s]
episodes                                  834
episode_length                      11.970024
returns                              9.334789
return_std                            5.59723
average_reward                       0.779751
round_time             0 days 00:10:25.209007
episodes_test                           834.0
episode_length_test                 11.983213
returns_test                         9.088767
return_std_test                      4.751424
average_reward_test                  0.758588
round_time_test        0 days 00:00:09.324465
round_time_total       0 days 00:10:25.210105
loss_total             22182843449413623808.0
loss_critic            27728553831417888768.0
loss_actor                -13549536070.143999
memory_size                         3596.1995 

=== epoch 6/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  836
episode_length                      11.934211
returns                              9.294501
return_std                           5.457684
average_reward                       0.780849
round_time             0 days 00:10:24.015776
episodes_test                           836.0
episode_length_test                 11.944976
returns_test                         8.951794
return_std_test                      5.448768
average_reward_test                  0.749293
round_time_test        0 days 00:00:09.291018
round_time_total       0 days 00:10:24.017048
loss_total             21785617328364490752.0
loss_critic            27232021183542444032.0
loss_actor                -13544007294.464001
memory_size                            3598.0 

=== epoch 6/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:23<00:00,  3.21it/s]
episodes                                  839
episode_length                        11.9118
returns                              9.280487
return_std                           5.772118
average_reward                       0.779001
round_time             0 days 00:10:23.792407
episodes_test                           852.0
episode_length_test                 11.726526
returns_test                         8.820887
return_std_test                       5.66327
average_reward_test                  0.752346
round_time_test        0 days 00:00:09.444912
round_time_total       0 days 00:10:23.793477
loss_total             21863289267917623296.0
loss_critic            27329111136708599808.0
loss_actor                -13528985708.544001
memory_size                         3621.5195 

=== epoch 6/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:25,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  838
episode_length                      11.922434
returns                              9.230928
return_std                           5.594345
average_reward                       0.774253
round_time             0 days 00:10:26.075732
episodes_test                           837.0
episode_length_test                 11.935484
returns_test                         9.169458
return_std_test                      5.152905
average_reward_test                   0.76831
round_time_test        0 days 00:00:09.250109
round_time_total       0 days 00:10:26.076906
loss_total             21996410069516562432.0
loss_critic            27495512107165032448.0
loss_actor                -13487136187.903999
memory_size                          3650.445 

=== epoch 6/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:45,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [10:25<00:00,  3.20it/s]
episodes                                  835
episode_length                      11.962874
returns                              9.158776
return_std                           5.656233
average_reward                       0.765653
round_time             0 days 00:10:25.648205
episodes_test                           839.0
episode_length_test                 11.915375
returns_test                         9.080835
return_std_test                      5.670908
average_reward_test                  0.762141
round_time_test        0 days 00:00:09.263975
round_time_total       0 days 00:10:25.649324
loss_total             21932754674782380032.0
loss_critic            27415942865190420480.0
loss_actor                   -13510274067.968
memory_size                          3654.509 

=== epoch 6/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:13,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:55<00:00,  3.36it/s]
episodes                                  842
episode_length                       11.86342
returns                                9.0298
return_std                           5.304243
average_reward                       0.761096
round_time             0 days 00:09:56.463953
episodes_test                           835.0
episode_length_test                  11.97006
returns_test                         9.114453
return_std_test                       5.82735
average_reward_test                  0.761505
round_time_test        0 days 00:00:09.042184
round_time_total       0 days 00:09:56.465082
loss_total             22162386082302271488.0
loss_critic            27702982100263587840.0
loss_actor                   -13501138874.368
memory_size                            3658.0 

=== epoch 6/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:42<00:00,  3.43it/s]
episodes                                  833
episode_length                      11.995198
returns                              9.118441
return_std                           5.680009
average_reward                       0.760179
round_time             0 days 00:09:43.224900
episodes_test                           834.0
episode_length_test                 11.979616
returns_test                         9.199464
return_std_test                      5.325132
average_reward_test                  0.767703
round_time_test        0 days 00:00:09.155863
round_time_total       0 days 00:09:43.226100
loss_total             21906153948079038464.0
loss_critic            27382691940868321280.0
loss_actor                -13504943349.247999
memory_size                         3673.2765 

=== epoch 6/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  837
episode_length                      11.925926
returns                              8.962999
return_std                           5.076305
average_reward                       0.751056
round_time             0 days 00:09:41.908576
episodes_test                           829.0
episode_length_test                 12.049457
returns_test                         9.395634
return_std_test                      7.215082
average_reward_test                  0.779754
round_time_test        0 days 00:00:09.119446
round_time_total       0 days 00:09:41.909786
loss_total             21614943840165593088.0
loss_critic            27018679321163522048.0
loss_actor                   -13450027620.864
memory_size                            3680.0 

=== epoch 6/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  845
episode_length                      11.827219
returns                              8.765988
return_std                           4.728114
average_reward                       0.741168
round_time             0 days 00:09:41.747135
episodes_test                           837.0
episode_length_test                 11.939068
returns_test                         8.677017
return_std_test                      5.119121
average_reward_test                  0.726851
round_time_test        0 days 00:00:09.073150
round_time_total       0 days 00:09:41.748361
loss_total             21731131250102132736.0
loss_critic            27163913593823395840.0
loss_actor                   -13449223631.872
memory_size                            3680.0 

=== epoch 6/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  840
episode_length                      11.894048
returns                              8.780146
return_std                           4.832483
average_reward                       0.738219
round_time             0 days 00:09:41.894713
episodes_test                           841.0
episode_length_test                 11.882283
returns_test                         8.793591
return_std_test                      5.469698
average_reward_test                    0.7402
round_time_test        0 days 00:00:09.176222
round_time_total       0 days 00:09:41.895784
loss_total             21995804703977951232.0
loss_critic            27494755392476471296.0
loss_actor                   -13499902258.688
memory_size                         3690.4985 

=== epoch 6/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  839
episode_length                      11.898689
returns                               8.71641
return_std                           4.860613
average_reward                       0.732663
round_time             0 days 00:09:41.991249
episodes_test                           835.0
episode_length_test                 11.966467
returns_test                         8.796518
return_std_test                       5.61594
average_reward_test                  0.735004
round_time_test        0 days 00:00:09.215041
round_time_total       0 days 00:09:41.992316
loss_total             21997728731678818304.0
loss_critic            27497160439472058368.0
loss_actor                   -13522294448.128
memory_size                         3705.8015 

=== epoch 6/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:42<00:00,  3.44it/s]
episodes                                  849
episode_length                      11.767962
returns                              8.399598
return_std                           4.169934
average_reward                       0.713934
round_time             0 days 00:09:42.619986
episodes_test                           802.0
episode_length_test                 12.453865
returns_test                         8.616773
return_std_test                      5.224118
average_reward_test                  0.691826
round_time_test        0 days 00:00:08.947984
round_time_total       0 days 00:09:42.621150
loss_total             22092843328642789376.0
loss_critic            27616053671933128704.0
loss_actor                   -13537540594.688
memory_size                         3711.6985 

=== epoch 6/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  860
episode_length                      11.619767
returns                              8.213394
return_std                           4.431119
average_reward                       0.706827
round_time             0 days 00:09:42.371061
episodes_test                           872.0
episode_length_test                 11.458716
returns_test                         7.815378
return_std_test                      4.485599
average_reward_test                  0.682033
round_time_test        0 days 00:00:09.309542
round_time_total       0 days 00:09:42.372296
loss_total             22370431439390277632.0
loss_critic            27963038810917249024.0
loss_actor                   -13555908010.496
memory_size                           3723.05 

=== epoch 6/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:16,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  860
episode_length                      11.610465
returns                              8.037018
return_std                           4.472071
average_reward                       0.692388
round_time             0 days 00:09:42.144061
episodes_test                           879.0
episode_length_test                 11.374289
returns_test                         7.816889
return_std_test                      5.639069
average_reward_test                  0.687276
round_time_test        0 days 00:00:09.133717
round_time_total       0 days 00:09:42.145292
loss_total             22537234482557292544.0
loss_critic            28171542598933094400.0
loss_actor                   -13623249136.128
memory_size                          3728.195 

=== epoch 6/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:05,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  875
episode_length                      11.406857
returns                              7.734839
return_std                           4.336104
average_reward                       0.677676
round_time             0 days 00:09:42.389209
episodes_test                           890.0
episode_length_test                 11.232584
returns_test                         7.744558
return_std_test                      4.843889
average_reward_test                  0.689557
round_time_test        0 days 00:00:09.165728
round_time_total       0 days 00:09:42.390298
loss_total             22776398068750790656.0
loss_critic            28470497098030202880.0
loss_actor                -13675114682.879999
memory_size                         3734.6605 

=== epoch 6/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:42<00:00,  3.43it/s]
episodes                                  880
episode_length                      11.357955
returns                               7.64965
return_std                           4.598402
average_reward                       0.673592
round_time             0 days 00:09:42.895904
episodes_test                           881.0
episode_length_test                 11.345062
returns_test                         7.703451
return_std_test                      4.570588
average_reward_test                  0.679052
round_time_test        0 days 00:00:09.328887
round_time_total       0 days 00:09:42.896978
loss_total             23149949352598536192.0
loss_critic            28937436166693441536.0
loss_actor                -13784817222.143999
memory_size                          3741.782 

=== epoch 6/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:42<00:00,  3.44it/s]
episodes                                  883
episode_length                      11.304643
returns                               7.63301
return_std                           4.541182
average_reward                       0.674794
round_time             0 days 00:09:42.725244
episodes_test                           889.0
episode_length_test                  11.24297
returns_test                         7.456146
return_std_test                      3.982793
average_reward_test                  0.663304
round_time_test        0 days 00:00:09.210223
round_time_total       0 days 00:09:42.726335
loss_total             23762700877608673280.0
loss_critic            29703375590135980032.0
loss_actor                    -13862466211.84
memory_size                            3767.0 

=== epoch 6/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:43<00:00,  3.43it/s]
episodes                                  877
episode_length                      11.385405
returns                              7.507234
return_std                           4.161993
average_reward                       0.658946
round_time             0 days 00:09:43.781544
episodes_test                           893.0
episode_length_test                  11.18925
returns_test                         7.269878
return_std_test                      4.180623
average_reward_test                  0.649965
round_time_test        0 days 00:00:09.061111
round_time_total       0 days 00:09:43.782611
loss_total             23995729237022814208.0
loss_critic            29994661046206885888.0
loss_actor                -13912442496.511999
memory_size                         3768.7065 

=== epoch 6/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:01,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:43<00:00,  3.43it/s]
episodes                                  873
episode_length                      11.447881
returns                              7.575576
return_std                            4.48598
average_reward                        0.66178
round_time             0 days 00:09:43.984684
episodes_test                           873.0
episode_length_test                 11.447881
returns_test                         7.378997
return_std_test                      3.988013
average_reward_test                  0.644709
round_time_test        0 days 00:00:09.213356
round_time_total       0 days 00:09:43.985853
loss_total             24186640086653415424.0
loss_critic            30233299567769366528.0
loss_actor                -14001830693.375999
memory_size                            3769.0 

=== epoch 6/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:41,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  860
episode_length                      11.611628
returns                               7.70365
return_std                           4.526771
average_reward                       0.663302
round_time             0 days 00:09:42.394557
episodes_test                           840.0
episode_length_test                 11.890476
returns_test                         7.919412
return_std_test                      4.031191
average_reward_test                  0.666448
round_time_test        0 days 00:00:09.196887
round_time_total       0 days 00:09:42.395672
loss_total             24774223743774515200.0
loss_critic            30967779172980723712.0
loss_actor                -14117533497.856001
memory_size                         3781.9565 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 7/10 ===== round 1/50 ======================================
  0%|          | 6/2000 [00:01<10:06,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  175
episode_length                      11.422857
returns                              7.544653
return_std                           3.891866
average_reward                       0.660618
round_time             0 days 00:09:38.513503
episodes_test                           870.0
episode_length_test                 11.481609
returns_test                         7.734128
return_std_test                       4.09482
average_reward_test                  0.673808
round_time_test        0 days 00:00:09.204208
round_time_total       0 days 00:09:38.514592
loss_total             24947771391377424384.0
loss_critic            31184713680807309312.0
loss_actor                   -14190195738.112
memory_size                         3789.8965 

=== epoch 7/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:15,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  351
episode_length                      11.387464
returns                              7.547875
return_std                           3.475929
average_reward                       0.663067
round_time             0 days 00:09:38.652127
episodes_test                           868.0
episode_length_test                 11.509217
returns_test                          7.73169
return_std_test                      4.041985
average_reward_test                  0.671847
round_time_test        0 days 00:00:09.329226
round_time_total       0 days 00:09:38.653194
loss_total             25395607032647262208.0
loss_critic            31744508242015334400.0
loss_actor                   -14319175822.848
memory_size                          3792.246 

=== epoch 7/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:15,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  524
episode_length                      11.438931
returns                              7.644141
return_std                           3.763498
average_reward                       0.668344
round_time             0 days 00:09:39.338891
episodes_test                           872.0
episode_length_test                 11.462156
returns_test                         7.515067
return_std_test                       3.78119
average_reward_test                  0.655775
round_time_test        0 days 00:00:09.278362
round_time_total       0 days 00:09:39.340131
loss_total             26268337231572787200.0
loss_critic            32835421000567848960.0
loss_actor                -14413363630.591999
memory_size                          3812.463 

=== epoch 7/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  698
episode_length                      11.448424
returns                              7.668134
return_std                           3.696032
average_reward                       0.669851
round_time             0 days 00:09:38.689020
episodes_test                           878.0
episode_length_test                 11.382688
returns_test                         7.574236
return_std_test                      3.624001
average_reward_test                  0.665469
round_time_test        0 days 00:00:09.119086
round_time_total       0 days 00:09:38.690335
loss_total             26493531766077218816.0
loss_critic            33116914137912061952.0
loss_actor                   -14549400829.952
memory_size                            3823.0 

=== epoch 7/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:45,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  868
episode_length                      11.514977
returns                              7.720004
return_std                           4.101125
average_reward                       0.670475
round_time             0 days 00:09:39.900967
episodes_test                           841.0
episode_length_test                 11.888228
returns_test                         7.935398
return_std_test                      4.547818
average_reward_test                  0.667544
round_time_test        0 days 00:00:09.032148
round_time_total       0 days 00:09:39.902065
loss_total             27271988415818940416.0
loss_critic            34089984944454213632.0
loss_actor                   -14673799981.056
memory_size                         3833.1035 

=== epoch 7/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:06,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  864
episode_length                      11.546296
returns                              7.776878
return_std                           4.129927
average_reward                       0.672329
round_time             0 days 00:09:40.263836
episodes_test                           848.0
episode_length_test                 11.787736
returns_test                         7.812659
return_std_test                      5.294569
average_reward_test                  0.662875
round_time_test        0 days 00:00:09.191263
round_time_total       0 days 00:09:40.264947
loss_total             27866762789375496192.0
loss_critic            34833452908238815232.0
loss_actor                -14815293050.879999
memory_size                            3834.0 

=== epoch 7/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  857
episode_length                      11.651109
returns                              7.782261
return_std                           4.232256
average_reward                       0.668327
round_time             0 days 00:09:40.390952
episodes_test                           860.0
episode_length_test                 11.626744
returns_test                         7.488958
return_std_test                      4.280935
average_reward_test                  0.644153
round_time_test        0 days 00:00:09.189265
round_time_total       0 days 00:09:40.392240
loss_total             28093762503169921024.0
loss_critic            35117202558453305344.0
loss_actor                -14992407073.280001
memory_size                            3834.0 

=== epoch 7/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.47it/s]
episodes                                  855
episode_length                      11.684211
returns                              7.778673
return_std                           4.354857
average_reward                       0.665686
round_time             0 days 00:09:37.631938
episodes_test                           864.0
episode_length_test                   11.5625
returns_test                         7.563099
return_std_test                      4.503424
average_reward_test                  0.654398
round_time_test        0 days 00:00:09.202600
round_time_total       0 days 00:09:37.633248
loss_total             28977707008215228416.0
loss_critic            36222133149627764736.0
loss_actor                   -15148626799.104
memory_size                         3835.8985 

=== epoch 7/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  848
episode_length                       11.78184
returns                              7.718397
return_std                           4.548351
average_reward                       0.655061
round_time             0 days 00:09:40.162837
episodes_test                           840.0
episode_length_test                      11.9
returns_test                         7.508567
return_std_test                      4.593413
average_reward_test                  0.631095
round_time_test        0 days 00:00:09.110362
round_time_total       0 days 00:09:40.164284
loss_total             29707926190037147648.0
loss_critic            37134907116734685184.0
loss_actor                    -15282557127.68
memory_size                         3838.0775 

=== epoch 7/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:31,  3.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  847
episode_length                      11.801653
returns                              7.670362
return_std                           4.815725
average_reward                        0.64985
round_time             0 days 00:09:39.859776
episodes_test                           850.0
episode_length_test                 11.752941
returns_test                         7.524967
return_std_test                      4.782113
average_reward_test                  0.640246
round_time_test        0 days 00:00:09.102458
round_time_total       0 days 00:09:39.861072
loss_total             30187781650601406464.0
loss_critic            37734726434468544512.0
loss_actor                   -15348119238.656
memory_size                          3854.685 

=== epoch 7/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  846
episode_length                      11.810875
returns                              7.536443
return_std                           4.727892
average_reward                       0.637876
round_time             0 days 00:09:41.468842
episodes_test                           837.0
episode_length_test                 11.936679
returns_test                         7.464084
return_std_test                      5.284115
average_reward_test                  0.625563
round_time_test        0 days 00:00:09.008051
round_time_total       0 days 00:09:41.469928
loss_total             30669214158285565952.0
loss_critic            38336517060552531968.0
loss_actor                -15486481667.072001
memory_size                          3884.903 

=== epoch 7/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:16,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  851
episode_length                      11.733255
returns                              7.407885
return_std                           4.695414
average_reward                       0.631468
round_time             0 days 00:09:40.973219
episodes_test                           855.0
episode_length_test                 11.688889
returns_test                         7.164532
return_std_test                      4.118943
average_reward_test                  0.613111
round_time_test        0 days 00:00:09.136959
round_time_total       0 days 00:09:40.974647
loss_total             31079257447458140160.0
loss_critic            38849071182188732416.0
loss_actor                -15641165377.535999
memory_size                            3887.0 

=== epoch 7/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:05,  3.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  861
episode_length                       11.59698
returns                              7.156536
return_std                           4.445099
average_reward                       0.616967
round_time             0 days 00:09:41.230892
episodes_test                           885.0
episode_length_test                 11.294915
returns_test                         6.816946
return_std_test                       4.26194
average_reward_test                  0.603648
round_time_test        0 days 00:00:09.179212
round_time_total       0 days 00:09:41.231993
loss_total             31773364037930188800.0
loss_critic            39716704405847703552.0
loss_actor                   -15794502198.784
memory_size                            3887.0 

=== epoch 7/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:08,  3.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  874
episode_length                       11.43135
returns                              6.852271
return_std                            4.20633
average_reward                       0.599415
round_time             0 days 00:09:40.724454
episodes_test                           877.0
episode_length_test                 11.388826
returns_test                         6.773831
return_std_test                      4.415319
average_reward_test                  0.595066
round_time_test        0 days 00:00:09.210002
round_time_total       0 days 00:09:40.725814
loss_total             32006776034268569600.0
loss_critic            40008469375981912064.0
loss_actor                -15920582605.823999
memory_size                          3889.961 

=== epoch 7/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:06,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  891
episode_length                      11.210999
returns                              6.598646
return_std                           3.578134
average_reward                       0.588686
round_time             0 days 00:09:39.571471
episodes_test                           887.0
episode_length_test                 11.264938
returns_test                         6.305593
return_std_test                      3.392157
average_reward_test                  0.559959
round_time_test        0 days 00:00:09.174679
round_time_total       0 days 00:09:39.572721
loss_total             33095814360791212032.0
loss_critic            41369767212117204992.0
loss_actor                   -16065034408.448
memory_size                            3891.0 

=== epoch 7/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:23,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  908
episode_length                      10.988987
returns                              6.354461
return_std                            3.45814
average_reward                       0.577934
round_time             0 days 00:09:41.404347
episodes_test                           891.0
episode_length_test                 11.219978
returns_test                         6.337482
return_std_test                      4.138267
average_reward_test                  0.564967
round_time_test        0 days 00:00:09.215205
round_time_total       0 days 00:09:41.405418
loss_total             33410181761659682816.0
loss_critic            41762726483131441152.0
loss_actor                -16218917360.639999
memory_size                            3891.0 

=== epoch 7/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:21,  2.93it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  913
episode_length                      10.951807
returns                              6.171568
return_std                            3.60545
average_reward                       0.563451
round_time             0 days 00:09:39.129165
episodes_test                           922.0
episode_length_test                 10.839479
returns_test                         5.816837
return_std_test                      2.814898
average_reward_test                  0.536776
round_time_test        0 days 00:00:09.278845
round_time_total       0 days 00:09:39.130240
loss_total             34358968304690520064.0
loss_critic            42948709565162962944.0
loss_actor                -16344345529.856001
memory_size                         3891.7065 

=== epoch 7/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  914
episode_length                      10.938731
returns                              6.070933
return_std                           3.607362
average_reward                       0.554863
round_time             0 days 00:09:39.039465
episodes_test                           904.0
episode_length_test                 11.056416
returns_test                         6.320714
return_std_test                      3.803668
average_reward_test                  0.571854
round_time_test        0 days 00:00:09.335980
round_time_total       0 days 00:09:39.040559
loss_total             35033218226987597824.0
loss_critic            43791521986176253952.0
loss_actor                   -16496793470.976
memory_size                         3896.3975 

=== epoch 7/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:42,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  916
episode_length                       10.90393
returns                              6.070662
return_std                            3.73539
average_reward                       0.556886
round_time             0 days 00:09:41.269698
episodes_test                           913.0
episode_length_test                 10.940854
returns_test                         5.887748
return_std_test                      3.462549
average_reward_test                  0.538715
round_time_test        0 days 00:00:09.242194
round_time_total       0 days 00:09:41.270783
loss_total             36372676639032479744.0
loss_critic            45465844998758449152.0
loss_actor                    -16696549483.52
memory_size                            3902.0 

=== epoch 7/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:59,  3.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  920
episode_length                      10.856522
returns                              5.956505
return_std                           3.418094
average_reward                       0.548881
round_time             0 days 00:09:39.558593
episodes_test                           917.0
episode_length_test                 10.901854
returns_test                         6.021598
return_std_test                      3.590592
average_reward_test                  0.552435
round_time_test        0 days 00:00:09.226731
round_time_total       0 days 00:09:39.559678
loss_total             36555152874107277312.0
loss_critic            45693940313217794048.0
loss_actor                   -16807051231.232
memory_size                            3902.0 

=== epoch 7/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:44,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  912
episode_length                      10.953947
returns                              6.083471
return_std                           3.932274
average_reward                       0.555521
round_time             0 days 00:09:38.543757
episodes_test                           918.0
episode_length_test                 10.885621
returns_test                         5.815896
return_std_test                      3.581053
average_reward_test                  0.534559
round_time_test        0 days 00:00:09.166465
round_time_total       0 days 00:09:38.545054
loss_total             37398062506791870464.0
loss_critic            46747577291126489088.0
loss_actor                   -16989614726.656
memory_size                          3913.607 

=== epoch 7/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                  918
episode_length                      10.888889
returns                              6.007656
return_std                           3.807345
average_reward                        0.55191
round_time             0 days 00:09:38.176324
episodes_test                           902.0
episode_length_test                 11.086475
returns_test                         5.803769
return_std_test                      3.382696
average_reward_test                    0.5235
round_time_test        0 days 00:00:09.152333
round_time_total       0 days 00:09:38.177533
loss_total             38101965801392742400.0
loss_critic            47627456418998304768.0
loss_actor                -17154679600.639999
memory_size                            3914.0 

=== epoch 7/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  922
episode_length                      10.840564
returns                              5.917213
return_std                           3.707211
average_reward                       0.545919
round_time             0 days 00:09:41.428727
episodes_test                           901.0
episode_length_test                  11.09212
returns_test                         5.877921
return_std_test                      3.458197
average_reward_test                  0.530085
round_time_test        0 days 00:00:09.107582
round_time_total       0 days 00:09:41.429839
loss_total             38929917989054857216.0
loss_critic            48662396640381812736.0
loss_actor                -17290436955.647999
memory_size                            3914.0 

=== epoch 7/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  919
episode_length                      10.873776
returns                              5.946695
return_std                           3.652608
average_reward                       0.546986
round_time             0 days 00:09:40.099185
episodes_test                           888.0
episode_length_test                 11.261261
returns_test                         5.799928
return_std_test                      3.889268
average_reward_test                  0.515034
round_time_test        0 days 00:00:09.159051
round_time_total       0 days 00:09:40.100269
loss_total             39713194747139653632.0
loss_critic            49641492568334041088.0
loss_actor                -17537351756.799999
memory_size                            3914.0 

=== epoch 7/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  904
episode_length                      11.048673
returns                              6.008467
return_std                           4.200461
average_reward                       0.543855
round_time             0 days 00:09:40.123197
episodes_test                           909.0
episode_length_test                 10.993399
returns_test                         5.605373
return_std_test                      3.011873
average_reward_test                  0.510002
round_time_test        0 days 00:00:09.153184
round_time_total       0 days 00:09:40.124296
loss_total             40713155196856557568.0
loss_critic            50891443053789233152.0
loss_actor                   -17723508022.784
memory_size                         3920.0375 

=== epoch 7/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<12:29,  2.66it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  905
episode_length                      11.043094
returns                              5.790674
return_std                           3.700258
average_reward                       0.524546
round_time             0 days 00:09:39.028664
episodes_test                           915.0
episode_length_test                 10.922404
returns_test                         5.467482
return_std_test                      3.586149
average_reward_test                   0.50084
round_time_test        0 days 00:00:09.196479
round_time_total       0 days 00:09:39.029758
loss_total             41625929263469281280.0
loss_critic            52032410695466688512.0
loss_actor                   -17927617468.416
memory_size                            3921.0 

=== epoch 7/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:41,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  909
episode_length                      10.984598
returns                              5.711988
return_std                           3.538915
average_reward                       0.520213
round_time             0 days 00:09:40.038248
episodes_test                           925.0
episode_length_test                 10.806486
returns_test                         5.535103
return_std_test                      3.389804
average_reward_test                  0.512362
round_time_test        0 days 00:00:09.110903
round_time_total       0 days 00:09:40.039515
loss_total             42239783248997376000.0
loss_critic            52799728134083543040.0
loss_actor                -18146900100.096001
memory_size                            3921.0 

=== epoch 7/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:36,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  912
episode_length                      10.951754
returns                              5.672899
return_std                           3.456864
average_reward                       0.517811
round_time             0 days 00:09:38.887225
episodes_test                           940.0
episode_length_test                 10.625532
returns_test                         5.411214
return_std_test                      2.919361
average_reward_test                  0.509584
round_time_test        0 days 00:00:09.232496
round_time_total       0 days 00:09:38.888535
loss_total             43684529164436545536.0
loss_critic            54605660483989716992.0
loss_actor                -18391482058.751999
memory_size                            3921.0 

=== epoch 7/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:24,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  904
episode_length                      11.050885
returns                              5.694986
return_std                           4.037398
average_reward                       0.514851
round_time             0 days 00:09:39.707241
episodes_test                           928.0
episode_length_test                 10.768319
returns_test                         5.600974
return_std_test                      3.503791
average_reward_test                  0.520224
round_time_test        0 days 00:00:09.370422
round_time_total       0 days 00:09:39.708484
loss_total             44713055274238500864.0
loss_critic            55891318069152800768.0
loss_actor                -18565942600.703999
memory_size                         3946.3235 

=== epoch 7/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  913
episode_length                      10.945235
returns                               5.60436
return_std                           3.638147
average_reward                        0.51188
round_time             0 days 00:09:39.202654
episodes_test                           903.0
episode_length_test                 11.071982
returns_test                         5.442292
return_std_test                      2.673162
average_reward_test                  0.491633
round_time_test        0 days 00:00:09.146141
round_time_total       0 days 00:09:39.203750
loss_total             46106296664792924160.0
loss_critic            57632869797725110272.0
loss_actor                -18783162459.647999
memory_size                            3955.0 

=== epoch 7/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  917
episode_length                      10.886587
returns                              5.617145
return_std                           3.597974
average_reward                       0.516055
round_time             0 days 00:09:40.250215
episodes_test                           915.0
episode_length_test                 10.918033
returns_test                         5.334997
return_std_test                      3.162455
average_reward_test                  0.488667
round_time_test        0 days 00:00:09.337550
round_time_total       0 days 00:09:40.251315
loss_total             47198943277316145152.0
loss_critic            58998678065303273472.0
loss_actor                -18955219691.007999
memory_size                            3955.0 

=== epoch 7/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:02,  3.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  909
episode_length                      10.990099
returns                               5.71264
return_std                           3.646431
average_reward                       0.519326
round_time             0 days 00:09:39.509986
episodes_test                           911.0
episode_length_test                 10.968167
returns_test                         5.704852
return_std_test                      4.189982
average_reward_test                  0.520319
round_time_test        0 days 00:00:09.158357
round_time_total       0 days 00:09:39.511045
loss_total             47988554690774712320.0
loss_critic            59985692318657462272.0
loss_actor                -19174079937.535999
memory_size                            3955.0 

=== epoch 7/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  901
episode_length                       11.08879
returns                              5.739205
return_std                           3.677507
average_reward                       0.517586
round_time             0 days 00:09:39.318213
episodes_test                           916.0
episode_length_test                 10.909389
returns_test                         5.509123
return_std_test                      2.550265
average_reward_test                  0.505036
round_time_test        0 days 00:00:09.119179
round_time_total       0 days 00:09:39.319313
loss_total             48909761264454320128.0
loss_critic            61137200484629684224.0
loss_actor                -19309520475.136002
memory_size                            3955.0 

=== epoch 7/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  912
episode_length                      10.952851
returns                              5.656077
return_std                           2.826711
average_reward                        0.51653
round_time             0 days 00:09:40.343162
episodes_test                           915.0
episode_length_test                 10.925683
returns_test                         5.720493
return_std_test                      3.384558
average_reward_test                  0.523716
round_time_test        0 days 00:00:09.296901
round_time_total       0 days 00:09:40.344593
loss_total             49830808994987606016.0
loss_critic            62288510201947234304.0
loss_actor                -19532537449.984001
memory_size                            3955.0 

=== epoch 7/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  919
episode_length                      10.862894
returns                              5.577039
return_std                            2.60529
average_reward                       0.513107
round_time             0 days 00:09:39.990305
episodes_test                           915.0
episode_length_test                 10.923497
returns_test                         5.821524
return_std_test                      3.973632
average_reward_test                  0.533156
round_time_test        0 days 00:00:09.180850
round_time_total       0 days 00:09:39.991519
loss_total             50196468163041812480.0
loss_critic            62745584103740882944.0
loss_actor                -19665317577.728001
memory_size                            3955.0 

=== epoch 7/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  914
episode_length                       10.93326
returns                              5.661576
return_std                             3.0268
average_reward                       0.517527
round_time             0 days 00:09:40.340777
episodes_test                           936.0
episode_length_test                  10.67735
returns_test                         5.349314
return_std_test                      3.407292
average_reward_test                   0.50112
round_time_test        0 days 00:00:09.125904
round_time_total       0 days 00:09:40.341828
loss_total             51574315765185257472.0
loss_critic            64467893582780694528.0
loss_actor                   -19844443809.792
memory_size                          3961.219 

=== epoch 7/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  921
episode_length                      10.847991
returns                              5.561735
return_std                           2.989017
average_reward                        0.51254
round_time             0 days 00:09:39.408912
episodes_test                           926.0
episode_length_test                 10.792657
returns_test                         5.611935
return_std_test                      3.067468
average_reward_test                  0.520116
round_time_test        0 days 00:00:09.312118
round_time_total       0 days 00:09:39.410018
loss_total             52795635320796405760.0
loss_critic            65994543077597274112.0
loss_actor                -20048557322.240002
memory_size                            3964.0 

=== epoch 7/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  921
episode_length                      10.849077
returns                              5.556722
return_std                           3.321432
average_reward                       0.512305
round_time             0 days 00:09:40.672558
episodes_test                           905.0
episode_length_test                 11.048619
returns_test                         5.634173
return_std_test                      3.629123
average_reward_test                  0.509991
round_time_test        0 days 00:00:09.239529
round_time_total       0 days 00:09:40.673988
loss_total             53201801391120834560.0
loss_critic            66502250647635746816.0
loss_actor                -20251251824.639999
memory_size                          3964.333 

=== epoch 7/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:22,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  923
episode_length                      10.825569
returns                              5.509065
return_std                           3.947335
average_reward                         0.5089
round_time             0 days 00:09:41.867213
episodes_test                           927.0
episode_length_test                  10.78425
returns_test                         5.553864
return_std_test                       3.24106
average_reward_test                  0.515122
round_time_test        0 days 00:00:09.277272
round_time_total       0 days 00:09:41.868313
loss_total             54953327464768839680.0
loss_critic            68691658213307473920.0
loss_actor                -20408107119.616001
memory_size                          3967.979 

=== epoch 7/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:52,  3.37it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  924
episode_length                      10.808442
returns                              5.504136
return_std                           3.931619
average_reward                       0.509555
round_time             0 days 00:09:39.481176
episodes_test                           931.0
episode_length_test                 10.729323
returns_test                         5.685519
return_std_test                      3.137427
average_reward_test                  0.530299
round_time_test        0 days 00:00:09.196370
round_time_total       0 days 00:09:39.482261
loss_total             55891625922512969728.0
loss_critic            69864531228313042944.0
loss_actor                   -20649969012.736
memory_size                            3973.0 

=== epoch 7/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  940
episode_length                      10.634043
returns                               5.29302
return_std                           3.591568
average_reward                        0.49792
round_time             0 days 00:09:40.980256
episodes_test                           930.0
episode_length_test                 10.752688
returns_test                         5.436797
return_std_test                      2.655393
average_reward_test                  0.505622
round_time_test        0 days 00:00:09.252513
round_time_total       0 days 00:09:40.981533
loss_total             57293395010903220224.0
loss_critic            71616742602269868032.0
loss_actor                -20802434193.408001
memory_size                            3973.0 

=== epoch 7/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:44,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  937
episode_length                      10.654216
returns                              5.351428
return_std                           3.825201
average_reward                       0.502564
round_time             0 days 00:09:40.234723
episodes_test                           935.0
episode_length_test                 10.694118
returns_test                         5.415992
return_std_test                      3.425225
average_reward_test                  0.506496
round_time_test        0 days 00:00:09.333926
round_time_total       0 days 00:09:40.235859
loss_total             58482417466880090112.0
loss_critic            73103020633758056448.0
loss_actor                -21063010799.616001
memory_size                         3979.8265 

=== epoch 7/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 4/2000 [00:01<10:00,  3.33it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  945
episode_length                      10.571429
returns                              5.269532
return_std                           3.528126
average_reward                       0.498132
round_time             0 days 00:09:41.616416
episodes_test                           953.0
episode_length_test                 10.491081
returns_test                         5.398629
return_std_test                      3.904052
average_reward_test                  0.514696
round_time_test        0 days 00:00:09.175651
round_time_total       0 days 00:09:41.617948
loss_total             59672371246589083648.0
loss_critic            74590462817162608640.0
loss_actor                -21243331018.751999
memory_size                            3991.0 

=== epoch 7/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:12,  2.97it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  950
episode_length                      10.514737
returns                              5.245047
return_std                            2.79363
average_reward                       0.498855
round_time             0 days 00:09:40.840745
episodes_test                           926.0
episode_length_test                 10.795896
returns_test                         5.580078
return_std_test                      3.117868
average_reward_test                  0.516986
round_time_test        0 days 00:00:09.136586
round_time_total       0 days 00:09:40.842156
loss_total             60680944773418737664.0
loss_critic            75851179772154036224.0
loss_actor                   -21470482164.736
memory_size                            3991.0 

=== epoch 7/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:08,  3.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  948
episode_length                      10.540084
returns                              5.213302
return_std                           2.798809
average_reward                       0.494538
round_time             0 days 00:09:41.481943
episodes_test                           943.0
episode_length_test                 10.597031
returns_test                         5.420256
return_std_test                      2.544633
average_reward_test                  0.511647
round_time_test        0 days 00:00:09.334618
round_time_total       0 days 00:09:41.483160
loss_total             62142136479548973056.0
loss_critic            77677669288543485952.0
loss_actor                   -21634194434.048
memory_size                            3991.0 

=== epoch 7/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:19,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  940
episode_length                       10.62234
returns                              5.257807
return_std                           2.990771
average_reward                       0.495722
round_time             0 days 00:09:40.180907
episodes_test                           949.0
episode_length_test                 10.531085
returns_test                         5.282814
return_std_test                      2.394123
average_reward_test                  0.501855
round_time_test        0 days 00:00:09.205571
round_time_total       0 days 00:09:40.181987
loss_total             63136008712381046784.0
loss_critic            78920009565839671296.0
loss_actor                -21880755764.223999
memory_size                            3991.0 

=== epoch 7/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:17,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  941
episode_length                      10.596174
returns                              5.229741
return_std                           3.136572
average_reward                       0.495512
round_time             0 days 00:09:40.154517
episodes_test                           934.0
episode_length_test                 10.702355
returns_test                           5.2447
return_std_test                      3.251039
average_reward_test                  0.490154
round_time_test        0 days 00:00:09.357560
round_time_total       0 days 00:09:40.156009
loss_total             64036465750122504192.0
loss_critic            80045580855045029888.0
loss_actor                -22053663751.167999
memory_size                            3991.0 

=== epoch 7/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  944
episode_length                      10.582627
returns                              5.301569
return_std                           3.439798
average_reward                       0.500669
round_time             0 days 00:09:41.552110
episodes_test                           951.0
episode_length_test                 10.504732
returns_test                         5.201829
return_std_test                      2.674481
average_reward_test                  0.495487
round_time_test        0 days 00:00:09.318515
round_time_total       0 days 00:09:41.553380
loss_total             65246160930155216896.0
loss_critic            81557699704466718720.0
loss_actor                   -22255735100.416
memory_size                         3997.9825 

=== epoch 7/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  939
episode_length                      10.638978
returns                              5.295026
return_std                            3.53222
average_reward                       0.497768
round_time             0 days 00:09:41.895680
episodes_test                           938.0
episode_length_test                 10.651386
returns_test                         5.308505
return_std_test                      2.702015
average_reward_test                  0.498545
round_time_test        0 days 00:00:09.296588
round_time_total       0 days 00:09:41.896774
loss_total             66251500219342503936.0
loss_critic            82814373916831023104.0
loss_actor                -22463967951.872002
memory_size                            3998.0 

=== epoch 7/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:42<00:00,  3.43it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  932
episode_length                      10.714592
returns                              5.333889
return_std                            3.92606
average_reward                       0.497981
round_time             0 days 00:09:42.952702
episodes_test                           953.0
episode_length_test                 10.491081
returns_test                         5.203154
return_std_test                      2.466993
average_reward_test                  0.496055
round_time_test        0 days 00:00:09.182871
round_time_total       0 days 00:09:42.953783
loss_total             69143751194386391040.0
loss_critic            86429687610896875520.0
loss_actor                -22718412147.712002
memory_size                         4034.6465 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 8/10 ===== round 1/50 ======================================
  0%|          | 7/2000 [00:02<10:12,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  187
episode_length                      10.695187
returns                              5.526734
return_std                           2.431975
average_reward                        0.51675
round_time             0 days 00:09:38.831092
episodes_test                           946.0
episode_length_test                 10.569767
returns_test                         5.278282
return_std_test                      2.551769
average_reward_test                  0.499425
round_time_test        0 days 00:00:09.334894
round_time_total       0 days 00:09:38.832723
loss_total             68683693409349984256.0
loss_critic            85854615358160879616.0
loss_actor                   -22617265558.528
memory_size                          4057.831 

=== epoch 8/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:27,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                  377
episode_length                      10.602122
returns                              5.416686
return_std                           2.422329
average_reward                       0.511187
round_time             0 days 00:09:38.477908
episodes_test                           963.0
episode_length_test                 10.377985
returns_test                         5.068288
return_std_test                      2.309717
average_reward_test                  0.488585
round_time_test        0 days 00:00:09.263906
round_time_total       0 days 00:09:38.478986
loss_total             67267917201429331968.0
loss_critic            84084895081767403520.0
loss_actor                -22442522324.992001
memory_size                            4066.0 

=== epoch 8/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:30,  3.50it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  572
episode_length                      10.472028
returns                              5.249878
return_std                           2.361843
average_reward                       0.501702
round_time             0 days 00:09:40.037923
episodes_test                           976.0
episode_length_test                 10.234631
returns_test                         5.110908
return_std_test                      3.260071
average_reward_test                  0.499269
round_time_test        0 days 00:00:09.139904
round_time_total       0 days 00:09:40.039006
loss_total             64508987630857216000.0
loss_critic            80636233180949544960.0
loss_actor                -22321261986.816002
memory_size                            4066.0 

=== epoch 8/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:46,  3.40it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  766
episode_length                      10.437337
returns                               5.25533
return_std                           2.340345
average_reward                       0.503833
round_time             0 days 00:09:39.009170
episodes_test                           968.0
episode_length_test                 10.330579
returns_test                         5.295337
return_std_test                        2.6399
average_reward_test                  0.512589
round_time_test        0 days 00:00:09.123388
round_time_total       0 days 00:09:39.010249
loss_total             63391977339296833536.0
loss_critic            79239970386318049280.0
loss_actor                -22289695469.568001
memory_size                            4066.0 

=== epoch 8/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:15,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  960
episode_length                      10.415625
returns                              5.223902
return_std                           2.569354
average_reward                       0.501599
round_time             0 days 00:09:38.890860
episodes_test                           954.0
episode_length_test                 10.476939
returns_test                         5.536671
return_std_test                      3.260801
average_reward_test                  0.528652
round_time_test        0 days 00:00:09.161801
round_time_total       0 days 00:09:38.892117
loss_total             63190489062737584128.0
loss_critic            78988110065907744768.0
loss_actor                -22216815244.287998
memory_size                            4066.0 

=== epoch 8/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:34,  3.47it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  961
episode_length                      10.401665
returns                              5.271942
return_std                           3.204072
average_reward                       0.506966
round_time             0 days 00:09:39.602765
episodes_test                           970.0
episode_length_test                 10.307216
returns_test                         5.366171
return_std_test                      2.779612
average_reward_test                  0.520719
round_time_test        0 days 00:00:09.277272
round_time_total       0 days 00:09:39.604005
loss_total             62011504020922884096.0
loss_critic            77514378686948442112.0
loss_actor                -22207571742.720001
memory_size                         4068.0405 

=== epoch 8/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  964
episode_length                      10.365145
returns                              5.302145
return_std                           3.665799
average_reward                       0.511603
round_time             0 days 00:09:39.011149
episodes_test                           974.0
episode_length_test                 10.258727
returns_test                         5.284503
return_std_test                      2.904105
average_reward_test                  0.515458
round_time_test        0 days 00:00:09.137978
round_time_total       0 days 00:09:39.012222
loss_total             62335416560958275584.0
loss_critic            77919269427138740224.0
loss_actor                   -22235474225.152
memory_size                         4081.0475 

=== epoch 8/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:54,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  964
episode_length                      10.364108
returns                              5.327603
return_std                           3.684673
average_reward                         0.5141
round_time             0 days 00:09:39.439435
episodes_test                           965.0
episode_length_test                 10.357513
returns_test                         5.347333
return_std_test                      3.198751
average_reward_test                  0.516498
round_time_test        0 days 00:00:09.030679
round_time_total       0 days 00:09:39.440527
loss_total             61743970636148146176.0
loss_critic            77179962058234609664.0
loss_actor                -22284049820.672001
memory_size                            4086.0 

=== epoch 8/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  964
episode_length                      10.356846
returns                              5.342438
return_std                           3.894996
average_reward                       0.515453
round_time             0 days 00:09:39.999079
episodes_test                           962.0
episode_length_test                 10.393971
returns_test                         5.523258
return_std_test                       3.29947
average_reward_test                  0.531431
round_time_test        0 days 00:00:09.268153
round_time_total       0 days 00:09:40.000164
loss_total             62097633961919021056.0
loss_critic            77622041138207506432.0
loss_actor                -22279385053.183998
memory_size                          4086.971 

=== epoch 8/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  965
episode_length                      10.345078
returns                              5.370911
return_std                            3.73751
average_reward                       0.519369
round_time             0 days 00:09:38.817783
episodes_test                           976.0
episode_length_test                 10.240779
returns_test                         5.363344
return_std_test                      2.444024
average_reward_test                  0.523939
round_time_test        0 days 00:00:09.274195
round_time_total       0 days 00:09:38.819026
loss_total             61450045574815858688.0
loss_critic            76812555707929739264.0
loss_actor                -22274871095.296001
memory_size                            4090.0 

=== epoch 8/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:10,  3.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  972
episode_length                       10.27572
returns                              5.282853
return_std                           3.187072
average_reward                       0.514318
round_time             0 days 00:09:39.024444
episodes_test                           961.0
episode_length_test                 10.400624
returns_test                          5.42806
return_std_test                      2.756946
average_reward_test                  0.522116
round_time_test        0 days 00:00:09.185952
round_time_total       0 days 00:09:39.025534
loss_total             61142971331881205760.0
loss_critic            76428712908934348800.0
loss_actor                -22278372914.175999
memory_size                            4090.0 

=== epoch 8/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:10,  3.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  979
episode_length                      10.202247
returns                               5.20937
return_std                           2.574407
average_reward                       0.510411
round_time             0 days 00:09:39.082002
episodes_test                           983.0
episode_length_test                 10.171923
returns_test                         5.245861
return_std_test                      2.844966
average_reward_test                  0.515762
round_time_test        0 days 00:00:09.276292
round_time_total       0 days 00:09:39.083099
loss_total             62142492389263859712.0
loss_critic            77678114197127561216.0
loss_actor                -22261008921.599998
memory_size                            4090.0 

=== epoch 8/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:00,  3.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  975
episode_length                      10.246154
returns                              5.291711
return_std                           2.945879
average_reward                       0.516625
round_time             0 days 00:09:40.103531
episodes_test                           988.0
episode_length_test                 10.117409
returns_test                         5.091312
return_std_test                      2.904697
average_reward_test                  0.503422
round_time_test        0 days 00:00:09.199971
round_time_total       0 days 00:09:40.104627
loss_total             61415971643500388352.0
loss_critic            76769963234411773952.0
loss_actor                -22293166517.248001
memory_size                          4103.155 

=== epoch 8/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  974
episode_length                      10.261807
returns                              5.310898
return_std                           2.672744
average_reward                       0.517294
round_time             0 days 00:09:40.658982
episodes_test                           975.0
episode_length_test                  10.25641
returns_test                         5.472437
return_std_test                      2.523329
average_reward_test                  0.533563
round_time_test        0 days 00:00:09.353333
round_time_total       0 days 00:09:40.660089
loss_total             61171522215079739392.0
loss_critic            76464401429094760448.0
loss_actor                -22239760906.240002
memory_size                            4105.0 

=== epoch 8/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  978
episode_length                      10.216769
returns                              5.281027
return_std                           2.628436
average_reward                       0.517135
round_time             0 days 00:09:39.200774
episodes_test                           988.0
episode_length_test                 10.118421
returns_test                         5.209279
return_std_test                      2.904137
average_reward_test                  0.514965
round_time_test        0 days 00:00:09.370458
round_time_total       0 days 00:09:39.201880
loss_total             60394278691151560704.0
loss_critic            75492847063492067328.0
loss_actor                -22052527352.832001
memory_size                            4105.0 

=== epoch 8/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  988
episode_length                      10.112348
returns                               5.23801
return_std                           2.590583
average_reward                       0.517678
round_time             0 days 00:09:40.186840
episodes_test                           985.0
episode_length_test                 10.145178
returns_test                         5.270528
return_std_test                      2.890192
average_reward_test                  0.519521
round_time_test        0 days 00:00:09.384448
round_time_total       0 days 00:09:40.188262
loss_total             59222657884650102784.0
loss_critic            74028321146624704512.0
loss_actor                -21827512765.439999
memory_size                            4105.0 

=== epoch 8/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  990
episode_length                       10.09596
returns                              5.221234
return_std                           2.585813
average_reward                       0.516977
round_time             0 days 00:09:40.023629
episodes_test                          1010.0
episode_length_test                   9.90099
returns_test                         5.203765
return_std_test                      3.168562
average_reward_test                   0.52558
round_time_test        0 days 00:00:09.353502
round_time_total       0 days 00:09:40.024886
loss_total             57597670534950297600.0
loss_critic            71997087035641143296.0
loss_actor                -21652270218.240002
memory_size                            4105.0 

=== epoch 8/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:19,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  996
episode_length                      10.018072
returns                              5.232515
return_std                           2.113123
average_reward                       0.522273
round_time             0 days 00:09:40.599903
episodes_test                          1000.0
episode_length_test                     9.999
returns_test                         5.419092
return_std_test                      3.154446
average_reward_test                  0.542006
round_time_test        0 days 00:00:09.398102
round_time_total       0 days 00:09:40.600996
loss_total             56295663830907240448.0
loss_critic            70369578588242223104.0
loss_actor                -21660959574.015999
memory_size                            4105.0 

=== epoch 8/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:25,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1004
episode_length                       9.955179
returns                              5.223265
return_std                            2.03837
average_reward                       0.524674
round_time             0 days 00:09:40.198198
episodes_test                          1010.0
episode_length_test                   9.90099
returns_test                         5.334343
return_std_test                      2.919182
average_reward_test                  0.538769
round_time_test        0 days 00:00:09.235135
round_time_total       0 days 00:09:40.199286
loss_total             55855246350498422784.0
loss_critic            69819056765493878784.0
loss_actor                -21738516885.504002
memory_size                            4105.0 

=== epoch 8/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:41,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1007
episode_length                       9.922542
returns                              5.268377
return_std                           2.027182
average_reward                       0.531115
round_time             0 days 00:09:40.229403
episodes_test                           989.0
episode_length_test                 10.109201
returns_test                         5.244414
return_std_test                       1.98159
average_reward_test                  0.518876
round_time_test        0 days 00:00:09.147900
round_time_total       0 days 00:09:40.230483
loss_total             55983339602468880384.0
loss_critic            69979173335404756992.0
loss_actor                -21870668751.872002
memory_size                            4105.0 

=== epoch 8/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1006
episode_length                       9.926441
returns                              5.266642
return_std                           1.988719
average_reward                       0.530666
round_time             0 days 00:09:40.322099
episodes_test                           986.0
episode_length_test                 10.141988
returns_test                          5.37608
return_std_test                      3.250639
average_reward_test                  0.530082
round_time_test        0 days 00:00:09.407575
round_time_total       0 days 00:09:40.323177
loss_total             56779475532220325888.0
loss_critic            70974343248143810560.0
loss_actor                -21906030130.175999
memory_size                            4105.0 

=== epoch 8/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:01,  3.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1007
episode_length                       9.930487
returns                              5.297954
return_std                           2.284491
average_reward                       0.533504
round_time             0 days 00:09:40.490993
episodes_test                           991.0
episode_length_test                 10.083754
returns_test                         5.314428
return_std_test                       2.65378
average_reward_test                  0.527202
round_time_test        0 days 00:00:09.338946
round_time_total       0 days 00:09:40.492259
loss_total             56363118426168107008.0
loss_critic            70453896801257111552.0
loss_actor                -21912635101.183998
memory_size                            4105.0 

=== epoch 8/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:31,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1019
episode_length                       9.803729
returns                              5.177321
return_std                           2.222176
average_reward                       0.528065
round_time             0 days 00:09:39.001546
episodes_test                          1058.0
episode_length_test                   9.44518
returns_test                         4.923758
return_std_test                      2.174953
average_reward_test                  0.521607
round_time_test        0 days 00:00:09.319083
round_time_total       0 days 00:09:39.002695
loss_total             56293082009379938304.0
loss_critic            70366351235741679616.0
loss_actor                -21864888612.863998
memory_size                            4105.0 

=== epoch 8/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:49,  3.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1027
episode_length                       9.729309
returns                              5.151242
return_std                           2.305968
average_reward                       0.529786
round_time             0 days 00:09:40.249762
episodes_test                          1022.0
episode_length_test                  9.783757
returns_test                         5.196704
return_std_test                      2.454618
average_reward_test                  0.531202
round_time_test        0 days 00:00:09.471301
round_time_total       0 days 00:09:40.250977
loss_total             56209240765608697856.0
loss_critic            70261549772012208128.0
loss_actor                -21892487317.504002
memory_size                            4105.0 

=== epoch 8/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1022
episode_length                       9.772016
returns                                5.2052
return_std                            2.98632
average_reward                        0.53274
round_time             0 days 00:09:38.836833
episodes_test                          1002.0
episode_length_test                  9.976048
returns_test                          5.37317
return_std_test                      2.463707
average_reward_test                  0.538801
round_time_test        0 days 00:00:09.275669
round_time_total       0 days 00:09:38.837905
loss_total             56245161933633437696.0
loss_critic            70306451245237280768.0
loss_actor                    -22041270712.32
memory_size                          4110.698 

=== epoch 8/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:48,  3.39it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1021
episode_length                       9.790402
returns                              5.264776
return_std                           3.000946
average_reward                       0.537799
round_time             0 days 00:09:38.822787
episodes_test                          1012.0
episode_length_test                  9.879447
returns_test                         5.269688
return_std_test                       2.59533
average_reward_test                  0.533497
round_time_test        0 days 00:00:09.382424
round_time_total       0 days 00:09:38.824016
loss_total             57350425940164149248.0
loss_critic            71688031251751403520.0
loss_actor                   -22092393276.416
memory_size                            4116.0 

=== epoch 8/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:50,  3.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1025
episode_length                        9.75122
returns                              5.301862
return_std                           2.762279
average_reward                       0.543871
round_time             0 days 00:09:40.908818
episodes_test                          1000.0
episode_length_test                     9.996
returns_test                         5.420353
return_std_test                      2.643942
average_reward_test                  0.542416
round_time_test        0 days 00:00:09.222353
round_time_total       0 days 00:09:40.909896
loss_total             57335850754652725248.0
loss_critic            71669812293501616128.0
loss_actor                   -22249842429.952
memory_size                            4116.0 

=== epoch 8/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:01,  3.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                 1013
episode_length                       9.856861
returns                              5.345386
return_std                           2.808034
average_reward                       0.542386
round_time             0 days 00:09:41.536142
episodes_test                          1017.0
episode_length_test                  9.828909
returns_test                         5.280471
return_std_test                       2.05337
average_reward_test                  0.537435
round_time_test        0 days 00:00:09.166141
round_time_total       0 days 00:09:41.537216
loss_total             57716563407871066112.0
loss_critic            72145703049276530688.0
loss_actor                   -22357704771.584
memory_size                            4116.0 

=== epoch 8/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:23,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                 1006
episode_length                       9.935388
returns                              5.370822
return_std                           2.805625
average_reward                       0.540467
round_time             0 days 00:09:41.205086
episodes_test                          1026.0
episode_length_test                   9.74269
returns_test                         5.172244
return_std_test                      2.160589
average_reward_test                  0.531022
round_time_test        0 days 00:00:09.204224
round_time_total       0 days 00:09:41.206383
loss_total             57874721182423900160.0
loss_critic            72343400259221241856.0
loss_actor                -22459204788.223999
memory_size                            4116.0 

=== epoch 8/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1008
episode_length                       9.910714
returns                              5.347489
return_std                           2.422577
average_reward                       0.539823
round_time             0 days 00:09:40.711390
episodes_test                          1019.0
episode_length_test                  9.805692
returns_test                         5.172868
return_std_test                      2.301088
average_reward_test                  0.527844
round_time_test        0 days 00:00:09.320636
round_time_total       0 days 00:09:40.712832
loss_total             59048693764032823296.0
loss_critic            73810865981009707008.0
loss_actor                -22553740969.984001
memory_size                            4116.0 

=== epoch 8/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:53,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                 1014
episode_length                        9.85503
returns                              5.318861
return_std                           2.405619
average_reward                       0.539786
round_time             0 days 00:09:41.422439
episodes_test                          1025.0
episode_length_test                  9.754146
returns_test                         5.102695
return_std_test                      2.107929
average_reward_test                  0.523232
round_time_test        0 days 00:00:09.363747
round_time_total       0 days 00:09:41.423510
loss_total             58583327929383165952.0
loss_critic            73229158703365677056.0
loss_actor                -22721454226.431999
memory_size                            4116.0 

=== epoch 8/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:04,  3.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1014
episode_length                       9.853057
returns                              5.295755
return_std                           2.428919
average_reward                       0.537289
round_time             0 days 00:09:40.569886
episodes_test                          1024.0
episode_length_test                  9.762695
returns_test                         5.328186
return_std_test                       3.02445
average_reward_test                   0.54588
round_time_test        0 days 00:00:09.233230
round_time_total       0 days 00:09:40.570961
loss_total             58831115483901837312.0
loss_critic            73538893121225244672.0
loss_actor                   -22828344075.264
memory_size                            4116.0 

=== epoch 8/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:43,  3.42it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1022
episode_length                       9.779843
returns                              5.276289
return_std                            2.51055
average_reward                       0.539514
round_time             0 days 00:09:40.632970
episodes_test                          1039.0
episode_length_test                  9.619827
returns_test                         5.044394
return_std_test                      2.268987
average_reward_test                  0.524598
round_time_test        0 days 00:00:09.341690
round_time_total       0 days 00:09:40.634052
loss_total             59886716097218764800.0
loss_critic            74858393884298002432.0
loss_actor                   -22976269592.576
memory_size                            4116.0 

=== epoch 8/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:04,  3.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1031
episode_length                       9.683802
returns                              5.239282
return_std                           2.412659
average_reward                       0.540858
round_time             0 days 00:09:40.107047
episodes_test                          1033.0
episode_length_test                  9.679574
returns_test                          5.15766
return_std_test                      2.671418
average_reward_test                  0.532885
round_time_test        0 days 00:00:09.442867
round_time_total       0 days 00:09:40.108143
loss_total             60111751487310675968.0
loss_critic            75139688106794598400.0
loss_actor                   -23117368585.216
memory_size                            4116.0 

=== epoch 8/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1037
episode_length                       9.629701
returns                              5.174883
return_std                            2.35327
average_reward                       0.537308
round_time             0 days 00:09:38.786995
episodes_test                          1036.0
episode_length_test                  9.648649
returns_test                         5.089567
return_std_test                      2.555126
average_reward_test                  0.527636
round_time_test        0 days 00:00:09.249842
round_time_total       0 days 00:09:38.788062
loss_total             61333771668223713280.0
loss_critic            76667213335134912512.0
loss_actor                   -23213087229.952
memory_size                            4116.0 

=== epoch 8/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:17,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1031
episode_length                       9.691562
returns                              5.143866
return_std                           2.379951
average_reward                       0.530511
round_time             0 days 00:09:39.257279
episodes_test                          1024.0
episode_length_test                  9.764648
returns_test                         5.187889
return_std_test                       2.28813
average_reward_test                  0.531342
round_time_test        0 days 00:00:09.167296
round_time_total       0 days 00:09:39.258576
loss_total             61389605399746289664.0
loss_critic            76737005518504722432.0
loss_actor                -23389753068.543999
memory_size                            4116.0 

=== epoch 8/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:49,  3.07it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1034
episode_length                       9.663443
returns                              5.065641
return_std                           2.339157
average_reward                       0.524115
round_time             0 days 00:09:40.405636
episodes_test                          1045.0
episode_length_test                  9.559809
returns_test                         5.017173
return_std_test                      2.182452
average_reward_test                  0.525036
round_time_test        0 days 00:00:09.311323
round_time_total       0 days 00:09:40.406831
loss_total             62549559228991791104.0
loss_critic            78186947808909885440.0
loss_actor                -23551274524.672001
memory_size                            4116.0 

=== epoch 8/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:48,  3.08it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1034
episode_length                       9.665377
returns                              5.041488
return_std                           2.204318
average_reward                       0.521551
round_time             0 days 00:09:40.100586
episodes_test                          1027.0
episode_length_test                  9.734177
returns_test                         5.184873
return_std_test                      3.095545
average_reward_test                  0.532728
round_time_test        0 days 00:00:09.264670
round_time_total       0 days 00:09:40.101655
loss_total             64036130571699355648.0
loss_critic            80045161887238782976.0
loss_actor                -23733911185.408001
memory_size                            4116.0 

=== epoch 8/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:26,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1026
episode_length                       9.734893
returns                              5.071516
return_std                            2.23817
average_reward                       0.520732
round_time             0 days 00:09:40.600458
episodes_test                          1032.0
episode_length_test                  9.686047
returns_test                         5.156391
return_std_test                      2.358547
average_reward_test                  0.532536
round_time_test        0 days 00:00:09.316148
round_time_total       0 days 00:09:40.601664
loss_total             64241214717474365440.0
loss_critic            80301517093646811136.0
loss_actor                -23942590090.240002
memory_size                            4116.0 

=== epoch 8/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:55,  3.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1026
episode_length                       9.740741
returns                              5.046735
return_std                           1.920846
average_reward                       0.518267
round_time             0 days 00:09:40.885309
episodes_test                          1028.0
episode_length_test                  9.723735
returns_test                         5.269327
return_std_test                      2.151665
average_reward_test                  0.542052
round_time_test        0 days 00:00:09.242401
round_time_total       0 days 00:09:40.886419
loss_total             65840632968250703872.0
loss_critic            82300789880454070272.0
loss_actor                -24037219716.096001
memory_size                            4116.0 

=== epoch 8/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1032
episode_length                       9.674419
returns                              5.057175
return_std                           1.877695
average_reward                       0.522914
round_time             0 days 00:09:40.625360
episodes_test                          1054.0
episode_length_test                  9.486717
returns_test                         4.971754
return_std_test                      1.839902
average_reward_test                  0.524124
round_time_test        0 days 00:00:09.388708
round_time_total       0 days 00:09:40.626432
loss_total             65535853778817662976.0
loss_critic            81919815846933512192.0
loss_actor                    -24205227924.48
memory_size                            4116.0 

=== epoch 8/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:24,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                 1028
episode_length                        9.72179
returns                              5.167637
return_std                           2.056649
average_reward                       0.531566
round_time             0 days 00:09:38.427462
episodes_test                          1027.0
episode_length_test                  9.737098
returns_test                         5.231085
return_std_test                      2.050789
average_reward_test                  0.537232
round_time_test        0 days 00:00:09.442499
round_time_total       0 days 00:09:38.428562
loss_total             66601815924732444672.0
loss_critic            83252268489744580608.0
loss_actor                -24339454002.175999
memory_size                            4116.0 

=== epoch 8/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:54,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1026
episode_length                       9.730994
returns                              5.185127
return_std                           2.074402
average_reward                       0.532434
round_time             0 days 00:09:40.478999
episodes_test                          1023.0
episode_length_test                  9.771261
returns_test                         5.327805
return_std_test                      2.910898
average_reward_test                  0.545339
round_time_test        0 days 00:00:09.251573
round_time_total       0 days 00:09:40.480087
loss_total             67582277766084255744.0
loss_critic            84477845799680688128.0
loss_actor                -24432134429.695999
memory_size                            4116.0 

=== epoch 8/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:10,  3.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1027
episode_length                       9.731256
returns                              5.182171
return_std                           2.061481
average_reward                       0.532584
round_time             0 days 00:09:40.956405
episodes_test                          1018.0
episode_length_test                  9.815324
returns_test                         5.126211
return_std_test                      2.658855
average_reward_test                  0.522378
round_time_test        0 days 00:00:09.404614
round_time_total       0 days 00:09:40.957473
loss_total             68075552049183940608.0
loss_critic            85094438572466307072.0
loss_actor                -24544915539.967999
memory_size                            4116.0 

=== epoch 8/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:08,  3.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1027
episode_length                       9.728335
returns                              5.188027
return_std                           2.139236
average_reward                       0.533143
round_time             0 days 00:09:40.802333
episodes_test                          1044.0
episode_length_test                  9.574713
returns_test                          5.01863
return_std_test                       2.29951
average_reward_test                   0.52427
round_time_test        0 days 00:00:09.080754
round_time_total       0 days 00:09:40.803425
loss_total             67845387033827680256.0
loss_critic            84806732414871404544.0
loss_actor                -24680538079.231998
memory_size                            4116.0 

=== epoch 8/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:17,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1026
episode_length                       9.741715
returns                              5.175085
return_std                           2.164488
average_reward                       0.531316
round_time             0 days 00:09:40.475971
episodes_test                          1042.0
episode_length_test                  9.591171
returns_test                         5.095613
return_std_test                      2.152562
average_reward_test                  0.531366
round_time_test        0 days 00:00:09.404945
round_time_total       0 days 00:09:40.477067
loss_total             68676792530226921472.0
loss_critic            85845989123192487936.0
loss_actor                -24870958948.352001
memory_size                            4116.0 

=== epoch 8/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:44,  3.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1026
episode_length                       9.737817
returns                              5.196324
return_std                           2.496505
average_reward                       0.533669
round_time             0 days 00:09:39.813401
episodes_test                          1045.0
episode_length_test                  9.562679
returns_test                         4.925233
return_std_test                      2.291984
average_reward_test                  0.515304
round_time_test        0 days 00:00:09.312190
round_time_total       0 days 00:09:39.814611
loss_total             70671119658756866048.0
loss_critic            88338898055570276352.0
loss_actor                   -24964887258.112
memory_size                          4119.175 

=== epoch 8/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:01,  3.32it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                 1029
episode_length                       9.703596
returns                               5.18566
return_std                           2.502546
average_reward                        0.53396
round_time             0 days 00:09:41.005728
episodes_test                          1052.0
episode_length_test                  9.502852
returns_test                         4.984918
return_std_test                      1.892298
average_reward_test                  0.524682
round_time_test        0 days 00:00:09.248233
round_time_total       0 days 00:09:41.006955
loss_total             70462868965673345024.0
loss_critic            88078584660078821376.0
loss_actor                -25134840199.167999
memory_size                            4120.0 

=== epoch 8/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                 1040
episode_length                       9.608654
returns                              5.084661
return_std                           2.500201
average_reward                       0.528988
round_time             0 days 00:09:41.226890
episodes_test                          1032.0
episode_length_test                  9.688953
returns_test                         5.165261
return_std_test                      2.300055
average_reward_test                  0.533151
round_time_test        0 days 00:00:09.381724
round_time_total       0 days 00:09:41.227951
loss_total             71774876090809868288.0
loss_critic            89718593512348532736.0
loss_actor                -25313314203.647999
memory_size                            4120.0 

=== epoch 8/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                 1043
episode_length                       9.574305
returns                              5.044981
return_std                           2.420775
average_reward                       0.527187
round_time             0 days 00:09:40.470112
episodes_test                          1045.0
episode_length_test                  9.568421
returns_test                         4.993152
return_std_test                      2.471761
average_reward_test                   0.52188
round_time_test        0 days 00:00:09.201451
round_time_total       0 days 00:09:40.471399
loss_total             72135803574345621504.0
loss_critic            90169752931089645568.0
loss_actor                -25433634914.304001
memory_size                            4120.0 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 9/10 ===== round 1/50 ======================================
  0%|          | 7/2000 [00:02<10:20,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  206
episode_length                       9.694175
returns                               5.43693
return_std                           3.646294
average_reward                       0.561468
round_time             0 days 00:09:40.042457
episodes_test                          1042.0
episode_length_test                  9.590211
returns_test                         5.066277
return_std_test                       2.61098
average_reward_test                  0.528491
round_time_test        0 days 00:00:09.248214
round_time_total       0 days 00:09:40.043550
loss_total             73122111082516201472.0
loss_critic            91402637295137275904.0
loss_actor                -25619841860.608002
memory_size                          4120.135 

=== epoch 9/10 ===== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:15,  3.24it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  412
episode_length                       9.686893
returns                              5.238467
return_std                           3.376564
average_reward                       0.540418
round_time             0 days 00:09:40.389221
episodes_test                          1059.0
episode_length_test                  9.434372
returns_test                         5.031768
return_std_test                      2.249522
average_reward_test                  0.533571
round_time_test        0 days 00:00:09.304270
round_time_total       0 days 00:09:40.390299
loss_total             74123969744500457472.0
loss_critic            92654960478306697216.0
loss_actor                   -25786516529.152
memory_size                           4127.49 

=== epoch 9/10 ===== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  628
episode_length                        9.55414
returns                              5.060687
return_std                           2.911362
average_reward                       0.529685
round_time             0 days 00:09:40.840339
episodes_test                          1038.0
episode_length_test                  9.625241
returns_test                         5.102855
return_std_test                      2.341045
average_reward_test                  0.529937
round_time_test        0 days 00:00:09.363045
round_time_total       0 days 00:09:40.841641
loss_total             75331907081417768960.0
loss_critic            94164882202229899264.0
loss_actor                -25841530057.728001
memory_size                            4128.0 

=== epoch 9/10 ===== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:09,  2.98it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  839
episode_length                       9.526818
returns                              4.998241
return_std                           2.680006
average_reward                       0.524873
round_time             0 days 00:09:40.998981
episodes_test                          1036.0
episode_length_test                  9.649614
returns_test                         5.061577
return_std_test                      2.288929
average_reward_test                  0.524643
round_time_test        0 days 00:00:09.300679
round_time_total       0 days 00:09:41.000069
loss_total             75978179622482837504.0
loss_critic            94972722928039247872.0
loss_actor                -26014037201.919998
memory_size                            4128.0 

=== epoch 9/10 ===== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.04it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1047
episode_length                       9.550143
returns                              5.006086
return_std                           2.557469
average_reward                       0.524232
round_time             0 days 00:09:39.344764
episodes_test                          1051.0
episode_length_test                  9.514748
returns_test                         5.153132
return_std_test                       3.16513
average_reward_test                  0.541594
round_time_test        0 days 00:00:09.341539
round_time_total       0 days 00:09:39.346008
loss_total             76884725793002307584.0
loss_critic            96105905555701563392.0
loss_actor                -26061931326.464001
memory_size                            4128.0 

=== epoch 9/10 ===== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1053
episode_length                       9.487179
returns                              4.885575
return_std                           2.137658
average_reward                       0.514991
round_time             0 days 00:09:38.724413
episodes_test                          1048.0
episode_length_test                  9.539122
returns_test                         5.104791
return_std_test                      2.161798
average_reward_test                  0.535261
round_time_test        0 days 00:00:09.327022
round_time_total       0 days 00:09:38.725495
loss_total             77472161288732540928.0
loss_critic            96840199953127014400.0
loss_actor                   -26231948965.888
memory_size                            4128.0 

=== epoch 9/10 ===== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:52,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1057
episode_length                       9.448439
returns                              4.873162
return_std                           1.836156
average_reward                       0.515799
round_time             0 days 00:09:38.673115
episodes_test                          1053.0
episode_length_test                  9.490028
returns_test                         4.933533
return_std_test                      1.974473
average_reward_test                  0.519977
round_time_test        0 days 00:00:09.341566
round_time_total       0 days 00:09:38.674554
loss_total             77234232722927632384.0
loss_critic            96542789129872408576.0
loss_actor                -26384329367.551998
memory_size                            4128.0 

=== epoch 9/10 ===== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                 1049
episode_length                       9.524309
returns                              4.954584
return_std                           1.894775
average_reward                        0.52018
round_time             0 days 00:09:39.245959
episodes_test                          1044.0
episode_length_test                  9.576628
returns_test                         5.096412
return_std_test                      2.405094
average_reward_test                  0.532262
round_time_test        0 days 00:00:09.193674
round_time_total       0 days 00:09:39.247271
loss_total             78548661611484774400.0
loss_critic            98185825342273650688.0
loss_actor                   -26515187002.368
memory_size                            4128.0 

=== epoch 9/10 ===== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:56,  3.35it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                 1035
episode_length                       9.658937
returns                              5.002471
return_std                           2.342433
average_reward                       0.517864
round_time             0 days 00:09:38.337213
episodes_test                          1047.0
episode_length_test                  9.551098
returns_test                         4.941833
return_std_test                      1.895767
average_reward_test                   0.51741
round_time_test        0 days 00:00:09.313917
round_time_total       0 days 00:09:38.338299
loss_total             79168832621717684224.0
loss_critic            98961039003909734400.0
loss_actor                -26601495357.439999
memory_size                         4130.7495 

=== epoch 9/10 ===== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:41,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                 1031
episode_length                       9.685742
returns                              5.046449
return_std                           2.341537
average_reward                       0.521099
round_time             0 days 00:09:37.810469
episodes_test                          1060.0
episode_length_test                  9.419811
returns_test                         4.890662
return_std_test                      2.203917
average_reward_test                  0.518901
round_time_test        0 days 00:00:09.452228
round_time_total       0 days 00:09:37.811564
loss_total             79527556784006938624.0
loss_critic            99409444248277843968.0
loss_actor                   -26738429266.944
memory_size                            4134.0 

=== epoch 9/10 ===== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                 1027
episode_length                       9.727361
returns                              5.072833
return_std                           2.349929
average_reward                       0.521285
round_time             0 days 00:09:40.260901
episodes_test                          1031.0
episode_length_test                  9.696411
returns_test                         5.056964
return_std_test                      2.532215
average_reward_test                  0.521638
round_time_test        0 days 00:00:09.120365
round_time_total       0 days 00:09:40.262415
loss_total             79395179731380568064.0
loss_critic            99243973023754354688.0
loss_actor                -26951215135.743999
memory_size                            4134.0 

=== epoch 9/10 ===== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1028
episode_length                        9.715953
returns                               5.060375
return_std                            2.356905
average_reward                        0.520838
round_time              0 days 00:09:41.008094
episodes_test                           1044.0
episode_length_test                   9.576628
returns_test                          5.003039
return_std_test                       1.857811
average_reward_test                   0.522506
round_time_test         0 days 00:00:09.217923
round_time_total        0 days 00:09:41.009190
loss_total              81993618692479680512.0
loss_critic            102492021605281480704.0
loss_actor                 -27030243705.855999
memory_size                             4134.0 

=== epoch 9/10 ===== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:05,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1034
episode_length                        9.669246
returns                                5.04363
return_std                            2.486566
average_reward                        0.521321
round_time              0 days 00:09:40.352300
episodes_test                           1051.0
episode_length_test                   9.514748
returns_test                          5.100238
return_std_test                       2.424622
average_reward_test                   0.536035
round_time_test         0 days 00:00:09.337631
round_time_total        0 days 00:09:40.353619
loss_total              82076037244070887424.0
loss_critic            102595044767282692096.0
loss_actor                    -27205635395.584
memory_size                             4134.0 

=== epoch 9/10 ===== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1041
episode_length                        9.592699
returns                               5.047669
return_std                            2.079198
average_reward                        0.526506
round_time              0 days 00:09:39.890525
episodes_test                           1050.0
episode_length_test                   9.514286
returns_test                          4.938932
return_std_test                       2.445939
average_reward_test                   0.519278
round_time_test         0 days 00:00:09.313087
round_time_total        0 days 00:09:39.891739
loss_total              82890627893753380864.0
loss_critic            103613283035405352960.0
loss_actor                 -27411271618.560001
memory_size                             4134.0 

=== epoch 9/10 ===== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:07,  3.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1041
episode_length                        9.596542
returns                               5.046367
return_std                             2.09118
average_reward                        0.525783
round_time              0 days 00:09:38.898515
episodes_test                           1039.0
episode_length_test                   9.617902
returns_test                          5.067164
return_std_test                       2.387694
average_reward_test                   0.526921
round_time_test         0 days 00:00:09.213445
round_time_total        0 days 00:09:38.899621
loss_total              84437880361807167488.0
loss_critic            105547348538558971904.0
loss_actor                     -27471879900.16
memory_size                             4134.0 

=== epoch 9/10 ===== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1039
episode_length                        9.614052
returns                               5.066282
return_std                            2.089222
average_reward                        0.526773
round_time              0 days 00:09:40.509673
episodes_test                           1044.0
episode_length_test                   9.577586
returns_test                          5.114948
return_std_test                       2.571236
average_reward_test                   0.534099
round_time_test         0 days 00:00:09.480964
round_time_total        0 days 00:09:40.511063
loss_total              85087871674398228480.0
loss_critic            106359837687269244928.0
loss_actor                     -27579077319.68
memory_size                             4134.0 

=== epoch 9/10 ===== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:27,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1038
episode_length                        9.624277
returns                               5.078618
return_std                             2.07247
average_reward                        0.527758
round_time              0 days 00:09:39.179464
episodes_test                           1047.0
episode_length_test                   9.545368
returns_test                          5.138839
return_std_test                       2.632985
average_reward_test                   0.538502
round_time_test         0 days 00:00:09.397743
round_time_total        0 days 00:09:39.180688
loss_total              85137180261806784512.0
loss_critic            106421473527357947904.0
loss_actor                 -27767063117.824001
memory_size                             4134.0 

=== epoch 9/10 ===== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1039
episode_length                        9.622714
returns                               5.044216
return_std                            1.866318
average_reward                        0.524297
round_time              0 days 00:09:39.231791
episodes_test                           1035.0
episode_length_test                   9.658937
returns_test                          5.000142
return_std_test                       2.563497
average_reward_test                   0.517784
round_time_test         0 days 00:00:09.216306
round_time_total        0 days 00:09:39.233216
loss_total              85723009590951165952.0
loss_critic            107153760018638995456.0
loss_actor                 -27845719678.976002
memory_size                             4134.0 

=== epoch 9/10 ===== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1044
episode_length                        9.562261
returns                               5.006258
return_std                             1.85583
average_reward                        0.522622
round_time              0 days 00:09:39.347797
episodes_test                           1029.0
episode_length_test                   9.709427
returns_test                           4.96383
return_std_test                       2.029396
average_reward_test                   0.511438
round_time_test         0 days 00:00:09.428286
round_time_total        0 days 00:09:39.348875
loss_total              86442407717982093312.0
loss_critic            108053007796999553024.0
loss_actor                    -27971305682.944
memory_size                             4134.0 

=== epoch 9/10 ===== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1050
episode_length                        9.521905
returns                               4.977103
return_std                            2.098198
average_reward                        0.522488
round_time              0 days 00:09:40.046152
episodes_test                           1046.0
episode_length_test                   9.559273
returns_test                          5.092271
return_std_test                       2.301824
average_reward_test                   0.532748
round_time_test         0 days 00:00:09.294482
round_time_total        0 days 00:09:40.047357
loss_total              86597470862082981888.0
loss_critic            108246836580890607616.0
loss_actor                    -28139692634.112
memory_size                             4134.0 

=== epoch 9/10 ===== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:26,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1054
episode_length                        9.486717
returns                               5.007893
return_std                            2.359767
average_reward                        0.527762
round_time              0 days 00:09:39.309976
episodes_test                           1048.0
episode_length_test                   9.534351
returns_test                          5.082166
return_std_test                       2.545116
average_reward_test                   0.532991
round_time_test         0 days 00:00:09.240822
round_time_total        0 days 00:09:39.311032
loss_total              88356987267049930752.0
loss_critic            110446232179458277376.0
loss_actor                 -28250426963.967999
memory_size                             4134.0 

=== epoch 9/10 ===== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1050
episode_length                         9.51619
returns                               5.028581
return_std                            2.386672
average_reward                        0.528556
round_time              0 days 00:09:41.056071
episodes_test                           1050.0
episode_length_test                       9.52
returns_test                          5.023814
return_std_test                       2.203048
average_reward_test                   0.527914
round_time_test         0 days 00:00:09.486516
round_time_total        0 days 00:09:41.057142
loss_total              88605268491294359552.0
loss_critic            110756583748796481536.0
loss_actor                 -28403001476.096001
memory_size                             4134.0 

=== epoch 9/10 ===== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:34,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1044
episode_length                        9.557471
returns                               5.128476
return_std                            2.626568
average_reward                        0.535632
round_time              0 days 00:09:39.724561
episodes_test                           1055.0
episode_length_test                   9.467299
returns_test                          5.102154
return_std_test                       2.493189
average_reward_test                   0.539006
round_time_test         0 days 00:00:09.390776
round_time_total        0 days 00:09:39.725688
loss_total              88837258678360539136.0
loss_critic            111046571379824852992.0
loss_actor                     -28615970012.16
memory_size                             4134.0 

=== epoch 9/10 ===== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.16it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1047
episode_length                        9.541547
returns                                 5.1493
return_std                            2.888501
average_reward                        0.539746
round_time              0 days 00:09:40.441473
episodes_test                           1042.0
episode_length_test                   9.591171
returns_test                          5.109549
return_std_test                       2.537654
average_reward_test                   0.532949
round_time_test         0 days 00:00:09.348443
round_time_total        0 days 00:09:40.442576
loss_total              90111743631093563392.0
loss_critic            112639677537755791360.0
loss_actor                 -28703898813.439999
memory_size                             4134.0 

=== epoch 9/10 ===== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1049
episode_length                        9.523356
returns                               5.111375
return_std                              2.6959
average_reward                        0.536656
round_time              0 days 00:09:41.470702
episodes_test                           1044.0
episode_length_test                   9.571839
returns_test                          5.276888
return_std_test                       2.671484
average_reward_test                   0.551543
round_time_test         0 days 00:00:09.221596
round_time_total        0 days 00:09:41.471939
loss_total              91441907692128452608.0
loss_critic            114302382509046038528.0
loss_actor                     -28820284231.68
memory_size                             4134.0 

=== epoch 9/10 ===== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:51,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1045
episode_length                        9.569378
returns                               5.112427
return_std                            2.458912
average_reward                        0.534249
round_time              0 days 00:09:40.255463
episodes_test                           1039.0
episode_length_test                   9.621752
returns_test                          5.120658
return_std_test                       2.743395
average_reward_test                   0.532303
round_time_test         0 days 00:00:09.195383
round_time_total        0 days 00:09:40.256769
loss_total              91306595820866207744.0
loss_critic            114133242794762797056.0
loss_actor                 -28984801591.296001
memory_size                             4134.0 

=== epoch 9/10 ===== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1048
episode_length                        9.530534
returns                               5.084449
return_std                            2.464322
average_reward                        0.533348
round_time              0 days 00:09:39.018959
episodes_test                           1048.0
episode_length_test                    9.53626
returns_test                          5.067161
return_std_test                       2.687746
average_reward_test                   0.531579
round_time_test         0 days 00:00:09.402176
round_time_total        0 days 00:09:39.020402
loss_total              92960572057415598080.0
loss_critic            116200713004687622144.0
loss_actor                 -29090669655.040001
memory_size                             4134.0 

=== epoch 9/10 ===== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1050
episode_length                        9.512381
returns                               5.052042
return_std                            2.248312
average_reward                        0.531207
round_time              0 days 00:09:40.405929
episodes_test                           1032.0
episode_length_test                   9.681202
returns_test                          5.051431
return_std_test                       2.516534
average_reward_test                   0.522081
round_time_test         0 days 00:00:09.508214
round_time_total        0 days 00:09:40.406998
loss_total              93450935087712108544.0
loss_critic            116813666868974338048.0
loss_actor                    -29243030106.112
memory_size                             4134.0 

=== epoch 9/10 ===== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:38,  3.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1047
episode_length                        9.546323
returns                               5.096863
return_std                             2.21529
average_reward                        0.533591
round_time              0 days 00:09:41.183869
episodes_test                           1047.0
episode_length_test                   9.542502
returns_test                          5.007734
return_std_test                       2.091675
average_reward_test                   0.524881
round_time_test         0 days 00:00:09.478136
round_time_total        0 days 00:09:41.185102
loss_total              93892709432539807744.0
loss_critic            117365884797260185600.0
loss_actor                 -29415851952.127998
memory_size                             4134.0 

=== epoch 9/10 ===== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:32,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1046
episode_length                        9.551625
returns                               5.151553
return_std                            2.207066
average_reward                        0.539236
round_time              0 days 00:09:39.667335
episodes_test                           1028.0
episode_length_test                   9.722763
returns_test                          5.384824
return_std_test                       2.773674
average_reward_test                   0.553965
round_time_test         0 days 00:00:09.368699
round_time_total        0 days 00:09:39.668412
loss_total              95638732457087860736.0
loss_critic            119548413416866791424.0
loss_actor                    -29546477985.792
memory_size                             4134.0 

=== epoch 9/10 ===== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:24,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1055
episode_length                        9.472038
returns                               5.106074
return_std                            2.192432
average_reward                        0.539323
round_time              0 days 00:09:40.830637
episodes_test                           1043.0
episode_length_test                   9.581975
returns_test                          5.130887
return_std_test                       2.733324
average_reward_test                   0.535695
round_time_test         0 days 00:00:09.381552
round_time_total        0 days 00:09:40.831724
loss_total              95026151276665438208.0
loss_critic            118782687007859228672.0
loss_actor                    -29660521276.416
memory_size                             4134.0 

=== epoch 9/10 ===== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:38,  3.45it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1048
episode_length                        9.538168
returns                               5.179543
return_std                            2.199466
average_reward                        0.542993
round_time              0 days 00:09:38.816285
episodes_test                           1043.0
episode_length_test                   9.581016
returns_test                          5.029835
return_std_test                       2.091485
average_reward_test                    0.52511
round_time_test         0 days 00:00:09.225821
round_time_total        0 days 00:09:38.817388
loss_total              96804023835843592192.0
loss_critic            121005027736243830784.0
loss_actor                 -29817206940.672001
memory_size                             4134.0 

=== epoch 9/10 ===== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:45,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1047
episode_length                        9.540592
returns                               5.212744
return_std                            2.438091
average_reward                          0.5464
round_time              0 days 00:09:39.690739
episodes_test                           1039.0
episode_length_test                   9.621752
returns_test                          5.073169
return_std_test                       1.938841
average_reward_test                   0.527372
round_time_test         0 days 00:00:09.377483
round_time_total        0 days 00:09:39.691831
loss_total              96896422607201107968.0
loss_critic            121120526119901528064.0
loss_actor                    -30064595047.424
memory_size                             4134.0 

=== epoch 9/10 ===== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:06,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1041
episode_length                        9.596542
returns                               5.232748
return_std                            2.204563
average_reward                        0.545529
round_time              0 days 00:09:39.798983
episodes_test                           1044.0
episode_length_test                   9.574713
returns_test                          5.155254
return_std_test                       2.221737
average_reward_test                   0.538525
round_time_test         0 days 00:00:09.385619
round_time_total        0 days 00:09:39.800077
loss_total              99139450934934388736.0
loss_critic            123924311563652972544.0
loss_actor                    -30276708071.424
memory_size                             4134.0 

=== epoch 9/10 ===== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:05,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1041
episode_length                        9.601345
returns                               5.242516
return_std                            2.431198
average_reward                        0.545544
round_time              0 days 00:09:39.559774
episodes_test                           1037.0
episode_length_test                   9.641273
returns_test                          5.269501
return_std_test                       2.480657
average_reward_test                   0.546643
round_time_test         0 days 00:00:09.461020
round_time_total        0 days 00:09:39.561119
loss_total             100313631024074276864.0
loss_critic            125392036672329056256.0
loss_actor                     -30428592235.52
memory_size                             4134.0 

=== epoch 9/10 ===== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:26,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1033
episode_length                        9.674734
returns                               5.250525
return_std                            2.463326
average_reward                         0.54249
round_time              0 days 00:09:39.030685
episodes_test                           1037.0
episode_length_test                   9.641273
returns_test                          5.254447
return_std_test                       2.763205
average_reward_test                   0.545069
round_time_test         0 days 00:00:09.307951
round_time_total        0 days 00:09:39.031844
loss_total             101053640786295013376.0
loss_critic            126317048926232264704.0
loss_actor                 -30564748421.119999
memory_size                             4134.0 

=== epoch 9/10 ===== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:35,  3.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1040
episode_length                        9.606731
returns                               5.192992
return_std                            2.476362
average_reward                        0.540498
round_time              0 days 00:09:40.660748
episodes_test                           1024.0
episode_length_test                   9.765625
returns_test                           5.35407
return_std_test                       3.034013
average_reward_test                   0.548257
round_time_test         0 days 00:00:09.335801
round_time_total        0 days 00:09:40.661841
loss_total             103081176459021107200.0
loss_critic            128851468350014128128.0
loss_actor                     -30740459284.48
memory_size                             4134.0 

=== epoch 9/10 ===== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:19,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1043
episode_length                        9.581975
returns                               5.179865
return_std                            2.451483
average_reward                         0.54055
round_time              0 days 00:09:39.201752
episodes_test                           1037.0
episode_length_test                    9.63838
returns_test                          5.158749
return_std_test                       2.442155
average_reward_test                   0.535399
round_time_test         0 days 00:00:09.340668
round_time_total        0 days 00:09:39.202824
loss_total             104196266578867912704.0
loss_critic            130245331086684045312.0
loss_actor                 -30945194939.391998
memory_size                             4134.0 

=== epoch 9/10 ===== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:26,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1047
episode_length                        9.547278
returns                               5.122453
return_std                             2.40989
average_reward                        0.536466
round_time              0 days 00:09:39.808487
episodes_test                           1042.0
episode_length_test                    9.59501
returns_test                          5.167215
return_std_test                       2.416711
average_reward_test                   0.538608
round_time_test         0 days 00:00:09.188833
round_time_total        0 days 00:09:39.809577
loss_total             104672988663289954304.0
loss_critic            130841233685064777728.0
loss_actor                    -31144787282.944
memory_size                             4134.0 

=== epoch 9/10 ===== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:44,  3.41it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1041
episode_length                        9.590778
returns                               5.100893
return_std                            2.195043
average_reward                        0.531987
round_time              0 days 00:09:40.826233
episodes_test                           1044.0
episode_length_test                   9.576628
returns_test                          5.217251
return_std_test                       2.509917
average_reward_test                   0.544877
round_time_test         0 days 00:00:09.339798
round_time_total        0 days 00:09:40.827531
loss_total             106139106807134748672.0
loss_critic            132673881403353677824.0
loss_actor                 -31238934659.071999
memory_size                             4134.0 

=== epoch 9/10 ===== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:36,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1041
episode_length                        9.599424
returns                               5.179891
return_std                            2.435173
average_reward                        0.539412
round_time              0 days 00:09:40.663713
episodes_test                           1034.0
episode_length_test                    9.66441
returns_test                           5.10107
return_std_test                       2.137471
average_reward_test                    0.52775
round_time_test         0 days 00:00:09.318443
round_time_total        0 days 00:09:40.665014
loss_total             105918575714860089344.0
loss_critic            132398217466542096384.0
loss_actor                 -31436577476.608002
memory_size                           4134.147 

=== epoch 9/10 ===== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:26,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1037
episode_length                        9.626808
returns                               5.241836
return_std                             2.42596
average_reward                        0.543822
round_time              0 days 00:09:40.513621
episodes_test                           1039.0
episode_length_test                   9.624639
returns_test                          5.265011
return_std_test                       2.579078
average_reward_test                   0.547035
round_time_test         0 days 00:00:09.397626
round_time_total        0 days 00:09:40.514793
loss_total             107224189814299312128.0
loss_critic            134030234980340187136.0
loss_actor                    -31627193481.216
memory_size                             4135.0 

=== epoch 9/10 ===== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:43,  2.84it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1035
episode_length                        9.654106
returns                               5.224345
return_std                            2.179019
average_reward                        0.541242
round_time              0 days 00:09:39.043011
episodes_test                           1033.0
episode_length_test                    9.67183
returns_test                          5.243577
return_std_test                       2.561115
average_reward_test                   0.542434
round_time_test         0 days 00:00:09.401431
round_time_total        0 days 00:09:39.044072
loss_total             108967806219000397824.0
loss_critic            136209755498860937216.0
loss_actor                 -31851035643.903999
memory_size                             4135.0 

=== epoch 9/10 ===== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1033
episode_length                        9.674734
returns                               5.277819
return_std                            2.458803
average_reward                        0.545444
round_time              0 days 00:09:40.698534
episodes_test                           1036.0
episode_length_test                   9.650579
returns_test                           5.15289
return_std_test                       2.931884
average_reward_test                    0.53405
round_time_test         0 days 00:00:09.358581
round_time_total        0 days 00:09:40.699601
loss_total             110639954944040140800.0
loss_critic            138299941324346507264.0
loss_actor                 -31995479641.088001
memory_size                             4135.0 

=== epoch 9/10 ===== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:43,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1035
episode_length                        9.658937
returns                               5.269209
return_std                            2.473151
average_reward                        0.545353
round_time              0 days 00:09:41.374041
episodes_test                           1028.0
episode_length_test                    9.72179
returns_test                          5.288093
return_std_test                       2.630614
average_reward_test                   0.544139
round_time_test         0 days 00:00:09.352498
round_time_total        0 days 00:09:41.375102
loss_total             110718791414291398656.0
loss_critic            138398486958340063232.0
loss_actor                 -31953224087.551998
memory_size                             4135.0 

=== epoch 9/10 ===== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:59,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1039
episode_length                        9.615977
returns                               5.206982
return_std                            2.228508
average_reward                        0.541653
round_time              0 days 00:09:39.221978
episodes_test                           1035.0
episode_length_test                   9.659903
returns_test                          5.283513
return_std_test                       2.388628
average_reward_test                   0.547041
round_time_test         0 days 00:00:09.300984
round_time_total        0 days 00:09:39.223028
loss_total             111035431844893818880.0
loss_critic            138794287471854075904.0
loss_actor                    -32097686193.152
memory_size                             4135.0 

=== epoch 9/10 ===== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:21,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  1038
episode_length                        9.627168
returns                               5.219903
return_std                              2.5541
average_reward                        0.542235
round_time              0 days 00:09:39.461992
episodes_test                           1055.0
episode_length_test                   9.475829
returns_test                           5.18797
return_std_test                       2.606745
average_reward_test                   0.547625
round_time_test         0 days 00:00:09.227929
round_time_total        0 days 00:09:39.463074
loss_total             112695200882643402752.0
loss_critic            140868998722311815168.0
loss_actor                 -32320347492.352001
memory_size                          4139.1895 

=== epoch 9/10 ===== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:09,  3.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1039
episode_length                        9.617902
returns                               5.200661
return_std                            2.800269
average_reward                         0.54071
round_time              0 days 00:09:41.041345
episodes_test                           1051.0
episode_length_test                    9.50999
returns_test                          5.130329
return_std_test                       2.250212
average_reward_test                   0.539639
round_time_test         0 days 00:00:09.375029
round_time_total        0 days 00:09:41.042413
loss_total             113494560589283573760.0
loss_critic            141868198333072048128.0
loss_actor                 -32474917647.360001
memory_size                             4141.0 

=== epoch 9/10 ===== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1035
episode_length                        9.658937
returns                               5.188943
return_std                             2.81664
average_reward                        0.537347
round_time              0 days 00:09:39.847898
episodes_test                           1053.0
episode_length_test                   9.494777
returns_test                          5.070766
return_std_test                       2.678131
average_reward_test                   0.534127
round_time_test         0 days 00:00:09.531578
round_time_total        0 days 00:09:39.848990
loss_total             115365812171818811392.0
loss_critic            144207262745270386688.0
loss_actor                 -32662140770.304001
memory_size                             4141.0 

=== epoch 9/10 ===== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:58,  3.03it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
episodes                                  1036
episode_length                        9.643822
returns                               5.211218
return_std                            2.820795
average_reward                        0.540345
round_time              0 days 00:09:39.980296
episodes_test                           1046.0
episode_length_test                   9.556405
returns_test                           5.02717
return_std_test                       1.885445
average_reward_test                   0.526238
round_time_test         0 days 00:00:09.289409
round_time_total        0 days 00:09:39.981651
loss_total             116859253403914207232.0
loss_critic            146074064349161308160.0
loss_actor                 -32867737947.136002
memory_size                             4141.0 


<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
=== epoch 10/10 ==== round 1/50 ======================================
  0%|          | 7/2000 [00:02<10:01,  3.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                   208
episode_length                        9.586538
returns                               5.039393
return_std                            2.176733
average_reward                        0.526414
round_time              0 days 00:09:38.209276
episodes_test                           1040.0
episode_length_test                   9.608654
returns_test                           5.15665
return_std_test                        2.32601
average_reward_test                   0.536749
round_time_test         0 days 00:00:09.273460
round_time_total        0 days 00:09:38.210400
loss_total             117150801989610307584.0
loss_critic            146438500068637065216.0
loss_actor                    -33139117170.688
memory_size                             4141.0 

=== epoch 10/10 ==== round 2/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:22,  3.55it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                   418
episode_length                        9.569378
returns                               5.142971
return_std                            1.967465
average_reward                         0.53744
round_time              0 days 00:09:38.262712
episodes_test                           1063.0
episode_length_test                   9.401693
returns_test                          5.070894
return_std_test                       2.447142
average_reward_test                   0.539499
round_time_test         0 days 00:00:09.282639
round_time_total        0 days 00:09:38.263917
loss_total             118905759546689634304.0
loss_critic            148632196924826255360.0
loss_actor                 -33342813563.903999
memory_size                             4141.0 

=== epoch 10/10 ==== round 3/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:05,  3.29it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                   623
episode_length                        9.622793
returns                               5.131613
return_std                            1.893227
average_reward                        0.533556
round_time              0 days 00:09:40.293779
episodes_test                           1054.0
episode_length_test                   9.483871
returns_test                          5.110122
return_std_test                       2.246776
average_reward_test                   0.538986
round_time_test         0 days 00:00:09.596981
round_time_total        0 days 00:09:40.294896
loss_total             120183757498447020032.0
loss_critic            150229694512407314432.0
loss_actor                 -33487610993.664001
memory_size                           4145.704 

=== epoch 10/10 ==== round 4/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:44,  3.10it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                   833
episode_length                        9.602641
returns                               5.188557
return_std                            1.924939
average_reward                         0.54038
round_time              0 days 00:09:39.613098
episodes_test                           1051.0
episode_length_test                   9.510942
returns_test                          5.024404
return_std_test                       1.816176
average_reward_test                   0.528383
round_time_test         0 days 00:00:09.424599
round_time_total        0 days 00:09:39.614535
loss_total             121480789377467809792.0
loss_critic            151850984291364306944.0
loss_actor                 -33561974438.911999
memory_size                             4148.0 

=== epoch 10/10 ==== round 5/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  1043
episode_length                        9.581975
returns                               5.156425
return_std                            1.920083
average_reward                         0.53833
round_time              0 days 00:09:39.467836
episodes_test                           1060.0
episode_length_test                   9.425472
returns_test                          5.045069
return_std_test                       2.033951
average_reward_test                   0.535539
round_time_test         0 days 00:00:09.365023
round_time_total        0 days 00:09:39.468930
loss_total             121877184772951949312.0
loss_critic            152346478489540001792.0
loss_actor                 -33681581333.504002
memory_size                             4148.0 

=== epoch 10/10 ==== round 6/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:05,  3.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1048
episode_length                        9.539122
returns                               5.128917
return_std                            1.841494
average_reward                        0.537399
round_time              0 days 00:09:39.336022
episodes_test                           1045.0
episode_length_test                   9.564593
returns_test                          5.149633
return_std_test                       2.537018
average_reward_test                   0.538597
round_time_test         0 days 00:00:09.332184
round_time_total        0 days 00:09:39.337107
loss_total             123182514235219001344.0
loss_critic            153978140256350928896.0
loss_actor                 -33972109687.807999
memory_size                             4148.0 

=== epoch 10/10 ==== round 7/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:01,  3.01it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1049
episode_length                        9.529075
returns                               5.098856
return_std                            1.866465
average_reward                        0.535268
round_time              0 days 00:09:38.696793
episodes_test                           1060.0
episode_length_test                   9.431132
returns_test                          4.952506
return_std_test                        1.82491
average_reward_test                    0.52527
round_time_test         0 days 00:00:09.262166
round_time_total        0 days 00:09:38.698007
loss_total             125841156146256445440.0
loss_critic            157301442391160520704.0
loss_actor                    -34092322885.632
memory_size                             4148.0 

=== epoch 10/10 ==== round 8/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:14,  3.25it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1049
episode_length                        9.523356
returns                               5.100098
return_std                            1.888302
average_reward                        0.535181
round_time              0 days 00:09:39.820123
episodes_test                           1056.0
episode_length_test                   9.464962
returns_test                          5.141939
return_std_test                        2.55965
average_reward_test                   0.543521
round_time_test         0 days 00:00:09.423476
round_time_total        0 days 00:09:39.821380
loss_total             126047092180446511104.0
loss_critic            157558862683487272960.0
loss_actor                 -34353604094.976002
memory_size                             4148.0 

=== epoch 10/10 ==== round 9/50 ======================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:37,  3.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1047
episode_length                        9.540592
returns                                5.05388
return_std                            1.831189
average_reward                        0.529565
round_time              0 days 00:09:39.877960
episodes_test                           1054.0
episode_length_test                   9.482922
returns_test                          5.157829
return_std_test                       2.321495
average_reward_test                   0.544011
round_time_test         0 days 00:00:09.311025
round_time_total        0 days 00:09:39.879034
loss_total             128338797561398132736.0
loss_critic            160423494434415771648.0
loss_actor                 -34552751389.695999
memory_size                             4148.0 

=== epoch 10/10 ==== round 10/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:07,  2.99it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                  1049
episode_length                        9.524309
returns                               5.043539
return_std                             1.82679
average_reward                        0.529446
round_time              0 days 00:09:37.995119
episodes_test                           1052.0
episode_length_test                   9.504753
returns_test                          5.007222
return_std_test                       2.210881
average_reward_test                   0.526863
round_time_test         0 days 00:00:09.387962
round_time_total        0 days 00:09:37.996353
loss_total             127408769328851943424.0
loss_critic            159260959195959853056.0
loss_actor                 -34387522920.447998
memory_size                             4148.0 

=== epoch 10/10 ==== round 11/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<09:23,  3.54it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1050
episode_length                        9.519048
returns                               5.076189
return_std                            1.826409
average_reward                        0.533399
round_time              0 days 00:09:39.104083
episodes_test                           1055.0
episode_length_test                   9.476777
returns_test                          5.042967
return_std_test                       1.920707
average_reward_test                   0.532234
round_time_test         0 days 00:00:09.360516
round_time_total        0 days 00:09:39.105342
loss_total             127050118403932700672.0
loss_critic            158812645406769905664.0
loss_actor                 -34217845715.967999
memory_size                             4148.0 

=== epoch 10/10 ==== round 12/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:00,  3.02it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1045
episode_length                        9.559809
returns                               5.142092
return_std                            2.319146
average_reward                         0.53788
round_time              0 days 00:09:39.609297
episodes_test                           1059.0
episode_length_test                   9.437205
returns_test                          5.080796
return_std_test                        2.41405
average_reward_test                   0.538665
round_time_test         0 days 00:00:09.383549
round_time_total        0 days 00:09:39.610490
loss_total             125552205611978768384.0
loss_critic            156940254391538712576.0
loss_actor                 -34526761339.903999
memory_size                          4152.4865 

=== epoch 10/10 ==== round 13/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<10:03,  3.30it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1048
episode_length                        9.534351
returns                               5.137267
return_std                            2.318422
average_reward                        0.538976
round_time              0 days 00:09:39.706100
episodes_test                           1052.0
episode_length_test                   9.504753
returns_test                          5.043542
return_std_test                       2.132826
average_reward_test                   0.530687
round_time_test         0 days 00:00:09.444879
round_time_total        0 days 00:09:39.707389
loss_total             128478120398301954048.0
loss_critic            160597647721610575872.0
loss_actor                 -34609204225.024002
memory_size                             4154.0 

=== epoch 10/10 ==== round 14/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:29,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1052
episode_length                        9.494297
returns                               5.155175
return_std                            2.574164
average_reward                         0.54311
round_time              0 days 00:09:41.135323
episodes_test                           1056.0
episode_length_test                   9.464015
returns_test                          5.044892
return_std_test                       2.357521
average_reward_test                   0.533355
round_time_test         0 days 00:00:09.421704
round_time_total        0 days 00:09:41.136400
loss_total             128586646688567951360.0
loss_critic            160733305752118591488.0
loss_actor                 -34892910456.832001
memory_size                             4154.0 

=== epoch 10/10 ==== round 15/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1050
episode_length                        9.514286
returns                               5.188333
return_std                            2.607315
average_reward                        0.544932
round_time              0 days 00:09:40.555138
episodes_test                           1038.0
episode_length_test                   9.633911
returns_test                          5.238179
return_std_test                       3.338263
average_reward_test                   0.543723
round_time_test         0 days 00:00:09.243190
round_time_total        0 days 00:09:40.556241
loss_total             131111227976973582336.0
loss_critic            163889032335687614464.0
loss_actor                 -35124220652.543999
memory_size                             4154.0 

=== epoch 10/10 ==== round 16/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:55,  3.05it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  1049
episode_length                        9.524309
returns                               5.141438
return_std                            2.631284
average_reward                        0.539868
round_time              0 days 00:09:39.514543
episodes_test                           1046.0
episode_length_test                   9.557361
returns_test                          5.064922
return_std_test                       1.968537
average_reward_test                   0.530047
round_time_test         0 days 00:00:09.331900
round_time_total        0 days 00:09:39.515776
loss_total             133044006433597161472.0
loss_critic            166305005244289122304.0
loss_actor                 -35341511548.928001
memory_size                             4154.0 

=== epoch 10/10 ==== round 17/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 6/2000 [00:01<10:02,  3.31it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1049
episode_length                        9.515729
returns                               5.132147
return_std                            2.244949
average_reward                        0.539511
round_time              0 days 00:09:40.292310
episodes_test                           1052.0
episode_length_test                   9.498099
returns_test                          4.962036
return_std_test                       2.202282
average_reward_test                    0.52247
round_time_test         0 days 00:00:09.534130
round_time_total        0 days 00:09:40.293399
loss_total             135226610398337581056.0
loss_critic            169033259978663067648.0
loss_actor                    -35671618221.056
memory_size                             4154.0 

=== epoch 10/10 ==== round 18/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1055
episode_length                        9.467299
returns                               5.129521
return_std                            2.213791
average_reward                        0.541999
round_time              0 days 00:09:39.741316
episodes_test                           1054.0
episode_length_test                   9.483871
returns_test                          5.051669
return_std_test                       2.372604
average_reward_test                   0.532809
round_time_test         0 days 00:00:09.414320
round_time_total        0 days 00:09:39.742670
loss_total             136741575056530833408.0
loss_critic            170926965816248041472.0
loss_actor                 -35851803189.248001
memory_size                             4154.0 

=== epoch 10/10 ==== round 19/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:10,  3.27it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1054
episode_length                         9.48482
returns                               5.081169
return_std                            1.936141
average_reward                        0.535497
round_time              0 days 00:09:40.126538
episodes_test                           1052.0
episode_length_test                   9.498099
returns_test                          5.167959
return_std_test                       2.854022
average_reward_test                    0.54422
round_time_test         0 days 00:00:09.334542
round_time_total        0 days 00:09:40.127608
loss_total             138649455835404500992.0
loss_critic            173311816964662460416.0
loss_actor                 -36070668030.975998
memory_size                             4154.0 

=== epoch 10/10 ==== round 20/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:29,  2.89it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  1057
episode_length                        9.451277
returns                                5.05844
return_std                            1.850891
average_reward                        0.534966
round_time              0 days 00:09:41.639690
episodes_test                           1038.0
episode_length_test                   9.629094
returns_test                          5.298658
return_std_test                       2.484874
average_reward_test                   0.550351
round_time_test         0 days 00:00:09.197434
round_time_total        0 days 00:09:41.640766
loss_total             140002765636550868992.0
loss_critic            175003454051718430720.0
loss_actor                 -36346810802.176003
memory_size                             4154.0 

=== epoch 10/10 ==== round 21/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:54,  3.36it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  1057
episode_length                        9.456008
returns                               5.084053
return_std                            1.829393
average_reward                        0.537675
round_time              0 days 00:09:39.510564
episodes_test                           1054.0
episode_length_test                   9.481025
returns_test                           5.13433
return_std_test                        2.72927
average_reward_test                   0.541651
round_time_test         0 days 00:00:09.455710
round_time_total        0 days 00:09:39.511673
loss_total             141717977194088316928.0
loss_critic            177147468503038296064.0
loss_actor                    -36575717586.944
memory_size                             4154.0 

=== epoch 10/10 ==== round 22/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:39,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1060
episode_length                        9.423585
returns                               5.089298
return_std                            2.147949
average_reward                        0.540102
round_time              0 days 00:09:41.048745
episodes_test                           1062.0
episode_length_test                   9.415254
returns_test                          5.024172
return_std_test                       1.960997
average_reward_test                   0.533662
round_time_test         0 days 00:00:09.273305
round_time_total        0 days 00:09:41.049809
loss_total             143902784111738847232.0
loss_critic            179878477067638079488.0
loss_actor                 -36707572150.272003
memory_size                           4154.313 

=== epoch 10/10 ==== round 23/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:45,  3.09it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1051
episode_length                         9.50999
returns                               5.131232
return_std                            2.160555
average_reward                        0.539487
round_time              0 days 00:09:40.701716
episodes_test                           1049.0
episode_length_test                   9.530029
returns_test                          5.166571
return_std_test                       2.089854
average_reward_test                   0.542243
round_time_test         0 days 00:00:09.316399
round_time_total        0 days 00:09:40.703014
loss_total             145826023467163615232.0
loss_critic            182282526071703502848.0
loss_actor                    -36947713486.848
memory_size                             4158.0 

=== epoch 10/10 ==== round 24/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:36,  3.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.45it/s]
episodes                                  1047
episode_length                        9.548233
returns                               5.197869
return_std                            2.163838
average_reward                        0.544496
round_time              0 days 00:09:39.448316
episodes_test                           1055.0
episode_length_test                   9.472986
returns_test                          5.106196
return_std_test                       2.094287
average_reward_test                   0.539208
round_time_test         0 days 00:00:09.286527
round_time_total        0 days 00:09:39.449651
loss_total             146568280752097787904.0
loss_critic            183210347744941441024.0
loss_actor                 -37201557950.463997
memory_size                             4158.0 

=== epoch 10/10 ==== round 25/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:38,  3.12it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1045
episode_length                        9.556938
returns                                5.21698
return_std                            2.159724
average_reward                        0.545958
round_time              0 days 00:09:39.306571
episodes_test                           1053.0
episode_length_test                   9.494777
returns_test                          5.248133
return_std_test                       2.183447
average_reward_test                   0.552807
round_time_test         0 days 00:00:09.309383
round_time_total        0 days 00:09:39.307671
loss_total             150369805855278891008.0
loss_critic            187962254028260311040.0
loss_actor                 -37300863827.968002
memory_size                             4158.0 

=== epoch 10/10 ==== round 26/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:50,  3.38it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1040
episode_length                        9.594231
returns                               5.227357
return_std                              2.3662
average_reward                        0.544846
round_time              0 days 00:09:40.185715
episodes_test                           1042.0
episode_length_test                   9.588292
returns_test                          5.144763
return_std_test                        2.19025
average_reward_test                   0.536644
round_time_test         0 days 00:00:09.296329
round_time_total        0 days 00:09:40.186818
loss_total             151335305562191134720.0
loss_critic            189169128418908569600.0
loss_actor                 -37552813257.727997
memory_size                          4158.4535 

=== epoch 10/10 ==== round 27/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:28,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1039
episode_length                        9.612127
returns                               5.189749
return_std                            2.186073
average_reward                        0.539747
round_time              0 days 00:09:40.753430
episodes_test                           1039.0
episode_length_test                   9.617902
returns_test                          5.176688
return_std_test                       2.118255
average_reward_test                   0.538407
round_time_test         0 days 00:00:09.369847
round_time_total        0 days 00:09:40.754504
loss_total             152209510472291319808.0
loss_critic            190261884774786859008.0
loss_actor                 -37743066904.575996
memory_size                           4160.475 

=== epoch 10/10 ==== round 28/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  1040
episode_length                        9.600962
returns                               5.145808
return_std                            2.227717
average_reward                        0.535969
round_time              0 days 00:09:41.733645
episodes_test                           1048.0
episode_length_test                   9.539122
returns_test                          5.158169
return_std_test                       2.224364
average_reward_test                   0.540842
round_time_test         0 days 00:00:09.350294
round_time_total        0 days 00:09:41.734725
loss_total             154211077973950627840.0
loss_critic            192763844049056628736.0
loss_actor                 -37908707496.959999
memory_size                             4164.0 

=== epoch 10/10 ==== round 29/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:20,  3.21it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1043
episode_length                        9.581975
returns                               5.117529
return_std                            2.224543
average_reward                        0.534041
round_time              0 days 00:09:38.762466
episodes_test                           1043.0
episode_length_test                   9.581016
returns_test                          5.166963
return_std_test                       2.186939
average_reward_test                   0.539421
round_time_test         0 days 00:00:09.177298
round_time_total        0 days 00:09:38.763539
loss_total             155921606146316402688.0
loss_critic            194902004067151511552.0
loss_actor                 -38152864551.935997
memory_size                             4164.0 

=== epoch 10/10 ==== round 30/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  1039
episode_length                        9.618864
returns                               5.131067
return_std                            2.255882
average_reward                        0.533508
round_time              0 days 00:09:41.511400
episodes_test                           1026.0
episode_length_test                   9.739766
returns_test                          5.346078
return_std_test                       2.594625
average_reward_test                   0.549082
round_time_test         0 days 00:00:09.413456
round_time_total        0 days 00:09:41.512649
loss_total             157132256563359612928.0
loss_critic            196415317107147243520.0
loss_actor                 -38333696615.424004
memory_size                             4164.0 

=== epoch 10/10 ==== round 31/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:35,  3.14it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1044
episode_length                        9.574713
returns                               5.106681
return_std                            2.060443
average_reward                         0.53338
round_time              0 days 00:09:40.987408
episodes_test                           1046.0
episode_length_test                   9.560229
returns_test                          5.105703
return_std_test                       2.230375
average_reward_test                   0.534057
round_time_test         0 days 00:00:09.237958
round_time_total        0 days 00:09:40.988487
loss_total             159672558110467293184.0
loss_critic            199590694250488791040.0
loss_actor                 -38530909026.304001
memory_size                             4164.0 

=== epoch 10/10 ==== round 32/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:24,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1046
episode_length                        9.554493
returns                               5.083338
return_std                            1.908165
average_reward                        0.532059
round_time              0 days 00:09:39.563610
episodes_test                           1044.0
episode_length_test                   9.574713
returns_test                          5.176382
return_std_test                       2.474545
average_reward_test                   0.540808
round_time_test         0 days 00:00:09.399830
round_time_total        0 days 00:09:39.564691
loss_total             161702545157561647104.0
loss_critic            202128177914221199360.0
loss_actor                 -38668919078.912003
memory_size                             4164.0 

=== epoch 10/10 ==== round 33/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:23,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1050
episode_length                        9.518095
returns                               5.118275
return_std                            1.911135
average_reward                        0.537622
round_time              0 days 00:09:39.021368
episodes_test                           1035.0
episode_length_test                   9.655072
returns_test                           5.33901
return_std_test                       2.211163
average_reward_test                   0.553151
round_time_test         0 days 00:00:09.294280
round_time_total        0 days 00:09:39.022435
loss_total             162008506447338897408.0
loss_critic            202510629439581323264.0
loss_actor                 -38888848294.912003
memory_size                             4164.0 

=== epoch 10/10 ==== round 34/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:30,  3.17it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1054
episode_length                        9.485769
returns                               5.089629
return_std                            1.902706
average_reward                        0.536657
round_time              0 days 00:09:39.527624
episodes_test                           1049.0
episode_length_test                   9.529075
returns_test                          5.301281
return_std_test                       2.710757
average_reward_test                    0.55635
round_time_test         0 days 00:00:09.408021
round_time_total        0 days 00:09:39.528693
loss_total             162273676669364240384.0
loss_critic            202842092401830952960.0
loss_actor                 -39085633094.655998
memory_size                             4164.0 

=== epoch 10/10 ==== round 35/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:22,  3.20it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.44it/s]
episodes                                  1054
episode_length                        9.483871
returns                               5.080668
return_std                            1.925476
average_reward                        0.535678
round_time              0 days 00:09:41.252458
episodes_test                           1049.0
episode_length_test                   9.531935
returns_test                          5.167255
return_std_test                        2.38411
average_reward_test                   0.542145
round_time_test         0 days 00:00:09.390694
round_time_total        0 days 00:09:41.253543
loss_total             164287826654437310464.0
loss_critic            205359779481850576896.0
loss_actor                 -39148324247.552002
memory_size                             4164.0 

=== epoch 10/10 ==== round 36/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:17,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1054
episode_length                        9.478178
returns                                5.08604
return_std                            1.880071
average_reward                         0.53668
round_time              0 days 00:09:39.662322
episodes_test                           1052.0
episode_length_test                   9.502852
returns_test                           5.15003
return_std_test                       2.464487
average_reward_test                   0.542095
round_time_test         0 days 00:00:09.321589
round_time_total        0 days 00:09:39.663405
loss_total             163356599806882283520.0
loss_critic            204195746171995947008.0
loss_actor                 -39407016092.671997
memory_size                             4164.0 

=== epoch 10/10 ==== round 37/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:11,  3.26it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1052
episode_length                        9.497148
returns                               5.142047
return_std                            1.895285
average_reward                        0.541495
round_time              0 days 00:09:40.634849
episodes_test                           1042.0
episode_length_test                   9.596929
returns_test                          5.238393
return_std_test                       2.222393
average_reward_test                   0.545841
round_time_test         0 days 00:00:09.396385
round_time_total        0 days 00:09:40.635928
loss_total             168062001240950767616.0
loss_critic            210077497761721614336.0
loss_actor                 -39518074468.351997
memory_size                             4164.0 

=== epoch 10/10 ==== round 38/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:05,  3.00it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  1049
episode_length                        9.516683
returns                               5.138233
return_std                            1.861374
average_reward                        0.539873
round_time              0 days 00:09:41.677999
episodes_test                           1045.0
episode_length_test                   9.569378
returns_test                          5.179226
return_std_test                       2.285972
average_reward_test                   0.541229
round_time_test         0 days 00:00:09.209103
round_time_total        0 days 00:09:41.679241
loss_total             167877376501881176064.0
loss_critic            209846716700995420160.0
loss_actor                 -39748126273.536003
memory_size                             4164.0 

=== epoch 10/10 ==== round 39/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:31,  3.49it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:41<00:00,  3.44it/s]
episodes                                  1047
episode_length                        9.536772
returns                               5.173409
return_std                            1.862216
average_reward                        0.542603
round_time              0 days 00:09:41.993046
episodes_test                           1035.0
episode_length_test                    9.66087
returns_test                          5.366488
return_std_test                       3.385253
average_reward_test                   0.555528
round_time_test         0 days 00:00:09.146756
round_time_total        0 days 00:09:41.994125
loss_total             169700944556501991424.0
loss_critic            212126176911108440064.0
loss_actor                    -39950087886.848
memory_size                             4164.0 

=== epoch 10/10 ==== round 40/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:50,  3.06it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:40<00:00,  3.45it/s]
episodes                                  1044
episode_length                        9.570881
returns                               5.221522
return_std                            1.829127
average_reward                         0.54546
round_time              0 days 00:09:40.780452
episodes_test                           1047.0
episode_length_test                   9.551098
returns_test                          5.316305
return_std_test                       3.603021
average_reward_test                   0.556617
round_time_test         0 days 00:00:09.356619
round_time_total        0 days 00:09:40.781908
loss_total             171958191130888896512.0
loss_critic            214947735104902856704.0
loss_actor                 -40220312561.664001
memory_size                             4164.0 

=== epoch 10/10 ==== round 41/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:19,  3.22it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1042
episode_length                        9.587332
returns                               5.270026
return_std                             1.92183
average_reward                        0.549826
round_time              0 days 00:09:39.568467
episodes_test                           1048.0
episode_length_test                   9.532443
returns_test                          5.208524
return_std_test                       2.089647
average_reward_test                   0.546583
round_time_test         0 days 00:00:09.317985
round_time_total        0 days 00:09:39.569556
loss_total             171344612396551962624.0
loss_critic            214180761713369939968.0
loss_actor                 -40502529689.599998
memory_size                             4164.0 

=== epoch 10/10 ==== round 42/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<11:14,  2.96it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:39<00:00,  3.45it/s]
episodes                                  1042
episode_length                        9.585413
returns                               5.234976
return_std                            1.926717
average_reward                        0.546183
round_time              0 days 00:09:40.463093
episodes_test                           1036.0
episode_length_test                   9.646718
returns_test                          5.147837
return_std_test                       2.301925
average_reward_test                   0.533736
round_time_test         0 days 00:00:09.355754
round_time_total        0 days 00:09:40.464406
loss_total             176839309491827277824.0
loss_critic            221049132961517797376.0
loss_actor                 -40594887956.480003
memory_size                             4164.0 

=== epoch 10/10 ==== round 43/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:24,  3.19it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:38<00:00,  3.46it/s]
episodes                                  1039
episode_length                        9.621752
returns                                5.30938
return_std                             1.99113
average_reward                        0.551774
round_time              0 days 00:09:39.104267
episodes_test                           1036.0
episode_length_test                   9.650579
returns_test                          5.298953
return_std_test                       2.638465
average_reward_test                   0.549167
round_time_test         0 days 00:00:09.274703
round_time_total        0 days 00:09:39.105351
loss_total             177524779191413997568.0
loss_critic            221905970089299738624.0
loss_actor                 -40908373843.968002
memory_size                             4164.0 

=== epoch 10/10 ==== round 44/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:18,  3.23it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:37<00:00,  3.46it/s]
episodes                                  1036
episode_length                        9.640927
returns                                5.35305
return_std                            2.233525
average_reward                        0.555097
round_time              0 days 00:09:38.556424
episodes_test                           1038.0
episode_length_test                   9.627168
returns_test                          5.211875
return_std_test                       2.045152
average_reward_test                   0.541525
round_time_test         0 days 00:00:09.233498
round_time_total        0 days 00:09:38.557490
loss_total             179916837942013689856.0
loss_critic            224896043397806981120.0
loss_actor                 -41030553714.688004
memory_size                           4164.796 

=== epoch 10/10 ==== round 45/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:40,  3.11it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:36<00:00,  3.47it/s]
episodes                                  1038
episode_length                        9.623314
returns                               5.320153
return_std                            2.244143
average_reward                        0.552852
round_time              0 days 00:09:36.996818
episodes_test                           1041.0
episode_length_test                   9.603266
returns_test                           5.25986
return_std_test                       2.368502
average_reward_test                   0.547821
round_time_test         0 days 00:00:09.336698
round_time_total        0 days 00:09:36.997902
loss_total             181624761729774059520.0
loss_critic            227030948012660686848.0
loss_actor                    -41157909553.152
memory_size                             4165.0 

=== epoch 10/10 ==== round 46/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:27,  3.18it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:32<00:00,  3.49it/s]
episodes                                  1035
episode_length                         9.65314
returns                               5.345069
return_std                            2.228794
average_reward                        0.553783
round_time              0 days 00:09:33.083554
episodes_test                           1053.0
episode_length_test                   9.493827
returns_test                          5.179798
return_std_test                       2.052729
average_reward_test                   0.545702
round_time_test         0 days 00:00:09.396682
round_time_total        0 days 00:09:33.084614
loss_total             181452267280170123264.0
loss_critic            226815330224434184192.0
loss_actor                 -41474514071.552002
memory_size                             4165.0 

=== epoch 10/10 ==== round 47/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:37,  3.13it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:31<00:00,  3.50it/s]
episodes                                  1033
episode_length                        9.672798
returns                                5.38049
return_std                            2.229538
average_reward                        0.556064
round_time              0 days 00:09:32.308022
episodes_test                           1038.0
episode_length_test                   9.633911
returns_test                          5.117801
return_std_test                        1.93257
average_reward_test                   0.531228
round_time_test         0 days 00:00:09.322694
round_time_total        0 days 00:09:32.309285
loss_total             183677274678294839296.0
loss_critic            229596589408318390272.0
loss_actor                 -41671607525.375999
memory_size                             4165.0 

=== epoch 10/10 ==== round 48/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:08,  3.28it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:33<00:00,  3.49it/s]
episodes                                  1040
episode_length                        9.608654
returns                               5.274271
return_std                            2.204334
average_reward                        0.549008
round_time              0 days 00:09:34.061311
episodes_test                           1061.0
episode_length_test                   9.419416
returns_test                          5.081933
return_std_test                       2.584063
average_reward_test                   0.539707
round_time_test         0 days 00:00:09.311072
round_time_total        0 days 00:09:34.062784
loss_total             186090482086454067200.0
loss_critic            232613098582755540992.0
loss_actor                 -41646132420.608002
memory_size                             4165.0 

=== epoch 10/10 ==== round 49/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<09:36,  3.46it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:34<00:00,  3.48it/s]
episodes                                  1042
episode_length                        9.590211
returns                               5.239232
return_std                            2.404681
average_reward                        0.546344
round_time              0 days 00:09:34.727147
episodes_test                           1065.0
episode_length_test                   9.387793
returns_test                          5.123696
return_std_test                       2.339814
average_reward_test                   0.545867
round_time_test         0 days 00:00:09.337245
round_time_total        0 days 00:09:34.728442
loss_total             185954701882612023296.0
loss_critic            232443373283972481024.0
loss_actor                 -41732096688.127998
memory_size                            4166.47 

=== epoch 10/10 ==== round 50/50 =====================================
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
  0%|          | 5/2000 [00:01<10:33,  3.15it/s]/<ANONYMIZED PATH>/python3.12/site-packages/gymnasium/envs/registration.py:519: DeprecationWarning: [33mWARN: The environment Hopper-v4 is out of date. You should consider upgrading to version `v5`.[0m
  logger.deprecation(
100%|██████████| 2000/2000 [09:23<00:00,  3.55it/s]
<GE123_Delay<NormalizeActionWrapper<Float64ToFloat32<CompatWrapper<NoisyActionWrapper<TimeLimit<OrderEnforcing<PassiveEnvChecker<HopperEnv<Hopper-v4>>>>>>>>>>
episodes                                  1049
episode_length                        9.520496
returns                               5.206424
return_std                            2.398329
average_reward                        0.547362
round_time              0 days 00:09:23.805031
episodes_test                           1049.0
episode_length_test                   9.530029
returns_test                          5.150431
return_std_test                       1.924415
average_reward_test                   0.540555
round_time_test         0 days 00:00:09.198494
round_time_total        0 days 00:09:23.806083
loss_total             186404353874209996800.0
loss_critic            233005438262474833920.0
loss_actor                 -41947261177.856003
memory_size                             4173.0 


