2025-08-07 09:17:03,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1108 [DEBUG]: logdir: _logs/benchmark-v3-tc8/noiseperc5-halfcheetah/MM1Queue_a033_s075-bpql-mem16
2025-08-07 09:17:03,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1109 [DEBUG]: trainer_prefix: benchmark-v3-tc8/noiseperc5-halfcheetah/MM1Queue_a033_s075-bpql-mem16
2025-08-07 09:17:03,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1110 [DEBUG]: args.trainer_eval_latencies: {'MM1Queue_a033_s075': <latency_env.delayed_mdp.MM1QueueDelay object at 0x14a343d9bcd0>}
2025-08-07 09:17:03,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1111 [DEBUG]: using device: cuda
2025-08-07 09:17:03,579 baseline-bpql-noiseperc5-halfcheetah:77 [WARNING]: args.assumed_delay != args.horizon: 16 != 24
2025-08-07 09:17:03,580 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1133 [INFO]: Creating new trainer
2025-08-07 09:17:03,596 baseline-bpql-noiseperc5-halfcheetah:113 [DEBUG]: pi network:
NNGaussianPolicy(
  (common_head): Sequential(
    (0): Flatten(start_dim=1, end_dim=-1)
    (1): Linear(in_features=113, out_features=256, bias=True)
    (2): ReLU()
    (3): Linear(in_features=256, out_features=256, bias=True)
    (4): ReLU()
  )
  (mu_head): Sequential(
    (0): Linear(in_features=256, out_features=6, bias=True)
    (1): Unflatten(dim=1, unflattened_size=(6,))
  )
  (log_std_head): Sequential(
    (0): Linear(in_features=256, out_features=6, bias=True)
    (1): Unflatten(dim=1, unflattened_size=(6,))
  )
  (tanh_refit): NNTanhRefit(scale: tensor([[2., 2., 2., 2., 2., 2.]]), shift: tensor([[-1., -1., -1., -1., -1., -1.]]))
)
2025-08-07 09:17:03,596 baseline-bpql-noiseperc5-halfcheetah:114 [DEBUG]: q network:
NNLayerConcat2(
  dim: -1
  (next): Sequential(
    (0): Linear(in_features=23, out_features=256, bias=True)
    (1): ReLU()
    (2): Linear(in_features=256, out_features=256, bias=True)
    (3): ReLU()
    (4): Linear(in_features=256, out_features=1, bias=True)
    (5): NNLayerSqueeze(dim: -1)
  )
  (init_left): Flatten(start_dim=1, end_dim=-1)
  (init_right): Flatten(start_dim=1, end_dim=-1)
)
2025-08-07 09:17:04,852 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1194 [DEBUG]: Starting training session...
2025-08-07 09:17:04,852 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 1/100
2025-08-07 09:18:42,857 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:18:54,446 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: -473.42609 ± 24.824
2025-08-07 09:18:54,447 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [-458.93484, -480.52713, -442.66946, -503.75555, -495.47784, -523.50793, -458.50827, -456.17157, -455.81525, -458.89328]
2025-08-07 09:18:54,447 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:18:54,447 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (-473.43) for latency MM1Queue_a033_s075
2025-08-07 09:18:54,489 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 2/100 (estimated time remaining: 3 hours, 54 seconds)
2025-08-07 09:20:39,259 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:20:52,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: -232.31567 ± 49.610
2025-08-07 09:20:52,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [-196.78514, -271.96112, -210.70364, -247.02127, -197.97858, -234.73032, -211.22589, -188.06795, -361.4013, -203.28162]
2025-08-07 09:20:52,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:20:52,221 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (-232.32) for latency MM1Queue_a033_s075
2025-08-07 09:20:52,229 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 3/100 (estimated time remaining: 3 hours, 5 minutes, 41 seconds)
2025-08-07 09:22:36,995 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:22:48,572 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: -147.29898 ± 73.539
2025-08-07 09:22:48,572 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [-154.66415, -18.80942, -163.79039, -90.73686, -154.96994, -169.97635, -279.61475, -167.64223, -223.89116, -48.894615]
2025-08-07 09:22:48,572 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:22:48,572 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (-147.30) for latency MM1Queue_a033_s075
2025-08-07 09:22:48,582 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 4/100 (estimated time remaining: 3 hours, 5 minutes, 13 seconds)
2025-08-07 09:24:35,343 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:24:46,887 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 19.26864 ± 87.841
2025-08-07 09:24:46,887 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [-32.66221, -107.352005, -8.707547, -36.93489, 175.26717, -24.604254, -53.647316, 42.37304, 153.3554, 85.599]
2025-08-07 09:24:46,887 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:24:46,888 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (19.27) for latency MM1Queue_a033_s075
2025-08-07 09:24:46,895 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 5/100 (estimated time remaining: 3 hours, 4 minutes, 49 seconds)
2025-08-07 09:26:31,277 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:26:42,600 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 196.93854 ± 145.069
2025-08-07 09:26:42,600 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [347.70157, 190.86081, 423.95947, 133.89754, 225.1771, 215.41193, -88.66827, 14.434268, 323.30597, 183.30501]
2025-08-07 09:26:42,600 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:26:42,600 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (196.94) for latency MM1Queue_a033_s075
2025-08-07 09:26:42,608 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 6/100 (estimated time remaining: 3 hours, 2 minutes, 57 seconds)
2025-08-07 09:28:28,481 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:28:39,925 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 224.30783 ± 150.303
2025-08-07 09:28:39,925 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [100.48511, 107.77155, 224.30664, 86.5366, 145.70735, 383.51047, 164.30946, 556.0399, 364.7072, 109.70413]
2025-08-07 09:28:39,925 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:28:39,925 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (224.31) for latency MM1Queue_a033_s075
2025-08-07 09:28:39,935 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 7/100 (estimated time remaining: 3 hours, 3 minutes, 26 seconds)
2025-08-07 09:30:23,914 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:30:35,452 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 513.14325 ± 207.754
2025-08-07 09:30:35,452 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [282.50134, 507.8778, 382.05582, 900.072, 699.3589, 520.0534, 466.76443, 787.6421, 323.7889, 261.3181]
2025-08-07 09:30:35,452 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:30:35,452 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (513.14) for latency MM1Queue_a033_s075
2025-08-07 09:30:35,461 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 8/100 (estimated time remaining: 3 hours, 48 seconds)
2025-08-07 09:32:22,330 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:32:35,265 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 598.94122 ± 218.849
2025-08-07 09:32:35,265 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [597.3884, 362.90317, 775.3815, 605.1234, 667.776, 242.9149, 955.6907, 437.3495, 452.91742, 891.9674]
2025-08-07 09:32:35,265 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:32:35,265 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (598.94) for latency MM1Queue_a033_s075
2025-08-07 09:32:35,275 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 9/100 (estimated time remaining: 2 hours, 59 minutes, 55 seconds)
2025-08-07 09:34:20,487 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:34:31,915 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1264.38733 ± 465.696
2025-08-07 09:34:31,915 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [925.49115, 1185.7537, 1834.7449, 1078.5714, 1564.5466, 1210.034, 1298.949, 705.71136, 2205.2246, 634.8458]
2025-08-07 09:34:31,915 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:34:31,915 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (1264.39) for latency MM1Queue_a033_s075
2025-08-07 09:34:31,941 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 10/100 (estimated time remaining: 2 hours, 57 minutes, 27 seconds)
2025-08-07 09:36:16,061 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:36:29,005 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1733.23181 ± 738.397
2025-08-07 09:36:29,005 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1436.7659, 854.56256, 2399.2017, 749.5415, 1828.3823, 970.87537, 2193.0344, 2774.5288, 2796.9175, 1328.5074]
2025-08-07 09:36:29,005 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:36:29,005 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (1733.23) for latency MM1Queue_a033_s075
2025-08-07 09:36:29,036 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 11/100 (estimated time remaining: 2 hours, 55 minutes, 55 seconds)
2025-08-07 09:38:14,424 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:38:25,975 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1541.80347 ± 486.051
2025-08-07 09:38:25,975 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1315.6838, 909.7794, 1902.1538, 914.9724, 1986.6897, 1398.3583, 1014.7724, 1588.0532, 2336.1602, 2051.412]
2025-08-07 09:38:25,975 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:38:26,038 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 12/100 (estimated time remaining: 2 hours, 53 minutes, 52 seconds)
2025-08-07 09:40:12,232 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:40:23,641 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1401.87866 ± 475.472
2025-08-07 09:40:23,641 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1350.436, 1924.4811, 1459.3733, 1043.8899, 1067.3508, 1041.9374, 1610.2445, 2517.9375, 1037.2673, 965.86847]
2025-08-07 09:40:23,641 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:40:23,653 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 13/100 (estimated time remaining: 2 hours, 52 minutes, 32 seconds)
2025-08-07 09:42:08,331 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:42:19,876 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1490.85742 ± 310.212
2025-08-07 09:42:19,876 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1144.4003, 1458.8425, 1197.1268, 1672.4346, 2114.8992, 1908.9803, 1305.4059, 1219.53, 1293.0516, 1593.9036]
2025-08-07 09:42:19,876 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:42:19,892 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 14/100 (estimated time remaining: 2 hours, 49 minutes, 32 seconds)
2025-08-07 09:44:05,216 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:44:16,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1491.99536 ± 552.160
2025-08-07 09:44:16,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1517.0662, 1246.8165, 2960.8484, 880.79, 1565.8593, 1649.4186, 1207.5189, 1690.4646, 1101.6702, 1099.501]
2025-08-07 09:44:16,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:44:16,737 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 15/100 (estimated time remaining: 2 hours, 47 minutes, 38 seconds)
2025-08-07 09:46:03,173 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:46:16,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3071.03174 ± 923.016
2025-08-07 09:46:16,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3672.1343, 3961.3494, 1514.5524, 2368.404, 3763.5403, 3188.1423, 1346.3538, 3573.0623, 3788.6929, 3534.0837]
2025-08-07 09:46:16,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:46:16,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (3071.03) for latency MM1Queue_a033_s075
2025-08-07 09:46:16,160 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 16/100 (estimated time remaining: 2 hours, 46 minutes, 21 seconds)
2025-08-07 09:48:00,913 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:48:12,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3741.28174 ± 703.256
2025-08-07 09:48:12,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3992.192, 4040.821, 4005.2483, 4137.7803, 1686.7075, 4015.0562, 4123.625, 3986.919, 3536.6106, 3887.8586]
2025-08-07 09:48:12,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:48:12,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (3741.28) for latency MM1Queue_a033_s075
2025-08-07 09:48:12,336 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 17/100 (estimated time remaining: 2 hours, 44 minutes, 9 seconds)
2025-08-07 09:49:57,461 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:50:10,403 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 1955.62854 ± 724.632
2025-08-07 09:50:10,403 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1810.8185, 3872.4316, 2660.2427, 1789.6637, 1882.7935, 1451.6143, 1589.5038, 1419.3528, 1590.9752, 1488.891]
2025-08-07 09:50:10,403 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:50:10,409 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 18/100 (estimated time remaining: 2 hours, 42 minutes, 20 seconds)
2025-08-07 09:51:56,242 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:52:07,666 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3492.36523 ± 761.884
2025-08-07 09:52:07,666 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3906.032, 2989.409, 4041.1035, 2132.527, 3004.7827, 4135.2095, 2349.7964, 4215.842, 3883.767, 4265.181]
2025-08-07 09:52:07,666 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:52:07,676 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 19/100 (estimated time remaining: 2 hours, 40 minutes, 39 seconds)
2025-08-07 09:53:53,784 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:54:05,249 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3468.56128 ± 914.773
2025-08-07 09:54:05,249 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3881.895, 4255.3438, 4241.664, 3847.2927, 2668.651, 4075.4973, 2641.0132, 1292.3121, 3731.1255, 4050.822]
2025-08-07 09:54:05,249 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:54:05,275 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 20/100 (estimated time remaining: 2 hours, 38 minutes, 54 seconds)
2025-08-07 09:55:50,933 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:56:02,422 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3053.57544 ± 1464.651
2025-08-07 09:56:02,423 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1749.1523, 4648.005, 4728.394, 1234.88, 2166.7942, 4714.616, 1199.2678, 2076.834, 4825.112, 3192.6997]
2025-08-07 09:56:02,423 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:56:02,439 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 21/100 (estimated time remaining: 2 hours, 36 minutes, 20 seconds)
2025-08-07 09:57:47,202 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:57:58,618 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3536.55322 ± 1172.230
2025-08-07 09:57:58,619 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4692.592, 2569.476, 5100.867, 2810.6318, 3037.9895, 2279.0242, 4951.8823, 3239.2122, 1842.2185, 4841.6377]
2025-08-07 09:57:58,619 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:57:58,625 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 22/100 (estimated time remaining: 2 hours, 34 minutes, 23 seconds)
2025-08-07 09:59:42,766 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 09:59:54,243 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3700.08276 ± 1403.525
2025-08-07 09:59:54,243 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5065.865, 2164.893, 5044.8804, 4825.536, 5052.8804, 1858.045, 3954.4434, 1483.6781, 2728.909, 4821.7]
2025-08-07 09:59:54,243 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 09:59:54,258 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 23/100 (estimated time remaining: 2 hours, 31 minutes, 48 seconds)
2025-08-07 10:01:38,673 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:01:50,109 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3420.18115 ± 1249.106
2025-08-07 10:01:50,109 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3020.0957, 4890.2065, 4187.6504, 4522.9634, 1279.0458, 2014.5, 3408.6008, 5257.226, 3442.5889, 2178.9329]
2025-08-07 10:01:50,109 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:01:50,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 24/100 (estimated time remaining: 2 hours, 29 minutes, 29 seconds)
2025-08-07 10:03:33,991 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:03:45,404 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3406.13477 ± 1262.785
2025-08-07 10:03:45,404 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4918.994, 3306.1475, 4238.966, 3241.9077, 1673.0889, 4921.076, 4856.4565, 2471.1504, 3171.1636, 1262.3976]
2025-08-07 10:03:45,404 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:03:45,409 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 25/100 (estimated time remaining: 2 hours, 26 minutes, 58 seconds)
2025-08-07 10:05:29,488 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:05:40,994 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3854.83984 ± 1253.780
2025-08-07 10:05:40,994 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5207.049, 3106.8179, 4216.9326, 4206.6025, 5130.933, 1937.47, 4517.5635, 4115.7095, 1324.8867, 4784.4355]
2025-08-07 10:05:40,994 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:05:40,994 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (3854.84) for latency MM1Queue_a033_s075
2025-08-07 10:05:41,021 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 26/100 (estimated time remaining: 2 hours, 24 minutes, 38 seconds)
2025-08-07 10:07:26,025 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:07:37,548 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3801.22998 ± 1341.755
2025-08-07 10:07:37,548 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4520.4927, 4910.7236, 4576.783, 3139.681, 1270.8762, 3421.9814, 4719.397, 5190.341, 4711.845, 1550.1786]
2025-08-07 10:07:37,548 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:07:37,555 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 27/100 (estimated time remaining: 2 hours, 22 minutes, 48 seconds)
2025-08-07 10:09:21,995 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:09:33,513 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4073.65356 ± 1134.768
2025-08-07 10:09:33,513 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4571.304, 2317.2349, 3581.493, 4961.927, 4146.743, 4803.2534, 1649.3501, 4768.4795, 5053.049, 4883.7007]
2025-08-07 10:09:33,513 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:09:33,513 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (4073.65) for latency MM1Queue_a033_s075
2025-08-07 10:09:33,531 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 28/100 (estimated time remaining: 2 hours, 20 minutes, 57 seconds)
2025-08-07 10:11:17,067 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:11:28,544 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3613.27222 ± 1331.976
2025-08-07 10:11:28,544 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5227.598, 2793.589, 4319.3994, 2750.8142, 5122.1025, 4485.7847, 1410.9266, 1971.9233, 2953.0264, 5097.5586]
2025-08-07 10:11:28,544 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:11:28,550 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 29/100 (estimated time remaining: 2 hours, 18 minutes, 49 seconds)
2025-08-07 10:13:14,082 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:13:27,050 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3853.61572 ± 1369.334
2025-08-07 10:13:27,051 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [2832.1506, 5277.3345, 4911.2607, 2569.493, 5024.702, 2544.395, 4158.4443, 5274.416, 4727.137, 1216.8231]
2025-08-07 10:13:27,051 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:13:27,059 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 30/100 (estimated time remaining: 2 hours, 17 minutes, 39 seconds)
2025-08-07 10:15:11,815 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:15:23,340 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3642.10229 ± 1342.499
2025-08-07 10:15:23,340 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3720.1953, 5217.026, 2593.701, 4729.874, 5129.287, 1258.03, 2351.6736, 4880.558, 4240.714, 2299.9666]
2025-08-07 10:15:23,341 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:15:23,380 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 31/100 (estimated time remaining: 2 hours, 15 minutes, 53 seconds)
2025-08-07 10:17:10,589 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:17:21,993 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3945.70166 ± 1408.672
2025-08-07 10:17:21,993 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1368.0835, 3412.9795, 1650.951, 5002.4487, 3038.0898, 5163.6133, 5080.5645, 4683.347, 5185.296, 4871.6406]
2025-08-07 10:17:21,993 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:17:22,005 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 32/100 (estimated time remaining: 2 hours, 14 minutes, 25 seconds)
2025-08-07 10:19:06,932 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:19:18,474 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3788.28906 ± 1640.867
2025-08-07 10:19:18,474 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5108.848, 3833.1758, 4850.3096, 5127.6245, 1276.4196, 4841.4795, 1539.0853, 5020.2593, 5070.055, 1215.6335]
2025-08-07 10:19:18,474 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:19:18,500 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 33/100 (estimated time remaining: 2 hours, 12 minutes, 35 seconds)
2025-08-07 10:21:01,993 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:21:13,499 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3842.35596 ± 1212.988
2025-08-07 10:21:13,499 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5212.9956, 5203.2847, 2913.9253, 3125.7847, 2041.6818, 3785.0432, 5072.2637, 4932.645, 1958.757, 4177.181]
2025-08-07 10:21:13,500 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:21:13,512 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 34/100 (estimated time remaining: 2 hours, 10 minutes, 38 seconds)
2025-08-07 10:22:58,823 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:23:11,775 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3963.06567 ± 1409.784
2025-08-07 10:23:11,775 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4885.116, 4833.662, 1716.3931, 5213.8433, 4924.5176, 2803.3425, 5113.8535, 5133.3306, 3605.5237, 1401.075]
2025-08-07 10:23:11,775 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:23:11,782 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 35/100 (estimated time remaining: 2 hours, 8 minutes, 38 seconds)
2025-08-07 10:24:58,114 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:25:09,617 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3698.51562 ± 1122.151
2025-08-07 10:25:09,617 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3196.6414, 4078.0532, 1982.0879, 2716.5115, 2880.7847, 4944.645, 4706.2837, 4890.482, 2451.444, 5138.2227]
2025-08-07 10:25:09,617 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:25:09,624 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 36/100 (estimated time remaining: 2 hours, 7 minutes, 1 second)
2025-08-07 10:26:53,962 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:27:05,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3751.09424 ± 1437.411
2025-08-07 10:27:05,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5058.2026, 5083.859, 4316.518, 1544.2881, 2559.9346, 1358.6779, 5072.453, 2930.814, 5149.5405, 4436.6562]
2025-08-07 10:27:05,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:27:05,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 37/100 (estimated time remaining: 2 hours, 4 minutes, 28 seconds)
2025-08-07 10:28:48,659 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:29:00,085 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4505.77197 ± 1059.134
2025-08-07 10:29:00,085 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4771.133, 4822.607, 5037.606, 4557.126, 5024.7085, 1401.2653, 4545.302, 5200.3643, 5108.363, 4589.2437]
2025-08-07 10:29:00,085 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:29:00,085 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (4505.77) for latency MM1Queue_a033_s075
2025-08-07 10:29:00,092 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 38/100 (estimated time remaining: 2 hours, 2 minutes, 8 seconds)
2025-08-07 10:30:44,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:30:56,165 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4460.82324 ± 713.028
2025-08-07 10:30:56,165 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5078.628, 2905.0862, 4786.6255, 4871.2417, 3479.3901, 5176.756, 4646.936, 3994.4075, 4825.7437, 4843.4165]
2025-08-07 10:30:56,165 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:30:56,179 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 39/100 (estimated time remaining: 2 hours, 25 seconds)
2025-08-07 10:32:40,053 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:32:51,526 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4086.88330 ± 911.070
2025-08-07 10:32:51,526 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4490.222, 4873.3467, 5100.448, 3163.7258, 1990.6205, 3865.9895, 4988.1436, 4025.4514, 3760.4963, 4610.387]
2025-08-07 10:32:51,526 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:32:51,534 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 40/100 (estimated time remaining: 1 hour, 57 minutes, 52 seconds)
2025-08-07 10:34:36,485 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:34:47,939 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4480.33594 ± 1125.185
2025-08-07 10:34:47,939 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3853.6838, 4460.598, 5247.187, 4682.2637, 5058.075, 1362.673, 5066.6074, 5290.887, 4538.9766, 5242.407]
2025-08-07 10:34:47,939 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:34:47,946 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 41/100 (estimated time remaining: 1 hour, 55 minutes, 39 seconds)
2025-08-07 10:36:32,636 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:36:44,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3597.53442 ± 1567.475
2025-08-07 10:36:44,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1483.6056, 3270.9072, 5057.5156, 5107.4907, 1433.8029, 5076.2705, 4755.806, 3434.8342, 5047.5396, 1307.5714]
2025-08-07 10:36:44,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:36:44,179 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 42/100 (estimated time remaining: 1 hour, 53 minutes, 48 seconds)
2025-08-07 10:38:29,713 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:38:42,601 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4620.14111 ± 1031.372
2025-08-07 10:38:42,601 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4917.2686, 5166.6006, 4740.6846, 4788.082, 5008.978, 5101.422, 4942.5767, 5055.7764, 4931.392, 1548.6274]
2025-08-07 10:38:42,601 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:38:42,601 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (4620.14) for latency MM1Queue_a033_s075
2025-08-07 10:38:42,607 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 43/100 (estimated time remaining: 1 hour, 52 minutes, 37 seconds)
2025-08-07 10:40:26,987 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:40:38,454 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4297.81885 ± 1009.701
2025-08-07 10:40:38,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4885.734, 5035.388, 4744.0425, 3220.5957, 4374.212, 4789.918, 1733.7631, 5088.1387, 4124.0317, 4982.365]
2025-08-07 10:40:38,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:40:38,463 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 44/100 (estimated time remaining: 1 hour, 50 minutes, 38 seconds)
2025-08-07 10:42:22,294 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:42:33,701 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4774.18604 ± 451.910
2025-08-07 10:42:33,702 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5021.7983, 5083.0527, 4716.508, 3547.087, 5126.464, 5197.7837, 4598.3267, 4903.8555, 4661.373, 4885.6074]
2025-08-07 10:42:33,702 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:42:33,702 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (4774.19) for latency MM1Queue_a033_s075
2025-08-07 10:42:33,708 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 45/100 (estimated time remaining: 1 hour, 48 minutes, 40 seconds)
2025-08-07 10:44:18,014 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:44:30,997 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4542.53223 ± 1137.002
2025-08-07 10:44:30,997 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4927.9116, 5103.639, 5060.6377, 5014.7603, 5111.548, 4756.5293, 4002.4512, 1269.3394, 5095.092, 5083.419]
2025-08-07 10:44:30,997 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:44:31,035 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 46/100 (estimated time remaining: 1 hour, 46 minutes, 53 seconds)
2025-08-07 10:46:15,615 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:46:28,591 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4139.25000 ± 1352.280
2025-08-07 10:46:28,591 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4099.908, 4951.2437, 5242.8945, 4841.9707, 5169.919, 2269.8328, 1417.9985, 2911.5234, 5294.3857, 5192.8247]
2025-08-07 10:46:28,591 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:46:28,614 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 47/100 (estimated time remaining: 1 hour, 45 minutes, 11 seconds)
2025-08-07 10:48:16,215 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:48:27,639 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3821.34814 ± 1018.609
2025-08-07 10:48:27,639 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3634.6943, 5006.2837, 4667.919, 3429.0103, 4605.53, 5073.038, 3529.5063, 2393.1567, 3985.028, 1889.3112]
2025-08-07 10:48:27,639 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:48:27,649 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 48/100 (estimated time remaining: 1 hour, 43 minutes, 21 seconds)
2025-08-07 10:50:12,833 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:50:25,807 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4273.99316 ± 1276.381
2025-08-07 10:50:25,807 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5059.072, 4824.4136, 3222.0115, 5230.5796, 1963.0841, 4791.418, 5177.1177, 5254.614, 5202.586, 2015.0356]
2025-08-07 10:50:25,807 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:50:25,826 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 49/100 (estimated time remaining: 1 hour, 41 minutes, 48 seconds)
2025-08-07 10:52:10,286 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:52:21,768 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4396.37012 ± 1283.603
2025-08-07 10:52:21,768 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4965.2427, 1710.6898, 4994.609, 1978.9928, 4810.971, 5139.046, 4870.873, 5176.7754, 5059.025, 5257.4795]
2025-08-07 10:52:21,769 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:52:21,777 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 50/100 (estimated time remaining: 1 hour, 39 minutes, 58 seconds)
2025-08-07 10:54:06,474 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:54:19,437 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4695.08496 ± 997.947
2025-08-07 10:54:19,437 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4718.891, 5149.662, 5035.7593, 5061.3545, 5168.656, 1758.6802, 4701.227, 5307.3867, 5201.6455, 4847.586]
2025-08-07 10:54:19,437 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:54:19,444 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 51/100 (estimated time remaining: 1 hour, 38 minutes, 4 seconds)
2025-08-07 10:56:03,830 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:56:15,352 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4360.09473 ± 1163.774
2025-08-07 10:56:15,353 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5213.331, 1283.9822, 5236.0986, 5053.6914, 4896.3594, 3413.6035, 4802.7197, 4522.478, 5134.2915, 4044.3857]
2025-08-07 10:56:15,353 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:56:15,375 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 52/100 (estimated time remaining: 1 hour, 35 minutes, 50 seconds)
2025-08-07 10:58:00,339 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 10:58:11,771 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4341.41504 ± 1201.974
2025-08-07 10:58:11,771 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3133.4844, 5035.155, 5204.603, 2286.7866, 5155.331, 4895.0957, 5156.0405, 5124.443, 2211.3792, 5211.83]
2025-08-07 10:58:11,771 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 10:58:11,779 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 53/100 (estimated time remaining: 1 hour, 33 minutes, 27 seconds)
2025-08-07 10:59:59,168 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:00:10,515 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4043.03271 ± 1224.527
2025-08-07 11:00:10,515 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4967.2705, 5162.503, 5216.391, 3273.4722, 5057.5854, 1695.4905, 3601.3127, 5206.452, 3881.0586, 2368.7922]
2025-08-07 11:00:10,516 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:00:10,523 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 54/100 (estimated time remaining: 1 hour, 31 minutes, 36 seconds)
2025-08-07 11:01:54,042 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:02:05,593 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4209.44775 ± 1503.027
2025-08-07 11:02:05,593 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5197.1694, 5090.208, 4967.8325, 5130.1187, 5155.183, 5175.952, 3717.166, 1228.4547, 5017.0938, 1415.2961]
2025-08-07 11:02:05,593 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:02:05,603 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 55/100 (estimated time remaining: 1 hour, 29 minutes, 31 seconds)
2025-08-07 11:03:49,962 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:04:02,751 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4124.18262 ± 1173.868
2025-08-07 11:04:02,751 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4663.139, 5223.1094, 5047.666, 1971.4183, 2456.298, 2737.1492, 4459.8286, 5096.5396, 4533.5728, 5053.106]
2025-08-07 11:04:02,751 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:04:02,759 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 56/100 (estimated time remaining: 1 hour, 27 minutes, 29 seconds)
2025-08-07 11:05:47,368 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:05:58,694 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4333.03125 ± 1517.434
2025-08-07 11:05:58,694 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5124.013, 5224.2183, 5138.825, 4950.937, 1323.4528, 4996.584, 5003.783, 1281.2888, 5151.07, 5136.14]
2025-08-07 11:05:58,694 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:05:58,704 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 57/100 (estimated time remaining: 1 hour, 25 minutes, 33 seconds)
2025-08-07 11:07:42,821 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:07:54,189 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4816.78857 ± 931.845
2025-08-07 11:07:54,189 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5186.0264, 2043.588, 5145.901, 5124.969, 5165.487, 5185.625, 4917.2314, 4903.541, 5304.1514, 5191.367]
2025-08-07 11:07:54,189 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:07:54,189 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (4816.79) for latency MM1Queue_a033_s075
2025-08-07 11:07:54,197 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 58/100 (estimated time remaining: 1 hour, 23 minutes, 28 seconds)
2025-08-07 11:09:35,601 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:09:46,872 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4525.74902 ± 1172.105
2025-08-07 11:09:46,872 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5053.1885, 4048.642, 5156.0776, 1303.073, 5126.5474, 3899.7825, 5209.296, 5002.212, 5308.162, 5150.503]
2025-08-07 11:09:46,872 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:09:46,885 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 59/100 (estimated time remaining: 1 hour, 20 minutes, 41 seconds)
2025-08-07 11:11:30,219 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:11:41,515 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4248.47754 ± 997.155
2025-08-07 11:11:41,515 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [2421.6213, 4404.126, 4660.2803, 4529.982, 5157.413, 3457.9226, 4992.741, 2590.3455, 5081.3677, 5188.977]
2025-08-07 11:11:41,515 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:11:41,523 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 60/100 (estimated time remaining: 1 hour, 18 minutes, 42 seconds)
2025-08-07 11:13:23,605 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:13:36,303 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4245.50098 ± 1218.877
2025-08-07 11:13:36,303 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4139.8413, 4865.056, 5230.908, 5235.1206, 4910.146, 4953.3813, 5103.6064, 1573.0563, 4114.898, 2328.9924]
2025-08-07 11:13:36,303 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:13:36,328 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 61/100 (estimated time remaining: 1 hour, 16 minutes, 28 seconds)
2025-08-07 11:15:19,077 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:15:30,344 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4536.64600 ± 1188.710
2025-08-07 11:15:30,345 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3122.4106, 5147.018, 5018.301, 5131.615, 5126.6387, 5148.9917, 5044.2583, 5106.411, 5078.2075, 1442.6097]
2025-08-07 11:15:30,345 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:15:30,353 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 62/100 (estimated time remaining: 1 hour, 14 minutes, 18 seconds)
2025-08-07 11:17:12,578 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:17:23,729 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3540.93091 ± 1429.443
2025-08-07 11:17:23,730 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [2174.0808, 5262.5786, 2421.7415, 3585.668, 4888.211, 1348.9171, 5214.9863, 2166.2363, 3116.6438, 5230.2456]
2025-08-07 11:17:23,730 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:17:23,749 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 63/100 (estimated time remaining: 1 hour, 12 minutes, 8 seconds)
2025-08-07 11:19:06,081 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:19:17,343 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4293.03809 ± 1181.772
2025-08-07 11:19:17,343 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5202.6553, 3655.5005, 4454.1997, 5122.6655, 5079.5103, 3313.7866, 4730.4204, 1276.102, 5110.2954, 4985.244]
2025-08-07 11:19:17,343 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:19:17,360 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 64/100 (estimated time remaining: 1 hour, 10 minutes, 21 seconds)
2025-08-07 11:21:01,483 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:21:12,781 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4776.42334 ± 789.558
2025-08-07 11:21:12,781 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4531.882, 5045.6567, 5198.6904, 5106.232, 2479.5085, 5230.959, 5141.875, 4991.863, 4886.9014, 5150.6646]
2025-08-07 11:21:12,781 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:21:12,793 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 65/100 (estimated time remaining: 1 hour, 8 minutes, 33 seconds)
2025-08-07 11:22:53,286 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:23:04,627 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 5048.80371 ± 194.936
2025-08-07 11:23:04,627 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4995.662, 5235.5723, 5167.7686, 5143.9336, 5100.4526, 4986.55, 4526.6626, 5011.411, 5243.025, 5077.0015]
2025-08-07 11:23:04,627 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:23:04,627 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1226 [INFO]: New best (5048.80) for latency MM1Queue_a033_s075
2025-08-07 11:23:04,635 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 66/100 (estimated time remaining: 1 hour, 6 minutes, 18 seconds)
2025-08-07 11:24:50,327 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:25:01,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4576.82373 ± 1094.565
2025-08-07 11:25:01,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4296.551, 4978.399, 5050.0557, 5161.687, 1393.5951, 5069.8164, 4956.541, 5153.7314, 5144.04, 4563.8257]
2025-08-07 11:25:01,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:25:01,589 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 67/100 (estimated time remaining: 1 hour, 4 minutes, 44 seconds)
2025-08-07 11:26:45,015 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:26:56,206 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4063.60107 ± 1289.087
2025-08-07 11:26:56,206 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3026.4268, 1699.4097, 4916.375, 5015.9775, 4932.151, 4345.1475, 1815.0554, 5000.3936, 4996.555, 4888.5186]
2025-08-07 11:26:56,206 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:26:56,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 68/100 (estimated time remaining: 1 hour, 2 minutes, 58 seconds)
2025-08-07 11:28:39,206 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:28:50,490 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4329.56885 ± 1216.701
2025-08-07 11:28:50,490 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5033.192, 4513.5396, 2700.625, 5221.4644, 4722.1826, 5261.1084, 3932.4387, 1485.8877, 5215.7095, 5209.5376]
2025-08-07 11:28:50,490 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:28:50,500 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 69/100 (estimated time remaining: 1 hour, 1 minute, 8 seconds)
2025-08-07 11:30:32,520 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:30:43,821 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3699.79053 ± 1480.645
2025-08-07 11:30:43,821 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3108.0461, 2431.5857, 4756.718, 5192.7334, 1825.7458, 2536.9895, 1497.1567, 5254.0913, 5222.091, 5172.7515]
2025-08-07 11:30:43,821 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:30:43,846 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 70/100 (estimated time remaining: 59 minutes)
2025-08-07 11:32:24,900 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:32:36,178 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4591.88916 ± 1021.844
2025-08-07 11:32:36,179 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4994.4927, 5145.382, 5054.984, 5063.6523, 3576.525, 5032.3335, 4931.5723, 1838.4087, 5217.0957, 5064.4434]
2025-08-07 11:32:36,179 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:32:36,202 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 71/100 (estimated time remaining: 57 minutes, 9 seconds)
2025-08-07 11:34:17,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:34:29,872 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4455.62354 ± 1149.094
2025-08-07 11:34:29,872 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4936.656, 5060.6133, 2597.0933, 5163.7324, 5069.2476, 5184.969, 4304.023, 1874.4597, 5183.6304, 5181.808]
2025-08-07 11:34:29,873 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:34:29,886 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 72/100 (estimated time remaining: 54 minutes, 56 seconds)
2025-08-07 11:36:10,120 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:36:21,393 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4801.96533 ± 585.569
2025-08-07 11:36:21,393 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4810.0317, 5135.257, 4950.3257, 5165.8257, 5098.376, 3093.6885, 5006.4688, 4947.2407, 5096.7705, 4715.6665]
2025-08-07 11:36:21,393 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:36:21,417 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 73/100 (estimated time remaining: 52 minutes, 44 seconds)
2025-08-07 11:38:04,307 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:38:15,569 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4956.94678 ± 208.010
2025-08-07 11:38:15,569 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4674.1123, 5172.709, 4572.2676, 4722.7153, 4945.092, 5068.238, 5102.603, 5158.1123, 5063.624, 5089.993]
2025-08-07 11:38:15,569 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:38:15,585 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 74/100 (estimated time remaining: 50 minutes, 51 seconds)
2025-08-07 11:39:58,311 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:40:09,563 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4235.90332 ± 1427.812
2025-08-07 11:40:09,563 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1408.4789, 5252.4917, 5131.7617, 1958.2181, 5282.9775, 5145.4385, 5163.1895, 4879.531, 3040.254, 5096.6934]
2025-08-07 11:40:09,563 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:40:09,582 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 75/100 (estimated time remaining: 49 minutes, 1 second)
2025-08-07 11:41:51,126 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:42:03,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4750.28857 ± 993.108
2025-08-07 11:42:03,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1978.8809, 5311.5415, 4432.359, 5271.0547, 4366.851, 5286.6797, 4877.622, 5479.445, 5165.0864, 5333.368]
2025-08-07 11:42:03,715 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:42:03,725 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 76/100 (estimated time remaining: 47 minutes, 17 seconds)
2025-08-07 11:43:46,608 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:43:57,827 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3657.90894 ± 1517.857
2025-08-07 11:43:57,827 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1908.939, 2309.3062, 1339.4156, 5034.608, 5241.56, 5017.73, 4680.162, 4771.4736, 4499.521, 1776.3761]
2025-08-07 11:43:57,827 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:43:57,843 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 77/100 (estimated time remaining: 45 minutes, 26 seconds)
2025-08-07 11:45:39,448 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:45:50,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3315.47974 ± 1537.822
2025-08-07 11:45:50,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4933.838, 1658.9658, 3405.4663, 1818.6174, 2900.6582, 1670.0012, 5181.6914, 5023.3525, 5118.9746, 1443.2323]
2025-08-07 11:45:50,564 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:45:50,593 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 78/100 (estimated time remaining: 43 minutes, 38 seconds)
2025-08-07 11:47:31,891 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:47:44,506 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4943.53418 ± 414.951
2025-08-07 11:47:44,506 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5085.1484, 3829.5588, 5232.12, 5282.5776, 4736.468, 5210.777, 5196.7285, 4717.8853, 5083.5825, 5060.4946]
2025-08-07 11:47:44,506 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:47:44,529 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 79/100 (estimated time remaining: 41 minutes, 43 seconds)
2025-08-07 11:49:27,886 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:49:39,117 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4314.86377 ± 1403.833
2025-08-07 11:49:39,117 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3773.512, 5155.152, 5149.8535, 5217.7876, 5225.1587, 5076.3564, 5125.333, 1921.5834, 1362.0928, 5141.81]
2025-08-07 11:49:39,117 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:49:39,150 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 80/100 (estimated time remaining: 39 minutes, 52 seconds)
2025-08-07 11:51:22,133 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:51:33,266 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4641.76123 ± 1020.985
2025-08-07 11:51:33,266 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4725.7656, 5228.4473, 5275.254, 1803.8989, 5267.029, 5277.2197, 4864.71, 4763.487, 3990.9998, 5220.802]
2025-08-07 11:51:33,266 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:51:33,278 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 81/100 (estimated time remaining: 37 minutes, 58 seconds)
2025-08-07 11:53:12,965 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:53:24,141 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4886.15869 ± 312.134
2025-08-07 11:53:24,141 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5248.633, 4653.512, 4741.233, 5163.9893, 5160.402, 5013.662, 4534.193, 5171.725, 4259.378, 4914.8545]
2025-08-07 11:53:24,141 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:53:24,153 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 82/100 (estimated time remaining: 35 minutes, 51 seconds)
2025-08-07 11:55:08,326 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:55:20,928 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3636.04614 ± 1383.879
2025-08-07 11:55:20,929 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1573.7441, 5100.5664, 5159.827, 3397.4583, 1676.2731, 2453.3645, 3634.7703, 5196.683, 3011.9036, 5155.8706]
2025-08-07 11:55:20,929 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:55:20,940 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 83/100 (estimated time remaining: 34 minutes, 13 seconds)
2025-08-07 11:57:02,531 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:57:13,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4465.04004 ± 1079.169
2025-08-07 11:57:13,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5108.979, 1932.1621, 2802.2256, 5097.8003, 4617.4873, 5142.3833, 4951.9937, 4840.0386, 5216.0337, 4941.2954]
2025-08-07 11:57:13,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:57:13,813 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 84/100 (estimated time remaining: 32 minutes, 15 seconds)
2025-08-07 11:58:54,287 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 11:59:05,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4597.06152 ± 1011.377
2025-08-07 11:59:05,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5106.7847, 1617.0724, 5137.916, 4695.7725, 4647.38, 5022.7573, 5092.17, 4614.357, 5053.737, 4982.6646]
2025-08-07 11:59:05,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 11:59:05,512 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 85/100 (estimated time remaining: 30 minutes, 12 seconds)
2025-08-07 12:00:48,970 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:01:00,123 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4730.04736 ± 495.783
2025-08-07 12:01:00,123 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5272.5005, 5145.5005, 5155.418, 4819.992, 5240.931, 4791.5605, 4672.937, 4049.7178, 3747.657, 4404.262]
2025-08-07 12:01:00,123 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:01:00,138 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 86/100 (estimated time remaining: 28 minutes, 20 seconds)
2025-08-07 12:02:42,906 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:02:54,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4854.16211 ± 334.981
2025-08-07 12:02:54,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5039.1147, 4566.4873, 5130.3696, 4639.6997, 5096.4814, 5117.562, 5200.097, 4924.734, 4065.8115, 4761.261]
2025-08-07 12:02:54,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:02:54,134 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 87/100 (estimated time remaining: 26 minutes, 35 seconds)
2025-08-07 12:04:37,072 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:04:48,346 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4378.66992 ± 924.402
2025-08-07 12:04:48,346 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5088.517, 4525.0186, 5105.2856, 3547.3416, 5208.11, 5061.3022, 5179.9595, 2789.817, 4474.658, 2806.6904]
2025-08-07 12:04:48,346 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:04:48,360 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 88/100 (estimated time remaining: 24 minutes, 35 seconds)
2025-08-07 12:06:30,965 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:06:42,091 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4087.85400 ± 1294.931
2025-08-07 12:06:42,091 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5067.3687, 4985.3125, 1563.8845, 3141.1506, 4684.4395, 5003.0205, 5057.8022, 1945.868, 5052.737, 4376.9526]
2025-08-07 12:06:42,091 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:06:42,102 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 89/100 (estimated time remaining: 22 minutes, 43 seconds)
2025-08-07 12:08:25,599 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:08:36,837 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4178.71973 ± 1289.834
2025-08-07 12:08:36,837 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [3641.0732, 5280.9272, 1339.7924, 5150.86, 5227.6553, 2542.8745, 4410.936, 5209.4473, 5212.785, 3770.8513]
2025-08-07 12:08:36,837 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:08:36,849 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 90/100 (estimated time remaining: 20 minutes, 56 seconds)
2025-08-07 12:10:17,661 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:10:30,274 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4400.91846 ± 1192.673
2025-08-07 12:10:30,274 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5092.71, 1642.27, 4985.9727, 4311.365, 5114.556, 2549.0469, 4988.801, 5085.0806, 5120.763, 5118.617]
2025-08-07 12:10:30,274 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:10:30,290 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 91/100 (estimated time remaining: 19 minutes)
2025-08-07 12:12:11,917 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:12:23,113 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3916.07031 ± 1291.929
2025-08-07 12:12:23,114 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1857.8948, 2185.55, 4861.0024, 4068.725, 2096.7588, 3855.543, 5069.6074, 5062.841, 5108.98, 4993.8]
2025-08-07 12:12:23,114 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:12:23,154 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 92/100 (estimated time remaining: 17 minutes, 4 seconds)
2025-08-07 12:14:03,903 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:14:16,490 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 3728.62939 ± 1374.144
2025-08-07 12:14:16,491 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5141.075, 4085.6208, 5237.658, 3006.6104, 3426.8005, 5162.3203, 1149.613, 2626.9004, 5130.203, 2319.489]
2025-08-07 12:14:16,491 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:14:16,528 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 93/100 (estimated time remaining: 15 minutes, 9 seconds)
2025-08-07 12:15:57,172 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:16:09,757 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4992.95312 ± 324.369
2025-08-07 12:16:09,757 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4992.8345, 5243.08, 5112.122, 5087.8535, 5249.1055, 4503.8394, 4250.1343, 5041.2827, 5230.2153, 5219.061]
2025-08-07 12:16:09,757 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:16:09,790 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 94/100 (estimated time remaining: 13 minutes, 14 seconds)
2025-08-07 12:17:50,270 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:18:02,855 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4247.85693 ± 1251.911
2025-08-07 12:18:02,855 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [1576.8524, 4908.3525, 5123.558, 3928.647, 5291.3745, 5241.196, 2334.69, 5224.359, 3934.0935, 4915.4453]
2025-08-07 12:18:02,855 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:18:02,899 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 95/100 (estimated time remaining: 11 minutes, 19 seconds)
2025-08-07 12:19:44,838 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:19:55,967 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4522.32617 ± 975.386
2025-08-07 12:19:55,967 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5009.1245, 5221.441, 5147.1704, 5223.6865, 3862.2615, 5117.986, 4993.9487, 5026.223, 2186.3665, 3435.0564]
2025-08-07 12:19:55,967 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:19:55,981 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 96/100 (estimated time remaining: 9 minutes, 25 seconds)
2025-08-07 12:21:36,360 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:21:47,432 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4801.43896 ± 701.877
2025-08-07 12:21:47,432 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5186.1514, 5185.6753, 2876.6462, 5194.387, 5225.441, 5275.226, 4752.628, 4744.027, 5212.0107, 4362.1997]
2025-08-07 12:21:47,432 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:21:47,459 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 97/100 (estimated time remaining: 7 minutes, 31 seconds)
2025-08-07 12:23:30,913 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:23:43,378 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4366.21191 ± 1472.701
2025-08-07 12:23:43,378 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5299.1235, 1318.9669, 5081.7715, 5243.6294, 4773.0786, 5286.015, 5214.4775, 1577.0388, 5152.1284, 4715.888]
2025-08-07 12:23:43,378 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:23:43,389 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 98/100 (estimated time remaining: 5 minutes, 40 seconds)
2025-08-07 12:25:26,735 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:25:39,219 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4655.44434 ± 1048.621
2025-08-07 12:25:39,219 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5013.169, 5062.402, 5154.2217, 5088.7173, 4797.0024, 5161.5713, 4653.472, 5147.4077, 1545.4891, 4930.9946]
2025-08-07 12:25:39,219 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:25:39,251 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 99/100 (estimated time remaining: 3 minutes, 47 seconds)
2025-08-07 12:27:18,549 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:27:29,626 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4820.52637 ± 749.139
2025-08-07 12:27:29,626 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [5158.7827, 4240.3423, 5238.5454, 4956.91, 5176.3296, 5210.7803, 2739.0085, 5093.9575, 5209.875, 5180.736]
2025-08-07 12:27:29,627 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:27:29,666 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1199 [INFO]: Iteration 100/100 (estimated time remaining: 1 minute, 53 seconds)
2025-08-07 12:29:07,841 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-08-07 12:29:18,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1221 [DEBUG]: Total Reward: 4191.67676 ± 1147.567
2025-08-07 12:29:18,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1222 [DEBUG]: All rewards: [4990.3613, 4254.3438, 3301.1675, 4995.914, 4989.7896, 5058.311, 2057.1545, 5162.4663, 2252.3582, 4854.8984]
2025-08-07 12:29:18,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-08-07 12:29:18,814 latency_env.delayed_mdp:training_loop(baseline-bpql-noiseperc5-halfcheetah):1251 [DEBUG]: Training session finished
