2025-05-10 00:19:21,978 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1108 [DEBUG]: logdir: _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16
2025-05-10 00:19:21,979 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1109 [DEBUG]: trainer_prefix: benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16
2025-05-10 00:19:21,979 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1110 [DEBUG]: args.trainer_eval_latencies: {'MM1Queue_a033_s075': <latency_env.delayed_mdp.MM1QueueDelay object at 0x7f1662dc8c70>}
2025-05-10 00:19:21,979 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1111 [DEBUG]: using device: cpu
2025-05-10 00:19:21,983 baseline-bpql-noisy-walker2d:77 [WARNING]: args.assumed_delay != args.horizon: 16 != 24
2025-05-10 00:19:21,983 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1133 [INFO]: Creating new trainer
2025-05-10 00:19:21,989 baseline-bpql-noisy-walker2d:113 [DEBUG]: pi network:
NNGaussianPolicy(
  (common_head): Sequential(
    (0): Flatten(start_dim=1, end_dim=-1)
    (1): Linear(in_features=113, out_features=256, bias=True)
    (2): ReLU()
    (3): Linear(in_features=256, out_features=256, bias=True)
    (4): ReLU()
  )
  (mu_head): Sequential(
    (0): Linear(in_features=256, out_features=6, bias=True)
    (1): Unflatten(dim=1, unflattened_size=(6,))
  )
  (log_std_head): Sequential(
    (0): Linear(in_features=256, out_features=6, bias=True)
    (1): Unflatten(dim=1, unflattened_size=(6,))
  )
  (tanh_refit): NNTanhRefit(scale: tensor([[2., 2., 2., 2., 2., 2.]]), shift: tensor([[-1., -1., -1., -1., -1., -1.]]))
)
2025-05-10 00:19:21,989 baseline-bpql-noisy-walker2d:114 [DEBUG]: q network:
NNLayerConcat2(
  dim: -1
  (next): Sequential(
    (0): Linear(in_features=23, out_features=256, bias=True)
    (1): ReLU()
    (2): Linear(in_features=256, out_features=256, bias=True)
    (3): ReLU()
    (4): Linear(in_features=256, out_features=1, bias=True)
    (5): NNLayerSqueeze(dim: -1)
  )
  (init_left): Flatten(start_dim=1, end_dim=-1)
  (init_right): Flatten(start_dim=1, end_dim=-1)
)
2025-05-10 00:19:22,207 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1194 [DEBUG]: Starting training session...
2025-05-10 00:19:22,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 1/100
2025-05-10 00:21:46,505 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:21:47,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 11.18022 ± 4.358
2025-05-10 00:21:47,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [11.504534, 10.027304, 17.879763, 4.8500085, 10.577365, 12.626117, 9.872222, 4.033113, 12.409147, 18.022606]
2025-05-10 00:21:47,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [61.0, 72.0, 64.0, 61.0, 62.0, 61.0, 65.0, 52.0, 65.0, 64.0]
2025-05-10 00:21:47,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (11.18) for latency MM1Queue_a033_s075
2025-05-10 00:21:47,502 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 00:21:47,506 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 00:21:47,511 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 2/100 (estimated time remaining: 3 hours, 59 minutes, 45 seconds)
2025-05-10 00:24:28,524 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:24:30,583 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 96.57295 ± 71.761
2025-05-10 00:24:30,583 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [92.760124, 37.5067, 68.83956, 148.8709, 25.50464, 68.02595, 280.09692, 76.10665, 129.87254, 38.14559]
2025-05-10 00:24:30,583 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [162.0, 214.0, 133.0, 124.0, 33.0, 169.0, 196.0, 87.0, 105.0, 63.0]
2025-05-10 00:24:30,584 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (96.57) for latency MM1Queue_a033_s075
2025-05-10 00:24:30,584 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 00:24:30,588 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 00:24:30,593 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 3/100 (estimated time remaining: 4 hours, 11 minutes, 50 seconds)
2025-05-10 00:27:07,043 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:27:09,201 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 71.09697 ± 59.596
2025-05-10 00:27:09,201 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [150.76216, 52.04241, 33.281853, 57.635933, 77.292595, 155.65427, -29.05702, 151.65346, 19.716938, 41.987144]
2025-05-10 00:27:09,201 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [126.0, 188.0, 221.0, 63.0, 267.0, 134.0, 142.0, 102.0, 29.0, 58.0]
2025-05-10 00:27:09,202 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 4/100 (estimated time remaining: 4 hours, 11 minutes, 39 seconds)
2025-05-10 00:29:46,082 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:29:48,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 86.62782 ± 65.814
2025-05-10 00:29:48,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [190.14194, -7.5034804, 23.166874, 167.15022, 18.085835, 134.53879, 49.03927, 42.60549, 126.83751, 122.215706]
2025-05-10 00:29:48,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [136.0, 205.0, 27.0, 163.0, 25.0, 578.0, 58.0, 49.0, 105.0, 151.0]
2025-05-10 00:29:48,579 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 5/100 (estimated time remaining: 4 hours, 10 minutes, 32 seconds)
2025-05-10 00:32:27,142 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:32:28,748 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 82.38338 ± 39.402
2025-05-10 00:32:28,748 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [37.04171, 65.71801, 145.09628, 90.178764, 88.366325, 63.08607, 39.46493, 120.9652, 139.00235, 34.91421]
2025-05-10 00:32:28,748 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [54.0, 174.0, 114.0, 93.0, 84.0, 68.0, 77.0, 204.0, 100.0, 44.0]
2025-05-10 00:32:28,750 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 6/100 (estimated time remaining: 4 hours, 9 minutes, 4 seconds)
2025-05-10 00:35:06,282 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:35:08,043 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 58.59084 ± 44.785
2025-05-10 00:35:08,043 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3.240676, 34.91532, 46.73342, 40.384014, 173.51509, 42.854748, 28.86078, 54.61431, 62.2957, 98.4944]
2025-05-10 00:35:08,043 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [186.0, 70.0, 66.0, 52.0, 132.0, 48.0, 77.0, 267.0, 108.0, 118.0]
2025-05-10 00:35:08,045 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 7/100 (estimated time remaining: 4 hours, 10 minutes, 50 seconds)
2025-05-10 00:37:47,771 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:37:49,831 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 93.60353 ± 43.527
2025-05-10 00:37:49,832 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [73.76439, 18.919107, 117.45418, 47.05896, 61.51397, 126.10616, 143.13017, 75.03321, 107.62982, 165.42534]
2025-05-10 00:37:49,832 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [116.0, 25.0, 123.0, 89.0, 232.0, 236.0, 108.0, 138.0, 110.0, 127.0]
2025-05-10 00:37:49,834 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 8/100 (estimated time remaining: 4 hours, 7 minutes, 45 seconds)
2025-05-10 00:40:27,904 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:40:29,763 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 54.51220 ± 49.477
2025-05-10 00:40:29,763 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [50.334213, 1.1276205, 39.502533, 40.040768, 187.95836, 26.707453, 77.454605, 57.140312, 55.749256, 9.106788]
2025-05-10 00:40:29,763 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [67.0, 178.0, 83.0, 99.0, 143.0, 127.0, 179.0, 119.0, 93.0, 119.0]
2025-05-10 00:40:29,765 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 9/100 (estimated time remaining: 4 hours, 5 minutes, 30 seconds)
2025-05-10 00:43:08,694 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:43:10,864 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 159.73468 ± 67.197
2025-05-10 00:43:10,864 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [192.34999, 186.55487, 126.83205, 242.52504, 137.33678, 180.17276, 271.8766, 63.762608, 148.45804, 47.47817]
2025-05-10 00:43:10,864 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [151.0, 143.0, 101.0, 194.0, 104.0, 129.0, 196.0, 65.0, 165.0, 160.0]
2025-05-10 00:43:10,864 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (159.73) for latency MM1Queue_a033_s075
2025-05-10 00:43:10,864 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 00:43:10,868 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 00:43:10,875 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 10/100 (estimated time remaining: 4 hours, 3 minutes, 21 seconds)
2025-05-10 00:45:50,654 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:45:52,186 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 71.72014 ± 66.501
2025-05-10 00:45:52,186 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [25.03642, 54.331154, 60.280518, 207.38058, 198.07814, 48.63608, 31.886755, 34.25593, 30.784101, 26.531664]
2025-05-10 00:45:52,186 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [135.0, 89.0, 71.0, 154.0, 142.0, 70.0, 68.0, 102.0, 50.0, 119.0]
2025-05-10 00:45:52,188 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 11/100 (estimated time remaining: 4 hours, 1 minute, 1 second)
2025-05-10 00:48:29,764 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:48:31,829 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 124.43242 ± 75.444
2025-05-10 00:48:31,829 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [51.901474, 45.64247, 86.81412, 177.62395, 15.460657, 133.54984, 268.48734, 128.48598, 216.40706, 119.95127]
2025-05-10 00:48:31,829 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [203.0, 119.0, 133.0, 139.0, 24.0, 150.0, 172.0, 124.0, 130.0, 139.0]
2025-05-10 00:48:31,832 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 12/100 (estimated time remaining: 3 hours, 58 minutes, 27 seconds)
2025-05-10 00:51:10,480 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:51:12,246 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 132.11571 ± 71.792
2025-05-10 00:51:12,246 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [59.082455, 116.786415, 84.74819, 219.47298, 126.15027, 151.43092, 70.89714, 144.71107, 55.018456, 292.85922]
2025-05-10 00:51:12,246 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [64.0, 90.0, 110.0, 161.0, 96.0, 108.0, 158.0, 95.0, 87.0, 175.0]
2025-05-10 00:51:12,249 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 13/100 (estimated time remaining: 3 hours, 55 minutes, 22 seconds)
2025-05-10 00:53:52,253 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:53:53,971 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 147.21779 ± 58.361
2025-05-10 00:53:53,971 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [176.82623, 109.076294, 183.99004, 223.72607, 232.70218, 53.967487, 144.14693, 154.306, 137.7054, 55.731228]
2025-05-10 00:53:53,971 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [125.0, 92.0, 142.0, 148.0, 126.0, 57.0, 143.0, 116.0, 112.0, 59.0]
2025-05-10 00:53:53,974 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 14/100 (estimated time remaining: 3 hours, 53 minutes, 13 seconds)
2025-05-10 00:56:31,915 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:56:33,534 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 146.27693 ± 77.965
2025-05-10 00:56:33,534 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [122.649124, 109.6758, 56.25726, 104.97644, 122.4091, 230.89233, 53.088192, 119.924034, 253.63692, 289.26]
2025-05-10 00:56:33,534 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [98.0, 91.0, 53.0, 126.0, 102.0, 137.0, 56.0, 78.0, 146.0, 160.0]
2025-05-10 00:56:33,537 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 15/100 (estimated time remaining: 3 hours, 50 minutes, 5 seconds)
2025-05-10 00:59:14,318 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 00:59:16,187 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 156.52194 ± 68.585
2025-05-10 00:59:16,187 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [197.7257, 304.72943, 97.86434, 123.25145, 147.11366, 210.31529, 202.57774, 71.76153, 80.25647, 129.62395]
2025-05-10 00:59:16,187 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [110.0, 154.0, 94.0, 101.0, 96.0, 217.0, 148.0, 62.0, 111.0, 83.0]
2025-05-10 00:59:16,190 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 16/100 (estimated time remaining: 3 hours, 47 minutes, 48 seconds)
2025-05-10 01:01:54,144 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:01:56,822 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 272.88837 ± 218.225
2025-05-10 01:01:56,823 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [263.25513, 297.91132, 256.50687, 230.6136, 29.425087, 825.6423, 202.85307, 119.72381, 52.284767, 450.66788]
2025-05-10 01:01:56,823 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [231.0, 137.0, 146.0, 121.0, 32.0, 464.0, 202.0, 72.0, 48.0, 225.0]
2025-05-10 01:01:56,823 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (272.89) for latency MM1Queue_a033_s075
2025-05-10 01:01:56,823 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 01:01:56,827 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 01:01:56,834 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 17/100 (estimated time remaining: 3 hours, 45 minutes, 24 seconds)
2025-05-10 01:04:37,202 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:04:38,654 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 157.65915 ± 105.857
2025-05-10 01:04:38,654 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [12.830464, 256.25906, 229.5708, 93.43793, 295.57135, 226.62102, 24.913794, 58.64629, 92.28106, 286.45984]
2025-05-10 01:04:38,654 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [22.0, 124.0, 114.0, 84.0, 129.0, 115.0, 29.0, 51.0, 112.0, 152.0]
2025-05-10 01:04:38,658 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 18/100 (estimated time remaining: 3 hours, 43 minutes, 6 seconds)
2025-05-10 01:07:18,221 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:07:20,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 227.84282 ± 136.336
2025-05-10 01:07:20,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [228.72807, 127.314964, 27.699892, 500.10193, 233.38956, 18.56704, 289.08188, 329.87894, 287.20526, 236.46063]
2025-05-10 01:07:20,220 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [147.0, 101.0, 33.0, 264.0, 121.0, 31.0, 135.0, 160.0, 147.0, 128.0]
2025-05-10 01:07:20,224 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 19/100 (estimated time remaining: 3 hours, 40 minutes, 22 seconds)
2025-05-10 01:10:02,381 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:10:04,624 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 177.06204 ± 83.721
2025-05-10 01:10:04,624 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [334.80475, 270.59747, 179.45447, 207.42404, 16.409286, 87.238686, 181.89812, 151.13692, 191.8365, 149.8201]
2025-05-10 01:10:04,624 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [168.0, 126.0, 157.0, 170.0, 24.0, 214.0, 238.0, 116.0, 120.0, 118.0]
2025-05-10 01:10:04,628 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 20/100 (estimated time remaining: 3 hours, 38 minutes, 59 seconds)
2025-05-10 01:12:43,825 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:12:46,686 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 257.91473 ± 102.508
2025-05-10 01:12:46,687 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [330.9926, 226.5086, 280.76996, 354.60712, 214.76277, 167.9669, 201.36507, 479.2976, 225.09196, 97.78463]
2025-05-10 01:12:46,687 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [167.0, 129.0, 260.0, 216.0, 201.0, 127.0, 133.0, 206.0, 274.0, 109.0]
2025-05-10 01:12:46,691 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 21/100 (estimated time remaining: 3 hours, 36 minutes, 8 seconds)
2025-05-10 01:15:26,652 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:15:29,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 219.61833 ± 115.206
2025-05-10 01:15:29,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [177.57681, 251.00104, 212.13498, 395.60898, 335.06097, 185.45747, 289.67377, 294.83328, 28.590456, 26.245674]
2025-05-10 01:15:29,170 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [156.0, 127.0, 132.0, 416.0, 278.0, 112.0, 139.0, 174.0, 31.0, 31.0]
2025-05-10 01:15:29,174 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 22/100 (estimated time remaining: 3 hours, 33 minutes, 54 seconds)
2025-05-10 01:18:10,561 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:18:14,728 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 381.56842 ± 320.094
2025-05-10 01:18:14,728 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [26.930286, 259.47595, 448.70282, 286.5107, 1064.6069, 30.062147, 302.56882, 875.63794, 278.88818, 242.30049]
2025-05-10 01:18:14,728 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [30.0, 287.0, 396.0, 261.0, 670.0, 32.0, 155.0, 451.0, 157.0, 129.0]
2025-05-10 01:18:14,728 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (381.57) for latency MM1Queue_a033_s075
2025-05-10 01:18:14,729 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 01:18:14,732 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 01:18:14,741 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 23/100 (estimated time remaining: 3 hours, 32 minutes, 10 seconds)
2025-05-10 01:20:50,248 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:20:52,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 263.25717 ± 155.737
2025-05-10 01:20:52,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [269.43985, 264.7281, 30.472778, 373.43164, 379.47696, 23.995338, 359.81094, 159.39104, 552.0752, 219.74992]
2025-05-10 01:20:52,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [190.0, 129.0, 33.0, 180.0, 200.0, 29.0, 160.0, 107.0, 276.0, 118.0]
2025-05-10 01:20:52,478 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 24/100 (estimated time remaining: 3 hours, 28 minutes, 28 seconds)
2025-05-10 01:23:31,022 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:23:34,448 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 362.96295 ± 153.958
2025-05-10 01:23:34,448 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [375.84473, 634.2622, 228.95377, 595.557, 320.99542, 210.53915, 210.63164, 254.69481, 518.8104, 279.34067]
2025-05-10 01:23:34,448 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [195.0, 252.0, 168.0, 332.0, 195.0, 142.0, 177.0, 335.0, 239.0, 152.0]
2025-05-10 01:23:34,453 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 25/100 (estimated time remaining: 3 hours, 25 minutes, 9 seconds)
2025-05-10 01:26:13,978 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:26:17,472 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 455.12094 ± 244.288
2025-05-10 01:26:17,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [734.1567, 301.09995, 263.15417, 223.18217, 130.56691, 666.36237, 717.06635, 223.65833, 818.5093, 473.453]
2025-05-10 01:26:17,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [323.0, 150.0, 163.0, 123.0, 97.0, 304.0, 296.0, 143.0, 354.0, 255.0]
2025-05-10 01:26:17,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (455.12) for latency MM1Queue_a033_s075
2025-05-10 01:26:17,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 01:26:17,477 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 01:26:17,486 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 26/100 (estimated time remaining: 3 hours, 22 minutes, 41 seconds)
2025-05-10 01:28:54,971 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:28:57,043 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 238.39836 ± 110.014
2025-05-10 01:28:57,044 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [296.89792, 285.9795, 285.51883, 50.470364, 295.97504, 251.44508, 8.720744, 379.1642, 251.67206, 278.13992]
2025-05-10 01:28:57,044 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [138.0, 139.0, 126.0, 100.0, 155.0, 154.0, 20.0, 223.0, 160.0, 127.0]
2025-05-10 01:28:57,049 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 27/100 (estimated time remaining: 3 hours, 19 minutes, 16 seconds)
2025-05-10 01:31:33,594 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:31:36,446 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 348.43686 ± 125.136
2025-05-10 01:31:36,446 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [379.94672, 367.94565, 148.16731, 409.5669, 146.96277, 541.80756, 364.50952, 484.01788, 239.44481, 401.99948]
2025-05-10 01:31:36,446 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [187.0, 188.0, 118.0, 236.0, 143.0, 229.0, 183.0, 205.0, 173.0, 161.0]
2025-05-10 01:31:36,451 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 28/100 (estimated time remaining: 3 hours, 15 minutes, 4 seconds)
2025-05-10 01:34:15,777 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:34:18,142 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 306.37555 ± 123.263
2025-05-10 01:34:18,142 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [351.8039, 414.51752, 15.587496, 324.17303, 407.8465, 450.0764, 177.71474, 299.92844, 364.53595, 257.57156]
2025-05-10 01:34:18,142 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [170.0, 226.0, 23.0, 167.0, 209.0, 205.0, 102.0, 137.0, 157.0, 120.0]
2025-05-10 01:34:18,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 29/100 (estimated time remaining: 3 hours, 13 minutes, 21 seconds)
2025-05-10 01:36:54,691 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:36:57,585 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 344.45740 ± 146.534
2025-05-10 01:36:57,585 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [378.04797, 737.4034, 291.29794, 302.17932, 266.5387, 229.11221, 400.50485, 375.38974, 286.03845, 178.06128]
2025-05-10 01:36:57,585 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [209.0, 438.0, 158.0, 140.0, 139.0, 116.0, 189.0, 200.0, 137.0, 100.0]
2025-05-10 01:36:57,591 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 30/100 (estimated time remaining: 3 hours, 10 minutes, 4 seconds)
2025-05-10 01:39:35,807 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:39:38,425 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 334.08966 ± 126.028
2025-05-10 01:39:38,425 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [385.71652, 313.4609, 456.87448, 350.75436, 435.95218, 27.71916, 180.09174, 404.00757, 376.36182, 409.95792]
2025-05-10 01:39:38,425 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [201.0, 174.0, 196.0, 166.0, 206.0, 30.0, 126.0, 184.0, 184.0, 211.0]
2025-05-10 01:39:38,431 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 31/100 (estimated time remaining: 3 hours, 6 minutes, 53 seconds)
2025-05-10 01:42:16,900 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:42:20,927 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 510.87817 ± 186.270
2025-05-10 01:42:20,928 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [285.81027, 638.6938, 319.5786, 852.3874, 664.5777, 656.0368, 321.66006, 360.41342, 613.2972, 396.3261]
2025-05-10 01:42:20,928 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [160.0, 290.0, 171.0, 320.0, 331.0, 329.0, 167.0, 172.0, 392.0, 209.0]
2025-05-10 01:42:20,928 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (510.88) for latency MM1Queue_a033_s075
2025-05-10 01:42:20,928 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 01:42:20,932 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 01:42:20,942 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 32/100 (estimated time remaining: 3 hours, 4 minutes, 53 seconds)
2025-05-10 01:45:00,013 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:45:02,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 328.07812 ± 156.850
2025-05-10 01:45:02,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [358.3643, 287.41452, 345.4754, 293.97623, 687.6427, 31.305737, 283.6079, 373.64444, 409.40085, 209.94916]
2025-05-10 01:45:02,473 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [173.0, 138.0, 174.0, 148.0, 301.0, 32.0, 138.0, 154.0, 211.0, 118.0]
2025-05-10 01:45:02,479 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 33/100 (estimated time remaining: 3 hours, 2 minutes, 41 seconds)
2025-05-10 01:47:41,697 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:47:45,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 405.09378 ± 116.600
2025-05-10 01:47:45,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [357.46988, 574.3524, 410.24228, 624.5598, 292.9665, 346.02365, 314.13745, 322.61874, 517.6601, 290.90704]
2025-05-10 01:47:45,317 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [167.0, 271.0, 358.0, 369.0, 145.0, 302.0, 149.0, 166.0, 227.0, 146.0]
2025-05-10 01:47:45,323 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 34/100 (estimated time remaining: 3 hours, 16 seconds)
2025-05-10 01:50:22,283 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:50:25,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 393.26187 ± 195.606
2025-05-10 01:50:25,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [236.94055, 353.62854, 772.1194, 386.19937, 375.7467, 482.06158, 25.436956, 336.4104, 641.0122, 323.06305]
2025-05-10 01:50:25,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [116.0, 172.0, 341.0, 208.0, 177.0, 225.0, 28.0, 175.0, 240.0, 146.0]
2025-05-10 01:50:25,168 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 35/100 (estimated time remaining: 2 hours, 57 minutes, 40 seconds)
2025-05-10 01:53:03,680 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:53:06,578 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 404.33575 ± 170.868
2025-05-10 01:53:06,578 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [376.39163, 610.26447, 335.3894, 471.8644, 682.9037, 446.45935, 278.89227, 356.61465, 453.8924, 30.6852]
2025-05-10 01:53:06,578 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [201.0, 221.0, 179.0, 194.0, 244.0, 210.0, 163.0, 206.0, 200.0, 35.0]
2025-05-10 01:53:06,584 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 36/100 (estimated time remaining: 2 hours, 55 minutes, 5 seconds)
2025-05-10 01:55:44,489 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:55:48,302 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 508.53360 ± 189.320
2025-05-10 01:55:48,302 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [247.08742, 734.6298, 636.9136, 819.2775, 330.18863, 714.57916, 407.65692, 341.03888, 434.44675, 419.5171]
2025-05-10 01:55:48,302 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [125.0, 359.0, 310.0, 369.0, 150.0, 283.0, 209.0, 178.0, 203.0, 213.0]
2025-05-10 01:55:48,309 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 37/100 (estimated time remaining: 2 hours, 52 minutes, 14 seconds)
2025-05-10 01:58:27,851 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 01:58:31,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 481.19214 ± 51.506
2025-05-10 01:58:31,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [479.9422, 491.32022, 394.0545, 472.58865, 608.52637, 466.2466, 498.1531, 482.42133, 436.14374, 482.5251]
2025-05-10 01:58:31,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [214.0, 264.0, 184.0, 206.0, 258.0, 217.0, 256.0, 208.0, 210.0, 266.0]
2025-05-10 01:58:31,456 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 38/100 (estimated time remaining: 2 hours, 49 minutes, 53 seconds)
2025-05-10 02:01:11,922 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:01:14,533 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 327.47800 ± 141.356
2025-05-10 02:01:14,533 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [332.77515, 406.39322, 432.77448, 466.52808, 551.00586, 320.81924, 76.96825, 268.19678, 307.37338, 111.945915]
2025-05-10 02:01:14,533 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [173.0, 186.0, 177.0, 205.0, 259.0, 183.0, 102.0, 133.0, 154.0, 114.0]
2025-05-10 02:01:14,540 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 39/100 (estimated time remaining: 2 hours, 47 minutes, 14 seconds)
2025-05-10 02:03:52,437 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:03:55,951 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 426.75278 ± 177.191
2025-05-10 02:03:55,951 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [561.9321, 433.4275, 374.61093, 224.11346, 63.684425, 542.89825, 716.76074, 372.45673, 565.6393, 412.00455]
2025-05-10 02:03:55,951 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [309.0, 266.0, 187.0, 122.0, 100.0, 250.0, 340.0, 204.0, 260.0, 174.0]
2025-05-10 02:03:55,958 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 40/100 (estimated time remaining: 2 hours, 44 minutes, 51 seconds)
2025-05-10 02:06:35,918 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:06:39,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 422.77997 ± 194.790
2025-05-10 02:06:39,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [418.82266, 716.21185, 25.215393, 397.57373, 504.4801, 405.0749, 274.19543, 267.54596, 532.37006, 686.30945]
2025-05-10 02:06:39,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [198.0, 293.0, 29.0, 261.0, 251.0, 168.0, 128.0, 142.0, 211.0, 398.0]
2025-05-10 02:06:39,215 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 41/100 (estimated time remaining: 2 hours, 42 minutes, 31 seconds)
2025-05-10 02:09:19,028 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:09:22,530 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 492.83124 ± 338.649
2025-05-10 02:09:22,530 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [337.70905, 344.8088, 396.60947, 274.38745, 457.303, 385.00903, 538.0306, 277.7179, 434.33182, 1482.4058]
2025-05-10 02:09:22,530 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [147.0, 136.0, 176.0, 142.0, 183.0, 236.0, 229.0, 150.0, 184.0, 603.0]
2025-05-10 02:09:22,537 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 42/100 (estimated time remaining: 2 hours, 40 minutes, 7 seconds)
2025-05-10 02:11:58,449 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:12:02,867 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 633.19855 ± 278.209
2025-05-10 02:12:02,867 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [377.6283, 1142.6324, 749.0714, 662.4809, 358.29907, 892.1159, 256.95023, 717.51666, 851.7006, 323.5898]
2025-05-10 02:12:02,867 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [194.0, 455.0, 327.0, 278.0, 160.0, 381.0, 146.0, 329.0, 356.0, 135.0]
2025-05-10 02:12:02,867 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (633.20) for latency MM1Queue_a033_s075
2025-05-10 02:12:02,867 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 02:12:02,871 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 02:12:02,883 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 43/100 (estimated time remaining: 2 hours, 36 minutes, 52 seconds)
2025-05-10 02:14:41,584 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:14:44,769 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 417.86176 ± 191.285
2025-05-10 02:14:44,769 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [232.57265, 593.7365, 559.00275, 619.27014, 527.79407, 60.394108, 629.6656, 275.89554, 447.81262, 232.47385]
2025-05-10 02:14:44,770 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [110.0, 328.0, 225.0, 258.0, 206.0, 102.0, 290.0, 142.0, 258.0, 119.0]
2025-05-10 02:14:44,777 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 44/100 (estimated time remaining: 2 hours, 33 minutes, 56 seconds)
2025-05-10 02:17:23,931 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:17:28,103 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 606.38519 ± 253.854
2025-05-10 02:17:28,103 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [691.85925, 326.1331, 1022.87805, 508.87814, 829.8721, 606.2546, 962.25464, 221.36389, 453.18726, 441.1716]
2025-05-10 02:17:28,103 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [238.0, 132.0, 438.0, 205.0, 356.0, 263.0, 424.0, 114.0, 224.0, 230.0]
2025-05-10 02:17:28,111 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 45/100 (estimated time remaining: 2 hours, 31 minutes, 36 seconds)
2025-05-10 02:20:07,675 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:20:11,882 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 635.81067 ± 239.375
2025-05-10 02:20:11,882 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [986.88525, 218.41168, 726.20026, 1004.5776, 673.9507, 493.07983, 322.20935, 636.9529, 574.08185, 721.75665]
2025-05-10 02:20:11,882 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [413.0, 111.0, 306.0, 431.0, 295.0, 182.0, 145.0, 240.0, 237.0, 284.0]
2025-05-10 02:20:11,882 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (635.81) for latency MM1Queue_a033_s075
2025-05-10 02:20:11,883 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 02:20:11,886 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 02:20:11,899 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 46/100 (estimated time remaining: 2 hours, 28 minutes, 59 seconds)
2025-05-10 02:22:46,830 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:22:53,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 879.66681 ± 584.410
2025-05-10 02:22:53,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [581.50214, 754.5049, 17.894232, 646.59985, 677.3791, 2223.0068, 1332.7809, 499.74585, 1400.7452, 662.5087]
2025-05-10 02:22:53,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [276.0, 390.0, 25.0, 284.0, 296.0, 939.0, 593.0, 201.0, 574.0, 262.0]
2025-05-10 02:22:53,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (879.67) for latency MM1Queue_a033_s075
2025-05-10 02:22:53,245 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 02:22:53,248 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 02:22:53,261 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 47/100 (estimated time remaining: 2 hours, 25 minutes, 55 seconds)
2025-05-10 02:25:37,076 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:25:40,908 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 618.40546 ± 382.144
2025-05-10 02:25:40,908 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1338.3269, 799.17944, 798.55255, 700.64594, 22.30321, 918.48145, 543.3439, 666.12054, 377.86365, 19.236961]
2025-05-10 02:25:40,909 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [516.0, 297.0, 292.0, 252.0, 27.0, 335.0, 226.0, 255.0, 176.0, 26.0]
2025-05-10 02:25:40,917 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 48/100 (estimated time remaining: 2 hours, 24 minutes, 31 seconds)
2025-05-10 02:28:17,518 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:28:21,319 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 606.33356 ± 397.536
2025-05-10 02:28:21,319 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1433.1489, 26.106947, 1075.8013, 592.2449, 75.2549, 626.85, 616.41614, 369.975, 609.27893, 638.2588]
2025-05-10 02:28:21,319 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [531.0, 31.0, 365.0, 239.0, 89.0, 239.0, 226.0, 142.0, 249.0, 265.0]
2025-05-10 02:28:21,327 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 49/100 (estimated time remaining: 2 hours, 21 minutes, 32 seconds)
2025-05-10 02:30:48,532 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:30:51,680 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 549.90918 ± 236.951
2025-05-10 02:30:51,680 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [361.32928, 688.56195, 220.8067, 324.5009, 400.10242, 773.3339, 765.83, 858.3532, 811.4126, 294.86075]
2025-05-10 02:30:51,680 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [161.0, 274.0, 120.0, 136.0, 175.0, 305.0, 325.0, 332.0, 333.0, 161.0]
2025-05-10 02:30:51,690 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 50/100 (estimated time remaining: 2 hours, 16 minutes, 36 seconds)
2025-05-10 02:33:04,959 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:33:08,137 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 618.90442 ± 167.228
2025-05-10 02:33:08,137 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [726.98004, 1013.38, 722.67816, 559.6299, 597.7141, 396.14777, 592.6737, 511.9416, 430.3984, 637.49994]
2025-05-10 02:33:08,137 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [274.0, 379.0, 245.0, 213.0, 244.0, 169.0, 230.0, 192.0, 162.0, 224.0]
2025-05-10 02:33:08,145 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 51/100 (estimated time remaining: 2 hours, 9 minutes, 22 seconds)
2025-05-10 02:35:21,409 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:35:24,024 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 501.48395 ± 191.148
2025-05-10 02:35:24,024 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [709.2604, 518.8007, 502.94586, 16.04939, 533.8227, 479.68036, 477.26663, 521.1993, 788.83185, 466.98196]
2025-05-10 02:35:24,024 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [230.0, 203.0, 216.0, 24.0, 203.0, 185.0, 196.0, 207.0, 294.0, 182.0]
2025-05-10 02:35:24,031 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 52/100 (estimated time remaining: 2 hours, 2 minutes, 37 seconds)
2025-05-10 02:37:34,734 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:37:38,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 696.29675 ± 179.665
2025-05-10 02:37:38,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [829.15784, 689.6175, 619.20984, 740.3753, 581.6801, 892.12006, 244.73415, 753.4584, 891.0128, 721.6016]
2025-05-10 02:37:38,244 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [334.0, 246.0, 241.0, 243.0, 211.0, 340.0, 133.0, 277.0, 305.0, 245.0]
2025-05-10 02:37:38,251 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 53/100 (estimated time remaining: 1 hour, 54 minutes, 46 seconds)
2025-05-10 02:39:51,684 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:39:56,145 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 837.11414 ± 236.711
2025-05-10 02:39:56,145 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [867.75146, 798.23535, 884.3066, 834.0432, 535.7721, 1099.1006, 1374.449, 579.19714, 737.0263, 661.25946]
2025-05-10 02:39:56,145 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [357.0, 350.0, 340.0, 310.0, 201.0, 420.0, 487.0, 215.0, 257.0, 295.0]
2025-05-10 02:39:56,153 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 54/100 (estimated time remaining: 1 hour, 48 minutes, 51 seconds)
2025-05-10 02:42:10,895 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:42:15,427 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 847.00995 ± 541.134
2025-05-10 02:42:15,427 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1124.6406, 172.38728, 730.05853, 387.38022, 578.7858, 2280.4167, 662.0279, 794.73065, 812.68536, 926.98663]
2025-05-10 02:42:15,427 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [415.0, 118.0, 300.0, 212.0, 223.0, 831.0, 245.0, 280.0, 281.0, 303.0]
2025-05-10 02:42:15,444 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 55/100 (estimated time remaining: 1 hour, 44 minutes, 50 seconds)
2025-05-10 02:44:24,730 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:44:30,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1077.89758 ± 518.024
2025-05-10 02:44:30,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [840.28595, 1265.5128, 1142.0774, 1132.8545, 616.2583, 427.85645, 660.77356, 1255.5035, 1040.0845, 2397.7683]
2025-05-10 02:44:30,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [293.0, 556.0, 435.0, 402.0, 208.0, 170.0, 250.0, 441.0, 386.0, 859.0]
2025-05-10 02:44:30,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (1077.90) for latency MM1Queue_a033_s075
2025-05-10 02:44:30,455 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 02:44:30,458 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 02:44:30,470 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 56/100 (estimated time remaining: 1 hour, 42 minutes, 20 seconds)
2025-05-10 02:46:42,461 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:46:46,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 858.62177 ± 317.729
2025-05-10 02:46:46,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [963.3274, 509.26147, 769.99927, 94.774765, 881.0008, 1094.5837, 957.7769, 1118.9342, 963.1077, 1233.4519]
2025-05-10 02:46:46,800 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [341.0, 218.0, 280.0, 60.0, 314.0, 369.0, 325.0, 372.0, 415.0, 429.0]
2025-05-10 02:46:46,808 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 57/100 (estimated time remaining: 1 hour, 40 minutes, 8 seconds)
2025-05-10 02:48:59,315 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:49:02,402 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 628.83228 ± 248.856
2025-05-10 02:49:02,402 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [844.63055, 942.19507, 729.8346, 872.63556, 341.0134, 738.4102, 473.05527, 798.74786, 323.64215, 224.15813]
2025-05-10 02:49:02,403 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [288.0, 312.0, 284.0, 318.0, 127.0, 243.0, 183.0, 271.0, 132.0, 108.0]
2025-05-10 02:49:02,411 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 58/100 (estimated time remaining: 1 hour, 38 minutes, 3 seconds)
2025-05-10 02:51:17,147 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:51:22,281 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 945.37970 ± 407.171
2025-05-10 02:51:22,281 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1428.4396, 401.45648, 977.6838, 429.3142, 642.9905, 829.2736, 913.73145, 1667.006, 1396.0746, 767.82715]
2025-05-10 02:51:22,281 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [533.0, 147.0, 379.0, 191.0, 245.0, 313.0, 353.0, 618.0, 526.0, 357.0]
2025-05-10 02:51:22,289 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 59/100 (estimated time remaining: 1 hour, 36 minutes, 3 seconds)
2025-05-10 02:53:31,101 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:53:36,498 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1035.09546 ± 531.032
2025-05-10 02:53:36,498 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [386.89896, 749.05237, 2079.339, 973.79034, 1641.6115, 1508.0764, 661.5685, 784.0449, 394.1666, 1172.4055]
2025-05-10 02:53:36,498 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [144.0, 259.0, 760.0, 368.0, 562.0, 564.0, 250.0, 297.0, 143.0, 431.0]
2025-05-10 02:53:36,506 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 60/100 (estimated time remaining: 1 hour, 33 minutes, 4 seconds)
2025-05-10 02:55:50,446 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:55:55,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1103.45410 ± 597.091
2025-05-10 02:55:55,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [555.5168, 1179.1699, 971.71106, 1101.1932, 672.37933, 684.5683, 764.6955, 2638.4648, 799.9719, 1666.8705]
2025-05-10 02:55:55,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [218.0, 388.0, 340.0, 398.0, 229.0, 225.0, 253.0, 847.0, 273.0, 569.0]
2025-05-10 02:55:55,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (1103.45) for latency MM1Queue_a033_s075
2025-05-10 02:55:55,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 02:55:55,770 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 02:55:55,783 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 61/100 (estimated time remaining: 1 hour, 31 minutes, 22 seconds)
2025-05-10 02:58:06,977 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 02:58:10,495 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 655.28406 ± 130.732
2025-05-10 02:58:10,495 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [631.272, 727.9792, 494.08655, 802.17633, 703.8392, 845.919, 595.3759, 592.13336, 753.0643, 406.99463]
2025-05-10 02:58:10,495 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [253.0, 301.0, 219.0, 318.0, 266.0, 311.0, 222.0, 229.0, 308.0, 153.0]
2025-05-10 02:58:10,504 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 62/100 (estimated time remaining: 1 hour, 28 minutes, 52 seconds)
2025-05-10 03:00:21,859 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:00:25,194 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 691.02954 ± 228.391
2025-05-10 03:00:25,194 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [795.29016, 645.0804, 694.73206, 309.51096, 916.3451, 364.29367, 707.6509, 1141.8892, 689.1341, 646.36847]
2025-05-10 03:00:25,194 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [280.0, 233.0, 232.0, 119.0, 348.0, 127.0, 229.0, 412.0, 224.0, 234.0]
2025-05-10 03:00:25,203 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 63/100 (estimated time remaining: 1 hour, 26 minutes, 29 seconds)
2025-05-10 03:02:44,337 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:02:48,647 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 885.95789 ± 429.477
2025-05-10 03:02:48,647 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [344.91016, 2024.3069, 735.8976, 499.54742, 737.98224, 816.37305, 746.4897, 913.90784, 1044.1293, 996.03436]
2025-05-10 03:02:48,647 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [147.0, 715.0, 241.0, 187.0, 266.0, 270.0, 273.0, 295.0, 383.0, 306.0]
2025-05-10 03:02:48,656 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 64/100 (estimated time remaining: 1 hour, 24 minutes, 39 seconds)
2025-05-10 03:04:56,505 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:05:00,598 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 882.15332 ± 408.474
2025-05-10 03:05:00,598 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [811.08887, 790.3629, 1526.3737, 427.8335, 1052.1497, 347.5872, 1102.196, 285.42096, 1066.9404, 1411.58]
2025-05-10 03:05:00,598 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [279.0, 272.0, 520.0, 164.0, 338.0, 133.0, 345.0, 117.0, 324.0, 463.0]
2025-05-10 03:05:00,607 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 65/100 (estimated time remaining: 1 hour, 22 minutes, 5 seconds)
2025-05-10 03:07:11,741 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:07:16,758 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1047.28748 ± 754.215
2025-05-10 03:07:16,758 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1580.0415, 803.95465, 785.68097, 1117.3202, 358.6188, 2591.6094, 2006.792, 361.7908, 66.87294, 800.19354]
2025-05-10 03:07:16,758 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [540.0, 260.0, 267.0, 367.0, 132.0, 830.0, 637.0, 135.0, 76.0, 272.0]
2025-05-10 03:07:16,767 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 66/100 (estimated time remaining: 1 hour, 19 minutes, 26 seconds)
2025-05-10 03:09:30,600 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:09:34,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 838.55066 ± 490.908
2025-05-10 03:09:34,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [745.6357, 812.73553, 1873.6145, 373.64542, 686.32135, 1410.7805, 758.23157, 15.759842, 1052.5714, 656.2113]
2025-05-10 03:09:34,577 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [259.0, 268.0, 583.0, 135.0, 245.0, 512.0, 241.0, 23.0, 345.0, 231.0]
2025-05-10 03:09:34,587 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 67/100 (estimated time remaining: 1 hour, 17 minutes, 31 seconds)
2025-05-10 03:11:49,441 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:11:57,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1684.45959 ± 720.735
2025-05-10 03:11:57,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1331.6176, 3252.9011, 1570.5983, 1256.0315, 1093.654, 1088.4198, 2094.418, 2204.0916, 706.8135, 2246.05]
2025-05-10 03:11:57,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [438.0, 934.0, 511.0, 423.0, 364.0, 342.0, 623.0, 687.0, 226.0, 713.0]
2025-05-10 03:11:57,118 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (1684.46) for latency MM1Queue_a033_s075
2025-05-10 03:11:57,119 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 03:11:57,122 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 03:11:57,135 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 68/100 (estimated time remaining: 1 hour, 16 minutes, 6 seconds)
2025-05-10 03:14:11,261 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:14:15,683 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 909.87305 ± 215.222
2025-05-10 03:14:15,683 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [915.93475, 783.0255, 1072.2147, 372.7931, 1040.5057, 970.57025, 1046.1344, 730.465, 1098.4039, 1068.6835]
2025-05-10 03:14:15,683 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [308.0, 255.0, 406.0, 141.0, 339.0, 443.0, 334.0, 299.0, 352.0, 326.0]
2025-05-10 03:14:15,693 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 69/100 (estimated time remaining: 1 hour, 13 minutes, 17 seconds)
2025-05-10 03:16:28,265 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:16:32,678 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 876.01105 ± 326.523
2025-05-10 03:16:32,678 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [527.9211, 796.40454, 663.2689, 1768.6288, 788.7371, 748.83594, 814.03265, 764.3771, 790.13586, 1097.7684]
2025-05-10 03:16:32,678 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [266.0, 276.0, 226.0, 659.0, 277.0, 242.0, 332.0, 264.0, 258.0, 369.0]
2025-05-10 03:16:32,688 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 70/100 (estimated time remaining: 1 hour, 11 minutes, 30 seconds)
2025-05-10 03:18:54,013 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:19:01,264 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1519.19165 ± 951.580
2025-05-10 03:19:01,264 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [541.2148, 621.4577, 2165.2627, 813.94, 1779.4712, 2252.71, 3132.605, 794.513, 395.91998, 2694.8218]
2025-05-10 03:19:01,264 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [185.0, 228.0, 690.0, 277.0, 579.0, 693.0, 1000.0, 276.0, 142.0, 834.0]
2025-05-10 03:19:01,274 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 71/100 (estimated time remaining: 1 hour, 10 minutes, 27 seconds)
2025-05-10 03:21:05,331 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:21:15,333 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2046.23560 ± 943.264
2025-05-10 03:21:15,333 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [777.10095, 2020.9146, 2132.817, 2985.2117, 3155.2207, 759.56384, 3064.6377, 3106.269, 1328.5426, 1132.0775]
2025-05-10 03:21:15,334 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [295.0, 612.0, 694.0, 884.0, 1000.0, 310.0, 1000.0, 1000.0, 436.0, 421.0]
2025-05-10 03:21:15,334 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (2046.24) for latency MM1Queue_a033_s075
2025-05-10 03:21:15,334 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 03:21:15,337 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 03:21:15,351 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 72/100 (estimated time remaining: 1 hour, 7 minutes, 44 seconds)
2025-05-10 03:23:35,292 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:23:39,768 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 964.00958 ± 155.204
2025-05-10 03:23:39,769 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1092.3232, 985.9974, 735.96387, 1134.5552, 752.2351, 1124.647, 948.59283, 1143.5194, 957.47296, 764.78906]
2025-05-10 03:23:39,769 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [365.0, 314.0, 253.0, 372.0, 268.0, 367.0, 327.0, 366.0, 357.0, 253.0]
2025-05-10 03:23:39,779 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 73/100 (estimated time remaining: 1 hour, 5 minutes, 34 seconds)
2025-05-10 03:25:54,744 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:26:00,308 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1163.60876 ± 789.019
2025-05-10 03:26:00,308 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [378.31284, 1793.9106, 3224.2131, 794.6764, 872.26886, 1194.2787, 753.82367, 1110.7004, 1104.8438, 409.05832]
2025-05-10 03:26:00,308 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [140.0, 591.0, 1000.0, 278.0, 294.0, 400.0, 254.0, 378.0, 361.0, 148.0]
2025-05-10 03:26:00,319 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 74/100 (estimated time remaining: 1 hour, 3 minutes, 24 seconds)
2025-05-10 03:28:09,485 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:28:18,606 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1928.72583 ± 924.296
2025-05-10 03:28:18,606 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2919.1, 3090.551, 1063.6251, 1076.7638, 373.2138, 1355.6288, 1694.2427, 2055.4214, 2444.025, 3214.6863]
2025-05-10 03:28:18,606 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [876.0, 1000.0, 328.0, 333.0, 145.0, 444.0, 564.0, 643.0, 751.0, 1000.0]
2025-05-10 03:28:18,616 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 75/100 (estimated time remaining: 1 hour, 1 minute, 10 seconds)
2025-05-10 03:30:28,102 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:30:37,028 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1985.07007 ± 746.546
2025-05-10 03:30:37,029 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2022.1294, 2941.3896, 2121.4253, 1492.1134, 2419.1245, 3337.2122, 863.41614, 1036.4551, 2090.5996, 1526.8364]
2025-05-10 03:30:37,029 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [622.0, 903.0, 617.0, 454.0, 738.0, 1000.0, 277.0, 309.0, 628.0, 468.0]
2025-05-10 03:30:37,039 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 76/100 (estimated time remaining: 57 minutes, 58 seconds)
2025-05-10 03:32:49,461 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:32:57,071 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1652.89941 ± 1237.784
2025-05-10 03:32:57,071 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1091.7188, 1510.8508, 1766.5509, 3398.7634, 3522.263, 637.8898, 408.33978, 3339.1992, 408.73636, 444.6823]
2025-05-10 03:32:57,071 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [347.0, 441.0, 518.0, 1000.0, 1000.0, 285.0, 155.0, 1000.0, 155.0, 170.0]
2025-05-10 03:32:57,082 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 77/100 (estimated time remaining: 56 minutes, 8 seconds)
2025-05-10 03:35:18,886 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:35:27,930 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1946.97327 ± 1087.381
2025-05-10 03:35:27,931 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1890.4686, 3207.685, 3291.7493, 763.0273, 731.3689, 3034.471, 2765.7605, 850.9604, 2474.2727, 459.96606]
2025-05-10 03:35:27,931 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [582.0, 950.0, 974.0, 247.0, 235.0, 1000.0, 805.0, 295.0, 736.0, 172.0]
2025-05-10 03:35:27,942 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 78/100 (estimated time remaining: 54 minutes, 17 seconds)
2025-05-10 03:37:36,661 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:37:45,512 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1827.60181 ± 1138.766
2025-05-10 03:37:45,512 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3056.6262, 1496.3092, 709.22394, 1089.5012, 733.964, 3271.1858, 3169.4187, 719.4794, 3274.7837, 755.52545]
2025-05-10 03:37:45,512 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 480.0, 232.0, 362.0, 270.0, 1000.0, 1000.0, 242.0, 1000.0, 247.0]
2025-05-10 03:37:45,523 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 79/100 (estimated time remaining: 51 minutes, 42 seconds)
2025-05-10 03:39:59,192 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:40:09,383 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2215.05908 ± 964.570
2025-05-10 03:40:09,383 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2130.0474, 3331.5823, 982.78955, 3220.8743, 2681.8403, 2056.9807, 2496.62, 746.85205, 1018.7193, 3484.2832]
2025-05-10 03:40:09,383 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [641.0, 1000.0, 321.0, 1000.0, 798.0, 681.0, 733.0, 258.0, 317.0, 1000.0]
2025-05-10 03:40:09,383 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (2215.06) for latency MM1Queue_a033_s075
2025-05-10 03:40:09,383 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 03:40:09,386 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 03:40:09,401 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 80/100 (estimated time remaining: 49 minutes, 45 seconds)
2025-05-10 03:42:26,794 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:42:35,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1776.95190 ± 919.716
2025-05-10 03:42:35,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2520.0837, 501.78592, 2494.3804, 2418.8105, 3299.8467, 1828.742, 1460.5486, 280.2451, 973.468, 1991.6074]
2025-05-10 03:42:35,162 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [794.0, 224.0, 758.0, 729.0, 1000.0, 554.0, 481.0, 121.0, 314.0, 629.0]
2025-05-10 03:42:35,174 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 81/100 (estimated time remaining: 47 minutes, 52 seconds)
2025-05-10 03:44:50,444 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:45:01,108 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2352.02393 ± 1012.091
2025-05-10 03:45:01,108 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2336.824, 3521.2341, 1727.0422, 1139.4227, 3507.772, 3379.064, 2449.731, 1413.5514, 3348.9568, 696.64325]
2025-05-10 03:45:01,108 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [671.0, 1000.0, 527.0, 382.0, 1000.0, 1000.0, 717.0, 459.0, 1000.0, 254.0]
2025-05-10 03:45:01,108 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (2352.02) for latency MM1Queue_a033_s075
2025-05-10 03:45:01,108 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 03:45:01,111 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 03:45:01,127 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 82/100 (estimated time remaining: 45 minutes, 51 seconds)
2025-05-10 03:47:12,049 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:47:21,499 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2069.72314 ± 1059.359
2025-05-10 03:47:21,499 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1173.6603, 3521.532, 1047.6132, 2901.6187, 1241.6082, 3162.1057, 770.6105, 3432.563, 2397.6443, 1048.2761]
2025-05-10 03:47:21,500 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [377.0, 1000.0, 343.0, 861.0, 402.0, 929.0, 272.0, 1000.0, 735.0, 335.0]
2025-05-10 03:47:21,511 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 83/100 (estimated time remaining: 42 minutes, 48 seconds)
2025-05-10 03:49:45,143 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:49:59,539 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 3050.55225 ± 553.591
2025-05-10 03:49:59,539 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3401.194, 3406.7754, 3271.3113, 3052.6416, 3263.2546, 2150.7776, 3418.1704, 1799.5356, 3424.4868, 3317.378]
2025-05-10 03:49:59,539 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 965.0, 1000.0, 666.0, 1000.0, 577.0, 1000.0, 1000.0]
2025-05-10 03:49:59,539 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (3050.55) for latency MM1Queue_a033_s075
2025-05-10 03:49:59,539 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 03:49:59,542 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 03:49:59,558 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 84/100 (estimated time remaining: 41 minutes, 35 seconds)
2025-05-10 03:52:04,207 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:52:15,320 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2386.88184 ± 971.111
2025-05-10 03:52:15,320 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1903.3972, 3463.2258, 1079.9784, 1808.0616, 1079.0159, 3125.1208, 3441.8206, 1423.0276, 3018.7183, 3526.451]
2025-05-10 03:52:15,320 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [591.0, 1000.0, 334.0, 595.0, 401.0, 1000.0, 1000.0, 438.0, 878.0, 1000.0]
2025-05-10 03:52:15,332 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 85/100 (estimated time remaining: 38 minutes, 42 seconds)
2025-05-10 03:54:39,668 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:54:48,215 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 1859.61584 ± 1052.068
2025-05-10 03:54:48,215 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1105.7268, 840.9788, 3361.1646, 742.99365, 1698.7693, 3434.872, 1961.0027, 1031.8734, 3330.4543, 1088.3221]
2025-05-10 03:54:48,215 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [327.0, 294.0, 1000.0, 228.0, 526.0, 1000.0, 603.0, 323.0, 1000.0, 377.0]
2025-05-10 03:54:48,227 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 86/100 (estimated time remaining: 36 minutes, 39 seconds)
2025-05-10 03:57:00,400 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:57:12,562 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2659.48291 ± 1035.197
2025-05-10 03:57:12,562 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3458.385, 3562.874, 3489.027, 3529.302, 2087.9482, 1061.6755, 877.54443, 3415.6965, 3320.7449, 1791.6315]
2025-05-10 03:57:12,562 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 612.0, 330.0, 347.0, 1000.0, 1000.0, 575.0]
2025-05-10 03:57:12,574 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 87/100 (estimated time remaining: 34 minutes, 8 seconds)
2025-05-10 03:59:17,267 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 03:59:30,199 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2801.77954 ± 1001.770
2025-05-10 03:59:30,199 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3386.1582, 3407.6409, 3334.3047, 3423.6243, 3465.0806, 2627.297, 3378.5315, 799.80005, 903.0888, 3292.2695]
2025-05-10 03:59:30,199 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 774.0, 1000.0, 270.0, 319.0, 1000.0]
2025-05-10 03:59:30,211 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 88/100 (estimated time remaining: 31 minutes, 34 seconds)
2025-05-10 04:01:51,874 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:02:02,791 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2408.05615 ± 1239.256
2025-05-10 04:02:02,791 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1084.0454, 3410.3884, 708.4629, 3528.877, 3405.2812, 3516.977, 1096.3442, 714.7859, 3418.1272, 3197.2708]
2025-05-10 04:02:02,791 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [348.0, 1000.0, 220.0, 1000.0, 1000.0, 1000.0, 335.0, 226.0, 1000.0, 936.0]
2025-05-10 04:02:02,804 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 89/100 (estimated time remaining: 28 minutes, 55 seconds)
2025-05-10 04:04:13,060 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:04:28,798 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 3385.08081 ± 137.551
2025-05-10 04:04:28,798 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3381.0552, 3327.5156, 3437.1191, 3538.127, 3553.1755, 3088.0083, 3438.034, 3483.9363, 3210.1958, 3393.643]
2025-05-10 04:04:28,798 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 948.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-05-10 04:04:28,798 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (3385.08) for latency MM1Queue_a033_s075
2025-05-10 04:04:28,798 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 04:04:28,801 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 04:04:28,818 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 90/100 (estimated time remaining: 26 minutes, 53 seconds)
2025-05-10 04:06:48,127 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:07:02,545 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2910.00195 ± 857.467
2025-05-10 04:07:02,545 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3223.376, 3250.3298, 356.35107, 3073.1821, 3070.2383, 3385.9216, 3047.1743, 3175.3323, 3312.1606, 3205.9526]
2025-05-10 04:07:02,545 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 143.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-05-10 04:07:02,558 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 91/100 (estimated time remaining: 24 minutes, 28 seconds)
2025-05-10 04:09:18,011 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:09:33,351 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 3270.23779 ± 282.445
2025-05-10 04:09:33,351 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3526.702, 3474.4365, 3397.214, 3364.692, 3367.9119, 2481.823, 3387.7546, 3291.9497, 3280.0798, 3129.813]
2025-05-10 04:09:33,351 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 754.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-05-10 04:09:33,364 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 92/100 (estimated time remaining: 22 minutes, 13 seconds)
2025-05-10 04:11:47,963 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:12:00,972 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2812.34448 ± 1051.528
2025-05-10 04:12:00,972 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1418.6118, 3397.5627, 3329.05, 3431.1716, 2477.3928, 288.67697, 3439.7295, 3605.0396, 3394.5767, 3341.634]
2025-05-10 04:12:00,972 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [450.0, 1000.0, 1000.0, 1000.0, 759.0, 130.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-05-10 04:12:00,985 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 93/100 (estimated time remaining: 20 minutes, 1 second)
2025-05-10 04:14:13,210 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:14:24,989 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2613.74634 ± 1252.962
2025-05-10 04:14:24,989 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3496.9849, 3449.7603, 516.631, 355.9872, 3087.0623, 3506.7832, 3353.742, 1361.3694, 3507.8127, 3501.3308]
2025-05-10 04:14:24,989 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 192.0, 140.0, 885.0, 1000.0, 943.0, 430.0, 1000.0, 1000.0]
2025-05-10 04:14:25,003 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 94/100 (estimated time remaining: 17 minutes, 19 seconds)
2025-05-10 04:16:41,772 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:16:51,975 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2332.34155 ± 838.732
2025-05-10 04:16:51,976 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2263.923, 3573.0085, 2541.1238, 1640.0587, 1665.06, 3500.694, 1089.234, 3382.322, 1915.0695, 1752.9191]
2025-05-10 04:16:51,976 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [687.0, 1000.0, 692.0, 489.0, 489.0, 1000.0, 329.0, 1000.0, 559.0, 519.0]
2025-05-10 04:16:51,989 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 95/100 (estimated time remaining: 14 minutes, 51 seconds)
2025-05-10 04:19:04,464 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:19:15,298 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2435.25317 ± 1200.199
2025-05-10 04:19:15,299 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [766.20123, 2352.383, 3478.9702, 701.97217, 3415.2783, 3577.0632, 822.8487, 2110.103, 3538.0828, 3589.628]
2025-05-10 04:19:15,299 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [244.0, 673.0, 1000.0, 220.0, 1000.0, 1000.0, 292.0, 595.0, 1000.0, 1000.0]
2025-05-10 04:19:15,312 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 96/100 (estimated time remaining: 12 minutes, 12 seconds)
2025-05-10 04:21:40,341 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:21:53,438 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2867.72583 ± 1188.045
2025-05-10 04:21:53,438 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3419.5857, 3540.0935, 3612.0938, 3448.4685, 3526.572, 3396.0444, 3323.6436, 3404.1143, 699.44385, 307.19736]
2025-05-10 04:21:53,438 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 222.0, 125.0]
2025-05-10 04:21:53,452 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 97/100 (estimated time remaining: 9 minutes, 52 seconds)
2025-05-10 04:24:08,461 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:24:22,674 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 3093.59473 ± 652.458
2025-05-10 04:24:22,674 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3529.492, 3418.041, 3516.068, 3365.281, 3408.586, 1529.1355, 3282.6553, 3299.1853, 2121.117, 3466.3845]
2025-05-10 04:24:22,674 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 459.0, 1000.0, 1000.0, 612.0, 1000.0]
2025-05-10 04:24:22,687 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 98/100 (estimated time remaining: 7 minutes, 25 seconds)
2025-05-10 04:26:26,208 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:26:38,766 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2837.42212 ± 919.174
2025-05-10 04:26:38,766 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [2850.3088, 1788.9674, 3548.6697, 3368.93, 1183.2192, 3582.8184, 3473.794, 3480.2832, 1479.9816, 3617.2454]
2025-05-10 04:26:38,766 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [781.0, 525.0, 1000.0, 1000.0, 354.0, 1000.0, 1000.0, 1000.0, 436.0, 1000.0]
2025-05-10 04:26:38,780 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 99/100 (estimated time remaining: 4 minutes, 53 seconds)
2025-05-10 04:28:58,808 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:29:10,346 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 2648.41870 ± 954.175
2025-05-10 04:29:10,347 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [1895.2416, 3604.661, 3531.94, 1142.2733, 2178.6257, 3634.1255, 2544.8503, 3282.1965, 3512.5757, 1157.6978]
2025-05-10 04:29:10,347 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [578.0, 1000.0, 1000.0, 345.0, 609.0, 1000.0, 747.0, 892.0, 1000.0, 366.0]
2025-05-10 04:29:10,362 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1199 [INFO]: Iteration 100/100 (estimated time remaining: 2 minutes, 27 seconds)
2025-05-10 04:31:26,044 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1214 [DEBUG]: Evaluating for latency MM1Queue_a033_s075...
2025-05-10 04:31:41,722 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1221 [DEBUG]: Total Reward: 3547.72778 ± 133.509
2025-05-10 04:31:41,723 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1222 [DEBUG]: All rewards: [3497.279, 3443.4976, 3585.682, 3697.7927, 3283.253, 3664.0027, 3675.0786, 3531.6182, 3692.5852, 3406.4893]
2025-05-10 04:31:41,723 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1223 [DEBUG]: All trajectory lengths: [1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0, 1000.0]
2025-05-10 04:31:41,723 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1226 [INFO]: New best (3547.73) for latency MM1Queue_a033_s075
2025-05-10 04:31:41,723 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1229 [INFO]: saving network
2025-05-10 04:31:41,726 latency_env.training.utils:544 [DEBUG]: Saving evalcopy of BPQL to _logs/benchmark-v3-tc8/noisy-walker2d/MM1Queue_a033_s075-bpql-mem16/checkpoints/best_MM1Queue_a033_s075.pkl
2025-05-10 04:31:41,746 latency_env.delayed_mdp:training_loop(baseline-bpql-noisy-walker2d):1251 [DEBUG]: Training session finished
