wandb: Currently logged in as: 804703098. Use `wandb login --relogin` to force relogin
wandb: wandb version 0.17.5 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.16.6
wandb: Run data is saved locally in /home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/scripts/results/StarCraft2/2s3z/mappotrsyn/train_2s3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240802_190133-dmpjzh53
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run StarCraft2_2s3z_4
wandb: ⭐️ View project at https://wandb.ai/804703098/2s3z_sigma_v0
wandb: 🚀 View run at https://wandb.ai/804703098/2s3z_sigma_v0/runs/dmpjzh53
choose to use cpu...
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 42097 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-pw1imlas/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 44747 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-idpdt76r/'
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 38737 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-naxq4v4u/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 36175 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-a8t3fzw7/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 35293 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-ubxs2ev8/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 39147 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-y5hgpzlt/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 37329 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-mttzv407/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 37793 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-40q1_wjn/'
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:38737
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:44747
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:37793
Listening on: 127.0.0.1:36175
Listening on: 127.0.0.1:35293
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:42097
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:37329
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:39147
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:49724 accepted
ConnectHandler: Request from 127.0.0.1:40906 accepted
ReadyHandler: 127.0.0.1:49724 ready
ReadyHandler: 127.0.0.1:40906 ready
ConnectHandler: Request from 127.0.0.1:45698 accepted
ReadyHandler: 127.0.0.1:45698 ready
ConnectHandler: Request from 127.0.0.1:57914 accepted
ReadyHandler: 127.0.0.1:57914 ready
ConnectHandler: Request from 127.0.0.1:46990 accepted
ReadyHandler: 127.0.0.1:46990 ready
ConnectHandler: Request from 127.0.0.1:40854 accepted
ReadyHandler: 127.0.0.1:40854 ready
ConnectHandler: Request from 127.0.0.1:53142 accepted
ReadyHandler: 127.0.0.1:53142 ready
ConnectHandler: Request from 127.0.0.1:40452 accepted
ReadyHandler: 127.0.0.1:40452 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 8
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Next launch phase started: 2
Next launch phase started: 3
Launching next game.
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Requesting to join a single player game
Configuring interface options
Next launch phase started: 4
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Launching next game.
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Game has started.
Game has started.
Game has started.
Game has started.
Game has started.
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Sending ResponseJoinGame
Sending ResponseJoinGame
Sending ResponseJoinGame
Sending ResponseJoinGame
Sending ResponseJoinGame
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
CloseHandler: 127.0.0.1:40854 disconnected
CloseHandler: 127.0.0.1:40906 disconnected
CloseHandler: 127.0.0.1:53142 disconnected
CloseHandler: 127.0.0.1:46990 disconnected
CloseHandler: 127.0.0.1:57914 disconnected
CloseHandler: 127.0.0.1:45698 disconnected
CloseHandler: 127.0.0.1:40452 disconnected
CloseHandler: 127.0.0.1:49724 disconnected
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 42547 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-pue4ijud/'
Starting up...
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:42547
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:45316 accepted
ReadyHandler: 127.0.0.1:45316 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py:394: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  eval_episode_team_rewards = np.array(eval_episode_team_rewards)
idv policy and team policy use same initial params!

 Map 2s3z Algo mappotrsyn Exp train_2s3z_sigma_v0_CMT_s2r2_v0 updates 0/1000 episodes, total num timesteps 3200/3200000, FPS 99.

incre win rate is 0.0.
team_policy eval win rate is 0.0.
idv_policy eval win rate is 0.0.
RequestQuit command received.
DataHandler: unable to parse websocket frame.
Closing Application...
CloseHandler: 127.0.0.1:45316 disconnected
wandb: - 0.006 MB of 0.006 MB uploaded
wandb: \ 0.006 MB of 0.006 MB uploaded
wandb: | 0.270 MB of 0.819 MB uploaded
wandb: / 0.819 MB of 0.819 MB uploaded
wandb: 
wandb: Run history:
wandb:                              Aa_idv_actor_loss ▁
wandb:                                 Ab_policy_loss ▁
wandb:                            Ac_idv_ppo_loss_abs ▁
wandb:                                Ad_idv_ppo_prop ▁
wandb:                                         Ae_eta ▁
wandb:                           Af_noclip_proportion ▁
wandb:                           Ag_update_proportion ▁
wandb:                                 Ah_update_loss ▁
wandb:                                Ai_idv_epsilon' ▁
wandb:                                   Aj_idv_sigma ▁
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') ▁
wandb:                       Al_idv_noclip_proportion ▁
wandb:              Am_idv_(sigma*A)update_proportion ▁
wandb:                    An_idv_(sigma*A)update_loss ▁
wandb:                            Ao_idv_entropy_prop ▁
wandb:                                Ap_dist_entropy ▁
wandb:                                 Aq_idv_kl_prop ▁
wandb:                                 Ar_idv_kl_coef ▁
wandb:                                 As_idv_kl_loss ▁
wandb:                           At_idv_cross_entropy ▁
wandb:                                  Au_value_loss ▁
wandb:                                  Av_advantages ▁
wandb:                              Aw_idv_actor_norm ▁
wandb:                             Ax_idv_critic_norm ▁
wandb:                            Ba_idv_org_min_prop ▁
wandb:                            Bb_idv_org_max_prop ▁
wandb:                            Bc_idv_org_org_prop ▁
wandb:                            Bd_idv_new_min_prop ▁
wandb:                            Be_idv_new_max_prop ▁
wandb:                             Ta_team_actor_loss ▁
wandb:                            Tb_team_policy_loss ▁
wandb:                           Tc_team_ppo_loss_abs ▁
wandb:                               Td_team_ppo_prop ▁
wandb:                               Te_team_epsilon^ ▁
wandb:                                 Tf_team_sigma^ ▁
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') ▁
wandb:                      Th_team_noclip_proportion ▁
wandb:            Ti_team_(sigma^*A)update_proportion ▁
wandb:                  Tj_team_(sigma^*A)update_loss ▁
wandb:                           Tk_team_entropy_prop ▁
wandb:                           Tl_team_dist_entropy ▁
wandb:                                Tm_team_kl_prop ▁
wandb:                                Tn_team_kl_coef ▁
wandb:                                To_team_kl_loss ▁
wandb:                          Tp_team_cross_entropy ▁
wandb:                             Tq_team_value_loss ▁
wandb:                             Tr_team_advantages ▁
wandb:                             Ts_team_actor_norm ▁
wandb:                            Tt_team_critic_norm ▁
wandb:                           average_step_rewards ▁
wandb:                      average_step_team_rewards ▁
wandb:                                     dead_ratio ▁
wandb:        idv_policy_eval_average_episode_rewards ▁
wandb:   idv_policy_eval_average_episode_team_rewards ▁
wandb:                       idv_policy_eval_win_rate ▁
wandb:                                 incre_win_rate ▁
wandb:       team_policy_eval_average_episode_rewards ▁
wandb:  team_policy_eval_average_episode_team_rewards ▁
wandb:                      team_policy_eval_win_rate ▁
wandb: 
wandb: Run summary:
wandb:                              Aa_idv_actor_loss -0.00988
wandb:                                 Ab_policy_loss 0.00912
wandb:                            Ac_idv_ppo_loss_abs 0.92011
wandb:                                Ad_idv_ppo_prop 0.97977
wandb:                                         Ae_eta 0.99994
wandb:                           Af_noclip_proportion 0.99858
wandb:                           Ag_update_proportion 0.57792
wandb:                                 Ah_update_loss -0.43267
wandb:                                Ai_idv_epsilon' 1.0
wandb:                                   Aj_idv_sigma 0.99994
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') 0.99994
wandb:                       Al_idv_noclip_proportion 1.0
wandb:              Am_idv_(sigma*A)update_proportion 0.42173
wandb:                    An_idv_(sigma*A)update_loss -0.25792
wandb:                            Ao_idv_entropy_prop 0.02023
wandb:                                Ap_dist_entropy 1.89998
wandb:                                 Aq_idv_kl_prop 0.0
wandb:                                 Ar_idv_kl_coef 0.0
wandb:                                 As_idv_kl_loss 5e-05
wandb:                           At_idv_cross_entropy 0.0
wandb:                                  Au_value_loss 0.67992
wandb:                                  Av_advantages -0.32915
wandb:                              Aw_idv_actor_norm 0.05998
wandb:                             Ax_idv_critic_norm 1.74881
wandb:                            Ba_idv_org_min_prop 0.4928
wandb:                            Bb_idv_org_max_prop 0.08512
wandb:                            Bc_idv_org_org_prop 0.0
wandb:                            Bd_idv_new_min_prop 0.18387
wandb:                            Be_idv_new_max_prop 0.23786
wandb:                             Ta_team_actor_loss -0.02187
wandb:                            Tb_team_policy_loss -0.00434
wandb:                           Tc_team_ppo_loss_abs 0.74818
wandb:                               Td_team_ppo_prop 0.97337
wandb:                               Te_team_epsilon^ 0.05
wandb:                                 Tf_team_sigma^ 0.9999
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') 0.99982
wandb:                      Th_team_noclip_proportion 0.98299
wandb:            Ti_team_(sigma^*A)update_proportion 0.98888
wandb:                  Tj_team_(sigma^*A)update_loss 0.06817
wandb:                           Tk_team_entropy_prop 0.02472
wandb:                           Tl_team_dist_entropy 1.89981
wandb:                                Tm_team_kl_prop 0.00191
wandb:                                Tn_team_kl_coef 9.0
wandb:                                To_team_kl_loss 0.00016
wandb:                          Tp_team_cross_entropy 0.0
wandb:                             Tq_team_value_loss 0.27669
wandb:                             Tr_team_advantages 0.07858
wandb:                             Ts_team_actor_norm 0.11388
wandb:                            Tt_team_critic_norm 0.49616
wandb:                           average_step_rewards 0.07762
wandb:                      average_step_team_rewards 0.0
wandb:                                     dead_ratio 0.31253
wandb:        idv_policy_eval_average_episode_rewards 4.79289
wandb:   idv_policy_eval_average_episode_team_rewards 0.0
wandb:                       idv_policy_eval_win_rate 0.0
wandb:                                 incre_win_rate 0.0
wandb:       team_policy_eval_average_episode_rewards 6.65525
wandb:  team_policy_eval_average_episode_team_rewards 0.0
wandb:                      team_policy_eval_win_rate 0.0
wandb: 
wandb: 🚀 View run StarCraft2_2s3z_4 at: https://wandb.ai/804703098/2s3z_sigma_v0/runs/dmpjzh53
wandb: ⭐️ View project at: https://wandb.ai/804703098/2s3z_sigma_v0
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 4 other file(s)
wandb: Find logs at: ./results/StarCraft2/2s3z/mappotrsyn/train_2s3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240802_190133-dmpjzh53/logs
Traceback (most recent call last):
  File "train/train_smac_trsyn.py", line 247, in <module>
    main(sys.argv[1:])
  File "train/train_smac_trsyn.py", line 232, in main
    runner.run()
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py", line 56, in run
    obs, share_obs, rewards, dones, infos, available_actions = self.envs.step(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 106, in step
    self.step_async(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 367, in step_async
    remote.send(('step', action))
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
    self._send(header + buf)
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
