wandb: Currently logged in as: 804703098. Use `wandb login --relogin` to force relogin
wandb: wandb version 0.16.6 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.16.5
wandb: Run data is saved locally in /home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/scripts/results/StarCraft2/3s_vs_3z/mappotrsyn/train_3s_vs_3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240412_154257-v7pjvf4i
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run StarCraft2_3s_vs_3z_1
wandb: ⭐️ View project at https://wandb.ai/804703098/3s_vs_3z_sigma_v0
wandb: 🚀 View run at https://wandb.ai/804703098/3s_vs_3z_sigma_v0/runs/v7pjvf4i/workspace
choose to use gpu...
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 45319 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-1m6ynb75/'
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 41713 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-00_b4q89/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 46229 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-rxrd1j87/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 38739 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-2vb5sdk5/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 43227 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-7yl70_nt/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 34561 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-yjgva5_a/'
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 38339 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-gios1q8l/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 42473 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-0iy7lkmz/'
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:41713
Listening on: 127.0.0.1:43227
Listening on: 127.0.0.1:45319
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:38739
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:34561
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:38339
Listening on: 127.0.0.1:46229
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:42473
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:47026 accepted
ConnectHandler: Request from 127.0.0.1:44338 accepted
ConnectHandler: Request from 127.0.0.1:60954 accepted
ReadyHandler: 127.0.0.1:47026 ready
ReadyHandler: 127.0.0.1:44338 ready
ReadyHandler: 127.0.0.1:60954 ready
ConnectHandler: Request from 127.0.0.1:37232 accepted
ReadyHandler: 127.0.0.1:37232 ready
ConnectHandler: Request from 127.0.0.1:51224 accepted
ReadyHandler: 127.0.0.1:51224 ready
ConnectHandler: Request from 127.0.0.1:47110 accepted
ReadyHandler: 127.0.0.1:47110 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 2
Next launch phase started: 5
Next launch phase started: 3
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Requesting to join a single player game
Configuring interface options
Launching next game.
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Launching next game.
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
ConnectHandler: Request from 127.0.0.1:36036 accepted
ReadyHandler: 127.0.0.1:36036 ready
ConnectHandler: Request from 127.0.0.1:33796 accepted
ReadyHandler: 127.0.0.1:33796 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Sending ResponseJoinGame
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Sending ResponseJoinGame
Successfully loaded stable ids: GameData\stableid.json
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Sending ResponseJoinGame
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 37703 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-avtgxxem/'
Starting up...
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:37703
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:49502 accepted
ReadyHandler: 127.0.0.1:49502 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
CloseHandler: 127.0.0.1:47026 disconnected
CloseHandler: 127.0.0.1:51224 disconnected
CloseHandler: 127.0.0.1:44338 disconnected
CloseHandler: 127.0.0.1:60954 disconnected
CloseHandler: 127.0.0.1:47110 disconnected
CloseHandler: 127.0.0.1:36036 disconnected
CloseHandler: 127.0.0.1:37232 disconnected
CloseHandler: 127.0.0.1:33796 disconnected
/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py:394: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  eval_episode_team_rewards = np.array(eval_episode_team_rewards)
idv policy and team policy use same initial params!

 Map 3s_vs_3z Algo mappotrsyn Exp train_3s_vs_3z_sigma_v0_CMT_s2r2_v0 updates 0/1000 episodes, total num timesteps 3200/3200000, FPS 332.

incre win rate is 0.0.
team_policy eval win rate is 0.0.
idv_policy eval win rate is 0.0.
RequestQuit command received.
Closing Application...
DataHandler: unable to parse websocket frame.
CloseHandler: 127.0.0.1:49502 disconnected
wandb: - 0.006 MB of 0.006 MB uploaded
wandb: \ 0.502 MB of 0.502 MB uploaded
wandb: | 0.502 MB of 0.502 MB uploaded
wandb: 
wandb: Run history:
wandb:                              Aa_idv_actor_loss ▁
wandb:                                 Ab_policy_loss ▁
wandb:                            Ac_idv_ppo_loss_abs ▁
wandb:                                Ad_idv_ppo_prop ▁
wandb:                                         Ae_eta ▁
wandb:                           Af_noclip_proportion ▁
wandb:                           Ag_update_proportion ▁
wandb:                                 Ah_update_loss ▁
wandb:                                Ai_idv_epsilon' ▁
wandb:                                   Aj_idv_sigma ▁
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') ▁
wandb:                       Al_idv_noclip_proportion ▁
wandb:              Am_idv_(sigma*A)update_proportion ▁
wandb:                    An_idv_(sigma*A)update_loss ▁
wandb:                            Ao_idv_entropy_prop ▁
wandb:                                Ap_dist_entropy ▁
wandb:                                 Aq_idv_kl_prop ▁
wandb:                                 Ar_idv_kl_coef ▁
wandb:                                 As_idv_kl_loss ▁
wandb:                           At_idv_cross_entropy ▁
wandb:                                  Au_value_loss ▁
wandb:                                  Av_advantages ▁
wandb:                              Aw_idv_actor_norm ▁
wandb:                             Ax_idv_critic_norm ▁
wandb:                            Ba_idv_org_min_prop ▁
wandb:                            Bb_idv_org_max_prop ▁
wandb:                            Bc_idv_org_org_prop ▁
wandb:                            Bd_idv_new_min_prop ▁
wandb:                            Be_idv_new_max_prop ▁
wandb:                             Ta_team_actor_loss ▁
wandb:                            Tb_team_policy_loss ▁
wandb:                           Tc_team_ppo_loss_abs ▁
wandb:                               Td_team_ppo_prop ▁
wandb:                               Te_team_epsilon^ ▁
wandb:                                 Tf_team_sigma^ ▁
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') ▁
wandb:                      Th_team_noclip_proportion ▁
wandb:            Ti_team_(sigma^*A)update_proportion ▁
wandb:                  Tj_team_(sigma^*A)update_loss ▁
wandb:                           Tk_team_entropy_prop ▁
wandb:                           Tl_team_dist_entropy ▁
wandb:                                Tm_team_kl_prop ▁
wandb:                                Tn_team_kl_coef ▁
wandb:                                To_team_kl_loss ▁
wandb:                          Tp_team_cross_entropy ▁
wandb:                             Tq_team_value_loss ▁
wandb:                             Tr_team_advantages ▁
wandb:                             Ts_team_actor_norm ▁
wandb:                            Tt_team_critic_norm ▁
wandb:                           average_step_rewards ▁
wandb:                      average_step_team_rewards ▁
wandb:                                     dead_ratio ▁
wandb:        idv_policy_eval_average_episode_rewards ▁
wandb:   idv_policy_eval_average_episode_team_rewards ▁
wandb:                       idv_policy_eval_win_rate ▁
wandb:                                 incre_win_rate ▁
wandb:       team_policy_eval_average_episode_rewards ▁
wandb:  team_policy_eval_average_episode_team_rewards ▁
wandb:                      team_policy_eval_win_rate ▁
wandb: 
wandb: Run summary:
wandb:                              Aa_idv_actor_loss -0.00587
wandb:                                 Ab_policy_loss 0.0125
wandb:                            Ac_idv_ppo_loss_abs 0.90767
wandb:                                Ad_idv_ppo_prop 0.98016
wandb:                                         Ae_eta 0.99974
wandb:                           Af_noclip_proportion 0.95644
wandb:                           Ag_update_proportion 0.59268
wandb:                                 Ah_update_loss -0.47568
wandb:                                Ai_idv_epsilon' 1.0
wandb:                                   Aj_idv_sigma 0.99974
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') 0.99974
wandb:                       Al_idv_noclip_proportion 1.0
wandb:              Am_idv_(sigma*A)update_proportion 0.39689
wandb:                    An_idv_(sigma*A)update_loss -0.22787
wandb:                            Ao_idv_entropy_prop 0.01984
wandb:                                Ap_dist_entropy 1.83696
wandb:                                 Aq_idv_kl_prop 0.0
wandb:                                 Ar_idv_kl_coef 0.0
wandb:                                 As_idv_kl_loss 0.00021
wandb:                           At_idv_cross_entropy 0.0
wandb:                                  Au_value_loss 1.44435
wandb:                                  Av_advantages -0.3376
wandb:                              Aw_idv_actor_norm 0.06374
wandb:                             Ax_idv_critic_norm 1.8483
wandb:                            Ba_idv_org_min_prop 0.52729
wandb:                            Bb_idv_org_max_prop 0.06539
wandb:                            Bc_idv_org_org_prop 0.0
wandb:                            Bd_idv_new_min_prop 0.15065
wandb:                            Be_idv_new_max_prop 0.24624
wandb:                             Ta_team_actor_loss -0.02227
wandb:                            Tb_team_policy_loss -0.00576
wandb:                           Tc_team_ppo_loss_abs 0.70227
wandb:                               Td_team_ppo_prop 0.972
wandb:                               Te_team_epsilon^ 0.05
wandb:                                 Tf_team_sigma^ 0.99969
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') 0.99943
wandb:                      Th_team_noclip_proportion 0.9649
wandb:            Ti_team_(sigma^*A)update_proportion 0.97935
wandb:                  Tj_team_(sigma^*A)update_loss -0.10012
wandb:                           Tk_team_entropy_prop 0.02542
wandb:                           Tl_team_dist_entropy 1.83693
wandb:                                Tm_team_kl_prop 0.00257
wandb:                                Tn_team_kl_coef 9.0
wandb:                                To_team_kl_loss 0.00021
wandb:                          Tp_team_cross_entropy 0.0
wandb:                             Tq_team_value_loss 0.85988
wandb:                             Tr_team_advantages -0.07102
wandb:                             Ts_team_actor_norm 0.18518
wandb:                            Tt_team_critic_norm 1.30588
wandb:                           average_step_rewards 0.0869
wandb:                      average_step_team_rewards 0.0
wandb:                                     dead_ratio 0.3219
wandb:        idv_policy_eval_average_episode_rewards 7.14269
wandb:   idv_policy_eval_average_episode_team_rewards 0.0
wandb:                       idv_policy_eval_win_rate 0.0
wandb:                                 incre_win_rate 0.0
wandb:       team_policy_eval_average_episode_rewards 8.32801
wandb:  team_policy_eval_average_episode_team_rewards 0.0
wandb:                      team_policy_eval_win_rate 0.0
wandb: 
wandb: 🚀 View run StarCraft2_3s_vs_3z_1 at: https://wandb.ai/804703098/3s_vs_3z_sigma_v0/runs/v7pjvf4i/workspace
wandb: Synced 6 W&B file(s), 0 media file(s), 0 artifact file(s) and 4 other file(s)
wandb: Find logs at: ./results/StarCraft2/3s_vs_3z/mappotrsyn/train_3s_vs_3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240412_154257-v7pjvf4i/logs
Traceback (most recent call last):
  File "train/train_smac_trsyn.py", line 247, in <module>
    main(sys.argv[1:])
  File "train/train_smac_trsyn.py", line 232, in main
    runner.run()
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py", line 56, in run
    obs, share_obs, rewards, dones, infos, available_actions = self.envs.step(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 106, in step
    self.step_async(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 367, in step_async
    remote.send(('step', action))
  File "/home/user/anaconda3/envs/zypython38/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/user/anaconda3/envs/zypython38/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
    self._send(header + buf)
  File "/home/user/anaconda3/envs/zypython38/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
