wandb: Currently logged in as: 804703098. Use `wandb login --relogin` to force relogin
wandb: wandb version 0.17.5 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade
wandb: Tracking run with wandb version 0.16.6
wandb: Run data is saved locally in /home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/scripts/results/StarCraft2/2s3z/mappotrsyn/train_2s3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240802_190126-stltib29
wandb: Run `wandb offline` to turn off syncing.
wandb: Syncing run StarCraft2_2s3z_3
wandb: ⭐️ View project at https://wandb.ai/804703098/2s3z_sigma_v0
wandb: 🚀 View run at https://wandb.ai/804703098/2s3z_sigma_v0/runs/stltib29
choose to use cpu...
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 36359 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-y_mtu7i_/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 32943 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-x311t9lu/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 44371 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-rcy0_bhk/'
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 40553 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-9govgtjl/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 45283 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-7grhyuaf/'
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 45755 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-li4zbas1/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 34547 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-lpyp4a6u/'
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 45691 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-_5g9vvo7/'
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Starting up...
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:40553
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:44371
Listening on: 127.0.0.1:32943
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:45283
Listening on: 127.0.0.1:34547
Startup Phase 3 complete. Ready for commands.
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:45755
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:36359
Startup Phase 3 complete. Ready for commands.
Listening on: 127.0.0.1:45691
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:47082 accepted
ReadyHandler: 127.0.0.1:47082 ready
ConnectHandler: Request from 127.0.0.1:40556 accepted
ReadyHandler: 127.0.0.1:40556 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
ConnectHandler: Request from 127.0.0.1:34020 accepted
ReadyHandler: 127.0.0.1:34020 ready
ConnectHandler: Request from 127.0.0.1:59020 accepted
ConnectHandler: Request from 127.0.0.1:57936 accepted
ReadyHandler: 127.0.0.1:59020 ready
ReadyHandler: 127.0.0.1:57936 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
ConnectHandler: Request from 127.0.0.1:51726 accepted
ReadyHandler: 127.0.0.1:51726 ready
ConnectHandler: Request from 127.0.0.1:43422 accepted
ConnectHandler: Request from 127.0.0.1:42742 accepted
ReadyHandler: 127.0.0.1:43422 ready
ReadyHandler: 127.0.0.1:42742 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Requesting to join a single player game
Configuring interface options
Next launch phase started: 2
Configure: raw interface enabled
Next launch phase started: 3
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Requesting to join a single player game
Requesting to join a single player game
Configuring interface options
Configuring interface options
Configure: raw interface enabled
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: score interface disabled
Configure: render interface disabled
Configure: render interface disabled
Launching next game.
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
CloseHandler: 127.0.0.1:59020 disconnected
CloseHandler: 127.0.0.1:43422 disconnected
CloseHandler: 127.0.0.1:47082 disconnected
CloseHandler: 127.0.0.1:40556 disconnected
CloseHandler: 127.0.0.1:51726 disconnected
CloseHandler: 127.0.0.1:57936 disconnected
CloseHandler: 127.0.0.1:34020 disconnected
CloseHandler: 127.0.0.1:42742 disconnected
Version: B75689 (SC2.4.10)
Build: Aug 12 2019 17:16:57
Command Line: '"/home/user/StarCraftII/Versions/Base75689/SC2_x64" -listen 127.0.0.1 -port 33825 -dataDir /home/user/StarCraftII/ -tempDir /tmp/sc-ndkkhjkf/'
Starting up...
Startup Phase 1 complete
Startup Phase 2 complete
Creating stub renderer...
Listening on: 127.0.0.1:33825
Startup Phase 3 complete. Ready for commands.
ConnectHandler: Request from 127.0.0.1:55860 accepted
ReadyHandler: 127.0.0.1:55860 ready
Requesting to join a single player game
Configuring interface options
Configure: raw interface enabled
Configure: feature layer interface disabled
Configure: score interface disabled
Configure: render interface disabled
Launching next game.
Next launch phase started: 2
Next launch phase started: 3
Next launch phase started: 4
Next launch phase started: 5
Next launch phase started: 6
Next launch phase started: 7
Next launch phase started: 8
Game has started.
Using default stable ids, none found at: /home/user/StarCraftII/stableid.json
Successfully loaded stable ids: GameData\stableid.json
Sending ResponseJoinGame
/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py:394: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  eval_episode_team_rewards = np.array(eval_episode_team_rewards)
idv policy and team policy use same initial params!

 Map 2s3z Algo mappotrsyn Exp train_2s3z_sigma_v0_CMT_s2r2_v0 updates 0/1000 episodes, total num timesteps 3200/3200000, FPS 57.

incre win rate is 0.0.
team_policy eval win rate is 0.0.
idv_policy eval win rate is 0.0.
DataHandler: unable to parse websocket frame.
CloseHandler: 127.0.0.1:55860 disconnected
wandb: - 0.006 MB of 0.006 MB uploaded
wandb: \ 0.222 MB of 0.819 MB uploaded
wandb: | 0.819 MB of 0.819 MB uploaded
wandb: 
wandb: Run history:
wandb:                              Aa_idv_actor_loss ▁
wandb:                                 Ab_policy_loss ▁
wandb:                            Ac_idv_ppo_loss_abs ▁
wandb:                                Ad_idv_ppo_prop ▁
wandb:                                         Ae_eta ▁
wandb:                           Af_noclip_proportion ▁
wandb:                           Ag_update_proportion ▁
wandb:                                 Ah_update_loss ▁
wandb:                                Ai_idv_epsilon' ▁
wandb:                                   Aj_idv_sigma ▁
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') ▁
wandb:                       Al_idv_noclip_proportion ▁
wandb:              Am_idv_(sigma*A)update_proportion ▁
wandb:                    An_idv_(sigma*A)update_loss ▁
wandb:                            Ao_idv_entropy_prop ▁
wandb:                                Ap_dist_entropy ▁
wandb:                                 Aq_idv_kl_prop ▁
wandb:                                 Ar_idv_kl_coef ▁
wandb:                                 As_idv_kl_loss ▁
wandb:                           At_idv_cross_entropy ▁
wandb:                                  Au_value_loss ▁
wandb:                                  Av_advantages ▁
wandb:                              Aw_idv_actor_norm ▁
wandb:                             Ax_idv_critic_norm ▁
wandb:                            Ba_idv_org_min_prop ▁
wandb:                            Bb_idv_org_max_prop ▁
wandb:                            Bc_idv_org_org_prop ▁
wandb:                            Bd_idv_new_min_prop ▁
wandb:                            Be_idv_new_max_prop ▁
wandb:                             Ta_team_actor_loss ▁
wandb:                            Tb_team_policy_loss ▁
wandb:                           Tc_team_ppo_loss_abs ▁
wandb:                               Td_team_ppo_prop ▁
wandb:                               Te_team_epsilon^ ▁
wandb:                                 Tf_team_sigma^ ▁
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') ▁
wandb:                      Th_team_noclip_proportion ▁
wandb:            Ti_team_(sigma^*A)update_proportion ▁
wandb:                  Tj_team_(sigma^*A)update_loss ▁
wandb:                           Tk_team_entropy_prop ▁
wandb:                           Tl_team_dist_entropy ▁
wandb:                                Tm_team_kl_prop ▁
wandb:                                Tn_team_kl_coef ▁
wandb:                                To_team_kl_loss ▁
wandb:                          Tp_team_cross_entropy ▁
wandb:                             Tq_team_value_loss ▁
wandb:                             Tr_team_advantages ▁
wandb:                             Ts_team_actor_norm ▁
wandb:                            Tt_team_critic_norm ▁
wandb:                           average_step_rewards ▁
wandb:                      average_step_team_rewards ▁
wandb:                                     dead_ratio ▁
wandb:        idv_policy_eval_average_episode_rewards ▁
wandb:   idv_policy_eval_average_episode_team_rewards ▁
wandb:                       idv_policy_eval_win_rate ▁
wandb:                                 incre_win_rate ▁
wandb:       team_policy_eval_average_episode_rewards ▁
wandb:  team_policy_eval_average_episode_team_rewards ▁
wandb:                      team_policy_eval_win_rate ▁
wandb: 
wandb: Run summary:
wandb:                              Aa_idv_actor_loss -0.00824
wandb:                                 Ab_policy_loss 0.01079
wandb:                            Ac_idv_ppo_loss_abs 0.91362
wandb:                                Ad_idv_ppo_prop 0.9796
wandb:                                         Ae_eta 0.99971
wandb:                           Af_noclip_proportion 0.98882
wandb:                           Ag_update_proportion 0.60064
wandb:                                 Ah_update_loss -0.4292
wandb:                                Ai_idv_epsilon' 1.0
wandb:                                   Aj_idv_sigma 0.99971
wandb:     Ak_idv_clip(sigma, 1-epislon', 1+epislon') 0.99971
wandb:                       Al_idv_noclip_proportion 1.0
wandb:              Am_idv_(sigma*A)update_proportion 0.39783
wandb:                    An_idv_(sigma*A)update_loss -0.22712
wandb:                            Ao_idv_entropy_prop 0.0204
wandb:                                Ap_dist_entropy 1.90232
wandb:                                 Aq_idv_kl_prop 0.0
wandb:                                 Ar_idv_kl_coef 0.0
wandb:                                 As_idv_kl_loss 0.0001
wandb:                           At_idv_cross_entropy 0.0
wandb:                                  Au_value_loss 0.66962
wandb:                                  Av_advantages -0.31773
wandb:                              Aw_idv_actor_norm 0.05713
wandb:                             Ax_idv_critic_norm 1.5999
wandb:                            Ba_idv_org_min_prop 0.51425
wandb:                            Bb_idv_org_max_prop 0.08639
wandb:                            Bc_idv_org_org_prop 0.0
wandb:                            Bd_idv_new_min_prop 0.17225
wandb:                            Be_idv_new_max_prop 0.22558
wandb:                             Ta_team_actor_loss -0.02168
wandb:                            Tb_team_policy_loss -0.004
wandb:                           Tc_team_ppo_loss_abs 0.73023
wandb:                               Td_team_ppo_prop 0.97287
wandb:                               Te_team_epsilon^ 0.05
wandb:                                 Tf_team_sigma^ 0.99998
wandb: Tg_team_clip(sigma^, 1-epislon^', 1+epislon^') 0.9999
wandb:                      Th_team_noclip_proportion 0.98454
wandb:            Ti_team_(sigma^*A)update_proportion 0.9902
wandb:                  Tj_team_(sigma^*A)update_loss 0.06169
wandb:                           Tk_team_entropy_prop 0.02534
wandb:                           Tl_team_dist_entropy 1.90226
wandb:                                Tm_team_kl_prop 0.00179
wandb:                                Tn_team_kl_coef 9.0
wandb:                                To_team_kl_loss 0.00015
wandb:                          Tp_team_cross_entropy 0.0
wandb:                             Tq_team_value_loss 0.27681
wandb:                             Tr_team_advantages 0.07164
wandb:                             Ts_team_actor_norm 0.11138
wandb:                            Tt_team_critic_norm 1.85342
wandb:                           average_step_rewards 0.0784
wandb:                      average_step_team_rewards 0.0
wandb:                                     dead_ratio 0.30948
wandb:        idv_policy_eval_average_episode_rewards 6.77589
wandb:   idv_policy_eval_average_episode_team_rewards 0.0
wandb:                       idv_policy_eval_win_rate 0.0
wandb:                                 incre_win_rate 0.0
wandb:       team_policy_eval_average_episode_rewards 7.56748
wandb:  team_policy_eval_average_episode_team_rewards 0.0
wandb:                      team_policy_eval_win_rate 0.0
wandb: 
wandb: 🚀 View run StarCraft2_2s3z_3 at: https://wandb.ai/804703098/2s3z_sigma_v0/runs/stltib29
wandb: ⭐️ View project at: https://wandb.ai/804703098/2s3z_sigma_v0
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 4 other file(s)
wandb: Find logs at: ./results/StarCraft2/2s3z/mappotrsyn/train_2s3z_sigma_v0_CMT_s2r2_v0/wandb/run-20240802_190126-stltib29/logs
Traceback (most recent call last):
  File "train/train_smac_trsyn.py", line 247, in <module>
    main(sys.argv[1:])
  File "train/train_smac_trsyn.py", line 232, in main
    runner.run()
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/runner/shared/smac_runner_trsyn.py", line 56, in run
    obs, share_obs, rewards, dones, infos, available_actions = self.envs.step(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 106, in step
    self.step_async(actions)
  File "/home/user/zhangyang/PycharmProjects/Nips2024-ITPC-v2/Nips2024-ITPC-v2/onpolicy/envs/env_wrappers.py", line 367, in step_async
    remote.send(('step', action))
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 411, in _send_bytes
    self._send(header + buf)
  File "/home/user/anaconda3/envs/zypy38/lib/python3.8/multiprocessing/connection.py", line 368, in _send
    n = write(self._handle, buf)
BrokenPipeError: [Errno 32] Broken pipe
