requirement :
(following all can be installed by pip)

python3.9
pytorch1.12
mujoco
mujoco-py
opencv-python
gym 0.20.0
tensorboard
rltorch
numpy
glfw



gymnasium
gymnasium['box2d']
gym_minigrid


miniworld


PyQt5
win32gui (pip install pypiwin32)



### the following can be installed by git or downloading installation package

babyai:  https://github.com/mila-iqia/babyai, 
torcs: if you use ubuntu, you can build it by https://github.com/ugo-nama-kun/gym_torcs

attention:
if you use window like us, download package from https://sourceforge.net/projects/torcs/,  with a patch https://sourceforge.net/projects/cig/files/SCR Championship/.
unzip the patch to the source path of torcs. Then install gym-torcs. Capture the pixel input by PyQt5 from screenshot. 



#######################

all the code can be run with a NVIDIA GeForce GTX 3090, requiring less then 10 GB memory.

######################

attention:

If you use win10, do not install mujoco-py by 'pip install mujoco-py', use 'pip install mujoco-py==2.0.2.8' instead.

'mjkey.txt' for old version of mujoco < 210

#######################


To start:

If you want to start to train an agent of our method,  run 'python our_main2.py --env_name env_name' for maze navigation task.
env_name can be 'Maze_square_random', 'Maze_square_blocked' for training task, 'Maze_general_1' for generalization test

run 'python our_main2.py --env_name env_name' for object interaction task
env_name can be 'Maze_square_random', 'Maze_square_blocked2' for training task, 'Maze_square_blocked_general' for generalization test


If you want to train agent in car-racing task or test generalization in babyai task, please cover the source file with the ones we provided in './main_code/envs'

This version of code contains the main part of our works and can be used. We will provide the rest including isolated complete environments after collating.

#############################
Debug:

As our method utilizes the package of 'multiprocessing', if you run without bugs but with no effect,
please check the contents  in the instance of Queue, especially the tensor of 'cuda'.


If cuda locks the network when running with multiprocessing,  try to put the network in the main process after the ones in subprocesses.


If in torcs you find the received observations do not change, try to clean up the buffer of socket before you receiving observation.



