Explore Outworld Knowledge in Large Language Models: A Case Study in Pokemon Game

Hongqiu Wu, Xingyuan Liu, hai zhao, Min Zhang

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: large language model, outworld knowledge, self-supervised

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: Large language models (LLMs) show great power by gathering almost all knowledge in our human world. An appealing curiosity now arises regarding their adaption to a new world setting, e.g from fictions and films, one with disparate fundamental laws, which is much more challenging than transferring between domains of the same human world. This carries significant research potential for expanding AI to multiple universes in the future. This paper chooses \textsc{Pokémon} as the target, a popular strategy game with a unique worldview. We introduce \textsc{Pokemon-Py}, a Python library that provides an interactive playground as in the pokemon world. Our analysis demonstrates that the outworld context can exacerbate knowledge distortions and logical flaws in today's LLMs, and this phenomenon has a significant negative impact. Based on \textsc{Pokemon-Py}, we propose \textit{Self-Training with Self-Competition}, a novel self-supervised learning method to effectively adapt the model to a new or even unknown world setting, where the model is programmed to keep learning through self-competition, and ultimately grows into a superior individual. Our method achieves remarkable improvement to adapt LLaMA2-7b to two downstream tasks within the pokemon world.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2297