A Study on Brain Decoding of Image Stimuli Using a Diffusion Model

Fumi Ishizaki, Ichiro Kobayashi

Abstract: Humans recognize the external world by processing information received by the eyes and other sensory organs in the brain. Understanding how the human brain processes complex information from the outside world is expected to improve the performance of image and speech recognition technologies, which have made remarkable progress in recent years. In this study, we focus on brain activity decoding of visual experience, aim to read what humans are looking at by predicting image features from brain activity data, and further attempt to develop a method to output high-definition and semantically valid images by generating images from predicted features using a diffusion model. As a result, similarity to the stimulus image was confirmed in the image generated by the training data, but the evaluation data confirmed that there is still room for further study.