Image Authenticity Detection using Eye Gazing Data: A Performance Comparison Beyond Human Capabilities via Attention Mechanism, ResNet, and Cascade Strategies

Ping Zhang

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Image Manipulation Detection, Cascade Networks, Eye-tracking Data, Model Stability

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Optimization of a classifier based on human eye tracking to determine whether an image has been manipulated.

Abstract:

In the digital age, determining the authenticity of images has become increasingly crucial. This study aims to explore the capability of machine learning models in identifying manipulated images using eye movement data and compares this with human judgment. We collected a series of both manipulated and unaltered images and conducted eye-tracking experiments on a set of participants. After data preprocessing, various machine learning models were trained and validated, including a simple classifier, cascade-optimized classifier, and models integrating attention mechanisms with ResNet architectures. Results indicate that all models outperformed the baseline set by human judgment. Specifically, the Attention-ResNet model achieved the highest accuracy at 0.685, making it the top-performing model. Our analysis delves further into the stability, generalization capabilities, and practical value of these models. Ultimately, this research underscores the immense potential of deep learning strategies in verifying image authenticity, providing valuable insights for future research and applications.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9489