Image Authenticity Detection using Eye Gazing Data: A Performance Comparison Beyond Human Capabilities via Attention Mechanism, ResNet, and Cascade Strategies

Ping Zhang

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Image Manipulation Detection, Cascade Networks, Eye-tracking Data, Model Stability

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: Optimization of a classifier based on human eye tracking to determine whether an image has been manipulated.

Abstract:

In the digital age, determining the authenticity of images has become
increasingly crucial. This study aims to explore the capability of machine
learning models in identifying manipulated images using eye movement data and
compares this with human judgment. We collected a series of both manipulated
and unaltered images and conducted eye-tracking experiments on a set of
participants. After data preprocessing, various machine learning models were
trained and validated, including a simple classifier, cascade-optimized classifier,
and models integrating attention mechanisms with ResNet architectures. Results
indicate that all models outperformed the baseline set by human judgment.
Specifically, the Attention-ResNet model achieved the highest accuracy at 0.685,
making it the top-performing model. Our analysis delves further into the stability,
generalization capabilities, and practical value of these models. Ultimately, this
research underscores the immense potential of deep learning strategies in
verifying image authenticity, providing valuable insights for future research and
applications.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 9489