SecNet: Semantic Eye Completion in Implicit FieldDownload PDF

Published: 20 Oct 2022, Last Modified: 05 May 2023Gaze Meets ML 2022 OralReaders: Everyone
Keywords: Eye completion, Implicit Field, Semantic Completion
TL;DR: We propose an end-to-end approach to train a neural network, called \emph{SecNet} (semantic eye completion network), that predicts a point cloud with an accurate eye-geometry coupled with the semantic labels of each point.
Abstract: If we take a depth image of an eye, noise artifacts and holes significantly affect the depth values on the eye due to the specularity of the sclera. This paper aims at solving this problem through semantic shape completion. We propose an end-to-end approach to train a neural network, called \emph{SecNet} (semantic eye completion network), that predicts a point cloud with an accurate eye-geometry coupled with the semantic labels of each point. These labels correspond to the essential eye-regions, \ie pupil, iris and sclera. Particularly, our work performs implicit estimation of the query points with semantic labels where both the semantic and occupancy predictions are trained in an end-to-end way. To evaluate the approach, we then use the synthetic eye-scans rendered in UnityEyes simulator environment. Compared to the state of the art, the proposed method improves the accuracy for shape-completion for 3D eye-scan by 8.2\%. In practice, we also demonstrate the application of our semantic eye completion for gaze estimation.
Submission Type: Full Paper
Travel Award - Academic Status: Ph.D. Student
Travel Award - Institution And Country: Technical University of Munich, Germany
Travel Award - Low To Lower-middle Income Countries: No, my institution does not qualify.
Camera Ready Latexfile: zip
4 Replies

Loading