SecNet: Semantic Eye Completion in Implicit Field

Yida Wang, Yiru Shen, David Joseph Tan, Federico Tombari, Sachin S. Talathi

Keywords: Eye completion, Implicit Field, Semantic Completion

TL;DR: We propose an end-to-end approach to train a neural network, called \emph{SecNet} (semantic eye completion network), that predicts a point cloud with an accurate eye-geometry coupled with the semantic labels of each point.

Abstract:

If we take a depth image of an eye, noise artifacts and holes significantly affect the depth values on the eye due to the specularity of the sclera.
This paper aims at solving this problem through semantic shape completion.
We propose an end-to-end approach to train a neural network, called \emph{SecNet} (semantic eye completion network), that predicts a point cloud with an accurate eye-geometry coupled with the semantic labels of each point. These labels correspond to the essential eye-regions, \ie pupil, iris and sclera.
Particularly, our work performs implicit estimation of the query points with semantic labels where both the semantic and occupancy predictions are trained in an end-to-end way.
To evaluate the approach, we then use the synthetic eye-scans rendered in UnityEyes simulator environment.
Compared to the state of the art, the proposed method improves the accuracy for shape-completion for 3D eye-scan by 8.2\%.
In practice, we also demonstrate the application of our semantic eye completion for gaze estimation.

Submission Type: Full Paper

Travel Award - Academic Status: Ph.D. Student

Travel Award - Institution And Country: Technical University of Munich, Germany

Travel Award - Low To Lower-middle Income Countries: No, my institution does not qualify.