Material Segmentation Using 1-D Convolutional Neural Network With Transient Histogram

Yohanssen Pratama, Kazuya Kitano, Takahiro Kushida, Yuki Fujimura, Takuya Funatomi, Yasuhiro Mukaigawa

Abstract: This study introduces a method for material classification using transient histograms obtained via a single-photon avalanche diode (SPAD) sensor. Temporal resolution in optical sensing plays a crucial role in material classification and surface segmentation, particularly for distinguishing materials with similar visual properties. In this study, SPAD sensors were utilized to capture transient histograms with temporal resolutions ranging from 13 picoseconds to 208 picoseconds, enabling precise extraction of temporal signatures for various materials. A comparative evaluation of classification techniques, including one-dimensional convolutional neural networks (1-D CNN), random forest (RF), support vector classifier (SVC), and k-nearest neighbors (KNN), was conducted to assess the impact of temporal resolution and exposure time on classification accuracy. 1-D CNN achieved the highest classification accuracy of 99.25% at a temporal resolution of 13 ps and an exposure time of 0.09 s, significantly outperforming other methods. Additionally, the proposed SPAD-based system was evaluated for material segmentation on non-planar surfaces. In a real-world experiment, 1-D CNN achieved an overall accuracy of 87.5% in differentiating visually similar materials, demonstrating the effectiveness of transient histograms for material classification where conventional RGB-based methods fail. These findings highlight the potential of SPAD sensors combined with advanced classification techniques to enhance material classification and segmentation, providing a versatile framework for applications in robotics, computer vision, and optical sensing.