Vision-Based Relative State Estimation for Non-Cooperative Spacecraft Using Deep Learning and Adaptive Kalman Filtering

Safa Mesut BOSTANCI, Halil Ersin Soken

Keywords: Uncooperative spacecraft pose estimation, motion estimation, Multiplicative Extended Kalman Filter, deep learning, proximity operations

TL;DR: A 13-state Multiplicative Extended Kalman Filter with PnP-derived measurement covariance for robust pose and motion estimation of noncooperative spacecraft from monocular images.

Abstract: Autonomous proximity operations for on-orbit servicing and debris removal require accurate estimation of a target's translational and rotational motion. While deep learning approaches have achieved success in estimating spacecraft pose (position and attitude) from monocular imagery, they typically lack the velocity and angular rate estimates essential for trajectory prediction and control. This paper presents a pose-to-motion estimation framework that fuses pose measurements from a keypoint-based Convolutional Neural Network (CNN) pipeline with a Multiplicative Extended Kalman Filter (MEKF) to recover full six-degree-of-freedom (6-DoF) motion states. The measurement covariance is derived directly from Perspective-n-Point (PnP) reprojection geometry, providing adaptive uncertainty quantification. We evaluate the framework on trajectory sequence from the SPEED-UE-Cube dataset, demonstrating that the MEKF recovers velocity with RMSE of 0.002\,m/s and angular rate with RMSE of 0.27°/s from noisy pose-only measurements. The filter simultaneously improves pose accuracy, reducing position RMSE from 6.96\,m to 0.27\,m and attitude RMSE from 34.4° to 7.8°. The results show that temporal filtering with geometry-derived measurement covariance enables reliable motion estimation from pose-only measurements, bridging the gap between single-frame pose estimation and the dynamic state information required for proximity operations.

Submission Number: 17