Sequential Dataset for Satellite Pose Estimation and a Frequency-Space Neural Operator for HIL-Free Generalization Benchmarking

Woojin Cho, Junghwan Park, Steve Andreas Immanuel, JunminPark, Seokhyun Chin, Jiayun Wang

Keywords: Sim2Real, Space robotics

Abstract:

Accurate six-degree-of-freedom (6-DoF) pose estimation of non-cooperative satellites is critical for the success of on-orbit servicing, assembly, and manufacturing missions. However, the development of deep learning models is severely hampered by a significant scarcity of labeled, real-world on-orbit imagery, which creates a substantial sim-to-real domain gap. While existing datasets are valuable, they lack the large-scale, diverse sequential data necessary to develop and rigorously validate tracking, filtering, and test-time adaptation algorithms under realistic orbital dynamics.
To address this sequential data gap, we introduce a new large-scale, public benchmark: ASTRA-HST, the Hubble Space Telescope (HST) sequential dataset. It consists of 512 unique and physically plausible rendezvous trajectory sequences of the geometrically complex HST. The dataset was generated via a high-fidelity simulator with a wide range of parameters, including orbital dynamics, illumination conditions, and camera properties, providing a new resource for the research community.
To tackle the fundamental sim-to-real problem, we reframe domain adaptation as a function space mapping problem. We propose the frequency-space neural operator (FRESCO), a novel architecture that learns to translate synthetic images to the real domain by operating on distinct frequency bands of the Fourier amplitude spectrum while preserving the phase, which encodes geometric structure. We benchmark several state-of-the-art methods on ASTRA-HST and demonstrate how FRESCO, trained on existing hardware-in-the-loop (HIL) data, can be used to generate a realistic testbed for quantifying the sim-to-real gap on our new dataset.

Submission Number: 12