SimFormer: Real-to-Sim Transfer with Recurrent Restoration

Yingnan Ma, Fan Yang, Xudong Li, Chen Jiang, Anup Basu

Abstract: Real-to-Sim transfer is a popular research topic in robotics. Utilizing a simulated environment, the development processes can achieve lower costs and make the testing process easier. In addition, after the Real-to-Sim transfer, the simulated environment can lower the texture effect and light effect, which can be further applied to other computer vision tasks, such as robot grasping. Differing from artistic style transfer, Real-to-Sim transfer has higher accuracy requirements for content preservation. In this paper, we utilize the transformer to solve the Real-to-Sim transfer. We creatively design the restoration stage to preserve the content information. We also propose the restoration loss function. After these improvements, our architecture can achieve better performance on light removal, content preservation, and feature embedding.