TartanDrive 1.5: Improving Large Multimodal Robotics Dataset Collection and Distribution

Matthew Sivaprakasam, Samuel Triest, Mateo Guaman Castro, Micah Nye, Mukhtar Maulimov, Cherie Ho, Parv Maheshwari, Wenshan Wang, Sebastian Scherer

Keywords: Dataset, pretraining, multimodal

TL;DR: We present our goals for the next generation of TartanDrive dataset, including more modalities, better data pipelines, and a framework for others to process their own data through our pipelines.

Abstract:

The complexity of the real world presents numerous challenges in robotics that must be overcome, such as handling complex physical interactions, learning novel tasks, and planning in unknown environments. Recently, large data-driven algorithms and deep learning models have been adopted and modified to solve these problems, but with the rise of these approaches there also arises a need for large amounts of diverse robotics data to train them on.
In this work we discuss the improvements to our previous dataset, TartanDrive, that we are currently working on to fulfill these needs in the context of off-road driving. We address the challenges of copious data collection in order to provide an expansive dataset containing several modalities collected in an outdoor area with approximately 225 acres of diverse terrain. Moreover, we will provide scripts capable of re-configuring this data (such as by filtering by location or formatting to fit specific use-cases/conventions) and release a framework that will allow others to not only use our data but collect their own in a way that enables them to use our scripts. By leveraging this dataset, we hope to facilitate the advancement of robotics and reduce the barrier to entry that is often associated with data at this scale.