Data Discovery and Indexing for Semi-Structured Scientific Data

Kaushik Jagini, Yifan Zhang, Yichen Guo, Julian Goddy, Dale Stansberry, Joshua Agar, Jeff Heflin

Abstract: There is a need for powerful, user-friendly tools for scientific data management and discovery. We present an architecture based on DataFed and Elasticsearch that allows scientists to easily share data they produce and a novel interface that allows other scientists to easily discover data of interest. This interface supports summary-level information about a collection of datasets that can be easily refined using schema-free search. We extend the recent idea of cell-centric search to semi-structured data, describe the architecture of the system, present a use case from the context of materials science, and evaluate the efficacy of the system.