Debugging Big Data Analytics in Spark with BigDebug

Muhammad Ali Gulzar, Matteo Interlandi, Tyson Condie, Miryung Kim

Abstract: To process massive quantities of data, developers leverage Data-Intensive Scalable Computing (DISC) systems such as Apache Spark. In terms of debugging, DISC systems support only post-mortem log analysis and do not provide any debugging functionality. This demonstration paper showcases BigDebug: a tool enhancing Apache Spark with a set of interactive debugging features that can help users in debug their Big Data Applications.