A Zero-Positive Learning Approach for Diagnosing Software Performance Regressions

Mejbah Alam, Justin Gottschlich, Nesime Tatbul, Javier S Turek, Timothy Mattson, Abdullah Muzahid

Abstract: A performance regression is a degradation of software performance due to a change in code. A common way to identify such regressions is to manually create tests for them. This is known as regression testing. Although widely used, regression testing can be slow and error-prone due to its historically manual nature. Moreover, such tests’ utility is usually constrained by the expertise of the developer creating them. Due to these limitations, researchers have turned their attention to the automatic generation of regression tests. In this paper, we present AutoPerf – a novel approach to automate regression testing that utilizes three core techniques: (i) zero-positive learning, (ii) autoencoders, and (iii) hardware telemetry. We demonstrate AutoPerf’s generality and efficacy against 3 types of performance regressions across 10 real performance bugs in 7 benchmark and open-source programs. On average, AutoPerf exhibits 4% profiling overhead and accurately diagnoses more performance bugs than prior state-of-the-art approaches.

Code Link: https://github.com/mejbah/AutoPerf

CMT Num: 6217