Exploratory Analysis of Speedup Learning Data Using Epectation Maximization

Alberto Maria Segre, Geoffrey J. Gordon, Charles Elkan

Abstract: Experimental evaluations of speedup learning methods have in the past used non-parametric hypothesis testing to determine whether or not learning is beneficial. We show here how to obtain deeper insight into the comparative performance of learning methods through a complementary parametric approach to data analysis. In this approach experimental data is used to estimate values for the parameters of a statistical model of the performance of a problem solver. To model problem solvers that use speedup learning methods, we propose a two-component linear model that captures how learned knowledge may accelerate the solution of some problems while leaving the solution of others relatively unchanged. We show how to apply expectation maximization (EM), a statistical technique, to fit this kind of multi-component model. EM allows us to fit the model in the presence of censored data, a methodological difficulty common to experiments involving speedup learning.