Online model selection by learning how compositional kernels evolve
Abstract: Motivated by the need for efficient, personalized learning in health, we investigate the problem of online compositional kernel selection for multi-task Gaussian Process regression. Existing composition selection methods do not satisfy our strict criteria in health; selection must occur quickly, and the selected kernels must maintain the appropriate level of complexity, sparsity, and stability as data arrives online. We introduce the Kernel Evolution Model (KEM), a generative process on how to evolve kernel compositions in a way that manages the bias--variance trade-off as we observe more data about a user. Using pilot data, we learn a set of kernel evolutions that can be used to quickly select kernels for new test users. KEM reliably selects high-performing kernels for a range of synthetic and real data sets, including two health data sets.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: We thank the reviewers and action editors for the constructive feedback regarding our paper.
We included an experiment to explore the scalability of our approach during pilot training in section 7.3.
In summary, we found that runtime in pilot training scales better to more users (scales linearly) than to more timesteps (scales exponentially). This implies that our method is best suited for pilot data that has either a high number of users and limited time steps, or a larger number of batches per time step with fewer total time steps. The number of features (and size of the base kernel set) did not affect the amount of time it took per training iteration in our method; however, we expect that pilot data with a larger number of features will require more iterations overall.
Finally, we addressed each reviewer's suggestions during the discussion period and amended the final manuscript as noted in each of the individual responses.
Code: https://github.com/dtak/kernel-evolutions-public
Assigned Action Editor: ~Benjamin_Guedj1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 1187
Loading