SCRAM: Simple Checks for Realtime Analysis of Model Training for Non-Expert ML Programmers

Eldon Schoop, Forrest Huang, Björn Hartmann

Abstract: Many non-expert Machine Learning users wish to apply powerful deep learning models to their own domains but encounter hurdles in the opaque model tuning process. We introduce SCRAM, a tool which uses heuristics to detect potential error conditions in model output and suggests actionable steps and best practices to help such users tune their models. Inspired by metaphors from software engineering, SCRAM extends high-level deep learning development tools to interpret model metrics during training and produce human-readable error messages. We validate SCRAM through three author-created example scenarios with image and text datasets, and by collecting informal feedback from ML researchers with teaching experience. We finally reflect upon our feedback for the design of future ML debugging tools.