POSTER: ML-Compass: A Comprehensive Assessment Framework for Machine Learning Models

Zhibo Jin, Zhiyu Zhu, Hongsheng Hu, Minhui Xue, Huaming Chen

Abstract: Machine learning models have made significant breakthroughs across various domains. However, it is crucial to assess these models to obtain a complete understanding of their capabilities and limitations and ensure their effectiveness and reliability in solving real-world problems. In this paper, we present a framework, termed ML-Compass, that covers a broad range of machine learning abilities, including utility evaluation, neuron analysis, robustness evaluation, and interpretability examination. We use this framework to assess seven state-of-the-art classification models on four benchmark image datasets. Our results indicate that different models exhibit significant variation, even when trained on the same dataset. This highlights the importance of using the assessment framework to comprehend their behavior.