Adapting fisher vectors for histopathology image classification

Yang Song, Ju Jia Zou, Hang Chang, Weidong Cai

Abstract: Histopathology image classification can provide automated support towards cancer diagnosis. In this paper, we present a transfer learning-based approach for histopathology image classification. We first represent the image feature by Fisher Vector (FV) encoding of local features that are extracted using the Convolutional Neural Network (CNN) model pretrained on ImageNet. Next, to better transfer the pretrained model to the histopathology image dataset, we design a new adaptation layer to further transform the FV descriptors for higher discriminative power and classification accuracy. We used the publicly available BreaKHis image dataset for classifying between benign and malignant breast tumors, and obtained improved performance over the state-of-the-art.