% Fully automatic Deep Learning (DL) has become the start-of-the-art technique in many medical imaging-related tasks to extract clinically useful information for computer-aided diagnosis, treatment planning, and prevention. 
% %
% However, it is challenging to develop such software that performs at the level of an experienced radiologist. 
% %
% This is due to the requirement of large-scale and high quality labeled datasets for training a DL system to archived a human level of accuracy. 
Deep learning in medical image analysis often requires an extensive amount of high-quality labeled data for training to achieve human-level accuracy. 
%
We propose Gist-set Online Active Learning (GOAL), a novel solution for limited high-quality labeled data in medical imaging analysis.
%
Our approach advances the existing active learning methods in three aspects. 
%
Firstly, we improve the classification performance with fewer manual annotations by presenting a sample selection strategy called gist set selection. 
%
Secondly, unlike traditional methods focusing only on random uncertain samples of low prediction confidence, we propose a new method in which only informative uncertain samples are selected for human annotation. 
%
Thirdly, we propose an application of online learning where high-confidence samples are automatically selected, iteratively assigned, and pseudo-labels are updated.
%
We validated our approach on two private and one public datasets. The experimental results show that, by applying GOAL, we can reduce required labeled data up to 88\% while maintaining the same F1 scores compared to the models trained on full datasets. 