Open Peer Review. Open Publishing. Open Access. Open Discussion. Open Directory. Open Recommendations. Open API. Open Source.
Who Said What: Modeling individual labelers improves classification
Melody Y. Guan, Varun Gulshan, Andrew M. Dai, Geoffrey E. Hinton
Feb 17, 2017 (modified: Mar 18, 2017)ICLR 2017 workshop submissionreaders: everyone
Abstract:Data are often labeled by many different experts, with each expert labeling a small fraction of the data and each sample receiving multiple labels. When experts disagree, the standard approaches are to treat the majority opinion as the truth or to model the truth as a distribution, but these do not make any use of potentially valuable information about which expert produced which label. We propose modeling the experts individually and then learning averaging weights for combining them, possibly in sample-specific ways. This allows us to give more weight to more reliable experts and take advantage of the unique strengths of individual experts at classifying certain types of data. We show that our approach performs better than three competing methods in computer-aided diagnosis of diabetic retinopathy.
Keywords:Computer vision, Deep learning, Supervised Learning
Enter your feedback below and we'll get back to you as soon as possible.