Scalable design of Error-Correcting Output Codesusing Discrete Optimization with Graph Coloring

Samarth Gupta, Saurabh Amin

Abstract: We study the problem of scalable design of Error-Correcting Output Codes (ECOC)for multi-class classification. Prior works on ECOC-based classifiers are limited tocodebooks with small number of rows (classes) or columns, and do not provide op-timality guarantees for the codebook design problem. We address these limitationsby developing a codebook design approach based on a Mixed-Integer QuadraticallyConstrained Program (MIQCP). This discrete formulation is naturally suited formaximizing the error-correction capability of ECOC-based classifiers and incorpo-rates various design criteria in a flexible manner. Our solution approach is tractablein that it incrementally increases the codebook size by adding columns to maximizethe gain in error-correcting capability. In particular, we show that the maximalgain in error-correction can be upper bounded by solving a graph-coloring problem.As a result, we can efficiently generate near-optimal codebooks for very largeproblem instances. These codebooks provide competitive multi-class classificationperformance on small class datasets such as MNIST and CIFAR10. Moreover,by leveraging transfer-learned binary classifiers, we achieve better classificationperformance over transfer-learned multi-class CNNs on large class datasets such asCIFAR100, Caltech-101/256. Our results highlight the advantages of simple andmodular ECOC-based classifiers in improving classification accuracy without therisk of overfitting.