Insect Cyborgs: Bio-mimetic Feature Generators Improve ML Accuracy on Limited Data

Charles B. Delahunt, J. Nathan Kutz

TL;DR: Features auto-generated by the bio-mimetic MothNet model significantly improve the test accuracy of standard ML methods on vectorized MNIST. The MothNet-generated features also outperform standard feature generators.

Keywords: feature selection, bio-mimesis, neural networks, insect olfaction, sparsity

Abstract:

We seek to auto-generate stronger input features for ML methods faced with limited training data.
Biological neural nets (BNNs) excel at fast learning, implying that they extract highly informative features. In particular, the insect olfactory network learns new odors very rapidly, by means of three key elements: A competitive inhibition layer; randomized, sparse connectivity into a high-dimensional sparse plastic layer; and Hebbian updates of synaptic weights. In this work we deploy MothNet, a computational model of the moth olfactory network, as an automatic feature generator. Attached as a front-end pre-processor, MothNet's readout neurons provide new features, derived from the original features, for use by standard ML classifiers. These ``insect cyborgs'' (part BNN and part ML method) have significantly better performance than baseline ML methods alone on vectorized MNIST and Omniglot data sets, reducing test set error averages 20% to 55%. The MothNet feature generator also substantially out-performs other feature generating methods including PCA, PLS, and NNs. These results highlight the potential value of BNN-inspired feature generators in the ML context.