Deep Transfer Learning for NLP on Small Data Sets

Hanoz Bhathena

Abstract:

Nvidia GTC 2019 invited speaker talk:

Massive quantities of domain specific labeled data have been the fuel, but also the primary bottleneck for using deep learning algorithms in industry. Organizations which lack the budget, time or have data privacy issues face hurdles in collecting such large amounts of domain specific human annotated data. I will review and compare methods to tackle this problem for text classification tasks via transfer learning using deep learning models. The models discussed will include Universal Sentence Encoders, ELMo and BERT. I will describe the model architectures used, specifics of training mechanisms, the evaluation criteria that guided the experiments and finally provide an attribution analysis of which components contributed most to end performance results.