Conformal Prompting for Largescale Multiclass Food Risk Classification

Anonymous

Abstract: Contaminated or adulterated food poses a substantial risk to human health. Given sets of labeled web texts for training, Machine Learning and Natural Language Processing can be applied to automatically extract pointers towards such risks in order to generate early warnings. We publish a dataset of 7,619 short texts describing food recalls. Each text is manually labeled, on two granularity levels (coarse and fine), for food products and hazards that the recall corresponds to. We describe the dataset, also presenting baseline scores of naive, traditional, and transformer models. We show that Support Vector Machines based on a Bag of Words representation outperform RoBERTa and XLM-R on classes with low support. We also apply in-context learning with PaLM, leveraging Conformal Prediction to improve it by reducing the number of classes used to select the few-shots. We call this method Conformal Prompting.

Paper Type: long

Research Area: Machine Learning for NLP

Contribution Types: NLP engineering experiment, Data resources, Data analysis

Languages Studied: English, German, French, Danish, Greek, Italian

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.