My Climate Advisor: An Application of NLP in Climate Adaptation for Agriculture

Vincent Nguyen, Sarvnaz Karimi, Willow Hallgren, Ashley Harkin, Mahesh Prakash

Keywords: LLM, NLP, Climate Adaptation, Agriculture, Question Answering, Generation, Evaluation

TL;DR: We create a question answering system for climate adaptation in agriculture and propose a framework for evaluating such a system.

Abstract: Climate adaptation in the agricultural sector necessitates tools that equip farmers and farm advisors with relevant and trustworthy information to help increase their resiliency to climate change. We introduce {\em My Climate Advisor}, a question-answering (QA) prototype that synthesises information from different data sources, such as peer-reviewed scientific literature and high-quality, industry-relevant grey literature to generate answers, with references, to a given user's question. Our prototype uses open-source generative models for data privacy and intellectual property protection, and retrieval augmented generation for answer generation, grounding and provenance. While there are standard evaluation metrics for QA systems, no existing evaluation framework suits our LLM-based QA application in the climate adaptation domain. We design an evaluation framework with seven metrics based on the requirements of the domain experts to judge the generated answers from 12 different LLM-based models. Our initial evaluations through a user study via domain experts show promising usability results.

Archival Submission: arxival

Arxival Submission: arxival

Submission Number: 6