A Natural Way of Building Financial Domain Expert Agents

Gagandeep Singh Kaler, Dimitrios Vamvourellis, Deran Onay, Yury Krongauz, Stefano Pasquali, Dhagash Mehta

Keywords: GenAI, Large Language Model, Memory, Finance, Natural agent, Zero-Shot, Financial domain expert

TL;DR: A natural way of creating financial domain experts using a zero-shot approach. The memory layer introduced in our framework provides capabilities complimentary to few-shot prompting enabling a more natural way of building domain expert LLM agents.

Abstract:

Financial experts possess specialized knowledge that is not easily accessible, making the acquisition of such expertise a time-intensive process.
Leveraging Large Language Models (LLMs) to emulate financial domain experts offers a promising solution, which can offload routine responsibilities from human experts, allowing them to focus on more strategic tasks.
However, developing a GenAI agent that matches the capabilities of a financial domain expert requires more than just LLMs with Retrieval-Augmented Generation (RAG) capabilities.
The agent must interact with domain-specific data sources, perform complex analyses, and understand niche terminologies and processes.
We propose a natural way of developing of GenAI-powered financial domain experts by following a zero-shot approach.
Our agent's memory layer has complimentary capabilities to few-shot prompting and provides a natural way of remembering information as it interacts with domain experts.

This paper is presented as a case study where we propose a comprehensive framework for building financial domain expert agents.
Our approach involves iteratively enhancing a basic LLM with data extraction layer, coding capabilities, and a memory layer to perform complex analyses.
We show how addition of each layer to our LLM agent improves its performance and also address the necessary safety and governance processes to ensure the robustness and accuracy of production ready agent.
We also introduce a custom dataset (having roots in the financial domain) for evaluating the agent’s performance in numerical analysis and multi-step reasoning, providing a clearer picture of the agent's capability to mimic a financial domain expert.

Submission Number: 9