Finny: A Multi-Agent System for Structured Decision-Making with LLMs

Harshitha Ravindra, Utkarsh Bajaj, Madhur Mehta

Track: tiny / short paper (up to 4 pages)

Keywords: Large Language Models, Multi-Agent Systems, Retrieval-Augmented Generation, Explainable AI, Logical Reasoning, Forecast Adjustment, Domain Knowledge Integration

TL;DR: Finny is a multi-agent system that demonstrates how large language models can perform structured decision-making by applying domain-specific rules to multiple related scenarios.

Abstract:

Finny is a multi-agent system that demonstrates how large language models can perform structured decision-making by applying domain-specific rules to multiple related scenarios. Leveraging foundation models with Retrieval-Augmented Generation (RAG), the system applies Standard Operating Procedures (SOPs) for intelligent forecast refinement at scale. Finny employs a two-stage architecture: a knowledge base agent that retrieves and applies domain rules while analyzing historical patterns, and a conversational agent enabling interactive refinement. In user acceptance testing (UAT), the system achieved 97.6% alignment with expert judgment across 124 evaluations (31.5% complete, 66.1% partial), with quantitative validation showing 5.89% mean deviation and 0.993 correlation against human decisions across 1,280 data points. This production-deployed system reduces manual analysis time by 70%, translating to 2,400 annual hours savings in
the piloted teams.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Funding: Yes, the presenting author of this submission falls under ICLR’s funding aims, and funding would significantly impact their ability to attend the workshop in person.

Submission Number: 88