Performant LLM Agentic Framework for Conversational AI

Alex Casella, Wayne Wang

Keywords: agentic, agentic ai, conversational ai, machine learning, workflow navigation, ai automation, agent, performant, latency

TL;DR: Improving the performance of navigating Agentic workflows in Conversational AI, via a combination of logical decision making and vector scoring to improve alignment and reduce latency.

Abstract:

The rise of Agentic applications and automation in the Voice AI industry has led to an increased reliance on Large Language Models (LLMs) to navigate graph-based logic workflows composed of nodes and edges. However, existing methods face challenges such as alignment errors in complex workflows and hallucinations caused by excessive context size. To address these limitations, we introduce the Performant Agentic Framework (PAF), a novel system that assists LLMs in selecting appropriate nodes and executing actions in order when traversing complex graphs. PAF combines LLM-based reasoning with a mathematically grounded vector scoring mechanism, achieving both higher accuracy and reduced latency. Our approach dynamically balances strict adherence to predefined paths with flexible node jumps to handle various user inputs efficiently. Experiments demonstrate that PAF significantly outperforms baseline methods, paving the way for scalable, real-time Conversational AI systems in complex business environments.

Submission Number: 2