Mentor-Mind: Risk-Aware, Constraint-Grounded Advice Agents Beyond Chain-of-Thought

Yun Wing Kiang

Keywords: Large Language Models, Chain-of-Thought Prompting, Influence Diagrams, Risk-Aware Decision Making, Conditional Value-at-Risk (CVaR), Constraint-Grounded Reasoning, Advice Agents

Abstract: Large language models (LLMs) have shown remarkable reasoning abilities through prompting techniques like chain-of-thought (CoT) prompting and self-consistency decoding, achieving state-of-the-art results on complex tasks. However, these methods rely on the model’s generated rationales, which can be unreliable – often hallucinating plausible-sounding but unfaithful content – and do not account for risk or hard constraints in decision making. We propose Mentor-Mind, an influence-diagram–grounded advice agent that marries LLM reasoning with decision-theoretic planning. Mentor-Mind uses domain- and mentor-specific decision graphs (influence diagrams) as structured scaffolds for reasoning, ensuring that recommendations satisfy hard domain constraints and optimize a risk-sensitive objective (e.g. Conditional Value-at-Risk). In synthetic yet complex advisory scenarios (energy facility siting, code review, early-career planning), Mentor-Mind generates advice that is more aligned, faithful, and risk-aware than baseline prompting methods. Experimental results show that our approach maintains high decision quality under uncertainty while strictly respecting constraints, outperforming CoT and self-consistency prompts in both success rate and adherence to safety constraints. This work demonstrates a practical integration of LLMs with symbolic decision frameworks, yielding advice agents that replace the “make-up” CoT reasoning with grounded, trustworthy decision analysis.

Submission Number: 291