AmI HMAS: Hybrid Agents with Individual and Collective Experience-Aware Code-Based Planning for Smart Environments

AAMAS 2026 Workshop EMAS Submission39 Authors

Keywords: Smart Environments, Hypermedia MAS, LLM Agents, Behavior Tree Plans, Structured Goal Representation, Individual and Collective Experience Reuse

TL;DR: AmI HMAS uses LLM-powered agents and reusable code-based plans to enable flexible, goal-driven interactions with smart environments.

Abstract:

Ad-hoc, goal-driven interactions in smart environments (homes, offices, hotels) have been a long lasting objective in Ambient Intelligence (AmI). Advances in Large Language Models (LLM) for reasoning and use of hypermedia environments for multi-agent systems are bringing this objective closer to achievement.

We describe the functionality and implementation of AmI HMAS, a framework for agent-based, goal-driven, LLM-supported inter-
actions with smart environments. AmI HMAS maps existing HomeAssistant deployments into semantically represented, navigable Hypermedia Environments, enabling discovery of real-world smart devices. The framework combines classic agency with LLM reasoning to perform environment exploration, request interpretation, community-based exchange of experience, and action planning.

AmI HMAS leverages an engine that enables storage and reuse of past interaction experiences during reasoning, distinguishing between environment state requests, explicit commands and implicit / ambiguous requests. The planning approach is designed to produce BehaviorTree code-based procedural plans, that enable plan life cycle management and reuse. Plan components can be exchanged in a community of agents that manage different smart environments, leveraging the power of the community to improve solving requests.

We evaluate the system quantitatively across two distinct setups (simulated homes in the HomeBench benchmark and cross-environment transfer in a smart research lab simulation), measuring planning success rates, signifier fast-path hit rates, LLM call reduction, and planning latency across different request types (explicit, ambiguous, single or multi-command, achievable or impossible) and experience reuse settings.

Paper Type: Regular paper

Demo: Yes, we would love to present a demo.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 39