HealthBot: An Open-Source AI Assistant for Longitudinal Personal Health Management

Zheng Yu, Yanyuan Qiao, Qi Wu, Yutong Xie

Keywords: Multimodal Foundation Models, Large Language Models, Biomedical Multimodal Data, Health Agent Reasoning, Personalized Healthcare Applications

Abstract: Medical LLMs and VLMs perform well on clinical QA and report generation, but remain largely stateless and cannot manage personal health data over time. Existing medical agents mainly target specific benchmarks or workflows, while commercial systems such as ChatGPT Health are closed-source and cloud-dependent. We present HealthBot, an open-source personal health assistant that runs locally, stores data on-device, supports messaging-platform interfaces, and works with model-agnostic LLM backends. HealthBot integrates multimodal extraction of heterogeneous health inputs into normalized records, Hierarchical Health Context (H-Context) for budget-aware longitudinal reasoning, and tool-augmented reasoning grounded in the local archive for report interpretation, trend analysis, consultation preparation, and medication tracking. We evaluate longitudinal context on a diagnosis prediction benchmark, where models infer the current diagnosis from laboratory and vital measurements under three settings: no history, raw history, and H-Context. Across two LLM backbones, history substantially improves accuracy, and H-Context further improves over raw history while reducing context length by 39\% (up to +20~pp over no-history), demonstrating the value of structured longitudinal context for personal health agents.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 114