Hindsight Preference Optimization for Financial Time Series Advisory

Yanwei CUI, Guanghui Wang, Xing Zhang, Peiyang He, Ziyuan Li, Bing Zhu, Qucy Wei Qiu, Xusheng WANG, Zheng Yu, Anqi Xin

Presentation Attendance: No, we cannot present in-person

Keywords: Time Series Forecasting, Preference Optimization, Vision-Language Models Direct Preference Optimization (DPO), LLM-as-Judge

Abstract: Time series models predict numbers; decision-makers need advisory---directional signals with reasoning, actionable suggestions, and risk management. Training language models for such predictive advisory faces a fundamental challenge: quality depends on outcomes unknown at prediction time. We bridge two ideas from reinforcement learning---using information unavailable during execution to retrospectively generate training signal, and preference alignment---and propose Hindsight Preference Optimization: observed outcomes let an LLM judge rank candidate advisories on dimensions that scalar metrics cannot capture, producing preference pairs for DPO without human annotation. We apply this to Vision-Language-Model-based predictive advisories on S\&P 500 equity time series, demonstrated by a 4B model outperforming its 235B teacher on both accuracy and advisory quality.

Track: Research Track (max 4 pages)

Submission Number: 19