Pre-trained Financial Model for Price Movement Forecasting

Chenyou Fan, Tianqi Pang, Aimin Huang

Abstract: We propose the Pre-trained Financial Model (PFM) for price movement forecasting, which is critical in the automated trading systems in the Stock and Futures markets. Inspired by recent successes of pre-trained large language models in tackling NLP tasks, our PFM adopts a pretraining-and-finetuning strategy for obtaining capable models that are adapted to various downstream price-forecasting tasks. During the pre-training stage, we train a sequence prediction backbone with multi-task learning by adopting both a supervised learning objective and an unsupervised regularization target. Our approach differs from the common masked language modeling (MLM) used in NLP studies. We develop a per-step target variable generation strategy for eliciting future predictions from the transformer encoder-decoder architecture. We verify our pre-trained model on various practical downstream forecasting tasks, including lagged movement regression, movement direction classification, and selective trading with best performing stocks. Specifically, during the fine-tuning stage, we retain the pre-trained encoder and replace the decoder with specific downstream task decoders. We then perform supervised task-specific target generation learning as the fine-tuning process. Through extensive numerical studies and analysis, we demonstrate that our fine-tuned financial model can achieve a 5–15% improvement over downstream regression and classification tasks and over 40% in selective trading task.