Learning Monotonic Probabilities with a Generative Cost Model

Yongxiang Tang, Yanhua cheng, Xiaocheng Liu, Jiaochenchen, Yanxiang Zeng, Ning Luo, Pengjia Yuan, Xialong Liu, Peng Jiang

Abstract: In many machine learning tasks, it is often necessary for the relationship between input and output variables to be monotonic, including both strictly monotonic and implicitly monotonic relationships. Traditional methods for maintaining monotonicity mainly rely on construction or regularization techniques, whereas this paper shows that the issue of strict monotonic probability can be viewed as a partial order between an observable revenue variable and a latent cost variable. This perspective enables us to reformulate the monotonicity challenge into modeling the latent cost variable. To tackle this, we introduce a generative network for the latent cost variable, termed the Generative Cost Model (**GCM**), which inherently addresses the strict monotonic problem, and propose the Implicit Generative Cost Model (**IGCM**) to address the implicit monotonic problem. We further validate our approach with a numerical simulation of quantile regression and conduct multiple experiments on public datasets, showing that our method significantly outperforms existing monotonic modeling techniques. The code for our experiments can be found at [https://github.com/tyxaaron/GCM](https://github.com/tyxaaron/GCM).

Lay Summary:

In many machine learning tasks, ensuring a monotonic relationship between input and output variables is crucial. Traditional methods often struggle with maintaining this monotonicity effectively.

To address this challenge, we reformulated the problem by viewing it as a relationship between an observable revenue variable and a latent cost variable. This led us to develop a new approach focusing on modeling the latent cost variable. We introduced the Generative Cost Model (GCM) to handle strict monotonic relationships and the Implicit Generative Cost Model (IGCM) for more implicit monotonic relationships. Our models inherently address the monotonicity issue without relying on traditional construction or regularization techniques.

Our research is significant because it offers a novel probabilistic perspective and a generative solution to the monotonic problem in machine learning, making it a valuable contribution to the field.

Link To Code: https://github.com/tyxaaron/GCM

Primary Area: Deep Learning->Generative Models and Autoencoders

Keywords: monotonic model, generative model

Submission Number: 9162