Multi-Agent Deep Reinforcement Learning Based Pricing Strategy for Competing Cloud Platforms in the Evolutionary Market

Bing Shi, Rongjian Shi, Bingzhen Li

Abstract: In the cloud market, there exist multiple cloud providers competing against each other in order to attract cloud users and make profits. Each provider needs an appropriate pricing strategy. In this paper, we analyze how a cloud provider sets the price effectively when competing against other cloud providers. The price charged by the cloud provider is affected by many factors such as the prices of its opponents, the price set in the previous round, the choice of cloud users and the marginal cost of the cloud provider. Therefore, we model this problem as a Partially Observable Markov Game where cloud providers compete against each other in the evolutionary market. In more detail, in this game, the amount of cloud users is increasing before it eventually becomes saturated and the marginal value of users is also changing. Then we use a gradient-based multi-agent reinforcement learning algorithm to generate the pricing strategy for the cloud service provider. Finally, we evaluate our pricing strategy against other typical pricing strategies by conducting extensive experiments. The experimental results show that our pricing strategy can not only adapt the price according to the opponents' prices, but also adapt the price according to the changes of the cloud users' valuations on the cloud resource, and therefore it can outperform other pricing strategies. Furthermore, we also find that when training the pricing strategy against the pricing strategy generated by our algorithm, our pricing strategy can still win in terms of the long-term profits. The experimental results can provide useful insights for designing practical pricing strategies.