Towards Constraint-aware Learning for Resource Allocation in NFV Networks

Tianfu Wang, Long yang, Chao Wang, Chuan Qin, Liwei Deng, Li Shen, Hui Xiong

Keywords: Resource Allocation, Network Function Virtualization, Reinforcement Learning, Graph Network Network

TL;DR: We study a crucial network resource allocation problem in network function virtualization, known as Virtual Network Embedding (VNE), and propose a CONstraint-Aware Learning framework to enhance constraint management, named CONAL.

Abstract:

Virtual Network Embedding (VNE) is a fundamental resource allocation challenge that is associated with hard and multifaceted constraints in network function virtualization (NFV). Existing works for VNE struggle to handle such complex constraints, leading to compromised system performance and stability. In this paper, we propose a \textbf{CON}straint-\textbf{A}ware \textbf{L}earning framework, named \textbf{CONAL}, for efficient constraint handling in VNE. Concretely, we formulate the VNE problem as a constrained Markov decision process with violation tolerance, enabling precise assessments of both solution quality and constraint violations. To achieve the persistent zero violation to guarantee solutions' feasibility, we propose a reachability-guided optimization with an adaptive reachability budget method. This method also stabilizes policy optimization by appropriately handling scenarios with no feasible solutions. Furthermore, we propose a constraint-aware graph representation method to efficiently learn cross-graph relations and constrained path connectivity in VNE. Finally, extensive experimental results demonstrate the superiority of our proposed method over state-of-the-art baselines. Our code is available at https://github.com/GeminiLight/conal-vne.

Submission Number: 30