Uncertainty-based Decision Making Using Deep Reinforcement Learning

Xujiang Zhao, Shu Hu, Jin-Hee Cho, Feng Chen

Abstract: This work proposes an opinion inference algorithm in large graph network data using subjective, uncertain opinions. In the graph network data, an opinion is associated with an edge between two nodes where the edge indicates a known opinion while no edge refers to an unknown opinion for their relationship. The examples include the predictions of a road traffic condition (i.e., an edge indicates a road between two intersections and an opinion represents congested or non-congested) or trust relationships (i.e., an edge refers to a trust relationship between two users where an opinion indicates a user's trust in another user). To derive an unknown opinion between two nodes, we identify a set of best paths in the graph network data that can maximize decision performance (e.g., prediction accuracy). To solve this problem, we formulate each opinion using Subjective Logic (SL) and leverage a policy-based deep reinforcement learning (DRL) technique. We propose three DRL-based schemes combining SL and DRL where a reward is given based on a different type of uncertainty, including vacuity, dissonance, or monosonance. Via extensive simulation experiments, we investigate what type of uncertainty is a more critical factor than others in improving decision performance when a different uncertainty type is considered as a reward in DRL. We validated the outperformance of the proposed DRL-based schemes in terms of belief errors, prediction accuracy, and computation time based on both a semi-synthetic and real world datasets.