Collaborative Q-Learning Based Routing Control in Unstructured P2P Networks

Xiangjun Shen, Qing Chang, Jianping Gou, Qirong Mao, Zheng-Jun Zha, Ke Lu

Abstract: Query routing among peers whilst locating required resources is still an acute issue discussed P2P networking, especially in unstructured P2P networks. Such an issue becomes worse when there is frequent in and out movement of the peers in the network and also with node failures. We propose a new method to assure alternative routing path to balance the query loads among the peers under higher network churns. The proposed collaborative Q-learning method learns the networks parameters such as processing capacity, number of connections, and number of resources in the peers, along with their state of congestion. By this technique, peers are avoided to forward queries to the congested peers. Our simulation results show that the required resources are located more quickly and queries in the whole network are also balanced. Also our proposed protocol exhibits more robustness and adaptability under high network churns and heavy workloads than that of the random walk method.