Smoothing Deep Reinforcement Learning for Power Control for Spectrum Sharing in Cognitive Radios

Wanli Wang

Keywords: deep reinforcement learning, deep Q-network, spectrum sharing, Q-value

TL;DR: A novel smoothing deep reinforcement learning considers kernel and weighting methods for smoothing network outputs.

Abstract:

The spectrum sharing in a cognitive radio system is related with a secondary user
sharing common spectrum with a primary user for power transmit without induc-
ing harmful inference. The deep reinforcement learning has been considered as
an intelligent power control method via an agent continuously interacting with en-
vironment. Traditional deep Q-network in the frame work of deep reinforcement
learning utilizes a deep neural network for learning a nonlinear function which
maps the state or observation to accumulated rewards conditional on current state
and agent action also called Q-value. The state or observation in the radio sys-
tem is collected from wireless network and corrupted by noises. The deep neural
network may therefore yield undesirable result due to the presence of noises and
induced degraded network parameters. Considering that the kernel-based adaptive
filter is beneficial for adaptive filtering, we aim to apply the kernel-based adaptive
filter into traditional deep Q-network for smoothing network outputs. In addi-
tion, a weighting approach on the basis of past Q-values also works together with
the deep neural network for further network output smoothing. The weighting
approach is especially beneficial for alleviating the over-smoothing issue of the
kernel-based adaptive filter. Simulation results have shown the efficiency of the
proposed smoothing deep Q-network in spectrum sharing in cognitive radios in
comparison with traditional deep Q-network.