Using an Information Theoretic Metric for Compressive Recovery under Poisson Noise

Ajit Rajwade

Abstract:

Recovery error bounds in compressed sensing under Gaussian or uniform bounded noise do not translate
easily to the case of Poisson noise. Reasons for this include the signal dependent nature of Poisson noise, and
also the fact that the negative log likelihood in case of a Poisson distribution (which is directly related to the
generalized Kullback-Leibler divergence) is not a metric and does not obey the triangle inequality. There
exist prior theoretical results in the form of provable error bounds for computationally tractable estimators
for compressed sensing problems under Poisson noise. However, these results do not apply to realistic
compressive systems, which must obey some crucial constraints such as non-negativity and flux preservation.
On the other hand, there exist provable error bounds for such realistic systems in the published literature,
but they are for estimators that are computationally intractable. In this paper, we develop error bounds
for a computationally tractable estimator which also applies to realistic compressive systems obeying the
required constraints. The focus of our technique is on the replacement of the generalized Kullback-Leibler
divergence, with an information theoretic metric - namely the square root of the Jensen-Shannon divergence,
which is related to an approximate, symmetrized version of the Poisson log likelihood function. We show
that our method allows for very simple proofs of the error bounds. We also propose and prove several
interesting statistical properties of the square root of Jensen-Shannon divergence, a well-known informationtheoretic metric, and exploit other known ones. Numerical experiments are performed showing the practical
use of the technique in signal and image reconstruction from compressed measurements under Poisson noise.
Our technique has the following features: (i) It is applicable to signals that are sparse or compressible in
any orthonormal basis. (ii) It works with high probability for any randomly generated sensing matrix that
obeys the non-negativity and flux preservation constraints, and is derived from a ‘base matrix’ that obeys the restricted isometry property. (iii) Most importantly, our proposed estimator uses parameters that are
purely statistically motivated and signal independent, as opposed to techniques (such as those based on
the Poisson negative log-likelihood or `2 data-fidelity) that require the choice of a regularization or signal
sparsity parameter which are unknown in practice.