Theoretical analysis and image reconstruction for multi-bit quanta image sensors

Hiu Tung Wong, Chi-Sing Leung, Derek Ho

Abstract: Highlights•This paper formally analyzes the properties of multi-bit quanta image sensor (QIS) systems. It derives the log likelihood function of the received photon counts in a spatiotemporal jot kernel, and introduces the concept of the probability of all jots being saturated.•From the likelihood function result, we can have a maximum likelihood (ML) estimate for the exposure level and present a construction algorithm, namely ML multi-bit (MLM). As our estimate is ML based, the variance of the estimated exposure achieves the Cramér-Rao bound (CRB) asymptotically.•From the Fisher information concept, this paper derives the CRB on the variance of the estimated exposure. This CRB value can be considered as a performance indicator for different hardware settings.•From the jot saturation result, we can accurately formulate the relationship between maximum exposure and the kernel size, as well as i.e., the relationship between dynamic range and the kernel size.•From two analysis results, we can model the relationships between sensor design parameters and performance metrics (variance of the estimated exposure and the dynamic range). Since the analysis results are independent of the construction algorithm used, they give us some guidelines to vary the spatiotemporal jot kernel size and the bit resolution of QIS jots to control dynamic range, and signal to noise ratio of constructed images.•This paper empirically studies the effect of readout Gaussian noise.•With our framework in likelihood function, we can develop some joint-construction-and-denosing algorithms for MBQIS. We use an enhanced version of MLM, namely MLM with denoising (MLMDN), to demonstrate this joint-construction-and-denosing concept.