Enhancing Images with Coupled Low-Resolution and Ultra-Dark Degradations: A Tri-level Learning Framework

Jiaxin Gao, Yaohua Liu

Abstract: Due to device constraints and lighting conditions, captured images frequently exhibit coupled low-resolution and ultra-dark degradations. Enhancing the visibility and resolution of ultra-dark images simultaneously is crucial for practical applications. Current approaches often address both tasks in isolation or through simplistic cascading strategies, while also relying heavily on empirical and manually designed composite loss constraints, which inevitably results in compromised training efficacy, increased artifacts, and diminished detail fidelity. To address these issues, we propose TriCo, the first to adopt a Tri-level learning framework that explicitly formulates the bidirectional Cooperative relationship and devises algorithms to tackle coupled degradation factors. In the optimization across Upper (U)-Middle (M)-Lower (L) levels, we model the synergistic dependencies between illumination learning and super-resolution tasks within the M-L levels. Moving to the U-M levels, we introduce hyper-variables to automate the learning of beneficial constraints for both learning tasks, moving beyond the traditional trial-and-error pitfalls of the learning process. Algorithmically, we establish a Phased Gradient-Response (PGR) algorithm as our training mechanism, which facilitates a dynamic, inter-variable gradient feedback and ensures efficient and rapid convergence. Moreover, we present the Integrated Hybrid Expert Modulator (IHEM), which merges inherent illumination priors with universal semantic model features to adaptively guide pixel-level high-frequency detail recovery. Extensive experimentation validates the framework's broad generalizability across challenging ultra-dark scenarios, outperforming current state-of-the-art methods across 4 real and synthetic benchmark datasets over 8 metrics (e.g., 5.8\%$\uparrow$ in PSNR, 26.6\%$\uparrow$ in LPIPS, and 13.9\%$\uparrow$ in RMSE).

Primary Subject Area: [Content] Media Interpretation

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference: This research makes a pivotal contribution to the multimedia domain by unveiling TriCo, a groundbreaking framework that holistically addresses the intertwined challenges of amplifying, illuminating, and detailing images under extreme dark conditions. TriCo's novel tri-level constraint optimization approach not only marks a departure from conventional disjointed methodologies but also harmonizes the enhancement process, ensuring seamless integration of brightness adjustment and super-resolution. The introduction of an integrated hybrid expert modulator, which adeptly bridges intrinsic illumination cues with semantic features, paves the way for unprecedented detail preservation and artifact mitigation. Moreover, the phased gradient response algorithm underscores the efficiency of TriCo, optimizing multiple variables in concert to expedite convergence without compromising on quality. By setting new benchmarks across diverse datasets and metrics, TriCo not only enhances the visual fidelity of multimedia content but also strengthens the foundation for subsequent advanced multimedia analyses in challenging lighting scenarios, thereby broadening the horizon for multimedia applications.

Submission Number: 5588