Computational end‑to‑end and super‑resolution methods to improve thermal infrared remote sensing for agriculture

Iftach Klapp, Peretz Yafin, Navot Oz, Omri Brand, Idan Bahat, Eitan Goldshtein, Yafit Cohen, Victor Alchanatis, Nir Sochen

Abstract:

Increasing global water deficit and demand for yield improvement call for high-resolution
monitoring of irrigation, crop water stress, and crops’ general condition. To provide high
spatial resolution with high-temperature accuracy, remote sensing is conducted at low
altitudes using radiometric longwave thermal infrared cameras. However, the radiometric
cameras’ price, and the low altitude leading to low coverage in a given time, limit the use
of radiometric aerial surveys for agricultural needs. This paper presents progress toward
solving both limitations using algorithmic and computational imaging methods: stabiliz-
ing the readout of low-cost thermal cameras to obtain radiometric data, and improving the
latter’s low resolution by applying convolutional neural network-based super-resolution.
The two methods were merged by an end-to-end algorithm pipeline, providing a large
mosaicked image of the field. First, the potential capabilities of a joint estimation method
to correct unknown offset and gain were simulated on remotely sensed agricultural data.
Comparison to ground-truth measurements showed radiometric accuracy with a root mean
square error (RMSE) of 1.3 °C to 1.8 °C. Then, the proposed super-resolution method was
demonstrated on experimental and simulated remotely sensed agricultural data. Prelimi-
nary experimental results showed 50% improvement in image sharpness relative to bicubic
interpolation. The performance of the algorithm was evaluated on 22 simulated cases at
× 2 and × 4 magnification. Finally, image mosaicking using the proposed pipeline was
demonstrated. A mosaicked image composed of sub-images pre-processed by the pro-
posed computational methods resulted in a RMSE in temperature of 0.8 °C, as compared to
8.2 °C without the initial processing.