Pixel-wise optical encoder driven by video prediction: towards single-shot high dynamic range moving object recognition

Conventional low dynamic range (LDR) imaging device fails to preserve plenty of information for further vision task because of saturation effect. Thus, high dynamic range (HDR) imaging is an important imaging technology in extreme illuminance conditions, which enables a wide range of applications, including photography, autonomous driving and robotics. The mainstream approaches require multi-shot because the conventional camera can only control the exposure globally. Although performing well on static HDR imaging, they face a challenge on real time HDR imaging for motional scene because of the artifact and time latency caused by multi-shot and post processing. To this end, we propose a framework, termed POE-VP, which achieves single-shot HDR imaging via a pixel-wise optical encoder driven by video prediction. We use highlight motional license plate recognition as a downstream vision task to demonstrate the performance of POE-VP. From the results of simulation and real scene experiments, we validate that POE-VP outperforms conventional LDR camera by more than 5 times in recognition accuracy and by more than 200% in information entropy. The dynamic range could reach 120dB and the captured data size is verified to be lower than mainstream multi-shot methods by 67%. The running time of POE-VP is also validated to satisfy the needs for high speed HDR imaging.