Deep Camera Obscura: An Image Restoration Pipeline for Pinhole Photography

Lensless cameras using thin optics such as amplitude or phase masks, diffraction gratings, or diffusion layers have recently been proposed as a compelling alternative to normal lens-based cameras for mobile photography and tiny robotics which are size and cost constrained. However, these multiplexing cameras typically have limited 2D image quality at low resolution sizes (~1 MP) in reconstruction due to severe optical blurring. In contrast, a pinhole camera trades off less light and higher noise due to the aperture in exchange for slightly better optical blurring. Yet there has been minimal research on enhancing pinhole camera images for everyday photography applications. In this paper, we develop an image restoration pipeline of the pinhole system to enhance the pinhole image quality through joint denoising and deblurring. Our pipeline integrates optics-based filtering and reblur losses for reconstructing high resolution still images (2600x1952) as well as temporal consistency for video reconstruction to enable practical exposure times (30 FPS) for high resolution video (1920x1080). We demonstrate comparably higher 2D image quality of 1-2 dB in PSNR than two state-of-the-art lensless cameras FlatCam and DiffuserCam while keeping size, cost, and hardware complexity low. This work opens up the potential of pinhole cameras to be used for photography in size-limited devices such as smartphones in the future.