Document Rectification and Illumination Correction using a Patch-based CNN

Xiaoyu Li 1           Bo Zhang 1, 2           Jing Liao 3           Pedro V. Sander 1          

1 The Hong Kong University of Science and Technology           2 Microsoft Research Asia           3 City University of Hong Kong

Fig. 1. Geometric and illumination correction. The top row shows the input images and the bottom row shows the results of our approach.


We propose a novel learning method to rectify document images with various distortion types from a single input image. As opposed to previous learning-based methods, our approach seeks to first learn the distortion flow on input image patches rather than the entire image. We then present a robust technique to stitch the patch results into the rectified document by processing in the gradient domain. Furthermore, we propose a second network to correct the uneven illumination, further improving the readability and OCR accuracy. Due to the less complex distortion present on the smaller image patches, our patch-based approach followed by stitching and illumination correction can significantly improve the overall accuracy in both the synthetic and real datasets.


Paper [arXiv] Supp [PDF] Code [Github Repository]


    author = {Li, Xiaoyu and Zhang, Bo and Liao, Jing and Sander, Pedro V.},
    title = {Document Rectification and Illumination Correction using a Patch-based CNN},
    journal={ACM Transactions on Graphics (TOG)},
    month = {11},
    year = {2019}


We thank the anonymous reviewers for valuable feedback on our manuscript. This work was partly supported by Hong Kong ECS grant # 21209119, Hong Kong UGC, and HKUST DAG06/07.EG07 grants.