Content Authentication for Neural Imaging Pipelines: End-to-end Optimization of Photo Provenance in Complex Distribution Channels

Abstract

Forensic analysis of digital photo provenance relies on intrinsic traces left in the photograph at the time of its acquisition. Such analysis becomes unreliable after heavy post-processing, such as down-sampling and re-compression applied upon distribution in the Web. This paper explores end-to-end optimization of the entire image acquisition and distribution workflow to facilitate reliable forensic analysis at the end of the distribution channel. We demonstrate that neural imaging pipelines can be trained to replace the internals of digital cameras, and jointly optimized for high-fidelity photo development and reliable provenance analysis. In our experiments, the proposed approach increased image manipulation detection accuracy from 45% to over 90%. The findings encourage further research towards building more reliable imaging pipelines with explicit provenance-guaranteeing properties.

Comment: Camera ready + supplement, CVPR'19

Original document

The different versions of the original document can be found in:

• http://arxiv.org/abs/1812.01516

• http://arxiv.org/pdf/1812.01516

• http://xplorestaging.ieee.org/ielx7/8938205/8953184/08953353.pdf?arnumber=8953353,

http://dx.doi.org/10.1109/cvpr.2019.00882

• https://dblp.uni-trier.de/db/conf/cvpr/cvpr2019.html#KorusM19,

https://nyuscholars.nyu.edu/en/publications/content-authentication-for-neural-imaging-pipelines-end-to-end-op,

https://openaccess.thecvf.com/content_CVPR_2019/html/Korus_Content_Authentication_for_Neural_Imaging_Pipelines_End-To-End_Optimization_of_Photo_CVPR_2019_paper.html,

https://ui.adsabs.harvard.edu/abs/2018arXiv181201516K/abstract,

https://academic.microsoft.com/#/detail/2982544354