A Hybrid Framework for Matching Printing Design Files to Product Photos

Abstract

We propose a real-time image matching framework, which is hybrid in the sense that it uses both hand - crafted features and deep features obtained from a well -tuned deep convolutional network. The matching problem, which we concentrate on, is specific to a certain application, that is, printing design to product photo matching. Printing designs are any kind of template image files, created using a design tool, thus are perfect image signals. For this purpose, we create an image set that includes printing design and corresponding product photo pairs with collaboration of an actual printing facility. Using this image set, we benchmark various hand-crafted (SIFT, SURF, GIST, HoG) and deep features for matching performance. Various segmentation algorithms including deep learning based segmentation methods are applied to select feature regions. Results show that SIFT features selected from deep segmented regions achieves up to 96% product photo to design file matching success in our dataset. We propose a framework in which deep learning is utilized with highest contribution, but without disabling real-time operation using an ordinary desktop computer.