British Machine Vision Conference (BMVC) 2017

Image Completion with Intrinsic Reflectance Guidance Image Completion with Intrinsic Reflectance Guidance

by Soomin Kim , Taeyoung Kim, Min H. Kim, and Sung-Eui Yoon

Korea Advanced Institute of Science and Technology (KAIST)

This figure shows the basic intrinsic image property we utilize for image completion. Image (a) shows an input image, where the blanket is removed (masked as red). Image (b) presents correspondences of source patches using color and Laplacian properties. Image (c) visualizes our correspondence search using intrinsic reflectance. Note that our method brings source patches from virtually similar materials.


Our method inpaints the input image by separating intrinsic reflectance and shading so that our method can complete the image with intrinsic scene materials, rather than shadowed color values. Other state-of-the-art methods inpaint the blank region using color values directly, resulting in unrealistic artifacts.


Abstract

Patch-based image completion methods often fail in searching patch correspondences of similar materials due to shading caused by scene illumination, resulting in inappropriate image completion with dissimilar materials. We therefore present a novel image completion method that additionally accounts for intrinsic reflectance of scene objects, when searching patch correspondences. Our method examines both intrinsic reflectances and color image structures to avoid false correspondences of different materials so that our method can search and vote illumination-invariant patches robustly, allowing for image completion mainly with homogeneous materials. Our results validate that our reflectance-guided inpainting can produce more natural and consistent images than state-of-the-art inpainting methods even under various illumination conditions.

Contents

Paper (author preprint)
Supplementary material, BMVC poster
Source code: Github page, ZIP file (Intrinsic decomposition code is not included, see the readme file for further information)
BMVC 2017

Bibtex: 
@InProceedings{Inpainting:BMVC:2017,
  author  = {Soomin Kim and Taeyoung Kim and Min H. Kim and Sung-Eui Yoon},
  title   = {Image Completion with Intrinsic Reflectance Guidance},
  booktitle = {Proc. British Machine Vision Conference (BMVC 2017)},
  address = {London, England},
  year = {2017},
  pages = {},
  volume  = {},
}