QbS - Searching for Known Images using User-Drawn Sketches

Michael Springmann, Dietmar Kopp, Heiko Schuldt

In Proceedings

2010/3

Proceedings of the 11th ACM SIGMM International Conference on Multimedia Information Retrieval (MIR

Philadelphia, Pennsylvania

Sheridan Printing, Alpha, New Jersey

With the increasingly growing size of digital image collections, known image search is gaining more and more importance. Especially when the objects in such collections do not possess appropriate metadata (e.g., tags, annotations), content-based image retrieval (CBIR) is a promising approach. However, the application of CBIR to known item search usually suffers from the unavailability of query images that are good enough to express the user's information need. In order to improve this situation, we propose the QbS system which provides an approach to content-based search in large image collections based on user-drawn sketches. By exploiting novel devices for human-computer interaction like interactive paper, tablet PCs, or graphic tablets, users are able to draw a sketch that reflects their information need and start a content-based search using this sketch. The QbS system provides query support and offers several invariances that allow the user-generated sketch to slightly deviate from the searched image in terms of rotation, translation, relative size, and/or unknown objects in the background. To illustrate the benefits of the approach, we show search results from the evaluation of QbS on the basis of the MIRFLICKR collection with 25'000 objects.