VideoSketcher: Innovative Query Modes for Searching Videos through Sketches, Motion and Sound
This text reports on the results of the work carried on within the VideoSketcher project during the eNTERFACE 2015 summer workshop help in Mons, Belgium.
An additional report in video form can also be found on-line. Information retrieval technologies are key enablers of a range of applications in an environment where finding the right information and data becomes critical in many sectors, for efficient decision-making, research, and creative thinking. Multimedia content deserves a particular treatment given its unstructured (non-symbolic) and hidden meaning to the computer, also known as the semantic gap. This calls for research on the way multimedia documents can be indexed, queried and stored efficiently. Despite the numerous recent research advances, we are far from there yet, as can be seen through the increasing number of challenging benchmarks and competitions related to Multimedia Information Retrieval (MIR). In this context, the study of query modes beyond keywords or text search is of particular interest.
In this paper, we focus on audiovisual databases and develop an integrated prototype enabling to make use of content analysis and recognition through machine learning, multiple query modes (symbolic and non-symbolic enabling to specify the visual as well as audio characteristics of searched video shots), and a scalable database back-end. We evaluate the system in a known-item search task.