Interactive Search or Sequential Browsing? A Detailed Analysis of the VBS 2018
This work summarizes the findings of the seventh iteration of the Video Browser Showdown (VBS) competition organized as a workshop at the 24th International Conference on Multimedia Modeling in Bangkok. The competition focuses on video retrieval scenarios in which the searched scenes were either previously observed or described by another person (i.e., an example shot is not available). During the event, nine teams competed with their video retrieval tools in providing access to a shared video collection with 600 hours of video content. Evaluation objectives, rules, scoring, tasks and all the participating tools are described in the paper. In addition, we provide some insights into how the different teams interacted with their video browsers, which was made possible by a novel interaction logging mechanism introduced for this iteration of VBS. The results collected at the Video Browser Showdown evaluation server confirm that searching for one particular scene in the collection given a limited time is still a challenging task for many of the approaches that were showcased during the event. Given only a short textual description, finding the correct scene is even harder. In ad-hoc search with multiple relevant scenes, the tools were mostly able to find at least one scene, while recall was the issue for many teams. The logs also reveal that, even though recent exciting advances in machine learning narrow the classical semantic gap problem, user centric interfaces are still required to mediate access to specific content. Finally, open challenges and lessons learned are presented for future VBS events.