Competitive Interactive Video Retrieval in Virtual Reality with vitrivr-VR
Authors
Florian Spiess, Ralph Gasser, Silvan Heller, Luca Rossetto, Loris Sauter, Heiko Schuldt
Type
In Proceedings
Date
2021/6
Appears in
Proceedings of the 27th International Conference on Multimedia Modeling (MMM 2021)
Location
Prague, Czech Republic (held virtually)
Abstract
Virtual Reality (VR) has emerged and developed as a new modality to interact with multimedia data. In this paper, we present vitrivr-VR, a prototype of an interactive multimedia retrieval system in VR based on the open source full-stack multimedia retrieval system vitrivr. We have implemented query formulation tailored to VR: Users can use speech-to-text to search collections via text for concepts, OCR and ASR data as well as entire scene descriptions through a video-text co-embedding feature that embeds sentences and video sequences into the same feature space. Result presentation and relevance feedback in vitrivr-VR leverages the capabilities of virtual spaces.
Research Projects