Spatiotemporal Lifelog Analytics in Virtual Reality with vitrivr-VR

Authors
Florian Spiess, Heiko Schuldt, Luca Rossetto
Type
Article
Date
2024/6
Appears in
Abstract

Modern wearables and smart devices make it easier than ever to collect a detailed, digital record of biometric as well as visual and aural information. Reasons to collect such a lifelog range from health applications to vacation documentation. With the large quantities of data that can be collected in very short periods of time, it remains a challenging problem to find specific events and answer questions based on such lifelogs. In order to support lifelog multimedia analytics within collections of large sizes, interactive analytics methods must be developed that take advantage of the diverse and multimodal data available.

Through rapid technological improvement, immersive interfaces, such as virtual reality (VR) devices, are quickly becoming more affordable and accessible to the general public. Due to their immersive capabilities, these interfaces are uniquely suited to visualizing and allowing interaction with diverse multimodal data. Even so, research on interactive lifelog analytics has been heavily focused on conventional desktop interfaces with pointer controls, and little research has been conducted on interfaces for virtual reality. While many interfaces developed for conventional interfaces can also be used within VR, the advantages of immersive interfaces can only be utilized by methods tailored to this new interface modality.

In this paper, we describe the vitrivr-VR virtual reality multimedia analytics system in the form in which it will participate in the Lifelog Search Challenge (LSC) 2024. In order to take greater advantage of the variety of available lifelog data and the affordances of immersive interfaces, we implement new search interfaces that allow easier and more flexible temporal and spatial query formulation as well as spatiotemporally contextualized results visualization in VR.

Research Projects