MediaMix: Multimedia Retrieval in Mixed Reality
Extended reality (XR), which encompasses the concepts of augmented (AR), mixed (MR), and virtual reality (VR), enables unique user interactions. Focusing on mixed reality is essential to enabling interaction between the virtual and the real world. One application in which MR can have a significant impact is multimedia retrieval. Moving away from desktop user interfaces towards MR creates new, immersive, and more natural ways for users to interact with the retrieval engine, which can enhance the user experience. In an MR world, there are many new ways of creating queries and interacting with results, which are much more intuitive and user-friendly than in a traditional 2D setting. By introducing MediaMix, a multimedia retrieval system in mixed reality, we introduce a concept and system for user interactions regarding query generation and result interaction in an MR environment. MediaMix will participate in the Video Browser Showdown (VBS) 2025 for the first time. The system utilises the newest MR hardware, the Apple Vision Pro, to facilitate easy user input for query generation and investigates new result presentation and interaction utilising eye tracking and gesture recognition. For the retrieval backend, vitrivr-engine, connected to Postgres, is employed to provide advanced retrieval functionality.