XRECO: XR mEdia eCOsystem (Ongoing)
While media organisations increasingly support non-linear experiences for the consumer, those are still limited to single channels and media domains. Although several media organisations have recently succeeded in breaking data silos, data sharing is mostly limited to the organisation. Thus, there are challenges for producing content feeding multiple channels with different granularities and structures, mainly related to data discovery, management and (re-)use. XRECO will create a new data-driven ecosystem for the media industry, focusing on facilitating data sharing, search and discovery and supporting creation of news and entertainment content, in particular, the creation and (re-)use of location-related 2D and 3D assets and the creation of XR experiences. The ecosystem core, represented by a Neural Media Repository (NMR), will foster inter-organisation content sharing and provide increased access to content for media creators, considering novel data monetization and rights management policies. A set of AI-based media transformation services are built around the NMR to produce novel media- and XR experiences, including 3D neural reconstructions, neural based device localisation, image stitching, de-/re-lighting and holoportation. The developed technology will be validated in use case scenarios for (i) the news media for XR-based broadcasting and automatic and customized multitarget news publishing, and for (ii) location-based information and entertainment content, with applications in tourism and the automotive industry.
Website
Since
01.09.2022
Partners
- Deutsche Welle, Germany
- Fundació Privada i2CAT, Internet i Innovació Digital a Catalunya, Spain
- JOANNEUM RESEARCH Forschungsgesellschaft mbH, Austria
- Centre for Research and Technology Hellas, Greece
- Estudio Bay, S.L.U., Spain
- MOG Technologies, SA, Portugal
- Sound Holding BV, Netherland
- Universidad Politécnica de Madrid, Spain
- Fincons SPA, Italy
- University of Basel, Switzerland
- ATOS Spain SA, Spain
- ZAUBAR UG (haftungsbeschränkt), Germany
- Virtual Reality Berlin-Brandenburg e.V., Germany
- Rai - Radiotelevisione Italiana S.p.A., Italy
- NVIDIA Mellanox Technologies, Israel
- Katholieke Universiteit te Leuven, Belgium
- F.F.P. Film- & Fernsehproduktion GmbH, Austria
- Unity Technologies ApS, Denmark
- Continental Automotive Romania SrL, Romania
- THEODOROS CHIOU & PARTNERS INTELLECTUAL PROPERTY & INTERNET LAW OFFICE | IPRights, Greece
Funding Agencies
EU - Horizon Europe (TOPIC ID: HORIZON-CL4-2021-HUMAN-01-06)
SERI - Swiss State Secretariat for Education, Research and Innovation
Funding
542'932.00 CHF
Staff
Research Topics
Publications
2024
- Loris Sauter, Ralph Gasser, Laura Rettig, Heiko Schuldt, Luca Rossetto
General Purpose Multimedia Retrieval with vitrivr at LSC’24
LSC ‘24: Proceedings of the 7th Annual ACM Workshop on the Lifelog Search Challenge, Phuket, Thailand 2024/6 - Ralph Gasser, Rahel Arnold, Fynn Faber, Heiko Schuldt, Raphael Waltenspül, Luca Rossetto
A new Retrieval Engine for vitrivr
MMM 2024 - The 30th International Conference on Multimedia Modeling, Amsterdam, The Netherlands 2024/1 - Rahel Arnold, Heiko Schuldt
Multimedia Retrieval in Mixed Reality: Leveraging Live Queries for Immersive Experiences
2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR), Los Angeles, United Sates 2024/1
2023
- Werner Bailer, Rahel Arnold, Vera Benz, Davide Coccomini, Anastasios Gkagkas, Gylfi Þór Guðmundsson, Silvan Heller, Björn Þór Jónsson, Jakub Lokoc, Nicola Messina, Nick Pantelidis, Jiaxin Wu
Improving Query and Assessment Quality in Text-Based Interactive Video Retrieval Evaluation
Proceedings of the International Conference on Multimedia Retrieval (ICMR ‘23), 2023/6