Data Management for Dynamic Multimedia Analytics and Retrieval (PhD Thesis, finished)

Multimedia data in its various manifestations poses a unique challenge from a data storage and data management perspective, especially if search, analysis and analytics in large data corpora is considered. The inherently unstructured nature of the data itself and the curse of dimensionality that afflicts the representations we typically work with in its stead are cause for a broad range of issues that require sophisticated solutions at different levels. This has given rise a huge corpus of research that puts focus on techniques that allow for effective and efficient multimedia retrieval. Many of these contributions have led to an array of purpose-built, multimedia search systems.

However, recent progress in multimedia analytics and interactive multimedia retrieval have demonstrated, that several of the assumptions usually taken for multimedia retrieval workloads do not hold once a session has a human user in the loop. Firstly, many of the required query operations cannot be expressed by mere similarity search and since the concrete requirement cannot alway be anticipated, one needs a flexibel and adaptible data management and query framework. Secondly, the widespread notion of staticity of data collections does not hold if one considers analytics workloads, whose purpose it is to produce and store new data and insights. And finally, it is impossible even for an expert user to specify exactly how a data management system should produce and arrive at the desired outcomes of the potentially many different queries.

Guided by these shortcomings and motivated by the fact that similar questions have once been answered for structured data in classical database research, this Thesis presents three contributions that seek to mitigate the aforementioned issues. We present a query model that generalises the notion of proximity based query operations and formalises the connection between those queries and high-dimensional indexing. We complement this by a cost-model that makes the often implicit trade-off between query execution speed and results quality transparent to the system and the user. And we describe a model for the transactional and durable maintenance of high-dimensional index structures.

All contributions are implemented in the open-source multimedia database system Cottontail DB, on top of which we present an evaluation that demonstrates the effectiveness of the proposed models. We conclude by discussing avenues for future research in the quest of converging the fields of databases on the one hand and (interactive) multimedia retrieval and analytics on the other.

 

More Information

• Edoc: Thesis
• GitHub: Thesis repository
• GitHub: Cottontail DB repository