High-Velocity Data Management in the PolyDBMS Architecture (Bachelor Thesis, Ongoing)
Author
Description
High-velocity, dynamic data has become a critical asset across various domains, ranging from industrial sensor networks and system status monitoring to real-time financial telemetry. In these fields, massive volumes of fast-moving data must be processed within strict time constraints. This requires addressing inherent challenges such as information irregularity (incompleteness, duplicates, or bursty loads) and a chronic lack of format standardization.
With recent technological advancements, classical streaming approaches, once reserved for specialized real-time systems,have become increasingly relevant for traditional data management in historically static environments. Today, the boundary between static and dynamic data management is blurring, and technologies are being adapted to provide tools for handling dynamic data within classically static infrastructures.
While many modern systems allow for the rapid processing of dynamic data, they often aim to remove the need for persistence entirely. However, in many industrial and research use cases, long-term persistence is mandatory. Current solutions generally fall into two polarized ends of a spectrum:
- Streaming Databases: These systems provide high performance but typically enforce a single, rigid data model. This forces heterogeneous incoming data to be "pressed" into a predefined schema, sacrificing flexibility.
- Stream Processing Platforms: These offer maximum flexibility with no predefined schema, but place a significant burden on developers. Engineers must manually implement and configure custom pipelines to handle, transform, and persist the data.
The goal of this Bachelor thesis is to address the "middle ground" between these two extremes. By leveraging Multi-Model, Multi-Engine Database Management Systems (PolyDBMS), this work will develop an architectural extension model that allows for the seamless persistence of dynamic data. This will be achieved by utilizing and extending existing heterogeneous data stores within the PolyDBMS.
The research will be implemented as a functional prototype within the Polypheny ecosystem.
Start / End Dates
2026/03/04 - 2026/07/04