A Client for Dynamic Replication and Partitioning of Big Data (Master Project, Finished)
Author
Description
The data stores of today come in many shapes, forms and flavors. Traditional row-based relational databases with ACID guarantees compete with NoSQL data stores, columnar stores and in-memory databases. Recently, so called Polystore systems have been developed which attempt to combine different storage engines into one hybrid database system to exploit the individual advantadges. Comparing these data stores and quantifying the tradeoffs involved in choosing one is a difficult task. The use cases, access methods, query languages and configurations are diverse.
In this master project, we present Polypheny-client, a client for Database Management Systems which dynamically replicate and partition big data. Polypheny-client features two standardized scenarios, TPC-C and TPC-H as representatives of an OLTP respectively OLAP workload. It features a configurable and scalable Master-Worker architecture to stress Big Data DBMS. Polypheny-client presents another important step towards the vision of a Polystore by identifying, quantifying and visualizing bottlenecks and limitations of existing implementations. We use Polypheny-client to evaluate a commercial row-store, PostgreSQL and compare it to Icarus, a recently developed Polystore. Our evaluation is in line with expectations. While Icarus underperforms in the OLTP scenario TPC-C and shows issues for large numbers of concurrent requests, it outperforms PostgreSQL significantly in the OLAP Benchmark TPC-H . This shows promising results for the Polystore concept.
Start / End Dates
2017/02/15 - 2017/07/31