Dynamic Replication and Partitioning of Big Data (Master Project, Finished)
Author
Khushbu Rajendra Agrawal
Description
The goal of this Master Project is the development an infrastructure system which
- achieves fault-tolerance using data replication protocols and techniques and which
- efficiently stores data by distributing it using data partitioning protocols and techniques.
The overall goal is to use a combination of both to achieve efficient and fault-tolerant data management using user defined replication and partitioning parameters. The parameters should be adjustable; as soon as the parameters are changed, data should be automatically re-distributed accordingly.
Requirements
On the Software
- The user can specify which data to replicate, their replication degree, and how to partition the data on the basis of the parameters.
- These parameters can be changed by the user at run-time causing the system to properly copy/move the data to fulfill the newly set parameters.
- Dynamic adaptation of the data distribution when parameters change.
On the Project
- Description of the system architecture. This includes the selection of data stores (relational DBMS, key/value stores, etc. or any combination thereof). Implementation of the system architecture.
- Definition of the relevant parameters (e.g., the desired distribution and replication degree) and a means to collect/specify them.
- Milestone presentation presenting the project plan, the distribution of topics and responsibilities along with the proposed architecture, benchmarks, dimensions and parameters.
- Final presentation on the results/outcome of the project.
Optional
- Visualization of the system, data deployment, etc. *
Deliverables
- Final project report
- Presentation slides
- Source code
Modalities
The project can be designed either for one student or for a group of two students (in the latter case, the optional visualization will be mandatory part of the project). If run as a group project, in a first phase, the group should work out the details of a project plan, the distribution of topics and responsibilities within the group.
_________________________________________________________________
* The visualization is mandatory if this project is processed by two students.
Start / End Dates
2017/02/15 - 2017/07/31