Datenschutzerklärung|Data Privacy

Juan Soto

01.02.2015: Demo "Optimistic Recovery for Iterative Dataflows in Action" accepted at SIGMOD 2015

The Submission "Optimistic Recovery for Iterative Dataflows in Action" (Sergey Dudoladov, Sebastian Schelter, Chen Xu, Asterios Katsifodimos, Stephan Ewen, Kostas Tzoumas, Volker Markl) has been accepted for publication/demonstration at the demo track of ACM SIGMOD 2015 in Melbourne, Australia.

Abstract :
Over the past years, parallel dataflow systems have been employed for advanced analytical applications, especially in the fields of data mining where a large number of algorithms are of iterative nature. During the execution of iterative parallel dataflows at large scale, some form of failure is very likely to occur. Current systems typically achieve fault tolerance through rollback recovery by periodically checkpointing the algorithm's state and, in case of failure, restoring a consistent state from a checkpoint.

In prior work, we presented an optimistic recovery mechanism that eliminates the need for checkpointing and rollback recovery in certain cases. In case of failure, our recovery mechanism uses an algorithmic compensation function for transition to a consistent algorithm state, from which the execution can continue. Since this recovery mechanism does not checkpoint any state, it achieves optimal failure-free performance while guaranteeing fault tolerance.

In this paper, we demonstrate our optimistic recovery approach built upon the Apache Flink dataflow platform. During our demonstration, attendees can interact with a GUI that allows them to run graph algorithms with different input datasets. The attendees have the opportunity to induce failures while the algorithms are running, and watch the algorithms recovering with the use of compensation functions instead of checkpoints.