Datenschutzerklärung|Data Privacy
Impressum

28.05.2013
A. Borusan

13.06.2013, 16 Uhr c.t TU Berlin/DIMA, Thursday, June 13, 2013, 4:00 p.m. Location: TU Berlin, EN building, seminar room EN 719 (7th floor), Einsteinufer 17, 10587 Berlin: "Naiad: a system for iterative, incremental, and interactive distributed dataflow"

In this talk I’ll describe the Naiad system, based on a new model for low-latency incremental and iterative dataflow. Naiad is designed to provide three properties we do not think yet exist in a single system: the expressive power of loops, concurrent vertex execution, and fine-grained edge completion. Removing any one of these requirements yields an existing class of solutions (respectively: streaming systems like StreamInsight, iterative incremental systems like Nephele, and callback systems like Percolator), but all three together appear to require a new system design. We will describe Naiad’s structured cyclic dataflow model and protocol for tracking and coordinating outstanding work, more closely resembling memory fences than traditional distributed systems barriers. We give several examples of how Naiad can be used to efficiently implement many of the currently popular “big data” programming patterns, as well as several new ones, and experimental results indicating that Naiad’s relative performance ranges from “as good as” to “much better than” existing systems.

This is joint work with Derek G. Murray, Rebecca Isaacs, Michael Isard, Paul Barham, and Martin Abadi.