Datenschutzerklärung|Data Privacy
Impressum

12.10.2015
A. Borusan

19.10.2015, 16 Uhr c.t. TU Berlin, EN building, seminar room EN 719 (7th floor), Einsteinufer 17, 10587 Berlin: "Apache REEF – The Retainable Evaluator Execution Framework" (Markus Weimer Microsoft, Redmond)

Resource managers like Apache YARN emerged as a critical layer in the cloud computing stack. They offer a flexible, low level abstraction for leasing cluster resources and instantiating application logic on them. This flexibility comes at a high cost in terms of developer effort, as each application must repeatedly tackle the same challenges (e.g., fault-tolerance, task scheduling and coordination) and re-implement common mechanisms (e.g., caching, bulk-data transfers).

I present REEF, a development framework that provides a control-plane for scheduling and coordinating task-level (data-plane) work on cluster resources obtained from a resource manager. REEF provides mechanisms that facilitate resource re-use for data caching, and state management abstractions that greatly ease the development of elastic data processing work-flows on resource managed cloud platforms.

REEF is used to develop several commercial offerings such as the Azure Stream Analytics service at Microsoft. REEF is also an Apache Incubator project that has attracted contributors from several institutions.