Datenschutzerklärung|Data Privacy

Martin Pagel

The Paper "AJoin: Adhoc Stream Joins at Scale" was Accepted for Publication at VLDB 2020

AJoin: Adhoc Stream Joins at Scale, Jeyhun Karimov, Tilmann Rabl, Volker Markl . To be Presented at the 46th Conference on Very Large Data Bases (VLDB), Tokyo, Japan, 2020 and to Appear in the Proceedings of the VLDB Endowment, Vol. 13, Nr. 4.

The processing model of state-of-the-art stream processing engines is designed to execute long-running queries one at a time. However, with the advance of cloud technologies and multi-tenant systems, multiple users share the same cloud for stream query processing. This results in many ad-hoc stream queries sharing common stream sources. Many of these queries include joins. There are two main limitations that hinder performing ad-hoc stream join processing. The first limitation is missed optimization potential both in stream data processing and query optimization layers. The second limitation is the lack of dynamicity in query execution plans.
We present AJoin, a dynamic and incremental ad-hoc stream join framework. AJoin consists of an optimization layer and a stream data processing layer. The optimization layer periodically reoptimizes the query execution plan, performing join reordering and vertical and horizontal scaling at run-time without stopping the execution. The data processing layer implements pipeline-parallel join architecture. This layer enables incremental and consistent query processing supporting all the actions triggered by the optimizer. We implement AJoin on top of Apache Flink, an open-source data processing framework. AJoin outperforms Flink not only at ad-hoc multi-query workloads but also at single-query workloads.

A preprint version is available here.

If you want to learn more about VLDB 2020 visit