Datenschutzerklärung|Data Privacy

Martin Pagel

Do 18.11.2021, 1 pm - 2 pm, Virtual Event: Storage and Indexing in Cloud Data Warehouses (Benjamin Wagner, Firebolt)

DIMA/IAM presentation: "Storage and Indexing in Cloud Data Warehouses" by Benjamin Wagner (Firebolt)

Like so many things in life, database system research goes through phases. In the 2000s, the advent of large and comparably cheap main-memory systems seemed to eliminate the need for traditional research topics like buffer managers, on-disk compression, or spilling operators. In recent years, however, improvements in main memory capacity and pricing have stagnated. At the same time, the amount of data has kept growing exponentially. As such, many of the traditional questions facing database systems have re-emerged as exciting research topics.
This can be best seen in modern cloud data warehouses like Firebolt. Disaggregated storage and compute have added another layer to the already deep memory hierarchy. AWS S3 as the backing storage medium suffers from both very high latency and low throughput. This talk shows how to build a modern storage and indexing layer into a cloud data warehouse. We provide an in-depth examination of F3, Firebolt's custom file format. A columnar data layout, aggressive compression of ordered data, and sparse indexing enable high query performance in a challenging cloud environment.

Short bio:
Benjamin leads Firebolt's query processing team. After studying at TU Munich, he decided to join Firebolt to work on real-world data challenges facing the world's most data-driven companies.
He is passionate about all things data and excited by the possibilities arising from academia and industry collaborating to solve some of the hardest problems in data management.

Log-in information:
If you are interested in attending the event, please contact!