Disco: Efficient Distributed Window Aggregation

Lawrence Benson (HPI, University of Potsdam), Philipp M. Grulich, Steffen Zeuch, Volker Markl, Tilmann Rabl (HPI, University of Potsdam)

Proceedings of the 23rd International Conference on Extending Database Technology (EDBT 2020) | March 2020

Abstract:

Many business applications benefit from fast analysis of online data streams. Modern stream processing engines (SPEs) provide complex window types and user-defined aggregation functions to analyze streams. While SPEs run in central data centers, wireless sensors networks (WSNs) perform distributed aggregations close to the data sources, which is beneficial especially in modern IoT setups. However, WSNs support only basic aggregations and windows. To bridge the gap between complex central aggregations and simple distributed analysis, we propose Disco, a distributed complex window aggregation approach. Disco processes complex window types on multiple independent nodes while efficiently aggregating incoming data streams. Our evaluation shows that Disco’s throughput scales linearly with the number of nodes and that Disco already outperforms a centralized solution in a two-node setup. Furthermore, Disco reduces the network cost significantly compared to the centralized approach. Disco’s treelike topology handles thousands of nodes per level and scales to support future data-intensive streaming applications.

Talk:

Bibtex:

 
@inproceedings{DBLP:conf/edbt/BensonGZMR20,
  author    = {Lawrence Benson and
               Philipp M. Grulich and
               Steffen Zeuch and
               Volker Markl and
               Tilmann Rabl},
  editor    = {Angela Bonifati and
               Yongluan Zhou and
               Marcos Antonio Vaz Salles and
               Alexander B{\"{o}}hm and
               Dan Olteanu and
               George H. L. Fletcher and
               Arijit Khan and
               Bin Yang},
  title     = {Disco: Efficient Distributed Window Aggregation},
  booktitle = {Proceedings of the 23nd International Conference on Extending Database
               Technology, {EDBT} 2020, Copenhagen, Denmark, March 30 - April 02,
               2020},
  pages     = {423--426},
  publisher = {OpenProceedings.org},
  year      = {2020},
  url       = {https://doi.org/10.5441/002/edbt.2020.48},
  doi       = {10.5441/002/edbt.2020.48},
  timestamp = {Wed, 25 Mar 2020 15:46:06 +0100},
  biburl    = {https://dblp.org/rec/conf/edbt/BensonGZMR20.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org}
}

View Paper

View Talk