NebulaStream -
Data Management for the Internet of Things
NebulaStream is a general purpose, end-to-end data management system for the IoT. It provides an out-of-the box experience with rich data processing functionalities and a high ease-of-use.

NebulaStream is a joint research project being undertaken in the IoT-Lab at the Berlin Institute for the Foundations of Learning and Data (BIFOLD). The first contributors to NebulaStream are researchers from the DIMA group at TU Berlin and the DFKI IAM group.

Join our community and use NebulaStream for your research projects!

Try NebulaStream

We provide containers and prebuilt-binaries of NebulaStream. For an easy start, follow our tutorial.

Tutorial
Contribute to NebulaStream

We invite you to contribute to NebulaStream. If you have further questions, please contact us.

Request Access Contact us on Slack

Features

NebulaStream conducts state-of-the-art research, innovates, and integrates various technologies, including cloud, fog, and sensor networks, to create a unified sensor-fog-cloud environment and facilitate the development of foreseeable IoT applications.

Thousands of Queries

NebulaStream supports the execution of thousands of queries over millions of sensors.

Dynamic Topologies

NebulaStream detects and reacts to changes in the underlying topology without impacting query processing.

Heterogeneous Workloads

NebulaStream supports rich workloads beyond classical stream processing (e.g., geo-spatial analytics, complex-event processing, machine learning).

Efficient Resource Utilization

NebulaStream compiles queries into highly efficient code, which increases hardware utilization and reduces energy consumption significantly.

Efficient State Management

NebulaStream supports rich stateful operations that take event ordering into account and provide different levels of guarantee.

Heterogeneous Device Support

NebulaStream supports a wide range of devices including different architectures (e.g., ARM, x86) and accelerators (e.g., GPUs, TPUs).

Publications


Project Overview

The NebulaStream Platform: Data and Application Management for the Internet of Things
CIDR 2020 | Steffen Zeuch, Ankit Chaudhary, Bonaventura Del Monte, Haralampos Gavriilidis, Dimitrios Giouroukis, Philipp M. Grulich, Sebastian Bress, Jonas Traub, Volker Markl
NebulaStream: Complex Analytics Beyond the Cloud
VLIoT 2020 | Steffen Zeuch, Eleni Tzirita Zacharatou, Shuhao Zhang, Xenofon Chatziliadis, Ankit Chaudhary, Bonaventura Del Monte, Dimitrios Giouroukis, Philipp M. Grulich, Ariane Ziehn, Volker Markl
Showcasing Data Management Challenges for Future IoT Applications with NebulaStream
VLDB 2023 | Aljoscha Lepping, Hoang Mi Pham, Laura Mons, Balint Rueb, Ankit Chaudhary, Philipp M. Grulich, Steffen Zeuch, Volker Markl
Using and Enhancing NebulaStream – A Tutorial
DEBS 2024 | Steffen Zeuch, Ankit Chaudhary, Viktor Rosenfeld, Taha Tekdogan, Adrian Michalke, Matthis Gördel, Ariane Ziehn, Volker Markl

System Publications

Fault Tolerance Placement in the Internet of Things
SIGMOD 2024 | Anastasiia Kozar, Bonaventura Del Monte, Steffen Zeuch, Volker Markl
Query Compilation Without Regrets
SIGMOD 2024 | Philipp M. Grulich, Aljoscha P. Lepping, Dwi P. A. Nugroho, Bonaventura Del Monte, Varun Pandey, Steffen Zeuch, Volker Markl
Efficient Placement of Decomposable Aggregation Functions for Stream Processing over Large Geo-Distributed Topologies
VLDB 2024 | Xenofon Chatziliadis, Eleni Tzirita Zacharatou, Alphan Eracar, Steffen Zeuch, Volker Markl
Incremental Stream Query Merging
EDBT 2023 | Ankit Chaudhary, Jeyhun Karimov, Steffen Zeuch, Volker Markl
Rethinking Stateful Stream Processing with RDMA
SIGMOD 2022 | Bonaventura Del Monte, Steffen Zeuch, Tilmann Rabl, Volker Markl
Babelfish: Efficient Execution of Polyglot Queries
VLDB 2022 | Philipp M. Grulich, Steffen Zeuch, Volker Markl
An Energy-Efficient Stream Join for the Internet of Things
DAMON 2021 | Adrian Michalke, Philipp M. Grulich, Clemens Lutz, Steffen Zeuch, Volker Markl
Streaming Data through the IoT via Actor-Based Semantic Routing Trees
VLIoT 2021 | Dimitrios Giouroukis, Johannes Jestram, Steffen Zeuch, Volker Markl
Monitoring of Stream Processing EnginesBeyond the Cloud: an Overview
VLIoT 2021 | Xenofon Chatziliadis, Eleni Tzirita Zacharatou, Steffen Zeuch, Volker Markl
ExDRa: Exploratory Data Science on Federated Raw Data
SIGMOD 2021 | Sebastian Baunsgaard, Matthias Boehm, Ankit Chaudhary, Behrouz Derakhshan, Stefan Geißelsöder, Philipp Grulich, Michael Hildebrand, Kevin Innerebner, Volker Markl, Claus Neubauer, Sarah Osterburg, Olga Ovcharenko, Sergey Redyuk, Tobias Rieger, Alireza Rezaei Mahdiraji, Sebastian Benjamin Wrede, Steffen Zeuch
Parallelizing Intra-Window Join on Multicores: An Experimental Study
SIGMOD 2021 | Shuhao Zhang, Yancan Mao, Jiong He, Philipp M Grulich, Steffen Zeuch, Bingsheng He, Richard TB Ma, Volker Markl
Towards Resilient Data Management for the Internet of Moving Things
BTW 2021 | Elena Beatriz Ouro Paz, Eleni Tzirita Zacharatou, Volker Markl
Automatic Tuning of Read-Time Tolerances for Optimized On-Demand Data-Streaming from Sensor Nodes
EDBT 2021 | Julius Hülsmann, Chiao-Yun Li, Jonas Traub, Volker Markl
Demand-based Sensor Data Gathering with Multi-Query Optimization
VLDB 2020 | Julius Hülsmann, Jonas Traub, Volker Markl
Complex Event Processing for the Internet of Things
VLDB 2020 PhD Workshop | Ariane Ziehn
A Survey of Adaptive Sampling and Filtering Algorithms for the Internet of Things
DEBS 2020 | Dimitrios Giouroukis, Alexander Dadian, Jonas Traub, Steffen Zeuch, Volker Markl
Rhino: Efficient Management of Very Large Distributed State for Stream Processing Engines
SIGMOD 2020 | Bonaventura Del Monte, Steffen Zeuch, Tilmann Rabl, Volker Markl
Grizzly: Efficient Stream Processing Through Adaptive Query Compilation
SIGMOD 2020 | Philipp M. Grulich, Sebastian Breß, Steffen Zeuch, Jonas Traub, Janis von Bleichert, Zongxiong Chen, Tilmann Rabl, Volker Markl
Scaling a Public Transport Monitoring System to Internet of Things Infrastructures
EDBT 2020 | Haralampos Gavriilidis, Adrian Michalke, Laura Mons, Steffen Zeuch, Volker Markl
Governor: Operator Placement for a Unified Fog-Cloud Environment
EDBT 2020 | Ankit Chaudhary, Steffen Zeuch, Volker Markl
Disco: Efficient Distributed Window Aggregation
EDBT 2020 | Lawrence Benson, Philipp M. Grulich, Steffen Zeuch, Volker Markl, Tilmann Rabl
SENSE: Scalable Data Acquisition from Distributed Sensors with Guaranteed Time Coherence
arXiv Preprint 2019 | Jonas Traub, Julius Hülsmann, Sebastian Breß, Tilmann Rabl, Volker Markl
Analyzing Efficient Stream Processing on Modern Hardware
VLDB 2019 | Steffen Zeuch, Sebastian Breß, Tilmann Rabl, Bonaventura Del Monte, Jeyhun Karimov, Clemens Lutz, Manuel Renz, Jonas Traub, Volker Markl
Efficient Window Aggregation with General Stream Slicing
EDBT 2019 | Jonas Traub, Philipp Grulich, Alejandro Rodríguez Cuéllar, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl
Resense: Transparent Record and Replay of Sensor Data in the Internet of Things
EDBT 2019 | Dimitrios Giouroukis, Julius Hülsmann, Janis von Bleichert, Morgan Geldenhuys, Tim Stullich, Felipe Gutierrez, Jonas Traub, Kaustubh Beedkar, Volker Markl
Generating Reproducible Out-Of-Order Data Streams
DEBS 2019 | Philipp M. Grulich, Jonas Traub, Sebastian Breß, Asterios Katsifodimos, Tilmann Rabl, Volker Markl
Optimized On-Demand Data Streaming from Sensor Nodes
SoCC 2017 | Jonas Traub, Sebastian Breß, Tilmann Rabl, Asterios Katsifodimos, Volker Markl

Team


Contact Us & Join the Team

Join

We offer a variety of research positions, for PostDocs, PhD Students, Student Assistants and Bachelor/Master theses.

Topics include all aspects of the IoT: query compilation, query optimization, query processing, query languages, distributed data processing, complex-event processing, machine learning, signal processing, sensor networks, fog computing, temporal-spatial query processing, transactional data processing, modern hardware and many more.

Contact Us

Database Systems and Information Management (DIMA) Group Technische Universität Berlin
Sekr. E-N 7, Room E-N 728
Einsteinufer 17
10587 Berlin
Germany
+49 30 314 23555 nebulastream(at)dima.tu-berlin.de