Open Source Industrial IoT Data Platform

Apache StreamPipes helps teams connect industrial data sources, build real-time streaming pipelines, explore time-series data, and deliver live operational insights without requiring many different systems or custom code for every use case.

Overview

Apache StreamPipes is an end-to-end industrial IoT data platform that combines a graphical user interface for domain experts with an extensible framework for developers. It enables teams to connect industrial data sources, process real-time data streams, store historical data, and build data-driven applications on top of industrial equipment.

StreamPipes provides an opinionated architecture for managing industrial IoT data. Most users rely on StreamPipes to make data from machines, production systems, and other industrial assets available to analytics, monitoring, and application layers.

A standard StreamPipes installation includes everything needed to get started: a messaging system for real-time data exchange, with NATS as the default and support for additional protocols, as well as a time-series database for storing historical data. StreamPipes also includes a wide range of connectors tailored to industrial IoT requirements. For example, it provides extensive OPC UA support, including a web-based node browser and support for OPC UA events.

StreamPipes is fully open source, governed by the Apache Software Foundation, and includes enterprise-grade capabilities such as user and role management, OAuth 2.0 integration, and support for geographically distributed deployments that simplify communication between OT and IT networks.

As a ready-to-use platform, StreamPipes enables users to implement industrial IoT use cases on top of a reliable data foundation. Developers can extend the platform through REST APIs, client libraries for various programming languages, and an SDK for building custom connectors, processors, sinks, and application logic.

Product Tour

Feature	Preview
Connect data sources with a guided setup flow
Build stream analytics pipelines visually
Create beautiful charts and dashboards
Asset-centric organization and navigation
Administration, users, and platform operations

Built for users

• Connect industrial and messaging systems such as OPC UA, PLCs, MQTT, REST, Pulsar, Kafka, and more.
• Build harmonization and analytics pipelines with a library of ready-to-use adapters, processors, and sinks.
• Explore historical data visually with charts tailored to time-series use cases.
• Create live dashboards for real-time monitoring on the shop floor or in operations centers.
• Organize pipelines, streams, and dashboards around assets and operational structures.

Built for developers

• Extend the platform with custom adapters, processors, and sinks through the Java SDK.
• Run pipeline elements as standalone microservices in central or edge deployments.
• Integrate existing processing logic and ML models into reusable pipeline elements.
• Use our Python, Go or Java client to interact with StreamPipes from your own applications.

Quick Start

Run StreamPipes as a user

The fastest way to get a full installation with extensions is to use one of the Docker-based installers:

• StreamPipes Compose for container-based deployment
• StreamPipes CLI for developers who want to extend StreamPipes
• StreamPipes k8s for cluster-based operation

For most first-time users, installer/compose is the right starting point.

cd installer/compose
docker-compose up -d

After the services are up, open http://localhost to complete the setup in the browser.

Build StreamPipes as a developer

Prerequisites:

• Java 25 JDK
• Maven 3.8+
• Node.js and npm for the UI build
• Docker and Docker Compose

Backend build:

mvn clean package

UI build:

cd ui
npm install
npm run build

From the repository root, you can start the full stack with:

docker-compose up --build -d

This will start a development stack with no volumes! Choose the installation options from the installer page for production setups.

For backend changes, prefer targeted module validation first, for example:

mvn -pl <module> -am test

Documentation

The main documentation lives at streampipes.apache.org/docs.

• Quick start guide
• Create adapters
• Build pipelines
• Developer setup
• Write custom pipeline elements

Repository Structure

This repository contains the StreamPipes platform, SDKs, extensions, installers, and UI in a single monorepo. A few important entry points:

• streampipes-service-core for bootstrapping, security, migrations, and scheduling
• streampipes-rest for HTTP and resource APIs
• streampipes-extensions for bundled adapters and pipeline elements
• streampipes-sdk for extension development
• ui for the web application
• installer for Compose, CLI, and Kubernetes deployment options

Extending StreamPipes

StreamPipes is designed to be extended. Custom processors, sinks, and data sources can be packaged as pipeline elements and deployed independently of the core platform.

• Use the Java SDK to wrap existing processing logic.
• Package new functionality as containerized microservices.
• Deploy extensions centrally or near the edge, depending on latency and infrastructure constraints.

The bundled extension code lives in streampipes-extensions.

Community

Get help

• Support channels
• Mailing lists
• Quick start guide

Or simply use Github Discussions.

To subscribe directly to a mailing list:

• users-subscribe@streampipes.apache.org
• dev-subscribe@streampipes.apache.org

Contribute

Contributions are welcome across core services, UI, extensions, installers, and documentation.

• Review CONTRIBUTING.md
• Check the Get Involved page
• Browse open issues
• Start with a good first issue
• Visit the developer wiki

Report bugs or request features

• Bug reports
• Feature ideas and discussions

License

Apache License 2.0