IoT Stream Processing

Introduction of Wide Area Data Collection Infrastructure (Mobile SINET) and SINETStream

Official Site: SINETStream, https://www.sinetstream.net/

Overview

Big data generated by numerous indoor and outdoor IoT devices must be securely analyzed in real-time to create various innovative services. SINET allows the construction of an end-to-end isolated HPC and IoT environment using VPN over mobile and wired networks. SINETStream is a software library that enables the easy development of secure and efficient IoT applications over the environment.

Mobile Network for IoT Applications by SINET

The SINET Japanese R&E network has begun offering secure and high-performance VPN services over mobile and wired networks (Fig.1).
Each IoT application can construct an end-to-end isolated HPC and IoT environment using the SINET VPN.

SINETStream: Software Library for IoT Applications

SINETStream (Fig.2) enables easy development of secure and efficient IoT applications for Java, Python, and Android.

It provides the following capabilities:
- User-friendly APIs to write and read stream data
- Authentication and authorization
- TLS and data encryption
- Passive metrics collection
- Cooperation with stream processing
- Adaptation for various brokers such as MQTT and Apache Kafka. (Fig.3)

relationship — Fig.3. Relationship between SINETStream, message brokers and stream processing platforms.

Support for Android: SINETStream Android

SINETStream Android consists of three libraries:
- Core library - provides the SINETStream basic functions. - Enables to use MQTT brokers via Eclipse Paho (https://www.eclipse.org/paho).
- Helper library - Collects data from various sensors built-in an Android device. - Publishes it via the core library.
- Sample app - Android app implemented using the helper library. - allows users to collect sensors’ data without development.

SINETStream Use Cases

Secure online video processing

We employ YOLOv3 and OpenPose for object detection and human keypoint detection, respectively.
Image stream data captured at the sensor are sent to the cloud via the VPN, stored and analyzed in real-time.
IoT application servers including GPU nodes are easily deployed by using VCP.

Collection of Sensor Data from Android Smartphones

SINETStream also provides an Android app that collects various sensor data from smartphones.
The app can be utilized as a sensor program for IoT apps without coding.
Selected sensor data captured by the app are sent to the cloud via the VPN, where they are stored and visualized in real-time.

android app — Fig.6. Android sensor data collection.

Environment monitoring live demo

With SINETStream you can easily develop a program that collect and analyze data from sensors distributed over a wide area network.
The writer program running on a Raspberry Pi with temperature and humidity sensors continuously sends data to a messaging server called “SINETStream server” on AWS.
The reader program on AWS then receives the data from the SINETStream server and the processed data are periodically visualized here.
These programs use the SINETStream API to implement such functions as writing data from the sensor devices to the SINETStream server, and reading data from the server for visualization.

SINETStream Web site: https://www.sinetstream.net/index.en.html
Live demo: https://www.sinetstream.net/docs/livedemo/livedemo.en.html
Tutorial: https://www.sinetstream.net/docs/tutorial/index.en.html
Promotion video: https://www.youtube.com/watch?v=Z0wlUi4lr6c