Streaming Sensor Data in Real-Time with Azure IoT Hub

/

Note: If you are just unpacking your Raspberry Pi, check out how to get up and running via a headless setup (no external keyboard or mouse required).

This post builds upon my previous demo, How to Build a Raspberry Pi Temperature Sensor. I will show how to extend the project to send temperature data into the cloud via Azure IoT Hub, then consume the incoming events to be visualised in real-time using Stream Analytics and Power BI.

iot_hub_stream_analytics_power_bi.png

Prerequisites

1. Register an IoT Device

  1. Navigate to IoT Devices within your IoT Hub.
  2. Click Add.
  3. Populate the Device ID (e.g. raspberrypi).
  4. Click Save.
iot_hub_register_device.gif

2. Create a Consumer Group

  1. Navigate to Endpoints within your IoT Hub.
  2. Click on the Events (messages/events) endpoint.
  3. Add a Consumer Group (e.g. streamanalytics_consumergroup).
  4. Click Save.
iothub_consumer_group.gif

3. Setup Stream Analytics (Input and Output)

  1. Navigate to Inputs within your Stream Analytics resource.
  2. Click Add stream input > IoT Hub
    1. Provide an Input Alias (e.g. iothubinput).
    2. Select the Consumer Group (streamanalytics_consumergroup).
    3. Click Save.
  3. Navigate to Outputs.
  4. Click Add > Power BI.
    1. Provide an Output Alias (e.g. powerbioutput).
    2. Provider a Dataset Name (e.g. temperatureDataset).
    3. Provide a Table Name (e.g. temperatureTable).
    4. Click Authorize.
    5. Click Save.
  5. Click on Query to confirm both Input and Output have been configured correctly.
stream_analytics_input_output.gif

4. Stream Sensor Data to IoT Hub from a Raspberry Pi

If you haven't done so already, I highly recommend running through How to Build a Raspberry Pi Temperature Sensor as this step assumes the circuit and required Raspberry Pi configuration is complete and ready to go.

  1. Upload the Python script below onto your Raspberry Pi (e.g. rpi_temp_sensor.py).
  2. Update the following variables:
    • SENSOR_DEVICE_ID (refer to previous post on how to get the ID of your Temperature sensor).
    • URI (e.g. your_iot_hub.azure-devices.net).
    • KEY (IoT Hub > Shared Access Policies > iothubowner > Copy & Paste the Primary Key).
    • IOT_DEVICE_ID (the ID of the registered IoT device within IoT Hub).
  3. Start running the script from the command line (e.g. python rpi_temp_sensor.py).
  4. If successful, you should see the temperature readings being printed to the console. Messages are now sent to IoT Hub :)

Note:

  • If you are running Raspbian, pip will need to be installed (sudo apt-get install python-pip).
  • The code below has a dependency on the requests library (sudo pip install requests).
  • Messages are being sent to IoT Hub via the REST API.
from base64 import b64encode, b64decode
from hashlib import sha256
from urllib import quote_plus, urlencode
from hmac import HMAC
import requests
import json
import os
import time
 
# Temperature Sensor
BASE_DIR = '/sys/bus/w1/devices/'
SENSOR_DEVICE_ID = 'YOUR_DEVICE_ID'
DEVICE_FILE = BASE_DIR + SENSOR_DEVICE_ID + '/w1_slave'

# Azure IoT Hub
URI = 'YOUR_IOT_HUB_NAME.azure-devices.net'
KEY = 'YOUR_IOT_HUB_PRIMARY_KEY'
IOT_DEVICE_ID = 'YOUR_REGISTED_IOT_DEVICE_ID'
POLICY = 'iothubowner'

def generate_sas_token():
    expiry=3600
    ttl = time.time() + expiry
    sign_key = "%s\n%d" % ((quote_plus(URI)), int(ttl))
    signature = b64encode(HMAC(b64decode(KEY), sign_key, sha256).digest())

    rawtoken = {
        'sr' :  URI,
        'sig': signature,
        'se' : str(int(ttl))
    }

    rawtoken['skn'] = POLICY

    return 'SharedAccessSignature ' + urlencode(rawtoken)

def read_temp_raw():
    f = open(DEVICE_FILE, 'r')
    lines = f.readlines()
    f.close()
    return lines

def read_temp():
    lines = read_temp_raw()
    while lines[0].strip()[-3:] != 'YES':
        time.sleep(0.2)
        lines = read_temp_raw()
    equals_pos = lines[1].find('t=')
    if equals_pos != -1:
        temp_string = lines[1][equals_pos+2:]
        temp_c = float(temp_string) / 1000.0
        return temp_c

def send_message(token, message):
	url = 'https://{0}/devices/{1}/messages/events?api-version=2016-11-14'.format(URI, IOT_DEVICE_ID)
    headers = {
        "Content-Type": "application/json",
        "Authorization": token
    }
    data = json.dumps(message)
    print data
    response = requests.post(url, data=data, headers=headers)

if __name__ == '__main__':
    # 1. Enable Temperature Sensor
    os.system('modprobe w1-gpio')
    os.system('modprobe w1-therm')

    # 2. Generate SAS Token
    token = generate_sas_token()

    # 3. Send Temperature to IoT Hub
    while True:
        temp = read_temp() 
        message = { "temp": str(temp) }
        send_message(token, message)
        time.sleep(1)

5. Stream Analytics SQL

  1. Navigate to Query within Stream Analytics.
  2. Copy and paste the SQL below into your Stream Analytics query window.
  3. Click on the ellipsis next to your IoT Hub input and click Sample data from input.
  4. Set duration to 1 minute and click OK.
  5. Once the sample data has returned, click Test.
  6. Click Save.
  7. Click Overview.
  8. Click Start and then click Start once more.
SELECT
    CAST(iothub.EnqueuedTime AS datetime) AS event_date,
    CAST(temp AS float) AS temp
INTO
    powerbioutput
FROM
    iothubinput

5. Power BI

  1. Log on to Power BI.
  2. Create a new Dashboard (My Workspace > Create > Dashbaord).
  3. Add two tiles.

Tile #1: Card

  1. Click Add tile.
  2. Click Custom Streaming Data.
  3. Click Next.
  4. Select the temperatureDataset and click Next.
  5. Populate the properties.
    1. Visualization Type: Card
    2. Fields: temp
    3. Decimal Places: 2
    4. Title: Temperature (Celsius)

Tile #2: Line Chart

  1. Click Add tile.
  2. Click Custom Streaming Data.
  3. Click Next.
  4. Select the temperatureDataset and click Next.
  5. Populate the properties.
    1. Visualization Type: Line chart
    2. Axis: event_date
    3. Values: temp
    4. Time window: 1 minute
    5. Title: Temperature (Celsius)

Video

/ /
, , , , , , ,

Taygan Rifat