Real-Time Big Data Processing with Azure Lab 2 - Getting Started with IoT Hubs

Overview In this lab, you will create an Azure IoT Hub and use it to collect data from a client application.

What You’ll Need To complete the labs, you will need the following: • • • • •

A web browser A Microsoft account A Microsoft Azure subscription A Windows, Linux, or Mac OS X computer The lab files for this course

Note: To set up the required environment for the lab, follow the instructions in the Setup document for this course. Specifically, you must have signed up for an Azure subscription and installed Node.JS on your computer.

Exercise 1: Provision an IoT Hub In this exercise, you will create an IoT hub. Note: The Microsoft Azure portal is continually improved in response to customer feedback. The steps in this exercise reflect the user interface of the Microsoft Azure portal at the time of writing, but may not match the latest design of the portal exactly.

Create an IoT Hub In this procedure, you will use the Azure portal to create an IoT hub. 1. 2.

In the Microsoft Azure portal, in the Hub Menu, click New. Then in the Internet of Things menu, click IoT Hub. In the IoT Hub blade, enter the following settings, and then click Create: • Name: Enter a unique name (and make a note of it!) • Pricing and scale tier: Free • IoT Hub Units: 1 • Device-to-cloud partitions: 2 partitions • Subscription: Select your Azure subscription • Resource Group: Select the existing resource group you created previously

3.

• Location: Select any available region • Pin to dashboard: Not selected In the Azure portal, view Notifications to verify that deployment has started. Then wait for the service bus namespace to be deployed (this can take a few minutes.)

Exercise 2: Register a Client Device Unlike event hubs, all devices that use an IoT hub must be individually registered, and use their own endpoint and shared access key to access the hub. In this exercise, you will register a client device.

Create a Device Identity Each device that sends data to the IoT hub must be registered with a unique identity. 1. In the Azure portal, browse to the blade for the IoT Hub that you created in the previous exercise, and click the Device Explorer tab. 2. Click the + Add button and add a new device with the ID MyDevice that uses a Symmetric Key authentication type (auto-generate the keys). 3. After it has been added, click MyDevice and view the device-specific keys and connection strings that have been generated. Then copy the connection string-primary key for MyDevice to the clipboard. You will use this in the next exercise.

Exercise 3: Submit Data from a Client Device Application Now that you have registered a client device, you can create an application that the device can use to submit data to the IoT Hub.

Create a Client Device Application Now that you have registered a device, it can submit data to the IoT hub. Note: The code for MyDevice is similar to the code you used in the previous lab to submit events to an event hub – it generates readings from ten random devices. In a real solution, each device would use its own device-specific ID and connection string to enable you to manage them individually. 1. In the Node.JS console, navigate to the iotdevice folder in the folder where you extracted the lab files. 2. Enter the following command, and press RETURN to accept all the default options. This creates a package.json file for your application: npm init

3. Enter the following command to install the Azure IoT device and AMQP protocol packages: npm install azure-iot-device azure-iot-device-amqp

4. Use a text editor to edit the iotdevice.js file in the iotdevice folder. 5. Modify the script to set the connStr variable to reflect the device connection string for the MyDevice device (which you copied to the clipboard in the previous exercise), as shown here: 'use strict'; var clientFromConnectionString = require('azure-iot-deviceamqp').clientFromConnectionString; var Message = require('azure-iot-device').Message; var connStr = '';

var client = clientFromConnectionString(connStr); function printResultFor(op) { return function printResult(err, res) { if (err) console.log(op + ' error: ' + err.toString()); if (res) console.log(op + ' status: ' + res.constructor.name); }; } var connectCallback = function (err) { if (err) { console.log('Could not connect: ' + err); } else { console.log('Client connected'); // Create a message and send it to the IoT Hub every second setInterval(function(){ var r = Math.random(); var data = JSON.stringify({ device: 'MyDevice', reading: r }); var message = new Message(data); console.log("Sending message: " + message.getData()); client.sendEvent(message, printResultFor('send')); }, 1000); } }; client.open(connectCallback);

6. Save the script and close the file.

Submit data to the IoT Hub Now you can run your client application to submit data to the IoT hub. 1. In the Node.JS console window, enter the following command to run the script: node iotdevice.js

2. Observe the script running as it starts to submit device readings. Then leave the script running and start the next exercise.

Exercise 4: Read Data from the IoT Hub When devices submit messages to your IoT hub, you can read the messages from its event hub endpoint.

Create an Application to Read Messages from the IoT Hub In this procedure, you will create a client application that reads from the event hub endpoint of the IoT hub. 1. In the Azure portal, browse to the blade for the IoT Hub. 2. In the blade for your IoT Hub, click Shared access policies. 3. In the Shared access policies blade, click the service policy, and then copy the connection string-primary key for your IoT Hub to the clipboard – you will need this later. 4. Open a new Node.JS console, and navigate to the iotreader folder in the folder where you extracted the lab files.

5. Enter the following command, and press RETURN to accept all the default options. This creates a package.json file for your application: npm init

6. Enter the following command to install the Azure IoT Hub package: npm install azure-event-hubs

7. Use a text editor to edit the iotreader.js file in the iotreader folder. 8. Modify the script to set the connStr variable to reflect the service shared access policy connection string for your IoT Hub (not the device-specific connection string you used in the previous procedure), as shown here: 'use strict'; var EventHubClient = require('azure-event-hubs').Client; var connStr = ''; var printError = function (err) { console.log(err.message); }; var printMessage = function (message) { console.log('Message received: '); console.log(JSON.stringify(message.body)); console.log(''); }; var client = EventHubClient.fromConnectionString(connStr); client.open() .then(client.getPartitionIds.bind(client)) .then(function (partitionIds) { return partitionIds.map(function (partitionId) { return client.createReceiver('$Default', partitionId, { 'startAfterTime' : Date.now()}).then(function(receiver) { console.log('Created partition receiver: ' + partitionId) receiver.on('errorReceived', printError); receiver.on('message', printMessage); }); }); }) .catch(printError);

9. Save the script and close the file.

Read Data from the IoT Hub Now you arew ready to run your application and read data from the IoT Hub. 1. Ensure that the iotdevice.js script is still running. 2. In the Node.JS console window for the iotreader.js application, enter the following command to run the script: node iotreader.js

3. Observe the script running as it reads the device readings that were submitted to the IoT hub by the iotdevice.js script.

4. After both scripts have been running for a while, stop them by pressing CTRL+C in each of the console windows. Then close the console windows. Note: You will use the resources you created in this lab when performing the next lab, so do not delete them. Ensure that all Node.js scripts are stopped to minimize ongoing resource usage costs.

Microsoft Learning Experiences - GitHub

Real-Time Big Data Processing with Azure. Lab 2 - Getting Started with IoT Hubs. Overview. In this lab, you will create an Azure IoT Hub and use it to collect data ...

1004KB Sizes 2 Downloads 291 Views

Recommend Documents

Microsoft Learning Experiences - GitHub
Performance for SQL Based Applications. Then, if you have not already done so, ... In the Save As dialog box, save the file as plan1.sqlplan on your desktop. 6.

Microsoft Learning Experiences - GitHub
A Windows, Linux, or Mac OS X computer. • Azure Storage Explorer. • The lab files for this course. • A Spark 2.0 HDInsight cluster. Note: If you have not already ...

Microsoft Learning Experiences - GitHub
Start Microsoft SQL Server Management Studio and connect to your database instance. 2. Click New Query, select the AdventureWorksLT database, type the ...

Microsoft Learning Experiences - GitHub
performed by writing code to manipulate data in R or Python, or by using some of the built-in modules ... https://cran.r-project.org/web/packages/dplyr/dplyr.pdf. ... You can also import custom R libraries that you have uploaded to Azure ML as R.

Microsoft Learning Experiences - GitHub
Developing SQL Databases. Lab 4 – Creating Indexes. Overview. A table named Opportunity has recently been added to the DirectMarketing schema within the database, but it has no constraints in place. In this lab, you will implement the required cons

Microsoft Learning Experiences - GitHub
create a new folder named iislogs in the root of your Azure Data Lake store. 4. Open the newly created iislogs folder. Then click Upload, and upload the 2008-01.txt file you viewed previously. Create a Job. Now that you have uploaded the source data

Microsoft Learning Experiences - GitHub
will create. The Azure ML Web service you will create is based on a dataset that you will import into. Azure ML Studio and is designed to perform an energy efficiency regression experiment. What You'll Need. To complete this lab, you will need the fo

Microsoft Learning Experiences - GitHub
Lab 2 – Using a U-SQL Catalog. Overview. In this lab, you will create an Azure Data Lake database that contains some tables and views for ongoing big data processing and reporting. What You'll Need. To complete the labs, you will need the following

Microsoft Learning Experiences - GitHub
The final Execute R/Python Script. 4. Edit the comment of the new Train Model module, and set it to Decision Forest. 5. Connect the output of the Decision Forest Regression module to the Untrained model (left) input of the new Decision Forest Train M

Microsoft Learning Experiences - GitHub
Page 1 ... A web browser and Internet connection. Create an Azure ... Now you're ready to start learning how to build data science and machine learning solutions.

Microsoft Learning Experiences - GitHub
In this lab, you will explore and visualize the data Rosie recorded. ... you will use the Data Analysis Pack in Excel to apply some statistical functions to Rosie's.

Microsoft Learning Experiences - GitHub
created previously. hbase org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles. /data/storefile Stocks. 8. Wait for the MapReduce job to complete. Query the Bulk Loaded Data. 1. Enter the following command to start the HBase shell. hbase shell. 2.

Microsoft Learning Experiences - GitHub
videos and demonstrations in the module to learn more. 1. Search for the Evaluate Recommender module and drag it onto the canvas. Then connect the. Results dataset2 (right) output of the Split Data module to its Test dataset (left) input and connect

Microsoft Learning Experiences - GitHub
In this lab, you will create schemas and tables in the AdventureWorksLT database. Before starting this lab, you should view Module 1 – Designing a Normalized ...

Microsoft Learning Experiences - GitHub
Challenge 1: Add Constraints. You have been given the design for a ... add DEFAULT constraints to columns based on the requirements. Challenge 2: Test the ...

Microsoft Learning Experiences - GitHub
Data Science and Machine Learning ... A web browser and Internet connection. ... Azure ML offers a free-tier account, which you can use to complete the labs in ...

Microsoft Learning Experiences - GitHub
Processing Big Data with Hadoop in Azure. HDInsight. Lab 1 - Getting Started with HDInsight. Overview. In this lab, you will provision an HDInsight cluster.

Microsoft Learning Experiences - GitHub
Real-Time Big Data Processing with Azure. Lab 1 - Getting Started with Event Hubs. Overview. In this lab, you will create an Azure Event Hub and use it to collect ...

Microsoft Learning Experiences - GitHub
Data Science Essentials. Lab 6 – Introduction to ... modules of this course; but for the purposes of this lab, the data exploration tasks have already been ... algorithm requires all numeric features to be on a similar scale. If features are not on

Microsoft Learning Experiences - GitHub
Selecting the best features is essential to the optimal performance of machine learning models. Only features that contribute to ... Page 3 .... in free space to the right of the existing modules: ... Use Range Builder (all four): Unchecked.

Microsoft Learning Experiences - GitHub
Implementing Predictive Analytics with. Spark in Azure HDInsight. Lab 3 – Evaluating Supervised Learning Models. Overview. In this lab, you will use Spark to ...

Microsoft Learning Experiences - GitHub
Microsoft Azure Machine Learning (Azure ML) is a cloud-based service from Microsoft in which you can create and run data science experiments, and publish ...

Microsoft Learning Experiences - GitHub
A Microsoft Windows, Apple Macintosh, or Linux computer ... In this case, you must either use a Visual Studio Dev Essentials Azure account, or ... NET SDK for.

Microsoft Learning Experiences - GitHub
In the new browser tab that opens, note that a Jupyter notebook named ... (Raw) notebook has been created, and that it contains two cells. The first ..... Page 9 ...