Event Hub – jack of all trades master of some https://jackofalltradesmasterofsome.com/blog Consultant - Real Estate - Author - Business Intelligence Tue, 09 Apr 2019 03:26:10 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 Create a NodeJS Client Application to Submit Data to Event Hubs https://jackofalltradesmasterofsome.com/blog/2019/04/09/create-a-nodejs-client-application-to-submit-data-to-event-hubs/ Tue, 09 Apr 2019 03:19:15 +0000 http://jackofalltradesmasterofsome.com/blog/?p=385 Get Tools and Features” and from the window, select “node.js” development to add the appropriate libraries to your Visual Studio Instance. Start […]]]> Now that we have provisioned a Event Hub in Azure, let’s Create a NodeJS Client Application to Submit Data to Event Hubs.

Prerequisites

  • Visual Studio 2017
  • Install NodeJS SDK
  • From Visual Studio, select “tool->Get Tools and Features” and from the window, select “node.js” development to add the appropriate libraries to your Visual Studio Instance.
  1. Start a command prompt window in the location of where your script will reside. For this example we will use C:\EventHub
  2. Create a package for the application using “npm init”
    1. Accept all defaults in the set up and select “yes”
    2. You will be prompted with the package.json that will be created for you
  • You must install the Azure SDK Package by running the command “npm install azure-event-hubs”
  • Navigate back to Azure to the security profile you created and copy the connection key. Place this in the js script file for the connection. This script just intermittently sends data to event hubs using a json string.
  • Run the following application in command “node eventclient.js” to begin sending messages to the Event Hub.
  • If you navigate back to Azure, you will see the events being recorded in the Event Hub.

Reading Data in Event Hubs

  1. Follow the same commands from the previous section to set up node and the json file via command prompt.
    1. npm init
    2. npm install azure-event-hubs
  2. Update the script with the connection string that was set up as send and listen shared access policy.
  • Run the command “node eventreader.js” to being to read the messages going into Event Hub.

Part 1 – Provisioning Event Hubs

]]> Provisioning an Azure Event Hub to capture real time streaming data https://jackofalltradesmasterofsome.com/blog/2019/04/09/provisioning-an-azure-event-hub-to-capture-real-time-streaming-data/ Tue, 09 Apr 2019 03:18:03 +0000 http://jackofalltradesmasterofsome.com/blog/?p=372 Provisioning an Azure Event Hub to capture real time streaming data is fairy easy once you have an Azure account. Event Hubs can be used to capture data from many different sources including databases or IoT devices. As we look at building a CDC streaming ETL, let’s take a look at the basics of Event Hubs

  1. Create a new Event Hubs in Azure, by finding it in the search bar.
  • Create a new namespace.
    • Name it something unique and use the basic pricing tier to limit cost since our needs are fairly limited and do not need the full horse power of Azure Event Hubs.
    • Select your basic subscription and create a resource group if you do not already have one.
    • Select “Create” to begin deployment process to create.
  • Once the deployment process is created, navigate to your new Namespace and select “Add Event Hub”
    • Give it a name and leave all settings as it to keep it small.
    • Once you hit create, the deployment process will start.


  • After completion you should now have an active event hub in your namespace.

Granting Access to the Event Hub

  1. On the right of the name space window, select “Shared Access Policies”
  • Add a new policy and give it “manage” access rights to it may send and listen to messages coming from and leaving the event hub. Different policies can be used to different applications as a best practice. Once created, this will create a key for access to be used to send and receive messages.

Part 2 – Build a client app in NodeJS to send data to Event Hubs

]]>