Category: Data Warehouse
-
Modern Data Architecture – Part 2 – Resource Groups and Subscriptions
Modern Data Architecture – Part 2 – Resource Groups and Subscriptions Resource Groups in Azure are a mechanism of grouping a collection of services and assets in Azure together. Resource groups can assist in automatic provisioning, deprovisioning, monitoring, and access control as well as provide logical groupings for business processes, units or even environment. A…
-
Modern Data Architecture – Part 1 – Navigating the Azure Portal
Modern Data Architecture – Part 1 – Navigating the Azure Portal Logging into Azure and the Portal Open up a web browser and navigate to https://azure.microsoft.com/en-us/ and click on “Portal” to log in. You will be navigated to authentication portal to log in. You can either sign up for a new account which comes with…
-
Auto Spin up and Down Azure Virtual Machines (VM)
One of the best perks of moving your infrastructure to the cloud is the ability to only pay for the resources you use. When using Virtual Machines that usage can be even further limited to servers that stay on only when you need them (business hours or nightly data runs) and then shut down to…
-
Create a NodeJS Client Application to Submit Data to Event Hubs
Now that we have provisioned a Event Hub in Azure, let’s Create a NodeJS Client Application to Submit Data to Event Hubs. Prerequisites Visual Studio 2017 Install NodeJS SDK
-
Provisioning an Azure Event Hub to capture real time streaming data
Provisioning an Azure Event Hub to capture real time streaming data is fairy easy once you have an Azure account. Event Hubs can be used to capture data from many different sources including databases or IoT devices. As we look at building a CDC streaming ETL, let’s take a look at the basics of Event…
-
Visual Studio Dev Essentials to download and install SQL Server Developer Edition
With the Microsoft BizSpark program now gone, so is the easy way to download and install SQL Server to your local machine for development purposes. Let’s take a look at using Visual Studio Dev Essentials to download and install SQL Server Developer Edition on your local machine for data development.
-
Streaming ETL using CDC and Azure Event Hub. A Modern Data Architecture.
In Modern Data architecture, As Data Warehouses have gotten bigger and faster, and as big data technology has allowed us to store vast amounts of data it is still strange to me that most data warehouse refresh processes found in the wild are still some form of batch processing. Even Hive queries against massive Hadoop…
-
Configure basic SSRS (SQL Server Reporting Services) and Set up Firewall Rules
Learn how to Configure basic SSRS (SQL Server Reporting Services) and Set up Firewall Rules so you can access SSRS remotely. Side Note: Want to learn SQL or Python for free. In less then 10 minutes a day and less than an hour total? Signup for my free classes delivered daily right to your email…
-
The Modern Data Warehouse; Azure Data Lake and U-SQL to combine data
The modern data warehouse will need to use Azure Data Lake and U-SQL to combine data. Begin by navigating to your Azure Portal and searching for the Data Lake Analytics Resource. Let’s start by creating a new Data Lake. Don’t worry, this service only charges on data in and out, not just remaining on like…
-
An Introduction to Consulting Book!
A few years ago, as I reflected on my career, I decided to begin documenting some of my stories and lessons learned to share on a blog. This exercise quickly turned into pages of An Introduction to Consulting Book. After fourteen years in Consulting which landed me on over twenty projects, across several different industries…