Category: Azure
-
Streaming ETL with Azure Data Factory and CDC – Setting up Audit Tables
In this series we look at building a Streaming ETL with Azure Data Factory and CDC – Setting up Audit Tables. This is Part 1, The rest of the series is below. Enabling CDC Setting up Audit Tables Provisioning Azure Data Factory Provisioning Azure Blog Storage Create Data Source Connection in ADF Create Incremental Pipeline…
-
Streaming ETL with Azure Data Factory and CDC – Enabling CDC
In this series we look at building a Streaming ETL with Azure Data Factory and CDC – Enabling CDC. This is Part 1, The rest of the series is below. Enabling CDC Setting up Audit Tables Provisioning Azure Data Factory Provisioning Azure Blog Storage Create Data Source Connection in ADF Create Incremental Pipeline in ADF…
-
Setting up Azure Data Factory Integration Runtime for On-Prem Connections
If you need to connect your Azure Data Factory to an on-premise SQL server you will need help Setting up Azure Data Factory Integration Runtime for On-Prem Connections Set up On-Prem Integration Run Time Click on Author & Monitor from Azure Data Factory Create two new linked services Linked Service to Database. Since we are…
-
Direct Custom SQL Query Denodo from Power BI M Query
If you are planning to use Power BI on top of Denodo sometimes a base view or a Power Query won’t be enough. If you need to embed your own complex query to pull data from Denodo, there are a few items you must be aware of. See below on how to Direct Custom SQL…
-
Download free backgrounds to make your Power BI reports pop!
Download free backgrounds to make your Power BI reports pop! The difference between a good looking Power BI report and a great looking Power BI report is just a short extra mile using a custom background template. It is easy to create custom Backgrounds in Paint or Powerpoint to make your Power BI reports Pop!…
-
Modern Data Architecture – Part 9 – Load Data into Synapse Data Warehouse
Modern Data Architecture – Part 9 – Loading Data into Synapse Data Warehouse Now that we have provisioned a Synapses Data Analytics environment, we are now ready to begin loading data into this environment. In lab 7, we loaded a single table “sales.customer” to our data lake. To complete this lab, you will either need…
-
Modern Data Architecture – Part 8 – Provisioning a Synapsis SQL Data Warehouse
Modern Data Architecture – Part 8 – Provisioning a Synapsis SQL Data Warehouse Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale. Azure Synapse brings these two worlds…
-
Modern Data Architecture – Part 7 – Staging Data into Data Lake
Modern Data Architecture – Part 7 – Staging Data into Data Lake In Lab 5, we demonstrated loaded data from our sample SQL server into our data lake. Although this tool is very handy, it creates a new service and set of tools that need to be provisioned and monitored. It is best to use…
-
Modern Data Architecture – Part 6 – Configuring and setting up Data Bricks
Modern Data Architecture – Part 6 – Configuring and setting up Data Bricks Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Databricks is an industry-leading, cloud-based data engineering tool used for processing and transforming massive quantities of data and exploring the data through machine learning models Azure…
-
Modern Data Architecture – Part 5 – Loading Data Lake with Data Factory
Modern Data Architecture – Part 5 – Loading Data Lake with Data Factory The Azure Data Factory (ADF) is a service designed to allow developers to integrate disparate data sources. It is a platform somewhat like SSIS in the cloud to manage the data you have both on-prem and in the cloud. It provides access…