{"id":1020,"date":"2021-01-25T02:23:08","date_gmt":"2021-01-25T02:23:08","guid":{"rendered":"http:\/\/jackofalltradesmasterofsome.com\/blog\/?p=1020"},"modified":"2021-02-05T14:37:53","modified_gmt":"2021-02-05T14:37:53","slug":"streaming-etl-with-azure-data-factory-and-cdc-provisioning-azure-data-factory","status":"publish","type":"post","link":"https:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/25\/streaming-etl-with-azure-data-factory-and-cdc-provisioning-azure-data-factory\/","title":{"rendered":"Streaming ETL with Azure Data Factory and CDC &#8211; Provisioning Azure Data Factory"},"content":{"rendered":"\n<p>In this series we look at building a Streaming ETL with Azure Data Factory and CDC &#8211;  Provisioning Azure Data Factory . This is Part 1, The rest of the series is below.<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/25\/streaming-etl-with-azure-data-factory-and-cdc-enabling-cdc\/\">Enabling CDC<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/25\/streaming-etl-with-azure-data-factory-and-cdc-setting-up-audit-tables\/\">Setting up Audit Tables<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/25\/streaming-etl-with-azure-data-factory-and-cdc-provisioning-azure-data-factory\/\">Provisioning Azure Data Factory<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/25\/streaming-etl-with-azure-data-factory-and-cdc-provisioning-provisioning-azure-blob-storage\/\">Provisioning Azure Blog Storage<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/27\/streaming-etl-with-azure-data-factory-and-cdc-creating-a-data-source-connection-in-azure-data-factory\/\">Create Data Source Connection in ADF<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/27\/streaming-etl-with-azure-data-factory-and-incremental-pipeline-azure-data-factory\/\">Create Incremental Pipeline in ADF<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/27\/streaming-etl-with-azure-data-factory-and-create-a-parameter-driver-pipeline\/\">Create a Parameter Driven Pipeline<\/a><\/li><li><a href=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/2021\/01\/27\/streaming-etl-with-azure-data-factory-and-cdc-creating-the-rolling-etl-window\/\">Create a Rolling Trigger<\/a><\/li><\/ol>\n\n\n\n<p>This series uses the Adventureworks database. For more information on how to get that set up see my Youtube video for <a href=\"https:\/\/www.youtube.com\/watch?v=JsrQGU7rhYA\">Downloading <\/a>and <a href=\"https:\/\/www.youtube.com\/watch?v=YL4-wMriI8s\">Restoring<\/a> the database.  <\/p>\n\n\n\n<p>Provisioning Azure Data Factory<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>For this section, we are assuming you have\nalready signed up for a Microsoft Azure Account. If you have not, navigate to a\n<a href=\"https:\/\/azure.microsoft.com\/\">https:\/\/azure.microsoft.com\/<\/a> to get\nyour free account. <\/li><\/ol>\n\n\n\n<ul class=\"wp-block-list\"><li>Open the Azure Portal and provision a new Data Factory &nbsp;<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"526\" height=\"122\" src=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-27.png\" alt=\"\" class=\"wp-image-1026\" srcset=\"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-27.png 526w, https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-27-300x70.png 300w\" sizes=\"auto, (max-width: 526px) 100vw, 526px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>Set up a Resource Group, and you will need to pick a name that has not been used before. Select Review and Create to complete set up of Azure Data Factory.<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"430\" height=\"292\" src=\"http:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-28.png\" alt=\"\" class=\"wp-image-1027\" srcset=\"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-28.png 430w, https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-content\/uploads\/2021\/01\/image-28-300x204.png 300w\" sizes=\"auto, (max-width: 430px) 100vw, 430px\" \/><\/figure>\n\n\n\n<p>Streaming ETL with Azure Data Factory and CDC &#8211;  Provisioning Azure Data Factory <\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this series we look at building a Streaming ETL with Azure Data Factory and CDC &#8211; Provisioning Azure Data Factory . This is Part 1, The rest of the series is below. Enabling CDC Setting up Audit Tables Provisioning Azure Data Factory Provisioning Azure Blog Storage Create Data Source Connection in ADF Create Incremental [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":822,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4,28,27,11,1],"tags":[104],"class_list":["post-1020","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analytics","category-azure","category-big-data","category-data-warehouse","category-uncategorized","tag-azure-data-factory"],"_links":{"self":[{"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/posts\/1020","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/comments?post=1020"}],"version-history":[{"count":2,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/posts\/1020\/revisions"}],"predecessor-version":[{"id":1083,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/posts\/1020\/revisions\/1083"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/media\/822"}],"wp:attachment":[{"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/media?parent=1020"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/categories?post=1020"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/jackofalltradesmasterofsome.com\/blog\/wp-json\/wp\/v2\/tags?post=1020"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}