jack of all trades master of some http://jackofalltradesmasterofsome.com/blog Consultant - Real Estate - Author - Business Intelligence Fri, 17 Mar 2023 18:36:38 +0000 en-US hourly 1 https://wordpress.org/?v=6.5.3 Decrypt and Save a Text or CVS with PGPY and Python http://jackofalltradesmasterofsome.com/blog/2023/03/17/decrypt-and-save-a-text-or-cvs-with-pgpy-and-python/ Fri, 17 Mar 2023 18:36:18 +0000 https://jackofalltradesmasterofsome.com/blog/?p=1321 Hopefully this helps someone to Decrypt and Save a Text or CVS file with PGPY and Python. Use Kleopatra and  OpenPGP to generate and save your key. You can obtain it by Exporting “Backup Secret Keys”.

Then run this code to decrypt and save the file in whatever location you need.

import pgpy

emsg = pgpy.PGPMessage.from_file('YOUR FILE.pgp')
key,_  = pgpy.PGPKey.from_file('YOUR PRIVATE KEY.asc')
with key.unlock('YOUR PGP PASSWORD'):
    print (key.decrypt(emsg).message)
    with open("my_file.txt", "wb") as binary_file:
   
        # Write bytes to file
        binary_file.write(key.decrypt(emsg).message)

That’s it! Keeping it short and simple.

]]>
Best Science-Themed Gifts: Get Your Nerd On http://jackofalltradesmasterofsome.com/blog/2022/12/20/best-science-themed-gifts-get-your-nerd-on/ Tue, 20 Dec 2022 00:21:37 +0000 https://jackofalltradesmasterofsome.com/blog/?p=1272 Are you struggling to find the perfect gift for the scientist in your life? Look no further! We’ve compiled a list of the best creative gift ideas that are sure to impress the science-loving person in your life. Here are our top 5 Best Science-Themed Gifts.

A periodic table throw blanket

  • This cozy blanket is not only functional, but it’s also educational! The periodic table of elements is featured prominently, making it the perfect gift for the chemistry enthusiast in your life
Best Science-Themed Gifts blanket
Periodic Table Throw Blanket

Custom Handmade Patent

  • Has your favorite scientist been awarded a patent. Help them show off their hard work with a custom patent designed for them to hang in their office or home.
Best Science-Themed Gifts patent
Custom Handmade Patent on Etsy

A science-themed cookbook:

  • If your scientist friend loves to cook, consider gifting them a cookbook filled with recipes that incorporate scientific principles. From molecular gastronomy to edible chemistry experiments, there are plenty of options to choose from.
Best Science-Themed Gifts books
The Food Lab: Better Home Cooking Through Science

A DIY terrarium kit

  • For the botanist or plant lover in your life, consider giving a DIY terrarium kit. This gift allows them to create their own miniature ecosystem and learn about plant care in the process.
Best Science-Themed Gifts
DIY Terrarium Kit

A Science-Themed Gifts board game

  • Board games make for a great gift, and there are plenty of options available with a science twist. From chemistry-themed games like “Elements” to biology-themed games like “Pandemic,” there’s something for every science enthusiast.
Best Science-Themed Gifts board game
Pandemic Board Game

5. Science fiction novel

  • Give the gift of a good read with a science fiction novel. From classic works by authors like Isaac Asimov to more modern offerings, there are plenty of options to choose from.

No matter what gift you choose, your scientist friend is sure to appreciate the thought and creativity you put into it. Happy shopping! Best Science-Themed Gifts

]]>
Get Data from NetSuite using Azure Data Factory http://jackofalltradesmasterofsome.com/blog/2022/05/17/get-data-from-netsuite-using-azure-data-factory/ Tue, 17 May 2022 16:25:29 +0000 https://jackofalltradesmasterofsome.com/blog/?p=1263 Get Data from NetSuite using Azure Data Factory. As Netsuite is an oracle product, getting data into your data warehouse can be a little tricky. Here is a helpful guide to get started.

Side Note: Want to learn SQL or Python for free. In less then 10 minutes a day and less than an hour total? Signup for my free classes delivered daily right to your email inbox for free!

Back to the article…

Helpful Links

https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_3994744300.html

https://www.netsuite.com/help/helpcenter/en_US/srbrowser/Browser2020_1/analytics/record/transactionLine.html

1.      Setup VM

Virtual Machine will be required since a server with the ability to install the correct ODBC drivers and a SQL Server to create the linked services will be needed. This will allow you to install an Integration runtime for Azure Data Factory to connect to and leverage the Linked Server

  1. Items to install on VM
    1. SQL Express Local DB
      1. Needed to test and verify ODBC connection to NetSuite and created Linked Server Connection
      1. SQL Server Express LocalDB – SQL Server | Microsoft Docs
      1. Install the SQL Server as a Mixed Authentication Mode
        1. Create a User on SQL Server that has rights to all databases and linked servers needed in the final steps
    1. SQL Server Management Studio
      1. Needed to test and verify Linked Server ODBC connection to NetSuite
    1. Azure Data Factory Integration Runtime
      1. Needed to connect a VM to Data Factory
      1. Download Microsoft Integration Runtime from Official Microsoft Download Center
    1. NetSuite ODBC Driver for Windows
      1. See next steps for download and setup instructions

2.      Setup the NetSuite ODBC driver for Windows

You will need to install the NetSuite ODBC drivers on the VM so that it can communicate with the NetSuite Servers

  1. This will be provided by your Admin or Via the NetSuite Portal
    1. You will need to obtain the server host, service port, service data source, account ID and Role ID from NetSuite as well as configure a user that has access to these settings.
  2. NetSuiteODBCDrivers_Windows64bit
  3. Navigate to your ODBC Settings in Windows and Add a new System DSN. If everything installed correctly you will now see a “NetSuite Drivers 64bit” to select from.
  4. In the Settings for this page, enter the Account ID, and Role ID from the setup instructions from Netsuite
  5. Once you select “Test Connect” you will be prompted to enter your credentials you configured in NetSuite.

3.      Setup a Linked Server Connection

With SQL Server Express and SQL Server Management Studio Setup, you can now create a Linked Server Connection to NetSuite.

  1. In the General tab, enter the following information:
    1. Linked server: NETSUITE (Name which you want to appear in the object explorer)
    1. Provider: Microsoft OLE DB Provider for ODBC Drivers
    1. Product name: NetSuite.com
    1. Data Source: The name of you 64-Bit ODBC Data Source from Previous Step. This was “NetSuite” in this example.
    1. Provider string: DSN=NetSuite.com
  • In the Security tab, select “Be made using this security context:”. For remote login, use your NetSuite login and password.
  • The linked server should now appear in the object explorer.
    • you can now query the tables with queries similar to “Select * from [NETSUITE].[Database].[Administrator].[ACCOUNT_ACTIVITY]”

4.      Set Up Integration Runtime in Azure Data Factory

Now that a linked server has been set up from your VM to the NetSuite box, you can now use this connection in Azure Data Factory. Since this is considered a on-premise server, you will need to install an Azure Integrated Runtime so that Azure Data Factory which is on the cloud can find and communicate with this VM.

5.      Setup and Install Integration Runtime on VM

You now will need to install the Integration Runtime on the VM and configure it so that it can find and communicate with the Azure Data Factory Service you configured in previous step

If all was done correctly, you will now see a valid connection inside Azure Data Factory

6.      Azure Data Factory Setup

Everything should now be setup and to move data from your NetSuite to your destination in Azure Data Factory

a.       Setup a Linked Service in Data Factory

b.      Create a Data Source

c.       Create a Pipeline and Move Data

]]>
Use Azure Blob Storage for SFTP http://jackofalltradesmasterofsome.com/blog/2022/03/08/use-azure-blob-storage-for-sftp/ Tue, 08 Mar 2022 15:55:29 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1244 This feature is in preview at time of writing and may be subject to change. Here is a quick guide on how to Use Azure Blob Storage Gen2 for SFTP for cheap serverless SFTP. Step by step guide on setup and access. By using local users, this allows you to create a series of SSH users and passwords for segmented folders and control access to your SFTP file storage.

Use Azure Blob Storage for SFTP

  1. Navigate to Subscriptions and select the subscription you want to activate the SFTP features against
  • Select “Preview Features” and then search for “SFTP”. Select “SFTP support in Azure Blob Storage and click Register.
  • Create a new Gen2 storage account in Azure. Use any name you need and set as Standard.
  • Under advanced setting you must Enable hierarchical namespace in the storage settings as well as Enable SFTP.

Setup SFTP in Azure Blob Storage

  1. In your storage, create a new container.
  • Under Settings you now should have a SFTP option to navigate too.
  • Create a new local user and select “SSH Password”
  • Next give this user the correct permissions to the containers and folders. Once completed the password will be shared with you.

Connect to Azure SFTP in Cleint

  1. Open a SFTP tool such as WinSCP or FileZilla
  2. For the host name enter the blob storage name, and user name and password from the previous steps.
    1. storagename.blob.core.windows.net
  3. Hit connect. You may be prompted for approval of a key. Once you accept, you should now have access to the SFTP on blob storage!

For more information be sure to check out the rest of this blog for azure tips and tricks and business intelligence tutorials. Azure belongs to the Microsoft suite of products and for more details please see the portal site on Microsoft’s documentation.

]]>
Connect to Azure SQL using Active Directory and Grant Access outside Organization http://jackofalltradesmasterofsome.com/blog/2022/02/13/connect-to-azure-sql-using-active-directory-and-grant-access-outside-organization/ Sun, 13 Feb 2022 20:08:58 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1235 Learn how to Connect to Azure SQL using Active Directory and Grant Access outside users outside of your Organization. This comes in handy when you have a user that sits outside of your organization and they need to log into a SQL environment you have provisioned for them. They will need to be invited as a guest user and then the appropriate access setup so they can login with out needed a hard coded SQL authentication login which creates risk.

1. Allowing Active Directory to Authenticate to SQL Server

  • Navigate to your server and Click on “Not Configured” for Active Directory admin inside the Server you are interested in allowing AD access to.
  • Set Admin and set your Azure Portal account as admin.
  • Leave the check box for Azure Active Directoy authentication only set to unchecked. This way you can continue to use defined crecentianls when needed.
  • Be sure to click Save to save changes.
  • Head back to SSMS and log in using Azure Active Directory – Universal with MFA or use the correct setting approved by your administrator.

2. Invite User to Your Organization

  • Add a new Guest User from the User section in your Azure Portal
  • User will get an email to activate their account

3. Adding a External AD user to your Database

  • Create a new Query window and run the following commands. and edit the user name and business to the users created in step 2

CREATE USER [username.com#EXT#@business.onmicrosoft.com] FROM EXTERNAL PROVIDER;

ALTER ROLE [db_datareader] ADD MEMBER [username.com#EXT#@business.onmicrosoft.com];

4. Conclusion

And that is all you need to Connect to Azure SQL using Active Directory and Grant Access outside Organization.

]]>
Best Men’s Fashion Blog Links http://jackofalltradesmasterofsome.com/blog/2021/12/26/beset-mens-fashion-blog-links/ Sun, 26 Dec 2021 19:40:27 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1223
  • Cheap Business Casual Capsule Wardrobe for Men
  • The Capsule Wardrobe Guide for men 2022
  • Best Male Fashion Watches on Amazon Under $50
  • The Guide on How to Care for Men’s Sweaters
  • The Basic Bastard Wardrobe Guide
  • Basic Layering Tips for Men
  • Affordable Wedding Bands For Men: The Best Cheap Wedding Bands
  • Ideal Men’s Haircut for Every Face Type
  • Beset Men’s Fashion Blog Links

    ]]>
    Copy data to and from Salesforce with Azure Data Factory http://jackofalltradesmasterofsome.com/blog/2021/06/04/copy-data-to-and-from-salesforce-with-azure-data-factory/ Fri, 04 Jun 2021 22:56:07 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1188 Salesforce is a key tool in many organizations, so it is important to be sure the data in the tool is tightly integrated to other systems and moved to a data warehouse for analytics. Here is how to Copy data to and from Salesforce with Azure Data Factory.

    Side Note: Want to learn SQL or Python for free. In less then 10 minutes a day and less than an hour total? Signup for my free classes delivered daily right to your email inbox for free!

    Now back to the article…

    1. Log into Salesforce as an admin account and navigate to settings.
    • Reset your security token on the left hand side menu under “My Personal Information”. This will email you a token that will be used in later steps.
    • Navigate to “Setup”
    • Navigate to “Setup -> Permission Sets -> (name) -> System permissions” and edit the “Security Center Integration User” and then the “System Permissions”
    • Verify that Enable “API Enabled”

    Create a Linked Service to Salesforce in Azure Data Factory

    1. Create a new Linked Service for Azure
    2. Obtain your environment name from Salesforce and log in with your credentials and service key from the previous steps. You should be able to now test your connection to verify it works. you will need the security token that was emailed to you as well in this step.

    Create a Azure Data Factory Dataset to Salesforce

    1. Create a new dataset in ADF and Select Salesforce
    • Point to the linked Service you created for Salesforce in the previous step. Select the table you wish to interact with. In this case, it will be the Contact table. Hit preview data to ensure the connection is working and set up correctly. Name the dataset and Save to complete.

    Create a pipeline to move Data

    The last step is the easy part. Create copy commands in the Pipelines to move data in and out of Salesforce. This can be used to update contacts, accounts, leads and other values automatically form other systems and processes.

    Copy data to and from Salesforce with Azure Data Factory

    Get Salesforce here

    Get Azure here

    Enjoying Learning about Azure and Azure Data Factory? Check out some of my online classes here or how to set up a run time to move data on-prem.

    ]]>
    Power BI to Yardi Real Estate Data http://jackofalltradesmasterofsome.com/blog/2021/02/16/power-bi-to-yardi-real-estate-data/ Tue, 16 Feb 2021 14:07:59 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1142 Most of the basic CRE tools have a deep suite of reports and analytics baked in with standard reporting, but everyone’s business is unique, and everyone’s reporting and integration needs are slightly different. Data from a single property ERP usually needs to be combined with other data points from other systems such as Salesforce, Accounting Software, Construction Software and even static excel files. This is why most CRE organizations must use tools such as Power BI to conduct analytics outside of the core tools. With Yardi, the data is usually harder to get too and there is no ODBC connection, it is doable just with a few extra steps. Let’s review how to Connect Power BI to Yardi Real Estate Data.

    If the contents of this post are a outside of your wheelhouse, don’t worry, this is why I started CREXchange.io! We have free Commercial Real Estate Power BI Reports to help clients get up and off the ground with Yardi Data Integration and Advanced Power BI Custom Reports to help run your CRE business. Contact us there for more details or a demo!

    1.     Export and Restore your Yardi Database backup locally

    The first step to be able to analyze your data is going to involve getting access to your data. Most SaaS platforms including Yardi do not allow direct access to the database or data model but most do allow for you to export this data to your local instance to do as you please. You will need to build a process that imports and restores the back up database copy to a local SQL Server. This process can be tricky, but it can be done. See our whitepapers below to get more informational on this step. You will require a SQL server license and a virtual machine to host the back up.

    Connect Power BI to Yardi Real Estate Data
    Get our free Power BI reports here!

    2.     Transform the Yardi data into a usable data model

    There are thousands of tables in the database, most which you will never need. Extract the handful of tables you need for your data that pertain to occupancy, revenue, properties, tenants, units and accounting and others you may need and organize this data into a data model that can be easily sliced and diced. The model can be complex so be sure to use the right ID and ensure you have similar rules and logic tied to the type id’s and param to be sure you are getting the right data for the right timeline. If you need help with this piece, reach out for more help to get access to our proprietary real estate data model.

    3.     Connect Power BI do your Data Model

    Once you have your data model created and a process to refresh and reload this data nightly, you are ready to connect your visualization tools to being building cool new reports, analytics, and visualizations. Using Power BI’s default SQL server connector, you can connect to your database weather it is on the cloud or on-prem and import the data into Power BI. Your data model should have the foreign keys establish so that it is inherited and be able to create reports and dashboards. Here is an example of one of our Reports that enables tenant and leasing data.

    Connect Power BI to Yardi Real Estate Data
    Get our free Power BI reports here!

    4.     Connect Power BI to other data sources and combine to your Yardi real estate data

    Having Yardi or Real Estate data is never enough. Now with Power BI, you can connect to numerous data sources in your business to begin to combine data too create analytics across all your business units and lifecycle of the real estate business.

    Step 1 and 2 are the more technical heavy steps of the process. In previous post I have written white papers about getting data from Yardi nightly or even hourly using transaction log shipping, both available for download.

    Connect Power BI to Yardi Real Estate Data
    Get our free Power BI reports here!

    If the contents of this post are a outside of your wheelhouse, don’t worry, this is why I started CREXchange.io to help clients get up and off the ground with Yardi Data Integration and Advanced Power BI Custom Reports to help run your CRE business. Our reports are free with integration! Contact us there for more details or a demo!

    ]]>
    Visual Studio Database projects to Deploy Azure Synapse Pool http://jackofalltradesmasterofsome.com/blog/2021/02/15/visual-studio-database-projects-to-deploy-azure-synapse-pool/ Mon, 15 Feb 2021 20:36:56 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1129 Visual Studio Database projects to Deploy Azure Synapse Pool

    Side Note: Want to learn SQL or Python for free. In less then 10 minutes a day and less than an hour total? Signup for my free classes delivered daily right to your email inbox for free!

    Now back to the article…

    Get Visual Studio 2019

    1. Download and install Visual Studio 2019 Community Edition
      1. https://visualstudio.microsoft.com/

    b) Verify Data Storage and Processing and make sure all updates are up to date.

    Create the database project

    1. Create a new project in Visual Studio
    • Create a new SQL Server Database project.
    • To add your first item, select from the new solution and select Add->New Item
    • From the list of items, select “Table (Data Warehouse) as this will allow for slightly different create table statements with columnstore indexes.
    • Add your code to the editor. Some items may still show an error but it will not be an issue. Save when ready.

    Update the Target Platform

    1. Right click the solution and select properties
    • Set the Target Platform to Microsoft Azure SQL Data Warehouse and save

    Publishing Changes to Server

    1. Right click the project in solutions explorer and select “Publish”
    2. Select the Azure SQL Data warehouse as the target database platform and select “Publish”

    Visual Studio Database projects to Deploy Azure Synapse Pool

    ]]>
    Automatically pausing and resuming Azure Workspace Synapse Pool Using Azure Data Factory http://jackofalltradesmasterofsome.com/blog/2021/02/10/automatically-pausing-and-resuming-azure-workspace-synapse-pool-using-azure-data-factory/ Wed, 10 Feb 2021 20:59:10 +0000 http://jackofalltradesmasterofsome.com/blog/?p=1100 Automatically pausing and resuming Azure Workspace Synapse Pool Using Azure Data Factory.

    1. Create a new Azure Data Factory Pipeline
    2. Add the Web Task and name it “PauseDW”
    • Set the setting to “POST”
    • In the advanced section, set the method to “MSI” and the resource to “https://management.core.windows.net”
    • In your IAM Security for your workspace add the contributor role for your data factory.
    • Debug your pipeline and your service should now pause. Update the API call to the replace the word “pause” with “resume” to have it work the other way around.
    • Add these steps to triggers at specific times a day to run to turn your resources on and off.

    Automatically pausing and resuming Azure Workspace Synapse Pool Using Azure Data Factory.

    ]]>