Skip to content
Business Central Musings

Business Central Musings

BC/NAV – the easy way

  • About
  • BC/NAV Community

Category: copy files

Copy files from Azure Blob Storage to File System (using Azure CLI)

In the previous blog post I’ve described the usage of Power Automate when copying files from Azure Blob Storage somewhere locally.

Today, we’ll focus on pursuing the same objective, but this time using a different tool: Azure CLI.

Azure Command-Line Interface (CLI) is a cross-platform command-line tool that can be installed locally on Windows computers. You can get this tool from here.

Almost anything you can do in Azure Portal GUI you can achieve with Azure CLI.

Visit this list to see what Azure CLI can manage.

In my previous blog I showed that, as soon as a new file makes it in the Azure Storage container, the file is pushed locally via a Power Automate flow.

Today, I’ll describe the steps I took to generate a script that runs under Task Scheduler shell and pulls daily files(blobs) from Azure Storage Blob Container locally at a scheduled time.

In VS Code(or your favorite scripting environment) open a terminal window.

  • First, from the console, login in your azure portal:
az login

Output:

                                                                                                                                            
The default web browser has been opened at https://login.microsoftonline.com/common/oauth2/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.
You have logged in. Now let us find all the subscriptions to which you have access...
[
  {
    "cloudName": "AzureCloud",
    "homeTenantId": "80e51828-6b27-4102-9478-a14375194b20",
    "id": "48b7fb28-08c7-444f-8e6b-7615735db9b2",
    "isDefault": true,
    "managedByTenants": [],
    "name": "Azure subscription 1",
    "state": "Enabled",
    "tenantId": "80e51828-6b27-4102-9478-a14375194b20",    
    "user": {
      "name": "admin@CRMbc691816.onmicrosoft.com",
      "type": "user"
    }
  }
]
  • Then, in the console, create a service principal with the role Owner:
az ad sp create-for-rbac -n "SVservicePrincipal" --role Owner --create-cert

In plain English this means, create in Azure a service principal that is role based access control with the assigned role of Owner. Also, create a certificate for this service principal.

The result:

Creating 'Owner' role assignment under scope '/subscriptions/48b7fb28-08c7-444f-8e6b-7615735db9b2'
The output includes credentials that you must protect. Be sure that you do not include these credentials in your code or check the credentials into your source control. For more information, see https://aka.ms/azadsp-cli
'name' property in the output is deprecated and will be removed in the future. Use 'appId' instead.
Please copy C:\Users\svir\tmpa41iqdm1.pem to a safe place. When you run 'az login', provide the file path in the --password argument  
  "appId": "a448d6fc-f8b7-4847-9bf7-93f56bc7451f",
  "displayName": "SVservicePrincipal",
  "fileWithCertAndPrivateKey": "C:\\Users\\svir\\tmpnow6fl5e.pem",
  "name": "a448d6fc-f8b7-4847-9bf7-93f56bc7451f",
  "password": null,
  "tenant": "80e51828-6b27-4102-9478-a14375194b20"
  • With the service principal created let’s logout from Azure:
az logout
  • let’s log back in using the service principal created and attempt to copy locally some blobs from an Azure Storage Blob Container:

Log in part:

az login --service-principal --username "a448d6fc-f8b7-4847-9bf7-93f56bc7451f" --password 'C:\\Users\\svir\\tmpnow6fl5e.pem' --tenant "80e51828-6b27-4102-9478-a14375194b20"

Output:


{
  "appId": "a74da28b-5dfd-49eb-b2a3-7dfdc8279998",
  "displayName": "SVservicePrincipal1",
  "fileWithCertAndPrivateKey": "C:\\Users\\svir\\tmpa41iqdm1.pem",
  "name": "a74da28b-5dfd-49eb-b2a3-7dfdc8279998",
  "password": null,
  "tenant": "80e51828-6b27-4102-9478-a14375194b20"
}

Copy part:

az storage azcopy blob download -c vendorlist --account-name svflorida -s * -d "c:\\temp\\localsvflorida" --recursive

Outcome:


INFO: Scanning...
INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder support
INFO: azcopy.exe: A newer version 10.11.0 is available to download


Job 4b2742c1-1e57-a44a-6a24-a40e6b60f01f has started
Log file is located at: C:\Users\svir\.azcopy\4b2742c1-1e57-a44a-6a24-a40e6b60f01f.log

0.0 %, 0 Done, 0 Failed, 3 Pending, 0 Skipped, 3 Total, 


Job 4b2742c1-1e57-a44a-6a24-a40e6b60f01f summary
Elapsed Time (Minutes): 0.0334
Number of File Transfers: 3
Number of Folder Property Transfers: 0
Total Number of Transfers: 3
Number of Transfers Completed: 3
Number of Transfers Failed: 0
Number of Transfers Skipped: 0
TotalBytesTransferred: 884
Final Job Status: Completed

We can see below the blobs located in the vendorlist azure storage container are now downloaded locally in my folder C:\Temp\LocalSVFlorida.

  • last step is automatization. I will be using Task Scheduler in Windows.
    • I will create a basic task that will execute the login part and the copy part wrapped in a powershell script.
    • First open Task Scheduler and create a basic task:
      • give it a name
      • schedule it at the desired time
      • when triggered the action is to execute a powershell script:

And below is the script:

az login --service-principal --username "a448d6fc-f8b7-4847-9bf7-93f56bc7451f" --password 'C:\\Users\\svir\\tmpnow6fl5e.pem' --tenant "80e51828-6b27-4102-9478-a14375194b20"

az storage azcopy blob download -c vendorlist --account-name svflorida -s * -d "c:\\temp\\localsvflorida" --recursive

To test, upload a new file in your azure blob container and schedule your task to run.

Conclusion:

We saw two ways to copy blobs/files from an azure storage container locally.

First method was using a Microsoft template flow, the second creating a powershell script and schedule it to run on a regular basis.

In the next blog will see how we can generate extracts in Business Central and store them in the Azure Storage Blob Containers.

Hope this helps!

You can find the script here.

Silviu az, Azure CLI, copy files, create service principal, login with service principal, script to copy files from azure, Task Scheduler 1 Comment August 28, 2021August 29, 2021 4 Minutes

Recent Posts

  • An on-demand subscriber with AL
  • How to get disk usage per folder
  • Say Hello to the new “Performance Profiler”
  • Knowing your Business Central data using the Page Inspection
  • BC20 in SaaS (preview)

Recent Comments

Miuru Mahagamage on Business Central On-Premise in…
Say Hello to the new… on Say Hello to the new “Pe…
Getting started with… on Getting started with Snapshot…
There is a new Posti… on There is a new Posting Preview…
Sunday Bokanovich on How to generate Azure Containe…

Archives

  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • August 2020
  • January 2020
  • October 2019
  • January 2019
  • December 2018
  • January 2018
  • October 2017
  • September 2017
  • August 2017
  • April 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016
  • September 2016

Categories

  • .NET
  • Access Key
  • AL
  • Application Testing
  • az
  • Azure Business Central
  • Azure CLI
  • Azure Functions
  • Azure Portal
  • Azure powershell
  • Azure Storage
  • Azure Storage Blob Containers
  • Azure Storage Table
  • Background Error Check
  • Background Tasks
  • batch
  • BcContainerHelper
  • BindSubscription
  • Business Central headlines in role center
  • Business Central On-Premise installation
  • Chart of accounts overview
  • Cloud Migration
  • Cloud Migration Tool
  • Code Analyzers
  • company backup
  • company data restore
  • Container
  • copy files
  • create service principal
  • Data Exchange
  • Data Management
  • default line type
  • Deployment Templates
  • Dimensions
  • Docker
  • Dynamics 365 Business Central
  • Dynamics GP
  • Dynamics NAV
  • Dynamics NAV backup
  • Dynamics365 For Financials
  • Event Subscriber
  • Excel
  • Extension Permissions
  • Field Selection codeunit
  • flow
  • Get-WmiObject
  • Git
  • Global Dimensions
  • GP to BC enhancements
  • GP to BC migration enhancements
  • hardware requirements
  • import
  • lists in AL
  • Local Business Central
  • login with service principal
  • MB-800
  • Microsoft Certification
  • Microsoft Flow
  • Microsoft System
  • NAS service
  • nassql
  • NAV 2009
  • NAV 2013
  • NAV 2015
  • NAV 2016
  • NAV 2017
  • NAV 2018
  • NAV extension
  • NAV Upgrades
  • NAVContainerHelper
  • NavigatePage
  • NAVX
  • New-BcContainerWizzard
  • New-NavContainer
  • New-NavContainerNewUser
  • OnResolveDateFilterToken
  • OnResolveTextFilterToken
  • pageextension
  • Positive Pay fields
  • Post Request
  • Posting Preview
  • Postman
  • Power Automate
  • Power Platform
  • PowerApps
  • PowerQuery
  • Report Layout
  • Report Layouts
  • Rest Client
  • Restful API
  • SaaS
  • Sandbox
  • SAS Token
  • script to copy files from azure
  • SelectLatestVersion
  • Shortcut Dimensions
  • Snapshot Debugging
  • software requirements
  • Task Scheduler
  • The windows account could not be mapped to a valid security identifier (sid)
  • UnBindSubscription
  • Uncategorized
  • UseRequestPage
  • Visual Studio Code
  • Web Services
  • web services
  • Word Layout
  • XML Buffer

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 292 other followers

Follow Business Central Musings on WordPress.com
  • RSS - Posts
Blog at WordPress.com.
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Business Central Musings
    • Join 292 other followers
    • Already have a WordPress.com account? Log in now.
    • Business Central Musings
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...