Thursday, 6 October 2022

CICD pipeline for Azure Blob storage with CDN - Part 3 Powershell to provision Azure infrastructure

Now the meat let's set up our AzureInfrastructure.ps1 script, open it in your ms Code editor, and let's start by setting up some input parameters and creating a simple function to create a resource group in azure for us.

Param (
  [Parameter()][String]$location,
  [Parameter()][String]$name,
  [Parameter()][String]$env)

#create resource group
function CreateResourceGroup {
    Param(
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)

    #setup resource Group name using prefix
    $rgName = ("rg-$name-$env").ToLower()

    #check if resource group already exists
    $resourceGroup = Get-AzResourceGroup `
        -Name $rgName `
        -ErrorAction SilentlyContinue
    if($resourceGroup){
        Write-host "Resource group '$rgName' already exists" -foregroundcolor yellow
        return $rgName
    }

    #create resource group
    try {
        Write-Host "Createing '$rgName' resource group in '$location'" -ForegroundColor Magenta
        $resourceGroup = New-AzResourceGroup `
            -Name $rgName `
            -Location $location
        Write-host "Resource group '$rgName' created  in '$location'" -foregroundcolor Green
        return $rgName
    }
    catch {
        #failed to create resourcec group  
         Write-error "Resource group '$rgName' NOT created" -ErrorAction Stop  
    }
}


$rgName = CreateResourceGroup $name $location $env

These will let our YAML file input parameters to our PowerShell which will make our lives easier in the long run, our powershell then will use those parameters to build our infrastructure. 

Now that we have parameterized our PowerShell script, let's jump back to our YAML file and pass those parameters to our script

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- dev
- test
- master

variables:
  ${{ if startsWith(variables['Build.SourceBranch'], 'refs/heads/') }}:
    branchName: $[ replace(variables['Build.SourceBranch'], 'refs/heads/', '') ]
  ${{ if startsWith(variables['Build.SourceBranch'], 'refs/pull/') }}:
    branchName: $[ replace(variables['System.PullRequest.TargetBranch'], 'refs/heads/', '') ]
  name: 'pav'
  location: 'westeurope'
  azureSubscription: 'Pavs Subscription(6e72246e-0000-0000-0000-000000000000)'

pool:
  vmImage: 'windows-latest'

steps:
- script: |
    echo BranchName = '$(branchName)'
    echo name = '$(name)'
    echo location = '$(location)'
    echo azureSubscription = '$(azureSubscription)'
  displayName: 'List YAML variables'

- task: AzurePowerShell@5
  inputs:
    azureSubscription: '$(azureSubscription)'
    ScriptType: 'FilePath'
    ScriptPath: '$(Build.SourcesDirectory)/AzureInfrastructure.ps1'
    ScriptArguments: >
      -location: $(location)
      -name: $(name)
      -env: $(branchName)
    errorActionPreference: 'continue'
    FailOnStandardError: true
    azurePowerShellVersion: 'LatestVersion'

with those two changes complete and SAVED, let's test our work, in theory all we have to do now, is commit our changes and push our code, Devops should pick them up, connect to azure and create our resource group.

the command we execute in our command line are

  1. Add our changes to our repo: git add .
  2. commit our changes : git commit -m 'a comment for our source control'
  3. push our changes: git push
and that's it, now we can go back to our azure devops portal to see if our pipeline ran correctly.



from the looks of our job, it was a success, now lets do one final check and login to our Azure portal and see if in fact we have a resource group called 'rg-pav-master'




and sure enough there it is our resource group, alright next step is to update our powershell to create a Blob storage in our resource group.

let's create a powershell function that does just that.

#create Blob storage account
function CreateStorageAcccount {
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)  
   
    $stName = "st$name$env".ToLower();

    $storageAccount = Get-AzStorageAccount `
        -ResourceGroupName $rgName `
        -Name $stName `
        -ErrorAction SilentlyContinue

    if($storageAccount){
        Write-host "Sorage account '$stName' already exists in resouce group '$rgName'" -foregroundcolor yellow
        return $stName
    }

     #create storage account
     try {
        Write-Host "Createing storage account '$stName' in resource group in '$rgname'" -ForegroundColor Magenta
        $storageAccount =  New-AzStorageAccount `
            -ResourceGroupName $rgName `
            -Name $stName `
            -Location $location `
            -SkuName Standard_LRS `
            -Kind BlobStorage `
            -AccessTier Hot
   
        Write-host "Storage account created" -foregroundcolor Green
        return $stName
    }
    catch {
        #failed to create storagte account  
         Write-error "storage account '$stName' NOT created in resouce group '$rgName'" -ErrorAction Stop  -foregroundcolor red
    }
}

notice the section in the try catch statement, here we create a new storage account, we set it to blob storage and to 'Locally-redundant storage (LRS)' because that's the cheapest option.

Now that we have our Blob storage set up, we have to configure a public container for our files, this is where they will be hosted.

#create Blob storage container
function CreateStorageContainer {
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$env,
        [Parameter(Mandatory=$true)][String]$stName)

    $strContainerName = "container-public-$name-$env".ToLower()

    $key = Get-AzStorageAccountKey `
        -ResourceGroupName $rgName `
        -Name $stName

    $stContext = New-AzStorageContext `
        -StorageAccountName $stName `
        -StorageAccountKey $key[0].Value
    $strContainer = Get-AzStorageContainer `
        -Name $strContainerName `
        -Context $stContext `
        -ErrorAction SilentlyContinue

    if($strContainer){
        Write-host "storage container '$strContainerName' already exists" -foregroundcolor yellow
        return $strContainer
    }

     # Create new storage container
     try {
        Write-Host "Creating storage container $strContainerName" -ForegroundColor Magenta
        $strContainer = New-AzStorageContainer `
            -Name $strContainerName `
            -Permission Blob `
            -Context $stContext
        Write-host "storage container '$strContainerName' created" -foregroundcolor Green
        return $strContainer
    }
    catch {
        #failed to create resource group  
        Write-error "CDN profile '$cdnProfileName' NOT created in '$rgName'" -ErrorAction Stop  
    }
}

with that set up, now we will start building our CDN, that is what is going to cache our images all over the world, so that if our site is hosted in West Europe, but our users are in Australia they will still receive optimal performance.

Let's create a cdn profile function.

# create a CDN profile
function CreateCdnProfile{
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)  

        $cdnProfileName = "cdn-profile-$name-$env"
        $cdnProfile = Get-AzCdnProfile `
            -ProfileName $cdnProfileName `
            -ResourceGroupName $rgName `
            -ErrorAction SilentlyContinue

        if($cdnProfile){
            Write-host "CDN '$cdnProfileName' already exists in resource group '$rgName'" -foregroundcolor yellow
            return $cdnProfileName
        }

        # Create a new cdn profile
        try {
            Write-Host "Createing cdn profile $cdnProfileName in resource group in $rgname" -ForegroundColor Magenta
            $cdnProfile =  New-AzCdnProfile `
                -ProfileName $cdnProfileName `
                -ResourceGroupName $rgName `
                -Sku Standard_Microsoft `
                -Location $location
            Write-host "Cdn profile '$cdnProfileName' created in '$rgName'" -foregroundcolor Green
            return $cdnProfileName
        }
        catch {
            #failed to create resourcec group  
            Write-error "CDN profile '$cdnProfileName' NOT created in '$rgName'" -ErrorAction Stop  
        }
}

With that complete, we now create a cdn endpoint.

#create a cdn endpoint
function CreateCdnEndPoint{
    param (
        [Parameter(Mandatory=$true)][String]$cdnProfileName,
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$stName,
        [Parameter(Mandatory=$true)][String]$env)  

    $cdnEndPointName = "cdn-endpoint-$name-$env-$location"
    Write-Host "-ResourceGroupName $rgName -ProfileName $cdnProfileName -Name $cdnEndPointName "
   
   $cdnEndpoint = Get-AzCdnEndpoint `
        -ResourceGroupName $rgName `
        -ProfileName $cdnProfileName `
        | Where-Object {$_.Name -eq $cdnEndPointName}

   if($cdnEndpoint){
       Write-host "cdn endpoint '$cdnEndpointName' already exists in resource group $rgName" -foregroundcolor yellow
       return $cdnEndpoint
   }

    # Create new cdn endpoint
    try {
        Write-Host "Createing CDN endPoint $cdnEndPointName in resource group $rgName" -ForegroundColor Magenta
        $originHostHeader = "$stName.blob.core.windows.net"
       
        $origin = @{
            Name = $originHostHeader.Replace('.','-')
            HostName = $originHostHeader
        };
        New-AzCdnEndpoint `
            -ResourceGroupName $rgName `
            -ProfileName $cdnProfileName `
            -OriginHostHeader  $originHostHeader `
            -Name $cdnEndPointName `
            -Location $location `
            -OptimizationType "GeneralWebDelivery" `
            -Origin $origin
           
        Write-host "cdn endpoint '$cdnEndPointName' created in '$rgName'" -foregroundcolor Green
        return $cdnEndpoint
    }
    catch {
        #failed to create cdn endpoint
        Write-error "CDN endpoint '$cdnEndPointName' NOT created in '$rgName'" -ErrorAction Stop  
    }  
}

With all of our function written let's call them at the end of our script.

$rgName = CreateResourceGroup $name $location $env
$stName = CreateStorageAcccount $name $rgName $location $env
$stContainer = CreateStorageContainer $name $rgName $env $stName
$cdnProfileName = CreateCdnProfile $name $rgName $location $env
CreateCdnEndPoint $name $rgName $location $stName $cdnProfileName $env

and for good measure, here's our script in it's entirety 

Param (
    [Parameter()][String]$location,
    [Parameter()][String]$name,
    [Parameter()][String]$env)

    $name= 'pav'
    $location= 'westeurope'
    $env = "master"
    $tenantId= '00000000-0000-00000-00000-0000000000000'
    $subscriptionId= '000000000-00000-0000-0000-000000000000'
#create resource group
function CreateResourceGroup {
    Param(
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)

    #setup resource Group name using prefix
    $rgName = ("rg-$name-$env").ToLower()

    #check if resource group already exists
    $resourceGroup = Get-AzResourceGroup `
        -Name $rgName `
        -ErrorAction SilentlyContinue

    if($resourceGroup){
        Write-host "Resource group '$rgName' already exists" -foregroundcolor yellow
        return $rgName
    }

    #create resource group
    try {
        Write-Host "Createing '$rgName' resource group in '$location'" -ForegroundColor Magenta
        $resourceGroup = New-AzResourceGroup `
            -Name $rgName `
            -Location $location
        Write-host "Resource group '$rgName' created  in '$location'" -foregroundcolor Green
        return $rgName
    }
    catch {
        #failed to create resourcec group  
         Write-error "Resource group '$rgName' NOT created" -ErrorAction Stop  
    }
}

#create Blob storage account
function CreateStorageAcccount {
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)  
   
    $stName = "st$name$env";
   
    Write-Host "Create storage account with $stName $rgName $env"

    $storageAccount = Get-AzStorageAccount -ResourceGroupName $rgName -Name $stName -ErrorAction SilentlyContinue

    if($storageAccount){
        Write-host "Sorage account '$stName' already exists in resouce group '$rgName'" -foregroundcolor yellow
        return $stName
    }

     #create storage account
     try {
        Write-Host "Createing storage account '$stName' in resource group in '$rgname'" -ForegroundColor Magenta
        $storageAccount =  New-AzStorageAccount `
            -ResourceGroupName $rgName `
            -Name $stName `
            -Location $location `
            -SkuName Standard_LRS `
            -Kind BlobStorage `
            -AccessTier Hot
   
        Write-host "Storage account created" -foregroundcolor Green
        return $stName
    }
    catch {
        #failed to create storagte account  
         Write-error "storage account '$stName' NOT created in resouce group '$rgName'" -ErrorAction Stop  -foregroundcolor red
    }
}

#create Blob storage container
function CreateStorageContainer {
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$env,
        [Parameter(Mandatory=$true)][String]$stName)

    $strContainerName = "container-public-$name-$env".ToLower()
    Write-Host "$strContainerName $rgName $env $stName"
    $key = Get-AzStorageAccountKey `
        -ResourceGroupName $rgName `
        -Name $stName

    $stContext = New-AzStorageContext `
        -StorageAccountName $stName `
        -StorageAccountKey $key[0].Value

    $strContainer = Get-AzStorageContainer `
        -Name $strContainerName `
        -Context $stContext `
        -ErrorAction SilentlyContinue

    if($strContainer){
        Write-host "storage container '$strContainerName' already exists" -foregroundcolor yellow
        return $strContainer
    }

     # Create new storage coantainer
     try {
        Write-Host "Createing storage container $strContainerName" -ForegroundColor Magenta
        $strContainer = New-AzStorageContainer `
            -Name $strContainerName `
            -Permission Blob `
            -Context $stContext
        Write-host "storage container '$strContainerName' created" -foregroundcolor Green
        return $strContainer
    }
    catch {
        #failed to create resourcec group  
        Write-error "CDN profile '$cdnProfileName' NOT created in '$rgName'" -ErrorAction Stop  
    }
}

# create a CDN profile
function CreateCdnProfile{
    param (
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$env)  

        $cdnProfileName = "cdn-profile-$name-$env"
        $cdnProfile = Get-AzCdnProfile `
            -ProfileName $cdnProfileName `
            -ResourceGroupName $rgName `
            -ErrorAction SilentlyContinue

        if($cdnProfile){
            Write-host "CDN '$cdnProfileName' already exists in resource group '$rgName'" -foregroundcolor yellow
            return $cdnProfileName
        }

        # Create a new cdn profile
        try {
            Write-Host "Createing cdn profile $cdnProfileName in resource group in $rgname" -ForegroundColor Magenta
            $cdnProfile =  New-AzCdnProfile `
                -ProfileName $cdnProfileName `
                -ResourceGroupName $rgName `
                -Sku Standard_Microsoft `
                -Location $location
            Write-host "Cdn profile '$cdnProfileName' created in '$rgName'" -foregroundcolor Green
            return $cdnProfileName
        }
        catch {
            #failed to create resourcec group  
            Write-error "CDN profile '$cdnProfileName' NOT created in '$rgName'" -ErrorAction Stop  
        }
}

#create a cdn endpoint
function CreateCdnEndPoint{
    param (
        [Parameter(Mandatory=$true)][String]$cdnProfileName,
        [Parameter(Mandatory=$true)][String]$name,
        [Parameter(Mandatory=$true)][String]$rgName,
        [Parameter(Mandatory=$true)][String]$location,
        [Parameter(Mandatory=$true)][String]$stName,
        [Parameter(Mandatory=$true)][String]$env)  

    $cdnEndPointName = "cdn-endpoint-$name-$env-$location"
    Write-Host "-ResourceGroupName $rgName -ProfileName $cdnProfileName -Name $cdnEndPointName "
   
   $cdnEndpoint = Get-AzCdnEndpoint `
        -ResourceGroupName $rgName `
        -ProfileName $cdnProfileName `
        | Where-Object {$_.Name -eq $cdnEndPointName}

   if($cdnEndpoint){
       Write-host "cdn endpoint '$cdnEndpointName' already exists in resource group $rgName" -foregroundcolor yellow
       return $cdnEndpoint
   }

    # Create new cdn endpoint
    try {
        Write-Host "Createing CDN endPoint $cdnEndPointName in resource group $rgName" -ForegroundColor Magenta
        $originHostHeader = "$stName.blob.core.windows.net"
       
        $origin = @{
            Name = $originHostHeader.Replace('.','-')
            HostName = $originHostHeader
        };
        New-AzCdnEndpoint `
            -ResourceGroupName $rgName `
            -ProfileName $cdnProfileName `
            -OriginHostHeader  $originHostHeader `
            -Name $cdnEndPointName `
            -Location $location `
            -OptimizationType "GeneralWebDelivery" `
            -Origin $origin
           
        Write-host "cdn endpoint '$cdnEndPointName' created in '$rgName'" -foregroundcolor Green
        return $cdnEndpoint
    }
    catch {
        #failed to create cdn endpoint
        Write-error "CDN endpoint '$cdnEndPointName' NOT created in '$rgName'" -ErrorAction Stop  
    }  
}

Register-AzResourceProvider -ProviderNamespace "Microsoft.Cdn"

$rgName = CreateResourceGroup $name $location $env
$stName = CreateStorageAcccount $name $rgName $location $env
$stContainer = CreateStorageContainer $name $rgName $env $stName
$cdnProfileName = CreateCdnProfile $name $rgName $location $env
$cdnEndPoint = CreateCdnEndPoint $cdnProfileName $name $rgName $location $stName $env


we are done, provisioning our azure infrastructure, now if you do a git add . , git commit -m 'comment', and a git push, our Continuous Integration pipeline, should execute our script and build our Azure infrastructure.

there is one problem, the script will fail to create an endpoint, you'll have to do it manually.