Deploy Azure ARM Template with Github Actions Pipeline and Powershell

In this article, I will walk through my process of creating a repository containing an Azure ARM template, parameters file, pester tests, and Github Workflow yaml file. The point of this is to showcase a method for deploying your Azure resources as Infrastructure As Code.

In this article, I won’t go in-depth on how to create an ARM Template and Parameters file. Instead, I will go deeper into how I have set up the repository, how I have orchestrated my pipeline, and how you can utilize Pester for testing your Azure ARM Templates and resources.

Organizing The Repository

First of all, you can find the repository I am showcasing on my Github Account Here. Feel free to reuse any code you want to.

Second of all, I will shortly just go over the Resources I want to deploy with this repository. This template is designed to create the following resources:

  • Storage Account
  • Log Analytics Workspace
  • AutomationAccount
  • Azure KeyVault
  • Add Secrets from a Service Principal into the KeyVault

The Service Principal will be created through Powershell, I will explain this later in the article

How I Organize the Repo

I organize the repository in a very simple layout with folders containing files according to their function

Root\
|
|__.github\workflows\
|
|__parameters\
|
|__scripts\
|
|__templates\
|
|__tests\
|
|__LICENSE
|__README.md

parameters\

This folder contains the parameters file for the ARM Template.

scripts\

The scripts folder contains scripts for running or support the pipeline, in this case, it contains the script run.ps1 which will primarily run the pipeline actions and deploy things, such as Service Principal and Resource Group

templates\

The templates folder contains the ARM templates. In this case, it only contains the single ARM template azuredeploy.json, but if needed you could deploy multiple templates and store them here.

tests\

The tests folder will contain all pester tests you create for quality assurance for your deployment

Creating the Run.ps1 Script

For actually deploying the resources I have chosen to use Powershell. You could also deploy your resources in Github Actions with the Action: azure/arm-deploy@v1.

You can read more about this method on Microsoft Docs

The reason why I chose to use Powershell is for making it easy to transfer the pipeline to a different site such as Azure DevOps. And I like doing this with Powershell.

The Purpose of the script

The run.ps1 script has a couple of purposes.

  • Connect to Azure through Powershell
  • Create a new Resource Group
  • Create a new Service Principal
  • Deploy the Azure resources stated in the ARM template

Connecting to Azure through Powershell

To deploy the resources with Powershell I will need to connect with Azure. For connecting to Azure and to deploy the resources later on I will need to separate Powershell modules, Az.Accounts and Az.Resources.

The first thing I want the script to do is to check for these two modules. If they don’t exist in the environment then download them

Write-Verbose -Message "Checking and Installing Azure Powershell Module"
if (-not(Get-Module -Name Az.Accounts -ListAvailable)){
	Write-Warning "Module 'Az.Accounts' is missing or out of date. Installing module now."
	Install-Module -Name Az.Accounts, Az.Resources -Scope CurrentUser -Force -AllowClobber
}

The next thing I want to do is to connect to Azure.

Write-Verbose -Message "Connecting to Azure"
$ServicePrincipalPassword = ConvertTo-SecureString -AsPlainText -Force -String $ServicePrincipalPass
$azureAppCred = New-Object System.Management.Automation.PSCredential ($ServicePrincipalName,$ServicePrincipalPassword)
Connect-AzAccount -ServicePrincipal -Credential $azureAppCred -Tenant $tenantId -Subscription $SubscriptionId

The variables:

  • $ServicePrincipalPass
  • $ServicePrincipalName

Will be defined in the parameters for the script

Deploying the Resource Group to Azure

I want the script to deploy a new resource group in Azure, and if the resource group already exists, then just continue. The reason for this is that this way I can utilize the pipeline for creating completely new Azure resources, but also update already existing resources.

To deploy a new resource group I will run the following:

if(!(Get-AzResourceGroup | Where-Object ResourceGroupName -eq $ResourceGroupName)){
	try {
		Write-Verbose -Message "Creating the Azure Resource Group: $($ResourceGroupName)"
		New-AzResourceGroup -Name $ResourceGroupName -Location $ResourceLocation -Force -Confirm:$false 
	}
	catch {
		Write-Error "$($_)"
		Exit
	}
}
else {
	Write-Verbose "Resource Group already Exists, updating resource"
}

This snippet will check your Azure environments if the resource group already exists, and if not then deploy a new resource group.

The Variables:

  • $ResourceGroupName
  • $ResourceLocation

Will be defined by the script parameters

 Creating a new Service Principal

I want the pipeline to deploy a new Azure AD Service Principal and grant it contributor rights to the resource group created. The point of this is that when the deployment is complete you would have a working Service Principal with rights to the resource group, and the credentials stored in the Azure KeyVault.

if(!(Get-AzADServicePrincipal | Where-Object DisplayName -eq $NewServicePrincipalName)) {
	Write-verbose -Message "Creating Service Principal"
	$sp = New-AzADServicePrincipal -DisplayName $NewServicePrincipalName -Scope "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroupName)" -Role "Contributor"
	$spSecret = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
		[System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
			$sp.Secret
		)
	)
}
else {
	Write-Verbose -Message "Service Principal Already exists, removing the old and creating new"
	$appId = (Get-AzADServicePrincipal | Where-Object DisplayName -eq $NewServicePrincipalName | Select-Object -ExpandProperty ApplicationId).Guid
	Remove-AzADServicePrincipal -ApplicationId $appId -Force

	Write-verbose -Message "Creating New Service Principal"
	$sp = New-AzADServicePrincipal -DisplayName $NewServicePrincipalName -Scope "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroupName)" -Role "Contributor"
	$spSecret = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
		[System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
			$sp.Secret
		)
	)
}

The variables:

  • $NewServicePrincipalName
  • $SubscriptionId

Will be defined  by the script parameters

Deploying the Azure ARM Template

To deploy the ARM template I will use the Powershell cmdlet New-AzResourceGroupDeployment. For deploying the template I will give the deployment a name, defined by the parameter [-Name]. I will define the path for my template file with the [-TemplateFile] parameter. I will define the path to my parameters file for the template by the [-TemplateParameterFile].  The parameters: tenantId, secretName, secretValue and objectId is for passing to the parameters in the ARM Template, and will be used for deploying the resource Azure KeyVault. I could define these parameters in my parameter file for the ARM template. But since the parameter: secretName and secretValue is created during the script I would need to define them when running the cmdlet.

Write-Verbose -Message "Deploying Azure resources"
New-AzResourceGroupDeployment -Name "fullazureresourcedeployment" `
	-ResourceGroupName $ResourceGroupName `
	-TemplateFile ".\templates\azuredeploy.json" `
	-TemplateParameterFile ".\parameters\azuredeploy.parameters.json" `
	-tenantId $TenantId `
	-secretName $sp.applicationId `
	-secretValue (ConvertTo-SecureString $spSecret -AsplainText -Force) `
	-objectId $ServicePrincipalName

Finishing the script

The last thing I need to do is configure the parameters needed to run the script and divide the script with two switch parameters for better controlling what should be run in the script.

The parameters section will look like so:

param (
    [Parameter(Mandatory=$false)][String]$ResourceGroupName,
    [Parameter(Mandatory=$false)][String]$ResourceLocation,
    [Parameter(Mandatory=$false)][String]$ServicePrincipalName,
    [Parameter(Mandatory=$false)][String]$ServicePrincipalPass,
    [Parameter(Mandatory=$false)][String]$SubscriptionId,
    [Parameter(Mandatory=$false)][String]$TenantId,
    [Parameter(Mandatory=$false)][String]$NewServicePrincipalName,
    [Parameter(Mandatory=$false)][Switch]$ConnectAzure,
    [Parameter(Mandatory=$false)][Switch]$DeployAzureResources
)

The last two parameters $ConnectAzure and $DeployAzureResources are two switches I use for controlling different sections in the script.

The first parameter: $ConnectAzure, makes sure that the correct Powershell modules and that connecting to Azure don’t fail.

The second parameter: $DeployAzureResources, is in charge of deploying all resources such as Resource Group, Service Principal, and Azure ARM templates.

The reason why I am splitting these two actions into separate actions is that when I am running the PIpeline, it gives a better overview of when an action is running prerequisites or deploying resources.

The Complete Script

<# 
.DESCRIPTION 
    Script for deploying AzureLab-AutomationAccount to Azure 
.PARAMETER ResourceGroupName
    Name of the resource group for which the resources should be deployed
.PARAMETER ResourceLocation
    The location of where the resources should be deployed.
.PARAMETER ServicePrincipalName
    The name of the Service PrincipalName who is deploying the azure resources
.PARAMETER ServicePrincipalPass
    The Password in Clear text for the service principal who is deploying the resources
.PARAMETER SubscriptionId
    The SubscriptionId to where the azure resources should be deployed
.PARAMETER TenantId
    The Tenant Id for which RBAC will be integrated
.PARAMETER NewServicePrincipalName
    The Name of the Service Principal Which will be created during the deployment
    The Service Principal will be granted contributor RBAC for the resource group and
    will be added to the Azure KeyVault

    The service principal used for running this script and creating a new service principal
    should have the following api permissions:
    - Windows Azure Active Directory: "Manage apps that this app creates or owns"
    - Microsoft Graph: "Read and write directory data", "Access directory as the signed in user"
.PARAMETER ConnectAzure
    Switch parameter to define if the script should run the installation of powershell
    module and connect to Azure
.PARAMETER DeployAzureResources
    Switch parameter to define if the script should deploy resources to azure
#> 

param (
    [Parameter(Mandatory=$false)][String]$ResourceGroupName,
    [Parameter(Mandatory=$false)][String]$ResourceLocation,
    [Parameter(Mandatory=$false)][String]$ServicePrincipalName,
    [Parameter(Mandatory=$false)][String]$ServicePrincipalPass,
    [Parameter(Mandatory=$false)][String]$SubscriptionId,
    [Parameter(Mandatory=$false)][String]$TenantId,
    [Parameter(Mandatory=$false)][String]$NewServicePrincipalName,
    [Parameter(Mandatory=$false)][Switch]$ConnectAzure,
    [Parameter(Mandatory=$false)][Switch]$DeployAzureResources
)

if($ConnectAzure.IsPresent) {
    Write-Verbose -Message "Checking and Installing Azure Powershell Module"
    if (-not(Get-Module -Name Az.Accounts -ListAvailable)){
        Write-Warning "Module 'Az.Accounts' is missing or out of date. Installing module now."
        Install-Module -Name Az.Accounts, Az.Resources -Scope CurrentUser -Force -AllowClobber
    }

    Write-Verbose -Message "Connecting to Azure"
    $ServicePrincipalPassword = ConvertTo-SecureString -AsPlainText -Force -String $ServicePrincipalPass
    $azureAppCred = New-Object System.Management.Automation.PSCredential ($ServicePrincipalName,$ServicePrincipalPassword)
    Connect-AzAccount -ServicePrincipal -Credential $azureAppCred -Tenant $tenantId -Subscription $SubscriptionId
}


if($DeployAzureResources.IsPresent) {
    if(!(Get-AzResourceGroup | Where-Object ResourceGroupName -eq $ResourceGroupName)){
        try {
            Write-Verbose -Message "Creating the Azure Resource Group: $($ResourceGroupName)"
            New-AzResourceGroup -Name $ResourceGroupName -Location $ResourceLocation -Force -Confirm:$false 
        }
        catch {
            Write-Error "$($_)"
            Exit
        }
    }
    else {
        Write-Verbose "Resource Group already Exists, updating resource"
    }

    if(!(Get-AzADServicePrincipal | Where-Object DisplayName -eq $NewServicePrincipalName)) {
        Write-verbose -Message "Creating Service Principal"
        $sp = New-AzADServicePrincipal -DisplayName $NewServicePrincipalName -Scope "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroupName)" -Role "Contributor"
        $spSecret = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
            [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
                $sp.Secret
            )
        )
    }
    else {
        Write-Verbose -Message "Service Principal Already exists, removing the old and creating new"
        $appId = (Get-AzADServicePrincipal | Where-Object DisplayName -eq $NewServicePrincipalName | Select-Object -ExpandProperty ApplicationId).Guid
        Remove-AzADServicePrincipal -ApplicationId $appId -Force

        Write-verbose -Message "Creating New Service Principal"
        $sp = New-AzADServicePrincipal -DisplayName $NewServicePrincipalName -Scope "/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroupName)" -Role "Contributor"
        $spSecret = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
            [System.Runtime.InteropServices.Marshal]::SecureStringToBSTR(
                $sp.Secret
            )
        )
    }

    
    Write-Verbose -Message "Deploying Azure resources"
    New-AzResourceGroupDeployment -Name "fullazureresourcedeployment" `
        -ResourceGroupName $ResourceGroupName `
        -TemplateFile ".\templates\azuredeploy.json" `
        -TemplateParameterFile ".\parameters\azuredeploy.parameters.json" `
        -tenantId $TenantId `
        -secretName $sp.applicationId `
        -secretValue (ConvertTo-SecureString $spSecret -AsplainText -Force) `
        -objectId $ServicePrincipalName
}

Pester Tests

Pester Tests are always a good thing to do, but even more when you working with Infrastructure As Code. The reason for this is that the tests help you make sure that the resources and changes deployed will be exactly as you want them. When you start running everything through code it is easy to make mistakes, especially when multiple people are working on the same repos. Once you deploy your code, the changes will happen in your environment. The tests can help you catch misconfigurations or mistakes in the code.

Pester Tests is a great tool for testing your ARM templates, and your scripts used for the pipeline. I have created a small test for showcasing how you could utilize pester tests, to make sure your deployments are happening exactly as you had intended.

The script

Describe 'ARM Template Validation' {
    Context 'Template File Validation' {
        It "Template File Exists" {
            Test-Path -Path ".\templates\azuredeploy.json" | Should -BeTrue
        }
        It "ARM Template is a valid json file" {
            Get-Content ".\templates\azuredeploy.json" -raw | ConvertFrom-json -ErrorAction SilentlyContinue | Should -Not -Be $null
        }
    }

    Context 'Template Content Validation' {
        $Resources = @(
            'Microsoft.Storage/storageAccounts',
            'Microsoft.OperationalInsights/workspaces',
            'Microsoft.Automation/automationAccounts',
            'Microsoft.KeyVault/vaults',
            'Microsoft.KeyVault/vaults/secrets'
        )
        It "Only has approved resources" {
            $resourcesFromTemplate = $TemplateJson.resources.type | Sort-Object
            $strResourcesFromTemplate = $resourcesFromTemplate -join ','
            $resources = $resources | Sort-Object
            $strResources = $resources -join ','
            $strResourcesFromTemplate | Should -be $strResources
        }
    }
}

This pester test for the following things:

  • Does the ARM template exist
  • Is the ARM template an actual valid json file
  • The resources deployed through the template is as described in the tes

Does the ARM template exist:

The first section of the test will, test if the azuredeploy.json ARM template exists in the repository. This is tested with the command Test-Path

It "Template File Exists" {
	Test-Path -Path ".\templates\azuredeploy.json" | Should -BeTrue
}

Is the ARM template an actual valid json file:

The second section of the test will, test if the azuredeploy.json file is a valid json file which can be imported into powershell. This is testet with the command ConvertFrom-Json

It "ARM Template is a valid json file" {
	Get-Content ".\templates\azuredeploy.json" -raw | ConvertFrom-json -ErrorAction SilentlyContinue | Should -Not -Be $null
}

The resources deployed through the template is as described in the test:

In the third section an array of resources is defined in the variable: $Resources. The test will then import the json file and read the resources inserted in the ARM template. Then it will compare the resources stated and the actual resources placed in the template. If other resources has been placed in the template, which has not been defined in the test it will fail.

It "Only has approved resources" {
	$resourcesFromTemplate = $TemplateJson.resources.type | Sort-Object
	$strResourcesFromTemplate = $resourcesFromTemplate -join ','
	$resources = $resources | Sort-Object
	$strResources = $resources -join ','
	$strResourcesFromTemplate | Should -be $strResources
}

Creating the pipeline

The pipeline will be defined by the language YAML. To create a new Pipeline you can go into Github –> Actions –> set up a workflow yourself”

This will give you a starter template. you can go ahead and erase everything.

Now with a fresh .yml file, we can start creating the pipeline.

The first to do is to define a name and how the pipeline can be activated. In this case, I have chosen that the pipeline can be started with a push of pull-request into the main branch. I also state the switch “workflow_dispatch”. This switch gives you the possibility to run the pipeline directory from the Github Repo, instead of having to do a push or pull request into the repo.

name: DeployAzureResources

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  workflow_dispatch:

Next i wont to define the Jobs. The Jobs are the actual actions the pipeline should perform. In this case i want the pipeline to perform two jobs.

The first job is to run the Pester Test to make sure that the template is compliant according to my needs.

I will start by naming this job “Tests”. Then to run this job I will create a runner(a “virtual machine”/”container”) running Windows. The acutal job will perform two steps:

Step 1:

  • Will checkout the code repository so that the new runner has the code base localy. This can be compared to do a git clone of the repo.

Step 2:

  • Will Run the powershell command Invoke-Pester on the Pester Test i have created.
jobs:
  Tests:
    runs-on: windows-latest
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2
      - name: Performin Pester Tests
        shell: powershell
        run: |
          Invoke-Pester -Script ".\tests\ARMTemplate.Tests.ps1" -Passthru

The second job will be named Release and likewise to the first job will create a Windows runner, for running the actions on. This job will also have a statement named “needs”. This statement defines that this job cannot start before the first job has completed successfully.

This job will have three steps:

Step 1:

  • Will check out the code repository to the local machine, just like in Job 1

Step 2:

  • Will run the powershell script run.ps1. It will only run the ConnectAzure section defined by the parameter “-ConnectAzure”.

Step 3:

  • Will run the powershell script run.ps1. This step will run the actual deployment of the resources to Azure

The reason why I am not running steps 2 and 3 in separate Jobs is that I would have to authenticate to azure two times, so to save time and resources I chose to split the ps1 script into two steps in a single job.

  Release:
    needs: Tests
    runs-on: windows-latest
    steps:
    - name: Checkout Code Repository
      uses: actions/checkout@v2
    - name: Run Prerequisites
      run: .\scripts\run.ps1 -ConnectAzure -ServicePrincipalPass ${{secrets.SERVICEPRINCIPALPASS}} -ServicePrincipalName ${{secrets.SERVICEPRINCIPALNAME}} -SubscriptionId ${{secrets.SUBSCRIPTIONID}} -TenantId ${{secrets.TENANTID}} -Verbose
      shell: powershell
    - name: Deploy Azure Resources
      run: .\scripts\run.ps1 -DeployAzureResources -ResourceGroupName "sc-dev-aa-rg" -ResourceLocation "North Europe" -SubscriptionId ${{secrets.SUBSCRIPTIONID}} -TenantId ${{secrets.TENANTID}} -NewServicePrincipalName "sc-dev-aa-sp" -Verbose
      shell: powershell

The complete pipeline.yml file

name: DeployAzureResources

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

  workflow_dispatch:

jobs:
  Tests:
    runs-on: windows-latest
    steps:
      - name: Check out repository code
        uses: actions/checkout@v2
      - name: Performin Pester Tests
        shell: powershell
        run: |
          Invoke-Pester -Script ".\tests\ARMTemplate.Tests.ps1" -Passthru

  Release:
    needs: Tests
    runs-on: windows-latest
    steps:
    - name: Checkout Code Repository
      uses: actions/checkout@v2
    - name: Run Prerequisites
      run: .\scripts\run.ps1 -ConnectAzure -ServicePrincipalPass ${{secrets.SERVICEPRINCIPALPASS}} -ServicePrincipalName ${{secrets.SERVICEPRINCIPALNAME}} -SubscriptionId ${{secrets.SUBSCRIPTIONID}} -TenantId ${{secrets.TENANTID}} -Verbose
      shell: powershell
    - name: Deploy Azure Resources
      run: .\scripts\run.ps1 -DeployAzureResources -ResourceGroupName "sc-dev-aa-rg" -ResourceLocation "North Europe" -SubscriptionId ${{secrets.SUBSCRIPTIONID}} -TenantId ${{secrets.TENANTID}} -NewServicePrincipalName "sc-dev-aa-sp" -Verbose
      shell: powershell

As you see in the yaml file I use environment secrets as Environment variables for some of the run.ps1 script parameters.

To create an environment secret you can go to Settings tab inside your repository –> Secret and select “New repository secret”

Finishing Touches

The cool thing about deploying your resources to Azure this way is that when you ever have a change to your environment, you just edit the ARM Template and run the pipeline again. The pipeline will make sure to update your Azure resources accordingly.

This can be utilized for endless possibilities. One use case I learned is for Azure labs, for example, if you need to test Azure Automations with DSC configurations to VM’s running in Azure. You could configure everything in an ARM template, then deploy it do all the testing you need, and then when you are done you can just delete the resources in Azure. This way you can very quickly test all sorts of environments in Azure without having a huge cost of having them running all the time, or having the cost of just having the resources deployed.

If you configure more advanced environments you could also create a workflow/pipeline for deleting the resources, This way you can deploy and delete all your resources with just a touch of a button in Github.

On the last note, I just want to highlight the Microsoft Docs for Learning about ARM Templates. This Quick guide took me an hour and gave me all the basic understandings of utilizing ARM Templates for Azure resource deployments.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This website uses cookies. By continuing to use this site, you accept our use of cookies.