iac | Tom's Notes

Using a self-hosted runner with GitHub Actions

By Tomica Kaniski in Other 17/10/2022 0 Comment

As I was going through the excellent short course called Azure Infrastructure as Code with GitHub (by fellow MVP, Barbara Forbes), a thought appeared – what do I need to do to use my custom runner machine inside a pipeline for… I don’t know… security/privacy concerns, isolation, special requirements, different OS, control, price… or just to complicate things a bit?

Of course, GitHub supports this and it’s called a self-hosted runner.

So, what do I need to do to use this self-hosted runner with my GitHub Actions?

It’s relatively simple – there is an application package, which will be installed on your runner machine, and which will listen for and eventually do all the work defined in your workflow!

But first, let’s introduce my environment.

I have a simple GitHub Action (workflow), which creates a simple storage account on my Azure environment (there is actually no need to convert Bicep to ARM before deployment, but it seemed cool 😀). It’s currently using the „ubuntu-latest“ runner, provided by GitHub… which has also all the needed components inside (like Azure CLI, Azure PowerShell, …).

And it works fine. When there is a push to my GitHub repository, GitHub Actions starts and does what is needed on my Azure environment via this workflow:

name: deploy-to-azure
on:
  push:
    branches: [ "main" ]
  workflow_dispatch:

env:
  resourceGroupName: SimpleAzureRG

jobs:
  deploy-to-azure:
    runs-on: ubuntu-latest

    steps:
      - name: "Checkout the code from GitHub"
        uses: actions/checkout@v3

      - name: "Build Bicep code into ARM (JSON)"
        uses: Azure/cli@v1
        with:
          inlineScript: az bicep build --file ./main.bicep --outfile ./main.json

      - name: "Login to Azure"
        uses: Azure/login@v1.4.6
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
          enable-AzPSSession: true

      - name: "Create the resource group, if needed"
        uses: Azure/powershell@v1
        with:
          inlineScript: New-AzResourceGroup -Name ${{ env.resourceGroupName }} -location WestEurope -Force
          azPSVersion: latest

      - name: "Deploy the ARM template"
        uses: Azure/arm-deploy@v1.0.8
        with:
          scope: resourcegroup
          resourceGroupName: ${{ env.resourceGroupName }}
          template: main.json
          parameters: storageAccountPrefix=simplegh
          deploymentName: "gh${{ github.run_id }}"

on:

push:

branches: [ "main" ]

workflow_dispatch:

env:

resourceGroupName: SimpleAzureRG

jobs:

deploy-to-azure:

runs-on: ubuntu-latest

steps:

- name: "Checkout the code from GitHub"

uses: actions/checkout@v3

- name: "Build Bicep code into ARM (JSON)"

uses: Azure/cli@v1

with:

inlineScript: az bicep build --file ./main.bicep --outfile ./main.json

- name: "Login to Azure"

uses: Azure/[email protected]

with:

creds: ${{ secrets.AZURE_CREDENTIALS }}

enable-AzPSSession: true

- name: "Create the resource group, if needed"

uses: Azure/powershell@v1

with:

inlineScript: New-AzResourceGroup -Name ${{ env.resourceGroupName }} -location WestEurope -Force

azPSVersion: latest

- name: "Deploy the ARM template"

uses: Azure/[email protected]

with:

scope: resourcegroup

resourceGroupName: ${{ env.resourceGroupName }}

parameters: storageAccountPrefix=simplegh

deploymentName: "gh${{ github.run_id }}"

And the mighty Bicep file (😀) it’s using for the deployment is:

@maxLength(11)
param storageAccountPrefix string = 'bicep'
param location string = resourceGroup().location

var sta = '${storageAccountPrefix}${uniqueString(subscription().id)}'

resource storageaccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {
  name: sta
  location: location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_LRS'
  }
}

@maxLength(11)

param storageAccountPrefix string = 'bicep'

param location string = resourceGroup().location

var sta = '${storageAccountPrefix}${uniqueString(subscription().id)}'

resource storageaccount 'Microsoft.Storage/storageAccounts@2021-02-01' = {

location: location

kind: 'StorageV2'

sku: {

}

Of course, this runs just fine on a standard (hosted) runner:

To run this workflow (successfully) not that much is needed.

First, I’ve created a new virtual machine (I’ll use a simple Ubuntu Hyper-V VM, no autoscaling, no… nothing) called hermes (god of speed 😀), with freshly installed Ubuntu 22.04.1-LTS (minimized).

After that, I went to the Settings of my GitHub repository and got the download and install scripts for the x64 Linux runner:

### get and install the runner application (script provided by GitHub)
## download the runner application
# create a runner folder
mkdir actions-runner && cd actions-runner

# download the latest runner package
curl -o actions-runner-linux-x64-2.298.2.tar.gz -L https://github.com/actions/runner/releases/download/v2.298.2/actions-runner-linux-x64-2.298.2.tar.gz

# (optional) validate the hash
echo "0bfd792196ce0ec6f1c65d2a9ad00215b2926ef2c416b8d97615265194477117  actions-runner-linux-x64-2.298.2.tar.gz" | shasum -a 256 -c

# extract the installer
tar xzf ./actions-runner-linux-x64-2.298.2.tar.gz


## configure the runner application
# create the runner and start the configuration experience
./config.sh --url https://github.com/TomicaKaniski/AzureIaCGH --token "<<--TOKEN_GOES_HERE!-->>"

# last step, run it! (we will actually run it via crontab at machine restart!)
# ./run.sh

### get and install the runner application (script provided by GitHub)

## download the runner application

# create a runner folder

mkdir actions-runner && cd actions-runner

# download the latest runner package

curl -o actions-runner-linux-x64-2.298.2.tar.gz -L https://github.com/actions/runner/releases/download/v2.298.2/actions-runner-linux-x64-2.298.2.tar.gz

# (optional) validate the hash

echo "0bfd792196ce0ec6f1c65d2a9ad00215b2926ef2c416b8d97615265194477117 actions-runner-linux-x64-2.298.2.tar.gz" | shasum -a 256 -c

# extract the installer

tar xzf ./actions-runner-linux-x64-2.298.2.tar.gz

## configure the runner application

# create the runner and start the configuration experience

./config.sh --url https://github.com/TomicaKaniski/AzureIaCGH --token "<<--TOKEN_GOES_HERE!-->>"

# last step, run it! (we will actually run it via crontab at machine restart!)

# ./run.sh

As you can see, I’ll be using crontab later to automatically (re)start my self-hosted runner.

If everything went well, you should see your runner “up and running” (😀) in the GitHub portal:

Next, I’ll use the following script to install all prerequisites for my workflow (like Azure CLI, Azure PowerShell, etc. – it really depends on your workflow and things you use):

### install (my) workflow prerequisites on self-runner machine
sudo su -

## install Azure CLI (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt)
curl -sL https://aka.ms/InstallAzureCLIDeb | bash

## install PowerShell  (https://learn.microsoft.com/en-us/powershell/scripting/install/install-ubuntu?view=powershell-7.2#installation-via-direct-download)
wget -q "https://github.com/PowerShell/PowerShell/releases/download/v7.2.6/powershell-lts_7.2.6-1.deb_amd64.deb" && dpkg -i powershell-lts_7.2.6-1.deb_amd64.deb

## install Bicep
az bicep install

## install other useful things
apt install apt-utils cron vim -y

## install Docker (https://cloudcone.com/docs/article/how-to-install-docker-on-ubuntu-22-04-20-04/)
# install prerequisites
apt install apt-transport-https curl gnupg-agent ca-certificates software-properties-common -y

# add GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# add repository
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" -y

# install docker
apt install docker-ce docker-ce-cli containerd.io -y

# create docker group
newgrp docker
exit

# add current user (tomica) to docker group
sudo usermod -aG docker $USER

# configure Docker autostart
sudo systemctl enable docker
sudo systemctl restart docker
sudo systemctl status docker

## install Az module for PowerShell (https://learn.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-9.0.0)
pwsh
Install-Module Az -Repository PSGallery -Confirm:$false -Force
exit


### configure ./run.sh (GitHub runner application) to start automatically
crontab -e

# add the following line (uncommented, of course; and check if the path is right!):
# @reboot /home/tomica/actions-runner/run.sh

sudo reboot

### install (my) workflow prerequisites on self-runner machine

sudo su -

## install Azure CLI (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux?pivots=apt)

curl -sL https://aka.ms/InstallAzureCLIDeb | bash

## install PowerShell (https://learn.microsoft.com/en-us/powershell/scripting/install/install-ubuntu?view=powershell-7.2#installation-via-direct-download)

wget -q "https://github.com/PowerShell/PowerShell/releases/download/v7.2.6/powershell-lts_7.2.6-1.deb_amd64.deb" && dpkg -i powershell-lts_7.2.6-1.deb_amd64.deb

## install Bicep

az bicep install

## install other useful things

apt install apt-utils cron vim -y

## install Docker (https://cloudcone.com/docs/article/how-to-install-docker-on-ubuntu-22-04-20-04/)

# install prerequisites

apt install apt-transport-https curl gnupg-agent ca-certificates software-properties-common -y

# add GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# add repository

add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" -y

# install docker

apt install docker-ce docker-ce-cli containerd.io -y

# create docker group

newgrp docker

exit

# add current user (tomica) to docker group

sudo usermod -aG docker $USER

# configure Docker autostart

sudo systemctl enable docker

sudo systemctl restart docker

sudo systemctl status docker

## install Az module for PowerShell (https://learn.microsoft.com/en-us/powershell/azure/install-az-ps?view=azps-9.0.0)

pwsh

Install-Module Az -Repository PSGallery -Confirm:$false -Force

exit

### configure ./run.sh (GitHub runner application) to start automatically

crontab -e

# add the following line (uncommented, of course; and check if the path is right!):

# @reboot /home/tomica/actions-runner/run.sh

sudo reboot

Once this is done, my self-hosted runner hermes should be ready to run the workflow.

To try this, I need to make a slight update to my workflow file – line 12 inside the job configuration should be updated from “runs-on: ubuntu-latest” to “runs-on: self-hosted“.

So, my workflow YAML file now looks like this:

name: deploy-to-azure
on:
  push:
    branches: [ "main" ]
  workflow_dispatch:

env:
  resourceGroupName: SimpleAzureRG

jobs:
  deploy-to-azure:
    runs-on: self-hosted

    steps:
      - name: "Checkout the code from GitHub"
        uses: actions/checkout@v3

      - name: "Build Bicep code into ARM (JSON)"
        uses: Azure/cli@v1
        with:
          inlineScript: az bicep build --file ./main.bicep --outfile ./main.json

      - name: "Login to Azure"
        uses: Azure/login@v1.4.6
        with:
          creds: ${{ secrets.AZURE_CREDENTIALS }}
          enable-AzPSSession: true

      - name: "Create the resource group, if needed"
        uses: Azure/powershell@v1
        with:
          inlineScript: New-AzResourceGroup -Name ${{ env.resourceGroupName }} -location WestEurope -Force
          azPSVersion: latest

      - name: "Deploy the ARM template"
        uses: Azure/arm-deploy@v1.0.8
        with:
          scope: resourcegroup
          resourceGroupName: ${{ env.resourceGroupName }}
          template: main.json
          parameters: storageAccountPrefix=simplegh
          deploymentName: "gh${{ github.run_id }}"

on:

push:

branches: [ "main" ]

workflow_dispatch:

env:

resourceGroupName: SimpleAzureRG

jobs:

deploy-to-azure:

runs-on: self-hosted

steps:

- name: "Checkout the code from GitHub"

uses: actions/checkout@v3

- name: "Build Bicep code into ARM (JSON)"

uses: Azure/cli@v1

with:

inlineScript: az bicep build --file ./main.bicep --outfile ./main.json

- name: "Login to Azure"

uses: Azure/[email protected]

with:

creds: ${{ secrets.AZURE_CREDENTIALS }}

enable-AzPSSession: true

- name: "Create the resource group, if needed"

uses: Azure/powershell@v1

with:

inlineScript: New-AzResourceGroup -Name ${{ env.resourceGroupName }} -location WestEurope -Force

azPSVersion: latest

- name: "Deploy the ARM template"

uses: Azure/[email protected]

with:

scope: resourcegroup

resourceGroupName: ${{ env.resourceGroupName }}

parameters: storageAccountPrefix=simplegh

deploymentName: "gh${{ github.run_id }}"

And once I push the configuration to my GitHub, my workflow automatically starts and runs on hermes, my self-hosted runner:

If we prepared our runner right, all is good! 😊

Of course, our resources are deployed successfully:

So, this is how you can use your own, self-hosted runner, to execute your GitHub Actions (workflows).

Cheers!

What about this Bicep?

By Tomica Kaniski in Other 04/08/2021 0 Comment

You’ve probably heard about Azure Resource Manager (ARM) – the deployment and management service/layer of Azure, which enables you to manage (create, configure, delete) your Azure resources. Also, you are probably aware that ARM uses so called ARM templates – basically, JSON files that actually define the infrastructure and configuration you want to deploy to Azure (think Infrastructure as Code, IaC).

So, if you have dealt with ARM/JSON in the past, you may have been finding it difficult to start with, and somewhat complex.

Bicep is here to help.

Here is a short overview of Bicep – basically, it’s a language which enables you easier deployment of Azure resources, without messing around (too much) with JSON. To be frank, it somehow reminds of Terraform, but it’s also different. It has many cool features, immediately supports all new Azure features and APIs, can be built (converted) into .json and deployed as such or it can be deployed straight away as .bicep, doesn’t require state file, it’s open and free, has great support in Visual Studio Code and much more. And it’s still in active development!

If you’re dealing with IaC and Azure, try it.

To show you the power (and simplicity) of Bicep, here is a short example of deploying Linux virtual machine in Azure (together with a resource group, virtual network, virtual network subnet, virtual NIC and network security group), done “the old way” (in JSON, which was actually converted from Bicep… it’s easier than writing JSON from the scratch) and then done via Bicep (“the right way”? 😀).

Additionally, you’ll see that I’ve tried to break stuff into modules – with more or less sucess. 😀

The ARM/JSON way (could be done nicer/shorter, with parameters inside .parameters.json… if you know what you’re doing – this is converted from Bicep and serves just for illustrative purposes):

{
  "$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "metadata": {
    "_generator": {
      "name": "bicep",
      "version": "0.4.451.19169",
      "templateHash": "6031449369953332876"
    }
  },
  "parameters": {
    "resourceGroupName": {
      "type": "string",
      "defaultValue": "Gym"
    },
    "location": {
      "type": "string",
      "defaultValue": "westeurope"
    }
  },
  "functions": [],
  "resources": [
    {
      "type": "Microsoft.Resources/resourceGroups",
      "apiVersion": "2021-04-01",
      "name": "[parameters('resourceGroupName')]",
      "location": "[parameters('location')]"
    },
    {
      "type": "Microsoft.Resources/deployments",
      "apiVersion": "2019-10-01",
      "name": "vm-module",
      "resourceGroup": "[parameters('resourceGroupName')]",
      "properties": {
        "expressionEvaluationOptions": {
          "scope": "inner"
        },
        "mode": "Incremental",
        "parameters": {
          "location": {
            "value": "[parameters('location')]"
          },
          "adminName": {
            "value": "tomica"
          },
          "adminSSHKey": {
            "value": "ssh-rsa <SOME SSH PUBLIC KEY>"
          },
          "machineName": {
            "value": "tkVM"
          },
          "machineSize": {
            "value": "Standard_A1_v2"
          },
          "networkName": {
            "value": "tkNet"
          },
          "networkSubnet": {
            "value": "default"
          },
          "networkSG": {
            "value": "tkNetSG"
          }
        },
        "template": {
          "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
          "contentVersion": "1.0.0.0",
          "metadata": {
            "_generator": {
              "name": "bicep",
              "version": "0.4.451.19169",
              "templateHash": "141276028935159012"
            }
          },
          "parameters": {
            "location": {
              "type": "string"
            },
            "adminName": {
              "type": "string"
            },
            "adminSSHKey": {
              "type": "string"
            },
            "machineName": {
              "type": "string"
            },
            "machineSize": {
              "type": "string"
            },
            "networkName": {
              "type": "string"
            },
            "networkSubnet": {
              "type": "string"
            },
            "networkSG": {
              "type": "string"
            }
          },
          "functions": [],
          "variables": {
            "networkNIC": "[format('{0}-nic', parameters('machineName'))]",
            "networkSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), parameters('networkSubnet'))]",
            "osConfiguration": {
              "disablePasswordAuthentication": true,
              "ssh": {
                "publicKeys": [
                  {
                    "path": "[format('/home/{0}/.ssh/authorized_keys', parameters('adminName'))]",
                    "keyData": "[parameters('adminSSHKey')]"
                  }
                ]
              }
            }
          },
          "resources": [
            {
              "type": "Microsoft.Network/networkSecurityGroups",
              "apiVersion": "2021-02-01",
              "name": "[parameters('networkSG')]",
              "location": "[parameters('location')]",
              "properties": {
                "securityRules": [
                  {
                    "name": "allow-ssh",
                    "properties": {
                      "priority": 1000,
                      "access": "Allow",
                      "direction": "Inbound",
                      "destinationPortRange": "22",
                      "protocol": "Tcp",
                      "sourceAddressPrefix": "*",
                      "sourcePortRange": "*",
                      "destinationAddressPrefix": "*"
                    }
                  }
                ]
              }
            },
            {
              "type": "Microsoft.Network/virtualNetworks",
              "apiVersion": "2021-02-01",
              "name": "[parameters('networkName')]",
              "location": "[parameters('location')]",
              "properties": {
                "addressSpace": {
                  "addressPrefixes": [
                    "10.0.0.0/16"
                  ]
                },
                "subnets": [
                  {
                    "name": "[parameters('networkSubnet')]",
                    "properties": {
                      "addressPrefix": "10.0.3.0/24",
                      "networkSecurityGroup": {
                        "id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"
                      }
                    }
                  }
                ]
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"
              ]
            },
            {
              "type": "Microsoft.Network/networkInterfaces",
              "apiVersion": "2021-02-01",
              "name": "[variables('networkNIC')]",
              "location": "[parameters('location')]",
              "properties": {
                "ipConfigurations": [
                  {
                    "name": "ipConfig",
                    "properties": {
                      "privateIPAllocationMethod": "Dynamic",
                      "subnet": {
                        "id": "[variables('networkSubnetID')]"
                      }
                    }
                  }
                ]
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/virtualNetworks', parameters('networkName'))]"
              ]
            },
            {
              "type": "Microsoft.Compute/virtualMachines",
              "apiVersion": "2021-03-01",
              "name": "[parameters('machineName')]",
              "location": "[parameters('location')]",
              "properties": {
                "hardwareProfile": {
                  "vmSize": "[parameters('machineSize')]"
                },
                "osProfile": {
                  "computerName": "[parameters('machineName')]",
                  "adminUsername": "[parameters('adminName')]",
                  "adminPassword": "[parameters('adminSSHKey')]",
                  "linuxConfiguration": "[variables('osConfiguration')]"
                },
                "storageProfile": {
                  "imageReference": {
                    "publisher": "Canonical",
                    "offer": "UbuntuServer",
                    "sku": "18.04-LTS",
                    "version": "latest"
                  }
                },
                "networkProfile": {
                  "networkInterfaces": [
                    {
                      "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"
                    }
                  ]
                }
              },
              "dependsOn": [
                "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"
              ]
            }
          ]
        }
      },
      "dependsOn": [
        "[subscriptionResourceId('Microsoft.Resources/resourceGroups', parameters('resourceGroupName'))]"
      ]
    }
  ]
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

{

"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",

"contentVersion": "1.0.0.0",

"metadata": {

"_generator": {

"name": "bicep",

"version": "0.4.451.19169",

"templateHash": "6031449369953332876"

}

"parameters": {

"resourceGroupName": {

"type": "string",

"defaultValue": "Gym"

"location": {

"type": "string",

"defaultValue": "westeurope"

}

"functions": [],

"resources": [

{

"type": "Microsoft.Resources/resourceGroups",

"apiVersion": "2021-04-01",

"name": "[parameters('resourceGroupName')]",

"location": "[parameters('location')]"

{

"type": "Microsoft.Resources/deployments",

"apiVersion": "2019-10-01",

"name": "vm-module",

"resourceGroup": "[parameters('resourceGroupName')]",

"properties": {

"expressionEvaluationOptions": {

"scope": "inner"

"mode": "Incremental",

"parameters": {

"location": {

"value": "[parameters('location')]"

"adminName": {

"value": "tomica"

"adminSSHKey": {

"value": "ssh-rsa <SOME SSH PUBLIC KEY>"

"machineName": {

"value": "tkVM"

"machineSize": {

"value": "Standard_A1_v2"

"networkName": {

"value": "tkNet"

"networkSubnet": {

"value": "default"

"networkSG": {

"value": "tkNetSG"

}

"template": {

"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

"contentVersion": "1.0.0.0",

"metadata": {

"_generator": {

"name": "bicep",

"version": "0.4.451.19169",

"templateHash": "141276028935159012"

}

"parameters": {

"location": {

"type": "string"

"adminName": {

"type": "string"

"adminSSHKey": {

"type": "string"

"machineName": {

"type": "string"

"machineSize": {

"type": "string"

"networkName": {

"type": "string"

"networkSubnet": {

"type": "string"

"networkSG": {

"type": "string"

}

"functions": [],

"variables": {

"networkNIC": "[format('{0}-nic', parameters('machineName'))]",

"networkSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), parameters('networkSubnet'))]",

"osConfiguration": {

"disablePasswordAuthentication": true,

"ssh": {

"publicKeys": [

{

"path": "[format('/home/{0}/.ssh/authorized_keys', parameters('adminName'))]",

"keyData": "[parameters('adminSSHKey')]"

}

]

}

"resources": [

{

"type": "Microsoft.Network/networkSecurityGroups",

"apiVersion": "2021-02-01",

"name": "[parameters('networkSG')]",

"location": "[parameters('location')]",

"properties": {

"securityRules": [

{

"name": "allow-ssh",

"properties": {

"priority": 1000,

"access": "Allow",

"direction": "Inbound",

"destinationPortRange": "22",

"protocol": "Tcp",

"sourceAddressPrefix": "*",

"sourcePortRange": "*",

"destinationAddressPrefix": "*"

}

]

}

{

"type": "Microsoft.Network/virtualNetworks",

"apiVersion": "2021-02-01",

"name": "[parameters('networkName')]",

"location": "[parameters('location')]",

"properties": {

"addressSpace": {

"addressPrefixes": [

"10.0.0.0/16"

]

"subnets": [

{

"name": "[parameters('networkSubnet')]",

"properties": {

"addressPrefix": "10.0.3.0/24",

"networkSecurityGroup": {

"id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"

}

]

"dependsOn": [

"[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"

]

{

"type": "Microsoft.Network/networkInterfaces",

"apiVersion": "2021-02-01",

"name": "[variables('networkNIC')]",

"location": "[parameters('location')]",

"properties": {

"ipConfigurations": [

{

"name": "ipConfig",

"properties": {

"privateIPAllocationMethod": "Dynamic",

"subnet": {

"id": "[variables('networkSubnetID')]"

}

]

"dependsOn": [

"[resourceId('Microsoft.Network/virtualNetworks', parameters('networkName'))]"

]

{

"type": "Microsoft.Compute/virtualMachines",

"apiVersion": "2021-03-01",

"name": "[parameters('machineName')]",

"location": "[parameters('location')]",

"properties": {

"hardwareProfile": {

"vmSize": "[parameters('machineSize')]"

"osProfile": {

"computerName": "[parameters('machineName')]",

"adminUsername": "[parameters('adminName')]",

"adminPassword": "[parameters('adminSSHKey')]",

"linuxConfiguration": "[variables('osConfiguration')]"

"storageProfile": {

"imageReference": {

"publisher": "Canonical",

"offer": "UbuntuServer",

"sku": "18.04-LTS",

"version": "latest"

}

"networkProfile": {

"networkInterfaces": [

{

"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"

}

]

}

"dependsOn": [

"[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"

]

}

]

}

"dependsOn": [

"[subscriptionResourceId('Microsoft.Resources/resourceGroups', parameters('resourceGroupName'))]"

]

}

]

}

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "metadata": {
    "_generator": {
      "name": "bicep",
      "version": "0.4.451.19169",
      "templateHash": "141276028935159012"
    }
  },
  "parameters": {
    "location": {
      "type": "string"
    },
    "adminName": {
      "type": "string"
    },
    "adminSSHKey": {
      "type": "string"
    },
    "machineName": {
      "type": "string"
    },
    "machineSize": {
      "type": "string"
    },
    "networkName": {
      "type": "string"
    },
    "networkSubnet": {
      "type": "string"
    },
    "networkSG": {
      "type": "string"
    }
  },
  "functions": [],
  "variables": {
    "networkNIC": "[format('{0}-nic', parameters('machineName'))]",
    "networkSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), parameters('networkSubnet'))]",
    "osConfiguration": {
      "disablePasswordAuthentication": true,
      "ssh": {
        "publicKeys": [
          {
            "path": "[format('/home/{0}/.ssh/authorized_keys', parameters('adminName'))]",
            "keyData": "[parameters('adminSSHKey')]"
          }
        ]
      }
    }
  },
  "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "apiVersion": "2021-02-01",
      "name": "[parameters('networkSG')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "allow-ssh",
            "properties": {
              "priority": 1000,
              "access": "Allow",
              "direction": "Inbound",
              "destinationPortRange": "22",
              "protocol": "Tcp",
              "sourceAddressPrefix": "*",
              "sourcePortRange": "*",
              "destinationAddressPrefix": "*"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/virtualNetworks",
      "apiVersion": "2021-02-01",
      "name": "[parameters('networkName')]",
      "location": "[parameters('location')]",
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "10.0.0.0/16"
          ]
        },
        "subnets": [
          {
            "name": "[parameters('networkSubnet')]",
            "properties": {
              "addressPrefix": "10.0.3.0/24",
              "networkSecurityGroup": {
                "id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"
              }
            }
          }
        ]
      },
      "dependsOn": [
        "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"
      ]
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "apiVersion": "2021-02-01",
      "name": "[variables('networkNIC')]",
      "location": "[parameters('location')]",
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipConfig",
            "properties": {
              "privateIPAllocationMethod": "Dynamic",
              "subnet": {
                "id": "[variables('networkSubnetID')]"
              }
            }
          }
        ]
      },
      "dependsOn": [
        "[resourceId('Microsoft.Network/virtualNetworks', parameters('networkName'))]"
      ]
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2021-03-01",
      "name": "[parameters('machineName')]",
      "location": "[parameters('location')]",
      "properties": {
        "hardwareProfile": {
          "vmSize": "[parameters('machineSize')]"
        },
        "osProfile": {
          "computerName": "[parameters('machineName')]",
          "adminUsername": "[parameters('adminName')]",
          "adminPassword": "[parameters('adminSSHKey')]",
          "linuxConfiguration": "[variables('osConfiguration')]"
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "Canonical",
            "offer": "UbuntuServer",
            "sku": "18.04-LTS",
            "version": "latest"
          }
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"
            }
          ]
        }
      },
      "dependsOn": [
        "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"
      ]
    }
  ]
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

{

"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",

"contentVersion": "1.0.0.0",

"metadata": {

"_generator": {

"name": "bicep",

"version": "0.4.451.19169",

"templateHash": "141276028935159012"

}

"parameters": {

"location": {

"type": "string"

"adminName": {

"type": "string"

"adminSSHKey": {

"type": "string"

"machineName": {

"type": "string"

"machineSize": {

"type": "string"

"networkName": {

"type": "string"

"networkSubnet": {

"type": "string"

"networkSG": {

"type": "string"

}

"functions": [],

"variables": {

"networkNIC": "[format('{0}-nic', parameters('machineName'))]",

"networkSubnetID": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('networkName'), parameters('networkSubnet'))]",

"osConfiguration": {

"disablePasswordAuthentication": true,

"ssh": {

"publicKeys": [

{

"path": "[format('/home/{0}/.ssh/authorized_keys', parameters('adminName'))]",

"keyData": "[parameters('adminSSHKey')]"

}

]

}

"resources": [

{

"type": "Microsoft.Network/networkSecurityGroups",

"apiVersion": "2021-02-01",

"name": "[parameters('networkSG')]",

"location": "[parameters('location')]",

"properties": {

"securityRules": [

{

"name": "allow-ssh",

"properties": {

"priority": 1000,

"access": "Allow",

"direction": "Inbound",

"destinationPortRange": "22",

"protocol": "Tcp",

"sourceAddressPrefix": "*",

"sourcePortRange": "*",

"destinationAddressPrefix": "*"

}

]

}

{

"type": "Microsoft.Network/virtualNetworks",

"apiVersion": "2021-02-01",

"name": "[parameters('networkName')]",

"location": "[parameters('location')]",

"properties": {

"addressSpace": {

"addressPrefixes": [

"10.0.0.0/16"

]

"subnets": [

{

"name": "[parameters('networkSubnet')]",

"properties": {

"addressPrefix": "10.0.3.0/24",

"networkSecurityGroup": {

"id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"

}

]

"dependsOn": [

"[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSG'))]"

]

{

"type": "Microsoft.Network/networkInterfaces",

"apiVersion": "2021-02-01",

"name": "[variables('networkNIC')]",

"location": "[parameters('location')]",

"properties": {

"ipConfigurations": [

{

"name": "ipConfig",

"properties": {

"privateIPAllocationMethod": "Dynamic",

"subnet": {

"id": "[variables('networkSubnetID')]"

}

]

"dependsOn": [

"[resourceId('Microsoft.Network/virtualNetworks', parameters('networkName'))]"

]

{

"type": "Microsoft.Compute/virtualMachines",

"apiVersion": "2021-03-01",

"name": "[parameters('machineName')]",

"location": "[parameters('location')]",

"properties": {

"hardwareProfile": {

"vmSize": "[parameters('machineSize')]"

"osProfile": {

"computerName": "[parameters('machineName')]",

"adminUsername": "[parameters('adminName')]",

"adminPassword": "[parameters('adminSSHKey')]",

"linuxConfiguration": "[variables('osConfiguration')]"

"storageProfile": {

"imageReference": {

"publisher": "Canonical",

"offer": "UbuntuServer",

"sku": "18.04-LTS",

"version": "latest"

}

"networkProfile": {

"networkInterfaces": [

{

"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"

}

]

}

"dependsOn": [

"[resourceId('Microsoft.Network/networkInterfaces', variables('networkNIC'))]"

]

}

]

}

The Bicep way:

// main.bicep
targetScope = 'subscription'

param resourceGroupName string = 'Gym'
param location string = 'westeurope'

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {
  name: resourceGroupName
  location: location
}

module vm 'vm.bicep' = {
  name: 'vm-module'
  scope: rg
  params: {
    location: location
    adminName: 'tomica'
    adminSSHKey: 'ssh-rsa <SOME SSH PUBLIC KEY>'
    machineName: 'tkVM'
    machineSize: 'Standard_A1_v2'
    networkName: 'tkNet'
    networkSubnet: 'default'
    networkSG: 'tkNetSG'
  }
}

// main.bicep

targetScope = 'subscription'

param resourceGroupName string = 'Gym'

param location string = 'westeurope'

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' = {

location: location

}

module vm 'vm.bicep' = {

scope: rg

params: {

location: location

adminName: 'tomica'

adminSSHKey: 'ssh-rsa <SOME SSH PUBLIC KEY>'

machineName: 'tkVM'

machineSize: 'Standard_A1_v2'

networkName: 'tkNet'

networkSubnet: 'default'

networkSG: 'tkNetSG'

}

// vm.bicep
targetScope = 'resourceGroup'

param location string
param adminName string
param adminSSHKey string
param machineName string
param machineSize string
param networkName string
param networkSubnet string
param networkSG string

var networkNIC = '${machineName}-nic'
var networkSubnetID = resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, networkSubnet)
var osConfiguration = {
  disablePasswordAuthentication: true
  ssh: {
    publicKeys: [
      {
        path: '/home/${adminName}/.ssh/authorized_keys'
        keyData: adminSSHKey
      }
    ]
  }
}

resource networkSecurityGroupName 'Microsoft.Network/networkSecurityGroups@2021-02-01' = {
  name: networkSG
  location: location
  properties: {
    securityRules: [
      {
        name: 'allow-ssh'
        properties: {
          priority: 1000
          access: 'Allow'
          direction: 'Inbound'
          destinationPortRange: '22'
          protocol: 'Tcp'
          sourceAddressPrefix: '*'
          sourcePortRange: '*'
          destinationAddressPrefix: '*'
        }
      }
    ]
  }
}

resource virtualNetworkName 'Microsoft.Network/virtualNetworks@2021-02-01' = {
  name: networkName
  location: location
  properties: {
    addressSpace: {
      addressPrefixes: [
        '10.0.0.0/16'
      ]
    }
    subnets: [
      {
        name: networkSubnet
        properties: {
          addressPrefix: '10.0.3.0/24'
          networkSecurityGroup: {
            id: networkSecurityGroupName.id
          }
        }
      }
    ]
  }
}

resource nicName 'Microsoft.Network/networkInterfaces@2021-02-01' = {
  name: networkNIC
  location: location
  properties: {
    ipConfigurations: [
      {
        name: 'ipConfig'
        properties: {
          privateIPAllocationMethod: 'Dynamic'
          subnet: {
            id: networkSubnetID
          }
        }
      }
    ]
  }
  dependsOn: [
    virtualNetworkName
  ]
}

resource vmName 'Microsoft.Compute/virtualMachines@2021-03-01' = {
  name: machineName
  location: location
  properties: {
    hardwareProfile: {
      vmSize: machineSize
    }
    osProfile: {
      computerName: machineName
      adminUsername: adminName
      adminPassword: adminSSHKey
      linuxConfiguration: osConfiguration
    }
    storageProfile: {
      imageReference: {
        publisher: 'Canonical'
        offer: 'UbuntuServer'
        sku: '18.04-LTS'
        version: 'latest'
      }
    }
    networkProfile: {
      networkInterfaces: [
        {
          id: nicName.id
        }
      ]
    }
  }
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

// vm.bicep

targetScope = 'resourceGroup'

param location string

param adminName string

param adminSSHKey string

param machineName string

param machineSize string

param networkName string

param networkSubnet string

param networkSG string

var networkNIC = '${machineName}-nic'

var networkSubnetID = resourceId('Microsoft.Network/virtualNetworks/subnets', networkName, networkSubnet)

var osConfiguration = {

disablePasswordAuthentication: true

ssh: {

publicKeys: [

{

path: '/home/${adminName}/.ssh/authorized_keys'

keyData: adminSSHKey

}

]

}

resource networkSecurityGroupName 'Microsoft.Network/networkSecurityGroups@2021-02-01' = {

location: location

properties: {

securityRules: [

{

properties: {

priority: 1000

access: 'Allow'

direction: 'Inbound'

destinationPortRange: '22'

protocol: 'Tcp'

sourceAddressPrefix: '*'

sourcePortRange: '*'

destinationAddressPrefix: '*'

}

]

}

resource virtualNetworkName 'Microsoft.Network/virtualNetworks@2021-02-01' = {

location: location

properties: {

addressSpace: {

addressPrefixes: [

'10.0.0.0/16'

]

}

subnets: [

{

properties: {

addressPrefix: '10.0.3.0/24'

networkSecurityGroup: {

id: networkSecurityGroupName.id

}

]

}

resource nicName 'Microsoft.Network/networkInterfaces@2021-02-01' = {

location: location

properties: {

ipConfigurations: [

{

properties: {

privateIPAllocationMethod: 'Dynamic'

subnet: {

id: networkSubnetID

}

]

}

dependsOn: [

virtualNetworkName

]

}

resource vmName 'Microsoft.Compute/virtualMachines@2021-03-01' = {

location: location

properties: {

hardwareProfile: {

vmSize: machineSize

}

osProfile: {

computerName: machineName

adminUsername: adminName

adminPassword: adminSSHKey

linuxConfiguration: osConfiguration

}

storageProfile: {

imageReference: {

publisher: 'Canonical'

offer: 'UbuntuServer'

sku: '18.04-LTS'

version: 'latest'

}

networkProfile: {

networkInterfaces: [

{

id: nicName.id

}

]

}

Bicep seems a bit easier to read and shorter, right (while still doing basically the same thing)? 😀

If we deploy the .bicep files above (note that I’m deploying the “raw” .bicep file directly – which is cool!):

New-AzDeployment -TemplateFile ".\main.bicep" -DeploymentName "vm-example" -Location "West Europe"

1	New-AzDeployment -TemplateFile ".\main.bicep" -DeploymentName "vm-example" -Location "West Europe"

We finally get our resources:

So, where should you start if you’re new to Bicep?

I would certainly recommend starting with free and official Deploy and manage resources in Azure by using Bicep learning path on Microsoft Learn.

After that, you can probably pick up Freek Berson’s book Getting started with Bicep: Infrastructure as Code on Azure (first and only book on Bicep that I know of – really liked it because of the simple (yet effective) examples with storage accounts, it connects everything and flows naturally – building up “brick by brick” and not “jumping around”, just to show off what Bicep can do).

Another great resource are also the Bicep examples – there’s plenty to learn from them too!

Of course, you’ll also need to practice – install the Azure CLI or Azure PowerShell module, add Bicep and use Visual Studio Code for your first steps with creating, deleting, configuring and breaking stuff… powered by Bicep! 😀

Cheers!

Fixing things with… terraform destroy

By Tomica Kaniski in Other 29/06/2020 0 Comment

I like Terraform, because it’s so clean, fast and elegant; OK, I also suck at it, but hey – I’m trying! 🙂

The long story

Usually, Terraform and its providers are very good at doing things in the order they should be done. But sometimes people do come up with silly ideas, and mine was such (of course) – I’ve decided to rename something and it broke things. Just a little.

I have a simple lab in Azure, with a couple of virtual machines behind the Azure Load Balancer, no big deal. All this is being deployed (and redeployed) via Terraform, using the official azurerm provider. It’s actually a Standard SKU Azure Load Balancer (don’t ask why), with a single backend pool and a few probes and rules. Nothing special.

I’ve deployed this lab a few days ago (thanks to good people at Microsoft, I have some free credits to spare), everything worked just fine, but today I’ve got the wild idea – I’ve decided to rename my backend pool.

With all the automation in place, this shouldn’t be a problem… one would think. 🙂

So, as I’ve updated my code and during the make it so phase (terraform apply), I’ve got some errors (truncated, with only the useful stuff):

Error:
Error Creating/Updating LoadBalancer: network.LoadBalancersClient
CreateOrUpdate:
Failure sending request:
StatusCode=400
--
Original Error:
Code="InvalidResourceReference... .../k8sthw-lb/backendAddressPools/k8sthw-lb-pool referenced by resource
/subscriptions/fake-azure-subscription-id/resourceGroups/k8sthw-rg/providers/Microsoft.Network/... ... or verify that both resources are in the same region." Details=[]

Error:

Error Creating/Updating LoadBalancer: network.LoadBalancersClient

CreateOrUpdate:

Failure sending request:

StatusCode=400

Original Error:

Code="InvalidResourceReference... .../k8sthw-lb/backendAddressPools/k8sthw-lb-pool referenced by resource

/subscriptions/fake-azure-subscription-id/resourceGroups/k8sthw-rg/providers/Microsoft.Network/... ... or verify that both resources are in the same region." Details=[]

Issue

After going through these errors, I’ve realized that my resources are indeed in the same region, but existing rules are referencing the current backend pool by name and actually blocking Terraform in renaming the backend pool.

There are a couple of options at this stage – you can destroy your deployment and run it again (as I normally would) and it should all be fine. Or you can try to fix only the dependent resources and make it work as part of the existing deployment.

With some spare time at my hands, I’ve tried to fix it using the second one and it actually worked.

Resolution

Terraform has a nice option for destroying just some parts of the deployment.

If you look at help for the terraform destroy command, you can see the target option:

terraform destroy -help
# ...
# -target=resource Resource to target. Operation will be limited to this
#                  resource and its dependencies. This flag can be used
#                  multiple times.
# ...

terraform destroy -help

# ...

# -target=resource Resource to target. Operation will be limited to this

# resource and its dependencies. This flag can be used

# multiple times.

# ...

And if you run it to fix your issues, you’ll get a nice red warning saying that this is only for exceptional situations (and they mean it!):

terraform destroy -target azurerm_lb_outbound_rule.k8sthw-lb-out
# Warning: Resource targeting is in effect
#
# You are creating a plan with the -target option, which means that the result
# of this plan may not represent all of the changes requested by the current
# configuration.
# 
# The -target option is not for routine use, and is provided only for
# exceptional situations such as recovering from errors or mistakes, or when
# Terraform specifically suggests to use it as part of an error message.
# 
# Do you really want to destroy all resources?
# Terraform will destroy all your managed infrastructure, as shown above.
# There is no undo. Only 'yes' will be accepted to confirm.
# 
# Enter a value: yes
# ...

terraform destroy -target azurerm_lb_outbound_rule.k8sthw-lb-out

# Warning: Resource targeting is in effect

# You are creating a plan with the -target option, which means that the result

# of this plan may not represent all of the changes requested by the current

# configuration.

# The -target option is not for routine use, and is provided only for

# exceptional situations such as recovering from errors or mistakes, or when

# Terraform specifically suggests to use it as part of an error message.

# Do you really want to destroy all resources?

# Terraform will destroy all your managed infrastructure, as shown above.

# There is no undo. Only 'yes' will be accepted to confirm.

# Enter a value: yes

# ...

So… BE CAREFUL! (can’t stress this enough!)

Anyhow, I’ve destroyed rules which were preventing my rename operation:

terraform destroy -target azurerm_lb_outbound_rule.k8sthw-lb-out -target azurerm_lb_rule.k8sthw-lb-80-rule -target azurerm_lb_rule.k8sthw-lb-443-rule --auto-approve
# ...
#
# Destroy complete! Resources: 3 destroyed.

terraform destroy -target azurerm_lb_outbound_rule.k8sthw-lb-out -target azurerm_lb_rule.k8sthw-lb-80-rule -target azurerm_lb_rule.k8sthw-lb-443-rule --auto-approve

# ...

# Destroy complete! Resources: 3 destroyed.

And then another terraform apply –auto-approve recreated everything that was needed, and finally – my backend pool got renamed:

# ...
# azurerm_lb_backend_address_pool.k8sthw-controllers-lb-pool: Refreshing state... [id=/subscriptions/fake-azure-subscription-id/resourceGroups/k8sthw-rg/providers/Micro...
# ...
# 
# Apply complete! Resources: 2 added, 0 changed, 3 destroyed.

# ...

# azurerm_lb_backend_address_pool.k8sthw-controllers-lb-pool: Refreshing state... [id=/subscriptions/fake-azure-subscription-id/resourceGroups/k8sthw-rg/providers/Micro...

# ...

# Apply complete! Resources: 2 added, 0 changed, 3 destroyed.

Another idea I’ve had was to taint the resources (terraform taint -help), which would probably be a lot nicer. Oh, well… maybe next time. 🙂

As things are constantly improving, it shouldn’t be long until this is fixed (should it even be fixed?!). Until then, hope this will help you with similar issues!

Cheers!