Decided to have some fun with (nested) Microsoft Azure Stack HCI in my lab.
If you want to do the same, I’ve scripted most the stuff you need, so… maybe it will be useful.
Steps to prepare a brand new, shiny, nested Azure Stack HCI lab are (roughly):
- prepare your (parent) Windows Server 2022 Hyper-V host (ensure enough resources are available)
- it already hosts my Active Directory, DNS, DHCP, router, …,ย VMs
- everything will be saved locally to D:\AzureStackHCI
- (optional) install Windows Admin Center (WAC)ย for easier management
- download it and install with simple command:
1 2 3 |
### install Windows Admin Center Invoke-WebRequest 'https://aka.ms/WACDownload' -OutFile "D:\AzureStackHCI\WindowsAdminCenter2110.msi" C:\Windows\System32\msiexec.exe /i "D:\AzureStackHCI\WindowsAdminCenter2110.msi" /qn /L*v "D:\AzureStackHCI\WindowsAdminCenter2110.log" SME_PORT=443 SSL_CERTIFICATE_OPTION=generate |
- obtain the Azure Stack HCI 60-day trial ISO image from here
- make VHD(X) from the obtained ISO image:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
### check what's in the Azure Stack HCI ISO file dism /Get-WimInfo /WimFile:E:\sources\install.wim # Deployment Image Servicing and Management tool # Version: 10.0.20348.1 # Details for image : E:\sources\install.wim # Index : 1 # Name : Azure Stack HCI # Description : This option installs Azure Stack HCI. # Size : 7.996.171.122 bytes # The operation completed successfully. ### convert ISO to VHDX # https://github.com/x0nn/Convert-WindowsImage $iso = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.iso" $vhd = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.vhdx" Convert-WindowsImage -SourcePath $iso -VHDFormat "VHDX" -Edition "Azure Stack HCI" -SizeBytes 200GB -DiskLayout "UEFI" -VHDPath $vhd |
-
- note that I’m using Convert-WindowsImage.ps1 available here
- which gives me nice, generalized Azure Stack HCI VHD(X), which we will “upgrade with things” and later use for VM creation
- install prerequisites into VHD(X)
- this one is fairly easy – install Windows roles and features directly to the VHD(X) itself:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
### install Azure Stack HCI cluster prerequisites $vhd = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.vhdx" $roles = @( "BitLocker", "Data-Center-Bridging", "EnhancedStorage", "Failover-Clustering", "FS-FileServer", "FS-Data-Deduplication", "Hyper-V", "Hyper-V-PowerShell", "NetworkATC", "RSAT-AD-PowerShell", "RSAT-Clustering-PowerShell", "Storage-Replica" ) Install-WindowsFeature -Vhd $vhd -Name $roles -IncludeAllSubFeature -IncludeManagementTools |
-
- NOTE: If you try to install the Hyper-V role later, it may fail as we’re running Azure Stack HCI on “normal” Windows Server, so it get’s confused with nested virtualization availability. With preinstaling it, we make sure it just works.
- update VHD(X) with latest patches:
1 2 3 4 5 6 7 8 9 |
### install Windows updates (into an offline VHDX image) $vhd = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.vhdx" $packages = "D:\AzureStackHCI\Updates" # mount image $drive = (Mount-VHD -Path $vhd -PassThru | Get-Disk | Get-Partition | Get-Volume).DriveLetter # install updates from folder Add-WindowsPackage -Path "$(([string]$drive).Trim()):\" -PackagePath $packages -IgnoreCheck # dismount image Dismount-Vhd -Path $vhd |
-
- I have previously downloaded all the Azure Stack HCI patches available to D:\AzureStackHCI\Updates
- add Unattend.xml to handle the “set password at first login issue”
- it annoys me that I need to set up the initial password, so… simple Unattend.xml file, injected into VHD(X) should take care of this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
<?xml version="1.0" encoding="utf-8"?> <unattend xmlns="urn:schemas-microsoft-com:unattend"> <settings pass="oobeSystem"> <component name="Microsoft-Windows-Shell-Setup" processorArchitecture="wow64" publicKeyToken="31bf3856ad364e35" language="neutral" versionScope="nonSxS" xmlns:wcm="http://schemas.microsoft.com/WMIConfig/2002/State" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <UserAccounts> <AdministratorPassword> <Value>UABAAHMAcwB3ADAAcgBkACEAQQBkAG0AaQBuAGkAcwB0AHIAYQB0AG8AcgBQAGEAcwBzAHcAbwByAGQA</Value> <PlainText>false</PlainText> </AdministratorPassword> </UserAccounts> </component> </settings> <cpi:offlineImage cpi:source="wim:c:/sources/install.wim#Azure Stack HCI SERVERAZURESTACKHCICORE" xmlns:cpi="urn:schemas-microsoft-com:cpi" /> </unattend> |
1 2 3 4 5 6 7 8 |
### inject Unattend.xml (into an offline VHDX image) $vhd = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.vhdx" # mount image $drive = (Mount-VHD -Path $vhd -PassThru | Get-Disk | Get-Partition | Get-Volume).DriveLetter # inject Unattend.xml Copy-Item -Path ".\Unattend.xml" -Destination "$(([string]$drive).Trim()):\" # dismount image Dismount-Vhd -Path $vhd |
-
- NOTE: Make sure you don’t use clear-text passwords in Unattend.xml file!
- create Azure Stack HCI VMs
- I’m creating two VMs from our prepared VHD(X), with a couple of additional data disks, few network adapters for different purposes, nested virtualization enabled, etc.:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
### create vms $vms = "AZS1", "AZS2" foreach ($vm in $vms) { # prepare paths $vmpath = "D:\AzureStackHCI\VMs" $vhd = "D:\AzureStackHCI\VMs\$vm\Virtual Hard Disks\$vm" $vhdt = "D:\AzureStackHCI\AzureStackHCI_20348.288_en-us.vhdx" # prepare folders New-Item -Name $vm -Type Directory -Path $vmpath -Force New-Item -Name "Virtual Machines" -Type Directory -Path "$vmpath\$vm" -Force New-Item -Name "Virtual Hard Disks" -Type Directory -Path "$vmpath\$vm" -Force # copy boot disk Copy-Item -Path $vhdt -Destination "$($vhd)_0.vhdx" # create vm New-VM -Name $vm -MemoryStartupBytes 64GB -VHDPath "$($vhd)_0.vhdx" -SwitchName Corporate -Generation 2 -Path $vmpath Set-VM -VMName $vm -AutomaticStartAction Nothing -AutomaticStopAction ShutDown -CheckpointType Standard Set-VMProcessor -VMName $vm -Count 16 -ExposeVirtualizationExtensions $true New-VHD -Path "$($vhd)_1.vhdx" -SizeBytes 480GB -Dynamic Add-VMHardDiskDrive -VMName $vm -Path "$($vhd)_1.vhdx" New-VHD -Path "$($vhd)_2.vhdx" -SizeBytes 480GB -Dynamic Add-VMHardDiskDrive -VMName $vm -Path "$($vhd)_2.vhdx" New-VHD -Path "$($vhd)_3.vhdx" -SizeBytes 480GB -Dynamic Add-VMHardDiskDrive -VMName $vm -Path "$($vhd)_3.vhdx" New-VHD -Path "$($vhd)_4.vhdx" -SizeBytes 480GB -Dynamic Add-VMHardDiskDrive -VMName $vm -Path "$($vhd)_4.vhdx" Rename-VMNetworkAdapter -VMName $vm -NewName "Corporate" Set-VMNetworkAdapter -VMName $vm -Name "Corporate" -DeviceNaming "On" Add-VMNetworkAdapter -VMName $vm -Name "Storage_1" -SwitchName "Storage_1" -DeviceNaming "On" Add-VMNetworkAdapter -VMName $vm -Name "Storage_2" -SwitchName "Storage_2" -DeviceNaming "On" Add-VMNetworkAdapter -VMName $vm -Name "VM" -SwitchName "Corporate" -DeviceNaming "On" Set-VMNetworkAdapter -VMName $vm -Name "VM" -MacAddressSpoofing "On" Set-VMFirmware -VMName $vm -BootOrder (Get-VMHardDiskDrive -VMName $vm -ControllerNumber 0 -ControllerLocation 0) Start-VM -VMName $vm } |
- set node networking, join them to the domain, prepare for cluster (by using PowerShell Direct):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
### customize AZS1 # set networking $localAdmin = Get-Credential -UserName "AZSx\Administrator" -Message "Enter password for the local Administrator account:" Enter-PSSession -VMName "AZS1" -Credential $localAdmin # rename network adapters per Hyper-V adapter name $adapters = "Corporate", "Storage_1", "Storage_2", "VM" foreach ($adapter in $adapters) { Get-NetAdapterAdvancedProperty -RegistryKeyword HyperVNetworkAdapterName | Where-Object { $_.DisplayValue -eq $adapter } | Rename-NetAdapter -NewName $adapter } # set IP addresses (AZS1) $ia = "Corporate" New-NetIPAddress -InterfaceAlias $ia -IPAddress "192.168.111.51" -PrefixLength "24" -DefaultGateway "192.168.111.1" Set-DnsClientServerAddress -InterfaceAlias $ia -ServerAddresses "192.168.111.5" Set-DnsClient -InterfaceAlias $ia -ConnectionSpecificSuffix "lab.tklabs.eu" Set-NetIPInterface -InterfaceAlias $ia -InterfaceMetric 1 # test if it works $dc = Resolve-DnsName "DOM1" Test-NetConnection $dc.IPAddress # set other IP addresses (AZS1) $ia = "Storage_1" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.111.1" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false $ia = "Storage_2" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.112.1" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false $ia = "VM" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.113.1" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false # enable RDP Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -Name "fDenyTSConnections" -Value 0 Enable-NetFirewallRule -DisplayGroup "Remote Desktop" # Power Options - High Performance (Get-WmiObject -Namespace root\cimv2\power -Class Win32_PowerPlan -Filter "ElementName = 'High Performance'").Activate() # Rename C: Set-Volume -DriveLetter "C" -NewFileSystemLabel "OSDisk" # timezone Set-Timezone "Central European Standard Time" # Windows Update PowerShell module [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force Set-PSRepository -Name 'PSGallery' -InstallationPolicy Trusted Install-Module PSWindowsUpdate # domain join Add-Computer -DomainName "lab.tklabs.eu" -Credential "LAB\Administrator" -NewName "AZS1" -Restart -Force ## additional settings $domainadmin = Get-Credential -UserName "LAB\Administrator" -Message "Enter password for the domain Administrator account:" Enter-PSSession -VMName "AZS1" -Credential $domainadmin # disable SConfig on login Set-SConfig -AutoLaunch $false # Windows Update Install-WindowsUpdate -MicrosoftUpdate -AcceptAll -Verbose |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
### customize AZS2 # set networking $localAdmin = Get-Credential -UserName "AZSx\Administrator" -Message "Enter password for the local Administrator account:" Enter-PSSession -VMName "AZS2" -Credential $localAdmin # rename network adapters per Hyper-V adapter name $adapters = "Corporate", "Storage_1", "Storage_2", "VM" foreach ($adapter in $adapters) { Get-NetAdapterAdvancedProperty -RegistryKeyword HyperVNetworkAdapterName | Where-Object { $_.DisplayValue -eq $adapter } | Rename-NetAdapter -NewName $adapter } # set IP addresses (AZS2) $ia = "Corporate" New-NetIPAddress -InterfaceAlias $ia -IPAddress "192.168.111.52" -PrefixLength "24" -DefaultGateway "192.168.111.1" Set-DnsClientServerAddress -InterfaceAlias $ia -ServerAddresses "192.168.111.5" Set-DnsClient -InterfaceAlias $ia -ConnectionSpecificSuffix "lab.tklabs.eu" Set-NetIPInterface -InterfaceAlias $ia -InterfaceMetric 1 # test if it works $dc = Resolve-DnsName "DOM1" Test-NetConnection $dc.IPAddress # set other IP addresses (AZS2) $ia = "Storage_1" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.111.2" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false $ia = "Storage_2" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.112.2" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false $ia = "VM" New-NetIPAddress -InterfaceAlias $ia -IPAddress "10.111.113.2" -PrefixLength "30" Set-DnsClient -InterfaceAlias $ia -RegisterThisConnectionsAddress $false # enable RDP Set-ItemProperty -Path 'HKLM:\System\CurrentControlSet\Control\Terminal Server' -Name "fDenyTSConnections" -Value 0 Enable-NetFirewallRule -DisplayGroup "Remote Desktop" # Power Options - High Performance (Get-WmiObject -Namespace root\cimv2\power -Class Win32_PowerPlan -Filter "ElementName = 'High Performance'").Activate() # Rename C: Set-Volume -DriveLetter "C" -NewFileSystemLabel "OSDisk" # timezone Set-Timezone "Central European Standard Time" # Windows Update PowerShell module [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force Set-PSRepository -Name 'PSGallery' -InstallationPolicy Trusted Install-Module PSWindowsUpdate # domain join Add-Computer -DomainName "lab.tklabs.eu" -Credential "LAB\Administrator" -NewName "AZS2" -Restart -Force ## additional settings $domainadmin = Get-Credential -UserName "LAB\Administrator" -Message "Enter password for the domain Administrator account:" Enter-PSSession -VMName "AZS2" -Credential $domainadmin # disable SConfig on login Set-SConfig -AutoLaunch $false # Windows Update Install-WindowsUpdate -MicrosoftUpdate -AcceptAll -Verbose |
- create the Azure Stack HCI cluster:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
### create Azure Stack HCI cluster # prepare and clean disks $servers = "AZS1", "AZS2" $cluster = "AzSCluster" $clusterIp = "192.168.111.50/24" Invoke-Command ($servers) { Update-StorageProviderCache Get-StoragePool | Where-Object { $_.IsPrimordial -eq $false } | Set-StoragePool -IsReadOnly:$false -ErrorAction SilentlyContinue Get-StoragePool | Where-Object { $_.IsPrimordial -eq $false } | Get-VirtualDisk | Remove-VirtualDisk -Confirm:$false -ErrorAction SilentlyContinue Get-StoragePool | Where-Object { $_.IsPrimordial -eq $false } | Remove-StoragePool -Confirm:$false -ErrorAction SilentlyContinue Get-PhysicalDisk | Reset-PhysicalDisk -ErrorAction SilentlyContinue Get-Disk | Where-Object { $_.Number -ne $null } | Where-Object { $_.IsBoot -ne $true } | Where-Object { $_.IsSystem -ne $true } | Where-Object { $_.PartitionStyle -ne "RAW" } | ForEach-Object { $_ | Set-Disk -IsOffline:$false $_ | Set-Disk -IsReadOnly:$false $_ | Clear-Disk -RemoveData -RemoveOEM -Confirm:$false $_ | Set-Disk -IsReadOnly:$true $_ | Set-Disk -IsOffline:$true } Get-Disk | Where-Object { $_.Number -ne $Null } | Where-Object { $_.IsBoot -ne $True } | Where-Object { $_.IsSystem -ne $True } | Where-Object { $_.PartitionStyle -eq "RAW" } | Group-Object -NoElement -Property "FriendlyName" } | Sort-Object -Property "PsComputerName", "Count" # Count Name PSComputerName # ----- ---- -------------- # 4 Msft Virtual Disk AZS1 # 4 Msft Virtual Disk AZS2 # check if cluster can be created Test-Cluster -Node $servers -Include "Storage Spaces Direct", "Inventory", "Network", "System Configuration" # create cluster New-Cluster -Name $cluster -Node $servers -StaticAddress $clusterIp -NoStorage # check cluster Get-Cluster -Name $cluster | Get-ClusterResource # Name State OwnerGroup ResourceType # ---- ----- ---------- ------------ # Cluster IP Address Online Cluster Group IP Address # Cluster Name Online Cluster Group Network Name # Virtual Machine Cluster WMI Online Cluster Group Virtual Machine Cluster WMI # set cluster quorum (file share witness) Set-ClusterQuorum -Cluster $cluster -FileShareWitness "\\DOM1.lab.tklabs.eu\Witness_AZSCluster$" # Cluster QuorumResource # ------- -------------- # AzSCluster File Share Witness Get-Cluster -Name $cluster | Get-ClusterResource # Name State OwnerGroup ResourceType # ---- ----- ---------- ------------ # Cluster IP Address Online Cluster Group IP Address # Cluster Name Online Cluster Group Network Name # File Share Witness Online Cluster Group File Share Witness # Virtual Machine Cluster WMI Online Cluster Group Virtual Machine Cluster WMI |
- register (optional) the Azure Stack HCI cluster:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
### https://docs.microsoft.com/en-us/azure-stack/hci/deploy/register-with-azure # prerequisites Install-Module -Name "Az.StackHCI" -Force -Confirm:$false # registration Register-AzStackHCI -SubscriptionId "<your_Azure_subscription_ID>" -ResourceGroupName "AzureStackHCI" # WARNING: To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code DW84XWSHV to authenticate. # WARNING: We have migrated the API calls for this cmdlet from Azure Active Directory Graph to Microsoft Graph. Visit https://go.microsoft.com/fwlink/?linkid=2181475 for any permission issues. # Result : Success # AzurePortalResourceURL : https://portal.azure.com/#@<some_resource_ID>/resource/subscriptions/<your_Azure_subscription_ID>/resourceGroups/AzureStackHCI/providers/Microsoft.AzureStackHCI/clusters/AzSCluster/overview # AzureResourceId : /Subscriptions/<your_Azure_subscription_ID>/resourceGroups/AzureStackHCI/providers/Microsoft.AzureStackHCI/clusters/AzSCluster # AzurePortalAADAppPermissionsURL : https://portal.azure.com/#blade/Microsoft_AAD_RegisteredApps/ApplicationMenuBlade/CallAnAPI/appId/<some_App_ID>/isMSAApp/ # Details : Azure Stack HCI is successfully registered. An Azure resource representing Azure Stack HCI has been created in your Azure subscription to enable an Azure-consistent monitoring, billing, and support experience. |
- create CSV(s) and virtual switches for child workloads (but add nodes/cluster to WAC before, if not using PowerShell)
- NOTE: I’m not using intent-based networking at this time. Nor SDN.
- play around with your new cluster
- (optional) clean all/redeploy if needed:
1 2 3 4 |
### cleanup script Get-VM "AZS*" | Stop-VM -TurnOff -Force Remove-VM "AZS*" -Force Remove-Item "D:\AzureStackHCI\VMs\AZS*" -Force -Recurse |
And now you have fully functional, nested, 2-node Azure Stack HCI cluster – nothing too fancy, but you can extend it how you wish! ๐
You can begin exploring the Azure Stack HCI itself, use it with Azure Arc, or perhaps install AKS on Azure Stack HCI and play around with it. Or something else.
Cheers!
P.S. You can use these scripts also for stuff other than Azure Stack HCI, of course! ๐
P.P.S. Code is also available on my GitHub page.