Working With Azure VM Scale Sets - Part 1

Late in 2015 Microsoft announced a "new" feature in Azure called Virtual Machine Scale Sets. VMSS makes it easier to deploy and manage identical virtual machines that can automagically scale on demand.

Those who have worked with Azure before know that this sort of functionality already exists within Azure's PaaS offerings and is more commonly known as auto scaling for Azure Cloud Services. However this capability was only available to VMs provisioned in the classic (IaaS V1) deployment model and had a number of limitations. VMSS extends this capability from the classic deployment model into the IaaS V2 (Azure Resource Manager) deployment model.

Working with VM Scale Sets isn't particularly hard if you've worked with Azure Resource Manager (ARM) templates before. VMSSs themselves are built on two resources:

  • Microsoft.Compute/virtualMachineScaleSets, the defintion of VMs that will autoscale, and,
  • Microsoft.Insights/autoscaleSettings, the auto scaling settings that will determine when to scale up and when to scale down

Lets break down a simple ARM template to deploy a VMSS. The template below is based on the VMSS template provided by the Azure SDK for Visual Studio.

Parameters

Paremeters are the input a template takes when deploying resources. They allow you to customise the deployment by using values that are specific to that deployment.

  "parameters": {
    "vmSize": {
      "type": "string",
      "defaultValue": "Standard_A1",
      "minLength": 1
    },
    "windowsOSVersion": {
      "type": "string",
      "defaultValue": "2012-R2-Datacenter",
      "allowedValues": [
        "2008-R2-SP1",
        "2012-Datacenter",
        "2012-R2-Datacenter"
      ],
      "metadata": {
        "description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter."
      }
    },
    "vmssName": {
      "type": "string",
      "metadata": {
        "description": "DNS name used for public IP addresses and as base for naming other resources. Must be globally unique and 3 to 9 characters long."
      },
      "minLength": 3,
      "maxLength": 9
    },
    "instanceCount": {
      "type": "int",
      "metadata": {
        "description": "Number of VM instances (100 or less)"
      },
      "minValue": 1,
      "maxValue": 100
    },
    "adminUsername": {
      "type": "string",
      "minLength": 1,
      "metadata": {
        "description": "Administrator username on all VMs"
      }
    },
    "adminPassword": {
      "type": "securestring",
      "metadata": {
        "description": "Administrator password on all VMs"
      }
    }
  }

The metadata in the parameters should be fairly self explanatory so I wont belabor the point.

Variables

Variables are values constructed within the template for use. They are usually built of parameters are statically defined. The variables used by the VMSS template can be grouped to make them slightly easier to understand.

General Variables

"location": "[resourceGroup().location]",
"namingInfix": "[toLower(parameters('vmssName'))]",

These are variables that are generally used across the template.

  • location is defined by calling a template function that returns location of the resource group. I'm not sure if its event possible to specify a resource group with resources in more than one Azure location so it's unclear to me why location isn't implied on all template resources, but maybe that's a question for another day.
  • namingInfix takes the vmssName and forces it to all lower case for consistency

Storage Variables

"storageAccountType": "Standard_LRS",
"newStorageAccountSuffix": "[concat(variables('namingInfix'), 'sa')]",

"uniqueStringArray": [
      "[concat(uniqueString(concat(resourceGroup().id, variables('newStorageAccountSuffix'), '0')), variables('newStorageAccountSuffix'))]",
      "[concat(uniqueString(concat(resourceGroup().id, variables('newStorageAccountSuffix'), '1')), variables('newStorageAccountSuffix'))]",
      "[concat(uniqueString(concat(resourceGroup().id, variables('newStorageAccountSuffix'), '2')), variables('newStorageAccountSuffix'))]",
      "[concat(uniqueString(concat(resourceGroup().id, variables('newStorageAccountSuffix'), '3')), variables('newStorageAccountSuffix'))]",
      "[concat(uniqueString(concat(resourceGroup().id, variables('newStorageAccountSuffix'), '4')), variables('newStorageAccountSuffix'))]"
],

These are variables that are used to define the storage accounts used by the VMSS

  • storageAccountType is statically defined as Standard_LRS. As a general rule you want scale set servers to be stateless so there's no need for any additional level of storage protection provided by Azure.
  • newStorageAccountSuffix concatenates the namingInfix variable and "sa" to create name suffixes for the storage accounts that are required.
  • uniqueStringArray is an array of unique names for the storage accounts that are going to be used by the VMSS. It creates the unique(ish) by concatenating the resource group id, the newStorageAccountSuffix variable, and a number, then hashing that string. It then appends the newStorageAccountSuffix onto the string again to ensure you know which VMSS the storage account belongs to. This somewhat convoluted process is required because all Azure storage account names must be globally unique. This constraint can often leave convention based names unavailable. Five of these unique names are created so that the VMSS virtual machine instances can be distributed across multiple storage accounts.

VM Variables

"vhdContainerName": "[concat(variables('namingInfix'), 'vhd')]",
"osDiskName": "[concat(variables('namingInfix'), 'osdisk')]",
"osType": {
  "publisher": "MicrosoftWindowsServer",
  "offer": "WindowsServer",
  "sku": "[parameters('windowsOSVersion')]",
  "version": "latest"
},
"imageReference": "[variables('osType')]",

VM variables define the configuration of the virtual machines that will make up the VMSS.

  • vhdContainerName concatenates the namingInfix variable and "vhd" to give a name to the blob storage container that VMSS VHDs on each storage account
  • osDiskName concatenates the namingInfix variable with "osdisk" to give a name to the operating system disk for each VMSS VM instance. Dont worry about name uniqueness here, Azure will automatically append a guid to the disk name when it creates it.
  • osType defines the various properties of the Windows OS that the VMSS virtual machines will use.
    • publisher and offer just tell Azure that our virtual machines will run Windows server images published by Microsoft. If you want to run another image from the marketplace as your starting point, change these values as appropriate to that image.
    • sku is populated by the windowsOSVersion parameter and will determine what OS package (08R2, 12, 12R2, etc) is actually deployed.
    • version is, in this case, defined as latest version of the Windows Server image published to Azure. If the application that will live on your VMSS requires a specific patch/update level then you may want to consider defining an actual version

Network Variables

"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.0.0/24",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
  • addressPrefix is the address range of the VNET being created for the VMSS.
  • subnetPrefix is the address range of the subnet within my VNET that will hold the VMSS instances.
  • virtualNetworkName is a descriptive name for the VNET, in this case we're joining the vmss name and "vnet"
  • publicIPAddressName is the descriptive name of the public IP that will load balance my VMSS services, in this case we join the VMSS name with "pip".
  • subnetName is the name of the subnet being deployed to hold the VMSS instances, in this case, the VMSS name and "subnet".

Azure DSC Variables

"DSCModuleUrl": "https://sfogdemo.blob.core.windows.net/demoassets/DSC-EnableIIS.ps1.zip",
"DSCFunction": "DSC-EnableIIS.ps1\\EnableIIS",

Load Balancing Variables

"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"lbID": "[resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName'))]",
"frontEndIPConfigID": "[concat(variables('lbID'), '/frontendIPConfigurations/loadBalancerFrontEnd')]",
"backEndIPConfigID": "[concat(variables('lbID'), '/backendAddressPools/LoadBalancerBackEnd')]",
  • loadBalancerName is the name of the load balancer.
  • lbID is the resource id of the load balancer that will be created. This is created by using the resourceid function, telling it what type of resource it is, and then the name of the resource. The function then returns the fully qualified resource ID of the resource, which looks something like this "/subscriptions/abcdefgh-ijkl-mnop-qrst-uvwxyz123456/resourceGroups/vmssdemo/providers/Microsoft.Network/loadBalancers/vmssdemolb"
  • frontEndIPConfigID is the resource id of the front end configuration of the load balancer. We build this variable by joining the load balancer ID variable to the path of the front end configuration.
  • backEndIPConfigID is the resource id of the back end configuration of the load balancer. We build this variable by joining the load balancer ID variable to the path of the front end configuration.

These last two variables are used to build the actual load balancing config, and to tell the VMSS instances which load balancer rules to attach to.

Diagnostics Variables

"diagnosticsStorageAccountName": "[variables('uniqueStringArray')[0]]",
"diagnosticsStorageAccountResourceGroup": "[resourceGroup().name]",
"accountid": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', variables('diagnosticsStorageAccountResourceGroup'), '/providers/', 'Microsoft.Storage/storageAccounts/', variables('diagnosticsStorageAccountName'))]",
"wadlogs": "<WadCfg> <DiagnosticMonitorConfiguration overallQuotaInMB=\"4096\" xmlns=\"http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration\"> <DiagnosticInfrastructureLogs scheduledTransferLogLevelFilter=\"Error\"/> <WindowsEventLog scheduledTransferPeriod=\"PT1M\" > <DataSource name=\"Application!*[System[(Level = 1 or Level = 2)]]\" /> <DataSource name=\"Security!*[System[(Level = 1 or Level = 2)]]\" /> <DataSource name=\"System!*[System[(Level = 1 or Level = 2)]]\" /></WindowsEventLog>",
"wadperfcounters1": "<PerformanceCounters scheduledTransferPeriod=\"PT1M\"><PerformanceCounterConfiguration counterSpecifier=\"\\Processor(_Total)\\% Processor Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"CPU utilization\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Processor(_Total)\\% Privileged Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"CPU privileged time\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Processor(_Total)\\% User Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"CPU user time\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Processor Information(_Total)\\Processor Frequency\" sampleRate=\"PT15S\" unit=\"Count\"><annotation displayName=\"CPU frequency\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\System\\Processes\" sampleRate=\"PT15S\" unit=\"Count\"><annotation displayName=\"Processes\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Process(_Total)\\Thread Count\" sampleRate=\"PT15S\" unit=\"Count\"><annotation displayName=\"Threads\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Process(_Total)\\Handle Count\" sampleRate=\"PT15S\" unit=\"Count\"><annotation displayName=\"Handles\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Memory\\% Committed Bytes In Use\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"Memory usage\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Memory\\Available Bytes\" sampleRate=\"PT15S\" unit=\"Bytes\"><annotation displayName=\"Memory available\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Memory\\Committed Bytes\" sampleRate=\"PT15S\" unit=\"Bytes\"><annotation displayName=\"Memory committed\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\Memory\\Commit Limit\" sampleRate=\"PT15S\" unit=\"Bytes\"><annotation displayName=\"Memory commit limit\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\% Disk Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"Disk active time\" locale=\"en-us\"/></PerformanceCounterConfiguration>",
"wadperfcounters2": "<PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\% Disk Read Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"Disk active read time\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\% Disk Write Time\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"Disk active write time\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Transfers/sec\" sampleRate=\"PT15S\" unit=\"CountPerSecond\"><annotation displayName=\"Disk operations\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Reads/sec\" sampleRate=\"PT15S\" unit=\"CountPerSecond\"><annotation displayName=\"Disk read operations\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Writes/sec\" sampleRate=\"PT15S\" unit=\"CountPerSecond\"><annotation displayName=\"Disk write operations\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Bytes/sec\" sampleRate=\"PT15S\" unit=\"BytesPerSecond\"><annotation displayName=\"Disk speed\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Read Bytes/sec\" sampleRate=\"PT15S\" unit=\"BytesPerSecond\"><annotation displayName=\"Disk read speed\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\PhysicalDisk(_Total)\\Disk Write Bytes/sec\" sampleRate=\"PT15S\" unit=\"BytesPerSecond\"><annotation displayName=\"Disk write speed\" locale=\"en-us\"/></PerformanceCounterConfiguration><PerformanceCounterConfiguration counterSpecifier=\"\\LogicalDisk(_Total)\\% Free Space\" sampleRate=\"PT15S\" unit=\"Percent\"><annotation displayName=\"Disk free space (percentage)\" locale=\"en-us\"/></PerformanceCounterConfiguration></PerformanceCounters>",
"wadcfgxstart": "[concat(variables('wadlogs'), variables('wadperfcounters1'), variables('wadperfcounters2'), '<Metrics resourceId=\"')]",
"wadmetricsresourceid": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name , '/providers/', 'Microsoft.Compute/virtualMachineScaleSets/')]",
"wadcfgxend": "[concat('\"><MetricAggregation scheduledTransferPeriod=\"PT1H\"/><MetricAggregation scheduledTransferPeriod=\"PT1M\"/></Metrics></DiagnosticMonitorConfiguration></WadCfg>')]"
  • diagnosticsStorageAccountName is the name of the Azure Storage account where we will store our diagonstic metric data for autoscaling. The [0] on the end means we're referring to the first value in the array.
  • diagnosticsStorageAccountResourceGroup is the name of the resource group this resource we're creating our VMSS under. The resourceGroup().name function returns this as a string.
  • accountid is the fully qualified resource ID of the storage account used to store diagnostic metric data.

The remaining unreadable mess is one of my least favourite parts of Azure right now. This blob of XML, stored inside a series of JSON variables, represents the performance metric collection configuration used by the Azure Diagnostics extension. As far as I am aware, the only way to supply this configuration is to use this horribly formatted XML stored in JSON. The variables define what metrics to collect and the rate at which to collect them, how to aggregate them, and then defines which resource the summary metrics should be stored against.

Resources

Now that we've gotten the variables out of the way, lets look at some of the resources we have to deploy.

VNETs

Starting from the ground up, I need a VNET to hold my VMSS.

    {
      "type": "Microsoft.Network/virtualNetworks",
      "name": "[variables('virtualNetworkName')]",
      "location": "[variables('location')]",
      "apiVersion": "2015-06-15",
      "tags": {
        "displayName": "VirtualNetwork"
      },
      "properties": {
        "addressSpace": {
          "addressPrefixes": [
            "[variables('addressPrefix')]"
          ]
        },
        "subnets": [
          {
            "name": "[variables('subnetName')]",
            "properties": {
              "addressPrefix": "[variables('subnetPrefix')]"
            }
          }
        ]
      }
    },

Here I'm defining my VNET, where it is, what its overall address space is, and the subnets I'm defining within that address space, and what each component will be called.
I'm using my variables to define all my VNET values where I don't want them to be static. This is what makes Azure Resources Manager templates so re-usable and flexible.

Storage Accounts

    {
      "copy": {
        "name": "storageLoop",
        "count": 5
      },
      "type": "Microsoft.Storage/storageAccounts",
      "name": "[variables('uniqueStringArray')[copyIndex()]]",
      "location": "[variables('location')]",
      "apiVersion": "2015-06-15",
      "tags": {
        "displayName": "StorageAccounts"
      },

      "properties": {
        "accountType": "[variables('storageAccountType')]"
      }
    },

Here I'm defining the storage accounts where the VMSS instances will be held. What the "copy" section does is tells ARM to create a loop, and execute the configuration 5 times. To ensure they have distinct names, the copyIndex() function to return the current iteration of the loop, and then references that array member inside the "uniqieStringArray" variable as the name of the storage account. This is a quick way to define multiple storage accounts inside a template without having to create a resource definition for each individual account, it also gives them each a name that can be programatically referred to.

Public IP Addresses

    {
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[variables('publicIPAddressName')]",
      "location": "[variables('location')]",
      "apiVersion": "2015-06-15",
      "tags": {
        "displayName": "PublicIP"
      },
      "properties": {
        "publicIPAllocationMethod": "Dynamic",
        "dnsSettings": {
          "domainNameLabel": "[variables('namingInfix')]"
        }
      }
    },

Nothing particularly fancy here, I'm requesting a public IP with a dynamic reservation and giving it a DNS name in the cloudapp namespace. These DNS names are created as "yourdnslabel"."yourazureregion".cloudapp.azure.com. The DNS name is useful so you can cname resources to point to things in Azure even if the dynamically reserved IP address changes.

Load Balancer

    {
      "type": "Microsoft.Network/loadBalancers",
      "name": "[variables('loadBalancerName')]",
      "location": "[variables('location')]",
      "apiVersion": "2015-06-15",
      "tags": {
        "displayName": "LoadBalancer"
      },
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
      ],
      "properties": {
        "frontendIPConfigurations": [
          {
            "name": "LoadBalancerFrontEnd",
            "properties": {
              "publicIPAddress": {
                "id": "[variables('publicIPAddressID')]"
              }
            }
          }
        ],
        "backendAddressPools": [
          {
            "name": "LoadBalancerBackEnd"
          }
        ],
        "loadBalancingRules": [
          {
            "name": "weblb",
            "properties": {
              "protocol": "Tcp",
              "frontendPort": 80,
              "backendPort": 80,
              "enableFloatingIP": false,
              "frontendIPConfiguration": {
                "id": "[variables('frontEndIPConfigID')]"
              },
              "backendAddressPool": {
                "id": "[variables('backEndIPConfigID')]"
              }
            }
          }
        ]
      }
    },

There's a bit of stuff going on with the load balancer config so we'll break it down bit by bit.

The first thing to is that the load balancer has a dependency on the VNET, the "dependsOn" property. This tells Azure to not try to create the load balancer until the VNET has been created, a useful feature when one resource may have dependencies on other resources existing first

frontendIpConfiguration tells the load balancer what IP it should load balance services on, in more familiar terms, this is the Virtual IP (VIP) of the load balancer. Here we are assigning it the public IP we created earlier, you could also assign it an internal IP if you only wanted to load balance an internal service.

backendIpConfiguration is the pool of back end interfaces that can receive traffic from the front end. In the world of Azure, we attach our backend interfaces to the pool from the NIC config rather than on load balancer itself (see the scale set config below).

loadBalancingRules is the load balancing configuration to be applied. Here we supply the front and back end ports, as well as which front end and back end pools this configuration applies to. In our example we only have a single set of pools and a single rule, but you can also attach multiple pools and rules to a single load balancer.

Virtual Machine Scale Set

    {
      "type": "Microsoft.Compute/virtualMachineScaleSets",
      "name": "[variables('namingInfix')]",
      "location": "[variables('location')]",
      "apiVersion": "2015-06-15",
      "tags": {
        "displayName": "VM Scale Set Configuration"
      },
      "dependsOn": [
        "storageLoop",
        "[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]",
        "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
        "[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]"
      ],
      "sku": {
        "name": "[parameters('vmSize')]",
        "tier": "Standard",
        "capacity": "[parameters('instanceCount')]"
      },
      "properties": {
        "upgradePolicy": {
          "mode": "Manual"
        },
        "virtualMachineProfile": {
          "storageProfile": {
            "osDisk": {
              "vhdContainers": [
                "[concat('https://', variables('uniqueStringArray')[0], '.blob.core.windows.net/', variables('vhdContainerName'))]",
                "[concat('https://', variables('uniqueStringArray')[1], '.blob.core.windows.net/', variables('vhdContainerName'))]",
                "[concat('https://', variables('uniqueStringArray')[2], '.blob.core.windows.net/', variables('vhdContainerName'))]",
                "[concat('https://', variables('uniqueStringArray')[3], '.blob.core.windows.net/', variables('vhdContainerName'))]",
                "[concat('https://', variables('uniqueStringArray')[4], '.blob.core.windows.net/', variables('vhdContainerName'))]"
              ],
              "name": "[variables('osDiskName')]",
              "caching": "ReadOnly",
              "createOption": "FromImage"
            },
            "imageReference": "[variables('imageReference')]"
          },
          "osProfile": {
            "computerNamePrefix": "[variables('namingInfix')]",
            "adminUsername": "[parameters('adminUsername')]",
            "adminPassword": "[parameters('adminPassword')]"
          },
          "networkProfile": {
            "networkInterfaceConfigurations": [
              {
                "name": "[variables('nicName')]",
                "properties": {
                  "primary": true,
                  "ipConfigurations": [
                    {
                      "name": "[variables('ipConfigName')]",
                      "properties": {
                        "subnet": {
                          "id": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'), '/subnets/', variables('subnetName'))]"
                        },
                        "loadBalancerBackendAddressPools": [
                          {
                            "id": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/backendAddressPools/LoadBalancerBackEnd')]"
                          }
                        ]
                      }
                    }
                  ]
                }
              }
            ]
          },
          "extensionProfile": {
            "extensions": [
              {
                "name": "DSC.Enable.IIS",
                "type": "extensions",
                "properties": {
                  "publisher": "Microsoft.Powershell",
                  "type": "DSC",
                  "typeHandlerVersion": "2.10",
                  "settings": {
                    "modulesUrl": "[variables('DSCModuleUrl')]",
                    "configurationFunction": "[variables('DSCFunction')]"
                  }
                }
              },
              {
                "name": "AzureDiagnostics",
                "properties": {
                  "publisher": "Microsoft.Azure.Diagnostics",
                  "type": "IaaSDiagnostics",
                  "typeHandlerVersion": "1.5",
                  "autoUpgradeMinorVersion": true,
                  "settings": {
                    "xmlCfg": "[base64(concat(variables('wadcfgxstart'), variables('wadmetricsresourceid'), variables('namingInfix'), variables('wadcfgxend')))]",
                    "storageAccount": "[variables('diagnosticsStorageAccountName')]"
                  },
                  "protectedSettings": {
                    "storageAccountName": "[variables('diagnosticsStorageAccountName')]",
                    "storageAccountKey": "[listkeys(variables('accountid'), '2015-06-15').key1]",
                    "storageAccountEndPoint": "https://core.windows.net"
                  }
                }
              }
            ]
          }
        }
      }
    },

There are many settings within a virtual machine scale set resource definition but most of them are the same as those you would normally see when deploying a VM with ARM. Some of the ones that are slightly different I have explained below:

  • sku defines what type of VM we're putting into the scale set, what size the VM is, and how many we're starting the scale set with.
  • upgradePolicy is how the scaleset behaves when updates are being made. If its set to manual only new VMs will get the changes.
  • vhdContainers is the list of storage account containers that can hold the VMSS instances. For larger or more intensive VMSSs it is essential to distribute the instances across multiple accounts to avoid running into the performance limits of a single storage account.
  • extensionProfile is where the Azure Diagnostic extension plugin gets installed to enable the capture of performance metrics and where we tell the VM to run the DSC module to install and configure IIS.

AutoScaling

    {
      "type": "Microsoft.Insights/autoscaleSettings",
      "apiVersion": "2015-04-01",
      "name": "[concat(variables('namingInfix'),'as1')]",
      "location": "[resourceGroup().location]",
      "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachineScaleSets/',variables('namingInfix'))]"
      ],
      "tags": {
        "displayName": "Auto Scaling Settings"
      },
      "properties": {
        "enabled": true,
        "name": "[concat(variables('namingInfix'),'as1')]",
        "profiles": [
          {
            "name": "Profile1",
            "capacity": {
              "minimum": "1",
              "maximum": "4",
              "default": "1"
            },
            "rules": [
              {
                "metricTrigger": {
                  "metricName": "\\Processor(_Total)\\% Processor Time",
                  "metricNamespace": "",
                  "metricResourceUri": "[concat('/subscriptions/',subscription().subscriptionId,'/resourceGroups/',resourceGroup().name,'/providers/Microsoft.Compute/virtualMachineScaleSets/',variables('namingInfix'))]",
                  "timeGrain": "PT1M",
                  "statistic": "Average",
                  "timeWindow": "PT15M",
                  "timeAggregation": "Average",
                  "operator": "GreaterThan",
                  "threshold": 60.0
                },
                "scaleAction": {
                  "direction": "Increase",
                  "type": "ChangeCount",
                  "value": "1",
                  "cooldown": "PT15M"
                }
              },
              {
                "metricTrigger": {
                  "metricName": "\\Processor(_Total)\\% Processor Time",
                  "metricNamespace": "",
                  "metricResourceUri": "[concat('/subscriptions/',subscription().subscriptionId,'/resourceGroups/',resourceGroup().name,'/providers/Microsoft.Compute/virtualMachineScaleSets/',variables('namingInfix'))]",
                  "timeGrain": "PT1M",
                  "statistic": "Average",
                  "timeWindow": "PT15M",
                  "timeAggregation": "Average",
                  "operator": "LessThan",
                  "threshold": 40.0
                },
                "scaleAction": {
                  "direction": "Decrease",
                  "type": "ChangeCount",
                  "value": "1",
                  "cooldown": "PT15M"
                }
              }
            ]
          }
        ],
        "targetResourceUri": "[concat('/subscriptions/',subscription().subscriptionId,'/resourceGroups/', resourceGroup().name,'/providers/Microsoft.Compute/virtualMachineScaleSets/',variables('namingInfix'))]"
      }
    }

The Auto Scaling resource is what actually adds or removes resources based on performance. It leverages the Insights Diagnostics provider of Azure to look at a resources performance metrics, and then act based on rules you define.
In this configuration we define the following behaviour:

  • Minimum of 1 instance
  • Maximum of 4 instances
  • Default is 1 instance
  • If average CPU load is greater than 60%, create a new instance
    • Load is averaged across all instances
    • Load is averaged over 15 minutes
    • Load is sampled every 1 minute
    • A new instance can only be created once every 15 minutes
  • If average CPU load is below than 40%, remove an existing instance
    • Load is averaged across all instances
    • Load is averaged over 15 minutes
    • Load is sampled every 1 minute
    • An instance can only be removed once every 15 minutes

So that's a basic VM scale set created from an ARM template. You can find the full template here: https://github.com/jj-ll/Azure/blob/master/WindowsVirtualMachineScaleSet.json

Before deploying that template would create our paremeters file so we can feed the settings we want to the template, and then deploy that via powershell.
To deploy, make sure you have the Azure Powershell installed and launch an Azure Powershell window, navigate to the path of your template and parameters file.

Login-AzureRMAccount

Enter your Azure AD Login Credentials

New-AzureRMResourceGroup -Name "foggyvmss" -Location "Australia East"

Substitute your desired name and region

New-AzureRMResourceGroupDeployment -ResourceGroupName "foggyvmss" -TemplateFile .\WindowsVirtualMachineScaleSet.json -TemplateParameterFile .\yourparametersfile.json

And then just sit back and wait. Right now there is no real feedback on deployment of VMSS from the portal GUI. However you can navigate https://resources.azure.com to view the actual status of scale set as it deploys.

Once the deployment finishes you can test your webserver by going to http://[namingInfix].[yourazureregion].cloudapp.azure.com

If you were to do something that generated lots of load on this server, RDP in and run CPUburn as an example, the auto scaling configuration would automatically create a new box with the same configuration and join it to the load balancing pool. Now your application servers scale themselves!

What Now?

Once you've played with VM Scale Sets you will notice some distinct challenges:

  1. Managing how long it takes to add new instances to the VMSS
  2. Lack of native deployment or lifecycle management options
  3. Collection and auditing of diagnostic information on instances that are ephemeral
  4. Lack of visibility and diagnostic information in the Azure Portal.
  5. Collecting and customising performance metrics for autoscaling

The next few posts in this series will cover off in more detail how to deal with each of these issues and hopefully shed some more light on how powerful VM scale sets can be.

comments powered by Disqus