Affinity Groups in Azure

I see a lot of confusion about the current status of affinity groups in Microsoft Azure and thought it worthwhile to describe that status (or my view of it).

Until 2012, Azure used a north-south network built to handle traditional internet traffic. In this scenario a request comes in from the internet at the top and gets routed down to a server. The response then gets routed back up to the internet. A problem with a datacenter designed in this way is that traffic from one server to another also gets routed up and down – at the cost of increased latency between servers.

Azure applications make heavy use of Azure Storage, so to minimize latency Azure supports the colocation of compute and storage inside a datacenter. This is achieved by the deployment of a cloud service and related storage accounts into an affinity group, which is a logical name for a colocated set of compute and storage nodes. An affinity group is created in a single region and can be used, instead of the region, as the hosting location for a cloud service or a storage account. Until 2012, the use of an affinity group was the recommended way to deploy a cloud service and any related storage accounts.

In 2012, Azure was upgraded to use an east-west network built to handle the compute-to-storage traffic that was dominant in an Azure datacenter. Furthermore, the network was converted to a flat network in which network traffic was routed directly from compute to storage nodes. Additionally network capacity and speed were both significantly upgraded. Brad Calder described this upgrade in a post on the Azure blog.

This network upgrade was essential to the 2012 release of Azure Virtual Machines, the IaaS offering in the Azure Platform. The OS and Data Disks used in the VMs in Azure Virtual Machines are backed by page blobs in Azure Storage accessed across the network. However, since the upgrade there has been less emphasis in documentation on the need to use an affinity group to colocate a cloud service and related storage accounts.

In 2012, Azure Virtual Network was released. Azure Virtual Network supported the creation of a VNET but required that the VNET be created in an affinity group. This meant that any cloud services in the VNET were also in the affinity group.

So what was the problem with affinity groups. The problem was that an affinity group is tied to a particular set of hardware. This was not an issue when Azure provided only one class of hardware – the A0-A4 standard instances. In 2013, however, Azure released high-memory instances (A5-A7) and high CPU instances (A8 and A9). Instances of the new size could not be deployed into a cloud service in an affinity group created before their release. The solution was to: create a new affinity group; recreate the cloud service in the new affinity group; and redeploy the VMs into the new cloud service. This problem was exacerbated by the use of a VNET since, being tied to an affinity group, no cloud service in the VNET could access new instance types. The solution was to: create a new affinity group; recreate the VNET in the new affinity group, recreate all the cloud services; and then redeploy the VMs into the VNET and cloud services.

In 2014, Azure Virtual Network was upgraded to support regional VNETs which span a region not an affinity group. This removes the issue of access to new instance types since a regional VNET has access to all the compute resources in a region. Regional VNETs essentially deprecate affinity group VNETs and at some point every affinity group VNET will be upgraded automatically into a regional VNET.

So where does this leave affinity groups?

An affinity group serves two purposes:

1. colocation of cloud service and storage account
2. host for an affinity group VNET

The second of these is now deprecated. The first is still possible but has not been emphasized as the latency optimization it was prior to the upgrade of the Azure network in 2012. In fact the standard way to deploy a VM in the Azure Portal does not support the simultaneous creation of a cloud service in an affinity group and a regional VNET. This confuses people who try to satisfy the modern best practice of using a regional VNET for deployments and the historic best practice of creating a cloud service in an affinity group.

At this point there seems to be little reason to deploy a cloud service into an affinity group. However, as with other deployment choices it might be wise for people to test this out for their specific workloads.

Note that it is actually possible to use the Azure Portal to deploy VMs into an cloud service hosted in an affinity group and resident in a regional VNET. The trick is to do this as two steps:

• create the cloud service in an affinity group
• deploy VMs into the cloud service but host them in the VNET

The same trick can be used with Azure cmdlets.