Using Azure Monitoring Service API with Azure Virtual Machines

Microsoft Azure Virtual Machines is the IaaS feature in the Azure platform that provides the ability to easily deploy Linux and Windows Server VMs into a public cloud infrastructure. Like other public clouds, Azure uses commodity hardware with horizontal scaling being used to provide scalable applications. As with any scalable system it is essential that monitoring, alerting and scaling be automated.

The Azure Portal provides a graphical view of various metrics for Virtual Machines, and a new preview Microsoft Azure Monitoring Services API provides direct access to the data displayed there. This data can be used to:

  • Build monitoring dashboards
  • Provide alerts
  • Implement autoscaling

Azure provides these capabilities out of the box, but the Monitoring Services API can be used to provide more sophisticated or proprietary functionality.

The Monitoring Services API is built on top of the Azure Service Management REST API, which also provides the core API for managing Azure compute services. The Monitoring Services API is documented on MSDN. Stephen Siciliano (@icsus) does a great overview of the API on the Cloud Cover show on Channel 9.

Monitoring

The Monitoring Services API provides access to various service metrics persisted by Azure. For Virtual Machines the following metrics are captured:

Name Units Reporting
Disk Read Bytes/sec Bytes / sec Max, Min, Ave
Disk Write Bytes/sec Bytes / sec Max, Min, Ave
Network Out Bytes Total
Network In Bytes Total
Percentage CPU Percentage Max, Min, Ave

These metrics are the same as those displayed on the Azure Portal. They are all available at the host hyper-visor level so consequently do not require the deployment of an agent into the VM. For Virtual Machines, metrics data is captured every 30 seconds and up to 90 days of data is made available. The data is reported in time grains that are 5, 60 or 720 minutes (12 hours) long. For 5 minute time grains, the data is averaged over 30 second intervals. For 60 and 720 minute buckets, the data is averaged over 5 minute intervals.

Microsoft Azure Monitoring Services API

The Monitoring Services API is available as a NuGet download. It can be installed using the following command in the Package Manager console:

Install-Package Microsoft.WindowsAzure.Management.Monitoring –Pre

It provides classes providing the ability to:

  • Download metrics
  • Configure alerting based on these metrics
  • Configure autoscaling based on the metrics

The Monitoring Services API provides monitoring support for the following Azure services:

  • Cloud Services (PaaS)
  • HdInsight (Hadoop on Azure)
  • Mobile Service
  • Service Bus
  • Service Bus Brokered Messaging
  • Storage
  • Virtual Machines (IaaS)
  • Web Site

These services are referred to as resource types in the Monitoring Services API, and are identified by resource Id. This post is focused on the Monitoring Services API support for Azure Virtual Machines.

Metric Definitions

The MetricsClient class provides the core connection to the Azure Service Management API. Authentication is provided using a self-signed X.509 Service Management certificate that has previously been uploaded as a Management Certificate to Azure (or alternatively downloaded as a publish settings file.

The MetricsClient class exposes a MetricDefinitions property which is used to access the metric definitions for a specified resource Id. A List() method can be invoked on this property to download a collection of MetricDefinition entities, each of which provides a single metric definition for the specified resource Id.

For Virtual Machines the resource Id is a String of the form:

/hostedservices/SERVICE_NAME/deployments/DEPLOYMENT_NAME/roles/VM_NAME

This information is available on the Azure Portal. The SERVICE_NAME is the name of the cloud service container hosting the VM. The DEPLOYMENT_NAME identifies the specific deployment of the VM, and VM_NAME is the name of the VM. This information is all available on the Azure Portal.

The ResourceIdBuilder class is a helper class exposing methods to create correctly-formatted resource Ids for the various types of resources (Virtual Machines, Cloud Services, Web Sites, etc.). For example, the BuildVirtualMachineResourceId() method creates a resource Id for Virtual Machines.

The MetricDefinition class is declared:

public class MetricDefinition {
public MetricDefinition();

  public String DisplayName { get; set; }
public Boolean IsAlertable { get; set; }
public IList<MetricAvailability> MetricAvailabilities { get; set; }
public TimeSpan MinimumAlertableTimeWindow { get; set; }
public String Name { get; set; }
public String Namespace { get; set; }
public String PrimaryAggregation { get; set; }
public String ResourceIdSuffix { get; set; }
public String Unit { get; set; }
}

DisplayName is a human-readable name while Name identifies the metric in API calls. IsAlertable specifies whether the metric can be used to trigger alerts, which is always true for Virtual Machines metrics. MetricAvailabilities specifies the time grain for metrics (5 minutes, 60 minutes and 720 minutes for Virtual Machines). MinimumAlertableTimeWindows specifies the minimum time windows for alerts, and is always 5 minutes for Virtual Machines metrics. Namespace provides additional parameterization of the resource for some resource types, but is not used for Virtual Machines metrics. PrimaryAggregation specifies whether the metric is specified as a Total or as an Average (with associated minimum and maximum) value. ResourceIdSuffix specifies the suffix identifying the actual resource for a given resource (roles/VM_NAME for a Virtual Machines metric). Unit specifies the units for the metric – for Virtual Machines metrics this is one of Bytes, Bytes / second or Percentage).

The following example shows how to retrieve the metrics definitions for Virtual Machines:

X509Store store = new X509Store(
StoreName.My, StoreLocation.CurrentUser);

store.Open(OpenFlags.ReadOnly);

X509Certificate2 x509Certificate = store.Certificates.Find(
X509FindType.FindByThumbprint, thumbprint, false)[0];

CertificateCloudCredentials credentials =
new CertificateCloudCredentials(SUBSCRIPTION_ID, x509Certificate);

String vmResourceId = ResourceIdBuilder.BuildVirtualMachineResourceId(
CLOUD_SERVICE_NAME, DEPLOYMENT_NAME, VM_NAME);

MetricsClient metricsClient = new MetricsClient(credentials);

MetricDefinitionListResponse metricListResponse =
metricsClient.MetricDefinitions.List(vmResourceId, null, null);

foreach (MetricDefinition metricDefinition in
metricListResponse.MetricDefinitionCollection.Value)
{
String displayName = metricDefinition.DisplayName;
String metricName = metricDefinition.Name;
TimeSpan alertableTimeWindows =
metricDefinition.MinimumAlertableTimeWindow;
String units = metricDefinition.Unit;
String primaryAggregation = metricDefinition.PrimaryAggregation;
}

The following shows the metric definition for Disk Read Bytes/sec:

  • Name: “Disk Read Bytes/sec”
  • Namespace: “”
  • ResourceIdSuffix: “roles/VM_NAME”
  • DisplayName: “Disk Read Bytes/sec”
  • Unit: “Bytes/sec”
  • PrimaryAggregation: “Average”
  • MetricAvailabilities: [
  •   {
  •     TimeGrain: “PT5M”
  •     Retention: “P90D”
  •   }
  •   {
  •     TimeGrain: “PT1H”
  •     Retention: “P90D”
  •   }
  •   {
  •     TimeGrain: “PT12H”
  •     Retention: “P90D”
  •   }
  • ]
  • MinimumAlertableTimeWindow: “PT5M”
  • IsAlertable: true

Metric Values

The MetricsClient class exposes a MetricValues property which is used to access metric values for a specified resource Id. A List() method can be invoked on this property to download a collection of MetricValue entities, each of which provides a single metric value definition for the specified resource Id and metric type.

The List() method has the following parameters:

  • resourceId
  • comma-separated list naming the metrics to be retrieved
  • time grain – must be one of 5 minutes, 60 minutes or 720 minutes (12 hours)
  • start time for the data to be retrieved
  • end time for the data to be retrieved

The List() method returns a MetricValueListResponse entity exposing a collection of MetricValueSet, with an entry for each type of metric value retrieved. MetricValueSet is declared:

public class MetricValueSet {
public MetricValueSet();

  public String DisplayName { get; set; }
public DateTime EndTime { get; set; }
public IList<MetricValue> MetricValues { get; set; }
public String Name { get; set; }
public String Namespace { get; set; }
public String PrimaryAggregation { get; set; }
public DateTime StartTime { get; set; }
public TimeSpan TimeGrain { get; set; }
public String Unit { get; set; }
}

DisplayName is a human-readable name while Name identifies the metric in other API calls. StartTime and EndTime indicate the start and end of the time interval containing the metric values. MetricValues is a list of the MetricValue entities containing the actual metric data. Namespace provides additional parameterization of the resource for some resource types, but is not used for Virtual Machines metrics. PrimaryAggregation specifies whether the metric is specified as a Total or as an Average (with associated minimum and maximum) value. TimeGrain indicates the interval between data points in the MetricValues collection, and for Virtual Machines metrics must be one of 5 minutes, 60 minutes or 720 minutes (12 hours). Unit specifies the units for the metric – for Virtual Machines metrics this is one of Bytes, Bytes / sec and Percentage).

MetricValue is declared:

public class MetricValue {
public MetricValue();

  public String Annotation { get; set; }
public Nullable<Double> Average { get; set; }
public Nullable<Int32> Count { get; set; }
public Nullable<Double> Maximum { get; set; }
public Nullable<Double> Minimum { get; set; }
public DateTime Timestamp { get; set; }
public Nullable<Double> Total { get; set; }
}

Annotation is not used for Virtual Machines metrics. Average, Maximum and Minimum specify the relevant value for a metric of type Average. Count indicates the number of individual data points in the average. These data points are every 30 seconds for metrics retrieved with a time grain of 5 minutes and 60 minutes, and every 30 minutes for metrics retrieved with a time grain of 720 minutes. Timestamp is the start time of the time grain represented by this metric value. Total specifies the value for a metric of type Total.

The following example shows the retrieval of all the Virtual Machines metrics for a VM for the preceding day, with the data reported at hourly intervals (on the hour):

String vmResourceId = ResourceIdBuilder.BuildVirtualMachineResourceId(
CLOUD_SERVICE_NAME, DEPLOYMENT_NAME, VM_NAME);

MetricsClient metricsClient = new MetricsClient(credentials);

List<String> metricNames = new List<String>() { “Disk Read Bytes/sec”,
“Disk Write Bytes/sec”, “Network In”, “Network Out”,
“Percentage CPU” };

// timeGrain must be 5, 60 or 720 minutes.
TimeSpan timeGrain = TimeSpan.FromMinutes(60);
DateTime startTime = DateTime.UtcNow.AddDays(-1);
DateTime endTime = DateTime.UtcNow;

MetricValueListResponse response = metricsClient.MetricValues.List(
vmResourceId, metricNames, String.Empty, timeGrain, startTime,
endTime);

foreach (MetricValueSet value in
response.MetricValueSetCollection.Value)
{
String valueName = value.Name;
foreach (MetricValue metricValue in value.MetricValues) {
Double? average = metricValue.Average;
Int32? count = metricValue.Count;
Double? maximum = metricValue.Maximum;
Double? minimum = metricValue.Minimum;
DateTime timestamp = metricValue.Timestamp;
Double? total = metricValue.Total;
}
}

The following shows a Network In metric value for a five minute interval:

  • Timestamp: “2014-06-21T19:45:00Z”
  • Average: 36713
  • Minimum: 36713
  • Maximum: 36713
  • Total: 36713
  • Count: 1

The following shows a CPU Percentage metric value for a five minute interval:

  • Timestamp: “2014-06-21T19:55:00Z”
  • Average: 4.105362
  • Minimum: 2.862055
  • Maximum: 10.254867
  • Total: 41.05362
  • Count: 10

Azure Service Management REST API

The Azure Service Management REST API is the core API for managing Azure compute resources. The Monitoring Services API invokes the REST API to interact with Azure. The Service Management REST API uses an X.509 certificate for authentication.

The host for the Service Management API is:

The Monitoring Services operations requires the use of a Service Management API version as specified in the following request headers:

  • x-ms-version: 2013-10-01

By default, the data is returned in XML format. However, the following request header indicates it should instead be returned as JSON:

  • Accept: application/json

The Service Management API uses different paths for different operations.

The path for the Get Metric Definitions operation is:

/SUBSCRIPTION_ID/services/monitoring/metricdefinitions/query?resourceId=/hostedservices/CLOUD_SERVICE_NAME/deployments/DEPLOYMENT_NAME/roles/VM_NAME

The path for the Get Metrics operation is:

/SUBSCRIPTION_ID/services/monitoring/metricvalues/query?resourceId=/hostedservices/CLOUD_SERVICE_NAME/deployments/DEPLOYMENT_NAME/roles/VM_NAME &namespace=&names=METRICS_LIST&timeGrain=TIME_GRAIN&startTime=START_TIME&endTime=END_TIME

METRICS_LIST is a comma-separated list of the metrics to be retrieved. START_TIME and END_TIME are the start and end times of the retrieval interval, specified as UTC date times in ISO 8601 format. The times should have both a T and Z indicators. For example: 2014-06-21T19:00:00Z. TIME_GRAIN specifies the time grain as an ISO 8601 time interval:

  • PT5M – 5 minutes
  • PT1H – 60 minutes
  • PT12H – 720 minutes

Note that although the alerts and autoscaling functionality in the Service Management REST API is documented, the monitoring functionality is currently not documented.

Summary

The Azure Monitoring Services API is a preview API that supports access to monitoring data captured by various Azure services. For Azure Virtual Machines it provides access to core performance data for a VM including: CPU Percentage; Network In and Network Out; Bytes Read / sec and Bytes Written / sec. This data can be displayed in a dashboard. The Monitoring Services API also provides the ability to configure alerting or autoscaling based on the monitored data.

About Neil Mackenzie

Cloud Solutions Architect. Microsoft
This entry was posted in Azure, Monitoring Service API, Virtual Machines and tagged , , . Bookmark the permalink.

12 Responses to Using Azure Monitoring Service API with Azure Virtual Machines

  1. Pingback: Dew Drop – June 23, 2014 (#1801) | Morning Dew

  2. This is gold! Awesome article it helped me a lot!!! However I was wondering if you could help me with something.. I was able to get values for the standard metrics, such as Network In, Network Out and those you listed. However I also have some custom metrics (and consequent alert rule) for “Uptime” and they show up in the monitoring interface of the VM. I am able to get a list of all the metrics and my “Uptime” ones show up. However if I try to retrieve the values of these “Min, Max, AVG etc” I cannot seem to do it! It always return “null”.😦 I tried to retrieve them by using either their “name” or their “displayName” and to avoid typos even used the property “metricDefinition.name or metricDefinition.displayName” but nothing! I am thinking that it may also be due to namespace. –> In your code you do the following: MetricValueListResponse response = metricsClient.MetricValues.List(vmResourceId, metricNames, String.Empty, timeGrain, startTime, endTime); so namespace is an emptyString, because for those defaults metrics the namespace is ” “. However for my metrics the namespace is WindowsAzure.Availability and i tried using that as string, but that didn’t work either.

    Do you have any ideas / experience / could you help me? I would appreciate it A LOT!

    Best Regards,
    Manuel

  3. Manuel – Sorry, I can’t help you. I wasn’t aware that it is possible to use custom metrics with Azure Virtual Machines, although it is possible with Azure (PaaS) Cloud Services. How did you configure custom metrics for Virtual Machines so that they show up in the portal?

  4. Dear Neil, what I did I went in a VM under the “Configure” tab, and added 2 endpoints in the monitoring section (see: http://puu.sh/9Sz5L/82289f4ec3.png). I also create an “alert” about these to get emails whenever the uptime is less then 99%. My code is like this: (http://puu.sh/9SzlJ/239a31e9ed.png) and is basically like you did except that I build a VirtualMachineResourceId and I define a namespace “MetricDefinitionListResponse metricListResponse = metricsClient.MetricDefinitions.List(resourceId, null, nspace);” rather than keeping that null as well (to refine my search), because I know these metrics are in the namespace “WindowsAzure.Availability”. (I could also leave that null but then I would get also the other metrics that I don’t want, like Network In, Network Out etc).

    The problem comes after, when I try to get a response with the following:
    “MetricValueListResponse response = metricsClient.MetricValues.List(vmResourceId, metricNames, NameSpace, timeGrain, startTime, endTime);”

    In your code instead of “NameSpace” you have String.Empty because the namespace of those metrics have no namespace (“”), but new metrics that I add get this WindowsAzure.Availability namespace (which i didn’t choose). Now I tried to do two things:

    1) Using the namespace that I KNOW FOR A FACT IS CORRECT (“WindowsAzure.Availability”) because I got it from the “metricDefinition.Namespace” property, I get an exception saying it cannot retrieve the metrics (xmlns=”http://schemas.microsoft.com/2003/10/Serialization/”>{“Code”:”InvalidRequest”,”Message”:”Could not retrieve metrics.”})

    2) Using String.Empty as namespace, I get empty result of course because those values I am looking for are not in ” ” namespace.

    Lastly it could have also been a problem of “the name” of the metric to retrieve but I tried to use both the metricDefinition.Name and metricDefinition.DisplayName, but neither worked (again, because none of them are in the empty namespace and whenever I try to specify a namespace in the line “MetricValueListResponse response = metricsClient.MetricValues.List(vmResourceId, metricNames, NameSpace, timeGrain, startTime, endTime);” I get that exception.

    It’s so frustrating that I cannot get this to work, and that microsoft doesn’t support this as it is a preview, and that their documentation doesn’t say much at all when it comes to examples, and imo this is obviously a mistake on their side since the exact same request fails whenever I try to specify a namespace in the “metricsClient.MetricValues.List” but worked ok when I specified in “metricsClient.MetricDefinitions.List”:/

    • I solved it!! What a “stupid” thing. So although those metrics are created on the VM when I do this “MetricValueListResponse response = metricsClient.MetricValues.List(resourceId, metricNames, NameSpace, timeGrain, startTime, endTime);” I actually have to use the cloud service as “resource” and not the VirtualMachine! YAY.

      However I gotta say that from microsoft I would expect much better documentation. I find them lacking A LOT on this side.

  5. Manuel, it is great that you worked it out. I suspect that it is not so much that the documentation is lacking as that you came across an undocumented feature. I noticed that the Uptime metric is available on the Cloud Service section on the Azure Portal and not on the Virtual Machines section. Remember that the Monitoring Service API is still in preview.

    • Oh yeah Neil, that’s true, but I am thinking that if you make documentation and put it publicly available it shouldn’t be that big of an effort to just put a very SIMPLE example with it. Or like for a method define what values are allowed etc. (like you did in your for the “timeGrain”). But yeah let’s hope it will be improved once it becomes “stable”.
      PS: I have access to those metrics (Uptime, ResponseTime) also in my VM monitoring🙂

  6. Midhun Kandoth says:

    It will be great if you could provide an example query for get metrics rest call. I have tried almost every possible permutations but the result is
    InvalidRequestCould not retrieve metrics.’
    I think an example will be sufficient

  7. Pavan Kale says:

    Hi All,
    This is great article…it help me a lot.
    I am able to get metric definition for role instance using following API call:
    https://management.core.windows.net//services/monitoring/metricdefinitions/query?resourceId=/hostedservices//deployments//roles//roleInstances/
    But now when I tried to get metrics for each of those metric definition using call:
    https://management.core.windows.net//services/monitoring/metricvalues/query?resourceId=/hostedservices//deployments//roles//roleInstances/&namespace=&names=Disk Read Bytes/sec&timeGrain=PT1H&startTime=2015-09-06T00:00:00Z&endTime=2015-09-07T00:00:00Z
    It is giving me error as below:

    HTTP error 400 for API call [https://management.core.windows.net/0f5d0018-8acb-4f26-90fe-28be48ac530e/services/monitoring/metricvalues/query?resourceId=/hostedservices/BMCCLoud/deployments/4366f3e1cb684bd59aaff2d160f2c2ca/roles/Composite.WindowsAzure.WebRole.SimpleBoot/roleInstances/Composite.WindowsAzure.WebRole.SimpleBoot_IN_0&namespace=&names=Disk Read Bytes/sec&timeGrain=PT1H&startTime=2015-09-06T00:00:00Z&endTime=2015-09-07T00:00:00Z]

    [Bad RequestBad RequestHTTP Error 400. The request is badly formed.]

    Is there anything wrong I am doing while making REST API call for metric values?

  8. Midhun Kandoth says:

    Great post. Is there any method to get billing information using this api ?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s