Configuring Windows Azure Diagnostics in Eclipse

The Windows Azure Developer Center provides downloads and documentation supporting the creation of Windows Azure PaaS cloud services in various environments and languages including Node.js, Java, PHP, Python and Ruby. The Java Developer Center contains tutorials demonstrating how to use various Windows Azure features. However, the Java Developer Center does not describe how to configure Windows Azure Diagnostics (WAD) which provides a convenient way to capture various logs on a Windows Azure role instance and persist them to Windows Azure Storage.

WAD has been part of Windows Azure for several years. Its use is essential to the development of maintainable services, since it allows easy access to diagnostics information about a role instance without the need to log into the VM. WAD can be used to capture and persist the following diagnostic information:

  • Event Logs
  • Performance Counters
  • Custom Logs
  • Windows Azure Infrastructure Logs
  • Trace Logs

There are various ways to configure WAD. The conventional method has been to use the .NET API in role startup to get the default WAD configuration, modify it, and then set the new configuration. This API is one of the few Windows Azure APIs with no equivalent REST API, which means it cannot be used from a Java application. The current recommendation is that WAD be configured using a configuration file – named diagnostics.wadcfg – that is contained in the package deployed to Windows Azure. This can be used to configure WAD in a Java application.

The Trace Logs are created using the System.Diagnostics.Trace class in .NET for which there is no Java equivalent. Consequently, there is no way to generate Trace Logs in Java. However, a Windows Azure cloud service developed in Java can configure and benefit from the other logs. Specifically, Custom Logs can be used to automatically persist arbitrary log files to Windows Azure Storage.

Custom Logs are configured by providing the name of a directory to contain the logs and the name of a Windows Azure Blob container into which the files are persisted. The correct way to configure the logging directory is to configure a Local Resource in the service-definition file. A Local Resource represents a directory on the resource disk (c drive) with the directory being provisioned automatically when the role instance is created. The Service Runtime API can be used to retrieve the actual location of the directory at runtime. Log files can be written or copied to the directory. An environment variable, containing the actual location for the Local Resource directory, can also be specified in the service-definition file. Multiple local resource directories can be configured, and WAD supports the ability to configure multiple Custom Logs directories. This allows for separation of different types of log information.

Adding Windows Azure Diagnostics to the Hello World Tutorial

The Java Developer Center has an introductory tutorial on Creating a Hello World Application for Windows Azure in Eclipse. This explains how to create a simple web site with an accompanying Windows Azure deployment project. It then describes how to deploy the application to Windows Azure. However, the tutorial does not explain how to configure WAD. This post builds on that tutorial and describes the steps required to configure WAD.

(Aside for Visual Studio/.NET users: Phil Hoff has a nice post showing how Role Content folders can be used to configure diagnostics.wadcfg in Visual Studio.)

Adding the diagnostics.wadcfg File

  1. Complete the Creating a Hello World Application for Windows Azure in Eclipse tutorial.
  2. Create an empty file named diagnostics.wadcfg in WorkerRole1\approot in the Windows Azure project in Eclipse.
  3. Copy the following into diagnostics.wadcfg:
<?xml version="1.0" encoding="utf-8" ?>
<DiagnosticMonitorConfiguration xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration"
      configurationChangePollInterval="PT1M"
      overallQuotaInMB="4096">

  <DiagnosticInfrastructureLogs bufferQuotaInMB="0"
     scheduledTransferLogLevelFilter="Verbose"
     scheduledTransferPeriod="PT1M" />

  <Directories bufferQuotaInMB="0"
     scheduledTransferPeriod="PT1M">  
     <DataSources>
      <DirectoryConfiguration container="diagnostics-custom-logs" directoryQuotaInMB="1024">
        <LocalResource name="CustomLogsLocalStore" relativePath="."/>
      </DirectoryConfiguration>     
    </DataSources>
  </Directories>

  <PerformanceCounters bufferQuotaInMB="0" scheduledTransferPeriod="PT1M">
    <PerformanceCounterConfiguration counterSpecifier="\Memory\Available Bytes" sampleRate="PT30S" />
    <PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT30S" />
  </PerformanceCounters>

  <WindowsEventLog bufferQuotaInMB="0"
     scheduledTransferLogLevelFilter="Verbose"
     scheduledTransferPeriod="PT1M">
    <DataSource name="Application!*" />
    <DataSource name="System!*" />
  </WindowsEventLog>

</DiagnosticMonitorConfiguration>

This configuration captures and persists all the diagnostics logs except for Trace Logs. The configuration persists the logs to Windows Azure Storage every minute – which is too short for production use, but convenient for trying out the feature. It captures and persists a couple of performance counters, as well as the Application and System event logs. It also captures and persists the Windows Azure Infrastructure Logs.

MSDN has documentation describing how to use diagnostics.wadcfg. The full schema documentation of diagnostics.wadcfg is also on MSDN.

Configure Local Store

In this configuration WAD also persists as a blob in the container named diagnostics-custom-logs any file which has been added to or modified in the directory associated with the Local Resource named CustomLogsLocalStore.

  1. Add the following line to the Imports section of ServiceDefinition.csdef:

    <Import moduleName=”Diagnostics”/>

  2. Add the following after the EndPoints section of ServiceDefinition.csdef:

    <LocalResources>
       <LocalStorage name=”CustomLogsLocalStore”
           sizeInMB=”100″
           cleanOnRoleRecycle=”false”/>
    </LocalResources>

    This defines the Local Resource used to persist “log files” to blob storage. It is configured to preserve the associated directory through role instance recycles.

Configure WAD Connection String

WAD persists the logs to Windows Azure Storage and the connection string for the storage account must be provided. Note that in a production system it is a best practice that production data and diagnostics data be stored in separate storage accounts. This serves to isolate production data and to ensure that the saving of diagnostics data does not impact the storage performance for production data.

  1. Add the following line to the ConfigurationSettings section of ServiceConfiguration.cscfg:

    <Setting name=”Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString”
       value=”UseDevelopmentStorage=true”/>

    This specifies the storage account WAD uses to persist the logs. The value specified above works only for the development environment. When deploying to the cloud an appropriate configuration string must be used, such as:

    DefaultEndpointsProtocol=https;AccountName=<AccountName>;AccountKey=<AccountKey>

    AccountName and AccountKey specify the storage account. Note that WAD requires that DefaultEndpointsProtocol be https.

Modify the Eclipse Build Configuration

The Eclipse build configuration must be modified to ensure that the diagnostics.wadcfg file is copied into the Windows Azure deployment package.

  1. Add the following line after the other component elements in the package.xml file in the Windows Azure project:

    <component cloudupload=”never” deploydir=”%DEPLOYROOT%” deploymethod=”none” importmethod=”copy” importsrc=”diagnostics.wadcfg”/>

    During the build process, this copies the diagnostics.wadcfg file into the Windows Azure deployment package.

Modifying the Configuration at Runtime

Once the application is deployed to either the Compute Emulator or to Windows Azure, the WAD agent will start automatically using the diagnostics.wadcfg for its configuration. It stores the configuration in an application/role/instance-specific blog in the wad-control-container container in the storage account configured for diagnostics. The WAD configuration can be modified for a running instance by modifying the contents of this blob. The Windows Azure Diagnostics .NET API can be used to do this – alternatively, the blob can be overwritten with an updated version.

Summary

Windows Azure Diagnostics is a powerful feature allowing the capture and persistence of logging data from a Windows Azure role instance. The diagnostics.wadcfg file can be used to configure WAD, allowing it to be used with Java applications.

About these ads

About Neil Mackenzie

Cloud Solutions Architect. Microsoft
This entry was posted in Diagnostics, Eclipse, Java, Windows Azure and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s