Custom Diagnostics in Windows Azure

This post follows on from the previous post on Diagnostics in Windows Azure which should probably be read before this one.

Windows Azure Diagnostics supports the ability to integrate custom logging with the provided diagnostics handling. This is achieved by the creation of a custom file-based data buffer configured through the DirectoriesBufferConfiguration and DirectoryConfiguration classes. DirectoriesBufferConfiguration is declared:

public class DirectoriesBufferConfiguration : DiagnosticDataBufferConfiguration {
    // Constructors
    public DirectoriesBufferConfiguration();    // Properties
    public IList<DirectoryConfiguration> DataSources { get; }
}

The DirectoryConfiguration class is declared:

public class DirectoryConfiguration {
    // Constructors
    public DirectoryConfiguration();    // Properties
    public String Container { get; set; }
    public Int32 DirectoryQuotaInMB { get; set; }
    public String Path { get; set; }
}

Configuring a custom log is a simple matter of providing:

• Container – name of the Azure Blob container where the custom log files are persisted
• DirectoryQuotaInMB – maximum size for the directory containing the custom log files
• Path – full path to the directory containing the custom log files

The Container name must satisfy the naming rules for an Azure Blob container but otherwise there are no restrictions on the name. The Azure Diagnostics infrastructure creates this container before persisting data to it.  The Path is the full path to a reserved piece of local storage configured through the LocalStorage element of the Azure Service definition file.

When a DirectoryConfiguration has been created and configured it can be added to the list of DataSources in the DirectoriesBufferConfiguration of the Diagnostics Monitor managing the Azure Diagnostics infrastructure.

Example

A named piece of Local Storage is configured as follows in the Azure Service Definition configuration file:

<LocalResources>
    <LocalStorage name=”MyLogs” cleanOnRoleRecycle=”true” sizeInMB=”10″ />
</LocalResources>

This reserves a 10MB portion of local storage named MyLogs. This storage is released when the role is recycled.This local storage is accessed through a LocalResource object as follows:

LocalResource localResource = RoleEnvironment.GetLocalResource(“MyLogs”);

where LocalResource is declared:

public abstract class LocalResource {
    // Properties
    public abstract Int32 MaximumSizeInMegabytes { get; }
    public abstract String Name { get; }
    public abstract String RootPath { get; }
}

The RootPath and MaximumSizeInMegabytes from the LocalResource are used to set the equivalent properties of a DirectoryConfiguration object as follows:

DirectoryConfiguration directoryConfiguration = new DirectoryConfiguration();
directoryConfiguration.Container = “wad-my-container”;
directoryConfiguration.DirectoryQuotaInMB = localResource.MaximumSizeInMegabytes;
directoryConfiguration.Path = localResource.RootPath;

This directoryConfiguration can then be added to the DiagnosticMonitorConfiguration and a scheduled transfer TimeSpan of 1 minute specified by:

DiagnosticMonitorConfiguration diagnosticMonitorConfiguration = DiagnosticMonitor.GetDefaultInitialConfiguration();
diagnosticMonitorConfiguration.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);
diagnosticMonitorConfiguration.Directories.DataSources.Add(directoryConfiguration);

The Azure Diagnostics Monitoring Agent Host, MonAgentHost,exe, can then be specified to use Development Storage as a persistent store:

CloudStorageAccount cloudStorageAccount = CloudStorageAccount.DevelopmentStorageAccount;
DiagnosticMonitor diagnosticMonitor = DiagnosticMonitor.Start(cloudStorageAccount, diagnosticMonitorConfiguration);

And that is it.Now when a file is inserted in the directory specified in directoryConfiguration.Path it will be persisted automatically as a blob in an Azure Storage container named wad-my-container when the ScheduledTransferPeriod TimeSpan expires. Note that the file is not removed from local storage when it is persisted to Azure Storage.The following is a trivial example of writing a file named logFile1 to local storage:

Byte[] bytesToWrite = new byte[] { 65, 122, 117, 114, 101 };
using (FileStream fileStream = new FileStream(localResource.RootPath + @”logFile1″, FileMode.Create))
{
    fileStream.Write(bytesToWrite, 0, bytesToWrite.Length);
}

Complete Example

The following comprises all the code above:

protected void UseCustomLogs()
{
    DiagnosticMonitorConfiguration diagnosticMonitorConfiguration =
        DiagnosticMonitor.GetDefaultInitialConfiguration();
    diagnosticMonitorConfiguration.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);    LocalResource localResource = RoleEnvironment.GetLocalResource(“MyLogs”);
    DirectoryConfiguration directoryConfiguration = new DirectoryConfiguration();
    directoryConfiguration.Container = “wad-my-container”;
    directoryConfiguration.DirectoryQuotaInMB = localResource.MaximumSizeInMegabytes;
    directoryConfiguration.Path = localResource.RootPath;
    diagnosticMonitorConfiguration.Directories.DataSources.Add(directoryConfiguration);    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.DevelopmentStorageAccount;
    DiagnosticMonitor diagnosticMonitor = DiagnosticMonitor.Start(cloudStorageAccount,
         diagnosticMonitorConfiguration);    Byte[] bytesToWrite = new byte[] { 65, 122, 117, 114, 101 };
    using (FileStream fileStream = new FileStream(localResource.RootPath + @”logFile1″, FileMode.Create))
    {
        fileStream.Write(bytesToWrite, 0, bytesToWrite.Length);
    }
}

UPDATE 4/27/2010

On this Azure Forum thread, f_r_a_n_k indicates out that Azure Diagnostics appears to need an exclusive lock on files when they are persisted.

Technorati Tags: Azure,Azure Diagnostics,Azure Storage

About these ads