• .NET
• Java
• PHP
• Node.js

.NET has always been the primary development environment for Windows Azure. Java and PHP have been around for many years and applications developed in these environments have been deployable to Windows Azure for some time.

Node.js is a lightweight platform, developed by Ryan Dahl (@ryah), for building highly-scalable network applications written in JavaScript. In particular, it can be used to develop and deploy web servers. socket.io is a Node.js package that provides a simple way to access HTML5 web sockets thereby facilitating the creation of applications supporting browser-to-browser conversation. The Node.js documentation is here.

This post is a brief introduction to Node.js and socket.io with that introduction being focused on the implementation of Node.js inside the Windows Azure environment.

Node.js

Node.js is an application hosting environment developed in C++ and using the Google V8  JavaScript engine to host applications written in JavaScript. An essential feature of applications written in Node.js is the heavy use of callbacks making support for an asynchronous programming model more natural. Node.js can be downloaded directly from the Node.js website.

Hello World in Node.js

The following example demonstrates how easy it is to write a website serving a single Hello World web page.

var port = 81;
var http = require('http');

app.listen(port);
console.log('Server running on port ' + port);

When the above code is saved in a file named server.js the following command can be used to launch the web server:

node server.js

The require() statement imports the Node.js http module into the application where it is used to create an HTTP server. The parameter to the createServer() method is an anonymous callback function invoked each time the server receives a request. This callback is passed the request and response streams for the request. In the example, a response header is added before res.end() is invoked to write Hello World to the response and flush the response back to the client. app.listen() starts the server listening on port 81.

This simple program demonstrates several features common to Node.js applications:

• the use of require() to import modules into the application
• the creation of a server
• the use of listen() to start the listening process

Application Frameworks

A web application can be coded directly in JavaScript and deployed as a website hosted in Node.js. As in other web-development environments it is common to use an application framework to structure the application and enforce a separation of concerns that aids the development of large-scale applications.

Many Node.js samples use the Express framework. This uses routes and views directories to store application routes and views. Express supports various view engines including Jade and EJS (embedded JavaScript). These provide different ways to specify the appearance of a web page.

The following commands can be invoked to download the Express module and create a default Node.js web server using the Express framework and the Jade view engine:

npm install express
.\node_modules\.bin\express
npm install

node app.js

This starts a web server, listening on port 3000, which responds with the web page defined in views\index.jade.

Node Package Manager (NPM)

The Node Package Manager (NPM) is an application that simplifies the local installation for the thousands of packages that have been created for Node.js. NPM stores downloaded packages in a node_modules folder under the invocation directory. It is possible to specify that downloaded packages be stored globally but, in general, they should be regarded as part of the application they are being used in and stored locally (which is the default).

The package.json file associated with a downloaded package specifies dependencies on other packages that it may have. NPM recursively downloads these dependent packages into a node_modules directory associated with the initial package. The NPM is invoked as follows:

npm install packageName

Windows Azure

Microsoft supports Node.js as a first-class development environment for Windows Azure. The Windows Azure developers portal has a section devoted to Node.js that contains various samples and a download link for the Windows Azure SDK for Node.js.

The SDK is installed in the following directory:

%ProgramFiles(x86)%\Microsoft SDKs\Windows Azure\Nodejs

The installation process creates a link on the Start menu for the Windows Azure PowerShell for Node.js which launches a PowerShell console with the Windows Azure Node.js cmdlets preloaded. These cmdlets can be used to manage the creation and deployment of hosted services both in the development and Windows Azure environment. As is typical with Windows Azure development environments, the PowerShell console should be launched using Run as Administrator.

The cmdlets include the following:

• New-AzureService serviceName
• Add-AzureNodeWorkerRole roleName
• Add-AzureNodeWebRole roleName
• Start-AzureEmulator
• Stop-AzureEmulator

These can be used sequentially to create a Windows Azure hosted service with zero or more web and worker roles and then deploy them to the compute emulator before tearing down the service. The developer portal has a sample that does precisely this. Other cmdlets support the deployment of the hosted service to Windows Azure.

Web Roles and Worker Roles

The Windows Azure SDK for Node.js implements web and worker roles in different ways leading to differences in the supported functionality. Specifically, since http.sys does not support HTML5 web sockets it is not possible to use them in a Node.js (or any) application deployed to a web role. Web sockets are supported for Node.js (or any) application deployed to a worker role. One benefit of using a web role is that, as shown by Aaron Stannard (@Aaronontheweb) in this post, IIS can handle the launching of multiple node.exe instances automatically. Otherwise, Node.js would only be able to use a single core, even in a multi-core instance.

A web role is implemented using a special IISNode module which is loaded as additional ASP.NET module in IIS. When creating a Windows Azure package for deployment, the packager creates a startup task to perform various tasks including installing IISNode in the web role instance. In the development machine, IISNode is installed in:

%ProgramFiles(x86)%\Microsoft SDKs\iisnode

A worker role is implemented using the new ProgramEntryPoint functionality exposed in ServiceDefinition.csdef which allows an arbitrary program to be specified as the role entry point. Specifically, on startup a worker role hosting a Node.js application invokes the following program entry point:

node.exe .\server.js

An obvious implication is that the Node.js application must be in a file named server.js – although it can import additional modules.

It is remarkably easy to port a Node.js application to Windows Azure. A particularly impressive sample on the Windows Azure developer portal takes a standard example from the socket.io website and ports it to Windows Azure by changing only two lines of code. One change is simply to the location of a module while the other is the specification of the port to listen on in a Windows Azure friendly manner. Specifically, when using Windows Azure SDK for Node.js it is necessary to specify the listening port using:

process.env.port

which allows the application to access the correct Windows Azure endpoint.

Furthermore, it can sometimes be helpful to test Node.js applications directly, using node.exe, without launching the Windows Azure development environment. This is feasible because of the simple manner with which Node.js applications are inserted into the development environment.

Node.js Packages for Windows Azure

The Windows Azure SDK for Node.js does not provide access to the various Windows Azure SDKs. In keeping with the style of Node.js applications, this SDK is instead deployed as a set of Node.js packages for Windows Azure which are downloaded into a Node.js application using

npm install azure

The packages expose Node.js APIs for:

• Windows Azure Storage Service (blobs, tables and queues)
• Service Bus Brokered Messaging (queues, topics and subscriptions)

The Windows Azure developer portal has a complete example showing how to access Windows Azure Table service from a Node.js application developed using Express/Jade. When the azure packages are installed, examples of all the supported functionality are installed in the node_modules\azure\examples directory for the application.

Some Node.js on Windows Azure Links

Glenn Block (@gblock) has an interesting Channel 9 interview in which he describes Node.js on Windows Azure. Matt Harrington (@mh415) has a post describing the environment variables available when using the Node.js azure packages as well as a post showing how to log messages. Aaron Stannard has a post on using the Node.js packages for Windows Azure outside the compute emulator.

socket.io

Guillermo Rauch (@rauchg) developed socket.io, a Node.js package that simplifies the creation of web applications supporting real-time communication between browsers and devices. The github repository and documentation for socket.io is here. The canonical example of a socket.io application is a chat application supporting a conversation between clients.

socket.io supports various transports including:

• web socket
• html file
• xhr-polling
• jsonp – polling

By default, the transports are used in that priority and socket.io automatically degrades a connection when the transport is not supported. Web sockets are not supported by IIS so, as mentioned earlier, a worker role must be used if web socket support is desired for a socket.io application hosted in Windows Azure. Similarly, IE9 does not support web sockets so IE9 use again leads to a degraded connection. (Hint: the Chrome browser provides a better development experience since it does not suffer any connection degradation delay.)

socket.io implements the Node.js EventEmitter interface and so provides a nice demonstration of idiomatic Node.js. The EventEmitter interface comprises:

• emitter.addListener(event, listener)
• emitter.on(event, listener)
• emitter.once(event, listener)
• emitter.removeListener(event, listener)
• emitter.removeAllListeners([event])
• emitter.setMaxListeners(n)
• emitter.listeners(event)
• emitter.emit(event, [arg1], [arg2], [...])
• Event: ‘newListener’

This interface supports the association of named events (or messages) with callbacks. Specifically, emitter.emit() sends a named event while emitter.on() listens for a named event and associates a callback with it. A socket.io application is created by the mutual exchange of named events between the client and the server.

A core sockets object manages connections. It handles a connection event by creating a socket object and passing that to the callback associated with the connection event. The socket object handles individual connections and listens for the disconnect event.  It also supports named events that contain the conversation messages providing the application functionality.

Simple Windows Azure Application using socket.io

A very basic socket.io application hosted in the Windows Azure development environment can be created as follows:

• Start the Windows Azure PowerShell for Node.js console (using Run as Administrator)
• cd to some directory
• Type New-AzureService chat
• Type Add-AzureNodeWorkerRole
• cd WorkerRole1
• Type npm install socket.io
• Open server.js in some editor (e.g., Sublime Text 2) and replace the contents with the following:

// Import various modules and start listening for socket.io connections
var app = require('http').createServer(handler)
  , io = require('socket.io').listen(app)
  , fs = require('fs');

  socket.on('message', function (data) {
    socket.emit('message', { message: 'Message forwarded' });
    socket.broadcast.emit('message', { message: data.message });
  });
});

• Create a file named index.html (in the same directory) and copy the following into it:

<html>
<head>
<script src="/socket.io/socket.io.js"></script>
<script src="http://ajax.aspnetcdn.com/ajax/jQuery/jquery-1.7.1.min.js"></script>
<script>
var socket = io.connect('http://localhost:81');

  socket.on('message', function (data) {
    socket.emit('message', { message: 'Message forwarded' });
    socket.broadcast.emit('message', { message: data.message });
  });
});

• Type Start-AzureEmulator –launch

Start multiple browser windows pointing to http://localhost:81.

The UI is simple – with a message box, a Send message button and a status line below. After a few seconds, the status line should display Connected. When a message is typed in one of the message boxes and submitted, the status line in the current windows displays “Message forwarded” while the status line in the other windows displays the message text. Using the appropriate developer tools for the browser it is possible to find out the transport used for communication between the browser and the server.

• Type Stop-AzureEmulator to shutdown the compute emulator.

This demonstration is very simple with the Node.js server serving up a simple web page to the browsers. The server listens for socket.io connections and on receiving one sends a connected message back to the client and sets up a socket listener for an event named message. On receiving such an event the socket responds by sending an event named fromServer with content Message forwarded back to the client and broadcasts an event named message, containing the message text, to all other clients. On loading, the browser connects to the server and, on connection, adds listeners for two events named fromServer and message. On receiving an event, the response is displayed in the status line. When the button is clicked, socket.emit() is invoked to send the message to the server.

This is a trivial demonstration that shows how easy it is to implement a Node.js application using socket.io inside Windows Azure. Mariano Vazquez uses an equally simple demonstration in a blog post in which he provides additional information about configuring the transport used by the connection.

NoSQL

During the last 30 years SQL databases have become the dominant data storage systems, with commercial offerings such as Oracle, Microsoft SQL Server and IBM DB2 achieving enormous commercial success. These data systems are characterized by their support for ACID semantics – atomicity, consistency, isolation and durability. These properties impose a certainty that is essential for many business processes dealing with valuable data. However, supporting ACID semantics becomes increasingly expensive as the volume of data increases.

The advent of Web 2.0 has led to an increasing interest in long-tail data whose value comes not from the value of a single piece of data but from the magnitude of the aggregate data.  Web logs provide a paradigmatic example of this type of data. Because of the low value of an individual data item this type of data can be managed by data systems which do not need to support full ACID semantics.

Over the last few years a new type of data storage semantics has become fashionable: BASE -basically available, soft state, eventually consistent. While the name may be a bit forced to make the pun, the general idea is that a system adhering to BASE semantics, when implemented as a distributed system, should be able to survive network partitioning of the individual components of the system at the cost of offering a lower level of consistency than available in traditional SQL systems.

In a strongly-consistent system, all reads following a write receive the written data. In an eventually-consistent system, reads immediately following a write are not guaranteed to return the newly written value. However, eventually all writes should return the written value.

NoSQL (not only SQL) is a name used to classify data systems that do not use SQL and which typically implement BASE semantics instead of ACID semantics. NoSQL systems have become very popular and many such systems have been created – most of them distributed in an open source model. Another important feature of NoSQL systems is that, unlike SQL systems, they do not impose a schema on stored entities.

NoSQL systems can be classified by how data is stored as:

• key-value
• document

In a key-value store, an entity comprises a primary key and a set of properties with no associated schema so that each entity in a “table” can have a different set of properties. Apache Cassandra is a popular key-value store. A document store provides for the storage of semantically richer entities with an internal structure. MongoDB is a popular document store.

Windows Azure Tables is a key-value NoSQL store provided in the Windows Azure Platform. Unlike other NoSQL stores, it supports strong consistency with no eventually-consistent option. The Windows Azure Storage team recently published a paper describing the implementation of Windows Azure Tables.

10gen, the company which maintains and supports MongoDB, has worked with Microsoft to make MongoDB available on Windows Azure. This provides Windows Azure with a document store NoSQL system to complement the key-value store provided by Windows Azure Tables.

MongoDB

MongoDB is a NoSQL document store in which individual entities are persisted as documents inside a collection hosted by a database. A single MongoDB installation can comprise many databases. MongoDB is schemaless so each document in a collection can have a different schema. Consistency is tunable from eventual consistency to strong consistency.

MongoDB uses memory-mapped files and performance is optimal when all the data and indexes fit in memory. It supports automated sharding allowing databases to scale past the limits of a single server. Data is stored in BSON format which can be thought of as a binary-encoded version of JSON.

High availability is supported in MongoDB through the concept of a replica set comprising one primary member and one or more secondary members, each of which contains a copy of all the data. Writes are performed against the primary member and are then copied asynchronously to the secondary members. A safe write can be invoked that returns only when the write is copied to a specified number of secondary members – thereby allowing the consistency level to be tuned as needed. Reads can be performed against secondary members to improve performance by reducing the load on the primary member.

MongoDB is open-source software maintained and supported by 10gen, the company which originally developed it. 10gen provides downloads for various platforms and versions. 10gen also provides drivers (or APIs) for many languages including C# and F#. MongoDB comes with a JavaScript shell which is useful for performing ad-hoc queries and testing a MongoDB deployment.

Kyle Banker, of 10gen, has written an excellent book called MongoDB in Action. I highly recommend it to anyone interested in MongoDB.

MongoDB on Windows Azure

David Makogon (@dmakogon) worked out how to deploy MongoDB onto worker role instances on Windows Azure. 10gen then worked with him and the Microsoft Interoperability team to develop an officially supported preview release of the MongoDB on Windows Azure wrapper which simplifies the task of deploying a MongoDB replica set onto worker role instances.

The wrapper deploys each member of the replica set to a separate instance of a worker role. The mongod.exe process for MongoDB is started in the OnStart() role entry point for the instance. The data for each member is persisted as a page blob in Windows Azure Blob storage that is mounted as an Azure Drive on the instance.

The MongoDB on Windows Azure wrapper can be downloaded from github. The download comprises two directory trees: ReplicaSets containing the core software as the MongoDBReplicaSet solution; and SampleApplications containing an MVC Movies sample application named MongoDBReplicaSetMvcMovieSample. The directory trees contain a PowerShell script, solutionsetup.ps1, that must be invoked to download the latest MongoDB binaries.

The MongoDBReplicaSetMvcMovieSample solution contains four projects:

• MongoDBAzureHelper – helper class to retrieve MongoDB configuration
• MongoDBReplicaSetSample – Windows Azure project
• MvcMovie – an ASP.NET MVC 3 application
• ReplicaSetRole – worker role to host the members of a MongoDB replica set.

The MVCMovie project is based on the Intro to ASP.NET MVC 3 sample on asp.net website. It displays movie information retrieved from the MongoDB replica set hosted in the ReplicaSetRole instances. The ReplicaSetRole is launched as 3 medium instances each if which hosts a replica set member. The MongoDBAzureHelper and ReplcaSetRole projects are from the MongoDBReplicaSet solution.

The MongoDBReplicaSetMvcMovieSample solution can be opened in Visual Studio, then built and deployed either to the local compute emulator or a Windows Azure hosted service. The application has two pages: an About page displaying the status of the replica set; and a Movies page allowing movie information to be captured and displayed. It may take a minute or two for the replica set to come fully online and the status to be made available on the About page. When a movie is added to the database via the Movies page, it may occasionally require a refresh for the updated information to become visible on the page. This is because MongoDB is an eventually consistent database and the Movies page may have received data from one of the secondary nodes.

This example provides a general demonstration of how a MongoDB installation with replica sets is added to a Windows Azure project: add the MongoDBAzureHelper and ReplicaSetRole projects from the MongoDBReplicaSet solution and add the appropriate configuration to the ServiceDefinition.csdef and ServiceConfiguration.cscfg files.

The ServiceDefinition.csdef entries for ReplicaSetRole are:

<Endpoints>
    <InternalEndpoint name=”MongodPort” protocol=”tcp” port=”27017″ />
</Endpoints>
<ConfigurationSettings>
    <Setting name=”MongoDBDataDir” />
    <Setting name=”ReplicaSetName” />
    <Setting name=”MongoDBDataDirSize” />
    <Setting name=”MongoDBLogVerbosity” />
</ConfigurationSettings>
<LocalResources>
    <LocalStorage name=”MongoDBLocalDataDir” cleanOnRoleRecycle=”false”
       sizeInMB=”1024″ />
    <LocalStorage name=”MongodLogDir” cleanOnRoleRecycle=”false”
       sizeInMB=”512″ />
</LocalResources>

Port 27017 is the standard port for a MongoDB installation. None of the settings need be changed for the sample project.

The ServiceDefinition.csdef entries for MvcMovie are:

<ConfigurationSettings>
    <Setting name=”ReplicaSetName” />
</ConfigurationSettings>

The ServiceConfiguration.cscfg settings for the ReplicaSetRole are:

<ConfigurationSettings>
      <Setting name=”MongoDBDataDir” value=”UseDevelopmentStorage=true” />
      <Setting name=”ReplicaSetName” value=”rs” />
      <Setting name=”MongoDBDataDirSize” value=”" />
      <Setting name=”MongoDBLogVerbosity” value=”-v” />
      <Setting name=”Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString” value=”UseDevelopmentStorage=true” />
</ConfigurationSettings>

The ServiceConfiguration.cscfg settings for the MvcMovie web role are:

<ConfigurationSettings>
      <Setting name=”ReplicaSetName” value=”rs” />
      <Setting name=”Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString” value=”UseDevelopmentStorage=true” />
</ConfigurationSettings>

It is critical that the value for the ReplicaSetName be the same in both the MvcMovie web role and the ReplicaSets worker role.

The replica set can also be accessed from the MongoDB shell, mongo, once it is running in the compute emulator. This is a useful way to ensure that everything is working since it provides a convenient way of accessing the data and managing the replica set. The application data is stored in the movies collection in the movies database. For example, the rs.stepDown() command can be invoked on the primary member to demote it to a secondary. MongoDB will automatically select one of the secondary members and promote it to primary. Note that in the compute emulator, the 3 replica set members are hosted at port numbers 27017, 27018 and 27019 respectively.

This sample demonstrates the process of adding support for MongoDB to a Windows Azure solution with a web role.

1) Add the following MongoDB projects to the solution;

• MongoDBAzureHelper
• ReplicaSetRole

2) Add the MongoDB assemblies to the web role:

• Mongo.DB.Bson (copy local)
• Mongo.DB.Driver (copy local)
• MongoDBAzureHelper  (from the added project)

3) Add the MongoDB settings described earlier to the Windows Azure service configuration.

MongoDB on Windows Azure with Node.js

Node.js is a popular system for developing web servers using JavaScript and an asynchronous programming model. Microsoft recently released the Windows Azure SDK for Node.js which makes it easy to deploy Node.js web applications to Windows Azure.

The SDK provides an extensive set of PowerShell scripts – such as Add-AzureNodeWebRole, Start-AzureEmulator, and Publish-AzureService – which simplify the lifecycle management of developing and deploying a Node.js web application. It contains several introductory tutorials including a Node.js Web Application with Storage on MongoDB. This tutorial shows how to add a replica set implemented, as described earlier, to a web application developed in Node.js.

Summary

The MongoDB on Windows Azure wrapper and the Windows Azure SDK for Node.js tutorial have made it very easy to try MongoDB out in a Windows Azure hosted service.

The deck can be viewed (or downloaded) on SlideShare.

When Packt Publishing first approached me about writing a book, I headed to Borders to look at a few books it published. I was impressed by the quality of the books, and signed up to write the Microsoft Windows Azure Development Cookbook. Packt has published a number of cloud-related books, and currently is offering an early release version of a book by David Burela (@DavidBurela) titled Microsoft Silverlight 4 and Windows Azure Enterprise Integration. Anyway, Borders no longer exists – but my book does.

Packt is offering a giveaway, through the end of October, in which it will give an eBook version of the Microsoft Windows Azure Development Cookbook to two people who visit the webpage for the book and leave a comment on this post with the one feature that makes them want to read the book. To participate in the giveaway, leave a comment and an email address. When November comes, Packt will give the eBook to two people chosen at random from the commenters.

Note that the chapter on Managing Hosted Services with the Service Management API can be downloaded for free.

In this post, I am going to show how to create and configure a service identity other than owner, and specifically how to ensure that this service identity is less privileged than owner. The intent is to show that, notwithstanding the lack of examples, it is pretty easy to use an alternate service identity. As of mid-September, the Service Bus support ACS v2 – and that is what the material in this post was tested with.

Service Bus Namespace

The starting point for all work with the Service Bus is the service namespace which is created and configured on the Windows Azure Portal. In creating a namespace, a unique name must be provided as well as the location of the Windows Azure datacenter which will host the service endpoints for the namespace. The namespace name becomes the first part of the URI used for these service endpoints. For example, if the namespace name is gress then the service gateway for the gress namespace becomes:

https://gress.servicebus.windows.net/

The various services supported by the Service Bus – including relayed and brokered messaging – are hosted at service paths under the service gateway:

https://{namespace}.servicebus.windows.net/{path}

For example, the following are valid service paths:

https://gress.servicebus.windows.net/topics/interestingtopic

https://gress.servicebus.windows.net/boringtopic/subscriptions/somesubscription

https://gress.servicebus.windows.net/EchoService/

Note that a service path must be specifically associated with a service for the path to have any meaning with regard to a service. For example, although the first service path above probably specifies a topic named topic/interestingtopic there is nothing to prevent it being used instead for a relayed service. The second probably specifies a subscription named somesubscription on a topic named boringtopic. The third likely specifies a relayed service named EchoService.

The paths for a namespace nominally form an infinite tree with the / indicating a branch point. For example, the following shows a namespace with two branches, one for services and the other for topics:

• https://gress.servicebus.windows.net/services/EchoService
• https://gress.servicebus.windows.net/services/OneWayService
• https://gress.servicebus.windows.net/topics/interesting
• https://gress.servicebus.windows.net/topics/boring

Service Bus Claims

The Service Bus uses the Windows Azure Access Control Service (ACS) for authentication and authorization. The ACS is a Security Token Service (STS) existing as part of claims-based infrastructure. Although it supports limited identity provider (IdP) capability, the primary function of the ACS is in transforming claims in support of federated identity.

This post assumes and requires only minimal understanding of claims-based authentication. The Microsoft Patterns and Practices Team has recently published the second edition of its excellent Guide to Claims-Based Identity and Access Control. This is a good resource for a much deeper understanding of the topic.

When a Service Bus namespace is created, it is associated with a default service identity named owner and a base64-encoded key. This service identity has absolute control of the namespace, including all service paths in it, so it is essential that the associated key be kept secure. This is made even more important by the fact that it is not possible to change the key on the Windows Azure Portal. The default service identity and key can be used to authenticate Service Bus API requests against services hosted by the namespace.

The Service Bus supports three claims which authorize specific operations on a service path. These claims are all of type net.windows.servicebus.action with values:

• Listen
• Manage
• Send

The different values authorize the obvious sets of operations on service paths. For example, the Send claim is required to perform the following operations:

• Send messages to a listener at a service namespace (relay)
• Get the queue description
• Send into to the queue
• Get the topic description
• Send to the topic

The complete list of operations and associated authorization requirements is provided in the MSDN documentation, which provides an official take on the material in this post.

When the default service identity, owner, authenticates against the ACS a rule is invoked that assigns all three claims to owner thereby authorizing it to perform any operation on any service path in the namespace.  Note that this discussion elides a lot of claims-based identity plumbing that is, in practice, hidden from the user of the Service Bus API. Although possible, it is strongly recommended that none of the claims be removed from the authorization provided to owner. Doing so can cause significant problems with the namespace.

It is possible to create another service identity with limited privileges, e.g. without the Manage claim. Indeed, it is possible to create a service identity and associate it with only one of the claims. This allows, for example, a sending service identity to be created with only the Send claim and a listening service identity to be created with only the Listen claim. Doing so allows a sending application to be distributed without the possibility of a service identity with a Manage claim being compromised.

By default, owner has Manage, Send and Listen claims over the infinite path tree of the namespace. It is possible to restrict additional service identities to only part of the service path tree. For example, if a service identity is used only to send messages to a topic there is no need for that service identity to have any claims other than Send on the specific service path associated with the topic. The ACS supports this type of restriction, and evaluates whether a service identity has the required permission for an operation by doing a longest string match on the service path. Essentially, it works its way back along the service path seeking a portion of the path for which the service identity has the needed permission. The service identity is authorized to perform a specific operation only if somewhere on the service path it is configured with the required claim and that permission has not been removed later in the path. This permission inheritance is similar to that on a file system. Note that it can cause problems if owner has permissions removed for a service path.

Best Practices

Suggesting that something is a best practice is probably a fair bit above my pay grade. But here goes with some recommended practices:

• Do not use owner for an operation that does not require the full privileges and claims of owner.
• Use a service identity with the minimum set of claims needed for a service path.
• Associate a service identity with the longest service path possible.

Consider the following two full service paths representing a topic and an associated subscription:

• https://gress.servicebus.windows.net/topics/interestingtopic
• https://gress.servicebus.windows.net/topics/interestingtopic/subscriptions/mysubscription

If the topic and subscription are created out of band, distinct service identities can be created to authorize sending messages to the topic and receiving messages from the subscription. The sending service identity can be given the Send claim for the topics/interestingtopic service path. The listening service identity can be given the Listen claim for the topics/interestingtopic/subscriptions/mysubscription service path. Separating the privileges like this minimizes the possible damage from a compromised service identity.

SBAzTool

Using an alternative service identity is a little bit more complicated than using owner. However, the Window Azure Service Bus team has provided a command-line utility, SBAzTool, that makes it very easy to add service identities and associate them with service paths.

The SBAzTool is one of the samples provided with the Windows Azure AppFabric SDK v1.5 (September 2011). It is in the ServiceBus/ExploringFeatures/Authorization/ManagingAccessControl directory of the samples. The source code is also supplied in case you want to roll your own service management utility.

The following commands create two new service identities, topicSender and subscriptionListener, and grants the topicSender the Send claim on the topic (from the previous section) and the subscriptionListener the Listen claim on the associated subscription.

sbaztool -n gress -k xkx…6o= storeoptions

sbaztool makeid topicSender
sbaztool grant Send /topics/interestingtopic topicSender

sbaztool makeid subscriptionListener
sbaztool grant Listen /topics/interestingtopic/subscriptions/mysubscription subscriptionListener

sbaztool clearoptions

The storeoptions option is used to store the namespace and the owner key in isolated storage for the SVAzTool – and the clearoptions option removes the information from isolated storage. The SBAzTool can also be used to delete service identities and revoke permissions, as well as display information about service identities and service paths.

SBAzTool makes it really easy to configure service identities and service paths so that service identities other than owner can be used. Consequently, there is no real excuse for not doing so.

Having done all this, we can replace the following code:

TokenProvider tokenProvider =
   TokenProvider.CreateSharedSecretTokenProvider(“owner”, ownerKey);

with

TokenProvider tokenProvider =
   TokenProvider.CreateSharedSecretTokenProvider(“topicSender”, topicSenderKey);

Windows Azure Portal – Access Control Service

The configuration performed by SBAzTool can also be performed manually on the ACS section of the Windows Azure Portal. The UI for ACS configuration on the portal may appear overkill in terms of what has been discussed so far. However, ACS has a much broader use than authorization on the Service Bus.

Clemens Vasters recently made a video of a presentation in which he steps carefully through the process of configuring an issuer and service path. Consequently, I will not repeat the steps here, and will merely put the configuration performed by SBAzTool into the context of the portal.

The makeid option with SBAzTool creates a Service Identity (visible on the portal) that can be authenticated with either a symmetric key or a password. Clemens suggests that the values of these should be the same, and that is what SBAzTool does.

The grant option with SBAzTool creates a Relying Party Application (visible on the portal) for the full service path, e.g.

http://gress.servicebus.windows.net/topics/interestingtopic

Note that the full service path is normalized to specify http rather than any other protocol. Additionally, the tool specifies a token format of SWT and sets the token lifetime to 1200s.

The tool also creates a Service Group (visible on the portal) and associates it with the relying party application configuration for the service path. It also associates the service group with each intermediate service path en route to the service path. This service group generates the appropriate Manage, Send and Listen claims configured by the grant (and revoke) option of SBAzTool. The tool also adds a default service group which generates the Manage, Send and Listen claims for owner on the service path and each intermediate service path. This is to prevent owner not having full rights on the namespace.

Note that currently there is a bug in the portal which causes a problem when following the instructions in the video. Specifically, when creating a relying party application on the portal it is not possible to save the configuration without generating a token signing key. However, doing so causes an error when accessing the service path from an application. The workaround is to go the Certificates and keys section of the ACS configuration on the portal and delete the created token-signing certificate. Everything works fine once that is done.

Summary

My intent in this post was not to give a deep explanation of how to use a claims-based identity framework to secure the Service Bus. Rather, my intent was to provide the minimum amount of information allowing the reader to replace the use of owner with that of a less-privileged service identity. Specifically, I wanted to show how easy it is to do so using the SBAzTool.

During the talk, I pointed people in the direction of a number of posts on Windows Azure AppFabric Applications by Alan Smith (@alansmith):

• Introduction to AppFabric Applications.
• Create Custom External Service in Azure AppFabric June CTP.

The Windows Azure AppFabric Applications CTP can be accessed on the Windows Azure AppFabrics Labs portal. The MSDN documentation is here.

The Windows Azure Customer Advisory Team, which supports the framework, describes it as follows:

The Transient Fault Handling Framework solution provides a reusable framework for building extensible retry policies capable of handling different types of transient conditions in applications leveraging SQL Azure, Windows Azure storage (queues, blobs, tables), Windows Azure AppFabric Service Bus and Windows Azure AppFabric Caching Service.

Although the post is specifically concerned with SQL Azure, the general idea can be implemented when using the Windows Azure Storage service, the Windows Azure AppFabric Service Bus and the Windows Azure AppFabric Caching Service.

I highly recommend the Windows Azure Customer Advisory Team blog. It has many posts showing real-world best practices for using the various features of the Windows Azure Platform.

Handling connection failures to SQL Azure

SQL Azure database is a distributed system in which each physical server hosts many databases. This sharing of resources leads to capacity constraints on operational throughput. SQL Azure handles these capacity constraints by throttling operations and closing connections that are using too many resources. SQL Azure also closes connections when it alleviates operational hot spots by switching from a primary SQL Azure database to one of its two backup copies. Furthermore, connectivity to a SQL Azure database is likely to be less reliable than connectivity to a Microsoft SQL Server database on a corporate LAN. It is imperative therefore that applications using SQL Azure be designed to withstand the connection failures that are far more likely to occur than with Microsoft SQL Server.

One of the mantras of cloud development is design for failure. It is important that applications using SQL Azure be designed to handle failures appropriately. There are two kinds of error: permanent errors indicating a general failure of part of the system and transient errors existing only for a brief time. Permanent errors perhaps indicate a logical problem with the application—and handling them may require code changes. However, an application should handle transient errors gracefully by retrying the operation that led to the error in the hope that it does not recur. A dropped connection should be regarded as transient, and an application should respond to a dropped connection by opening a new connection and retrying the operation.

There remains the problem of distinguishing permanent from transient errors. This can be done by comparing the error returned from a failed operation with a known list of transient errors. An application can therefore include a retry mechanism that checks the status of operations and retries any operations that experienced a transient error.

The Windows Azure AppFabric Customer Advisory Team has made available on the MSDN Code Gallery the source code and pre-compiled assemblies for the Transient Fault Handling Framework for Azure Storage, Service Bus, and SQL Azure. This comprises a set of classes that can be used to detect transient failures and retry SQL operations. It contains an extensible way to identify transient failures, with various examples including one that compares an error with a list of known transient failures. The Transient Fault Handling Framework provides various built-in retry backoff techniques that specify how often and frequently an operation should be retried following a transient failure. These include both a fixed interval and an exponential delay between retries. The classes in the Transient Fault Handling Framework include various extension methods that simplify the use of the framework, thereby minimizing the work required to add the handling of dropped connections and other transient failures to an application using SQL Azure.

In this recipe, we will learn how to use the Transient Fault Handling Framework for Azure Storage, Service Bus, and SQL Azure to handle dropped connections and other transient failures when using SQL Azure.

Getting ready

The recipe uses the Transient Fault Handling Framework for Azure Storage, Service Bus, and SQL Azure. It can be downloaded from the following URL:

http://code.msdn.microsoft.com/Transient-Fault-Handling-b209151f 

This download is a Visual Studio solution with precompiled output assemblies that are referenced in the project used in the recipe.

How to do it…

We are going to connect to SQL Azure using ADO.NET and perform various DDL and DML operations taking advantage of the transient-error handling provided by the Transient Fault Handling library. We do this as follows:

1. On the Project Properties dialog in Visual Studio, set the Target Framework to .NET Framework 4.

2. Add the following assembly references to the project:

Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling.dll
System.configuration.dll

3. Add a new class named RetryConnectionExample to the project.

4. Add the following using statements to the top of the class file:

using System.Data;
using System.Data.SqlClient;
using Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling;
using Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling.SqlAzure;
using Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling.Configuration;

5. Add the following private members to the class:

String connectionString;
RetryPolicy connectionRetryPolicy;
RetryPolicy commandRetryPolicy;

6. Add the following constructor to the class:

public RetryConnectionExample(String server, String database,
   String login, String password)
{
   SqlConnectionStringBuilder connStringBuilder;
   connStringBuilder = new SqlConnectionStringBuilder();
   connStringBuilder.DataSource = String.Format(“{0}.database.windows.net”, server);
   connStringBuilder.InitialCatalog = database;
   connStringBuilder.Encrypt = true;
   connStringBuilder.TrustServerCertificate = false;
   connStringBuilder.UserID = String.Format(“{0}@{1}”, login, server);
   connStringBuilder.Password = password;
   connectionString = connStringBuilder.ToString();

   connectionRetryPolicy = new RetryPolicy<SqlAzureTransientErrorDetectionStrategy>(
      5, TimeSpan.FromMilliseconds(100));

   connectionRetryPolicy.RetryOccurred += RetryConnectionCallback;

   RetryPolicyConfigurationSettings retryPolicySettings =
      ApplicationConfiguration.Current.GetConfigurationSection<
         RetryPolicyConfigurationSettings>(RetryPolicyConfigurationSettings.SectionName);

   RetryPolicyInfo retryPolicyInfo = retryPolicySettings.Policies.Get(“FixedIntervalDefault”);
   commandRetryPolicy =
      retryPolicyInfo.CreatePolicy<SqlAzureTransientErrorDetectionStrategy>();
   commandRetryPolicy.RetryOccurred += RetryCallbackCommand;
}

7. Add the following callback methods to the class:

private void RetryConnectionCallback(
   Int32 currentRetryCount, Exception lastException, TimeSpan delay)
{
   Int32 retryCount = currentRetryCount;
}

private void RetryCallbackCommand(
  Int32 currentRetryCount, Exception lastException, TimeSpan delay)
{
   Int32 retryCount = currentRetryCount;
}

8. Add the following method, retrieving the session tracing Id, to the class:

public String GetSessionTracingId()
{
  String commandText = “SELECT CONVERT(NVARCHAR(36), CONTEXT_INFO())”;

  String sessionTracingId;

  using (ReliableSqlConnection connection = new ReliableSqlConnection(connectionString))
  {
     connection.Open();
     using (SqlCommand sqlCommand = connection.CreateCommand())
     {
        sqlCommand.CommandText = commandText;
        sessionTracingId = sqlCommand.ExecuteScalarWithRetry() as String;
     }
   }  

   return sessionTracingId;
}

9. Add the following method, creating the Writer table, to the class:

public void CreateTable()
{
  String commandText = @”CREATE TABLE Writer (
     Id int PRIMARY KEY NOT NULL,
     Name nvarchar(20) NOT NULL,
     CountBooks int NULL)”;

  using (ReliableSqlConnection connection =
    new ReliableSqlConnection(connectionString, connectionRetryPolicy))
  {
     connection.Open();

     using (SqlCommand sqlCommand = connection.CreateCommand())
     {
       sqlCommand.CommandText = commandText;
       sqlCommand.ExecuteNonQueryWithRetry();
     }
   }
}

10. Add the following method, dropping the Writer table, to the class:

public void DropTable()
{
   String commandText = “DROP TABLE Writer”;

   using (ReliableSqlConnection connection = new ReliableSqlConnection(connectionString))
   {
      connection.Open(connectionRetryPolicy);

      using (SqlCommand sqlCommand = connection.CreateCommand())
      {
         sqlCommand.CommandText = commandText;
         sqlCommand.ExecuteNonQueryWithRetry(commandRetryPolicy);
      }
   }
}

11. Add the following method, querying the Writer table, to the class:

public void QueryTable()
{
   String commandText = “SELECT * FROM Writer”;

   using (ReliableSqlConnection connection =
      new ReliableSqlConnection(connectionString, connectionRetryPolicy, commandRetryPolicy))
   {
      connection.Open();

      using (SqlCommand sqlCommand = new SqlCommand(commandText, connection.Current))
      {
         using (IDataReader reader = connection.ExecuteCommand<IDataReader>(sqlCommand))
         {
            Int32 idColumn = reader.GetOrdinal(“Id”);
            Int32 nameColumn = reader.GetOrdinal(“Name”);
            Int32 countBooksColumn = reader.GetOrdinal(“CountBooks”);
            while (reader.Read())
            {
               Int32 id = (Int32)reader[idColumn];
               String name = reader[nameColumn] as String;
               Int32? countBooks = reader[countBooksColumn] as Int32?;
            }
         }
      }
   }
}

12. Add the following method, inserting rows in the Writer table, to the class:

public Int32 InsertRows()
{
   String commandText = @”INSERT INTO Writer (Id, Name, CountBooks)
     VALUES
        (1, N’Cervantes’, 2),
        (2, N’Smollett’, null),
        (3, ‘Beyle’, 4)”;

   Int32 rowsAffected;
   using (SqlConnection connection = new SqlConnection(connectionString))
   {
      connection.OpenWithRetry();

      using (SqlCommand sqlCommand = new SqlCommand(commandText, connection))
      {
         rowsAffected = sqlCommand.ExecuteNonQueryWithRetry();
      }
   }

   return rowsAffected;
}

13. Add the following method, updating a row in the Writer table, to the class:

public Int32 UpdateRow()
{
   RetryPolicy exponentialRetryPolicy = RetryPolicy.DefaultExponential;

   String commandText = @”UPDATE Writer
      SET Name=@Name
      WHERE Id=3″;

   Int32 rowsAffected;
   using (SqlConnection connection = new SqlConnection(connectionString))
   {
      connection.OpenWithRetry(exponentialRetryPolicy);

      using (SqlCommand sqlCommand = new SqlCommand(commandText, connection))
      {
         SqlParameter sqlParameter = new SqlParameter()
         {
            ParameterName = “@Name”,
            Value = “Stendhal”,
            SqlDbType = SqlDbType.NVarChar,
            Size = 20
         };

         sqlCommand.Parameters.Add(sqlParameter);

         rowsAffected = sqlCommand.ExecuteNonQueryWithRetry(exponentialRetryPolicy);
      }
   }
   return rowsAffected;
}

14. Add the following method, using the methods added earlier, to the class:

public static void UseRetryConnectionExample()
{
   String server = “SERVER_NAME”;
   String database = “DATABASE_NAME”;
   String login = “LOGIN”;
   String password = “PASSWORD”;

   RetryConnectionExample example = new RetryConnectionExample(
      server, database, login, password);
   example.GetSessionTracingId();
   example.CreateTable();
   example.InsertRows();
   example.QueryTable();
   example.UpdateRow();
   example.QueryTable();
   example.DropTable();
}

15. Add the following Transient Fault Handling Framework configuration to the app.config file for the project:

<configSections>
   <section name=”RetryPolicyConfiguration”
type=”Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling.Configuration.RetryPolicyConfigurationSettings,
Microsoft.AppFabricCAT.Samples.Azure.TransientFaultHandling”/>
</configSections>

<RetryPolicyConfiguration defaultPolicy=”FixedIntervalDefault”>
   <add name=”FixedIntervalDefault” maxRetryCount=”10″ retryInterval=”100″/>
</RetryPolicyConfiguration>

How it works…

In Step 1, we modify the output target of the project to make it consistent with the requirements of the Transient Fault Handling Framework. In Step 2, we add references to the Transient Fault Handling Framework assembly and to the System.configuration assembly used to access the Transient Fault Handling configuration in the app.config file.

In Steps 3 and 4, we set up the class. In Step 5, we add private members for the connection string and two RetryPolicy instances. In the constructor, we add in Step 6, we initialize the connection string using a SqlConnectionStringBuilder instance. Configuring a connection string for SQL Azure is precisely the same as for Microsoft SQL Server apart from the way in which the DataSource is specified—with the fully qualified host name. We turn encryption on, as this is required, and set TrustServerCertificate to false, so that the server certificate is validated. Instead of building the connection string like this, we could have loaded it from a configuration file.

For demonstration purposes, we initialize the RetryPolicy private members using different techniques. We create the connectionRetryPolicy member directly by providing initialization values in its constructor. We associate the RetryOccurred callback method with the connectionRetryPolicy member. We create the commandRetryPolicy member by retrieving a FixedIntervalDefault policy from the app.config file. We associate the RetryOccurred callback method with the commandRetryPolicy member. In both cases, we use SqlAzureTransientErrorDetectionStrategy to identify transient errors. This compares an error with a list of pre-defined transient errors.

In Step 7, we add two RetryOccurred callback methods the class. These have a trivial implementation that in a real application could be replaced by logging that a retry had occurred.

In Step 8, we create and open a ReliableSqlConnection which we use to create a SqlCommand. The connection is closed automatically when we exit the using block. We use SqlCommand to retrieve the session tracing ID for the connection. This is a GUID, identifying a particular connection, which can be provided to SQL Azure Support when its help is sought in debugging a problem. We use the default RetryPolicy when we open the connection and when we invoke the ExecuteScalarWithRetry() extension method. Note that the default RetryPolicy identifies all errors as being transient.

In Step 9, we invoke a CREATE TABLE operation on SQL Azure to create a table named Writer. The table has three columns: the Primary Key is the Id column; the remaining columns store the name of a writer and the number of books they wrote. We use the connectionRetryPolicy, configured in the constructor, when the connection is opened and the default RetryPolicy when we invoke the ExecuteNonQueryWithRetry() extension method.

In Step 10, we invoke a DROP TABLE operation on SQL Azure to drop the Writer table. We use the default RetryPolicy when the connection is opened and the commandRetryPolicy when we invoke the ExecuteNonQueryWithRetry() extension method.

In Step 11, we retrieve all rows from the Writer table and then iterate over them to examine the content of each column. We use the connectionRetryPolicy when the connection is opened and the commandRetryPolicy when we invoke the ExecuteComman<IDataReader>() extension method.

We insert three rows in the Writer table in Step 12. We invoke the OpenWithRetry() and ExecuteNonQueryWithRetry() extension methods to use the default RetryPolicy when we open and use the connection respectively. In Step 13, we use the same extension methods when we update a row in the Writer table. In this case, however, we parameterize them, so that we use the DefaultExponential retry policy when we open and use the connection. This default policy identifies all errors as transient.

In Step 14, we add a method that invokes the methods added earlier. We need to provide the server name, the database name, the login, and the password.

In Step 15, we add the configuration used to configure a RetryPolicy instance in Step 6. In doing so, we need to add a configSection element specifying the assembly used to access the configuration and then we add a RetryPolicyConfiguration element in which we specify a configuration we name FixedIntervalDefault.

Note that, with an appropriately configured connection string, all the code in this recipe can be run against Microsoft SQL Server—with the exception of the retrieval of the session tracing ID in Step 8.

There’s more…

The Transient Fault Handling Framework for Azure Storage, Service Bus, and SQL Azure can also be used for retrying operations against the Windows Azure Storage Service and the Windows Azure Service Bus.

See also

Valery Mizonov (@TheCATerminator) of the Windows Azure AppFabric Customer Advisory Team has written a blog post on Best practices for handling transient conditions in SQL Azure client applications. He explains how to use the Transient Fault Handling Framework. The post is available at the following URL:

http://windowsazurecat.com/2010/10/best-practices-for-handling-transient-conditions-in-sql-azure-client-applications/

Clemens Vasters (@clemensv) announced the release on the Windows Azure Team blog. The MSDN documentation for Windows Azure AppFabric v1.5 release, including Brokered Messaging, is here. Valery Mizonov (@TheCATerminator), of the Windows Azure Customer Advisory Team, has a post on Best Practices for Leveraging Windows Azure Service Bus Brokered Messaging API. Rick Garibay (@rickggaribay) has a post describing the differences between the preview and release versions that includes links to additional resources. Alan Smith (@alansmith) has a post providing an AppFabric Walkthrough: Simple Brokered Messaging (1st in a series). He has recently posted an eBook on Brokered Messaging as well as an update on deadletter sub-queues. There is also a post comparing Windows Azure Storage Queues with AppFabric Service Bus Brokered Messaging.

This post does not cover the WCF bindings and the REST API for Brokered Messaging that are also contained in the release. The post is a replacement for an earlier post describing the preview release of Service Bus Brokered Messaging.

Intro to Brokered Messaging

The name brokered messaging is used to distinguish the functionality from direct messaging in which a producer communicates directly with a consumer, and relayed messaging in which a producer communicates with a consumer through a relay. Both direct messaging and relayed messaging are subject to backpressure from the consumer being limited in how fast it can consume messages, and this causes the producer to throttle message production thereby limiting service scalability. By contrast, brokered messaging uses a high-capacity intermediate store to consume and durably store messages which can then be pulled by the consumer. The benefit of this is that the producer and consumer can scale independently of each other since the intermediate message broker buffers any difference in the real-time ability of the producer to send messages and the consumer to receive them.

The Windows Azure AppFabric Service Bus supports two distinct forms of brokered messages

• Queues
• Topics & Subscriptions

Queues represent a persistent sequenced buffer into which one or more producers send messages to the tail and one or more consumers compete to receive messages from the head. A queue has a single cursor, pointing to the current message, that is shared among all consumers. This cursor is moved each time a consumer receives a message. Service Bus Queues provide various methods to modify that default handling – including the timed visibility of messages on the queue, the deferral of message processing, and the ability of messages to reappear on the queue.

Topics supports a persistent sequenced buffer into which one or more producers send messages to the tail of the buffer which has multiple heads referred to as subscriptions, each of which receives distinct copies of the messages. One or more consumers compete to receive messages from the head of a subscription. Each subscription has its own cursor that is moved each time a message is retrieved.  A subscription can be filtered so that only a subset of messages are available through it. This provides for functionality where geographically-focused subscriptions contain messages only for particular regions while an auditing subscription contains all the messages.

TokenProvider Classes

The AppFabric ServiceBus v1.5 release introduced the following classes, derived from TokenProvider, to support authentication with Service Bus Brokered Messaging.

• SamlTokenProvider
• SharedSecretTokenProvider
• SimpleWebTokenProvider

The TokenProvider class creates factory methods to create instances of the various token providers. The factory methods for the SharedSecretTokenProvider supports the creation of a token from the issuer and shared secret for the service bus namespace.

These classes are all in the Microsoft.ServiceBus namespace in the Microsoft.ServiceBus.dll assembly.

NamespaceManager Class

The NamespaceManager and NamespaceManagerSettings classes are used to manage the Windows Azure AppFabric Service Bus namespace and, more specifically, the rendezvous endpoints used in Brokered Messaging.  It exposes methods supporting the creation and deletion of queues, topics and subscriptions. The NamespaceManagerSettings class exposes the OperationTimeout and the TokenProvider used with operations performed by an associated NamespaceManager.

The NamespaceManager class is declared (in truncated form):

public sealed class NamespaceManager {
    public NamespaceManager(String address, NamespaceManagerSettings settings);
    public NamespaceManager(Uri address, NamespaceManagerSettings settings);
    public NamespaceManager(String address, TokenProvider tokenProvider);
    public NamespaceManager(Uri address, TokenProvider tokenProvider);

    public Uri Address { get; }
    public NamespaceManagerSettings Settings { get; }

    public QueueDescription CreateQueue(String path);
    public QueueDescription CreateQueue(QueueDescription description);
    public SubscriptionDescription CreateSubscription(String topicPath, String name, Filter filter);
    public SubscriptionDescription CreateSubscription(String topicPath, String name,
       RuleDescription ruleDescription);
    public SubscriptionDescription CreateSubscription(SubscriptionDescription description,
       RuleDescription ruleDescription);
    public SubscriptionDescription CreateSubscription(String topicPath, String name);
    public SubscriptionDescription CreateSubscription(SubscriptionDescription description,
       Filter filter);
    public SubscriptionDescription CreateSubscription(SubscriptionDescription description);
    public TopicDescription CreateTopic(String path);
    public TopicDescription CreateTopic(TopicDescription description);
    public void DeleteQueue(String path);
    public void DeleteSubscription(String topicPath, String name);
    public void DeleteTopic(String path);
    public QueueDescription GetQueue(String path);
    public IEnumerable<QueueDescription> GetQueues();
    public IEnumerable<RuleDescription> GetRules(String topicPath, String subscriptionName);
    public SubscriptionDescription GetSubscription(String topicPath, String name);
    public IEnumerable<SubscriptionDescription> GetSubscriptions(String topicPath);
    public TopicDescription GetTopic(String path);
    public IEnumerable<TopicDescription> GetTopics();
    public Boolean QueueExists(String path);
    public Boolean SubscriptionExists(String topicPath, String name);
    public Boolean TopicExists(String path);
}

The constructors provide various ways of creating a NamespaceManager from a full namespace URI and a token provider, either directly or through a NamespaceManagerSettings instance. There are also asynchronous versions of all the methods.

The ServiceBusEnvironment class contains some helper methods to create the appropriate namespace URI. For example, the following code creates a shared secret token provider and a service URI – and uses them to create a NamespaceManager instance:

String serviceNamespace = “mynamespace”;
String issuer = “owner”;
String issuerKey = “base64-encoded key”;

TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
   issuer, issuerKey);
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri(“sb”, serviceNamespace,
   String.Empty);
NamespaceManager namespaceManager = new NamespaceManager(serviceUri, tokenProvider);

The NamespaceManager class provides an extensive set of methods supporting the management of queues, topics and subscriptions – including the creation, deletion and existence checking of them. It also provides methods allowing for the retrieval of the properties of either a specific queue, topic and subscription, or all of each type. Each of these methods is available in both synchronous and asynchronous versions.

The following example uses a NamespaceManager instance to create a queue asynchronously, and then synchronously a topic and an associated subscription.

IAsyncResult iAsyncResult = namespaceManager.BeginCreateQueue(
    “myqueue”,
    (result) =>
    {
        namespaceManager.EndCreateQueue(result);
    },
    null);

TopicDescription topicDescription = namespaceManager.CreateTopic(“mytopic”);
SubscriptionDescription subscriptionDescription =
   namespaceManager.CreateSubscription(topicDescription.Path, “mysubscription”);

The delete and existence-checking methods are invoked similarly to the create methods.

Note that service scalability can be significantly improved by using asynchronous calls rather than blocking synchronous calls. This is particularly true when receiving messages from queues and subscriptions. For clarity, this post primarily uses synchronous calls because it is easier to understand what is going on. However, the Valery Mizonov post on Best Practices for Leveraging Windows Azure Service Bus Brokered Messaging API has an extensive discussion on the robust use of asynchronous calls.

Note that other than to avoid namespace collisions no knowledge of how the namespace is allocated to queues, topics and subscriptions is needed when using the Brokered Messaging API. However, for completeness with a service named myservice, a queue named myqueue, a topic named mytopic and a subscription named mysubscription the following show examples of the full address for a queue, a topic and a subscription respectively:

• https://myservice.windows.net/myqueue
• https://myservice.windows.net/mytopic
• https://myservice.windows.net/mytopic/subscriptions/mysubscription

QueueDescription, TopicDescription and SubscriptionDescription

The QueueDescription, TopicDescription  and SubscriptionDescription classes are used with the NamespaceManager factory methods to parameterize the creation of queues, topics and subscriptions. It is interesting to look at the properties of these classes since they provide a direct way of understanding the functionality exposed through queues, topics and subscriptions. In particular, when displayed in tabular form it becomes immediately apparent which properties affect message sending and which affect message retrieval.

DefaultMessageTimeToLive specifies the default time to live for a message. DuplicateDetectionHistoryTimeWindow specifies the default time window for duplicate message detection. EnableBatchOperations specifies whether server-side batch operations are enabled. EnableDeadLetteringOnFilterEvaluationExceptions specifies whether messages encountering filter-evaluation exceptions on a subscription are sent to the dead letter subqueue. EnableDeadLetteringOnMessageExpiration specifies whether messages which have reached their time-to-live are sent to the dead letter subqueue. LockDuration specifies the default value for the lock duration when a message is retrieved using the PeekLock receive mode.

MaxDeliveryCount specifies the maximum number of
delivery attempts before a message is sent to the dead letter subqueue. MaxSizeInMegabytes specifies the maximum size in megabytes of the queue. Note that the current quotas allow for queues up to 5 GB. MessageCount returns the number of messages in the queue or subscription. Name specifies the name of a subscription. Path specifies the name of a queue or topic. RequiresDuplicateDetection specifies whether the queue or topic implements duplicate detection. RequiresSession specifies whether a MessageSession receiver must be used to retrieve messages from the queue or subscription. SizeInBytes returns the size of the queue or topic. TopicPath specifies the topic path for a subscription.

BrokeredMessage

The BrokeredMessage class represents a brokered message to be sent to a queue or topic or retrieved from a queue or subscription. It is declared:

public sealed class BrokeredMessage : IXmlSerializable, IDisposable {
public BrokeredMessage(Object serializableObject, XmlObjectSerializer serializer);
public BrokeredMessage(Stream messageBodyStream, Boolean ownsStream);
public BrokeredMessage();
public BrokeredMessage(Object serializableObject);

public String ContentType { get; set; }
public String CorrelationId { get; set; }
public Int32 DeliveryCount { get; }
public DateTime EnqueuedTimeUtc { get; }
public DateTime ExpiresAtUtc { get; }
public String Label { get; set; }
public DateTime LockedUntilUtc { get; }
public Guid LockToken { get; }
public String MessageId { get; set; }
public IDictionary<String,Object> Properties { get; }
public String ReplyTo { get; set; }
public String ReplyToSessionId { get; set; }
public DateTime ScheduledEnqueueTimeUtc { get; set; }
public Int64 SequenceNumber { get; }
public String SessionId { get; set; }
public Int64 Size { get; }
public TimeSpan TimeToLive { get; set; }
public String To { get; set; }

public void Abandon();
public IAsyncResult BeginAbandon(AsyncCallback callback, Object state);
public IAsyncResult BeginComplete(AsyncCallback callback, Object state);
public IAsyncResult BeginDeadLetter(AsyncCallback callback, Object state);
public IAsyncResult BeginDeadLetter(String deadLetterReason,
String deadLetterErrorDescription, AsyncCallback callback, Object state);
public IAsyncResult BeginDefer(AsyncCallback callback, Object state);
public void Complete();
public void DeadLetter();
public void DeadLetter(String deadLetterReason, String deadLetterErrorDescription);
public void Defer();
public void EndAbandon(IAsyncResult result);
public void EndComplete(IAsyncResult result);
public void EndDeadLetter(IAsyncResult result);
public void EndDefer(IAsyncResult result);
public T GetBody<T>();
public T GetBody<T>(XmlObjectSerializer serializer);

public void Dispose();
public override String ToString();
XmlSchema IXmlSerializable.GetSchema();
void IXmlSerializable.ReadXml(XmlReader reader);
void IXmlSerializable.WriteXml(XmlWriter writer);
}

The BrokeredMessage class exposes several constructors as well as various methods to manage methods received using the PeekLock receive mode:

• Abandon() to have the queue or subscription unlock the message and make it visible to consumers. 
• Complete() to have the queue or subscription delete the message
• DeadLetter() to have the queue or subscription transfer the message to the dead letter subqueue.
• Defer() to have the queue or subscription defer the message for subsequent processing.

The dead letter subqueue is used as a store where messages can be stored for out-of-band processing. This is typically used when there is a problem with the message which needs to be investigated outside normal processing. Note that a deferred message can only be received by the explicit provision of the sequence number for the message. Consequently, it is essential to save the sequence number for later use before invoking Defer().

These methods exist in both synchronous and asynchronous form. The BrokeredMessage class also exposes various accessor methods to the message body. The BrokeredMessage class has several properties which help describe some of the features of brokered messaging.

ContentType is an application-specific value that can be used to specify the type of data in the message. The CorrelationId is used with the CorrelationFilter to correlate two-way messaging between the producer and consumer. DeliveryCount specifies the number of times the message has been delivered to a consumer. EnqueuedTimeUtc specifies when the message was sent to the queue or topic. ExpiresAtUtc specifies when the message expires on the queue or subscription. Label is an application-specific label for the message. LockedUntilUtc specifies the unlock time for a message received using the PeekLock receive mode. LockToken specifies the lock token of the message received using the PeekLock receive mode. The MessageId is an application-specific identifier for a message.

The Properties collection contains a set of application-specific properties which can be used, instead of the message body, to pass message content. The Properties collection is also accessible during the rule processing for message retrieval from a subscription. ReplyTo contains the name of a queue to which any reply should be sent. The ReplyToSessionId specifies a session the message should be replied to. ScheduledEnqueueTimeUtc specifies a future time when a new message will become visible on the queue or topic. By default, a message becomes visible immediately. The SequenceNumber is a Service Bus provided unique identifier for the message.
The SessionId identifies a session, if any, that the message belongs to.
Size specifies the size of the message in bytes. TimeToLive specifies the time-to-live of the message after which it is either deleted or moved to the dead letter subqueue. With regard to To, your guess is as good as mine.

ReceiveMode

A consumer can receive a message in one of two receive modes: PeekLock (the default) and ReceiveAndDelete. In PeekLock receive mode, a consumer receives the message which remains invisible on the queue until the consumer either abandons the message (physically or through a timeout) and the message once again becomes visible or deletes the message on successful completion of processing. This mode supports at least once processing, since each message is consumed one or more times. In ReceiveAndDelete mode, a consumer receives the message and it is immediately deleted from the queue. This mode provides for at most once processing since each message is consumed at most once. A message can be lost in ReceiveAndDelete mode if the consumer fails while processing a message.

MessagingFactory and MessagingClientEntity

Messages are sent and received using instances of classes in the MessagingClientEntity hierarchy:

MessagingClientEntity
   MessageReceiver
      MessageSession
   MessageSender
   MessagingFactory
   QueueClient
   SubscriptionClient
   TopicClient

The MessagingFactory class contains factory methods that can be used to create a MessagingFactory instance which can then be used to create instances of the MessageReceiver, MessageSender, QueueClient, SubscriptionClient and TopicClient classes. The QueueClient and SubscriptionClient classes expose methods to create MessageSession intances.

The QueueClient class is used to send and receive messages to a queue. The TopicClient class is used to send messages to a topic while the SubscriptionClient class is used to receive messages from a subscription to a topic. The MessageSender class and MessageReceiver classes abstract the sending and receiving functionality so that they can be used against either queues or topics and subscriptions. The MessageSession class exposes functionality allowing queue and subscription receivers to receive related messages as a “session.” After some receiver receives a message in a session, all messages in that session are reserved for that receiver.

QueueClient

The QueueClient class is to some extent a superset of the TopicClient and SubscriptionClient classes. This is because it has to handle both the sending and receiving of messages. It lacks the functionality related to rules, filters and actions that the SubscriptionClient exposes but sending to a topic is pretty much the same as sending to a queue.

The following is a severely truncated class declaration for the QueueClient class:

public abstract class QueueClient : MessageClientEntity {
    public MessagingFactory MessagingFactory { get; }
    public String Path { get; }
    public Int32 PrefetchCount { get; set; }

    public static String FormatDeadLetterPath(String queuePath);

    public MessageSession AcceptMessageSession();
    public void Abandon(Guid lockToken);
    public void Complete(Guid lockToken);
    public void Defer(Guid lockToken);
    public void DeadLetter(Guid lockToken);
    public BrokeredMessage Receive();
    public void Send(BrokeredMessage message);
    public ReceiveMode Mode { get; }
}

The PrefetchCount property is a performance enhancement that specifies how many messages should be retrieved BEFORE the first call to Receive(). Note that this can lead to message loss if the consumer fails before completely processing the messages.

The instance methods of the class exist have several overloads, each of which has an associated asynchronous version. AcceptMessageSession() is used to request the next session (with an overload allowing a specific session to be requested). The returned MessageSession instance can then be used to retrieve sequentially the messages in that session.

The Abandon(), Complete(), DeadLetter()and Defer() methods have the same meaning as the equivalent methods in the BrokeredMessage class. The difference is that in this case the lock token contained in the message must be provided as a parameter.

The FormatDeadLetterPath() method is invoked to get the path for the dead letter subqueue. This path can then be used to create a QueueClient to retrieve messages from the dead letter subqueue.

A produces invokes Send() to send a message to the queue while a consumer invokes Receive() to receive a message from the queue. Note that a message received using the ReceiveAndDelete receive mode is deleted immediately from the queue. None of the Abandon(), Complete(), Defer() and DeadLetter() functionality, just described, applies to such a message.

The following code fragment shows messages being sent to two separate queues:

MessagingFactory messagingFactory = MessagingFactory.Create(serviceUri, tokenProvider);

QueueClient bodySender = messagingFactory.CreateQueueClient(“BodyQueue”);
BrokeredMessage brokeredMessage1 = new BrokeredMessage(“serializable message body”);
bodySender.Send(brokeredMessage1);

QueueClient propertySender = messagingFactory.CreateQueueClient(“PropertyQueue”);
BrokeredMessage brokeredMessage2 = new BrokeredMessage()
{
    Properties = {
        {“Month”, “July”},
        {“NumberOfDays”, 31}
    }
};
propertySender.Send(brokeredMessage2);

messagingFactory.Close();
messagingFactory = null;

These show two different ways of inserting information in a message – either as serializable content in the message body or as message properties. In the first case, a message containing body text is sent to a queue. In the second case, a message containing a pair of properties is sent to a different queue. These show distinct ways of using messages. Other than being convenient when a message contains only a few properties, the latter becomes essential when using subscriptions since the Service Bus messaging broker can modify message handling based on the value of the message properties. Indeed, it can even modify the values of the properties. This functionality is not available when the actionable content is stored in the message body.

The fragment also shows the closing of the MessagingFactory instance. Each MessagingClientEntity (QueueClient, TopicClient, etc.) gets its own connection and the resources consumed by the connection should be released when no longer needed. This can be done either by invoking Close() on the individual MessagingClientEntity or by invoking close on the MessagingFactory instance, which releases all resources created through the factory instance including any MessagingClientEntity instances created by it.

The following code fragment shows messages being retrieved from two separate queues:

QueueClient bodyReceiver = messagingFactory.CreateQueueClient(“BodyQueue”,
   ReceiveMode.ReceiveAndDelete);
BrokeredMessage brokeredMessage3 = bodyReceiver.Receive();
String messageBody = brokeredMessage3.GetBody<String>();
bodyReceiver.Close();

QueueClient propertyReceiver = messagingFactory.CreateQueueClient(“PropertyQueue”);
propertyReceiver.BeginReceive(ReceiveDone, propertyReceiver);

public static void ReceiveDone(IAsyncResult result)
{
    QueueClient queueClient = result.AsyncState as QueueClient;
    BrokeredMessage brokeredMessage = queueClient.EndReceive(result);
    String messageId = brokeredMessage.MessageId;
    String month = brokeredMessage.Properties["Month"] as String;
    Int32 numberOfDays = (Int32)brokeredMessage.Properties["NumberOfDays"];

    switch (numberOfDays)
    {
        case 28:
            brokeredMessage.Defer();
            break;
        case 30:
            brokeredMessage.Complete();
            break;
        case 31:
            brokeredMessage.Abandon();
            break;
    }
    queueClient.Close();
}

This sample shows both the synchronous and asynchronous retrieval of messages. In the synchronous case, the actionable content is retrieved from the message body. In the asynchronous case, the actionable content is retrieved from message properties. The property values are then used to select the disposition of the message as deferred, completed or abandoned. Note that this depends on the message receive mode being the default PeekLock. The MessageId needs to be retained for a deferred message since such a message can be retrieved only by MessageId.

TopicClient

A TopicClient is the MessageClientEntity used to send messages to a topic. It exposes synchronous and asynchronous versions of the Send() method.

SubscriptionClient

The SubscriptionClient class us used to handle messages received from a subscription. SubscriptionClient is declared (in truncated form):

public abstract class SubscriptionClient : MessageClientEntity {
    public MessagingFactory MessagingFactory { get; }
    public String Name { get; }
    public Int32 PrefetchCount { get; set; }
    public String TopicPath { get; }

    public IAsyncResult BeginAddRule(String ruleName, Filter filter, AsyncCallback callback,
       Object state);
    public IAsyncResult BeginAddRule(RuleDescription description, AsyncCallback callback,
       Object state);
    public IAsyncResult BeginRemoveRule(String ruleName, AsyncCallback callback,
       Object state);
    public void EndAddRule(IAsyncResult result);
    public void EndRemoveRule(IAsyncResult result);
    public static String FormatDeadLetterPath(String topicPath, String subscriptionName);
    public static String FormatSubscriptionPath(String topicPath, String subscriptionName);
    public void RemoveRule(String ruleName);

    public IAsyncResult BeginAcceptMessageSession(AsyncCallback callback,
       Object state);
    public IAsyncResult BeginAcceptMessageSession(String sessionId, AsyncCallback callback,
       Object state);
    public IAsyncResult BeginAcceptMessageSession(TimeSpan serverWaitTime,
       AsyncCallback callback, Object state);
    public IAsyncResult BeginAcceptMessageSession(String sessionId, TimeSpan serverWaitTime,
       AsyncCallback callback, Object state);
    public MessageSession EndAcceptMessageSession(IAsyncResult result);
    public IAsyncResult BeginAbandon(Guid lockToken, AsyncCallback callback, Object state);
    public void EndAbandon(IAsyncResult result);
    public IAsyncResult BeginComplete(Guid lockToken, AsyncCallback callback, Object state);
    public void EndComplete(IAsyncResult result);
    public IAsyncResult BeginDefer(Guid lockToken, AsyncCallback callback, Object state);
    public void EndDefer(IAsyncResult result);
    public IAsyncResult BeginDeadLetter(Guid lockToken, AsyncCallback callback, Object state);
    public IAsyncResult BeginDeadLetter(Guid lockToken, String deadLetterReason,
       String deadLetterErrorDescription, AsyncCallback callback, Object state);
    public void EndDeadLetter(IAsyncResult result);
    public IAsyncResult BeginReceive(AsyncCallback callback, Object state);
    public IAsyncResult BeginReceive(TimeSpan serverWaitTime, AsyncCallback callback,
       Object state);
    public IAsyncResult BeginReceive(Int64 sequenceNumber, AsyncCallback callback,
       Object state);
    public BrokeredMessage EndReceive(IAsyncResult result);
    public ReceiveMode Mode { get; }
}

The Begin/End asynchronous pairs of methods have equivalent synchronous methods. The Windows Azure AppFabric Service Bus team is serious about wanting us to do everything asynchronously. Much of the exposed functionality is familiar from the receive capability of a QueueClient – AcceptMessageSession(), Abandon(), Complete(), DeadLetter(), Defer() and Receive().

The novel functionality exposed by a SubscriptionClient is that related to rules used to modify the behavior of a subscription. Note that this affects the subscription NOT the subscription receiver. Any modification to the rules using these methods affect only those messages sent to the associated topic after the change. I’m not convinced that this functionality should be in SubscriptionClient as well as the NamespaceManager since the methods affect the subscription not the SubscriptionClient. As it is care must be taken when there is more than one rule for a subscription since a message can be received more than once if it satisfies more than one rule on the subscription.

Rules, Filters and Actions

The RuleDescription class associates a rule name with a Filter and an optional RuleAction. The default Filter for a RuleDescription is the TrueFilter which always evaluates to true. A Filter is used to filter the messages, based on the values of the message properties, that a subscription handles for a topic. A message passes the filter when it evaluates to true. RuleAction is used to modify the properties of a message when it is received through the subscription. Rules significantly enhance the power of a subscription over a simple queue in that different filter and rule actions can be used for different subscriptions against the same topic. This allows different consumers to have very different views of the messages sent to the topic. Note that a rule does not need to have an associated RuleAction.

There are several Filter classes in the following hierarchy:

Filter
    CorrelationFilter
    FalseFilter
    SqlFilter
    TrueFilter

The CorrelationFilter provides a filter on the value of the CorrelationId of a message. This filter is intended to be used when correlating a reply message from a consumer to a producer. A reply message is put on a reply queue with the associated CorrelationId, and the original producer can use a CorrelationFilter on this queue to receive a response to a message it sent. The FalseFilter always fails so that no message is received through a subscription with a FalseFilter. A TrueFilter always succeeds so that all messages are received through a subscription with a TrueFilter. A SqlFilter supports a simple SQL-92 style syntax that can be used to filter messages depending on the property values.

The RuleAction classes have the following hierarchy:

RuleAction
    SqlRuleAction

SqlRuleAction supports a simple SQL-92 style syntax allowing message properties to be modified as they are received through the subscription.

The following code fragment shows the creation of a topic and two subscriptions and the sending of some messages to the topic. Note that a subscription handles only those messages submitted to the topic after the subscription is created. Similarly, a rule only affects messages submitted after the rule is added.

TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
   issuer, issuerKey);
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri(“sb”, serviceNamespace,
   String.Empty);
NamespaceManager namespaceManager = new NamespaceManager(serviceUri, tokenProvider);

RuleDescription summerRule = new RuleDescription()
{
    Action = new SqlRuleAction(“SET HavingFun = ‘Yes’”),
    Filter = new SqlFilter(“Month = ‘June’ OR Month = ‘July’ OR [Month] = ‘August’”),
    Name = “SummerRule”
};

TopicDescription topicDescription = namespaceManager.CreateTopic(“WeatherTopic”);
SubscriptionDescription summerSubscriptionDescription =
   namespaceManager.CreateSubscription(“WeatherTopic”, “WeatherSubscription”, summerRule);
SubscriptionDescription allSubscriptionDescription =
   namespaceManager.CreateSubscription(“WeatherTopic”, “AllSubscription”);

IEnumerable<RuleDescription> rules = namespaceManager.GetRules(
   “WeatherTopic”, “WeatherSubscription”);
List<RuleDescription> listRules = rules.ToList();

MessagingFactory messagingFactory = MessagingFactory.Create(serviceUri, tokenProvider);
TopicClient weatherSender = messagingFactory.CreateTopicClient(“WeatherTopic”);

BrokeredMessage augustMessage = new BrokeredMessage()
{
    Properties = {
        {“HavingFun”, “No”},
        {“Month”, “August”},
        {“NumberOfDays”, 31}
    }
};
weatherSender.Send(augustMessage);

BrokeredMessage aprilMessage = new BrokeredMessage()
{
    Properties = {
        {“HavingFun”, “No”},
        {“Month”, “April”},
        {“NumberOfDays”, 30}
    }
};
weatherSender.Send(aprilMessage);

BrokeredMessage septemberMessage = new BrokeredMessage()
{
    Properties = {
        {“HavingFun”, “No”},
        {“Month”, “September”},
        {“NumberOfDays”, 30}
    }
};
weatherSender.Send(septemberMessage);

weatherSender.Close();
weatherSender = null;

This fragment creates a RuleDescription with an action that sets the HavingFun property of a message to Yes if the Month property is June, July or August. A topic and two associated subscriptions are created, one of which is assigned this rule while the other has no filter (and consequently passes all messages). A MessagingFactory is then created which, in turn, creates a TopicClient used to send to the topic three messages with different property values.

The following code fragment shows the retrieval of messages from two different subscriptions to the same topic. The first subscription has a rule that filters the messages and transforms the message properties while the second has rule that passes through all the message and performs no transformations.

SubscriptionClient summerReceiver = messagingFactory.CreateSubscriptionClient(“WeatherTopic”, “WeatherSubscription”, ReceiveMode.ReceiveAndDelete);

while (true)
{
    BrokeredMessage brokeredMessage = summerReceiver.Receive(TimeSpan.FromSeconds(1));
    if (brokeredMessage != null)
    {
        String havingFun = brokeredMessage.Properties["HavingFun"] as String;
        String month = brokeredMessage.Properties["Month"] as String;
        brokeredMessage.Dispose();
    }
    else
    {
        break;
    }
}

SubscriptionClient allReceiver = messagingFactory.CreateSubscriptionClient(“WeatherTopic”, “AllSubscription”, ReceiveMode.ReceiveAndDelete);

while (true)
{
    BrokeredMessage brokeredMessage = allReceiver.Receive(TimeSpan.FromSeconds(1));
    if (brokeredMessage != null)
    {
        String havingFun = brokeredMessage.Properties["HavingFun"] as String;
        String month = brokeredMessage.Properties["Month"] as String;
        brokeredMessage.Dispose();
    }
    else
    {
        break;
    }
}

messagingFactory.Close();
messagingFactory = null;

MessageSender and MessageReceiver

The MessageSender and MessageReceiver classes abstract out the sending and receiving functionality of queues, topics and subscriptions, and can be used instead of the QueueClient, TopicClient and SubscriptionClient classes. The MessageSender class exposes synchronous and asynchronous Send() methods while the MessageReceiver class exposes the usual synchronous and asynchronous Abandon(), Complete(), DeadLetter() and Defer() methods.

The earlier example using a SubscriptionClient to receive messages can be converted to use a MessageReceiver by replacing the following line:

SubscriptionClient summerReceiver = messagingFactory.CreateSubscriptionClient(
   “WeatherTopic”, “WeatherSubscription”, ReceiveMode.ReceiveAndDelete);

with

MessageReceiver summerReceiver = messagingFactory.CreateMessageReceiver(
   “WeatherTopic/subscriptions/WeatherSubscription”, ReceiveMode.ReceiveAndDelete);

with WeatherTopic/subscriptions/WeatherSubscription being the full entity path to the WeatherSubscription on WeatherTopic.

UPDATE 10.11.2011

A MessageReceiver can be used to receive messages from a dead letter subqueue for a subscription. For example, the following code fragment changes the earlier example to one that accesses the dead letter subqueue:

SubscriptionClient allReceiver =
   messagingFactory.CreateSubscriptionClient(“WeatherTopic”, “AllSubscription”,
   ReceiveMode.PeekLock);
String deadLetterPath =
   SubscriptionClient.FormatDeadLetterPath(“WeatherTopic”, allReceiver.Name);
MessageReceiver deadLetterReceiver =
   messagingFactory.CreateMessageReceiver(deadLetterPath);

while (true)
{
    BrokeredMessage brokeredMessage =
       deadLetterReceiver.Receive(TimeSpan.FromSeconds(1));
    if (brokeredMessage != null)
    {
        String havingFun = brokeredMessage.Properties["HavingFun"] as String;
        String month = brokeredMessage.Properties["Month"] as String;
        brokeredMessage.Dispose();
    }
    else
    {
        break;
    }
}

In the example, the value of deadLetterPath is:

WeatherTopic/Subscriptions/AllSubscription/$DeadLetterQueue

MessageSession

Service Bus Brokered Messaging supports sessions and session state. A queue or subscription must be set up to support sessions through the use of the RequiresSession property in the QueueDescription or SubscriptionDescription. Subsequently, each message may have a SessionId associated with it. The intent is that related messages share a SessionId and comprise a session. A MessageSession receiver can then be created on the queue or subscription to receive messages with the same SessionId. A persistent session state can be associated with the session. This can be used along with the PeekLock receive mode to implement exactly once processing of messages since the session state can be used to store the current state of processing regardless of any failure in the session consumer.

The MessageSession class is derived from the MessageReceiver class and adds synchronous and asynchronous versions of the following methods to maintain session state:

public Stream GetState();
public void SetState(Stream stream);

The following code fragment shows the creation of a session-aware queue and the sending of some messages to it:

TokenProvider tokenProvider = TokenProvider.CreateSharedSecretTokenProvider(
   issuer, issuerKey);
Uri serviceUri = ServiceBusEnvironment.CreateServiceUri(
   “sb”, serviceNamespace, String.Empty);
NamespaceManager namespaceManager = new NamespaceManager(serviceUri, tokenProvider);

QueueDescription queueDescription = new QueueDescription(“SessionQueue”)
{
    RequiresSession = true
};
namespaceManager.CreateQueue(queueDescription);

MessagingFactory messagingFactory = MessagingFactory.Create(serviceUri, tokenProvider);

QueueClient sender = messagingFactory.CreateQueueClient(“SessionQueue”);

for (Int32 i = 1; i < 10; i++)
{
    BrokeredMessage brokeredMessage = new BrokeredMessage()
    {
        MessageId = i.ToString(),
        SessionId = (i % 3).ToString(),
    };
    sender.Send(brokeredMessage);
}
sender.Close();

The novelty in this fragment is in the RequiresSession property for the QueueDescription and the setting of the SessionId on each message.

The following code fragment shows the use of a MessageSession to receive messages in a single session:

QueueClient receiver = messagingFactory.CreateQueueClient(
   “SessionQueue”, ReceiveMode.ReceiveAndDelete);
MessageSession sessionReceiver = receiver.AcceptMessageSession();
while ( true )
{
    BrokeredMessage message = sessionReceiver.Receive(TimeSpan.FromSeconds(1));
    if (message == null)
    {
        break;
    }
    String messageId = message.MessageId;
    String sessionId = message.SessionId;
}
messagingFactory.Close();

The MessageSession receiver is created by invoking AcceptMessageSession() on the QueueClient receiver. The session receiver receives the first message and then all subsequent messages with the same SessionId as the first one.

I was finally able to revisit the topic in my book, Microsoft Windows Azure Development Cookbook, in which I provide a pretty comprehensive overview of how to use the Windows Azure Service Management REST API. There is an entire chapter devoted to the topic – and the publisher, Packt Publishing, have made the chapter available as a free download. It covers the use of the Service Management API for:

• creating a Windows Azure hosted service
• deploying an application into a hosted service
• upgrading an application deployed to a hosted service
• retrieving the properties of a hosted service
• autoscaling with the Windows Azure Service Management REST API
• using the Windows Azure Platform PowerShell cmdlets

The rest of this post is the section of the book on creating a Windows Azure hosted service. (Apologies for the code formatting here – some day I will learn how to get that right on the blog.)

Creating a Windows Azure hosted service

A hosted service is the administrative and security boundary for an application deployed to Windows Azure. The hosted service specifies the service name, a label, and either the Windows Azure datacenter location or the affinity group into which the application is to be deployed. These cannot be changed once the hosted service is created. The service name is the subdomain under cloudapp.net used by the application, and the label is a human-readable name used to identify the hosted service on the Windows Azure Portal.

The Windows Azure Service Management REST API exposes a create hosted service operation. The REST endpoint for the create hosted service operation specifies the subscription ID under which the hosted service is to be created. The request requires a payload comprising an XML document containing the properties needed to define the hosted service, as well as various optional properties. The service name provided must be unique across all hosted services in Windows Azure, so there is a possibility that a valid create hosted service operation will fail with a 409 Conflict error if the provided service name is already in use. As the create hosted service operation is asynchronous, the response contains a request ID that can be passed into a get operation status operation to check the current status of the operation.

In this recipe, we will learn how to use the Service Management API to create a Windows Azure hosted service.

Getting ready

The recipes in this chapter use the ServiceManagementOperation utility class to invoke operations against the Windows Azure Service Management REST API. We implement this class as follows:

1. Add a class named ServiceManagementOperation to the project.

2. Add the following assembly reference to the project:

System.Xml.Linq.dll

3. Add the following using statements to the top of the class file:

using System.Security.Cryptography.X509Certificates;
using System.Net;
using System.Xml.Linq;
using System.IO;

4. Add the following private members to the class:

String thumbprint;
String versionId = "2011-02-25";

5. Add the following constructor to the class:

public ServiceManagementOperation(String thumbprint)
{
   this.thumbprint = thumbprint;
}

6. Add the following method, retrieving an X.509 certificate from the certificate store, to the class:

private X509Certificate2 GetX509Certificate2( String thumbprint)
{
    X509Certificate2 x509Certificate2 = null; 

    X509Store store = new X509Store("My", StoreLocation.LocalMachine); 

    try
    {
        store.Open(OpenFlags.ReadOnly); 

        X509Certificate2Collection x509Certificate2Collection =
            store.Certificates.Find(
        X509FindType.FindByThumbprint, thumbprint, false); 

        x509Certificate2 = x509Certificate2Collection[0];
    } 

    finally
    {
        store.Close();
    } 

    return x509Certificate2;
}

7. Add the following method, creating an HttpWebRequest, to the class:

private HttpWebRequest CreateHttpWebRequest( Uri uri, String httpWebRequestMethod)
{
    X509Certificate2 x509Certificate2 = GetX509Certificate2(thumbprint);
    HttpWebRequest httpWebRequest = (HttpWebRequest)HttpWebRequest.Create(uri);
    httpWebRequest.Method = httpWebRequestMethod;
    httpWebRequest.Headers.Add("x-ms-version", versionId);
    httpWebRequest.ClientCertificates.Add(x509Certificate2);
    httpWebRequest.ContentType = "application/xml";
    return httpWebRequest;
}

8. Add the following method, invoking a GET operation on the Service Management API, to the class:

public XDocument Invoke(String uri)
{
    XDocument responsePayload;
    Uri operationUri = new Uri(uri); 

    HttpWebRequest httpWebRequest = CreateHttpWebRequest(operationUri, "GET");
    using (HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse())
    {
        Stream responseStream = response.GetResponseStream();
        responsePayload = XDocument.Load(responseStream);
    } 

    return responsePayload; 

}

9. Add the following method, invoking a POST operation on the Service Management API, to the class:

public String Invoke(String uri, XDocument payload)
{
    Uri operationUri = new Uri(uri); 

    HttpWebRequest httpWebRequest = CreateHttpWebRequest(operationUri, "POST"); 

    using (Stream requestStream = httpWebRequest.GetRequestStream())
    {
        using (StreamWriter streamWriter =
        new StreamWriter(requestStream,
        System.Text.UTF8Encoding.UTF8))
        {
            payload.Save(streamWriter,
            SaveOptions.DisableFormatting);
        }
    } 

    String requestId;
    using (HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse())
    {
        requestId = response.Headers["x-ms-request-id"];
    } 

    return requestId;
}

How it works…

In steps 1 through 3, we set up the class. In step 4, we add a version ID for service management operations. Note that Microsoft periodically releases new operations for which it provides a new version ID, which is usually applicable for operations added earlier. In step 4, we also add a private member for the X.509 certificate thumbprint that we initialize in the constructor we add in step 5.

In step 6, we open the Personal (My) certificate store on the local machine level and retrieve an X.509 certificate identified by thumbprint. If necessary, we can specify the current user level, instead of the local machine level, by using StoreLocation.CurrentUser instead of StoreLocation.LocalMachine.

In step 7, we create an HttpWebRequest with the desired HTTP method type, and add the X.509 certificate to it. We also add various headers including the required x-ms-version.

In step 8, we invoke a GET request against the Service Management API and load the response into an XML document which we then return. In step 9, we write an XML document, containing the payload, into the request stream for an HttpWebRequest and then invoke a POST request against the Service Management API. We extract the request ID from the response and return it.

How to do it…

We are now going to construct the payload required for the create hosted service operation, and then use it when we invoke the operation against the Windows Azure Service Management REST API. We do this as follows:

1. Add a new class named CreateHostedServiceExample to the WPF project.

2. If necessary, add the following assembly reference to the project:

System.Xml.Linq.dll

3. Add the following using statement to the top of the class file:

using System.Xml.Linq;

4. Add the following private members to the class:

XNamespace wa = "http://schemas.microsoft.com/windowsazure";
String createHostedServiceFormat =
    "https://management.core.windows.net/{0}/services/hostedservices";

5. Add the following method, creating a base-64 encoded string, to the class:

private String ConvertToBase64String(String value)
{
    Byte[] bytes = System.Text.Encoding.UTF8.GetBytes(value);
    String base64String = Convert.ToBase64String(bytes);
    return base64String;
} 

6. Add the following method, creating the payload, to the class:

private XDocument CreatePayload( String serviceName, String label,
    String description, String location, String affinityGroup)
{
    String base64LabelName = ConvertToBase64String(label); 

    XElement xServiceName = new XElement(wa + "ServiceName", serviceName);
    XElement xLabel = new XElement(wa + "Label", base64LabelName);
    XElement xDescription = new XElement(wa + "Description", description);
    XElement xLocation = new XElement(wa + "Location", location);
    XElement xAffinityGroup = new XElement(wa + "AffinityGroup", affinityGroup); 

    XElement createHostedService = new XElement(wa +"CreateHostedService");
    createHostedService.Add(xServiceName);
    createHostedService.Add(xLabel);
    createHostedService.Add(xDescription);
    createHostedService.Add(xLocation);
    //createHostedService.Add(xAffinityGroup); 

    XDocument payload = new XDocument();
    payload.Add(createHostedService);
    payload.Declaration = new XDeclaration("1.0", "UTF-8", "no"); 

    return payload;
}

7. Add the following method, invoking the create hosted service operation, to the class:

private String CreateHostedService(String subscriptionId, String thumbprint,
    String serviceName, String label, String description, String location, String affinityGroup)
{
    String uri = String.Format(createHostedServiceFormat, subscriptionId); 

    XDocument payload = CreatePayload(serviceName, label, description, location, affinityGroup); 

    ServiceManagementOperation operation = new ServiceManagementOperation(thumbprint); 

    String requestId = operation.Invoke(uri, payload); 

    return requestId;
}

8. Add the following method, invoking the methods added earlier, to the class:

public static void UseCreateHostedServiceExample()
{
    String subscriptionId = "{SUBSCRIPTION_ID}";
    String thumbprint = "{THUMBPRINT}";
    String serviceName = "{SERVICE_NAME}";
    String label = "{LABEL}";
    String description = "Newly created service";
    String location = "{LOCATION}";
    String affinityGroup = "{AFFINITY_GROUP}"; 

    CreateHostedServiceExample example = new CreateHostedServiceExample(); 

    String requestId = example.CreateHostedService(
        subscriptionId, thumbprint, serviceName, label,
        description, location, affinityGroup);
} 

How it works…

In steps 1 through 3, we set up the class. In step 4, we add private members to define the XML namespace used in creating the payload and the String format used in generating the endpoint for the create hosted service operation. In step 5, we add a helper method to create a base-64 encoded copy of a String.

We create the payload in step 6 by creating an XElement instance for each of the required and optional properties, as well as the root element. We add each of these elements to the root element and then add this to an XML document. Note that we do not add an AffinityGroup element because we provide a Location element and only one of them should be provided.

In step 7, we use the ServiceManagementOperation utility class, described in the Getting ready section, to invoke the create hosted service operation on the Service Management API. The Invoke() method creates an HttpWebRequest, adds the required X.509 certificate and the payload, and then sends the request to the create hosted services endpoint. It then parses the response to retrieve the request ID which can be used to check the status of the asynchronous create hosted services operation.

In step 8, we add a method that invokes the methods added earlier. We need to provide the subscription ID for the Windows Azure subscription, a globally unique service name for the hosted service, and a label used to identify the hosted service in the Windows Azure Portal. The location must be one of the official location names for a Windows Azure datacenter, such as North Central US. Alternatively, we can provide the GUID identifier of an existing affinity group and swap the commenting out in the code adding the Location and AffinityGroup elements in step 6. We see how to retrieve the list of locations and affinity groups in the Locations and affinity groups section of this recipe.

There’s more…

Each Windows Azure subscription can create 6 hosted services. This is a soft limit that can be raised by requesting a quota increase from Windows Azure Support at:

http://www.microsoft.com/windowsazure/support/

There are also soft limits on the number of cores per subscription (20) and the number of Windows Azure storage accounts per subscription (5). These limits can also be increased by request to Windows Azure Support.

Locations and affinity groups

The list of locations and affinity groups can be retrieved using the list locations and list affinity groups operations respectively in the Service Management API. We see how to do this in the Using the Windows Azure Platform PowerShell Cmdlets recipe in this chapter.

As of this writing, the locations are:

• Anywhere US
• South Central US
• North Central US
• Anywhere Europe
• North Europe
• West Europe
• Anywhere Asia
• Southeast Asia
• East Asia

The affinity groups are specific to a subscription.

The book is one of a series of cookbooks published by Packt. They are intended to provide “recipes” showing how to implement specific techniques in a particular technology. They are not intended to be narrative books read from cover-to-cover – although they certainly can be.

In the Microsoft Windows Azure Development Cookbook, I show how to perform various development tasks across the Windows Azure Platform – including the Window Azure Storage Service, Windows Azure hosted services, Windows Azure Diagnostics, and the Windows Azure Service Management REST API. I also provided some “recipes” for the Windows Azure AppFabric and SQL Azure, although these areas are sufficiently large as to deserve their own books rather than be reduced to chapters in a general book. Packt has provided a free download of the chapter on the Windows Azure Service Management REST API.

One of the problems in writing about a new technology is that the technology develops as the book is being written. This was particularly true of the Windows Azure Access Control Service. This is an interesting part of the Windows Azure Platform but is one that is not covered in the book – partly because the technology was changing and partly because of the difficulty of shoehorning information about it into the structure of the book.

As with any large project, the creation of a book takes many people. I am particularly indebted to the technical editors who provided a lot of comments that immeasurable improved the book:

Maarten Balliauw (@maartenballiauw)

Michael Collier (@MichaelCollier)

Gaurav Mantri (@gmantri)

Brent Stineman (@BrentCodeMonkey)

They, of course, bear no responsibility for the errors that are inevitable in any project like this. I am also grateful to the many people at Packt who worked on the book and helped ensure it came out in a timely fashion.

The chapters on the Windows Azure Storage Service would have been very difficult to do had I not used Cerebrata’s Cloud Storage Studio. This is an excellent product and I highly recommend it to anyone using the Windows Azure Storage Service.

The genesis of the book is clearly in this blog. However, the coverage of the various features is much more extensive in the book. So, if you like the posts on the blog you should look at the Microsoft Windows Azure Development Cookbook.

And now, on to all that new stuff the Windows Azure Team keeps announcing.