Entities in Azure Tables

This post follows on from earlier posts, here and here, on Azure Tables and focuses specifically on the WritingEntity and ReadingEntity events exposed in the DataServiceContext class. These events form extension points providing for the modification of the default behavior of the DataServiceContext class as it processes the data sent to and received from the Azure Storage Service.

Writing Data to an Azure Storage Table

The pattern for using the Storage Client library to access Azure Tables is to create a class derived from TableServiceEntity to contain the data and a class derived from DataServiceContext to handle interaction with Azure Tables.  The following is a simple example with a model class, City, exposing two properties- Country and Name:

[DataServiceKey("PartitionKey", "RowKey")]
public class City : TableServiceEntity
{
    public City() {};

    public City(String country, String name)
        : base( “city”, Guid.NewGuid().ToString())
    {
        Country = country;
        Name = name;
    }

    public String Country { get; set; }
    public String Name { get; set; }
}

public class CityContext : TableServiceContext
{
    private String tableName = “city”;

    public CityContext(
         String baseAddress, StorageCredentials credentials)
        : base(baseAddress, credentials)
        {}

       public IQueryable<City> Cities
      {
        get
        {
            return this.CreateQuery<City>(tableName);
        }
    }

    public void AddCity(City city)
    {
        this.AddObject(tableName, city);
        DataServiceResponse dataServiceResponse =  
            SaveChanges();

    }
}

The SaveChanges() method creates an Atom Feed entry from the city object and POSTs it to the appropriate http endpoint – http://myaccount.table.core.windows.net/city.

An example of the Atom entry generated for a request from these classes is:

<?xml version=”1.0″ encoding=”utf-8″ 
       standalone=”yes”?>
<entry
xmlns:d=”http://schemas.microsoft.com/ado/2007/08/dataservices&#8221;
xmlns:m=”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata&#8221;
xmlns=”http://www.w3.org/2005/Atom”&gt;
<title />
  <updated>2009-12-22T22:43:44.9968515Z
 </updated>
  <author>
    <name />
  </author>
  <id />
  <content type=”application/xml”>
    <m:properties>
      <d:Country>USA</d:Country>
      <d:Name>Santa Barbara</d:Name>
      <d:PartitionKey>city</d:PartitionKey>
      <d:RowKey>
          acc737ae-f146-4ce7-9e69-d351ddd76425
      </d:RowKey>
      <d:Timestamp m:type=”Edm.DateTime”>
          0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>

The Atom Feed entry for the corresponding response is:

<?xml version=”1.0″ encoding=”utf-8″ standalone=”yes”?>
<entry>
xml:base=”http://myaccount.table.core.windows.net/&#8221;
xmlns:d=”http://schemas.microsoft.com/ado/2007/08/dataservices&#8221;
xmlns:m=”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata&#8221;
     m:etag=”W/&quot;datetime’2009-12-22T22%3A43%
     3A23.7757524Z’&quot;”
xmlns=”http://www.w3.org/2005/Atom”&gt;
<id>http://myaccount.table.core.windows.net/city(PartitionKey=’city&#8217;,
 RowKey=’acc737ae-f146-4ce7-9e69-d351ddd76425′)</id>
<title type=”text”></title>
 <updated>2009-12-22T22:43:23Z</updated>
     <author>
         <name />
      </author>
      <link rel=”edit” title=”city” href=”city(PartitionKey=’city’,RowKey=’acc737ae-f146-4ce7-9e69-d351ddd76425′)” />
     <category term=”myaccount.city” scheme=”http://schemas.microsoft.com/ado/2007/08/dataservices/scheme&#8221; />
      <content type=”application/xml”>
         <m:properties>
             <d:PartitionKey>city</d:PartitionKey>
             <d:RowKey>acc737ae-f146-4ce7-9e69-d351ddd76425</d:RowKey>
             <d:Timestamp m:type=”Edm.DateTime”>2009-12-22T22:43:23.7757524Z</d:Timestamp>
             <d:Country>USA</d:Country>
             <d:Name>Santa Barbara</d:Name>
         </m:properties>
     </content>
</entry>

The Atom entry contains a child of the properties element for both the Country and Name properties of the model class, City, as well as child elements for the PartitionKey, RowKey and Timestamp properties of the base class, TableServiceEntity. Note that the default Timestamp sent with the request has been supplied with a valid value in the response. The Id in the resource Atom entry is the URI of the Atom Table resource for this entity. The category element in the response contains specifies the account name and table name as myaccount.city.

The request generated in this example conforms to the default behavior of the TableServiceContext class which uses only the public properties of the associated model class. A corollary to this is that a property can be hidden from the default process and not transferred to/from Azure Storage by declaring it internal, protected or private. There are times when it might be useful to modify the default behavior and that is precisely the functionality exposed by the WritingEntity and ReadingEntity events. The WritingEntity event is raised when the DataServiceContext.SaveChanges() is invoked to convert a model class object into an Atom entry while ReadingEntity event is raised when the results of a query are enumerated over and the Atom entry is converted into a model class object.

WritingEntity

The WritingEntity event is raised by the SaveChanges() method of the DataServiceContext class. WritingEntity is declared:

public event EventHandler<ReadingWritingEntityEventArgs> WritingEntity

ReadingWritingEntityEventArgs has two public properties declared:

public XElement Data { get; }
public Object Entity { get; }

The Entity property contains the TableServiceEntity object to be persisted to an Azure Table while the Data property exposes the default Atom entry to be sent to the Azure Storage Service. XElement represents an XML element and, as part of the LINQ framework, supports modifying and searching inside the element. Consequently, the Atom entry that will be sent to the Azure Storage Service can be manipulated through the members and properties of the XElement Data object.

The WritingEntity event is handled by associating an event handler method with it. This can be done in the constructor as in the following example:

public CityContext(String baseAddress, StorageCredentials credentials)
    : base(baseAddress, credentials)
{
    WritingEntity += new 
         EventHandler<ReadingWritingEntityEventArgs>
          (AddCombinedName);
}

where AddCombinedName is a method declared:

void AddCombinedName(Object sender,
    ReadingWritingEntityEventArgs args);

Adding a Property

A simple use of the WritingEntity event is adding a property to the Atom entry that is not present in the TableServiceEntity. For example, combining the city Name and Country into a single property, CombinedName, and persisting that in a table in Azure Storage. The following event handler for WritingEntity does precisely this by adding a new property to the Atom entry in Data:

private void AddCombinedName(Object sender,
     ReadingWritingEntityEventArgs args)
{
    XNamespace d =
  “http://schemas.microsoft.com/ado/2007/08/dataservices&#8221;;
    XNamespace m = http://schemas.microsoft.com/ado/2007/08/dataservices/metadata;
City city = args.Entity as City;
if (city != null)
{
   String combinedName =
        String.Format(“{0}, {1}”, city.Name, city.Country );
   XElement xElement = new XElement(
        d + “CombinedName”, combinedName );
   XElement properties = args.Data.Descendants(
        m + “properties”).First();
    properties.Add(xElement);
 }
}

The example creates an XNamespace object for each of the two namespaces used in the Atom entry. The input args.Entity object is converted into a City object. A new XElement object is created with the combined names. Note that the first parameter of the XElement constructor is an XName object created using the XNamespace addition operator to combine the namespace with the property name. Then an XElement object, properties is assigned to the properties element in the Atom entry.  Finally, the newly created XElement is added to the properties element. This results in the following Atom entry being sent to the Azure Storage Service:

<?xml version=”1.0″ encoding=”utf-8″?>
<entry
      xmlns:d=http://schemas.microsoft.com/ado/2007/08/dataservices
      xmlns:m=http://schemas.microsoft.com/ado/2007/08/dataservices/metadata
      xmlns=”http://www.w3.org/2005/Atom”&gt;
  <title />
  <updated>2009-12-22T08:50:09.4394531Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type=”application/xml”>
    <m:properties>
      <d:Country>USA</d:Country>
      <d:Name>Chicago</d:Name>
      <d:PartitionKey>city</d:PartitionKey>
      <d:RowKey>db0568ef-0fda-439f-98c3-4a7511fd0dd3</d:RowKey>
      <d:Timestamp m:type=”Edm.DateTime”>
            0001-01-01T00:00:00</d:Timestamp>
      <d:CombinedName>Chicago, USA</d:CombinedName>
    </m:properties>
  </content>
</entry>

All the properties in this Atom entry are inserted as properties of a new entity in the city table including the new property, CombinedName.

Removing a Property

The WritingEntity event can also be used to remove a property from the Atom entry to be POSTed to the Azure Storage Service. This can be useful if the semantics for the model class indicate that a property should be public but that it should not be persisted in the corresponding Azure Storage table.The following event handler for WritingEntity does precisely this by removing a property from the Atom entry in Data:

private void RemoveCityName(Object sender,
    ReadingWritingEntityEventArgs args)
{
    XNamespace d = http://schemas.microsoft.com/ado/2007/08/dataservices;
    XElement cityNameElement =
         args.Data.Descendants(d + “Name”).First();
    cityNameElement.Remove();
}

This example identifies the element corresponding to the Name property to be removed, creates an XElement from it and then removes this XElement from its parent node in the Data object using the inherited XNode.Remove(). This results in the following Atom entry being sent to the Azure Storage Service:

<?xml version=”1.0″ encoding=”utf-8″?>
<entry
     xmlns:d=”http://schemas.microsoft.com/ado/2007/08/dataservices&#8221;
     xmlns:m=”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata&#8221;
     xmlns=”http://www.w3.org/2005/Atom”&gt;
  <title />
  <updated>2009-12-22T09:17:33.7832031Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type=”application/xml”>
    <m:properties>
      <d:Country>USA</d:Country>
      <d:PartitionKey>city</d:PartitionKey>
      <d:RowKey>b1accc5b-656a-4336-9674-df6004e084ce</d:RowKey>
      <d:Timestamp m:type=”Edm.DateTime”>
           0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>

Note there is no City child element of the properties element.

Custom Attribute

The default serialization behavior is that all public properties of a model class are serialized to an Azure Table. The WritingEntity event can be used to modify this behavior. However, so far all the functionality specifying the properties to be removed from the serialized property set has been hard-coded in the handler for the WritingEntity event.

On an ADO.Net Data Services thread on the MSDN forums, Ben Morris demonstrated how to define a custom attribute to mark a property as not serializable and then using this attribute in the WritingEntity handler to prevent serialization of the property. Phani Raj, of the ADO.Net Data Services team, expanded on the original post and provided an in-depth explanation of the technique. The idea is a fairly trivial extension of the above so rather than rewrite their code and posts I suggest you read the posts by Ben Morris and Phani Raj.

This technique can be used to decorate a public property with a custom attribute that indicates the property should not be serialized to an Azure Table. Consequently, it provides an additional technique ensuring this other than simply making the property internal, protected or private.

Reading Data from an Azure Storage Table

The ReadingEntity event is raised when a query is enumerated on the DataServiceContext. ReadingEntity is declared:

public event EventHandler<ReadingWritingEntityEventArgs> ReadingEntity

 DataServiceContext.SaveChanges() is invoked to convert a model class object into an Atom entry while ReadingEntity event is raised when the results of a query are enumerated over and the Atom entry is converted into a model class object. The ReadingWritingEntityEventArgs type is the same as that used with the WritingEntity event. In this case the input and output of interest are Data and Entity respectively.

The ReadingEntity event is handled by associating an event handler method with it. This can be done in the constructor as in the following example:

public CityContext(String baseAddress,
    StorageCredentials credentials)
    : base(baseAddress, credentials)
{
    ReadingEntity += new
         EventHandler<ReadingWritingEntityEventArgs>
         (AddCombinedNameToCity);
}

where AddCombinedName is a method declared:

void AddCombinedName(Object sender,
    ReadingWritingEntityEventArgs args);

Initializing a Property

A simple use of the ReadingEntity event is initializing a property of the model that is not present in the Azure Table. For example, combining the city Name and Country into a single property, CombinedName, and initializing a new property in a City object. (This is admittedly a somewhat contrived example.) The following event handler for ReadingEntity does precisely this by initializing the CombinedName property of the City object.:

private void AddCombinedNameToCity(Object sender, ReadingWritingEntityEventArgs args)
{
    XNamespace d = “http://schemas.microsoft.com/ado/2007/08/dataservices&#8221;;
    XElement countryElement =
        args.Data.Descendants(d + “Country”).First();
    XElement nameElement =
        args.Data.Descendants(d + “Name”).First();
    City city = args.Entity as City;
    if (city != null)
    {
        city.CombinedName = nameElement.Value + ” – ” +
            countryElement.Value;
    }
}

The example creates an XNamespace object for the namespace needed in the Atom entry. The input args.Entity object is converted into a City object. The City.CombinedName property is now initialized to a combination of the values of the Country and Name elements in the Atom entry. This property is accessible when the City object is exposed when enumerated over as the result of a query.

The following example shows a query that accesses the CombinedName initialized by the above:

protected void QueryCity()
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting
           (“DataConnectionString”);
     CloudTableClient cloudTableClient =
           cloudStorageAccount.CreateCloudTableClient();
     CityContext cityContext = new CityContext
          (cloudStorageAccount.TableEndpoint.ToString(),
           cloudStorageAccount.Credentials);
      IQueryable<City> cities =
           from city in cityContext.CreateQuery<City>(“city”)
           select city;
      foreach (City city in cities)
     {
        String combinedName = city.CombinedName;
     }
}

This method uses a CloudStorageAccount object to retrieve the required endpoint and authentication information from the Azure configuration file. It assumes that CloudStorageAccount.SetConfigurationSettingPublisher() has been invoked previously. A context is then created using the endpoint and authentication credentials. The context is used to invoke a query retrieving all records in the Azure Table named city. The context raises the ReadingEntity event during the enumeration on the foreach statement. With the code provided earlier this event is handled by the AddCombinedNameToCity() handler.

Serializing Generic Properties

Jai Haridas, of the Azure Storage team, has an interesting comment on an Azure thread on the MSDN Forums showing how to use a model class with a property of type Dictionary<String, Object> and specifically how to use a ReadingEntity handler to deserialize the properties of an Azure Table into entries of the Dictionary.  Yi-Lun Luo, of the Azure Team, completes the example by showing on another Azure thread on the MSDN Forums the use of a WritingEntity handler to write the Dictionary into an Atom entry that is then serialized in an Azure Table. Particular care must be taken with data types used in the Dictionary since Azure Tables support only support a limited set of data types. Note that Haridas credits Pablo Castro of the Ado.Net Data Services Team for the implementation of the ReadingEntity handler.

This is an interesting technique because it provides high-level .Net access to heterogeneous entities in a schema-less Azure Table.

Technorati Tags: ,

About Neil Mackenzie

Cloud Solutions Architect. Microsoft
This entry was posted in Storage Service, Windows Azure. Bookmark the permalink.

3 Responses to Entities in Azure Tables

  1. John Woakes says:

    This looks like it is code that works at the client side when making requests to read/write table data. Is there a way to hook into the read/write process at the Azure table end?

    I want to be able to munge all data coming and going into a table via the Azure REST API.

    Thanks.

  2. Pingback: Yet another application to handle Windows Azure Storage Services - Paolo Salvatori's Blog - Site Home - MSDN Blogs

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s