More on Azure Tables – Azure Storage Client v1.0

The Azure Storage Client v1.0 provides several classes to support Azure Table functionality:

CloudTableClient provides table management functionality such as create table, delete table and getting a list of tables. CloudTableQuery<TElement> represents an Azure Table query and provides support for retries. TableErrorCodeStrings provides strings describing various Azure Table errors. TableServiceContext adds Azure Table functionality to the ADO.Net Data Services class –  DataServiceContext. The TableServiceEntity class represents an entity in an Azure Table. The TableServiceExtensionMethods class provides an extension method that converts a TableServiceContext query into a CloudTableQuery<TElement> providing support for retries.

In the preceding post I looked at CloudTableClient. I will cover the CloudTableQuery<TElement>, TableServiceContext and TableServiceEntityclasses.

The Azure Storage Client is based on ADO.Net Data Services, once known as Astoria. The basic idea behind data access in the Storage Client is to use three classes: a model class representing the data; a context class providing access to the data; and a query class representing a query against the data.

TableServiceEntity

A model class represents an entity in the data so a new class must be created for each dataset. The Storage Client provides for a simple model class that is not much more than a set of public properties among which must be the PartitionKey and RowKey that are the primary key of the Azure Table. An example of a model class representing a table with two non-key properties, Singer and SongTitle, is:

[DataServiceKey("PartitionKey", "RowKey")]
public class Song
{
    public Song() {}

    public Song(String songTitle, String singer )
    {
        PartitionKey = “Songs”;
        RowKey = songTitle;
        Singer = singer;
        SongTitle = songTitle;
    }

    public String PartitionKey { get; set; }
    public String RowKey { get; set; }

    public String Singer { get; set; }
    public String SongTitle { get; set; }
}

The class must be annotated with DataServiceKey to indicate that PartitionKey and RowKeyform the primary key. The class must also have a default constructor taking no parameters.

The Storage Client provides a base class, TableServiceEntity, that simplifies the model class. TableServiceEntity is declared:

public abstract class TableServiceEntity {
    // Constructors
    protected TableServiceEntity();
    protected TableServiceEntity(String partitionKey, String rowKey);

    // Properties
    public virtual String PartitionKey { get; set; }
    public virtual String RowKey { get; set; }
    public DateTime Timestamp { get; set; }
}

The model class can be derived from TableServiceEntity. The example above becomes:

public class Song : TableServiceEntity
{
    public Song() {}

    public Song(String songTitle, String singer)
        : base( “Songs”, songTitle )
    {
        SongTitle = songTitle;
        Singer = singer;
    }

    public String SongTitle { get; set; }

    public String Singer { get; set; }
}

TableServiceContext

ADO.Net Data Services provides DataServiceContext to serve as a data access context class. DataServiceContext supports functionality such as AddObject() to add an entity, UpdateObject() to update an entity, SaveChanges() to save changes to an entity, and CreateQuery<T> to create a query against the dataset. The core functionality is supported in both synchronous and asynchronous versions. DataServiceContext also supports batch operations.

The Storage Client provides a TableServiceContext class that extends the functionality of DataServiceContext to support Azure Table authentication as well as providing for operations to be retried. The TableServiceContext class is declared:

public class TableServiceContext : DataServiceContext {
    // Constructors
    public TableServiceContext(String baseAddress, StorageCredentials credentials);

    // Properties
    public RetryPolicy RetryPolicy { get; set; }
    public StorageCredentials StorageCredentials { get; }

    // Methods
    public IAsyncResult BeginSaveChangesWithRetries(SaveChangesOptions options, AsyncCallback callback,
        Object state);
    public IAsyncResult BeginSaveChangesWithRetries(AsyncCallback callback, Object state);
    public DataServiceResponse EndSaveChangesWithRetries(IAsyncResult asyncResult);
    public DataServiceResponse SaveChangesWithRetries(SaveChangesOptions options);
    public DataServiceResponse SaveChangesWithRetries();
}

baseAddress is the Azure Table endpoint for the account such as “https://myaccount.table.core.windows.net/”.  The credentials are a StorageCredentialsAccountAndKey object containing the Azure Table account and key information.   RetryPolicy is a delegate with which a retry policy can be specified with the RetriesPolicies class providing some standard retry policies.

The TableServiceContext object is stateful and can store multiple changes which can then be applied in a single save operation.  There are both synchronous and asynchronous versions of save. The SaveChangesOptions parameter is an enumeration that qualifies the save: as a batch change in which the changes are sent in a single entity group; as having continue on error semantics; or replace on update semantics. The default behavior for a save is that it merges changes which means that a null value for a property is ignored – i.e. REST Merge Entity. Replace on update semantics is a true replace so that a null value for a property causes the property to be removed from the entity – i.e. REST Update Entity.

The following is an example of DataServiceContext being used to add a Song entity to an Azure Table named Songs.

protected void AddRecord(String tableName, String songTitle, String singer)
{
    CloudStorageAccount cloudStorageAccount =
         CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);

    Song song = new Song(songTitle, singer );

    TableServiceContext tableServiceContext = new
        TableServiceContext(cloudStorageAccount.TableEndpoint.ToString(), cloudStorageAccount.Credentials);

    tableServiceContext.AddObject( tableName, song);
    DataServiceResponse dataServiceResponse = tableServiceContext.SaveChangesWithRetries();
}

A TableServiceContext can also be retrieved from a CloudTableClient, as follows:

CloudStorageAccount cloudStorageAccount =
    CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
CloudTableClient cloudTableClient = cloudStorageAccount.CreateCloudTableClient();
TableServiceContext tableServiceContext = cloudTableClient.GetDataServiceContext();

Although TableServiceContext can be used like this, it is more usual to use a model-specific class derived from TableServiceContext. This derived class typically provides a method returning an IQueryable<T> that enumerates the entities in the table associated with the context. The following example shows a model-specific context class:

public class SongsContext : TableServiceContext
{
    private String tableName = “Songs”;

    public SongsContext(String baseAddress, StorageCredentials credentials)
        : base(baseAddress, credentials)
    {}

    public IQueryable<Song> Songs
    {
        get
        {
            return this.CreateQuery<Song>(tableName);
        }
    }

    public void AddSong(Song song)
    {
        this.AddObject(tableName, song);
        this.SaveChanges();
    }
}

As well as showing the IQueryable property this example also shows the use of a simple utility method to simplify adding an entity. Other actions can be handled similarly, if desired. The following demonstrated the use of SongsContext:

public void UseContext()
{
    CloudStorageAccount cloudStorageAccount =
          CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    SongsContext songsContext = new SongsContext(cloudStorageAccount.TableEndpoint.ToString(),
          cloudStorageAccount.Credentials);

    songsContext.AddSong(new Song(“Dry the Rain”, “Beta Band”));

    IQueryable<Song> songs = songsContext.Songs;
    foreach (Song song in songs)
    {
        String songTitle = song.SongTitle;
    }
}

The example adds a new song to the Songs table and then queries the table and retrieves the titles of all the songs in the table.

Azure Table handles concurrent updates or deletes through optimistic concurrency using an ETag value that is changed each time an entity is updated. The TableServiceContext stores the ETag of every entity it is tracking and submits this ETag in an If-Match request header when an update is requested. Azure Table rejects the update request if the submitted ETag does not match the current ETag for the entity. In the Storage Services REST API an update can be forced regardless of the current ETag by setting the ETag to the widcard character * in the If-Match request header.

It is not obvious how to use this special value in the Storage Client since the ETag  is not exposed directly in the API. The solution involves the trick of either preventing the context from entity tracking by setting DataServiceContext.MergeOption to the value MergeOption.NoTracking or cancelling tracking for the specific entity using DataServiceContext.Detach(). In either case, forced update is then specified by reattaching the entity to the context using DataServiceContext.AttachTo() with an ETag of *.

The following example, using MergeOption, retrieves a record from a table, updates it and does a forced update on the table:

protected void UpdateRecord(String tableName, String songTitle, String singer )
{
    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudTableClient cloudTableClient = cloudStorageAccount.CreateCloudTableClient();

    TableServiceContext tableServiceContext = cloudTableClient.GetDataServiceContext();

    tableServiceContext.MergeOption = MergeOption.NoTracking;

    Song theSong =
        (from song in tableServiceContext.CreateQuery<Song>(“Songs”)
        where song.Singer == singer && song.SongTitle == songTitle
        select song).FirstOrDefault<Song>();

    theSong.Singer = singer + ” won an award”;

    tableServiceContext.AttachTo(tableName, theSong, “*”);
    tableServiceContext.UpdateObject(theSong);
    DataServiceResponse dataServiceResponse =
        tableServiceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
}

Note that optimistic concurrency is also used for deletes so a similar technique is needed to force a delete whenever there is a possibility that an entity could have been updated while tracked by the current context.

UPDATE 2/10/2009

Note that version of ADO.Net Data Services currently used in Azure does not support server-side paging so that a DataServiceQuery, as used implicitly here, is not able to process the continuation tokens required to retrieve more than 1,000 entities. The next version does but is not yet released in the Azure environment. Consequently, you cannot rely on DataServiceQuery.Execute() to retrieve all the entities requested if there are more than 1,000 of them – or, indeed, if there is a need for continuation tokens which can happen on any query not including PartitionKey and RowKey.

CloudTableQuery<TElement>

The ADO.Net Data Service provides the DataServiceQuery<TElement> abstract class to “represent a single query request to a data service.” Instances of this class are created using the CreateQuery<TElement>() method of the DataServiceContext class. DataServiceQuery<TElement> is declared:

public class DataServiceQuery<TElement> : DataServiceQuery,
    IQueryable<TElement>, IEnumerable<TElement>, IQueryable, IEnumerable
{
    …
}

In the SongsContext sample given earlier, the IQueryable<Song> Songs property is implemented implicitly using DataServiceQuery<TElement> and DataServiceContext.CreateQuery<TElement().

Azure Storage provides an alternative query class, CloudTableQuery<TElement>, that implements additional functionality needed to support features unique to Azure Storage. CloudTableQuery is declared:

public class CloudTableQuery<TElement> : IQueryable<TElement>, IEnumerable<TElement>, IQueryable, IEnumerable {
    // Constructors
    public CloudTableQuery<TElement>(DataServiceQuery<TElement> query, RetryPolicy policy);
    public CloudTableQuery<TElement>(DataServiceQuery<TElement> query);

    // Properties
    public RetryPolicy RetryPolicy { get; set; }

    // Methods
    public IAsyncResult BeginExecuteSegmented(AsyncCallback callback, Object state);
    public ResultSegment<TElement> EndExecuteSegmented(IAsyncResult asyncResult);
    public IEnumerable<TElement> Execute();
    public CloudTableQuery<TElement> Expand(String path);

    // Implemented Interfaces and Overridden Members
    public IEnumerator<TElement> GetEnumerator();
    IEnumerator IEnumerable.GetEnumerator();
    public override String ToString();
    public Type ElementType { get; }
    public Expression Expression { get; }
    public IQueryProvider Provider { get; }
}

Many of these methods are the same as in DataServiceQuery<TElement>.  A CloudTableQuery<TElement> object can be created through a constructor or through the AsTableServiceQuery<TElement>() extension method in the TableServiceExtensionMethods class declared:

public static class TableServiceExtensionMethods {
    // Methods
    public static CloudTableQuery<TElement> AsTableServiceQuery<TElement>(IQueryable<TElement> query);
}

The BeginExecuteSegmented() and EndExecuteSegmented() methods can be used asynchronously to page through the entities in a table. EndExecuteSegmented() returns an object of the ResultSegment<TElement> class declared:

public class ResultSegment<TElement> {
    // Properties
    public Boolean HasMoreResults { get; }
    public IEnumerable<TElement> Results { get; }

    // Methods
    public IAsyncResult BeginGetNext(AsyncCallback callback, Object state);
    public ResultSegment<TElement> EndGetNext(IAsyncResult asyncResult);
}

BeginGetNext() and EndGetNext() can then be used to page through the remaining entities in the query.

The following is an example of the use of CloudTableQuery<TElement>:

protected void UseCloudTableQuery()
{
    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    TableServiceContext tableServiceContext = new TableServiceContext(
          cloudStorageAccount.TableEndpoint.ToString(), cloudStorageAccount.Credentials);

    CloudTableQuery<Song> cloudTableQuery = new CloudTableQuery<Song>(tableServiceContext.CreateQuery<Song>(“Songs”));

    foreach (Song s in cloudTableQuery)
    {
        String songTitle = s.SongTitle;
    }
}

UPDATE 1/27/2010:

Added information on optimistic concurrency in updates and deletes.

UPDATE 2/10/2010

Added information on DataServiceQuery.

I did a follow-up post, Queries in Azure Tables, which goes much deeper into querying.

About Neil Mackenzie

Azure Architect at Satory Global.
This entry was posted in Storage Service, Windows Azure. Bookmark the permalink.

7 Responses to More on Azure Tables – Azure Storage Client v1.0

  1. Alexander says:

    Really good article. Thanks

  2. sonam says:

    Really,Cleared my messy thoughts.Specially,I was confused that What does AddObject(entitySetName,Object)has entitySetName for?Then i got to know,that its actually the table name.Thanks.

  3. sonam says:

    Hey,I need to ask :suppose i have a class like this:public class X{ public string content{get;set;}}Can i do this:public class XEntity:TableServiceEntity{ public X xdata{get;set;}}My Main Aim is to differentiate the data from keys etc.After I get the entity from table storage,using just one line,I can get the data excluding keys or azure table storage specific things.r if you can think of any idea that can help me to do things like that.Note:All properties of X will statisfy the criteria for serialization and wll be allowed to store in Azure table storagethanks,Sonam

  4. Neil says:

    sonam -I did another post, Entities in Azure Tables, explaining the use of the ReadingEntity and WritingEntity events to modify the persistence behavior of the Storage Client – i.e. allow different behavior than the normal persist all simply-typed properties with a public getter and setter. That post links to some Microsoft blogs which show how to serialize and deserialize a Dictionary object containing all the data for an object.

  5. Pingback: Queries in Azure Tables | Convective

  6. Pingback: Entities in Azure Tables | Convective

  7. Pingback: Yet another application to handle Windows Azure Storage Services - Paolo Salvatori's Blog - Site Home - MSDN Blogs

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s