Examples of the Windows Azure Storage Services REST API

September 8, 2013 – The examples in this post were updated to work with the current version of the Windows Azure Storage REST API.

In the Windows Azure MSDN Azure Forum there are occasional questions about the Windows Azure Storage Services REST API. I have occasionally responded to these with some code examples showing how to use the API. I thought it would be useful to provide some examples of using the REST API for tables, blobs and queues – if only so I don’t have to dredge up examples when people ask how to use it. This post is not intended to provide a complete description of the REST API.

The REST API is comprehensively documented (other than the lack of working examples). Since the REST API is the definitive way to address Windows Azure Storage Services I think people using the higher level Storage Client API should have a passing understanding of the REST API to the level of being able to understand the documentation. Understanding the REST API can provide a deeper understanding of why the Storage Client API behaves the way it does.

Fiddler

The Fiddler Web Debugging Proxy is an essential tool when developing using the REST (or Storage Client) API since it captures precisely what is sent over the wire to the Windows Azure Storage Services.

Authorization

Nearly every request to the Windows Azure Storage Services must be authenticated. The exception is access to blobs with public read access. The supported authentication schemes for blobs, queues and tables and these are described here. The requests must be accompanied by an Authorization header constructed by making a hash-based message authentication code using the SHA-256 hash.

The following is an example of performing the SHA-256 hash for the Authorization header:


public static String CreateAuthorizationHeader(String canonicalizedString) {
String signature = String.Empty;
using (HMACSHA256 hmacSha256 = new HMACSHA256( Convert.FromBase64String(storageAccountKey) )) {
Byte[] dataToHmac = System.Text.Encoding.UTF8.GetBytes(canonicalizedString);
signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac));
}
String authorizationHeader = String.Format(
CultureInfo.InvariantCulture,
"{0} {1}:{2}",
AzureStorageConstants.SharedKeyAuthorizationScheme,
AzureStorageConstants.Account,
signature
);
return authorizationHeader;
}

This method is used in all the examples in this post.

AzureStorageConstants is a helper class containing various constants. Key is a secret key for Windows Azure Storage Services account specified by Account. In the examples given here, SharedKeyAuthorizationScheme is SharedKey.

The trickiest part in using the REST API successfully is getting the correct string to sign. Fortunately, in the event of an authentication failure the Blob Service and Queue Service responds with the authorization string they used and this can be compared with the authorization string used in generating the Authorization header. This has greatly simplified the us of the REST API.

Table Service API

The Table Service API supports the following table-level operations:

The Table Service API supports the following entity-level operations:

These operations are implemented using the appropriate HTTP VERB:

  • DELETE – delete
  • GET – query
  • MERGE – merge
  • POST – insert
  • PUT – update

This section provides examples of the Insert Entity and Query Entities operations.

Insert Entity

The InsertEntity() method listed in this section inserts an entity with two String properties, Artist and Title, into a table. The entity is submitted as an ATOM entry in the body of a request POSTed to the Table Service. In this example, the ATOM entry is generated by the GetRequestContentInsertXml() method. The date must be in RFC 1123 format in the x-ms-date header supplied to the canonicalized resource used to create the Authorization string. Note that the storage service version is set to “2012-02-12” which requires the DataServiceVersion and MaxDataServiceVersion to be set appropriately.


public void InsertEntity(String tableName, String artist, String title)
{
String requestMethod = "POST";
String urlPath = tableName;
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String contentMD5 = String.Empty;
String contentType = "application/atom+xml";
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath);
String stringToSign = String.Format(
"{0}\n{1}\n{2}\n{3}\n{4}",
requestMethod,
contentMD5,
contentType,
dateInRfc1123Format,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
UTF8Encoding utf8Encoding = new UTF8Encoding();
Byte[] content = utf8Encoding.GetBytes(GetRequestContentInsertXml(artist, title));
Uri uri = new Uri(AzureStorageConstants.TableEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Accept = "application/atom+xml,application/xml";
request.ContentLength = content.Length;
request.ContentType = contentType;
request.Method = requestMethod;
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.Headers.Add("Accept-Charset", "UTF-8");
request.Headers.Add("DataServiceVersion", "2.0;NetFx");
request.Headers.Add("MaxDataServiceVersion", "2.0;NetFx");
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(content, 0, content.Length);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream dataStream = response.GetResponseStream();
using (StreamReader reader = new StreamReader(dataStream))
{
String responseFromServer = reader.ReadToEnd();
}
}
}
private String GetRequestContentInsertXml(String artist, String title)
{
String defaultNameSpace = "http://www.w3.org/2005/Atom";
String dataservicesNameSpace = "http://schemas.microsoft.com/ado/2007/08/dataservices";
String metadataNameSpace = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";
XmlWriterSettings xmlWriterSettings = new XmlWriterSettings();
xmlWriterSettings.OmitXmlDeclaration = false;
xmlWriterSettings.Encoding = Encoding.UTF8;
StringBuilder entry = new StringBuilder();
using (XmlWriter xmlWriter = XmlWriter.Create(entry))
{
xmlWriter.WriteProcessingInstruction("xml", "version=\"1.0\" encoding=\"UTF-8\"");
xmlWriter.WriteWhitespace("\n");
xmlWriter.WriteStartElement("entry", defaultNameSpace);
xmlWriter.WriteAttributeString("xmlns", "d", null, dataservicesNameSpace);
xmlWriter.WriteAttributeString("xmlns", "m", null, metadataNameSpace);
xmlWriter.WriteElementString("title", null);
xmlWriter.WriteElementString("updated", String.Format("{0:o}", DateTime.UtcNow));
xmlWriter.WriteStartElement("author");
xmlWriter.WriteElementString("name", null);
xmlWriter.WriteEndElement();
xmlWriter.WriteElementString("id", null);
xmlWriter.WriteStartElement("content");
xmlWriter.WriteAttributeString("type", "application/xml");
xmlWriter.WriteStartElement("properties", metadataNameSpace);
xmlWriter.WriteElementString("PartitionKey", dataservicesNameSpace, artist);
xmlWriter.WriteElementString("RowKey", dataservicesNameSpace, title);
xmlWriter.WriteElementString("Artist", dataservicesNameSpace, artist);
xmlWriter.WriteElementString("Title", dataservicesNameSpace, title + "\n" + title);
xmlWriter.WriteEndElement();
xmlWriter.WriteEndElement();
xmlWriter.WriteEndElement();
xmlWriter.Close();
}
String requestContent = entry.ToString();
return requestContent;
}

view raw

InsertEntity

hosted with ❤ by GitHub

This generates the following request (as captured by Fiddler):

POST https://STORAGE_ACCOUNT.table.core.windows.net/authors HTTP/1.1
Accept: application/atom+xml,application/xml
Content-Type: application/atom+xml
x-ms-date: Sun, 08 Sep 2013 06:31:12 GMT
x-ms-version: 2012-02-12
Authorization: SharedKey STORAGE_ACCOUNT:w7Uu4wHZx4fFwa2bsxd/TJVZZ1AqMPwxvW+pYtoWHd0=
Accept-Charset: UTF-8
DataServiceVersion: 2.0;NetFx
MaxDataServiceVersion: 2.0;NetFx
Host: STORAGE_ACCOUNT.table.core.windows.net
Content-Length: 514
Expect: 100-continue
Connection: Keep-Alive

The body of the request is:

<?xml version="1.0" encoding="UTF-8"?>
 <entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom"><title /><updated>2013-09-08T06:31:13.0503771Z</updated><author><name /></author><id /><content type="application/xml"><m:properties><d:PartitionKey>Beckett</d:PartitionKey><d:RowKey>Molloy</d:RowKey><d:Artist>Beckett</d:Artist>
 <d:Title>Molloy
 Molloy</d:Title></m:properties></content></entry>

The Table Service generates the following response:

HTTP/1.1 201 Created
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
ETag: W/"datetime'2013-09-08T07%3A19%3A07.2189243Z'"
Location: https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy')
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 3818433a-4d89-4344-bcf1-ec248cf24d97
x-ms-version: 2012-02-12
Date: Sun, 08 Sep 2013 07:19:07 GMT
Content-Length: 1108

The Table Service generates the following response body:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="https://STORAGE_ACCOUNT.table.core.windows.net/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:etag="W/&quot;datetime'2013-09-08T07%3A19%3A07.2189243Z'&quot;" xmlns="http://www.w3.org/2005/Atom">
  <id>https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy')</id>
  <title type="text"></title>
  <updated>2013-09-08T07:19:07Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="authors" href="authors(PartitionKey='Beckett',RowKey='Molloy')" />
  <category term="STORAGE_ACCOUNT.authors" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>Beckett</d:PartitionKey>
      <d:RowKey>Molloy</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2013-09-08T07:19:07.2189243Z</d:Timestamp>
      <d:Artist>Beckett</d:Artist>
      <d:Title>Molloy
Molloy</d:Title>
    </m:properties>
  </content>
</entry>

Note that I should have URLEncoded the PartitionKey and RowKey but did not do so for simplicity. There are, in fact, some issues with the URL encoding of spaces and other symbols.

Get Entity

The GetEntity() method described in this section retrieves the single entity inserted in the previous section. The particular entity to be retrieved is identified directly in the URL.


public void GetEntity(String tableName, String partitionKey, String rowKey)
{
String requestMethod = "GET";
String urlPath = String.Format("{0}(PartitionKey='{1}',RowKey='{2}')", tableName, partitionKey, rowKey);
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath);
String stringToSign = String.Format(
"{0}\n\n\n{1}\n{2}",
requestMethod,
dateInRfc1123Format,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
Uri uri = new Uri(AzureStorageConstants.TableEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.Headers.Add("Accept-Charset", "UTF-8");
request.Accept = "application/atom+xml,application/xml";
request.Headers.Add("DataServiceVersion", "2.0;NetFx");
request.Headers.Add("MaxDataServiceVersion", "2.0;NetFx");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream dataStream = response.GetResponseStream();
using (StreamReader reader = new StreamReader(dataStream))
{
String responseFromServer = reader.ReadToEnd();
}
}
}

view raw

GetEntity

hosted with ❤ by GitHub

This generates the following request (as captured by Fiddler):

GET https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy') HTTP/1.1
x-ms-date: Sun, 08 Sep 2013 06:31:14 GMT
x-ms-version: 2012-02-12
Authorization: SharedKey STORAGE_ACCOUNT:1hWbr4aNq4JWCpNJY3rsLH1SkIyeFTJflbqyKMPQ1Gk=
Accept-Charset: UTF-8
Accept: application/atom+xml,application/xml
DataServiceVersion: 2.0;NetFx
MaxDataServiceVersion: 2.0;NetFx
Host: STORAGE_ACCOUNT.table.core.windows.net

The Table Service generates the following response:

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
ETag: W/"datetime'2013-09-08T06%3A31%3A14.1579056Z'"
Server: Windows-Azure-Table/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: f4bd4c77-6fb6-42a8-8dff-81ea8d28fa2e
x-ms-version: 2012-02-12
Date: Sun, 08 Sep 2013 06:31:15 GMT
Content-Length: 1108

The returned entities, in this case a single entity, are returned in ATOM entry format in the response body:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="https://STORAGE_ACCOUNT.table.core.windows.net/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:etag="W/&quot;datetime'2013-09-08T06%3A31%3A14.1579056Z'&quot;" xmlns="http://www.w3.org/2005/Atom">
  <id>https://STORAGE_ACCOUNT.table.core.windows.net/authors(PartitionKey='Beckett',RowKey='Molloy')</id>
  <title type="text"></title>
  <updated>2013-09-08T06:31:15Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="authors" href="authors(PartitionKey='Beckett',RowKey='Molloy')" />
  <category term="STORAGE_ACCOUNT.authors" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>Beckett</d:PartitionKey>
      <d:RowKey>Molloy</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2013-09-08T06:31:14.1579056Z</d:Timestamp>
      <d:Artist>Beckett</d:Artist>
      <d:Title>Molloy
Molloy</d:Title>
    </m:properties>
  </content>
</entry>

Blob Service API

The Blob Service API supports the following account-level operation:

The Blob Service API supports the following container-level operation:

The Blob Service API supports the following blob-level operation:

The Blob Service API supports the following operations on block blobs:

The Blob Service API supports the following operations on page blobs:

This section provides examples of the Put Blob and Lease Blob operations.

Put Blob

The Blob Service and Queue Service use a different form of shared-key authentication from the Table Service so care should be taken in creating the string to be signed for authorization. The blob type, BlockBlob or PageBlob, must be specified as a request header and consequently appears in the authorization string.


public void PutBlob(String containerName, String blobName)
{
String requestMethod = "PUT";
String urlPath = String.Format("{0}/{1}", containerName, blobName);
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String content = "Andrew Carnegie was born in Dunfermline";
UTF8Encoding utf8Encoding = new UTF8Encoding();
Byte[] blobContent = utf8Encoding.GetBytes(content);
Int32 blobLength = blobContent.Length;
const String blobType = "BlockBlob";
String canonicalizedHeaders = String.Format(
"x-ms-blob-type:{0}\nx-ms-date:{1}\nx-ms-version:{2}",
blobType,
dateInRfc1123Format,
storageServiceVersion);
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath);
String stringToSign = String.Format(
"{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}",
requestMethod,
blobLength,
canonicalizedHeaders,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
Uri uri = new Uri(AzureStorageConstants.BlobEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Headers.Add("x-ms-blob-type", blobType);
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.ContentLength = blobLength;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(blobContent, 0, blobLength);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
String ETag = response.Headers["ETag"];
}
}

view raw

PutBlob

hosted with ❤ by GitHub

This generates the following request:

PUT https://STORAGE_ACCOUNT.blob.core.windows.net/fife/dunfermline HTTP/1.1
x-ms-blob-type: BlockBlob
x-ms-date: Sun, 08 Sep 2013 06:28:29 GMT
x-ms-version: 2012-02-12
Authorization: SharedKey STORAGE_ACCOUNT:ntvh/lamVmikvwHhy6vRVBIh87kibkPlEOiHyLDia6g=
Host: STORAGE_ACCOUNT.blob.core.windows.net
Content-Length: 39
Expect: 100-continue
Connection: Keep-Alive

The body of the request is:

Andrew Carnegie was born in Dunfermline

The Blob Service generates the following response:

HTTP/1.1 201 Created
Transfer-Encoding: chunked
Content-MD5: RYJnWGXLyt94l5jG82LjBw==
Last-Modified: Sun, 08 Sep 2013 06:28:31 GMT
ETag: "0x8D07A73C5704A86"
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: b74ef0a2-294d-4581-b8f1-6cda724bbdbf
x-ms-version: 2012-02-12
Date: Sun, 08 Sep 2013 06:28:30 GMT

Lease Blob

The Blob Service allows a user to lease a blob for a minute at a time and so acquire a write lock on it. The use case for this is the locking of a page blob used to store the VHD backing an writeable Azure Drive.

The LeaseBlob() example in this section demonstrates a subtle issue with the creation of authorization strings. The URL has a query string, comp=lease. Rather than using this directly in creating the authorization string it must be converted into comp:lease with a colon replacing the equal symbol – see modifiedURL in the example. Furthermore, the Lease Blob operation requires the use of an x-ms-lease-action to indicate whether the lease is being acquired, renewed, released or broken.


public void LeaseBlob(String containerName, String blobName)
{
String requestMethod = "PUT";
String urlPath = String.Format("{0}/{1}?comp=lease", containerName, blobName);
String modifiedUrlPath = String.Format("{0}/{1}\ncomp:lease", containerName, blobName);
const Int32 contentLength = 0;
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String leaseAction = "acquire";
String leaseDuration = "60";
String canonicalizedHeaders = String.Format(
"x-ms-date:{0}\nx-ms-lease-action:{1}\nx-ms-lease-duration:{2}\nx-ms-version:{3}",
dateInRfc1123Format,
leaseAction,
leaseDuration,
storageServiceVersion);
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, modifiedUrlPath);
String stringToSign = String.Format(
"{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}",
requestMethod,
contentLength,
canonicalizedHeaders,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
Uri uri = new Uri(AzureStorageConstants.BlobEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-lease-action", leaseAction);
request.Headers.Add("x-ms-lease-duration", leaseDuration);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.ContentLength = contentLength;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
String leaseId = response.Headers["x-ms-lease-id"];
}
}

view raw

LeaseBlob

hosted with ❤ by GitHub

This generates the following request:

PUT https://STORAGE_ACCOUNT.blob.core.windows.net/fife/dunfermline?comp=lease HTTP/1.1
x-ms-date: Sun, 08 Sep 2013 06:28:31 GMT
x-ms-lease-action: acquire
x-ms-lease-duration: 60
x-ms-version: 2012-02-12
Authorization: SharedKey rebus:+SQ5+RFZg3hUaws5XCRHxsDgXb1ycdRIz5EKyHJWP7s=
Host: rebus.blob.core.windows.net
Content-Length: 0

The Blob Service generates the following response:

HTTP/1.1 201 Created
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 4b6ff77f-f885-4f74-803a-c92920d225c3
x-ms-version: 2012-02-12
x-ms-lease-id: b1320c2c-65ad-41d6-a7bd-85a4242c0ac5
Date: Sun, 08 Sep 2013 06:28:31 GMT
Content-Length: 0

Queue Service API

The Queue Service API supports the following queue-level operation:

The Queue Service API supports the following queue-level operation:

The Queue Service API supports the following message-level operations:

This section provides examples of the Put Message and Get Message operations.

Put Message

The most obvious curiosity about Put Message is that it uses the HTTP verb POST rather than PUT. The issue is presumably the interaction of the English language and the HTTP standard which states that PUT should be idempotent and that the Put Message operation is clearly not since each invocation merely adds another message to the queue. Regardless, it did catch me out when I failed to read the documentation well enough – so take that as a warning.

The content of a message posted to the queue must be formatted in a specified XML schema and must then be UTF8 encoded.


public void PutMessage(String queueName, String message)
{
String requestMethod = "POST";
String urlPath = String.Format("{0}/messages", queueName);
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String messageText = String.Format(
"<QueueMessage><MessageText>{0}</MessageText></QueueMessage>", message);
UTF8Encoding utf8Encoding = new UTF8Encoding();
Byte[] messageContent = utf8Encoding.GetBytes(messageText);
Int32 messageLength = messageContent.Length;
String canonicalizedHeaders = String.Format(
"x-ms-date:{0}\nx-ms-version:{1}",
dateInRfc1123Format,
storageServiceVersion);
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath);
String stringToSign = String.Format(
"{0}\n\n\n{1}\n\n\n\n\n\n\n\n\n{2}\n{3}",
requestMethod,
messageLength,
canonicalizedHeaders,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
Uri uri = new Uri(AzureStorageConstants.QueueEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.ContentLength = messageLength;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(messageContent, 0, messageLength);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
String requestId = response.Headers["x-ms-request-id"];
}
}

view raw

PostMessage

hosted with ❤ by GitHub

This generates the following request:

POST https://rebus.queue.core.windows.net/revolution/messages HTTP/1.1
x-ms-date: Sun, 08 Sep 2013 06:34:08 GMT
x-ms-version: 2012-02-12
Authorization: SharedKey rebus:nyASTVWifnxHKnj2wXwuzzzXz5CxUBZj58SToV5QFK8=
Host: rebus.queue.core.windows.net
Content-Length: 76
Expect: 100-continue
Connection: Keep-Alive

The body of the request is:

<QueueMessage><MessageText>Saturday in the cafe</MessageText></QueueMessage>

The Queue Service generates the following response:

HTTP/1.1 201 Created
Server: Windows-Azure-Queue/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 14c6e73b-15d9-480c-b251-c4c01b48e529
x-ms-version: 2012-02-12
Date: Sun, 08 Sep 2013 06:34:09 GMT
Content-Length: 0

Get Messages

The Get Messages operation described in this section retrieves a single message with the default message visibility timeout of 30 seconds.


public void GetMessage(String queueName)
{
string requestMethod = "GET";
String urlPath = String.Format("{0}/messages", queueName);
String storageServiceVersion = "2012-02-12";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
String canonicalizedHeaders = String.Format(
"x-ms-date:{0}\nx-ms-version:{1}",
dateInRfc1123Format,
storageServiceVersion);
String canonicalizedResource = String.Format("/{0}/{1}", AzureStorageConstants.Account, urlPath);
String stringToSign = String.Format(
"{0}\n\n\n\n\n\n\n\n\n\n\n\n{1}\n{2}",
requestMethod,
canonicalizedHeaders,
canonicalizedResource);
String authorizationHeader = Utility.CreateAuthorizationHeader(stringToSign);
Uri uri = new Uri(AzureStorageConstants.QueueEndPoint + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
request.Headers.Add("x-ms-date", dateInRfc1123Format);
request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.Accept = "application/atom+xml,application/xml";
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
Stream dataStream = response.GetResponseStream();
using (StreamReader reader = new StreamReader(dataStream))
{
String responseFromServer = reader.ReadToEnd();
}
}
}

view raw

GetMessage

hosted with ❤ by GitHub

This generates the following request:

GET https://rebus.queue.core.windows.net/revolution/messages HTTP/1.1
x-ms-date: Sun, 08 Sep 2013 06:34:11 GMT
x-ms-version: 2012-02-12
Authorization: SharedKey rebus:K67XooYhokw0i0AlCzYQ4GeLLrJih1r1vSqiO9DBo0c=
Accept: application/atom+xml,application/xml
Host: rebus.queue.core.windows.net

The Queue Service generates the following response:

HTTP/1.1 200 OK
Content-Type: application/xml
Server: Windows-Azure-Queue/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: efb21a86-7d66-47fd-b13d-7aa74fce0568
x-ms-version: 2012-02-12
Date: Sun, 08 Sep 2013 06:34:12 GMT
Content-Length: 484

The message is returned in the response body as follows:

<?xml version="1.0" encoding="utf-8"?><QueueMessagesList><QueueMessage><MessageId>05fd902f-6031-4ef4-8298-ef3844ec3bc6</MessageId><InsertionTime>Sun, 08 Sep 2013 06:34:11 GMT</InsertionTime><ExpirationTime>Sun, 15 Sep 2013 06:34:11 GMT</ExpirationTime><DequeueCount>1</DequeueCount><PopReceipt>AgAAAAMAAAAAAAAAAL+zgF2szgE=</PopReceipt><TimeNextVisible>Sun, 08 Sep 2013 06:34:43 GMT</TimeNextVisible><MessageText>Saturday in the cafe</MessageText></QueueMessage></QueueMessagesList>

UPDATE 10/24/2010: I noticed that some newline specifiers in strings (\n) were lost when the blog was auto-ported from Windows Live Spaces to WordPress. I have put them back in but it is possible I missed some. Consequently, in the event of a problem you should check the newlines in canonicalizedHeaders and stringToSign.

About Neil Mackenzie

Cloud Solutions Architect. Microsoft
This entry was posted in Storage Service, Windows Azure. Bookmark the permalink.

19 Responses to Examples of the Windows Azure Storage Services REST API

  1. Pingback: Windows Azure ストレージ サービス REST API を直接使うコード サンプル | fullvirtueの世界

  2. Egon says:

    Thank you for this very detailed example. I just want to know how I can modify the put/get/delete operations on the blob within the constraints of silverlight, for uploading/downloading images ? I have been trying to use your code in some form but have not been able to understand what I am doing wrong ? Can you please post some more sample code for my case if it won’t be too much trouble for you. Thank you very much again!
    -Egon

  3. Egon –
    I’m sorry but I don’t know anything about SilverLight. However, you should be very careful about exposing your storage credentials.

  4. Egon says:

    Can you atleast tell me how I can GET/PUT/DELETE jpg images using RESTful calls ?
    Thnx again!

  5. Egon –

    You should be able to use pretty much the Put Blob code in the post. You need to replace the code where I write a String to the request stream with code that writes your image. You would also need to set the content length to the correct value. You probably also need to add some metadata with the mime type.

  6. Greg Boyd says:

    Hello Neal,

    Benefiting very much from your contributions to the Azure community. Many thanks.

    I’ve been messing with the below code, built from your work, and would be forever grateful if you could attempt to see what I’m doing wrong that would cause an error while trying to query a single row/entity in an Azure storage table (production).
    Other than altering values for the variables sCloudKey and AzureStorage_Account, below is the verbatim code-behind in a VS2010 C# Winform
    And as an aside, here is the table/entity info queried:
    Table name: “tblplss”
    PartitionKey: “plss”
    RowKey: “1”

    at this line:
    using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())

    I get this error:
    WebException was unhandled: The remote server returned an error: (403) Forbidden.

    in the Debugger/Exception Snapshot . . .
    ResponseURI : {https://.table.core.windows.net/tblplss(PartitionKey=’plss’,RowKey=’1’)}

    Status : System.Net.WebExceptionStatus.ProtocolError

    (The ResponseURI looks like I think it should)

    The code . . .

    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Data;
    using System.Drawing;
    using System.Linq;
    using System.Text;
    using System.Windows.Forms;
    using System.Security.Cryptography; //needed for HMACSHA256
    using System.Globalization; //for CultureInfo
    using System.Net; //for HttpWebRequest
    using System.IO; //for StreamReader and Stream, etc.
    using System.Xml;

    namespace AzureStorageTableApps_Winforms
    {
    public partial class Form1 : Form
    {
    string sCloudKey = “xxxxxxxx”; // secondary key for storage account
    string AzureStorage_Account = “”;
    string AzureStorage_TableEndPoint = “https://.table.core.windows.net/”;
    string AzureStorage_SharedKeyAuthorizationScheme = “SharedKey”;
    byte[] AzureStorage_Key; //calculated below . . .

    public Form1()
    {
    InitializeComponent();
    AzureStorage_Key = Convert.FromBase64String(sCloudKey);
    }

    private void button1_Click(object sender, EventArgs e)
    {
    GetEntity(“tblplss”, “plss”, “1”);
    }

    private void GetEntity(String tableName, String partitionKey, String rowKey)
    {
    String requestMethod = “GET”;
    String urlPath = String.Format(“{0}(PartitionKey=’{1}’,RowKey=’{2}’)”, tableName, partitionKey, rowKey);
    String storageServiceVersion = “2009-09-19”;
    String dateInRfc1123Format = DateTime.UtcNow.ToString(“R”, CultureInfo.InvariantCulture);
    String canonicalizedResource = String.Format(“/{0}/{1}”, AzureStorage_Account, urlPath);
    String stringToSign = String.Format(
    “{0}\n\n\n{1}\n{2}”,
    requestMethod,
    dateInRfc1123Format,
    canonicalizedResource);
    String authorizationHeader = CreateAuthorizationHeader(stringToSign);

    Uri uri = new Uri(AzureStorage_TableEndPoint + urlPath);
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
    request.Method = requestMethod;
    request.Headers.Add(“x-ms-date”, dateInRfc1123Format);
    request.Headers.Add(“x-ms-version”, storageServiceVersion);
    request.Headers.Add(“Authorization”, authorizationHeader);
    request.Headers.Add(“Accept-Charset”, “UTF-8”);
    request.Accept = “application/atom+xml,application/xml”;

    request.Headers.Add(“DataServiceVersion”, “1.0;NetFx”);
    request.Headers.Add(“MaxDataServiceVersion”, “1.0;NetFx”);

    using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
    {
    Stream dataStream = response.GetResponseStream();
    using (StreamReader reader = new StreamReader(dataStream))
    {
    String responseFromServer = reader.ReadToEnd();
    }
    }
    }

    private string CreateAuthorizationHeader(string canonicalizedstring)
    {
    string signature = string.Empty;
    using (HMACSHA256 hmacSha256 = new HMACSHA256(AzureStorage_Key))
    {
    Byte[] dataToHmac = System.Text.Encoding.UTF8.GetBytes(canonicalizedstring);
    signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac));
    }

    string authorizationHeader = string.Format(
    CultureInfo.InvariantCulture,
    “{0} {1}:{2}”,
    AzureStorage_SharedKeyAuthorizationScheme,
    AzureStorage_Account,
    signature);

    return authorizationHeader;
    }
    }
    }

    //I’m trying not to get discouraged about accessing Azure resources, just because of this inability to properly access an Azure storage table—am desperately hoping you can help. Thanks . . . Greg B

  7. Greg Boyd says:

    This is probably an unnecessary follow-up, but my earlier comment/question omitted any reference to my storage account: both in the output of the ResponseURI and in the value of the variable AzureStorage_TableEndPoint.

    Thanks,
    Greg

  8. Greg –

    Offhand, I don’t see anything wrong with your code. I would do the following.

    Use Cerbrata’s Cloud Storage Studio to verify that your account/key combination is correct and that the data is actually where you think it is. Success here would also indicate that your machine is not too far adrift timewise from Azure Storage Service since too great a drift causes failure.

    Use Fiddler to verify that the request (and response) is correct. This is particularly helpful with blobs because a failed response provides the canonicalized headers string the Blob Service used in authentication.

    You might also look at the storage samples David Pallmann posted.

  9. Greg Boyd says:

    Wow! As has been the custom in my 3 decades of programming, the second I ask a question, I figure out the answer. Sorry to have been a bother . . .

    But maybe someone who happens on this page can benefit from the answer (albeit, and embarassingly silly one).

    BEWARE: WordPress changes single and double-quotes to the stylized versions. Normally, you’ll catch this when copying code back to Visual Studio, because you’ll get errors that are easy to find and fix, but in this case, it was the single-quotes in building the urlPath (which are not an error so much, as they are unacceptable to the Azure server parsing the data received on the server side).

    so this line was the problem. . .
    String urlPath = String.Format(“{0}(PartitionKey=’{1}’,RowKey=’{2}’)”, tableName, partitionKey, rowKey);

    —change those stylized quotation marks to the basic ones and Azure will reward you with what you’re seeking.

    Thanks again for your great posts,
    Greg B.

  10. Greg –

    I’m glad that you solved your problem and apologize for the error. The blog was moved automagically from Spaces to WordPress and that introduced some formatting bugs like this.

  11. WienerDog says:

    Thank you for this wonderfully detailed tutorial on the Azure Storage Services REST API. I can’t tell you how refreshing it is to find such a detailed example and explanation on a topic that is otherwise mostly devoid from examples.

    One comment that I found to be extremely helpful in using the Azure development storage with the REST api. I’ve found in my own testing that the CreateAuthorizationHeader method listed in this article didn’t work with my development storage account. The CloudStorageAccount class has an useful method built into it for creating the proper Authorization and Date header. CloudStorageAccount -> Credentials -> SignRequest()/SignRequestLite() eliminates the need for the CreateAuthorizationHeader method.

    I’m not sure if this functionality was added after the publishing of this article. Just thought I would share from my own struggles with Azure. Once again thanks for the great article!

  12. Wiener Dog –

    Thanks for your comment. In the post, I was trying to avoid using methods in the StorageClient library. I suspect the problem is not so much with the CreateAuthorizationHeader() method but with the creation of the canonicalizedResource String when using development storage. The Put Blob sample in the post works with development storage provided the creation of the canonicalizedString is changed to the following:

    String canonicalizedResource = String.Format(“/{0}/{0}/{1}”, AzureStorageConstants.Account, urlPath);

    This is to do with the difference in how URLs are created for cloud and development storage.

  13. Please note that in the following line:

    using (HMACSHA256 hmacSha256 = new HMACSHA256(AzureStorageConstants.Key))

    AzureStorageConstants.Key should not be base64-encoded string. If so, use the following to get correct signing:

    using (HMACSHA256 hmacSha256 = new HMACSHA256(Convert.FromBase64(AzureStorageConstants.Key)))

    will work but make sure that key is proper base64 string.

  14. Carl Cherry says:

    Can you please explain what the “secret key” is. I was assuming that it was the storage account key that I can get in the new Azure portal for the storage account (i.e. the Manage Keys link) but I cannot get that to work. No matter what I try I get back a 403.

    Perhaps it is the Shared Access key as described here?

    http://msdn.microsoft.com/en-us/library/windowsazure/hh508996.aspx

  15. I believe the above uses the Byte[] representation of the storage account access key for authentication. You can use something like the following to use the access key in String form:

    private static String CreateAuthorizationHeader(String accountName, String accessKey, String canonicalizedString)
    {
    String signature = string.Empty;
    Byte[] bytesAccessKey = Convert.FromBase64String(accessKey);
    using (HMACSHA256 hmacSha256 = new HMACSHA256(bytesAccessKey))
    {
    Byte[] dataToHmac = System.Text.Encoding.UTF8.GetBytes(canonicalizedString);
    signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac));
    }

    String authorizationHeader = String.Format(
    CultureInfo.InvariantCulture,
    “{0} {1}:{2}”,
    AuthorizationScheme,
    accountName,
    signature
    );

    return authorizationHeader;
    }

  16. Carl Cherry says:

    Thanks. After debugging a bit with Azure Diagnostics Manager (and Fiddler2) I was assuming the the secret key was the key from the storage account. I also assume that the account is only the account name and does not include the path (i.e. not account.table.core.windows.net but just account). If this is all true then I think I am just struggling to the the canonicalizedResource correct. Currently I am just trying to get a list of tables to I Think the string to sign would be this:
    “GET\n\n\nDate\n/Tables” but perhaps that is wrong.

  17. Carl Cherry says:

    Neil, I finally got the “Tables” call to work with this string.
    “GET\n\n\nDate\n/accountname/Tables”
    While I was trying various things to get it working I removed the Date field from the canonicalizedResource. I thought I had read that Date was optional for Table service calls, but it was the opposite – it can never be empty and must match the x-ms-date value.

  18. Pingback: Writing to an Azure Storage Queue from a Micro Framework device | devMobile's blog

  19. Joel Mendoza says:

    Neil, thank you very much for this example. This post guided me to insert an entity to table storage with REST API successfully. For those new to REST and the Azure REST API, don’t give up, read this post patiently and read over the API documentation. It will make sense with time 🙂

Leave a reply to Egon Cancel reply