Blobs – Azure Storage Client v1.0

The Azure Storage Client v1.0 provides many classes to support Azure Blob functionality of which the core classes are:

CloudBlob is the base class to each of CloudBlobDirectory, CloudBlockBlob and CloudPageBlob. It provides most of the functionality  for handling blobs including: creation and deletion of blobs; and maintenance of blob properties including attributes and metadata. CloudBlobDirectory adds functionality for traversing and accessing blobs with names structured as a directory hierarchy. CloudBlockBlob adds functionality specific to Block Blobs while CloudPageBlob adds functionality specific to Page Blobs. CloudBlobClient provides functionality to list blobs and gain direct access to the various types of blob. It also provides a convenient means to connect with a CloudStorageAccount object providing access to authentication credentials stored in the Azure configuration. CloudBlobContainer provides functionality for managing containers including the creation, deletion and listing of them.

CloudBlobClient

The CloudBlobClient class serves a similar role to that played by the CloudQueueClient and CloudTableClient classes for Azure Queue and Azure Tables. They provide a basic entry point for accessing the relevant Azure Storage Service. CloudBlobClient is declared:

public class CloudBlobClient {
    // Constructors
    public CloudBlobClient(Uri baseUri, StorageCredentials credentials);
    public CloudBlobClient(String baseAddress, StorageCredentials credentials);
    public CloudBlobClient(String baseAddress);

    // Events
    public event EventHandler<ResponseReceivedEventArgs> ResponseReceived;

    // Properties
    public Uri BaseUri { get; }
    public StorageCredentials Credentials { get; }
    public String DefaultDelimiter { get; set; }
    public Int32 ParallelOperationThreadCount { get; set; }
    public Int64 ReadAheadInBytes { get; set; }
    public RetryPolicy RetryPolicy { get; set; }
    public TimeSpan Timeout { get; set; }
    public Boolean UseIntegrityControlForStreamReading { get; set; }
    public Int64 WriteBlockSizeInBytes { get; set; }

    // Methods
    public IAsyncResult BeginListBlobsWithPrefixSegmented(String prefix, AsyncCallback callback, Object state);
    public IAsyncResult BeginListContainersSegmented(String prefix, AsyncCallback callback, Object state);
    public IAsyncResult BeginListContainersSegmented(String prefix, ContainerListingDetails detailsIncluded,
       AsyncCallback callback, Object state);
    public IAsyncResult BeginListContainersSegmented(AsyncCallback callback, Object state);
    public ResultSegment<IListBlobItem> EndListBlobsWithPrefixSegmented(IAsyncResult asyncResult);
    public ResultSegment<CloudBlobContainer> EndListContainersSegmented(IAsyncResult asyncResult);
    public CloudBlobDirectory GetBlobDirectoryReference(String blobDirectoryAddress);
    public CloudBlob GetBlobReference(String blobAddress);
    public CloudBlockBlob GetBlockBlob(String blobAddress);
    public CloudBlobContainer GetContainerReference(String containerAddress);
    public CloudPageBlob GetPageBlob(String blobAddress);
    public IEnumerable<IListBlobItem> ListBlobsWithPrefix(String prefix);
    public IEnumerable<CloudBlobContainer> ListContainers(String prefix, ContainerListingDetails detailsIncluded);
    public IEnumerable<CloudBlobContainer> ListContainers();
    public IEnumerable<CloudBlobContainer> ListContainers(String prefix);
}

Similarly to CloudQueueClient and CloudTableClient objects, a CloudBlobClient object can be created either using one of the constructors or through invoking the following CloudStorageAccountStorageClientExtensions extension method:

public static CloudBlobClient CreateCloudBlobClient(CloudStorageAccount account);

For example, providing a SetConfigurationSettingPublisher() has been invoked appropriately and the relevant Azure configuration settings have been done then the following creates a CloudBlobClient object:

CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
CloudBlobClient cloudBlobClient = new CloudBlobClient(cloudStorageAccount.BlobEndpoint.ToString(), cloudStorageAccount.Credentials);

or

CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

CloudBlobClient provides several synchronous methods to retrieve a list of containers as well as equivalent asynchronous methods that also support paging through the list. Note that the suffix Segmented is used throughout Azure Storage Client to indicate paging support. Paging is implemented through use of, for example, the ResultSegment<CloudBlobContainer> object returned from EndListContainersSegmented() method. The ResultSegment<TElement> class provides BeginGetNext() and EndGetNext() methods that can be used to page through the list of containers. The ListBlobWithPrefix() method retrieves a list of the containers in the $root container. GetContainerReference() provides a reference to a container.

The GetBlobDirectoryReference(), GetBlobReference(), GetBlockBlob(),and GetPageBlob() methods implement a pattern repeated in the CloudBlobContainer and  CloudBlob classes allowing the immediate creation of the pertinent blob object given an appropriate address.

The following example shows the use of GetBlockBlob():

protected void GetBlockBlobStream(String blobAddress)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlockBlob cloudBlockBlob = cloudBlobClient.GetBlockBlob(blobAddress);

    using( MemoryStream memoryStream = new MemoryStream())
    {
        cloudBlockBlob.DownloadToStream(memoryStream);
        Int64 streamSize = memoryStream.Length;
        memoryStream.Seek(0, SeekOrigin.Begin);
        using (StreamReader streamReader = new StreamReader(memoryStream))
        {
            String blobText = streamReader.ReadToEnd();
        }
    }
}

The example creates a reference to a CloudBlockBlob object which is then downloaded into a MemoryStream. Note that before the  memoryStream is used its Seek() method is invoked to set its current position to the beginning of the stream. Given a container named music containing a blob named British/StoneRoses the method can be invoked through:

GetBlockBlobStream(“music/British/StoneRoses”)

The created cloudBlobBlob object is truly a reference. The only time the Azure Storage Service is accessed in this example is when DownloadToStream() is invoked.

Note that if DefaultDelimiter is set to a value then the default delimiter, ‘/’, will be replaced by the value in any call to the Azure Storage Service.

CloudBlobContainer

The CloudBlobContainer class represents a blob container. It provides methods supporting the creation and deletion of containers as well as providing maintenance functionality such as setting and getting attributes, metadata and security settings on containers. The CloudBlobContainer class also provides methods allowing direct access to the blobs contained in the container either directly or using the directory namespace functionality. CloudBlobContainer is declared pretty much as:

public class CloudBlobContainer {
    // Constructors
    public CloudBlobContainer(String containerAddress, CloudBlobClient service);
    public CloudBlobContainer(String containerAddress, StorageCredentials credentials);
    public CloudBlobContainer(String containerAddress);

    // Properties
    public BlobContainerAttributes Attributes { get; }
    public NameValueCollection Metadata { get; }
    public String Name { get; }
    public BlobContainerProperties Properties { get; }
    public CloudBlobClient ServiceClient { get; }
    public Uri Uri { get; }

    // Methods
    public void Create();
    public void Create(BlobRequestOptions options);
    public Boolean CreateIfNotExist(BlobRequestOptions options);
    public Boolean CreateIfNotExist();
    public void Delete(BlobRequestOptions options);
    public void Delete();
    public void FetchAttributes();
    public void FetchAttributes(BlobRequestOptions options);
    public CloudBlob GetBlobReference(String blobAddressUri);
    public CloudBlockBlob GetBlockBlobReference(String blobAddressUri);
    public CloudBlobDirectory GetDirectoryReference(String relativeAddress);
    public CloudPageBlob GetPageBlobReference(String blobAddressUri);
    public BlobContainerPermissions GetPermissions(BlobRequestOptions options);
    public BlobContainerPermissions GetPermissions();
    public String GetSharedAccessSignature(SharedAccessPolicy policy);
    public String GetSharedAccessSignature(SharedAccessPolicy policy, String groupPolicyIdentifier);
    public IEnumerable<IListBlobItem> ListBlobs();
    public IEnumerable<IListBlobItem> ListBlobs(BlobRequestOptions options);
    public void SetMetadata();
    public void SetMetadata(BlobRequestOptions options);
    public void SetPermissions(BlobContainerPermissions permissions);
    public void SetPermissions(BlobContainerPermissions permissions, BlobRequestOptions options);
}

There are also asynchronous versions of all the methods.

A CloudBlobContainer is created using one of the constructors and providing at a minimum the name of the container. For example:

protected void CreateContainer(String containerName)
{
    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = new CloudBlobContainer(containerName, cloudBlobClient);
    cloudBlobContainer.Metadata["MetadataName"] = “MetadataValue”;
    cloudBlobContainer.Create();
}

GetBlobReference(), GetBlockBlobReference() and GetPageBlobReference() return references to the named CloudBlob, CloudBlockBlob and CloudPageBlob object. Note that none of these methods actually access Azure Storage they merely return references to blobs which may or may not exist. GetDirectoryReference() is similar except that it returns a reference to a CloudBlobDirectory allowing the blob namespace to be traversed like a directory. ListBlobs() can be used to enumerate the blobs in a container as in:

protected void GetBlobs(String containerName)
{
    CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    IEnumerable<IListBlobItem> blobList = cloudBlobContainer.ListBlobs();
    foreach (IListBlobItem item in blobList)
    {
        Uri uri = item.Uri;
    }
}

Container attributes can be set using the Attributes property. Metadata can be set using the Metadata NameValueCollection as shown in the CreateContainer() example. SetMetadata() can be used to update Metadata on a container once it has been created in Azure Storage – calling it earlier will raise an error.

There is no method to detect container existence and if this is required the pattern is to call FetchAttributes() and detect the error if it doesn’t exist.

There are a number of methods including GetPermissions()SetPermissions() and GetSharedAccessSignature() to handle access control and shared access signatures for containers. These are covered in a post on Access Control for Azure Blobs.

CloudBlob

The CloudBlob class represents a blob and implements the functionality not specific to the derived classes representing block blobs and page blobs. In fact, this is most of the functionality and consequently CloudBlob is a large class with many methods. CloudBlob implements the IListBlobItem interface declared as:

public interface IListBlobItem {
    // Properties
    CloudBlobContainer Container { get; }
    CloudBlobDirectory Parent { get; }
    Uri Uri { get; }
}

These properties can be used to access the container for the blob as well as the parent directory when the blob is accessed through a directory hierarchy.

Blob Addressing

A blob exists in a container and the simplest way to address a blob is to use containername/blobname form giving a full URL of:

https://myaccount.blob.core.windows.net/containername/blobname

However, blob names can include a delimiter, typically ‘/’. which can be used to simulate a directory hierarchy in the flat namespace for blobs in a container. For example, the following are all valid addresses for blobs in a container named music:

  • music/American/BobDylan/PositivelyFourthStreet
  • music/American/StevieWonder/InnerVisions
  • music/British/Portishead/SourTimes
  • music/British/Portishead/WanderingStar

The directory naming scheme simulates American and British as top-level directories with BobDylan, StevieWonder and Portishead are sub directories. The Portishead directory contains two blobs identified by SourTimes and WanderingStar although their full blob names are British/Portishead/SourTimes and British/Portishead/WanderingStar respectively. When invoked as:

GetDirectoryList(“music/British”, “Portishead”);

the following code demonstrates the traversal of the directory tree to enumerate these two blobs:

protected void GetDirectoryList(String topLevelDirectoryName, String subDirectoryName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobDirectory topLevelDirectory = cloudBlobClient.GetBlobDirectoryReferencetopLevelDirectoryName);

    CloudBlobDirectory subDirectory = topLevelDirectory.GetSubdirectory(subDirectoryName);

    IEnumerable<IListBlobItem> blobItems = subDirectory.ListBlobs();
    foreach (IListBlobItem blobItem in blobItems)
    {
        Uri uri = blobItem.Uri;
    }
}

The CloudBlob class is declared (in abbreviated form):

public class CloudBlob : IListBlobItem {
    // Constructors
    public CloudBlob(String blobUri, CloudBlobClient serviceClient);
    public CloudBlob(CloudBlob cloudBlob);
    public CloudBlob(String blobAbsoluteUri);
    public CloudBlob(String blobAbsoluteUri, StorageCredentials credentials);

    // Properties
    public BlobAttributes Attributes { get; }
    public NameValueCollection Metadata { get; }
    public BlobProperties Properties { get; }
    public CloudBlobClient ServiceClient { get; }
    public Nullable<DateTime> SnapshotTime { get; }
    public CloudBlockBlob ToBlockBlob { get; }
    public CloudPageBlob ToPageBlob { get; }

    // Methods
    public IAsyncResult BeginCopyFromBlob(CloudBlob source, BlobRequestOptions options, AsyncCallback callback,
       Object state);
    public IAsyncResult BeginCopyFromBlob(CloudBlob source, AsyncCallback callback, Object state);
    public void CopyFromBlob(CloudBlob source);
    public void CopyFromBlob(CloudBlob source, BlobRequestOptions options);
    public CloudBlob CreateSnapshot();
    public void Delete();
    public Boolean DeleteIfExists();
    public Byte[] DownloadByteArray();
    public String DownloadText();
    public void DownloadToFile(String fileName);
    public void DownloadToStream(Stream target);
    public void EndCopyFromBlob(IAsyncResult asyncResult);
    public void FetchAttributes();
    public String GetSharedAccessSignature(SharedAccessPolicy policy);
    public String GetSharedAccessSignature(SharedAccessPolicy policy, String groupPolicyIdentifier);
    public BlobStream OpenRead();
    public virtual BlobStream OpenWrite();
    protected void ParseSizeAndLastModified(HttpWebResponse response);
    public void SetMetadata();
    public void SetProperties();
    public virtual void UploadByteArray(Byte[] content);
    public virtual void UploadFile(String fileName);
    public virtual void UploadFromStream(Stream source);
    public virtual void UploadText(String content);
}

Almost all methods exist in both synchronous and asynchronous forms and with a BlobRequestOptions parameter – as with CopyFromBlob which is shown in all its variants. An important point is that although there are four synchronous upload and download techniques – from byte array, file, stream and text –  the only asynchronous technique is stream upload and download as in BeginUploadFromStream(), EndUploadFromStream(), BeginDownloadToStream() and EndDownloadToStream().

A CloudBlobClient object can be created using one of the constructors or through invoking the following CloudBlobClient method:

public CloudBlob GetBlobReference(String blobAddress);

These methods do not create a blob in the Azure Storage service merely an in-memory representation of one. The actual creation requires the uploading of the blob content through one of the upload methods. For example:

protected void UploadText(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlob cloudBlob = cloudBlobContainer.GetBlobReference(blobName);
    cloudBlob.Metadata[“MetadataName”] = “MetadataValue”;
    cloudBlob.UploadText(“Stately, plump Buck Mulligan came from the stairhead”);
}

This creates a block blob in the Azure Storage Service.

A blob can also be created in the Azure Storage Service by using OpenWrite() to create a BlobStream object and then writing to the stream:

protected void WriteToBlobStream(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlob cloudBlob = cloudBlobContainer.GetBlobReference(blobName);

    String blobText = “bearing a bowl of lather on which a mirror and a razor lay crossed“;
    UTF8Encoding utf8Encoding = new UTF8Encoding();
    Byte[] bytes = utf8Encoding.GetBytes(blobText);

    using (BlobStream blobStream = cloudBlob.OpenWrite())
    {
        blobStream.Write(bytes, 0, bytes.Count<Byte>());
    }
}

When the BlobStream is closed, in this case implicitly on exit from the using block, it uploads the blob in blocks and then commits the blocks.

The contents of blobs can be retrieved using one of the download methods or by using OpenRead() to create a BlobStream object and then reading from the stream. For example:

protected void DownloadText(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlob cloudBlob = cloudBlobContainer.GetBlobReference(blobName);
    String blobText = cloudBlob.DownloadText();
}

and:

protected void ReadBlobStream(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlob cloudBlob = cloudBlobContainer.GetBlobReference(blobName);

    Byte[] bytes = new Byte[1000];
    using ( BlobStream blobStream = cloudBlob.OpenRead())
    {
        blobStream.Read(bytes, 0, bytes.Count<Byte>());
    }
}

The other upload and download methods are used similarly. Note that OpenRead() downloads the block list and then downloads the blob.

Blob attributes and properties are accessed through the Attributes and Properties properties of CloudBlob. The FetchAttributes() method downloads the attributes without downloading the entire blob. Blob metadata is accessed through the Metadata property, a NameValueCollection, as shown in the UploadText example. The SetMetadata() method sets the metadata without uploading the entire blob although an error will be raised if this method is used before the blob has been uploaded.

Blobs can be deleted from the Azure Storage Service with the Delete() and DeleteIfExists() methods. The latter is a convenience method saving an additional roundtrip to verify existence before the blob is deleted.

Blobs can be copied directly on the Azure Storage Service using the CopyFromBlob() method.

protected void CopyFromBlob(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
      CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);

    CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(blobName);
    CloudBlockBlob copyCloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(“Copy” + blobName);
    copyCloudBlockBlob.CopyFromBlob(cloudBlockBlob);
}

Azure Blob supports the concept of a readonly snapshot taken at a specific time. Future modifications to the blob are recorded only as differences from the latest snapshot. A snapshot is made using the CreateSnapshot() method.

The ToBlockBlob and ToPageBlob properties can be used to downcast a CloudBlob to a CloudBlockBlob or a CloudPageBlob respectively.

CloudBlockBlock

The CloudBlockBlob class is derived from the CloudBlob class and adds functionality specific to block blobs to it. It is declared:

public class CloudBlockBlob : CloudBlob, IListBlobItem {
    // Constructors
    public CloudBlockBlob(String blobAbsoluteUri, StorageCredentials credentials, Boolean usePathStyleUris);
    public CloudBlockBlob(String blobAbsoluteUri, Boolean usePathStyleUris);
    public CloudBlockBlob(String blobAbsoluteUri);
    public CloudBlockBlob(String blobAbsoluteUri, StorageCredentials credentials);
    public CloudBlockBlob(String blobUri, CloudBlobClient client);

    // Methods
    public IAsyncResult BeginDownloadBlockList(BlockListingFilter blockListingFilter, AsyncCallback callback, Object state);
    public IAsyncResult BeginDownloadBlockList(BlockListingFilter blockListingFilter, BlobRequestOptions options,
        AsyncCallback callback, Object state);
    public IAsyncResult BeginPutBlock(String blockId, Stream blockData, String contentMD5, AsyncCallback callback,
        Object state);
    public IAsyncResult BeginPutBlock(String blockId, Stream blockData, String contentMD5, BlobRequestOptions options,
       AsyncCallback callback, Object state);
    public IAsyncResult BeginPutBlockList(IEnumerable<String> blockList, BlobRequestOptions options,
    AsyncCallback callback, Object state);
    public IAsyncResult BeginPutBlockList(IEnumerable<String> blockList, AsyncCallback callback, Object state);
    public IEnumerable<ListBlockItem> DownloadBlockList(BlobRequestOptions options);
    public IEnumerable<ListBlockItem> DownloadBlockList(BlockListingFilter blockListingFilter);
    public IEnumerable<ListBlockItem> DownloadBlockList(BlockListingFilter blockListingFilter,
       BlobRequestOptions options);
    public IEnumerable<ListBlockItem> DownloadBlockList();
    public IEnumerable<ListBlockItem> EndDownloadBlockList(IAsyncResult asyncResult);
    public void EndPutBlock(IAsyncResult asyncResult);
    public void EndPutBlockList(IAsyncResult asyncResult);
    public void PutBlock(String blockId, Stream blockData, String contentMD5);
    public void PutBlock(String blockId, Stream blockData, String contentMD5, BlobRequestOptions options);
    public void PutBlockList(IEnumerable<String> blockList);
    public void PutBlockList(IEnumerable<String> blockList, BlobRequestOptions options);
}

The CloudBlockBlob functionality is provided in both synchronous and asynchronous forms. Block blobs comprise a sequence of blocks which are uploaded independently of each other using PutBlock() and then committed to the blob using PutBlockList(). Blocks are identified by UTF8-encoded block IDs.The block list can be viewed using DownloadBlockList(). The following shows a single block being uploaded to a blob via a MemoryStream:

protected void PutBlockFromStream(String containerName, String blobName, Int32 blockId)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(blobName);

    String blobText = new String(‘b’, 1000);
    UTF8Encoding utf8Encoding = new UTF8Encoding();
    using (MemoryStream memoryStream = new MemoryStream(utf8Encoding.GetBytes(blobText)))
    {
        cloudBlockBlob.PutBlock(Convert.ToBase64String(System.BitConverter.GetBytes(blockId)), memoryStream, null);
    }
}

The following shows a block list being committed to create a block blob in the Azure Storage Service:

protected void PutBlockList(String containerName, String blobName, Int32 numBlocks)
{
    CloudStorageAccount cloudStorageAccount
      CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);
    CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference(blobName);

    String[] blockIds = new String[numBlocks];
    for (Int32 i = 0; i < numBlocks; i++)
    {
        blockIds[i] = Convert.ToBase64String(System.BitConverter.GetBytes(i)); ;
    }

    cloudBlockBlob.PutBlockList(blockIds);
}

This example assumes the blockIDs are sequential numbers from zero to one less than the number of blocks.

Note that CloudBlockBlob uses the CloudBlob implementation of the upload methods because CloudBlob uses block blob semantics for uploads.

CloudPageBlob

Microsoft added page blobs to Azure Storage in its November 2009 release. A page blob comprises a sequence of pages, aligned on 512-byte boundaries, and  addressable in a manner optimized for random reads and writes. Both pages and page blobs have a maximum size of 1TB. Block blobs are optimized for streaming.

Page blobs are exposed in Storage Client through the CloudPageBlob class declared:

public class CloudPageBlob : CloudBlob, IListBlobItem {
    // Constructors
    public CloudPageBlob(String blobAddress, CloudBlobClient serviceClient);
    public CloudPageBlob(CloudBlob cloudBlob);
    public CloudPageBlob(String blobAddress, StorageCredentials credentials);
    public CloudPageBlob(String blobAddress);

    // Methods
    public IAsyncResult BeginCreate(Int64 size, AsyncCallback callback, Object state);
    public IAsyncResult BeginCreate(Int64 size, BlobRequestOptions options, AsyncCallback callback, Object state);
    public IAsyncResult BeginGetPageRanges(AsyncCallback callback, Object state);
    public IAsyncResult BeginGetPageRanges(BlobRequestOptions options, AsyncCallback callback, Object state);
    public IAsyncResult BeginWritePages(Stream pageData, Int64 startOffset, BlobRequestOptions options,
       AsyncCallback callback, Object state);
    public IAsyncResult BeginWritePages(Stream pageData, Int64 startOffset, AsyncCallback callback, Object state);
    public void Create(Int64 size);
    public void Create(Int64 size, BlobRequestOptions options);
    public void EndCreate(IAsyncResult asyncResult);
    public IEnumerable<PageRange> EndGetPageRanges(IAsyncResult asyncResult);
    public void EndWritePages(IAsyncResult asyncResult);
    public IEnumerable<PageRange> GetPageRanges(BlobRequestOptions options);
    public IEnumerable<PageRange> GetPageRanges();
    public void WritePages(Stream pageData, Int64 startOffset, BlobRequestOptions options);
    public void WritePages(Stream pageData, Int64 startOffset);

    // Implemented Interfaces and Overridden Members
    public override BlobStream OpenWrite();
    public override BlobStream OpenWrite(BlobRequestOptions options);
    public override void UploadFromStream(Stream source);
    public override void UploadFromStream(Stream source, BlobRequestOptions options);
    public override IAsyncResult BeginUploadFromStream(Stream source, AsyncCallback callback, Object state);
    public override IAsyncResult BeginUploadFromStream(Stream source, BlobRequestOptions options,
        AsyncCallback callback, Object state);
    public override void EndUploadFromStream(IAsyncResult asyncResult);
    public override void UploadText(String content);
    public override void UploadText(String content, Encoding encoding, BlobRequestOptions options);
    public override void UploadFile(String fileName);
    public override void UploadFile(String fileName, BlobRequestOptions options);
    public override void UploadByteArray(Byte[] content);
    public override void UploadByteArray(Byte[] content, BlobRequestOptions options);
}

However, all methods listed in the Implemented Interfaces and Overridden Members section are documented as “not supported on CloudPageBlob.” This means that the only way of writing to a page blob is through WritePages() and its asynchronous version BeginWritePages() and EndWritePages().

A page blob is created in the Azure Storage Service using the Create() method which takes an Int64 specifying the size in bytes of the page blob. Data is written to individual pages in a page blob using WritePages(). Any of the CloudBlob download methods may be used to retrieve data from a page blob. On download, any unwritten pages will be initialized to 0. GetPageRanges() returns an IEnumerable<PageRange> with each PageRange specifying the start and end offsets of a page in the blob. (Note that there might be an error in the setting of PageRange.EndOffset which never appears to be set.)

The following shows the creation of a page blob and the uploading of some data to the first and third pages out of four:

protected void CreatePageBlob(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);

    CloudPageBlob cloudPageBlob = cloudBlobContainer.GetPageBlobReference(blobName);

    const Int32 pageSize = 512;
    const Int32 numPages = 4;
    cloudPageBlob.Create(numPages * pageSize);

    String blobText = new String(‘z’, pageSize);
    UTF8Encoding utf8Encoding = new UTF8Encoding();
    using (MemoryStream memoryStream = new MemoryStream(utf8Encoding.GetBytes(blobText)))
    {
        cloudPageBlob.WritePages(memoryStream, 0);
        memoryStream.Seek(0, SeekOrigin.Begin);
        cloudPageBlob.WritePages(memoryStream, 2 * pageSize);
    }
}

The following shows the listing of the page ranges in a page blob as well as the downloading of the page blob data to a MemoryStream:

protected void GetPageBlob(String containerName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName);

    CloudPageBlob cloudPageBlob = cloudBlobContainer.GetPageBlobReference(blobName);

    IEnumerable<PageRange> pageRanges = cloudPageBlob.GetPageRanges();
    foreach (PageRange pageRange in pageRanges)
    {
        Int64 startOffset = pageRange.StartOffset;
        Int64 endOffset = pageRange.EndOffset;
    }

    using (MemoryStream memoryStream = new MemoryStream())
    {
        cloudPageBlob.DownloadToStream(memoryStream);
        Int64 streamSize = memoryStream.Length;
    }
}

Since page blobs are written to in pages it it possible that two people might be updating the same page blob at the same time, possibly interfering with each others work. For this reason Azure Storage has the concept of a Lease Blob in which one user can acquire a one minute lease preventing any other user from writing to the page blob as long as the lease is active. The BlobProperties.LeaseStatus property exposes a readonly lease status. However, it is not possible to acquire a lease using this high-level Azure Storage Client library – it can be done only using the lower Storage Client Protocol library.

CloudBlobDirectory

The CloudBlobDirectory class allows the blob namespace in a container to be traversed as if the blob names specified full directory paths separated by a delimiter which is ‘/’ by default. CloudBlobDirectory is declared:

public class CloudBlobDirectory : IListBlobItem {
    // Properties
    public CloudBlobClient ServiceClient { get; }

    // Methods
    public IAsyncResult BeginListBlobsSegmented(AsyncCallback callback, Object state);
    public IAsyncResult BeginListBlobsSegmented(BlobRequestOptions options, AsyncCallback callback, Object state);
    public ResultSegment<IListBlobItem> EndListBlobsSegmented(IAsyncResult asyncResult);
    public CloudBlob GetBlobReference(String itemName);
    public CloudBlockBlob GetBlockBlobReference(String itemName);
    public CloudPageBlob GetPageBlobReference(String itemName);
    public CloudBlobDirectory GetSubdirectory(String itemName);
    public IEnumerable<IListBlobItem> ListBlobs(BlobRequestOptions options);
    public IEnumerable<IListBlobItem> ListBlobs();

    // Implemented Interfaces and Overridden Members
    public Uri Uri { get; }
    public CloudBlobContainer Container { get; }
    public CloudBlobDirectory Parent { get; }
}

A CloudBlobDirectory object is created using either CloudBlobClient.GetBlobReference() or CloudBlobContainer.GetDirectoryReference(). The blobs in that directory can be enumerated using the ListBlobs() method. A point to watch out for is that the IListBlobItem item in the enumeration can be any of a CloudBlockBlob, CloudPageBlob or a CloudBlobDirectory, all of which are derived from IListBlobItem. A CloudBlobDirectory object represents a subdirectory in the enumeration and these can be accessed by calling GetSubdirectory().

The following example shows the creation of a CloudBlobDirectory object, its use to get a reference to a CloudBlockBlob, a listing of the blobs in the directory and finally the creation of a CloudBlobDirectory object representing a subdirectory:

protected void GetDirectoryList(String directoryName, String subdirectoryName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobDirectory cloudBlobDirectory = cloudBlobClient.GetBlobDirectoryReference(directoryName);

    CloudBlockBlob cloudBlockBlob = cloudBlobDirectory.GetBlockBlobReference(blobName);

    IEnumerable<IListBlobItem> blobItems = cloudBlobDirectory.ListBlobs();
    foreach (IListBlobItem blobItem in blobItems)
    {
        Uri uri = blobItem.Uri;
    }

    CloudBlobDirectory anotherCloudBlobDirectory = cloudBlobDirectory.GetSubdirectory(subdirectoryName);
}

The following shows some of the CloudBlobDirectory methods being used:

protected void GetDirectoryList(String directoryName, String subdirectoryName, String blobName)
{
    CloudStorageAccount cloudStorageAccount =
       CloudStorageAccount.FromConfigurationSetting(“DataConnectionString”);
    CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();

    CloudBlobDirectory cloudBlobDirectory = cloudBlobClient.GetBlobDirectoryReference(directoryName);

    CloudBlockBlob cloudBlockBlob = cloudBlobDirectory.GetBlockBlobReference(blobName);

    IEnumerable<IListBlobItem> blobItems = cloudBlobDirectory.ListBlobs();
    foreach (IListBlobItem blobItem in blobItems)
    {
        Uri uri = blobItem.Uri;
    }

    CloudBlobDirectory anotherCloudBlobDirectory = cloudBlobDirectory.GetSubdirectory(subdirectoryName);
}

Given a container named music containing the following blobs:

  • British/StoneRoses
  • British/StoneRoses/FoolsGold
  • British/StoneRoses/IWannaBeAdored
  • British/StoneRoses/ThisIsTheOne

invoking the GetDirectoryList() method as follows:

GetDirectoryList(“music/British”, “StoneRoses”, “IWannaBeAdored”);

has the following effect. cloudBlobDirectory is a reference to “music/British”; blobItems enumerates over a CloudBlockBlob named “British/StoneRoses” and a CloudBlobDirectory named “British/StoneRoses” representing the three blobs in that subdirectory; and anotherCloudBlobDirectory enumerates these three blobs as CloudBlockBlobs

Other Functionality

There are many other blob-related classes in Azure Storage Client v1.0, In particular, the SharedAccessPolicies and SharedAccessPolicy classes are used with CloudBlob.GetSharedAccessSignature() to implement shared access signatures. I describe these in a another post on Access Control for Azure Blobs.

UPDATE: 11/30/2009

Added a new section on CloudBlobContainer inadvertently left out of the original post.

UPDATE: 1/21/2010

Added a link to a later post on Access Control for Azure Blobs.

About Neil Mackenzie

Cloud Solutions Architect. Microsoft
This entry was posted in Storage Service, Windows Azure. Bookmark the permalink.

13 Responses to Blobs – Azure Storage Client v1.0

  1. Joannes says:

    Thanks for the post, yet, how do we specify a different separator than "/" for the CloudBlobDirectory? Or rather, how do we specify no separator at all? It seems that the current behavior forces the "/" separator, which is not the convention used so far. In particular, all our apps are following the convention of not having any separator in many situation.

  2. Neil says:

    Joannes -I don’t think you can specify a different separator for CloudBlobDirectory – and when I tried it didn’t seem to work. I suspect the point is that it doesn’t make sense to use an alternative separator for a directory view of blobs. You certainly don’t need to go near CloudBlobDirectory if a directory view is not something you are interested in – and, if you don’t you don’t need to worry about using "/" in the blob name.

  3. Unknown says:

    I’ve been trying to figure out how I can download block blobs with progress indication and cancellation but have been drawing a blank. Any help in this regard would be greatly appreciated! Thanks….

  4. Neil says:

    If you want to download block blobs with progress indication you probably have to handle paging yourself using the Range or x-ms-range request header to control which parts of the blob you want to download.http://msdn.microsoft.com/en-us/library/ee691967.aspxI don’t know of any way of adding this request header with the Storage Client library. You should be able to do it with the slightly lower level Storage Client Protocol library. Steve Marx has a post showing how to use this to lease blobs – another piece of blob functionality not exposed in the Storage Client library – and it should be trivial to modify his code to handle adding the x-ms-range request header. You can, of course, add the x-ms-range request header in the low-level REST API but that is a LOT more work.http://blog.smarx.com/posts/leasing-windows-azure-blobs-using-the-storage-client-library

  5. sonam says:

    Hey Neil,Can help me out with this issue:I am doing this like: using (FileStream stream = f.GetStream()) { Byte[] bytes = new Byte[16*1024]; stream.Read(bytes,offset,bytes.Length); BlobUploadHelpers.WriteToBlobStream("temp", f.FileName, bytes,offset); offset = bytes.Length; }I am using a lsight modfication: public static void WriteToBlobStream(String containerName, String blobName,Byte[] bytes,int offset) { BlobUploadHelpers.CreateContainer("temp"); CloudStorageAccount cloudStorageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString"); CloudBlobClient cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient(); CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference(containerName); CloudBlob cloudBlob = cloudBlobContainer.GetBlobReference(blobName); using (BlobStream blobStream = cloudBlob.OpenWrite()) { blobStream.Write(bytes,offset, bytes.Length); } }Basically my point is to use buffer in between.But i see that the size of file/blob is exactly that of my buffer size.Any help?Thanks.

  6. Neil says:

    sonam -Since you appear to be uploading a file you should just use CloudBlob.UploadFile().http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storageclient.cloudblob.uploadfile.aspx

  7. sonam says:

    Hey Neil,No ,actually it might seem like the file from disk but its actually a stream which i am reading using buffer and then writing it to blob.Any help?

  8. Neil says:

    sonam -You should look at the UploadCloudDrive() example in my post on Azure Drive since it appears to do pretty much what you want to do. The difference is that it uses a page blob instead of a block blob. However, it should not be too difficult to modify to handle your situation.http://nmackenzie.spaces.live.com/blog/cns!B863FF075995D18A!582.entry

  9. sonam says:

    Thannks…I’ll look at it.Keep up the good work..

  10. Adrian Bezzina says:

    Hi Niel,

    I was wondering how is it possible to specifiy a proxy server for the Storage API to use, without going to the extreme and setting the GLOBAL proxy WebRequest.DefaultWebProxy (this code is running in a SharePoint process that is shared by SharePoint and other 3rd party applications). I haven’t been able to find the ‘right property or object’, and if its not there then I think its a HUGE oversight in the API, and was wondering when it would be added (even if you just exposed the underlying HttpWebRequest object somehow).

  11. Adrian –

    Sorry I don’t know the answer to your question. However, it is pretty easy to get access to the HttpWebRequest object using the classes in the Microsoft.WindowsAzure.StorageClient.Protocol namespace. Specifically, the BlobRequest class gets you the HttpWebRequest. These classes ship with the Azure Storage Client library.

    You should post your question on the MSDN Azure Data Forum.

  12. Pingback: Exploring Windows Azure Storage APIs By Building a Storage Explorer Application - Paolo Salvatori's Blog - Site Home - MSDN Blogs

  13. Pingback: Access Control for Azure Blobs | Convective

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s