BizTalk Server 2013: Connect to Window Azure Blob Storage

  • Article

Introduction

BizTalk Server 2013 supports connectivity with Windows Azure through its new adapters WCF-BasicHttpRelay, WCF-NetTcpRelay, WCF-WebHttp and SB-Messaging. The relay adapter enables connectivity with the relay service in the Windows Azure Service Bus. SB-Messaging adapters provides the means to connect to queues and subscriptions (topic). Finally the WCF-WebHttp for instance, supports connectivity to REST endpoints.

Windows Azure Storage, other services in Windows Azure and the cloud support a REST Interface to connect with. The WCF-WebHttp adapter can be used to connect to the Windows Azure Storage Service. A benefit of connecting to Windows Azure Storage and making use of this service is that you can store large amounts of unstructured text or binary data. In case you need to pull or push documents to Windows Azure you can connect to Windows Azure SQL Database or Windows Azure Storage. The latter offers storage of documents at lower costs. In this article the connection with the Windows Azure Storage through the WCF-WebHttp Adapter in BizTalk Server 2013 will be discussed.

Windows Azure Storage

The Storage service in Windows Azure provides a way to store and access data. This data can be unstructured binary data like video's, audio and images. Besides binary you can store and access text data like XML files, PDF's, word documents and so on. This can be done making use of what is called Binary Large Object (BLOB) storage. The service offers more than just Blob storage:

  • Queues to store messages;
  • Tables to store non-relational structured data.

In this article we will be using Blob storage. To connect to the Windows Azure Storage to store a Blob like an XML or text file you will need a Windows Azure Account and subsequently a storage account. In case you do not have a storage account, you need to create one.

Create a Windows Azure Storage Account

To create a Windows Azure Storage Account navigate to storage service and then at the lower left container click NEW.

http://i208.photobucket.com/albums/bb152/Steef-Jan/CreateStorageAccount_zps6dfc5fcd.png

Figure 1. Create a Windows Azure Storage Account.

After clicking QUICK CREATE a dialog will appear on the right hand side.

http://i208.photobucket.com/albums/bb152/Steef-Jan/CreateStorageAccount2_zps0a98f279.png

Figure 2. Specify Windows Azure Storage Account details.

In the dialog you will specify the URL i.e. first part of DNS, which will have the format of:

<unique name>.core.windows.net

Besides the URL you specify the location (region of a data center) or affinity group and which subscription the storage will be belonging too. You can also specify if you want to geo-replicate your data. The storage account will be provisioned for you as soon when you click CREATE STORAGE ACCOUNT. With a storage account you will have access to the storage service and it entities (BLOB, QUEUES, TABLES).

Create a container

Within a Windows Azure Storage account, you can create a container. A container gives you the ability to group a set of Binary Large Objects (BLOBs). To create a container Click containers in Windows Azure Portal and then click CREATE A CONTAINER.

http://i208.photobucket.com/albums/bb152/Steef-Jan/CreateContainer1_zps7b4208f6.png

Figure 3. Create a container through the Windows Azure Portal.

A dialog will appear.

http://i208.photobucket.com/albums/bb152/Steef-Jan/CreateContainer2_zpsf90f8d71.png

Figure 4. New Container dialog box.

In the dialog you specify a name for the container and define its access. When you click the check mark the container will be provisioned for you.

Note: A storage account can contain an unlimited number of containers, as long as their total size is under 100TB.

http://i208.photobucket.com/albums/bb152/Steef-Jan/CreateContainer3_zpseb2e1b88.png

Figure 5. A newly created container.

Configuring the WCF-WebHttp Adapter

You can access and store data in a container in Windows Azure Storage. That can be done through either code like .NET code, Node.js or through using BizTalk Server 2013 by leveraging the WCF-WebHttp adapter. In case you want to support a business process or messaging solution with BizTalk that needs to send a document to the Cloud (Windows Azure Storage) than the WCF-WebHttp adapter can be of great value.

The configuration of the WCF-WebHttp adapter in a messaging scenario is a follows:

  • A document needs to be sent to a container in Windows Azure Storage (see also the BizTalk Messaging Solution). Therefore a send port will be created and configured with WCF-WebHttp adapter.
  • In the General Tab of the WCF-WebHttp Transport properties the address of the service can be specified (URI). This address can be located through the Windows Azure Storage Dashboard under services.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WcfWebHttpTransportProperties2_zps4df458f9.png

Figure 6. Windows Azure Storage Services endpoints

  • Since we are going to make use of the blob service the endpoint (address) we will use is http://tnwiki.blob.core.windows.net or in case the Transport mode is set to Transport (i.e. HTTPS) than the address will be https://tnwiki.blob.core.windows.net
  • In the General Tab of the WCF-WebHttp Transport properties we will also specify the HTTP Method and URL Mapping. URL Mapping will be applied to dynamically have the file name that is the name of the file being offered to BizTalk Receive Port. The method is going to be PUT as we will create (place) the file in the container.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WcfWebHttpTransportProperties1_zps815fa7f9.png

Figure 7. WCF-WebHttp Transport Properties General Tab.

  • In HTTP Method and URL Mapping section you specify the method (operations) you are going to perform. In this scenario case it is going to be only PUT. The URL mapping you define what is going to added after the specified URI. To make it more dynamic instead of hard-coding in general, you can make use of the variable mapping configuration feature. So what's between the brackets is a variable that can be mapped to promoted property. The HTTP Method and URL Mapping looks in this case like what is visible in the picture above (figure 7).
  • Variable mapping is powerful technique to define any custom variable (or place holder) in your URL, in this scenario case {filename} and map that variable to any context property with the property name and namespace. The Variable mapping is specified by click the Edit… button.

http://i208.photobucket.com/albums/bb152/Steef-Jan/VariableMapping_zps5924520f.png

Figure 8. Variable Mapping of File Name variable.

  • In the Binding Tab you could specify time-out properties or message size.
  • In the Security tab you need to specify the security mode. You can set this to transport. Now with most cloud services have their own authentication scheme (like SalesForce that has OAuth protocol, or Service Bus that leverages the Access Control Service (ACS)). The Windows Azure Storage uses a key/value pair, which is the storage account and the key. Therefore you need that authentication scheme by using a endpointbehavior. Fortunately you do not have to create your own behavior to support authentication to the Azure Storage. Microsoft has created an azureStorageBehaviour you can use to authenticate with Windows Azure Storage. Here you can specify the name of the storage and the storage account key.
  • In the Behaviour Tab you right click EndPointBehaviour and click add extension.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WcfWebHttpTransportProperties3_zps6e89ff9b.png

**

**

Figure 9. Selection of a Behaviour Extension.

  • You select the azureStorageBehaviour and specify the Azure Account Key and Name.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WcfWebHttpTransportProperties4_zps5edb80f2.png

**

**

Figure 10. Specification of account key and name.

  • You can find the details when you go to your Azure Storage Account Dashboard and click Manage Access Keys.

http://i208.photobucket.com/albums/bb152/Steef-Jan/StorageAccountCredentials1_zps6a92e3d9.png

Figure 11. Manage the access keys.

  • In the Manage Access Keys dialog you will find the storage account name and access keys. You can paste the key and name in AzureStorageBehaviour (figure 10).
  • Proxy and Messages Tab you do not need to specify anything and leave it with default settings.

Now that the WCF-WebHttp is configured we can look at the complete BizTalk messaging solution that consists of a Send port and Receive port.

BizTalk Messaging Solution

The messaging solution is a very basic (see figure 12). A file will be dropped into a folder and a BizTalk receive port that listen to that folder will pick the file up. It will then be routed to a send port that subscribes to the files coming in through the receive port. The send port is configured with WCF-WebHttp adapter and will send according to its configuration the file to a container in Windows Azure Storage.

http://i208.photobucket.com/albums/bb152/Steef-Jan/Solution_zpsddec889a.png

Figure 12. BizTalk Messaging solution to send a file to Windows Azure Blob Storage.

However the way the adapter is configured the name of the file will be containing the path of the folder and the file name. Something like C:\foldername\filename. To prevent that from happening in a message based solution you will need a simple custom pipeline component that will strip off the path name. Below you can see the code to strip of the path from the ReceivedFileName context property.

01.public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg) 
02.       { 
03.           IBaseMessageContext contextReceivedFileName = pInMsg.Context; 
04. 
05.           //extract the filename
06.           string value = contextReceivedFileName.Read("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties").ToString();
07. 
08.           //set filename only, strip path
09.           FileInfo file = new  FileInfo(value);
10.           string fileString = file.Name;
11. 
12.           //write updated value back to context
13.           pInMsg.Context.Write("ReceivedFileName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", fileString);
14. 
15.           //return the message with modified context
16.           return pInMsg;
17.       }

When sending a document to Windows Azure Storage. A file will be dropped in the folder and will end up in Windows Azure Storage Container named data.

http://i208.photobucket.com/albums/bb152/Steef-Jan/WindowAzureStorageDemo_zps171d0be1.png

Figure 13. XML files in the data container.

Wrap up

This article demonstrated how to send a document (XML file) to Window Azure Storage Container by means of a BizTalk messaging solution. The WCF-WebHttp Adapter is used to connect to Windows Azure Storage. This service like other Azure Services provides a REST interface that support operations on the service. In case of the Azure Storage Service it will support operations like accessing or storing data. The Windows Azure Storage has its own authentication scheme and an out of the box behavior extension that needs to be added to configure the access to the storage. Once the behaviour in the adapter is configured with all the other necessary properties the data or document can be sent to the cloud as described in the messaging solution in this article.

Call for action

In case you want try it yourself, you will need a virtual machine with BizTalk Server 2013, a Windows Azure account and within your Azure account a Windows Azure Storage account. You can find the source code belonging to this article on MSDN Code Gallery: BizTalk Server 2013: Connect to Window Azure Blob Storage Sample.

If you are interested in trying out others sample using the WCF-WebHttp adapter see the resources below:

See Also

Read suggested related topics:

Another important place to find an extensive amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.