Customizing and Extending the BizTalk WCF Adapters

Introduction

The contents of the following post are taken from a presentation I created and delivered at TechReady 9 in July 2009 and replicated with Stephen Kaufman at Tech-Ed Europe 2009 .

The ABC of WCF

Windows Communication Foundation (WCF) is a runtime and a set of APIs for exchanging messages between components and applications. WCF was designed according to the tenets of service orientation and unified the existing communication technologies like .NET Remoting and ASMX web service into a single programming model and consistent architecture which provides high levels of functionality, interoperability and extensibility. As you know, a typical WCF service can expose one or multiple endpoints. In general, endpoints provide clients access to the functionality offered by a WCF service. Each endpoint consists of 3 properties as depicted in the picture below:

• An address that indicates where the endpoint can be found.
• A binding that specifies how a client app can communicate with the endpoint.
• A contract that identifies the operations exposed by the endpoint.

WCF Architecture: Messaging Runtime

• The WCF runtime is divided into 2 primary layers as shown by the following picture:
  The Service Layer aka Service Model defines the mechanisms and attributes used by developers to define and decorate service, message and data contracts.
• The  Messaging Layer  is responsible for preparing a WCF message for transmission on the send side and produce a WCF message for the dispatcher on the receive side. The messaging layer accomplishes this task using a Channel Stack. This latter is a pipeline of channel components that handle different processing tasks.  Each channel stack is composed of exactly one transport channel, one message encoder, and zero or more protocol channels.

It’s the responsibility of the proxy component on the client side and dispatcher component on the service side to mediate and translate between the two layers. In particular, the proxy component transforms .NET method calls into Message objects, whereas the dispatcher component turns WCF Messages into .NET method calls. WCF uses the Message class to model all incoming/outgoing messages within the Messaging Layer. The message represents a a SOAP Envelope, and therefore it’s composed of a payload and a set of headers. A typical WCF communication can be described as follows:

1. The client application creates one or more input parameters. Each of these parameters is defined by a data contract.
2. The client application invokes one of the methods of the service contract exposed by the proxy.
3. The proxy delivers a WCF Message object to the channel stack.
4. At this point each protocol channel has a chance to operate on the message before the transport channel uses a message encoder to transmit the final Message as a sequence of bytes to the target service. Each protocol channel can modify the content or the headers of the message to implement specific functionalities or WS-* protocols like WS-AtomicTransaction, WS-Security.
5. The raw stream of data is transmitted over the wire.
6. On the service side, the transport channel receives the stream of data and uses a message encoder to interpret the bytes and to produce a WCF Message object that can continue up the channel stack. At this point each protocol channel has a chance to work on the message.
7. The final Message is passed to the Dispatcher.
8. The Dispatcher receives the WCF Message from the underlying channel stack, individuates the target service endpoint using the destination address and Action property contained in the Message, deserializes the content of the WCF Message into objects.
9. Finally the target service method is invoked.

WCF Binding Comparison

Bindings provide a mechanism for configuring channel stacks. A binding defines a precise recipe for building a channel stack using a transport channel, a message encoder, and a set of protocol channels.
WCF ships with several built-in bindings that target common communication scenarios.

You should select the most appropriate binding based on your needs:

• For example, BasicHttpBinding is designed for interoperability with simple web services that conform to the SOAP 1.1 message version.
• Instead the WSHttpBinding is designed for interoperability with more advanced web services that might leverage different WS-* protocols. Both of these bindings use HTTP for the transport and the text message encoding because they are designed for maximum interoperability.
• NetTcpBinding and NetNamedPipeBinding, on the other hand, are designed for efficient and performant communication with other WCF applications across machines or on the same machine respectively.
• If you need maximum flexibility and you need to use one or multiple custom protocol channels at runtime, you can use the CustomBinding that gives you the possibility to decide which binding elements compose your binding.

Bindings have different characteristics in terms of response time and throughput, so the general advice to increase performance is using the NetTcpBindind and NetNamesPipeBinding whenever possible.

BizTalkServiceInstance Class

Each WCF Receive Location in BizTalk Server is indeed an instance of a WCF Service Class called BizTalkServiceInstance hosted by a separate instance of a ServiceHost-derived class

• the BtsServiceHost for RLs running in an in-process host
• the WebServiceHost for RLs running in an isolated host

When you “enable” a WCF receive location, the adapter initializes and opens the ServiceHost, which dynamically builds the WCF runtime components within the BizTalk service process (BtsNtSvc.exe for in in-process host, w3wp.exe for an isolated host). This includes the channel stack, dispatcher, and the singleton service instance. WCF Receive and Send Ports are message-type agnostic or if you prefer untyped. This design comes in handy when you need to configure a single Receive Port to accept numerous message types or versions that you can normalize (via BizTalk maps or using a custom pipeline component) into a common message type before being posted the BizTalkMsgBoxDb. However, this design also implies that the WCF adapters will need to build on generic service contracts in order to remain message-type agnostic within the WCF implementation.

WCF Receive Locations have the responsibility to receive the incoming message bytes, perform any necessary SOAP and WS-* processing, and publish the message (or some part of it) to the message box. The WCF Adapters create a separate ServiceHost and singleton service object of this class for each receive location to handle client requests for the lifetime of the BizTalk Host instance running WCF receive locations. The service object uses multiple threads to process messages concurrently unless the WCF-NetMsmq receive locations are used with the Ordered processing property being selected.. As shown in the picture below, the class is decorated with the ServiceBehavior attribute:

BizTalkServiceInstance class

 [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]internal sealed class BizTalkServiceInstance : ITwoWayAsync,                                                ITwoWayAsyncVoid,                                                IOneWayAsync,                                                IOneWayAsyncTxn,                                                ITwoWayAsyncVoidTxn{    ...}

In particular, you can note that:

-
InstanceContextMode = InstanceContextMode.Single

-
ConcurrencyMode = ConcurrencyMode.Multiple

Hence, all incoming messages to a WCF Receive Location are received and processed by a single well-known instance of the BizTalkServiceInstance class. This allows to avoid service activation/deactivation costs and improve performance/scalability.

The BizTalkServiceInstance class implements multiple untyped, generic service contracts.

-
IOneWayAsync

-
IOneWayAsyncTxn

-
ITwoWayAsync

-
ITwoWayAsyncVoid

-
ITwoWayAsyncVoidTxn

Each contract was designed for a different scope (as suggested by their name):

-
OneWay vs TwoWay Message Exchange  Pattern

-
Transactional vs Non-Transactional Communication

 [ServiceContract(Namespace = "https://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]public interface IOneWayAsync{    // Methods    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]    void BizTalkSubmit(Message message);    void EndOneWayMethod(IAsyncResult result);}[ServiceContract(Namespace = "https://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]public interface IOneWayAsyncTxn{    // Methods    [OperationContract(AsyncPattern = true, IsOneWay = true, Action = "*")]    IAsyncResult BeginOneWayMethod(Message message, AsyncCallback callback, object state);    [OperationContract(IsOneWay = true, Action = "BizTalkSubmit")]    void BizTalkSubmit(Message message);    void EndOneWayMethod(IAsyncResult result);}[ServiceContract(Namespace = "https://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]public interface ITwoWayAsync{    // Methods    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]    Message BizTalkSubmit(Message message);    Message EndTwoWayMethod(IAsyncResult result);}[ServiceContract(Namespace = "https://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]public interface ITwoWayAsyncVoid{    // Methods    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);    [OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]    void BizTalkSubmit(Message message);    void EndTwoWayMethod(IAsyncResult result);}[ServiceContract(Namespace = "https://www.microsoft.com/biztalk/2006/r2/wcf-adapter")]public interface ITwoWayAsyncVoidTxn{    // Methods    [TransactionFlow(TransactionFlowOption.Mandatory)]    [OperationContract(AsyncPattern = true, IsOneWay = false, Action = "*", ReplyAction = "*")]    IAsyncResult BeginTwoWayMethod(Message message, AsyncCallback callback, object state);    [TransactionFlow(TransactionFlowOption.Mandatory), OperationContract(IsOneWay = false, Action = "BizTalkSubmit")]    void BizTalkSubmit(Message message);    void EndTwoWayMethod(IAsyncResult result);}

All the methods exposed by these service contracts are generic, asynchronous and untyped. In fact, as shown in the picture above, each of this method is decorated by the OperationContract attribute and in particular:

-
AsyncPattern = True  indicates that an operation is implemented asynchronously using a Begin<methodName> and End<methodName> method pair in a service contractAction = “*” means that the method accepts a message with any Action.

-
ReplyAction = “*” means that the method can return a message with any Action.

-
Every method accepts as parameter or returns a generic WCF Message.

As a consequence, each WCF Receive Location can accept multiple  message types and versions that can be normalized into a canonical format using a different map before being published to the MessageBox.

Also Send Ports are message-type agnostic.

One-Way WCF Receive Locations

When you define a One-Way WCF Receive Location that uses the NetMsmqBinding, the underlying WCF service will expose an endpoint which uses the IOneWayAsync service interface. If instead the One-Way WCF Receive Location is not configured to use NetMsmqBinding, the underlying WCF service will expose an endpoint using the ITwoWayAsyncVoid contract interface that returns an acknowledgment message. Otherwise, when you define a two-way WCF receive location, the underlying WCF service will expose an endpoint using the ITwoWayASync service interface.

As you can note in the picture above, the IOneWayAsync service contract has a single logical operation named OneWayMethod that matches all incoming messages (Action=”*”). It’s implemented with the WCF asynchronous programming model that ties two methods – BeginOneWayMethod and EndOneWayMethod – together via AsyncPattern=true.

ITwoWayAsyncVoid is similar in design except that it also returns a WCF Message object (notice the addition of ReplyAction=”*”) and the operation is not marked as one-way. The only difference with ITwoWayAsync is that it also returns a Message object in order to properly model two-way ports.  The following picture depicts what happens when a message is received and processed through a One-way WCF Receive location.

1. The Raw Message Data are sent over the wire.
2. The Transport Channel receives and decodes the incoming stream of bytes and creates a WCF Message that is processed through the Channel Stack.
3. The WCF message is passed on to the Dispatcher
4. The WCF message is passed on to the BizTalkServiceInstance.
5. Based on the RL configuration, the entire SOAP Envelope, the Body of the SOAP message or a specific Xml Element is used as content of the BizTalk message.
6. The BizTalk message is processed through the pipeline.
7. A Map is eventually applied.
8. The BizTalk message is published to the MessageBox.

WCF Adapters

The WCF Adapters provided by BizTalk correspond 1:1 to the most commonly used WCF bindings.

WCF-Custom and WCF-CustomIsolated Adapters

The WCF-Custom and WCF-CustomIsolated adapters offer you complete control over the channel stack and behaviors configuration, and as a consequence, they are the only WCF adapters you really need. Compared with the other WCF Adapters, they are the only ones to provide the possibility to:

-
Implement and exploit extensibility points.

-
Have full access to properties exposed by bindings/behaviors.

-
Enable the use of the bamInterceptor endpoint behavior.

-
Export/Import the binding configuration.

-
Disable a receive location on failure.

-
Run an http-based RL within an in-process host.

-
Use bindings (e.g. wsDualHttpBinding) for which a WCF Adapter does not exist.

WCF allows developers to modify and extend its standard runtime behavior with several extensibility points, some of which are not supported by BizTalk Server: for example you cannot customize the BizTalkServiceInstance class, create you own ServiceHost or create a custom ContractBehavior. BizTalk Server supports the following extensibility points:

-
Custom Behaviors

  -   
    Service Behaviors: they enable the customization of the entire service runtime including the ServiceHost.
      
  -   
    Endpoint Behaviors: they enable the customization of service endpoints and their associated EndpointDispatcher. The most common technique is creating an endpoint behavior to add a Message Inspector to the collection of Message Inspectors used at run time by a proxy/dispatcher. This allows to intercept an in-flight incoming/outgoing WCF message before this latter is passed/transmitted to the intended operation/target service, inspect and eventually change its content (headers or body).
      
  

• Custom Binding Elements, Channels Binding Elements, Channels, Channel Factories, Binding Element Extension, etc. This technique is more complex, because in order to register and execute at runtime a custom channel within the channel stack used by a WCF Receive Location or Send Port, it’s necessary to create and properly register multiple components:

  • Custom BindingElementExtensionElement.

  • Custom BindingElement.

  • Custom ChannelFactory

  • Custom Channel.

• Custom Bindings: the WCF LOB Adapter SDK allows developers to create  new bindings to use with WCF-Custom and WCF-CustomIsolated Adapters. For example, the WCF based SQL Adapter (WCF-SQL) is based on a new binding called sqlBinding that is properly registered in the machine.config when installing the Adapter.

In my previous post  called How to Throw Typed Fault Exceptions from Orchestrations Published as WCF Services I created 2 different Endpoint Behaviors to customize the default runtime behavior of the WCF-Custom/WCF-CustomIsolated Adapter. In my post entitled How to create a custom WCF Channel that debatches an inbound message I show how creating a custom channel to debatch (on a WCF Send Port) an inbound message containing a collection of operations and make multiple calls to the underlying WCF service, one for each operation.

Conclusions

I strongly suggest you to read the following whitepapers to have a better insight in WCF Adapters and their extensibility points.

• The Using the Windows Communication Foundation (WCF) Adapters in BizTalk Server whitepaper describes the use of the WCF adapters in BizTalk Server.
• The Consuming and Hosting WCF Services with Custom Bindings in BizTalk Server whitepaper provides an in-depth explanation on how to use the BizTalk Server WCF adapters for hosting and consuming Windows Communication Foundation (WCF) services with custom bindings. The paper compiles a series of lessons learned based on a real-world implementation of integrating a custom WCF binding with the WCF adapters. Furthermore, key industrial infrastructure concerns such as transaction management and security are discussed in the context of integrating custom WCF bindings and behaviors with the BizTalk Messaging Engine. Also emphasized are a few pragmatic paradigms such as the use of dynamic ports in consuming WCF services. Finally, some key integration challenges are discussed to streamline the correct use of the adapters for solving complex business problems.

Moreover, I recommend to download and review the following WCF Adapters samples on MSDN:

• How to create a custom WCF Channel that debatches an inbound message available on my blog.
• Using Custom Behaviors with the BizTalk WCF Adapters: Extend WCF services through custom behaviors and the WCF adapters.
• Using a Custom Behavior to Secure a Receive Location with the WCF Adapters: Secure access to a BizTalk receive location by using a custom authorization behavior with the WCF adapters.
• Using a Pipeline Intermediary with the WCF Adapters: Use an intermediary BizTalk Server custom pipeline to dynamically channel an incoming WCF client call to the correct version of a WCF service.
• Using Transactions with Message Interception and the WCF Adapters: Custom message interception during BizTalk Server’s binding processing of an outgoing WCF message.
• Using BizTalk Server Messaging with the WCF Adapters: Use BizTalk Server messaging and the WCF adapters to send WCF messages from a WCF client application to a WCF service.
• Using Dynamic Send Ports with the WCF Adapters: Configure dynamic send ports by using WCF adapter context properties for the WCF-NetTcp transport and for the WCF-Custom transport with the WCF-NetTcp binding.
• Handling Exceptions in Orchestrations with the WCF Adapters: Handle typed fault exceptions in orchestrations.
• Using Custom SOAP Headers with the WCF Adapters: Access custom SOAP headers in inbound WCF messages, and add custom SOAP headers to outbound WCF messages.
• Programmatically Submitting a WCF Message Using the WCF Adapters: Submit a WCF message to an orchestration exposed as a WCF service. The WCF adapter uses an XPath command to extract specific SOAP property elements from the incoming message.
• Using the Messages Tab with the WCF Adapters: Use the Messages tab in the Transport Properties dialog box of a WCF adapter to control which part of an incoming WCF message is mapped to the body of the resulting BizTalk message.
• WCF Adapter Walkthroughs: Get sample code for the WCF adapter walkthroughs outlined in the WCF Adapter Walkthroughs topic of the MSDN Library.

Comments

• Anonymous
  October 29, 2010
  Hi Paolo! I wanted to expose an WCF Service (basicHTTPBinding) in BizTalk with two operations: one operation is Request-Response (Sync) and the other is OneWay (Async). The WCF Publishing Wizard accepts this configuration (a service with one operation async and the other operation sync) but it only creates a Receive Location related to a TwoWay Receive Port. So my question is how to manage correctly the Async operation within the TwoWay Receive Port/Location created (a TwoWay Receive Location always wait for a response). The only solution is to create two WCF Services? One with the OneWay operation and other with the TwoWay operation? I hope not. Sorry if this question is not related directly to this blog entry but I couldn´t find a blog entry better to ask for your recomendation regarding this issue. Regards, Diego

• Anonymous
  October 31, 2010
  Hi Diego, that's allright, no worries. I tried to follow the steps you mentioned, and I used the BizTalk WCF Service Publishing Wizard to create a Receive Location with a Two-Way operation and a One-Way operation respectively. Then I looked at the artifacts generated by the Wizard, and indeed the latter created a single Request-Response Receive Port and a Request-Response WCF-BasicHttp Receive Location. Then I opened the WSDL of the generated service and indeed the service exposes 2 operations, both using the BasicHttpBinding_ITwoWayAsync service contract, which is not what you were expecting. I'm afraid that in order to accomplish your objective, you have to create 2 separate Request Ports and 2 related Receive Locations, two-way and one-way respectively. Note: beware that the service contract exposed by a One-way WCF-BasicHttp Receive Location is not the IOneWayAsync, as you could expect, but the ITwoWayAsyncVoid (see my article above). In other words, the operation exposed is not truly one-way, is just a two-way operationb with a void response. This deisgn was intentional, because one-way opeartions do not return any feedback or exception to the caller, and the architect that designed the WCF Adapter wanted to avoid this situation. The only Adapter using the IOneWayAsync and IOneWayAsyncTxn service contracts is the WCF-NetMsmq Adapter. Ciao, Paolo

• Anonymous
  November 01, 2010
  Thanks for your time Paolo! In my opinion, this (creating two web services to manage two operations) is a big restriction. For each similar requirement (mixed operations) the web services inventory will grow considerably. I hope that the WCF Adapter architect could find a solution to this in a future adapter version. I already read your excellent article about the IOneWayAsync issue. Thanks again!, Diego

• Anonymous
  October 09, 2014
  Hello excellent post, but i got a question how is the implementation tu use  a custom encoder ( i want to ecnrypt/decrypt a piece of the body) in a custom binding. If you have other idea will be appreciated it!. I am very lost. I used message inspector but i read about the last stage of wcf is the encoder.

• Anonymous
  October 09, 2014
  Hi Venger Have you tried to use message level security to encrypt/decrypt only a section of the document? By the way, I think that if you want to encrypt the payload, you simply use a message inspector, if you instead want to encrypt/decrypt the whola SOAP message using a custom encoder/decoder, well, you should definitely create a custom channel. Ciao Paolo

• Anonymous
  October 09, 2014
  Hello Paola, thx for the quick response. I have tried to use the encrypt/decrypt of the security level but the thing that i found is wcf always sign the message and i dont know how to specified the payload of the message. With WCF Custom and WCF Basi the message is tottally encrypted but in other format that the client wont required. Do you have any example of custom channel or custom encoder? Thanks.

• Anonymous
  October 09, 2014
  Sorry my mystake Paolo, about your name!. I am very out of mind !!

• Anonymous
  October 09, 2014
  Hi Vanger for a sample of a custom channel look at my sample: blogs.msdn.com/.../how-to-create-a-custom-wcf-channel-that-debatches-an-inbound-message.aspx Ciao Paolo