Microsoft ESB Guidance for BizTalk Server 2006 R2

patterns & practices Developer Center

November 2007

Important: Version 2.0 of the ESB Guidance is called the BizTalk ESB Toolkit. This version extends BizTalk Server 2009 capabilities. To learn more, see BizTalk ESB Toolkit 2.0 on the BizTalk Developer Center or download the BizTalk ESB Toolkit from the MSDN Download Center. Customers who are using ESB Guidance 1.0 are strongly encouraged to upgrade to BizTalk Server 2009 and the BizTalk ESB Toolkit 2.0. The 1.0 release and documentation will be deprecated in August 2009.

Summary

The Microsoft ESB Guidance uses Microsoft BizTalk Server 2006 R2 to support a loosely coupled messaging architecture. BizTalk Server includes a powerful publish/subscribe mechanism for messaging applications that works by creating and filling subscriptions, which provides a highly efficient and scalable platform for service-oriented architecture (SOA) applications.

The Microsoft ESB Guidance extends the functionality of BizTalk Server to provide a range of new capabilities focused on building robust, connected, service-oriented applications that incorporate itinerary-based service invocation for lightweight service composition, dynamic resolution of endpoints and maps, Web service and WS-* integration, fault management and reporting, and integration with third-party SOA governance solutions.

The Microsoft ESB Guidance was developed jointly by the patterns & practices team and the Connected Systems Division.

Contents

Overview
Getting Started
Community
Future Plans
Feedback and Support
Authors and Contributors
Related Titles

Overview

The Microsoft ESB Guidance provides architectural guidance, patterns, practices, and a set of BizTalk Server R2 and .NET components to simplify the development of an Enterprise Service Bus (ESB) on the Microsoft platform and to allow Microsoft customers to extend their own messaging and integration solutions.

Common Scenarios

The term Enterprise Service Bus (ESB) is widely used in the context of implementing an infrastructure for enabling a service-oriented architecture (SOA). However, real-world experience with the deployment of SOAs has shown that an ESB is only one of many building blocks that make up a comprehensive Service-Oriented Infrastructure (SOI). The term ESB has morphed in a number of different directions, and its definition depends on the interpretation of individual ESB and integration platform vendors and on the requirements of particular SOA initiatives.

Based on the experience Microsoft has gathered from many successful real-world SOI implementations, you can think of an ESB as a collection of architectural patterns based on traditional enterprise application integration (EAI), message-oriented middleware, Web services, .NET and Java interoperability, host system integration, and interoperability with service registries and asset repositories.

Figure 1 shows a high-level view of the type of interconnectivity that an ESB architecture can provide.

Figure 1
A high-level example of the connectivity provided the Enterprise Service Bus architecture

Audience Requirements

The Microsoft ESB Guidance is intended for developers working with Microsoft BizTalk Server 2006 and who build solutions that leverage the Service Oriented Architecture pattern. To take full advantage of the Microsoft ESB Guidance, developers should possess knowledge and experience working with the following:

• Microsoft BizTalk Server 2006 R2
• Microsoft Visual Studio 2005
• Microsoft .NET development techniques, including the development of ASP.NET Web services and .NET Framework components

What's New

This is the first full release of the Microsoft ESB Guidance. Compared to the latest community technical preview release, this release contains the following new features:

• New ESB Web services, including the Itinerary and Resolver services
• New samples that demonstrate itinerary processing, UDDI service integration, and the BizTalk Operations and Resolver services; the ESB Management Portal, and an implementation of the Scatter/Gather pattern for Web services
• New core features, such as itinerary processing, centralized event logging, the Exception Management Framework; it also introduces the AmberPoint Embedded Nano Agent for BizTalk Server and the SOA Software Management Point for BizTalk Server.

System Requirements

The following list describes the Windows operating system, applications, and core services and features required by the ESB Guidance:

• Microsoft Windows Server 2003 with Service Pack 2 installed
• Microsoft BizTalk Server 2006 R2, including Business Activity Monitoring (BAM)
• Microsoft .NET Framework 3.0
• Internet Information Services (IIS) 6.0 (used for Web services and the ESB Management Portal)
• A UDDI service, such as Microsoft UDDI
• SQL Server 2005 SP2, including the Database Service, Analysis Services, and Integration Services; or SQL Server 2000, including the Database Service and Data Transformation Service, and with Service Pack 3 or later installed
• Visual Studio 2005 (C# for most projects; C++ for the JMS pipeline component) with Service Pack 1 or later installed
• InfoPath 2003 or InfoPath 2007 (used to render exception details generated by the ESB Exception Management Framework)
• Enterprise Library 3.1 (this is required only for the ESB Management Portal and ESB Alert Notification service samples)
• To be able to use the ESB JMS component or the samples that require MQSeries, you must also install the following:
  • BizTalk MQ Series Adapter installed and configured using the default name
  • IBM WebSphere MQ 5.3 (with CSD12) or WebSphere MQ 6.x Business Integration platform
  • The IBM "RfhUtil" JMS Test utility for queuing and retrieving messages

Design Goals

The Microsoft ESB Guidance consists of a series of interoperating components that support and implement a loosely coupled messaging environment that makes it easier to build message-based enterprise applications. The services and components fall naturally into the following seven categories:

• Web services. These expose internal services such as itinerary processing, exception management, resolution of endpoints and maps, BizTalk operations, UDDI interoperation, and transformation of message content.
• Itinerary services. These include agents for performing transformations and message delivery. You can create custom services that participate in Itinerary processing.
• Itinerary on-ramps. These receive external messages using either SOAP or WCF. On-ramps expose the itinerary SOAP header and perform itinerary processing, using the Microsoft ESB Guidance Resolver and Adapter Provider Framework for dynamic resolution of endpoints and metadata.
• On-ramps. These receive external messages in a range of formats and transports, such as HTTP, JMS, WMQ, FTP, Flat File, and XML. They are typical BizTalk receive locations that optionally use the Microsoft ESB Guidance interop pipeline components and the Microsoft ESB Guidance Resolver and Adapter Provider Framework for dynamic resolution of endpoints and metadata.
• Off-ramps. These implement send ports for the delivery of messages using formats and transports such as SOAP, WCF, JMS, WMQ, FTP, HTTP, Flat File, XML, or any other custom formats. They are typical BizTalk send ports that optionally use the Microsoft ESB Guidance interop pipeline components and the Microsoft ESB Guidance Resolver and Adapter Provider Framework for dynamic resolution of endpoints and metadata.
• Exception Management Framework. This includes the exception Web service, the exception management API, and components that enrich, process, and pass exception details to the ESB Management Portal.
• ESB Management Portal. This provides registry provisioning, exception mediation, alert notification, and analytics.

Many of these components and services rely on features implemented by BizTalk Server 2006 R2, such as the Orchestration, Transformation, and Business Rules engines and the Message Box database. Figure 2 shows a schematic view of the categories; the components and services typically occurring within each category; and the core BizTalk system components used by the Microsoft ESB Guidance.

Figure 2
The architecture and components of the Microsoft ESB Guidance

How the ESB Works

The Microsoft ESB Guidance accepts inbound messages and operates on them, perhaps (but not always) by performing processes such as transformation, delivery, or any other custom defined process. To specify the operations required, the core processing components require a message to contain associated instructions or metadata that define the processes to apply and the tasks to execute with the message content.

This approach provides loose coupling between services; this means that the ESB does not require prior knowledge of the specific processing for each message. It just has to know the possible range of processes and how to apply each process. The wide range of options for specifying the available processes and the mapping between the processes and the instructions within messages provides a flexible mechanism for configuring and adjusting behavior, without requiring changes to code and redeployment of components.

Design Patterns

The architecture that ESB uses, where processes deposit messages in the Message Box database and subscribers pick them up based on a processing instruction in the message, effectively implements the state-machine design pattern. In addition, the ESB implements and exposes its core functionality in a services-oriented manner, including to external applications through a set of core Web services.

This loosely coupled approach to designing BizTalk and ESB-based applications yields highly scalable solutions and has become an industry-accepted best practice.

Getting Started

After installing the Microsoft ESB Guidance, you should read the "Getting Started" topic in the documentation to understand the architecture, message flow, and the core components of the Microsoft ESB Guidance. You should also install and run the samples included with the Microsoft ESB Guidance, which demonstrate the common use cases described in the "Getting Started" topic and other messaging scenarios. This approach will help you quickly grasp how the Microsoft ESB Guidance works and how you can take advantage of its features in your own SOA applications.

Community

This guidance, like many patterns & practices deliverables, is associated with a community site. On this community site, you can post questions, provide feedback, or connect with other users for sharing ideas. Community members can also help Microsoft plan and test future releases and download additional content, such as extensions and training material.

Future Plans

Version 2.0 of the ESB Guidance is called the BizTalk ESB Toolkit. This version extends BizTalk Server 2009 capabilities. To learn more, see BizTalk ESB Toolkit 2.0 on the BizTalk Developer Center or download the BizTalk ESB Toolkit from the MSDN Download Center. Customers who are using ESB Guidance 1.0 are strongly encouraged to upgrade to BizTalk Server 2009 and the BizTalk ESB Toolkit 2.0. The 1.0 release and documentation will be deprecated in August 2009.

Feedback and Support

Questions? Comments? Suggestions? To provide feedback about this guidance, or to get help with any problems, please visit the Microsoft ESB Guidance Community site. The message board on the community site is the preferred feedback and support channel because it allows you to share your ideas, questions, and solutions with the entire community. The Microsoft ESB Guidance for BizTalk Server R2 is a guidance offering, designed to be reused, customized, and extended. It is not a Microsoft product. Code-based guidance is shipped "as is" and without warranties. Customers can obtain support through Microsoft Support Services for a fee, but the code is considered user-written by Microsoft support staff.

Authors and Contributors

The ESB Guidance for BizTalk Server R2 was produced by the following individuals:

• Program and Product Management: Dmitri Ossipov (Microsoft Corporation)
• Architect and Development Lead: Marty Wasznicky (Microsoft Corporation)
• Development: Jonathan Summers (Avanade), Craig Butler, Russell Waltz, and Hari Munikar (CTS, Inc.); Muthuraj Velayutham* *(Sogeti); Sandeep Kesiraju (Magenic Technologies); Kevin Orbaker, Patrick Zaw, and Doug Lam (SpeakTech)
• Test: Hanz Zhang and Carlos Farre (Microsoft Corporation); Anant Manuj Mittal, Bhavin Raichura, Gayatri Das, and Krishna Prakash Kalakuntla (Infosys Technologies Ltd)
• Documentation: Alex Homer (Content Master Ltd); Nelly Delgado and RoAnn Corbisier (Microsoft Corporation); Tina Burden McGrayne (TinaTech, Inc.)

Many thanks to the following advisors who provided invaluable assistance:

• Lukas Cudrigh, Chris Tavares, Kartik Paramasivam, and David Stucki (Microsoft Corporation)

Thanks also to the many contributors who assisted us in the production, in particular:

• Michael Holdorf and Christian Martinez (Sogeti); Jeff King, Jarrett Vance, Radhakrishnan Mukkai, Matthew Anderegg (CTS, Inc.)
• Thomas E. Canter and Brian Loesgen (Neudesic, LLC)

Related Titles

• BizTalk ESB Toolkit 2.0
• BizTalk Enterprise Service Bus (ESB) Guidance
• BizTalk Service Oriented Solutions
• Using Web Services with BizTalk Server