System Connections

Integration Patterns

May 2004

Summary: This chapter builds on Chapter 3 by describing how to connect with individual systems. Each system allows certain types of access and restricts others. This chapter presents a series of related patterns that will help you analyze the alternative methods and the tradeoffs to consider when you choose your system connections.

Contents

Connecting to Layered Applications

Data Integration

Presentation Integration

Functional Integration

System Connection Patterns

The previous chapter described different strategies for the design of an integration layer and the tradeoffs involved in choosing an alternative. After you decide to use an integrating layer, you must then decide exactly how you are going to connect each system that you intend to integrate.

As you consider how to connect to different information systems, you will encounter a mix of technical architectures. Each technical architecture is designed to allow certain types of access and to restrict others. These access restrictions constrain the set of integration options that are available to you. This chapter discusses a series of related patterns that will help you analyze the alternative methods and the tradeoffs to consider when you choose your system connections.

Connecting to Layered Applications

When trying to find connection points to an existing application, you need to study the way these applications are structured. The prevailing architectural style for business applications is Layered Application. Many applications fall into the more specific form of the Three-Layered Services Application pattern [Trowbridge03]. The components of this pattern are shown in Figure 1.

Figure 1. A three-layered services application

This architectural style defines three distinct layers:

• Presentation layer. The presentation layer displays information to the end user and allows for user input.
• Business logic layer. The business logic layer contains business functions that act on the business data
• Data layer. The data layer stores data persistently in a data store. This layer is also known as the resource layer.

Each application layer presents an opportunity for an application to interact with the integration layer, as shown in Figure 2.

Figure 2. Multiple choices when connecting an application

Because each layer has different requirements and purposes, you face different challenges when connecting to each layer. Similarly, there are three primary patterns you can use to connect an application to an integration layer:

• Presentation Integration. The integration layer can extract information from the application's user presentation layer through a technique known as screen scraping.
• Functional Integration. The integration layer can interact with the business logic layer through application or service interfaces.
• Data Integration. The integration layer can move data in and out of the data layer.

To make an informed choice about which pattern to use, you must know the options that are viable, and you must evaluate their impact on the overall integration architecture. This chapter describes each pattern and elaborates on the benefits and liabilities of each one.

As in other chapters, this chapter uses visual models to show the associations between the patterns. Figure 3 shows the patterns as circles and simple associations between them as lines. As the discussion progresses, more complex pattern relationships build upon this basic diagram.

Figure 3. System connection patterns

The first kind of integration to consider is Data Integration.

Data Integration

Many applications keep large volumes of data in data stores such as flat files and relational, hierarchical, and object-oriented databases. Other applications that need this information can access it directly from these data stores.

Connecting applications through the data store is relatively simple. Usually, you use FTP, scheduled batch files, database management system (DBMS) utilities, extract transform and load (ETL) tools, and integration servers.

If you decide to integrate at the data layer, there are three patterns that represent the three types of data integration to consider: Shared Database [Hohpe04], Maintain Data Copies [Teale03], and File Transfer [Hohpe04]. These patterns are shown in Figure 4.

Figure 4. Three kinds of data integration

Figure 4 introduces a new pattern relationship to the visual model from Figure 3. The triangle indicates refinement between the base pattern (Data Integration) and the derivative patterns (Shared Database, Maintain Data Copies, and File Transfer). More precisely, this relationship is defined as follows:

"A specific pattern refines a more abstract pattern if the specific pattern's full description is a direct extension of the more general pattern. That is, the specific pattern must deal with a specialization of the problem the general pattern addresses, must have a similar (but more specialized) solution structure, and must address the same forces as the more general pattern, but may also address additional forces. To make an analogy with object-oriented programming … the refines relationship is similar to inheritance." [Noble98]

Applying this relationship to the problem of data integration means that if you decide to integrate systems at the data layer, you must further refine your decision by choosing one of three alternatives. You could share a single instance of a database between multiple applications by using the Shared Database pattern, as shown in Figure 5.

Figure 5. Multiple applications sharing the same data store

Another approach is to create multiple copies of a database and distribute them throughout your enterprise according to the Maintain Data Copies pattern. If you do this, you must then maintain these separate copies, which introduces synchronization and replication. Figure 6 shows the Data Replication pattern, which refines Maintain Data Copies.

Figure 6. Data Replication, a refinement of Maintain Data Copies

Yet another approach is to use File Transfer. In this pattern, one application produces a file and transfers it so that other applications can consume it. Files can then be produced at regular intervals to synchronize two or more systems.

The Data Integration pattern, later in this chapter, describes these alternatives in detail and provides the benefits and liabilities of each approach. This discussion should help you choose the kind of data integration that is appropriate for your requirements.

Using Data Integration is made easier by the abundance of tool support provided by many companies. However, in spite of the low cost and the maturity of the tools, accessing the data store is not viable in some cases. A less invasive way of connecting applications is to connect through the presentation layer.

Presentation Integration

When applications that have user interfaces join an integration architecture, other applications can connect to them through the presentation byte stream as shown in Figure 7.

Figure 7. Connecting to an application through the presentation layer

Presentation connectivity represents the least invasive way of connecting applications because the other applications appear to the host application as humans who are interacting through the user interface. Therefore, this form of integration does not require any changes to the host. The disadvantage is that simulating user interaction is cumbersome and inefficient. It requires parsing the data or functionality out of the byte stream, which effectively reverses the transformations performed by the presentation logic. In addition, applications that connect through the presentation layer can only access what is also available to a human user. The granularity of the exposed data and functionality is very coarse; burying potentially-rich APIs.

One advantage of Presentation Integration is that it can be an inexpensive way to integrate applications. Although the connection is inexpensive, it is also easily disrupted. By its very nature, presentation integration is tightly coupled to the host application's presentation. If the application's presentation changes, the presentation integration solution must also change.

Instead of sharing data or parsing through a byte stream to access a system's functionality, it is often preferable to access a system's business logic directly through Functional Integration.

Functional Integration

By connecting directly to the business logic layer, Functional Integration enables other applications and services to take advantage of the business logic that is encapsulated in other systems, as shown in Figure 8.

Figure 8. Integrating applications and services at the business logic layer

Functional Integration connects applications through interfaces and specifications. Unfortunately, not all applications in an integration architecture have interfaces and specifications. Quite often, the applications that have APIs do not expose their data and functions at a granularity level that is suitable for integration.

Functional Integration can be very effective in the right circumstances. A good example of when to use Functional Integration is for a credit scoring application or service.

Credit Scoring Example

Many financial applications require a credit score to qualify an applicant for a specific loan. Salespeople often want a quick and accurate response from a credit scoring system so they can quickly prequalify customers and steer them to an affordable alternative. In addition, credit scores depend on a number of dynamic factors, which means that credit scores can quickly become out of date.

Because credit scores are calculated values, sharing data means relying on credit scores that might be out of date or duplicating raw input data and forcing each application to implement the credit scoring algorithm independently. A better solution is to share the business logic encapsulated in the credit scoring system and to then expose it through some type of functional interface that other systems can consume. For this level of integration, you need Functional Integration.

Kinds of Functional Integration

After you choose to use Functional Integration, you must further refine your decision by choosing one of the three alternatives:

• Distributed Object Integration. Distributed Object Integration is also known as instance-based collaboration because it extends the model of object-oriented computing to distributed solutions. Objects inside one application interact with objects in another remote application in the same way they would interact locally with another object.
• Message-Oriented Middleware Integration. Message-Oriented Middleware Integration connects systems by using asynchronous message queues that are based on proprietary message-oriented middleware. The connected systems then communicate through messages that contain small packets of data. Because the communication is asynchronous and durable, there is little chance that the messages will be lost during a network or system failure.
• Service-Oriented Integration. Service-Oriented Integration connects systems by enabling them to consume and provide XML Web services. This type of integration uses standards to provide both a portable type system and an extensible messaging framework that is not coupled to a proprietary implementation or transport. In addition, Service-Oriented Integration recommends the WS-I Basic Profile to ensure interoperability between endpoints. The WS-I Basic Profile is a carefully chosen subset of XML, SOAP, HTTP, and other standards.

Note: The term service is used in many different ways in the context of software engineering. It is also used in at least seven levels of scope: operating system, process, object, distributed object, proprietary message-oriented middleware, logical, and XML Web services. This guide uses the term service to mean XML Web services unless indicated otherwise.

Figure 9 shows these three alternatives as distinct derivatives, or refinement patterns, of Functional Integration.

Figure 9. Three kinds of Functional Integration

Later in this chapter, the Functional Integration pattern describes these alternatives in detail and discusses the benefits and liabilities of each approach. This discussion should help you choose the kind of functional integration that is appropriate for your requirements.

System Connection Patterns

Data, presentation, and functional integration represent ways in which seasoned integration architects connect applications. But given your specific requirements, how do you connect applications within your integration architecture?

Sometimes the applications you are integrating limit your choices. If there is more than one way you can connect them, you must evaluate the tradeoffs associated with each potential choice. The Data Integration, Presentation Integration, and Functional Integration patterns later in this chapter distill the knowledge required to make an informed decision.

Two general points are worth noting. First, an application that uses one style of connection can communicate with another application that uses a different style. For example, Presentation Integration may be the only way to extract data from a preexisting application, but you can still insert this data into the target application using Data Integration. Second, despite sometimes similar names, a specific integration layer pattern can be used with any system connection pattern. For example, a Portal Integration solution can extract data from existing applications by using Presentation Integration, Functional Integration, or Data Integration.

Figure 10 shows three system connection patterns, with derivative patterns and associations.

Figure 10. System connection patterns and their relationships

Table 1 summarizes these patterns and provides the problem/solution pairs they represent.

Table 1: System Connections Patterns

Start | Previous | Next

© Microsoft Corporation. All rights reserved.