Design for Flexibility

First of all, let’s level set on the definition of system Flexibility. I quite like M. Barbacci’s definition which is “Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.”

The beauty of this system quality attribute is that it is a key factor in providing Agility ("The ability of a system to be both flexible and undergo change rapidly" --MIT). In fact, I’ll blog some ideas I’ve been thinking as a formula for Agility because I feel Flexibility doesn’t seem to get the credit it deserves here…anyway, for arguments sake, let’s just assume that Flexibility is an important system quality attribute.

The problem I’d like to raise for discussion is how to help system designers build systems that are optimized for Flexibility. One approach I've been thinking about to help system designers build systems that are optimized for system Flexibility is to create an Architecture Principle which includes direction and explanation so that it can easily be consumed by system designers. I’d like to throw one out to this Architect Community to see what you all think.

The approach I’ve taken is

1.      Identify well-known patterns that optimize for Flexibility in high and low level system abstraction levels.

2.      Identify common scenarios in my IT environment and use them to explain what to do in those scenarios to optimize for Flexibility

3.      Identify Flexibility anti-patterns, or patterns which degrade system Flexibility.

Using this approach this, I’ve created a document which contains the content of this design principle. Below is a first go at this principle:

Design for Flexibility

Description: Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.

Rationale:

Optimizing for system Agility is a primary goal for enterprise systems with the purpose of enabling rapid delivery of enterprise applications to the business. There are several system quality attributes which are used to calculate system Agility such as Reusability, Interoperability and Flexibility. Within Microsoft IT’s enterprise application environment, system Flexibility is of particular importance as it is a key factor contributing to an agile architecture. For this reason, attention is paid to ensure guidance is provided to system designers to ensure systems are optimized for Flexibility.

Implications:

If systems are designed to optimize system Flexibility, the expected benefits are:

· Enabling business process changes with minimum effect on technology platforms.

· Enabling technology platform changes with minimum effect on business processes.

· Enabling Service Oriented Architecture (SOA).

· Enabling modularized systems library for rapid system reuse.

· Improving system Maintainability.

A note about when and where to optimize for Flexibility: This sections includes a few scenarios commonly observed in Microsoft IT to help guide system designers where and how to optimize for Flexibility.

1. System Integration. There are three primary Interface Types to send and receive data between systems and they are; Real-time Integration, Near Real-time Integration and Batch Integration. Each of these scenarios are explained in the sections below.

a. Real-time Integration

                                          i. The table below describes the scenario when Real-time Integration is often the best choice:

Term

Value

Transaction Volume – Traffic

Greater than 1 Peak Transactions Per Second

Transaction Volume – Message Size

Less than 10Mb Peak Message Size per Transaction

Interface Response Time

Less than 5 seconds

Latency Tolerance

Less than 5 seconds

Data State Staleness

Less than 60 seconds

                                         ii. The table below lists patterns and styles which optimize systems for Flexibility for Real-time Integration scenarios. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

b. Near Real-time Integration

                                          i. The table below describes the scenario when Real-time ETL is often the best choice:

Term

Value

Transaction Volume – Traffic

Less than 1 Transactions Per Second at Peak period and greater than 1 Transaction per 30 minutes at Trough period

Transaction Volume – Message Size

Greater than 10Mb and less than 30 Mb Peak Message Size per Transaction

Interface Response Time

Greater than 5 seconds and less than 3 minutes

Latency Tolerance

Greater than 5 seconds and less than 30 minutes

Data State Staleness

Less than 30 minutes

                                          i. The table below lists patterns and styles which optimize systems for Flexibility in Near Real-time Integration scenarios. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

c. Batch Integration (aka ETL )

                                          i. The table below describes the scenario when Batch Integration via ETL is often the best choice:

Term

Value

Transaction Volume – Traffic

Less than 10 Transactions per day at Peak period

Transaction Volume – Message Size

Greater than 30 Mb Peak Message Size per Transaction

Interface Response Time

Greater than 3 minutes

Latency Tolerance

Greater than 30 minutes

Data State Staleness

Greater than 30 minutes

                                         ii. To optimize Flexibility in Batch Integration scenarios,

1.  Use the following Object-oriented Design patterns; Façade and Adapter even within SQL Stored Procedures. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Façade

2. Follow guidelines to implement the concepts of encapsulation and ‘separation of concerns’.

d. SAP Web Services. This scenario is when an application requires integration to an SAP BAPI which is wrapped with a Web Service application. To optimize for Flexibility, the Web Services which directly wraps the SAP BAPI should implement the following Object-oriented Design patterns; Façade, Canonical Data Model and Adapter. See Appendix A - Pattern Reference for details on these patterns.

2. Configuration Management scenarios. Systems which have been designed to optimize for Flexibility take advantage of configuration mechanisms allowing for changes to the behavior of an application via changes in a system configuration file.

a. Types of Configuration Management changes that optimize for Flexibility are

                                          i. Configurable Logging sources and log message types

                                         ii. Configurable modules for in-process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.

Measure:

· é Green – The use of Architecture patterns which optimize Flexibility are used at all architecture and design abstraction levels as well as an implementation of Configuration Management for all major non-functional and functional sub-systems and system components.

o Examples of proven Architecture Styles and Architecture and Design patterns which optimize system Flexibility are in the table below. See Appendix A - Pattern Reference for details on these patterns.

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Functional Integration

Gateway

Message Broker

Message Bus

Message Endpoint

Messaging

Messaging Gateway

Observer

Pub/Sub

Remote Façade

Service Layer

Service-oriented Integration

o Types of Configuration Management changes that optimize for Flexibility are

§ Configurable Logging sources and log message types

§ Configurable modules for use process interception – Note that the Chain of Command Object-oriented Design pattern should be used in these situations.

Separation

· èYellow – Architecture patterns which optimize Flexibility at the design abstraction level. Examples of such design-level patterns which optimize system Flexibility are:

Name

Adapter

Chain of Responsibility

Command

Data Transfer Object

Entity Aggregation

Event Message

Event-Driven Consumer

Façade

Message Endpoint

Observer

Remote Façade

Note: See Appendix A - Pattern Reference for details on these patterns.

· êRed – System implementations which:

o Do not implement any of the styles and patterns listed in Appendix A - Pattern Reference with a Direct or Indirect value in the “Relevance to Flexibility” column.

o Are using patterns such as

§ Shared Database

§ Remote Procedure Invocation

o Do not follow guidelines for Encapsulation and Separation of Concerns

o Do not have configurable sub-systems or application components.

o Implement a Batch Integration system integration type for a Real-time or Near real-time scenario


1.1Appendix A - Pattern Reference

Name

Relevance to Flexibility

Type

Reference

Remote Procedure Invocation

Anti-pattern

Style

https://www.enterpriseintegrationpatterns.com/EncapsulatedSynchronousIntegration.html

Shared Database

Anti-pattern

Style

https://www.enterpriseintegrationpatterns.com/SharedDataBaseIntegration.html

Adapter

Direct

Design

https://www.dofactory.com/Patterns/PatternAdapter.aspx

Event Message

Direct

Design

https://www.enterpriseintegrationpatterns.com/EventMessage.html

Façade

Direct

Design

https://www.dofactory.com/Patterns/PatternFacade.aspx

Functional Integration

Direct

Architecture

https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archfunctionalintegration.asp

Messaging

Direct

Style

https://www.enterpriseintegrationpatterns.com/Messaging.html

Service Layer

Direct

Architecture

https://www.martinfowler.com/eaaCatalog/serviceLayer.html

Service-oriented Integration

Direct

Architecture

https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archserviceorientedintegration.asp

Chain of Responsibility

Indirect

Design

https://www.dofactory.com/Patterns/PatternChain.aspx

Command

Indirect

Design

https://www.dofactory.com/Patterns/PatternCommand.aspx

Data Transfer Object

Indirect

Design

https://www.martinfowler.com/eaaCatalog/dataTransferObject.html

Entity Aggregation

Indirect

Architecture

https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archentityaggregation.asp

Event-Driven Consumer

Indirect

Architecture

https://www.enterpriseintegrationpatterns.com/EventDrivenConsumer.html

Gateway

Indirect

Architecture

¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/desgateway.asp

¡https://www.martinfowler.com/eaaCatalog/gateway.html

Message Broker

Indirect

Architecture

¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebroker.asp

¡https://www.enterpriseintegrationpatterns.com/MessageBroker.html

Message Bus

Indirect

Architecture

https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp

Message Endpoint

Indirect

Architecture

https://www.enterpriseintegrationpatterns.com/MessageEndpoint.html

Messaging Gateway

Indirect

Architecture

https://www.enterpriseintegrationpatterns.com/MessagingGateway.html

Observer

Indirect

Design

¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpatterns/html/DesObserver.asp

¡https://www.dofactory.com/Patterns/PatternObserver.aspx

Pub/Sub

Indirect

Architecture

¡https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/despublishsubscribe.asp

¡https://www.enterpriseintegrationpatterns.com/PublishSubscribeChannel.html

Remote Façade

Indirect

Design

https://www.martinfowler.com/eaaCatalog/remoteFacade.html

Appendix A Glossary

Term

Definition

Agility

Ability of a system to be both flexible and undergo change rapidly.

Architecture Pattern

An architectural pattern expresses a fundamental structural organization schema for software systems. It provides a set of predefined subsystems, specifies their responsibilities, and includes rules and guidelines for organizing the relationships between them.

Architecture Style

An architectural approach to solving a common problem.

Business Capability

The unit describing the combination of people, business process and technology and the policy around its management

Component

Defined as executable/source code, practices, patterns, services as prescribed by Technical Architecture team.

Data State Staleness

The difference between the master state, from which the cached state was created, and the current version of the cached state.

Design Pattern

A design pattern provides a scheme for refining the subsystems or components of a software system, or the relationships between them. It describes a commonly recurring structure of communicating components that solves a general design problem within a particular context.

Encapsulation

Encapsulation conceals the exact details of how a particular class works from objects that use its code or send messages to it. So, for example, the Dogs class has a bark() method. The code for the bark() method defines exactly how a bark happens (e.g., by inhaling() and then exhaling(), at a particular pitch and volume). Timmy, Lassie's friend, however, does not need to know exactly how she barks. Encapsulation is achieved by specifying which classes may use the members of an object. The result is that each object exposes to any class a certain interface — those members accessible to that class. For example, an interface can ensure that puppies can only be added to an object of the class Dog by code in that class. Members are often specified as public, protected and private, determining whether they are available to all classes, sub-classes or only the defining class. Some languages go further: Java uses the protected keyword to restrict access also to classes in the same package, C# and VB.NET reserve some members to classes in the same assembly using keywords internal (C#) or Friend (VB.NET), and Eiffel allows one to specify which classes may access any member

Enterprise Solution

Any system that participates in an end to end business process that spans beyond the organization. The solution is SAP, Siebel, Clarify,/Metropolis, *ebis, or a solution that masters a key data entity such as Organization, Partner, Customer, Solution Opportunity, Campaign, Product or Order (among others).

Extract Transform and Load (ETL)

Extract Transform and Load is an approach to create nearly-identical copies of the data and to manage the integrity of the copies if they can be updated at both the source and target within a replication interval.

· https://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpatterns/html/ArcDataReplication.asp

Flexibility

Flexibility is the ease with which a system or component can be modified for use in applications or environments other than those for which it was specifically designed.

Implementation Pattern

An implementation pattern is a low-level pattern specific to a particular platform. An implementation pattern describes how to implement particular aspects of components or the relationships between them, using the features of a given platform.

Intentional Integration

Defining a public data and service contract to the process and data owned by a system. Should not be consumer driven decision

Interface Response Time

Amount of time a consumer application is expected to wait for a response from the provider application.

Interoperability

the ability of two or more systems or components to exchange information and to use the information that has been exchanged.

Latency Tolerance

Delay between updates between the Source Application and the Destination Application. Some forms of data integration imply a delay between updates to the data that is used by multiple applications. For example, in theData Replicationpattern, the data is extracted from the source system, and it is transported over a network. The data might then be modified, and then it is inserted in a target database. This delay means that one system may have access to data that is more up to date than another system

Maintainability

Maintainability is:

· The aptitude of a system to undergo repair and evolution.

· The ease with which a software system or component can be modified to correct faults, improve performance or other attributes, or adapt to a changed environment. (2) The ease with which a hardware system or component can be retained in, or restored to, a state in which it can perform its required functions.

Near Real-time Integration

General term for describing asynchronous inter-system communication typically an implementation of the Message Bus Integration Style.

· https://www.enterpriseintegrationpatterns.com/ComposedMessagingMSMQ.html

· https://msdn.microsoft.com/practices/topics/patterns/default.aspx?pull=/library/en-us/dnpag/html/archmessagebus.asp

Performance

Performance is the responsiveness of the system – the time required to respond to stimuli (events) or the number of events processed in some interval of time. Performance qualities are often expressed by the number of transactions per unit time or by the amount of time it takes to complete a transaction with the system.

Process

A process is the business process that an application may be involved in executing or automating

Real-time Integration

General term for describing synchronous inter-system communication typically an implementation of the Request-Reply pattern.Note that Real-Time Integration can also implement asynchronous communication but in a fashion to scale synchronous communication.

Relevance to Flexibility

§ Direct. The pattern/style directly contributes to optimizing system Flexibility.

§ Indirect. The pattern/style indirectly contributes to optimizing system Flexibility.

§ Anti-pattern. The use of the pattern adversely affects system Flexibility.

Reliability

Reliability is the ability of the system to keep operating over time. Reliability is usually measured by mean time to failure.

Reusability

Reusability is the degree to which a software module or other work product can be used in more than one computing program or software system.

This is typically in the form reusing software that is a encapsulated unit of functionality.

Role

A collection of responsibilities and motivations, to which authorizations are attached.

Scalability

Scalability is the ability to maintain or improve performance while system demand increases.

Security

Security is a measure of the system’s ability to resist unauthorized attempts at usage and denial of service while still providing its services to legitimate users. Security is categorized in terms of the types of threats that might be made to the system.

Separation of Concerns

Separation of concerns is the process of breaking a program into distinct features that overlap in functionality as little as possible. A concern is any piece of interest or focus in a program. Typically, concerns are synonymous with features or behaviors

Supportability

Supportability is the ease with which a software system is operationally maintained.

Testability

Testability is the degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met.

Transaction Volume – Peak Message Size

The largest message size expressed in bytes for a single transaction.

Transaction Volume – Peak Traffic

The largest number of transactions expected to be processed expressed in ‘transactions per second’.

Type

§ Architecture. See Architecture Pattern.

§ Design. See Design Pattern.

§ Style. See Architecture Style.

Usability

Usability is:

§ The measure of a user’s ability to utilize a system effectively.

§ The ease with which a user can learn to operate, prepare inputs for, and interpret outputs of a system or component.

§ A measure of how well users can take advantage of some system functionality. Usability is different from utility, a measure of whether that functionality does what is needed.

 

What do you think?

How do you provide Architecture/Design guidance to optimize for system Flexibility?

Comments

  • Anonymous
    October 20, 2006
    What is ETL?

  • Anonymous
    November 14, 2006
    Extract, Transform and Load.

  • Anonymous
    July 19, 2007
    In a previous blog , I posted some considerations to assist analyzing a business for opportunities to

  • Anonymous
    September 06, 2007
    Months ago I published a blog about how to implement system quality attributes ( found here ). In that

  • Anonymous
    July 01, 2008
    Welcome to the July 1, 2008 edition of Carnival of Enterprise Architecture (Issue #10). Business Process Management Stephan Grindley presents Enterprise Asset Management Software - a Great Way to Manage posted at Asset Management Articles. Bozidar Spirovski