Integration Product Options on the Microsoft Platform

Work in Progress!

↑ Return to Top

Introduction

Scope

This article describes which products are designed to address various integration patterns, scenarios and requirements. It also describes when these products “could, but should not” be used. Most large organisations will use a mix of a few of these products. But the time may come when their current products do not fit well with new requirements.  This article does not attempt to perform a full architecture section analysis. However, this article will help to identify candidate products. 

This article includes:

  • Microsoft products.
  • Microsoft Gold partner products.
  • Popular products used on the Microsoft platform.

↑ Return to Top

Other Resources

  • Book Applied Architecture Patterns on the Microsoft Platform by Richard Seroter provides an excellent “Architecture Selection Framework” to help organisations choose which products is the best fit for their requirements.  The selection framework is based on 4 criteria: Design, Delivery, Organisation, and Operations.  The book then goes on to use this selection framework to identify the best candidate products in various (mostly integration) scenarios.  The following candidate products were evaluated: BizTalk (inc. ESB Toolkit), AppFabric (WCF, WF, Cache), SSIS, SSBS, Master Data Services, StreamInsight, and Azure Cloud Services.
  • Video: Richard Seroter (from the BizTalk Summit 2014) : When To Use What: A Look at Choosing Integration Technology.  "There’s never been more options — or more confusion — about which technologies to use when connecting systems? When is BizTalk Server the right choice? Should I still use WCF? Is the Windows Azure Service Bus ready for production scenarios? In this session, we’ll first do an overview of each core technology in the integration portfolio. Then we’ll review a simple decision framework for down-selecting your choices. Finally, we’ll have an interactive discussion (and demonstration) of real-life scenarios and which technology is the right fit."
  • Video: Richard Seroter: The future of integration.
  • White Paper "Guidance for integration architecture on the Microsoft Application Platform (2014 edition)" by Gijs in 't Veld provides advice on "Architecting integration solutions using Microsoft technology now and in the future, on-premise and in the cloud."

↑ Return to Top

EAI - Enterprise Application Integration (Message Broker \ Enterprise Service Bus)

Microsoft BizTalk Server

Designed For
  • Enterprise Application Integration (EAI)
  • Business to Business Integration (B2B) supporting a wide variety of industry message standards (inc: RosettaNet, HL7) and EDI processing.
  • Support for a wide variety of integration patterns.
  • Large variety of adapters.
  • Reliability, availability, scalability.
  • Hybrid applications.
  • Single server options for small organisations to heavyweight multi-server integration platform for large organisations.
Not Recommended For
  • Very low latency services.
  • Very large messages.
  • No built-in failed message retry mechanism (need to design\build custom processes).
  • Message broadcast to 100's of target subscribers (need to work with other products).
Resources

↑ Return to Top

Microsoft BizTalk Server with ESB Toolkit

Designed For
  • Service Orientated Architecture
  • Adds the ‘Routing Slip’ pattern to BizTalk with transform service and routing to services or processes (orchestrations).
  • Supports both asynchronous and synchronous messaging.
Not Recommended For
  • Very low latency services.
  • Not ESB in a box - adds significant complexity to BizTalk implementations.
  • Exception Management Portal is just a sample. Needs customisation.
Resources

↑ Return to Top

Microsoft Windows Azure BizTalk Services

Designed For
  • Enterprise Application Integration (EAI)
  • Business to Business Integration (B2B) including EDI processing.
  • Cloud-scale hosting
  • Hybrid applications.
Not Recommended For
Resources

↑ Return to Top

MuleSoft AnyPoint

Designed For
  • Cloud platform for SOA, SaaS integration and APIs
Not Recommended For
  • .
Resources

↑ Return to Top

SnapLogic

Designed For
  • Integration Platform as a Service for both real-time and batch
Not Recommended For
Resources

↑ Return to Top

ETL - Extract Transform Load (Batch \ Data Integration)

Microsoft SQL Server Integration Services

Designed For
  • ETL (Export Transform Load)
  • Combining and transforming data from diverse sources into SQL Server.
  • Data Migration and Data Cleansing.
  • Large messages.
  • ERP Connectors.
Not Recommended For
  • Real time integration - Data Lag.
Resources

↑ Return to Top

Microsoft SQL Server Replication

Designed For
  • SQL Server data synchronization with copies of that data in other instances of SQL Server, Oracle, or DB2.
Not Recommended For
Resources

↑ Return to Top

Microsoft Windows Azure Data Sync

Designed For
  • Synchronize data across multiple SQL Database instances.
  • A sync group must have at least one Azure SQL Database instance which serves as the sync group hub in a hub-and-spoke topology.
Not Recommended For
Resources

↑ Return to Top

Informatica Cloud

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

Scribe Insight

Designed For
  • ETL (Export Transform Load)
  • Database and ERP adapters.
Not Recommended For
  • Application integration via web services.
Resources

↑ Return to Top

CQRS - Command Query Responsibility Segregation

Particular Software nServiceBus

Designed For
  • CQRS pattern (eventual consistency).
  • Highly scalable.
  • Loose-coupling with publish/subscribe communication - like .net events but asynchronously invoked across the network. Transparent addition of new subscribers
  • Centralised or Distributed, N-Tier or Peer-to-Peer, In-process or across the network.
  • Fault-tolerance by default - all messaging is enlisted into the same transaction.
Not Recommended For

"CQRS should not be your top-level architectural pattern – that would be SOA. CQRS, if used at all, would be used inside a service boundary only."

Resources

↑ Return to Top

MassTransit

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

BPM - Business Process Management (Workflow)

Microsoft BizTalk Server Orchestrations

Designed For
  • Application integration processes.
Not Recommended For
  • Human workflow.
Resources

↑ Return to Top

Microsoft Azure Workflow Manager

Designed For
  • Cloud scale hosting of Windows Workflow Foundation (4.5).
Not Recommended For
Resources

↑ Return to Top

Microsoft Windows Server AppFabric

Designed For
  • On-Premise hosting of Windows Workflow Foundation (plus WCF and Cache).
  • WCF - routing service, various protocol
  • WF -  from simple sequential workflows to complex state machine-based workflows with human interactions.
  • Workflow service - guaranteed delivery
  • AppFabric (SQL Server) - persistence points and tracking
  • AppFabric (IIS) - Load balancing \ high availability
Not Recommended For
  • No native transformation service
  • Low level exception handling.
  • Only primitive routing rules
Resources

↑ Return to Top

Microsoft SharePoint Workflow

Designed For
  • Business processes for (Microsoft Office) document management (e.g. collaboration, review, sign-off).
Not Recommended For
Resources

↑ Return to Top

Microsoft System Center Orchestrator

Designed For
  • Orchestrator is a workflow management solution for the data center. Orchestrator lets you automate the creation, monitoring, and deployment of resources in your environment.
  • Integration Packs are available for:
    •  Active Directory
    • Exchange
    • SharePoint
    •  FTP
    • REST
    •  Azure
    • System Centre
    •  VMware vSphere
    • .... many others
Not Recommended For
Resources

↑ Return to Top

K2 Workflow

Designed For
  • Human workflow.
  • Strong SharePoint integration.
Not Recommended For
Resources

↑ Return to Top

PNMSoft Sequence

Designed For
  • <todo>
Not Recommended For
Resources

Return to Top

Identity Integration

Microsoft Active Directory Federation Services

Designed For
  • <todo>
Not Recommended For
Resources

Return to Top

Microsoft Windows Azure Active Directory

Designed For
  • Now integrates with 227 different SaaS solutions (Office 365, SalesForce, Box, Concur, Google Apps, and more).
Not Recommended For
Resources

Return to Top

Microsoft Directory Sync

Designed For
  • One-way synchronization from an organization’s on-premises Active Directory to Windows Azure Active Directory (Windows Azure AD).
Not Recommended For
Resources

Return to Top

Microsoft Forefront Identity Manager

Designed For
  • Synchronising and consolidating identity information between disparate identity stores
  • Role-based access control
  • Portal-based governance and compliance for user identity
  • Self-service identity management
Not Recommended For
Resources

Return to Top

See also

↑ Return to Top

Services

Microsoft Windows Communication Foundation

Designed For
  • Unified programming model for building service-oriented applications.
  • Variety of transports: HTTP, TCP, IPC (named pipes), MSMQ.
  • Variety of HTTP protocols: SOAP, REST, OData and even PoX.
  • Instance and concurrency management, transactions, security, and discovery.
Not Recommended For
Resources

↑ Return to Top

Microsoft ASP.NET WebAPI

Designed For
  • HTTP REST services.
  • Version 2 supports: ODATA, OAuth, OWIN
Not Recommended For
Resources

↑ Return to Top

Microsoft ASP.NET SignalR

Designed For
  • ASP.NET SignalR is a library for ASP.NET to enable real-time web functionality. SignalR allows bi-directional communication between server and client. Servers can now push content to connected clients instantly as it becomes available. SignalR supports Web Sockets, and falls back to other compatible techniques for older browsers. SignalR includes APIs for connection management (for instance, connect and disconnect events), grouping connections, and authorization.
Not Recommended For
Resources

↑ Return to Top

Microsoft Azure Mobile Services

Designed For
  • Push notifications to mobile applications (plus windows store apps).
  • Can use SQL storage (default), table storage, BizTalk, etc
  • Social identity provider integration (OAUTH)
  • Can channel to specific devices or broadcast through integration with the Azure Notification Hub.
Not Recommended For
Resources

↑ Return to Top

Microsoft Azure Notification Hubs

Designed For
  • Broadcast push notifications to mobile applications.
Not Recommended For
Resources

↑ Return to Top

Microsoft SQL Server Master Data Management Services

Designed For
  • Central data hub that ensures the integrity of information and consistency of data is constant across different applications.
Not Recommended For
Resources

↑ Return to Top

ServiceStack

Designed For
  • .
Not Recommended For
  • .
Resources

↑ Return to Top

API

Microsoft Azure API Management

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

Queues

Microsoft MSMQ

Designed For
  • On-premise only.
Not Recommended For
  • Green field solutions.  The queues in 'Service Bus for Windows Server' is the successor to MSMQ.
Resources

↑ Return to Top

Microsoft Azure Queue Storage

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

Microsoft Azure Service Bus Queues and Topics

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

Pivotal RabbitMQ

Designed For
  • <todo>
Not Recommended For
Resources

↑ Return to Top

Microsoft SQL Server Service Broker

Designed For
  • Connecting SQL Server applications with other SQL Server applications using queued messaging.
Not Recommended For
  • Real Time integration - extensive customisations required.
Resources

↑ Return to Top

Other

Microsoft StreamInsight

Designed For
  • Complex event processing (CEP) applications. Monitor your streamed data sources for meaningful patterns, trends, exceptions, and opportunities. Alert on defined KPI’s.
Not Recommended For
Resources

↑ Return to Top

Notes on the Azure and Windows Service Bus

Microsoft Azure Service Bus

Designed For
  • Queues
    • Reliable message storage and retrieval with a choice of protocols and APIs.
    • Competing receivers pattern.
    • Other features: duplicate message detection; scheduled message delivery; performance optimization (using asynchronous calls, batching and prefetching of messages).
  • Topics
    • Publish-subscribe capabilities allowing multiple, concurrent subscribers to independently retrieve filtered or unfiltered views of the published messages.
  • Relay
    • Enable connections across network boundaries (e.g. firewalls).
    • The NetEventRelayBinding binding allows multicasting (multiple clients listening on a single endpoint)
  • Event Hubs
    • Highly scalable publish-subscribe ingestor.
Not Recommended For
  • Not yet an “ESB in the cloud” for general purpose integration.
  • No error handling/auditing - requires custom code in client\service layers.
  • Topics only have lightweight routing.
Resources

↑ Return to Top

Microsoft Service Bus for Windows Server

Designed For
  • On-premise implementation of the Azure Service Bus features.
  • Development symmetry between with the Windows Azure Service Bus making it easy to develop applications for either and switch between the two.
  • Staged adoption of Azure capability. Services can be built and deployed locally to allow for debugging and testing of applications with much more control before publishing to a public cloud.
Not Recommended For
  • New features are built in Azure Service Bus first.
Resources

↑ Return to Top

See Also

Another important place to find a huge amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.