BizTalk Server: Using BAM in the ESB Toolkit

Introduction

BAM is a very good way to monitor a process in BizTalk but often there is not enough time in a BizTalk project to implement BAM because it can take some time to implement it. The advantage of the ESB Toolkit is that you get (some) BAM out of the Box.

Normally you first have to create a BAM definition in Excel and after that deploy it with the command line tool BM. Also, you have to use the Tracking Profile Editor tool to map the activity definition to a process in BizTalk. With the installation of the ESB Toolkit, there are already some activity definitions created. The logic behind the mapping is also done so you don’t have to use the Tracking Profile Editor anymore. You only have to deploy the BAM definitions so now you can enable BAM tracking very easily.

The following activity definitions are available in the ESB Toolkit:

• Itineraries
  • Information about itineraries that are processed in BizTalk.
• Exceptions
  • Information about Exceptions that occur in BizTalk. 

The tracked BAM data give you answers on common questions such as:

• How many itineraries/services worked in past ?
• How many itineraries/services currently running ?
• What Exceptions have occurred in BizTalk?

Tracking Itineraries

Tracking Itineraries in BAM can be enabled in the Itinerary Designer Surface in Visual Studio. The Itinerary Designer Surface is a visual designer that is used to create Microsoft BizTalk ESB Toolkit itineraries and to export these itineraries into run-time format. 

Runtime are the following fields as an activity record added in the the BAM Primary Import database:

• ActivityID
• InterchangeId
• ItineraryBeginTime
• ItineraryEndTime
• ItineraryisRequestResponse
• ItineraryState
• Itineraryuuid
• MessageDirection
• ServiceisRequestResponse
• ServicePosition
• ServiceType
• ServiceState
• Serviceuuid
• ServiceIdGuid
• TimeStamp
• ServiceBusinessName
• ItineraryName
• ItineraryVersion
• ESBServiceName
• LastModified

Steps

To track an Itinerary Service enable the Tracking Enabled property in the properties window of the Itinerary Service. 

Figure 1. Itinerary Designer Surface in Visual Studio.

Note

You have to do this for all the Itinerary Services in your Itinerary. Every Itinerary Service that is tracked will result in a row in the bam_ItineraryServiceActivity_Completed table in the BAM database.

• Deploy the Itinerary.
• Restart Host Instances.

Tracking Exceptions

The ESB Exception Management Framework takes advantage of Business Activity Monitoring (BAM). This means that the ESB Exception Management Framework can track the exception data points with BAM, and then publish them to the BizTalk BAM Portal for monitoring.

When an error occurs, the following fields are as an activity record added in the the BAM Primary Import database:

• Application
• Description
• FaultSeverity
• ServiceName
• ErrorType
• FaultCode
• MachineName
• MessageID
• DateTime
• FaultDescription
• Scope
• FailureCategory
• FaultGenerator
• ServiceInstanceID 

The Microsoft BizTalk ESB Toolkit installs a Send Port named ALL.Exceptions that subscribes to all ESB fault messages and to all messages generated by the BizTalk Failed Message Routing mechanism.

Figure 2. ALL.Exceptions Send Port in Microsoft.Practices.ESB application.

ALL.Exceptions uses the ESBFaultProcessor Send Pipeline. The Fault Processor Pipeline BAM Tracker Component in the ESBFaultProcessor send pipeline adds the exception data to the BAM Primary Import database. Use the Enabled property to determine whether the component will process the message and write it to the BAM database. When set to False, the component just sends the message to the next component in the pipeline.

Figure 3. ESBFaultProcessor Send Pipeline properties.

Install BAM for the ESB Toolkit

Prerequisites

To use BAM in the ESB Toolkit solutions, you must install the following prerequisite applications and services. Follow this sequence to install the prerequisites and to configure your system.

1. Check if SQL Server Analysis Services is installed with the SQL Server Configuration Manager tool. Figure 4. SQL Server Configuration Manager tool.
2. Check if the BAM Analysis Database is installed in the BizTalk Server Configuration tool. Figure 5. BizTalk Server Configuration tool.

Install BAM definitions

• Install BAM itinerary tracking activity definition.

  BAM itinerary tracking can be used to track the progress and eventual completion of itineraries as they advance through various steps. The BAM definition file (Microsoft.BizTalk.ESB.BAM.Itinerary.xml) is located in the BAM folder in the installation directory and can be installed using the Bm.exe tool. Use the following command as an example.

   

  Bm.exe deploy-all -DefinitionFile:" C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\Bam\Microsoft.BizTalk.ESB.BAM.Itinerary.xml"
  

  

  Figure 6. Bm.exe tool.

• Install Exception Handling BAM tracking.

  The BAM definition file (Microsoft.BizTalk.ESB.BAM.Exceptions.xml) is located in the BAM folder in the installation directory and can be installed using the Bm.exe tool. Use the following command as an example.

  Bm.exe deploy-all -DefinitionFile:" C:\Program Files (x86)\Microsoft BizTalk ESB Toolkit\Bam\Microsoft.BizTalk.ESB.BAM.Exceptions.xml"
  

   

  

  Figure 7. Bm.exe tool.

To verify message tracking

You can verify the BAM results using a SQL query or the BAM Portal.

• SQL query

  Start SQL Server Management Studio and create a query on one of the ESB Toolkit views in the BAMPrimaryImport database.

  • Views on Exception tables:
    • bam_EsbExceptions_ActiveInstances
    • bam_EsbExceptions_AllInstances
    • bam_EsbExceptions_AllRelationships
    • bam_EsbExceptions_CompletedInstances
    • bam_EsbExceptions_InstancesForArchive
    • bam_EsbExceptions_RelationshipsForArchive
  • Views on Itinerary tables:
    • bam_ItineraryServiceActivity_ActiveInstances
    • bam_ItineraryServiceActivity_AllInstances
    • bam_ItineraryServiceActivity_AllRelationships
    • bam_ItineraryServiceActivity_CompletedInstances
    • bam_ItineraryServiceActivity_InstancesForArchive
    • bam_ItineraryServiceActivity_RelationshipsForArchive

  

  Figure 8. SQL query in SQL Server Management Studio.

• BAM Portal

  Start the BAM Portal from the Windows Start Menu and create a query on the BAM views.

  

  Figure 9. BAM Portal.

Creating a custom BAM Dashboard

With pure ESB runtime you don't have reports on the BAM data. You can edit the BAM definition files to create views on the BAM activities but another possibility is to create the charts yourself. 

I've created a custom BAM Dashboard in ASP.NET for the ESB Toolkit that shows information about:

• Itinerary Execution Times
• Processed Itineraries
• Running Itineraries
• Running ItineraryServices
• Exceptions in BizTalk Applications
• Exceptions Occurred Per Service

Figure 10. Custom BAM Dashboard in ASP.NET. 

Steps

1. Create a SQL query on the BAM data.

   In the following example is the bam_ItineraryServiceActivity_CompletedInstances view used to determine the execution times of the itineraries.

   

   Figure 11. SQL query in SQL Server Management Studio.

2. Create a Web Site application in Visual Studio. Figure 12. New Web Site application in Visual Studio.

3. Create a class in ASP.NET to get the data from the database and put it in a List.

    

   public class  BamManager
       {
           public List<ItineraryData> GetExecutionTimes()
           {
               List<ItineraryData> list = new  List<ItineraryData>();
    
               // Get ConnectionString & SQL Query from web.config
               string connectionString = ConfigurationManager.ConnectionStrings["BAMPrimaryImport"].ToString();
               string sqlQuery = ConfigurationManager.AppSettings["ExecutionTimes"];
    
               using (SqlConnection connection = new SqlConnection(connectionString))
               using (SqlCommand command = new SqlCommand(sqlQuery, connection))
               {
                   connection.Open();
                   using (SqlDataReader reader = command.ExecuteReader())
                   {
                       while (reader.Read())
                       {
                           ItineraryData iData = new  ItineraryData();
                           iData.Name = reader["ItineraryName"].ToString();
                           iData.ExecutionTime = int.Parse(reader["ExecutionTime"].ToString());
                           iData.Date = DateTime.Parse(reader["ItineraryEndTime"].ToString());
    
                           list.Add(iData);
                       }                    
                   }
               }
    
               return list;
           }
   

     

4. From the Data category in the Toolbox, drag a Chart control on the web page.

   

   **Figure 13. **Visual Studio Toolbox

5. Customize the Chart appearance.

    - Add a Legend to the Chart.

    - Add a Title to the Chart.

   

   Figure 14. ASP.NET Chart.

6. In the Code-behind put the data from the List in the Chart. 

    

   public void  FormatExecutionTimesChart()
   {
       // Get data from BAM database
       BamManager manager = new  BamManager();
       List<ItineraryData> executionTimes = manager.GetExecutionTimes();
    
       // Get distinct Itinerary names
       IEnumerable<string> DistinctItineraryNames = executionTimes.Select(x => x.Name).Distinct();
    
       // Create Series
       foreach (string value in DistinctItineraryNames)
       {
    
           if (!string.IsNullOrEmpty(value))
           {
               Series s = ExecutionTimesChart.Series.Add(value);
               s.ChartType = SeriesChartType.Line;  
               s.BorderWidth = 2;
               s.MarkerStyle = MarkerStyle.Square;
               s.YValueType = ChartValueType.Int64;
           }
       }
    
       // Add Points to the Chart
       foreach (ItineraryData iData in executionTimes)
       {
           if (!string.IsNullOrEmpty(iData.Name))
           {
               Series s = ExecutionTimesChart.Series.FindByName(iData.Name);
    
               int index = s.Points.AddY(iData.ExecutionTime);
               s.Points[index].LegendText = iData.Name;
               s.Points[index].ToolTip = iData.Date.ToString();
           }
       }
   }
   

Wrap up

BAM is a very good way to monitor a process in BizTalk and with the ESB Toolkit you can implement BAM without having to do a lot of effort. Out of the box you can monitor itineraries that are processed in BizTalk and monitor Exceptions that have occurred in BizTalk. With pure ESB runtime you only don't have reports on the BAM data. To solve this you can edit the BAM definition files to create views on the BAM activities but another possibility is to create the charts yourself.

Downloads

You can download the sample tool with the source code here:

• Creating a custom BAM Dashboard for the ESB Toolkit

See Also

Read suggested related topics:

• BizTalk Server: ESB Survival Guide

Another important place to find an extensive amount of BizTalk related articles is the TechNet Wiki itself. The best entry point is BizTalk Server Resources on the TechNet Wiki.