Everything you ever wanted to know about pipelines but were afraid to ask (Part I)

  • Article

What are Pipelines

Commerce Server pipeline is a framework that links one or more components and executes them in sequence. The Pipeline framework provides stages to abstract one or more categories of work. The Pipeline Stage also determine the sequence in which each category of work is performed. Each Pipeline Component is COM object that is put inserted into a stage and is executed in sequence by the Pipeline Object. Pipelines are linked and it’s configuration stored in files Pipeline configuration files (.pcf).

Pipeline Execution

The following diagram displays the Basket Pipeline and the execution starts with the Product Info Stage and continues down to the last Stage Order Subtotal. You can add or delete Stages as well as move the sequence of a particular Stage.

 

Figure 1 Commerce Server Pipeline

Pipeline Properties

Use the Pipeline Properties dialog box to change the name of the pipeline, to specify its transaction compatibility, to specify its currency mode, or to save notes about the pipeline.

Figure 2 Pipeline Property

The following is a description of each property for the UI above.

Use this

To do this

Label

Type the name that the Pipeline Editor displays for the pipeline.

Transaction

Select which pipeline object that you can use the pipeline configuration file (.pcf) with. The valid options are as follows:

· Any pipeline. You can use the configuration file together with an OrderPipeline object.

· Requires transacted pipeline. You can use the configuration file together with transacted pipeline objects (PooledTxPipeline, MtsTxPipeline).

· Requires non-transacted pipeline. You can use the configuration file together with non-transacted pipeline objects (PooledPipeline, MtsPipeline).

Supply a currency LCID

Specify a locale ID (LCID) for currency handling. A value of zero (0) indicates to use the default locale ID of the server. The default currency LCID is zero (0).

Supply the number of decimal places used for currency rounding

Specify how many digits of precision to use for the currency. For example, you use two decimal places for United States pennies. Commerce Server uses this value for rounding purposes, and to determine the scaling factor to use in legacy currency keys.

Description

Type descriptive comments about the pipeline.

Pipeline Stages

The following diagram displays the Product Info Stage which consists of three Pipeline components. You can add or delete components using the Pipeline Editor. You can also move the individual components sequence.

Figure 3 Product Info Stage

Stage Properties

Use the Stage Properties dialog box to change the properties assigned to a pipeline stage.

Figure 4 Stage Properties

The following is a description of each property for the UI above.

Use this

To do this

Label

Specify the name that the Pipeline Editor displays for the stage.

Description

Type descriptive comments about the stage.

GUID

View the GUID that identifies the component on any computer. This ID never changes.

Mode

A value of zero (0) prevents the stage from running.

Error level

Specify the error level for the stage.

Note

When any component in the stage generates an error above the level specified in the Error level box, the stage skips the remaining components in the stage, and logs the event in the log file.

 

The pipeline maintains the maximum error level encountered by any component in the pipeline. It uses the Error Level setting on the stage to compare this maximum error level and only runs components in the stage if the current error level is less than or equal to the error level limit specified for the component.

Pipeline Component

A component is a Component Object Model (COM) object designed to perform some operation on a dictionary -- a collection of name/value pairs -- that is a different representation of a runtime object. This can be a dictionary that represents an OrderForm object or a dictionary that represents another type of object.

 

Figure 5 Query Catalog Info Pipeline Component

Components Properties

Use the Component Properties dialog box to change the name of a pipeline component, to view the class ID or Program ID of a component, and to save notes about a component. Use the Component Properties dialog box, Values Read and Written tab, to view the values that the component reads from and writes to the order form and the values it reads from the Context dictionary object. For descriptions of each value read and written, access the corresponding component from the Pipeline Component Reference. These two “Component Properties” and “Values Read and Written” are common to all Pipeline components.

Figure 6 Components Properties

The following is a description of each property for the UI above.

Use this

To do this

Label

Specify the name that the Pipeline Editor displays for the component.

Class ID

View the GUID that identifies the component on any computer. This ID never changes.

Program ID

View the easily recognizable ID for the component. This ID never changes.

Required

Check to make sure that the pipeline component is not removed from the pipeline.

Description

Type descriptive comments about the component.

 

Figure 7 Component Values Read and Written

The following is a description of each property for the UI above.

Note: The Pipeline Editor cannot display values read and written for components that do not support the IPipelineComponent Interface. We will discuss the Interfaces that the Pipeline components implement later on.

Use this

To do this

Values Read

View the values that the component reads from the order form before it performs its function.

Values Written

View the values that the component writes to the order form or other dictionaries after it completes its function.

Context Values Read

View the values that the component reads from the Context dictionary object. The Context dictionary object is passed as a parameter to the Execute method of the pipeline.

 

There are however some out of the box components that need additional configuration or other information, this data is displayed on additional property pages.

Properties Dialog Box Tab Name

Description

Eval Expressions Tab

Determines how the CSFEvalTargetGroups component evaluates targeting expressions for advertising, content selection, and discounts.

Execution Mode Tab

Determines how the PaymentMethodRouter component processes credit cards and whether to validate payment totals.

History Penalty Tab

Determines how the CSFHistoryPenalty component applies penalties to content items based on how recently they have been selected.

Load History Tab

Determines where the CSFLoadHistory component retrieves the history string.

Order Discount Tab

Determines how the OrderDiscount component applies discounts scheduled by the Marketing Manager to the shopping basket.

Order Level Discount Tab

Determines how the OrderLevelDiscountApply component applies order level subtotal discounts.

Payment Router Tab

Determine the behavior of the PaymentMethodRouter pipeline component when processing credit cards and validating payment totals.

PO to File Tab

Determines how the POtoFile component retrieves and sends a purchase order or other textual data to a file.

Record History Tab

Determines how many items the CSFRecordHistory component saves in the history list string, which is used to contain the identifiers of winning content items.

Save History Tab

Determines where the CSFSaveHistory component saves the history string.

Scriptor Tab

Stores the script to be run by the Scriptor component.

Select Winners Tab

Determines how the CSFSelectWinners component displays house ads.

Send SMTP Tab

Determines where the SendSMTP component sends an e-mail message and the information sent.

Splitter Tab

Determines how the Splitter component divides an order into groups based on a list of item keys.

Test SendSMTP Tab

Tests the configuration of the SendSMTP component by sending a test e-mail message.

 

Commerce Server provides five objects to execute pipelines:

1. MtsPipeline

Use this object to run a non-transacted Microsoft Commerce Server pipeline.

2. MtsTxPipeline

Use this object to run a transacted Microsoft Commerce Server pipeline configuration.

3. PooledPipeline

Use this object to run a Microsoft Commerce Server pipeline configuration file. The PooledPipeline object uses COM+ object pooling to greatly increase the loading speed of pipelines, where the contained pipeline components support object pooling.

4. PooledTxPipeline

Use this object to execute a Microsoft Commerce Server pipeline. Use in place of the PooledPipeline object where you need transactions. The PooledTxPipeline object uses COM+ object pooling to increase the loading speed of pipelines.

5. OrderPipeline

Use this object to run a Microsoft Commerce Server Content Selection pipeline. To create an Order Processing pipeline (OPP), use a PooledPipeline, PooledTxPipeline, MtxPipeline, or MtsTxPipeline object.

The MtsPipeline and MtxTxPipeline is recommended for development only. All other Pipelines are recommended for production. The reason for this is that the MtsPipeline and MtxTxPipeline objects do not load the configuration into memory and makes it easier to develop rather than stopping and starting IIS every time a change occurs.

Note: MicroPipe is no longer supported in Commerce Server 2007. In earlier version of the Commerce Server you could only execute the Pipeline component.

By default Commerce Server has the following Pipeline files:

1. Advertising.pcf

2. Basket.pcf

3. Chckout.pcf

4. Discounts.pcf

5. DMLPipe.pcf

6. Product.pcf

7. RecordEvent.pcf

8. Total.pcf

When unpacking the CSharp site you will also get the above Pipeline files. These Pipeline files can be used as template to start a new Commerce Site.

Note: Commerce Server 2007 does not have Pipeline templates.

Pipelines in a nutshell

So to summarize everything, Pipeline framework consists of COM objects that are linked and stored in pcf file format. You can further categorize pipelines into stages. Pipelines are executed by the Pipeline object and configured by Pipeline Editor. Pipelines can be executed transactional and be pooled for performance. Commerce Server has a dozen or more Pipeline components and several default pcf files containing one or more Pipeline components. You can also create your own Pipeline components.

How are Pipelines used in Commerce Server

Commerce Server has three types of Pipelines:

1. Order Processing

The Order Processing pipelines (OPP) are software infrastructures that link several components and run them in sequence on an OrderForm or Dictionary object. The Order processing Pipeline implement aspects of business-to-consumer and business-to-business checkout transactions such as discounts, Inventory checking, Product Catalog checking etc…

2. Content Selection Framework

The Content Selection Framework (CSF) is a platform-level framework for building targeted content delivery applications on the Internet. The targeted advertising and discount features of Microsoft Commerce Server are built by using the CSF. The framework provides the means for building new types of content delivery applications and makes it easy to customize and extend the advertising and discount delivery systems.

3. Direct Mailer

The Direct Mailer pipeline runs components that screen recipients and personalize e-mail, and then sends the personalized e-mail.

You can add third-party or custom components to the Commerce Server pipeline. Those components need implement only the standard pipeline component interfaces. They also, however, need to understand the dictionary keys that components in the pipeline use.

How Pipelines are executed

To execute a pipeline, you add the name of the pipe to the pipelines node for your site in the site Web.config file, see example bellow.

<pipelines siteName="CSharpSite">

  <pipeline name="basket" path="pipelines\basket.pcf" transacted="true" type="OrderPipeline" loggingEnabled="false"/>

</pipelines>

 

Each pipeline node must have the following attributes:

· Name - The name that you will reference in your application when executing the pipeline.

· Path - The relative path of the pipeline configuration file (.pcf).

· Transacted - Set this attribute to true or false to indicate the pipeline’s transacted state.

· Type - You can use either OrderPipeline or CSFPipeline for this attribute. You use an OrderPipeline to process orders and a CSFPipeline to display advertisements.

· LoggingEnabled - Set to true to enable logging. When you enable logging, the log file is created in the pipelines\log directory. You must also set appropriate permissions for the current w3wp.exe process to be able to create log files.

Sample code to execute a pipeline.

PipelineInfo info1 = new PipelineInfo(pipelineName, OrderPipelineType.Basket)

Basket.RunPipeline(info1);

 

Pipeline Types available.

public enum OrderPipelineType

{

      Custom,

      Product,

      Basket,

      Total,

      Checkout,

      AcceptBasket

}

 

This enumerator provides pipeline type information to PipelineInfo to allow an object to tailor its behavior to the requirements of the particular pipeline in use, without relying on the name of the pipeline.

The AcceptBasket type is used by the LOB Adapter.

When using the Custom type, the PipelineInfo constructor will not attempt to set up the pipeline context.

Member name

Description

AcceptBasket

Indicates an AcceptBasket pipeline (e.g. AcceptBasket.pcf).

Basket

Indicates a Basket pipeline (e.g. Basket.pcf).

Checkout

Indicates a Checkout pipeline (e.g. Checkout.pcf).

Custom

Indicates a custom pipeline type.

Product

Indicates a Product pipeline (e.g. Product.pcf).

Total

Indicates a Total pipeline (e.g. Total.pcf).

 

Summary

At this point you should have a good context of what Pipelines are and in the next part we will get a deeper understanding of Pipelines by developing and understanding Pipeline internals. Keep an eye out for part II of the “Everything you ever wanted to know about pipelines but were afraid to ask”.

Comments

  • Anonymous
    August 23, 2006
    In my last post which listed a number of great new Commerce Server posts I forgot to mention that Max Akbar [MSFT] - Product Manager (Maxter Chief) has started blogging with a vengeance!

  • Anonymous
    August 29, 2006
    Hi Max,

    I'm creating a new CheckInventory Stage, to resolve some custom needs of my client and I have a doubt.

    I need to access the Invetory Catalog of the product item but I can't find a way to use a Current Context. The workaround I create every-time a new Context for the product, but the performance is going down.

    Do you know some way to do this?

    Thank you,
    Willian

  • Anonymous
    August 30, 2006
    The comment has been removed

  • Anonymous
    August 31, 2006
    The StarterSite CTP is now available for download. This should help and get everyone started on a solid...

  • Anonymous
    September 14, 2006
    The comment has been removed

  • Anonymous
    September 17, 2006
    Hi Willian,

    You are so very close good job. I am about to post part III of this ongoing post and has the sample code you are looking for :).

    -Max

  • Anonymous
    September 17, 2006
    Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server Pipelines system.

  • Anonymous
    September 18, 2006
    The comment has been removed

  • Anonymous
    September 18, 2006
    Check to make sure you have an inventory resource? If you are not using the Inventory system remove the Inventory pipelines.

    Also check the eventlog to see if you have anything there.

    Good luck,
    -Max

  • Anonymous
    October 08, 2006
    Max Akbar [MSFT] has recently finished a series of really good posts about the Commerce Server...

  • Anonymous
    November 11, 2006
    As previously discussed the pipeline system enables sequential workflow processing. It is a COM-based...

  • Anonymous
    March 12, 2007
    The comment has been removed

  • Anonymous
    March 14, 2007
    The comment has been removed

  • Anonymous
    March 15, 2007
    The comment has been removed

  • Anonymous
    April 23, 2008
    Hi Max, I want to use direct mail feature of commerce server. But Direct mail is used basically for campaigning purpose. My requirement is to send an email to selected users after placing an order. (Order Confirmation Email/Shipping confirmation email). Can we achieve this using commerce server direct mail?

  • Anonymous
    April 24, 2008
    Direct Mail was not designed for this. Nothing is stoping you form doing this but I don't see why you would use it for this purpose. -Max

  • Anonymous
    August 26, 2008
    Hello Max, I need some information regarding the Commerce server pipelines. We are in the process of implementing pipelines in our project. We have started with Basket pipelines. We are just using the available pipelines with commerce server 2007. When I run the basket pipeline, I always get a warning message like, 'Message: basketError: At least one item is out of stock', although the item is in stock (Verified it by checking the InStockQuantity and Quantity of the Basket object). Can you please tell us what other properties this pipeline look for as it says the item is out of stock. Also we are running the pipeline after adding all the information to the basket like, lineitems, Shipping details, Payment details and just before executing the SaveAsOrder method. Is this the right place to call?

  • Anonymous
    September 02, 2008
    Turn on the pipeline log that sould give you more details. -Max

  • Anonymous
    November 24, 2008
    PingBack from http://blogbastlers.wordpress.com/2008/11/25/einkaufserlebnisse-masgeschneidert-webshop-auf-basis-der-commerce-server-startersite/

  • Anonymous
    May 28, 2009
    PingBack from http://paidsurveyshub.info/story.php?title=max-akbar-everything-you-ever-wanted-to-know-about-pipelines-but-were

  • Anonymous
    June 01, 2009
    PingBack from http://indoorgrillsrecipes.info/story.php?id=3164

  • Anonymous
    June 02, 2009
    PingBack from http://indoorgrillsrecipes.info/story.php?id=9304