State-Based Processing in MIIS 2003

Applies To: Windows Server 2003 with SP1

Download Instructions

This document is available for download as a Microsoft Word document at https://go.microsoft.com/fwlink/?LinkId=30737.

Overview

This guide describes how MIIS 2003 processes data objects during synchronization. It examines in detail the different data structures that are used to represent the states of objects during the synchronization process.

In this subject

• What Is MIIS 2003 State-Based Processing?

• How MIIS 2003 State-Based Processing Works

What Is MIIS 2003 State-Based Processing?

MIIS 2003 processes identity information from various connected data sources based on your business requirements. The processing includes requesting the identity information from various data sources during the staging process. Your business requirements are then applied to that information by translating them to synchronization rules that are used during the synchronization process. The processing concludes with the export of required changes. These processes, each of which can run independently, are shown in the following illustration. For example, information that is requested from a connected data source can be synchronized and then resynchronized based on new business requirements before MIIS 2003 exports any changes.

During each of the three processes, MIIS 2003 processes a specific state, that is, the condition of the identity information that MIIS 2003 has at a particular time. MIIS 2003 organizes the identity information in a way that allows it to calculate the current state of the identity information at any given point in the identity management process.

During the staging process, MIIS 2003 compares incoming data with the data that is already staged in the connector space. MIIS 2003 stores changes to identity information about the staging object in the connector space. Staging ensures that only identity information that has not been processed yet is flagged for further processing.

During the synchronization process, MIIS 2003 differentiates between updates to identity information and the information that has already been synchronized. Updates to identity information include new information that is received from a connected data source and new information that needs to be exported to the connected data source. This allows MIIS 2003 to either process updates only or reprocess all identity information that is available for an object.

MIIS 2003 usually synchronizes only incoming updates for subsequent synchronization runs unless you change the synchronization logic. For example, MIIS 2003 can ensure that the e-mail name of a user account is comprised of the first name and the initial of the last name. MIIS 2003 synchronizes only updates to user accounts unless you change the synchronization logic (in this case, what the e-mail name is comprised of). However, if the synchronization logic has changed, all of the identity information must be reprocessed because the changes can produce different synchronization results.

During the export process, MIIS 2003 exports to the connected data source identity information that has been synchronized and is staged for export. It exports information that has not yet been exported and information that requires re-exporting. For improved efficiency, only the minimum amount of identity information, which includes updates to individual attributes, is exported to the connected data source.

How MIIS 2003 State-Based Processing Works

State-Based Identity Information Representation

MIIS 2003 accomplishes state-based processing of identity information by storing for an object in a connected data source both a complete representation of identity information and new information, which MIIS 2003 uses to calculate each state within the identity management process. The complete identity information for a state is called a hologram; the corresponding subset that represents new information is called a delta.

Using the hologram and its corresponding delta, MIIS 2003 can calculate another hologram of that object that incorporates the delta information. This hologram is the representation of the object that will be stored in the connected data source after either staging, synchronization, or export completes. A data structure that consists of a pre-process hologram, a delta, and a post-process hologram is called a triple.

Storing state-based representations of identity information in the form of holograms and deltas has many advantages:

• It allows MIIS 2003 to divide the identity management process into independent subprocesses that focus on information that is most likely to change.

• It minimizes the amount of data that is processed.

• It allows MIIS 2003 to request identity information at any time without having to immediately process it.

• It allows changes to be applied to the synchronization logic without requesting data from the connected data source.

• It allows updates to be exported to a connected data source at any time.

• It allows only required changes to be exported and applied to the connected data source.

As another important advantage, state-based representation of identity information facilitates recovery of a connected data source from catastrophic failure. The connected data source can be repopulated with the current identity information that is stored in MIIS 2003, which is not possible by using identity management solutions that are not state-based.

MIIS 2003 maintains holograms and corresponding delta information for two process states:

• Inbound. Information for the inbound state includes all identity information that has been imported from the connected data source.

• Outbound. Outbound state information includes all required changes and identity information that has already been exported to the connected data source.

Inbound State Information

Inbound state information comprises all identity information that has been imported from the connected data source. To determine all subsequent deltas for the different subprocesses of the identity management process, MIIS 2003 must maintain at least one complete set of identity information for an object in a connected data source. This object is known as a synchronized import hologram or hologram.

A synchronized import hologram is the representation of a connected data source object that was used as input for a successful synchronization process. A synchronized import hologram is stored with the staging object that represents the connected data source object. MIIS 2003 uses the synchronized import hologram to identify new identity information that arrives for a connected data source object and that has not yet been staged. This information is called the delta pending import. The delta pending import also is stored with the staging object.

MIIS 2003 uses the information stored in the synchronized import hologram and the delta pending import in a triple to produce a new representation of the object, which reflects all of the identity information that was received for this object from the connected data source. The resulting hologram is known as a pending import hologram, as shown in the following illustration.

The pending import hologram also shows how the representation of the object in the connected data source appears after synchronization. As such, the pending import hologram represents the future hologram of an object.

Outbound State Information

Holograms and deltas also are used to represent the state of information for a given object that is outbound to the connected data source. MIIS 2003 uses outbound state information to ensure that only the information that needs to be exported to a connected data source is ultimately exported to it. MIIS 2003 recognizes three types of outbound information:

• Information that must be exported.

• Information that is in the process of being exported.

• Information that has been exported successfully, but whose export has not yet been confirmed by being reimported into MIIS 2003.

Each state has corresponding delta and hologram pairs that MIIS 2003 uses to calculate the object representation for each state in the connected data source.

When the outbound synchronization process produces new data that must be exported to the connected data source, MIIS 2003 calculates the delta information that needs to be staged on a staging object for the next export. This delta information is known as the delta unapplied export.

During the export process, MIIS 2003 exports new information to the connected data source. The values of the delta unapplied export that are processed during the export operation are copied into a data structure called the delta escrowed export. The difference between the delta unapplied export and the delta escrowed export is that the components of the delta unapplied export have never been part of an export operation.

The values of the delta escrowed export can vary depending on whether the management agent is call-based or file-based. For a file-based management agent, MIIS 2003 does not receive notification of the success or failure of the export from the connected data source. In this case, the values of the delta escrowed export are copied into a data structure called the delta unconfirmed export after the export process is complete.

If a call-based management agent is used for communication with the connected data source, the communication occurs by using APIs that are implemented by the connected data source. In this case, MIIS 2003 receives notification from the connected data source in the form of a return value that indicates whether the export operation was successful.

The values of the delta escrowed export for which a notification of success is received are copied into the delta unconfirmed export. If an error is received from a connected data source that uses a call-based management agent, the values of the delta escrowed export remain part of the delta escrowed export until the next successful export operation occurs.

For both file-based and call-based management agents, outbound state information is also adjusted during the staging process. The identity information in a new delta pending import is compared with the delta unconfirmed export and the delta escrowed export to determine whether exported identity information has been imported successfully from the connected data source.

Recall that the pending import hologram is the complete view of an object in a connected data source that includes all of the information that was received for this object. The delta unconfirmed export is comprised of the identity information for this object that was exported to the connected data source and for which MIIS 2003 has received a notification of success. However, this information has not been reimported yet.

When MIIS 2003 combines the pending import hologram and delta unconfirmed export, the result is a hologram that represents the complete object as it appears in the connected data source. This representation is called the unconfirmed export hologram. The unconfirmed export hologram is calculated in a triple, as shown in the following illustration.

By using the values of the unconfirmed export hologram and the delta escrowed export that is in process of being exported to the connected data source, MIIS 2003 can calculate another triple for the state of an object in the connected data source. This triple represents the resulting object when these changes have been successfully applied. The corresponding hologram is called the escrowed export hologram.

The escrowed export hologram is a representation of the object in the connected data source that includes:

• All identity information that was received for this object.

• All identity information that has been successfully exported to the object.

• All identity information that is in the process of being exported to the object.

By using the values of the escrowed export hologram and the delta unapplied export, which includes the identity information that is stored on a staging object for the next export process, MIIS 2003 can calculate the future representation of an object in the connected data source after the identity information that is waiting to be exported has been successfully applied to it. The corresponding hologram is known as the unapplied export hologram, as shown in the following illustration.

Calculating the State-Based Identity Information

The four triples (the unapplied export, the escrowed export, the unconfirmed export, and the pending import) form a data structure called the synchronization tower. MIIS 2003 builds the synchronization tower in memory whenever it needs to calculate the state of an object or update delta information.

Because inbound and outbound identity information are correlated to each other, when MIIS 2003 needs to determine one of its holograms or deltas, it has to calculate the components of the synchronization tower from the bottom to the top, as shown in the following illustration.

MIIS 2003 minimizes the amount of information stored in a staging object by saving only the synchronized import hologram and all available delta information, as shown in the following illustration. This information is sufficient for MIIS 2003 to determine the various states of how an object will appear in the connected data source depending on the progress of the identity management process.

The unapplied export hologram, the escrowed export hologram, and the unconfirmed export hologram are previews of the object in the connected data source only if that export identity information has been persistently applied to the object and the identity information that is in process of being exported or that is planned for export is successfully applied to the object eventually.

Updating State-Based Identity Information

The following illustration shows how MIIS 2003 calculates the different holograms during the identity management process.

The delta unapplied export is calculated during outbound synchronization. The delta escrowed export is calculated during export. The delta unconfirmed export is calculated after export to a call-based connected data source. The delta pending import is calculated during staging.

During the staging process, MIIS 2003 calculates the delta pending import of the identity information that is received from the connected data source. In addition, MIIS 2003 also uses the new delta pending import information to evaluate whether previously exported identity information has been successfully re-imported. As such, the delta pending import received from the connected data source also serves as confirmation that identity information that was previously exported to a connected data source has been successfully and persistently recorded in the connected data source.

When a call-based management agent is used, MIIS 2003 can receive success notification for exported changes. However, such a notification does not confirm that those changes have been applied persistently to the object within the connected data source because another process in the connected data source might have applied changes to identity information that was changed by MIIS 2003. Importing previously exported changes from the connected data source conclusively indicates to MIIS 2003 that the changes have been applied persistently to the object.

The synchronization process consists of two subprocesses, inbound synchronization and outbound synchronization, as shown in the following illustration.

Each of the synchronization subprocesses affects different components of the state-based identity information.

During inbound synchronization, MIIS 2003 processes the identity information of imported objects that were received from a connected data source. Because the synchronized import hologram is the representation of an object in the connected data source that was used as input for the inbound synchronization process, the synchronized import hologram has to be updated and an available delta pending import needs to be cleared after inbound synchronization is complete.

The outbound synchronization process can produce changes that must be exported to the connected data source. Those changes have to be staged on an export object while they wait for the next export process to begin. MIIS 2003 stages the changes by updating the delta unapplied export on the export object.

The export process affects the three export-related deltas:

• Delta unapplied export. The changes that occurred as a result of outbound synchronization.

• Delta escrowed export. The changes that MIIS 2003 has attempted to export to the connected data source.

• Delta unconfirmed export. The changes that MIIS 2003 has exported to a call-based connected data source and for which MIIS 2003 has received a success notification of the export.

Updating Inbound State-Based Identity Information

Inbound state-based identity information is comprised of all of the identity information for an object that MIIS 2003 has received from the connected data source. This information consists of:

• Synchronized import hologram. The identity information that has been previously used as input to the inbound synchronization process.

• Delta pending import. The new delta information that still must be applied during the next inbound synchronization process.

Inbound state-based information is changed during the staging process and the synchronization process. The following table summarizes the inbound state-based identity information that is changed during these processes.

Changes Made to Inbound State-Based Identity Information

Inbound Changes Applied During the Staging Process

MIIS 2003 always calculates the delta for identity information received from a connected data source by comparing the identity information that it receives with the synchronized import hologram that is stored on a staging object.

When MIIS 2003 requests information from the connected data source, it limits the requested data to the object types specified in the list of designated object types and the attributes in the attribute inclusion list. Designated object types and the attribute inclusion list are specified during configuration of the management agent. MIIS 2003 builds an in-memory representation, also known as an import image, of each object that was received from the connected data source.

If this import image does not have a corresponding staging object, an imported object is created in the connector space, as shown in the following illustration.

To calculate the delta pending import during the staging process, MIIS 2003 has to compare the import image with the synchronized import hologram of the staging object. The difference between the synchronized import hologram and the import image is the delta pending import, as shown in the following illustration.

During the staging process, MIIS 2003 stores the delta pending import with the staging object and sets a flag that indicates if new pending information is available.

Inbound Changes Applied by the Synchronization Process

MIIS 2003 supports two different types of synchronization:

• Delta synchronization

• Full synchronization

Only staging objects with attributes that have not been processed yet — that is, objects with a delta pending import — are synchronized during delta synchronization. During full synchronization, all attributes in the pending import hologram are synchronized.

For both delta and full synchronization, the synchronization process consists of two subprocesses: inbound synchronization and outbound synchronization. Inbound synchronization is the first subprocess started by the synchronization process and it the only process that is relevant to the inbound state of a staging object. If no error occurs during inbound synchronization, the delta pending import is cleared and the content of the synchronized import hologram is replaced with the content of the pending import hologram.

Updating Outbound State-Based Identity Information

Holograms and deltas also are used to ensure that only the information that needs to be exported to a connected data source is exported. There are three different export states for an export object:

• Delta unapplied export. The changes that occurred as a result of outbound synchronization.

• Delta escrowed export. The changes that MIIS 2003 has attempted to export to the connected data source.

• Delta unconfirmed export. The changes that MIIS 2003 has exported to a call-based connected data source and for which MIIS 2003 has received a success notification of the export.

Outbound state-based information is changed during the staging process, the synchronization process, and the export process. The following table shows the outbound state-based identity information that is changed during each of these processes.

Changes Made to Outbound State-Based Identity Information

Outbound Changes Applied During the Staging Process

During the staging process, MIIS 2003 updates outbound state information. To calculate the changes to outbound state-based identity information, MIIS 2003 must first clear the delta escrowed export and the delta unconfirmed export, which were calculated by using a previous delta pending import.

Although changes might have been successfully applied to the connected data source, there is no guarantee that these changes are persistent. MIIS 2003 is usually only one authority with appropriate rights to apply changes. These changes can be overwritten either manually by the administrator of the connected data source or another service that is running on the connected data source. The identity information that is imported from the connected data source indicates whether the changes applied by MIIS 2003 are persistent.

To determine whether the changes applied by MIIS 2003 are persistent, the export status information that has been used during the export process (delta escrowed export, delta unconfirmed export) must be compared to the delta pending import, as shown in Figure 22.

Outbound Changes Applied During the Synchronization Process

Only during outbound synchronization, which is part of the synchronization process, does MIIS 2003 update outbound identity information about a staging object.

Whenever new data that is intended for export to the connected data source is generated during the outbound synchronization process, MIIS 2003 creates an in-memory representation of it as an export image. The export image must be compared with the escrowed export hologram. The escrowed export hologram is a representation of the object in the connected data source that includes:

• All identity information that has been received from the connected data source for this object.

• All changes that already have been exported to the connected data source.

• All changes that are in the process of being exported to the connected data source.

The escrowed export hologram is calculated in memory. The difference between the export image and the escrowed export hologram is the delta unapplied export. The delta unapplied export must be staged on the export object for the next export operation, as shown in the following illustration.

Outbound Changes Applied During the Export Process

Outbound updates that are applied by the export process have to account for all deltas that have been included in any previous export process. When the export process is initiated, the value of the delta unapplied export is copied to the delta escrowed export. When a call-based management agent is used, if the export process succeeded without error or interruption, the content of the delta escrowed export is copied into the delta unconfirmed export and the value of the delta escrowed export is cleared, as shown in the following illustration.

The delta pending import values that are received during staging and that are equal to values in the delta unconfirmed export can be removed from the delta unconfirmed export after a success notification is received from a call-based connected data source because those values are confirmed now by the delta pending import, as shown in the following illustration.

If the export process is interrupted, MIIS 2003 stores the export information differently depending on whether the changes that are being exported are additions to the connected data source or are only changes to existing objects in the connected data source. If the export of changes is interrupted, MIIS 2003 logs an error, then it moves the changes back to delta escrowed export, and flags the changes with an error.

If the export of additions is interrupted, MIIS 2003 moves the changes to delta unconfirmed export. The changes remain there until the next export process begins. MIIS 2003 moves the changes to delta unconfirmed export because additions to the connected data source can consume significant resources.

Because delta unconfirmed export contains a value only if a notification of success is received from a call-based management agent, if delta unconfirmed export contains no value, an error has occurred.

If a file-based management agent is used, all delta escrowed export values remain unchanged.