Understanding Exports in ILM 2007

  • Article

“I’ve tried to export data to a connected data source, didn’t get an error but nothing happened – no data was exported”.
If you have been in a situation like this, you might want to read this article.
In this article, I’m addressing some of the most popular questions I found in conjunction with export related problems on this forum.
Please feel free to provide feedback if you have questions or other feedback.

Understanding Exports

The objective of an export run is to update connected data source with physically authoritative information that was staged for being exported in the connector space of a management agent.
In other words, to export data to a connected data source, you need staged updates.

If no new data was applied to a connector space during a synchronization run and a subsequent export although you have expected it and you didn’t get an error during the export operation, it doesn’t necessarily mean that your system is not functioning correctly.
One common question, I have seen over the time been often asked in this context is like the following: “I imported from forest A and ran an export to forest B. How come no objects were exported to forest B? What is wrong with my export?”

In a case like this, there is not necessarily anything wrong with your exports.
It is also possible that your synchronization rules are not configured properly – which means, the system works correctly, no errors are reported but it doesn’t produce the desired results.
For example, it is possible that a configured attribute flow doesn’t have the required precedence to flow an attribute value to a target.

To track down why an export that should export data didn’t export data, you should perform the following steps:

  • Verify the export status
  • Analyze a synchronization run

 

Verifying the Export Status

If you have an issue related to exporting data, you should first verify, whether there anything staged for an export.

 

Note

You should always first verify the export statistic prior to running your first export to a connected data source!

 

To verify the export statistics, you have three options:

  • Check the statistics of a management agent in Identity Manager
  • Search for pending exports in a connector space
  • Verify the export statistics by running a script

 

Checking the management agent statistics

In ILM, the Identity Manager has a statistics dialog with related data for all management agents that are currently configured in your environment.
This includes:

  • Counters for the objects that are staged in a connector space
  • Export and import states of the objects in a connector space

To open the Statistics dialog:

  • In ILM, open Identity Manager
  • To open the Statistics dialog, click Statistics in the Tools menu

http://3pji8g.bay.livefilestore.com/y1pFRitvB2RJIO3S4ymLhs_QdodJEyikFKOGbsdn283197g6pDtEihr686zqrsFKYg11_FUNRFYdjVqVji8aImCiZCdE_GH5hmx/GHAMV27.jpg

 

Searching for pending exports in a connector space

While the management agent statistics only gives you the general counter for various states in your environment, you can use the Search Connector Space feature to list the affected objects.

The Search Connector Space feature enables you to perform a scope based connector space searches.
In conjunction with exports, you can set the scope to pending exports and select the type of pending operations (add, Modify, Delete) you care about.

To perform a scope based connector space search:

  • In ILM, open Identity Manager
  • To switch to the Management Agents view, select Management Agents from the Tools menu
  • Form the management agent list, select a management agent
  • To open the Search Connector Space dialog, select Search Connector Space from the Tools menu
  • From the Scope list, select Pending Exports
  • Select the pending type (Add, Modify, Delete) you are interested in
  • To perform a search, click Search

 

http://3pji8g.bay.livefilestore.com/y1pI2iABy0ysKWoSrP8n0zv897HR1eqRgKDNTH6Ao3n2A-1iFv-pEV63zDGKEoeV2YLYuE5NziqxtkdEq2759zPQw/GHAMV28.jpg

Note

You should familiarize yourself with the various scope options the Search Connector Space feature provides as they can help you to get more targeted information in troubleshooting situations.

 

Verifying the export statistics by running a script

Scripts provide a convenient method to retrieve the information you need in a targeted manner.
ILM provides a WMI interface you can use to extract the required information from the system.
Part of this interface is the MIIS_ManagementAgent agent class that gives you access to properties of a management agent.
To review the export statistics of a management agent, you check the related Add, Update and Delete parameter.
The following code block shows an example for a script that displays the export statistics of a management agent.

Option Explicit
Dim retVal
retVal = InputBox("Please enter the name of the management agent", Replace(WScript.ScriptName, ".vbs", "")) 
If(0 = Len(retVal)) Then
   WScript.Quit
End If   

Dim oService, lstMA, curObject
Set oService = GetObject("winmgmts:root\MicrosoftIdentityIntegrationServer")
Set lstMA = oService.ExecQuery("Select * from MIIS_ManagementAgent Where Name ='" & retVal & "'")
If(0 = lstMA.count) Then
   MSgBox "Management agent " & retVal & " not found.", vbInformation, Replace(WScript.ScriptName, ".vbs", "")
   WScript.Quit
End If


Dim retMsg
For Each curObject In lstMA
   retMsg = "Adds: "    & vbTab & curObject.NumExportAdd & vbCRLF & _
            "Updates: " & vbTab & curObject.NumExportUpdate & vbCRLF & _
            "Deletes: " & vbTab & curObject.NumExportDelete
Next

retMsg = "Export status on " & retVal & vbCRLF & retMsg 
MsgBox retMsg, vbInformation, Replace(WScript.ScriptName, ".vbs", "")

Note

You can use this script prior to an export run to determine whether export updates are staged on a management agent

 

While technically possible, backtracking why a desired export status was not accomplished can be a pretty time consuming and complex operation that doesn’t necessarily help to find the answer you are looking for quickly.

In most cases, you are better off using a push approach.
This means, processing a test object with prepared data through your system.
Since the export status is a result of a complete transaction that starts with staged changes in a connector space, which are processed through your configuration of your synchronization rules, you should prepare an object in a connected data source with specific test data and import these changes into the connector space.

ILM provides a very powerful feature called “Preview” that enables you to apply a “What If” run of an object without actually updating any data inside ILM.
In other words, you don’t need to worry about messing something up in your environment by staging a test object in a connector space.
After you have staged your test object in a connector space, you can use the Search Connector Space feature, to run a preview on an object.
Here are the simple steps to do this:

  • In the Identity Manager, select Management Agents from the Tools menu.
  • In the Management Agents list, select the affected management agent.
  • To open the Search Connector Space dialog, select Search Connector Space from the Actions menu.
  • In the Search Connector Space dialog, select the object you care about.
  • To open the Preview dialog, click the Preview button.

http://3pji8g.bay.livefilestore.com/y1pRt8CJvly_pyzdLmKTwKFBKWYwnSCtV35PC4eMoRaOBTKUNFMeiHohd9UZYY64cqSCMMGkPFyxbzbNsUH84bDmw/GHAMV29.jpg

The Preview dialog enables you to run two different “What if” scenarios – for a full synchronization and for a delta synchronization to cover the two options you have for synchronizing objects in ILM.
By looking at the preview result, you can determine how your object would be processed throughout the system in case of an actual run.
This also helps you to determine why nothing would be staged for an export.
When you are done with the analysis of the results your test object would produce in an actual run, you can safely delete your test object from a connector space.

Additional Export Status Issues

Next to no data that was exported, not enough data can be also a concern.
This is especially the case, when you want to export a complete connector space.
In this case, it is important to know that exports are by design always deltas!

Note

Exports are always deltas!

This means, during an export, only data that has to the knowledge of the system not been exported yet, is actually exported.

One common issue in conjunction with missing data on the outbound side is related to the attribute flow precedence configuration.
In addition to your attribute flow rules, attribute flow precedence is a second gate that can block data from being flown to a connector space.
Attribute flow precedence is applied on the inbound side and on the outbound side.
On the outbound side, attribute flow precedence protects a connector space from being updated with non authoritative data in the default configuration.
As long as the management agent with the highest flow precedence value has not contributed any data to the metaverse yet, it is possible for a management agent with a lower precedence to contribute data to the metaverse.
When you experience an issue with attribute updates, you should always check whether this use is related to the attribute flow precedence configuration.

http://3pji8g.bay.livefilestore.com/y1pG2SmNVU30x0Y31XwApYZl-VrWRvD3vHYoCgkyERUz4iI0rRL7WvObfdCCcy0z7FsFylRsDsg2zmsRGFccLCVwQ/GHAMV26.jpg

What if you need to export a complete connector space?
In this case, you have some options to address this.
The simplest method is to use a different connected data source.
Typically, the subject of a complete export comes up in the context of file based management agents.
However, files do naturally not provide any kind of data management capabilities.
This is the realm of databases and directories.
So, if you are in a frequent need to get complete exports of a connector space, you are better off using a database instead of a flat file to store your data.
If your target system needs the data in form of a file, it is very simple to extract the required data from a database by using a script.
If you need to use flat files and you need a complete export of a connector space, you can use CSExport, a command-line tool that ships with ILM to drop the content of a connector space into a XML file.
While CSExport is an extremely powerful and handy tool, there are also some obstacles associated with using CSExport to do a complete export of a connector space.
The first one is probably obvious – if you have large amount of objects that are managed in a scenario, dropping the information about all of these objects into a file can be time consuming operation.
The other issue is related to the understanding of role of a connector space in the ILM world.
While implemented as database, the connector space is no identity data repository!
The role of the connector space is to function as process related data layer.
As such, it links a metaverse object with different states of an object in a connected data source and also provides the data that enables system to function according to the design goals.
For example, one objective of the connector space is to enable the system to make smart decisions in conjunction with the data that actually needs to be processed.
In other words, one role of the connector space is to enable ILM to calculate deltas.
This is just one example and a complete discussion of the connector space is beyond the scope of this article.
However, what you need to be aware of is that connector space data is stored in a way that supports the system.
Interpreting the exported data requires an understanding of the synchronization tower architecture.
You can find more details on this in the Technical Reference.

Summary

In case of an ILM based scenario, you should not use the “deploy and see what happens” approach.
Trying to interpret how the system operates by just looking at the results you see has in many cases led to confusion in conjunction how this technology actually works.
In this context, I can highly recommend reading the article on Learning ILM a colleague of mine, Ahmad Abdel-wahed, wrote.
To operate an ILM environment, you should familiarize yourself with the basic concepts around this technology.
The ILM Technical library provides a collection of documentation that covers all aspects of using ILM in a production environment.
In conjunction with unexpected results, you should not immediately suspect a malfunction of the technology.
In many cases, it turns out that scenario issues are a result of a miss configuration.
Fortunately, ILM provides in the Preview feature a very powerful method to analyze your current configuration.
You should familiarize yourself with this feature because it can safe you a lot time while tracking an issue down.
I would also recommend looking into the options to script certain aspects of your environment.
Having a toolbox of scripts at your hands helps you to get faster to a solution.
Also, if you need help, providing details about the steps you have already taken to get to the root cause of an issue, will help you to get much faster to a result.
In this context, I can recommend reading "How to get the maximum return on your forum question", which helps you to get a better idea on how to approach a problem.

Additional Information

For more information, please see the Microsoft Identity Integration Server 2003 Technical Reference.