Walkthrough: Migrating ClearQuest Work Items to Team Foundation

  • Article

This walkthrough gives you the process for moving ClearQuest work items to Team Foundation work item tracking. Team Foundation includes a conversion utility, CQConverter.exe that you can use to perform this migration.

In this walkthrough, you will accomplish the following tasks:

  • Configure the migration process   Edit the configuration files to guide the migration process.

  • Analyze your ClearQuest database   Run the ClearQuest converter to analyze your current ClearQuest database. The analysis produces a report and a set of files that describes the work item types present in ClearQuest.

  • Define the migration mapping   Edit the XML files generated during the ClearQuest Database analysis to customize the migration process.

  • Migrate your ClearQuest database to Team Foundation work item tracking   Use the configuration and mapping files generated during the analysis of the database to migrate your data.

  • Review the migration report   Discover the cause of any errors and warnings seen during the migration, and review the migration status.

Prerequisites

To complete this walkthrough, you need the following software installed on the computer where you plan to run CQConverter.exe:

  • Team Explorer

  • ClearQuest Standalone client Version 2003.06.00

    Note

    Versions 2002.05.20 and 2003.05.00 may also work, but have not been tested.

You must also make a list of how you want the user names in ClearQuest mapped to Windows domain names used by Team Foundation work item tracking. Make sure that you include user names that are no longer being used.

The following information and knowledge is helpful:

  • A basic understanding of XML.

  • A list of how you want the map the work item fields in ClearQuest to Team Foundation work item tracking. Include information about transformations that are required. For example, you might map a string field in ClearQuest to an integer field in Team Foundation work item tracking.

Required Permissions

To perform this walkthrough, you must be a member of the Service Accounts security group. Also, you must be a member of the Readers security group or have the View work items in this node permission set to Allow. For more information, see Team Foundation Server Permissions. Also, you must have the Super User permission for the ClearQuest database. For more information, see the ClearQuest Administrator's Guide.

Pre-migration Tasks

Before you start the migration, you must do the following tasks:

  • Schedule time to lock the ClearQuest database   A good practice is to lock the database to prevent users from creating new work items during the migration process. The ClearQuest converter does not detect work item changes during conversion. For information about locking the ClearQuest database, see the ClearQuest documentation.

  • Create Visual Studio Issue Trackingteam projects   Create team projects and decide which work items go into what team project. For more information about how to create Team Foundation work item tracking team projects, see Walkthrough: Creating a New Team Project.

  • Add the user account you are using to run the converter to the Service Accounts group   The CQConverter requires service account permissions to run. Add the user account that will run the converter to the Service Accounts group on your Team Foundation Server by using the TFSSecurity utility. For more information, see TFSSecurity Command-Line Utility Commands.

Converter Limitations

Before you start your migration, you should review the limitations of the ClearQuest converter and how those limitations might affect the users in your organization. For more information, see CQConverter Command-Line Utility for ClearQuest Database Migration.

Definitions

The following terms are used in this walkthrough:

  • Work Item Type   Work item types are the basic building blocks of a work item tracking system. Examples of work item types are bugs, requirements, and tasks.

  • Entity   The basic unit of data in ClearQuest. For example, a defect could be linked to a customer and belong to certain project. The defect, customer, and project are all entities.

  • Field   Work items contain fields that contain information about the work item. For example, a defect work item might have fields such as opened by, status, resolution, history, links, and attachments.

  • States   The position in the process that a work item could be in. For example, a bug might be Open, Assigned, Resolved, or Postponed.

  • Field Behavior (Field Rules)   Specified how the fields behave in different states. For example, the Fixed In field could be non-editable when you create a new bug, but when you transition the bug into the Resolved state, the Fixed In field may require a value.

Configuring the Migration Process

During this step, you change the ClearQuest converter configuration files to guide the migration process.

The Configuration File

The configuration file is an XML file that provides settings for the ClearQuest converter. The settings are specified in the following sections of the XML file:

<ClearQuest>

The ClearQuest section contains information about the ClearQuest database that you want to migrate.

  • ConnectionName   ClearQuest connection name.

  • UserDatabase   ClearQuest user database name.

  • QueryName   Name of a query saved in ClearQuest.

  • UserID   ClearQuest user ID. The specified user ID must have the Super User permission for the ClearQuest database.

<VSTS>

The VSTS section contains information about the Team Foundation work item tracking server and team project to which you are migrating.

  • URI   Team Foundation Server friendly name or the complete URI. For example, "MyAtServer," or "http://MyAtServer:8080."

  • ProjectName   Team Foundation work item tracking team project name.

Other Configuration Settings

The other sections in the XML file let you specify the following settings:

  • ExitOnError   This section accepts a value of true or false. Specify True if you want to stop the ClearQuest converter if errors occur in the migration of some work items.

    Note

    This setting only applies to non-fatal errors. For fatal errors, such as the ClearQuest converter cannot connect to ClearQuest, the converter will always exit even if this setting is False.

  • OutputDirectory   Specify the directory where you want the generated files to go. If you do not specify an output directory, the converter creates a directory with Base Entity name for generating output files. If the directory already exists, the converter overwrites the files in that directory.

Creating the Configuration File

Creating the configuration file can be accomplished in any text editor, such as Notepad.

To create the configuration file

  1. Collect the following information:

    • ClearQuest connection name.

    • ClearQuest database name.

    • ClearQuest Administrator user ID and password. Make sure that this admin user has permissions to submit issues in ClearQuest.

    • Team Foundation work item tracking server name.

    • Team Foundation work item tracking team project name (the team project to which you want to migrate the work items).

  2. Create a query in Clearquest to select the work items that you want to migrate to Team Foundation work item tracking and save the query. Save the query by any name, for example TransitionQuery.

  3. Decide on a directory to store the files produced during the analysis phase.

  4. Copy the following XML configuration file into a text editor.

    <?xml version="1.0" encoding="utf-8"?>
<WorkItemConverter>
  <Source id="ClearQuest">
    <ClearQuest>
      <ConnectionName>CQRep</ConnectionName> 
      <UserDatabase>TEST</UserDatabase> 
      <QueryName>Public Queries/All Defects</QueryName> 
      <UserID>admin</UserID>
    </ClearQuest>
  </Source>
  <VSTS>
    <URI>AdventureWorksServer</URI>
    <ProjectName>AdventureWorksProject</ProjectName> 
  </VSTS>
  <ExitOnError>true</ExitOnError>
  <OutputDirectory>DefectSchema</OutputDirectory>
</WorkItemConverter>

  5. Change the file to specify your settings with information gathered in previous steps. For example, under <QueryName>, specify the query name as TransitionQuery and include the query path. See "The Configuration File" section earlier for information about what each section should contain.

    Note

    The password is not entered in the file, the ClearQuest converter tool asks for it later in the migration process.

  6. Save the file.

    For example, you can save the file as CqConverterConfig.xml.

Analyzing Your ClearQuest Database

During the analysis of your ClearQuest database, the ClearQuest to Team Foundation converter analyzes your ClearQuest database and creates the following XML files:

  • A Team Foundation work item tracking work item type definition (WITD) file for each ClearQuest entity type that contains the states, state transitions, fields, field types, and field behavior such as required, or optional.

  • A default field mapping file that maps ClearQuest fields and values to Team Foundation work item tracking fields and values.

  • A user map file that maps ClearQuest users to Windows users. This file applies to the whole system.

  • A schema map file that contains references to the other XML files, such as the SchemaMap file which contains entity names, and your field mapping file that has the path of the generated WorkItem file names and fieldMap files.

  • A report and warning file (CQAnalysisReport.xml).

To analyze the ClearQuest database

  1. At the Visual Studio 2008 Command Prompt, run the following command:

    CQConverter.exe /c:analyze /m:CqConverterConfig.xml

    The ClearQuest converter may take several minutes to finish running.

    Note   For more information about ClearQuest Converter commands, see CQConverter Command-Line Utility for ClearQuest Database Migration.

  2. In the output directory that you specified in the configuration file, look for the following XML files. These files are used later in the migration process:

    • A WITD file for each ClearQuest entity type named as EntityName.xml. For example, a file for the ClearQuest Defect entity would be named Defect.xml.

    • A field mapping file for each ClearQuest entity type. These files specify how the fields and field values in each ClearQuest entity will be mapped to the corresponding fields in Team Foundation work item tracking work item type. For example, a file for the ClearQuest Defect entity would be named DefectFieldMap.xml.

    • A usermap file. A file specifying how the ClearQuest users are to be mapped to Team Foundation work item tracking users.

    • A schemamap.xml file. A file describing the schema and field mappings.

  3. In the directory where you ran the converter, look for the CQAnalysisReport.xml file to see any warnings and errors before migration.

Defining the Migration Mapping

Analysis generates a set of files that captures the information from your ClearQuest database. You may have to change some of these files in order to migrate the database the way you want. At the very least, you have to change the user mapping file and the work item type definition file.

For more information about modifying the definition file for a work item type, see Walkthrough: Make Basic Customizations to a Work Item Type.

Editing the Schema Mapping File

To change the schemamap.xml file

  1. Open the Schemamap.xml file.

    A sample of this file follows:

    <Mappings>
  <SchemaMaps>
    <SchemaMap SourceEntity="Defect" TargetWIT="Defect"
      WITDFile="Defect.xml" FieldMapFile="DefectFieldMap.xml"/> 
    <SchemaMap SourceEntity="Customer" TargetWIT="Customer"
      WITDFile="Customer.xml" FieldMapFile="CustomerFieldMap.xml"/> 
  </SchemaMaps>
  <UserMap File="UserMap.xml" /> 
</Mappings>

  2. Change the sections in this file as appropriate.

    • SourceEntity   Ensure this section contains the correct ClearQuest entity.

    • TargetWIT   Ensure this section points to the correct work item type that should be created in Team Foundation work item tracking.

    • WITDFile   This section specifies the work item type definition (WITD) XML file.

    • FieldMapFile   This section specifies the field mapping XML file.

    • UserMap   This section specifies the user mapping XML file.

Editing the User Mapping File

The ClearQuest converter uses the user mapping file to map user IDs in the ClearQuest database to an equivalent Team Foundation work item tracking user ID. This helps preserve the work item history by converting the user ID field values to be valid values in Team Foundation work item tracking. Team Foundation work item tracking uses Windows domain user IDs.

A default version of this file is generated by the ClearQuest converter during the analysis phase. Make sure that this file is accurate before continuing with the conversion, because the converter checks this file and fails if there are problems.

To change the user mapping file

  1. Open the usermap.xml file. A sample follows:

    <UserMappings">
  <UserMap From="admin" To="admin" />
  <UserMap From="user" To="user" />
  <UserMap From="QE" To="QE" />
  <UserMap From="lead" To="lead" /> 
  <UserMap From="engineer" To="engineer" />
</UserMappings>

  2. Examine the user mappings and correct, add, and delete as required. The "to" side of the user mapping has to be an existing Team Foundation Server user.

  3. In Team Foundation work item tracking, add all the users specified in the user mapping file. For more information about adding users, see How to: Add Users to a Team Project Group.

    Note

    If you want any one of the ClearQuest users to be mapped as is into Team Foundation Server during migration, you can delete the rows in the user mapping file for those users. If you do not want any user mapping performed, you can delete all the rows specifying user maps in the <UserMap> section. You should only delete the individual entries in the <UserMap> section - do not delete the section or any other parts of the file.

    Warning

    If users are not mapped to valid Team Foundation Server users during migration, you may have to fix the user fields in your work items when you start working with them. The converter runs without validating the field values, but as soon as you start editing the work items, you will be required to provide valid users for the user fields.

Editing the Field Mapping File

The ClearQuest converter uses the field mapping files to map fields in each ClearQuest entity to an equivalent Team Foundation work item tracking work item field.

To change the field mapping file

  1. Open one of the field mapping files. For example, open DefectFieldMap.xml. A sample follows:

    <FieldMap from="Criticality" to="Severity">
  <ValueMaps>
        <ValueMap from="1-Critical" to = "1"/>
        <ValueMap from="2-Major" to = "2"/>
  </ValueMaps>
</FieldMap>

  2. Review the field mapping lines (starting with ValueMap), and make sure that the field indicated after "from" is mapped to the correct field after "to." You can add more field mappings by adding more ValueMap lines. The values should be in the list of the allowed values if this has been specified in the schema file, in this case, defect.xml.

    Note

    If you change the field mappings, make sure that the required fields on the work item form are mapped

  3. Open the other field mapping files and edit them as appropriate.

Defining a Work Item Type Definition

After the analysis phase, the converter generates work item type definitions (WITDs) for each ClearQuest record type. For example, a file for the ClearQuest Defect record type is named Defect.xml. The Converter generates a default work item form as a part of the schema generation during the analysis phase. However, this default form contains minimal formatting and will almost always require editing to have the work item layout that you want. For information about how to change the definition file for a work item type, see Walkthrough: Make Basic Customizations to a Work Item Typeand Walkthrough: Make Advanced Customizations to a Work Item Type.

Migrating the ClearQuest Work Items

To migrate work items

  1. At the Visual Studio 2008 Command Prompt, run the following command:

    CQConverter.exe /c:migrate /m:CqConverterConfig.xml /s:path\Schemamap.xml

    Where path is the path to the schema map file. The schema map file is generated during the analysis phase in the specified output directory.

    Note

    The ClearQuest converter can take hours to finish running for large databases.

    Note

    For more information about ClearQuest Converter commands, see Migrating from ClearQuest.

  2. When the migration is completed, review the migration report for the status of migration and other issues such as defects not migrating caused by field mapping inconsistencies.

    The migration report is named CQMigrationReport.xml and created in the folder from which you ran the ClearQuest converter.

    Note

    Since the converter converts all work items included in the query you specify, plus all work items that are linked to from those work items, the actual number of work items migrated may be different from the number of work items in the query.

  3. If all the work items have not been migrated, you can run the conversion again after resolving issues outlined in the migration report.

To migrate more work items from ClearQuest to Team Foundation work item tracking, you can run the migration process again.

To run migration again on the same ClearQuest database

  1. Correct any issues outlined in the migration report for the previous migration.

  2. Make a query in ClearQuest that returns the desired work items, or run the same query if you want to migrate work items that did not migrate the first time.

  3. Change the <QueryName> section in the configuration file to specify the new query name.

  4. Open the Visual Studio 2008 Command Prompt and type the command for migration:

    CQConverter.exe /c:migrate /m:CqConverterConfig.xml /s:path\Schemamap.xml

    The ClearQuest converter can take hours to finish running for large databases.

    Note

    For more information about ClearQuest Converter commands, see Migrating from ClearQuest.

  5. When the migration is completed, review the migration report for the status of migration. The migration report is named CQMigrationReport.xml and created in the folder from which you ran the ClearQuest converter.

Verifying Work Item Migration

After the migration is performed, you can examine some of the migrated work items to verify that the information has been migrated successfully.

To verify the migrated work items

  1. Open a migrated work item in Team Foundation work item tracking.

  2. Use the number in the ClearQuest ID field to compare the Team Foundation work item tracking work item with the ClearQuest bug with that same ID.

To find the Team Foundation work item corresponding to a specific ClearQuest work item

  1. Create a work item query with the following query clause:

    And/Or

    Field

    Operator

    Value

     

    VSTS SourceID

    =

    ClearQuest work item ID

    The VSTS SourceID field corresponds to the ClearQuest ID field label that you see on your work item form.

    For more information about how to create work item queries, see How to: Add New Work Item Queries.

  2. Run the query.

See Also

Tasks

Walkthrough: Tracking Work Items

Other Resources

Migration Walkthroughs