How to create and manage critical data (Preview)

Not all data elements have the same importance or sensitivity, and dedicating resources to manage the quality of all data indiscriminately can be impractical and costly. Critical data elements (CDEs) are a logical grouping of important columns across tables in your data sources that you can use to focus your efforts.

These columns should be critical pieces of information that are necessary for decision making and so need to be governed with the highest care. For example: A "Customer ID" critical data element can map "CustID" from one table and "CID" from another table into the same logical container. Users can match this value across data assets to make connections, and when data producers create a new asset they can use this element as a blueprint to provide quality information in the correct format.

Critical data elements are created within governance domains and can have policies set to manage these important pieces of information.

By creating CDEs, organizations can allocate resources strategically, focusing governance effort on areas that have the most significant impact on the business.

Prerequisites

• You must be using the new data catalog experience.
• To create critical data elements, the user must have data steward permissions.
• To access data assets in the data map, they need at least data reader permissions on the collection where the asset is housed. For more information about data map permissions, see the data map permissions article.

Access critical data

Critical data is defined in governance domains. To see critical data in your data catalog:

1. Open the Data Catalog in the Microsoft Purview portal.
2. Select the Catalog management drop-down.
3. Select Governance domains
4. You see a list of all the governance domains you have access to based on your permissions. Select the governance domain where you want to see your critical data.
5. Under the Details tab select Critical data elements.
6. Here you can see the list of your critical data elements, and select any one to view its details.

Critical data element details

On the details page for your critical data element, you'll find:

• Columns - all data columns mapped from your assets to this critical data element. You can add columns.
• Associated data products - an automatically generated list of data products that have assets associated with your critical data element.

  Note

  The list of associated data products only refresh when you add or remove columns.

• Manage policies - Critical data elements allow you to attach access policies, to protect your critical information. For instructions, see the steps later in the article.

Create critical data

1. Open the Data Catalog in the Microsoft Purview Portal.

2. Select the Catalog management drop-down.

3. Select Governance domains

4. You see a list of all the governance domains you have access to based on your permissions. Select the governance domain where you want to add critical data.

5. Under the Details tab, select Critical data elements.

6. Select the + New.

7. Select the Create dropdown. Select Create.

8. Give your critical data a unique name, and description, owner, and expected data type.

9. Select Create.

10. You see your critical data in the list of critical data elements.

    Note

    • Ensure your governance domain is published before you publish your critical data element.
    • Other users can't see this critical data as a filter in the data product search until its status is set to Published.

Add columns

Once you create your critical data, you can add data columns to map critical columns from your assets to this logical concept.

1. Select your critical data element from the list of critical data elements in your governance domain.

2. Select the + Add column button.

3. Search for your data sources using keywords and filters, and select a data source.

4. In that data source, select any relevant column.

5. Select Add.

6. Now you should be able to see all your columns mapped to your critical data element.

7. Repeat for any number of data sources and columns.

   Below the columns you've added, you'll also now see the list of data products that are associated with the assets you've added to your critical data element.

Remove column

To remove a column from a critical data element, select the ellipsis ... button in the row where the column is listed, and select Remove.

If you can't find the column you want to remove, select the View all columns button to see the full list of columns.

Update critical data

1. On the critical data's details page, select Edit.

2. Here you're able to edit name, description, owner, and expected data type.

3. Select Save to save your changes.

   To update the status of your critical data element, select the Status button to change between: Draft, Published, and Expired.

Manage related terms

You can link your critical data elements to related glossary terms.

1. Select the governance domain where your critical data element resides.
2. Select the Critical data elements button.
3. Search or browse for the critical data element and select it.
4. To link your critical data element to a glossary term, select the + Add term button.
5. Search for the term, or terms, you want to link and select them.
6. To add your selected terms, select Add.
7. To remove a related term, select the term, then select the ... ellipsis button, and then the Remove button.

Delete critical data

To delete a critical data element, you will need to unpublish it and delete all columns within it, as well as any links to glossary terms. You can then select the Delete button and proceed with deleting the critical data element.

Manage access policies

To manage access policies for critical data, you need data steward permissions.

1. On your governance domain's page, select your critical data elements.

2. Select a critical data element.

3. Select Manage policies.

4. From the policy configuration window, you're able to create and manage access policies for your critical data elements.

5. For more information about setting up access policies, see the documentation about managing access policies.

   These policies will be applied to all associated data products in your critical data element.