Comparing Data Quality Services (DQS) and Master Data Services (MDS)
This post describes some of the ways that Data Quality Services (DQS) and Master Data Services (MDS) differ in their processes and some of the ways in which they are similar. The information in this post is taken from the Cleanse and Match Master Data by Using EIM Whitepaper to ensure that the content is indexed and available for users on the Internet when they search for this information.
Some of their (DQS and MDS) functionality can overlap because they are designed to work together and they are each designed to work without one another.
Both DQS and MDS play important roles in enterprise information management (EIM) scenarios.
- DQS emphasizes the cleansing, correction, and standardization of data through the collection of business knowledge in a DQS knowledge base, and the application of that knowledge in interactive and automated cleansing scenarios.
- MDS emphasizes the ongoing "care and feeding" of master data, providing a set of tools that allow business users to keep records up to date, while allowing IT departments to secure and manage the master data models, while integrating with external systems that utilize the master data.
Import and export
Both DQS and MDS have multiple ways to import and export data. They both work with SQL Server and Excel, but they do so in different ways.
DQS can import and export data to and from Excel and SQL Server:
- You can cleanse data from SQL Server or Excel source data files.
- You can export cleansed data into a SQL Server, CSV, or Excel file.
- You can import domains or values into a domain of a knowledge base by using an Excel file.
- You can import or export a knowledge base or domain by using a .dqs file.
- You can import values from a cleansing project (such as is created when the DQS Cleansing transformation is run) back into the knowledge base used by the project.
For MDS, you have several choices for importing and exporting data to and from MDS:
- You can load master data from an entity into Excel, and you can publish data into a model from Excel, by using the Master Data Services Add-in for Excel.
- You can also import data from SQL Server staging tables by using entity-based staging.
- You can export master data from MDS into a subscription view.
Whether to use Excel or SQL Server to import or export master data is likely to depend on who is performing the tasks, the size of the data, and whether the tasks are part of an automated process.
- Information workers are likely to find the Master Data Services Add-in for Excel useful because they are probably familiar with the product and used to performing manual tasks using it. IT pros are likely to find SQL Server processes useful given their skill sets.
- If you have a very large set of data, you are likely to want to use SQL Server processes rather than Excel. At a certain point, a large set of data becomes unmanageable in Excel.
- Entity-based staging and subscription views are easy to automate using SQL Server Integration Services and Transact-SQL. If you need to bulk-load data into an entity, use entity-based staging. Subscription views are used as a source for lookups because any data that goes into the MDM automatically shows up in a subscription view.
DQS Domain Rules and MDS Business Rules
Domain rules in DQS and business rules in MDS are both used to maintain the integrity of data, but they are used in different circumstances, as part of different operations, and from within different tools. Domain rules are used to identify data problems as part of a cleansing project, while business rules are used to prevent bad data from being introduced into MDS. The two types of rules can apply to the same data at different places in the information management lifecycle, so the same rule can end up being defined in both places.
DQS Domain Rules
A domain rule is used in the cleansing processes performed in DQS to identify problematic records. Domain rules are established in the domain management performed in the Data Quality client, and they are included in the applicable knowledge base. A domain rule enables you to specify conditions for the data that you are cleansing. You can add rules to specify values that are not allowed, and what the available values are or what the range of values is. Each record you cleanse is validated against the rules that apply to the domains in the record. If a check of a value fails, the value is marked as invalid.
MDS Business Rules
A business rule is used in Master Data Services to prevent the entry of problematic records. Business rules are created within the Master Data Manager Explorer, and they are part of the master data model. A business rule prevents the improper entry or modification of records. It is not about identifying master data within an entity that is already incorrect. It's about keeping bad data from being added to the MDM or introduced within the MDM. Master Data Services will prevent you from entering a new record or modifying an existing value if doing so would violate a business rule. Multiple processes trigger this validation of master data. Whenever you add or change data into the MDM, whether it's through the Excel Add-in, entity-based staging, the Web UI, or the API, applicable business rules are run on the records that are added or changed. If you are manually making changes in the MDM, you can use a business rule to validate the changes.