Data Masking in Azure data Factory

Himanshu Garg 1 Reputation point
2020-10-18T07:24:11.37+00:00

We are using Azure Data factory to move data from Source like Azure SQL and Azure Postgres to destination as Azure data lake.There is some sensitive data which needs to be masked. Is it possible to have data masking in Azure Data factory during transformation phase only?

Thanks! in advance

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,410 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,026 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2020-10-19T22:18:23.313+00:00

    Hello @Himanshu Garg ,

    Thanks for the ask and also using the Microsoft Q&A.

    ADF does not have any function for masking the data , but since you mentioned that you have the source as SQL and Postgres ,
    you can make some updates with the query and hopefully it should work out with , please have the look in the query below , you can use something similar on the Source side and it should do the trick .

    CREATE TABLE MaskingTest   
    (   
    CC bigint   
    ,SSN varchar(100)   
    )   
      
    INSERT INTO MaskingTest values (1234567812345678,'123-23-1122')  
      
      
      
    SELECT *  
    ,SUBSTRING(CONVERT(VARCHAR,CC),1,1) + REPLICATE('X',LEN(CC)-1) AS [MASKED-CCC]  
    ,SUBSTRING(CONVERT(VARCHAR,SSN),1,1) + REPLICATE('X',LEN(SSN)-1) AS [MASKED-SSN]   
    FROM MASKINGTEST  
    

    CC SSN Masked-CCC Masked-SSN
    1234567812345678 123-23-1122 1XXXXXXXXXXXXXXX 1XXXXXXXXXX

    Let me know how it goes .

    Thanks Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

    0 comments No comments

  2. Kiran-MSFT 691 Reputation points Microsoft Employee
    2020-10-21T01:29:56.217+00:00

    This use case can be easily solved by dataflow derived transformation.

    0 comments No comments