UserErrorInvalidColumnMappingColumnNotFound

Sandip Shelke 5 Reputation points
2024-08-05T03:36:01.2666667+00:00

using ADF for below activity

Copying data from source table to destination table with same column name both side and column present in both source and destination table, still receiving the error saying

Failure happened on 'Sink' side. ErrorCode=UserErrorInvalidColumnMappingColumnNotFound,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column name ' OR_NO' cannot be found in either source data or column mapping.,Source=Microsoft.DataTransfer.DataContracts,'

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,566 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Vinodh247 18,101 Reputation points
    2024-09-11T06:07:05.08+00:00

    Hi Sandip Shelke,

    Thanks for reaching out to Microsoft Q&A.

    The error usererrorinvalidcolumnmappingcolumnnotfound in adf usually occurs when there's a discrepancy in the column mapping configuration within your data copy activity, even if the column names appear correct in both the source and destination tables.

    1. Verify that the column name 'or_no' exists exactly as named in both the source and destination tables. Pay special attention to spaces or special characters in the column names.
    2. Inspect copy activity mapping: go to the mapping tab of the copy data activity in your adf pipeline.
    • Ensure that the column mappings are explicitly specified and that 'or_no' is correctly mapped from the source to the destination. Sometimes, auto-mapping can cause issues if not accurately detected.
    1. Check if schema validation is enabled and might be causing this error. You can try disabling schema validation as a test to see if the error persists.
    2. Since the error message includes a space before 'or_no', it's possible that there might be leading or trailing spaces in the column name in the source query or in the mapping configuration. Make sure there are no unintended spaces:
    • in sql dbs, you can use trim() function to ensure there are no leading/trailing spaces around your column names.
      In adf mapping, ensure the column name does not include unwanted spaces.
    1. Check if the database settings on either source or destination enforce case sensitivity. Sql server, for instance, is not case-sensitive by default, but other databases or systems like postgresql can be.
    2. Review the detailed error messages in the monitoring logs of adf. This might give you more context or clues on why 'or_no' is reported as not found. Ensure that the data types and nullable settings of or_no are compatible between the source and destination schemas.

    If after checking these steps the issue still occurs, you might consider explicitly defining the column in the source query (if applicable) and ensuring that the query output is exactly as expected. This can sometimes resolve hidden issues not apparent in direct schema comparisons.

    Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.