Can we remove the artificial limitation of ADF Snowflake V2 supporting Multiple Statements

Macolino, Nick (Health) 1 Reputation point
2024-06-25T06:37:47.1+00:00

 

Issue with Snowflake V2

https://video2.skills-academy.com/en-us/azure/data-factory/connector-snowflake?tabs=data-factory#differences-between-snowflake-and-snowflake-legacy

 

How can I get feedback to the engineers that are working on this new connector for ADF for Snowflake.

 

This will be enforced come late October this year.

The highlighted item seems insane to me. Can’t find anywhere on the MS forums to express how crazy this seems and it should be reverted.

Reason is to get around it all I have to do is wrap my Script Multi Statements in a “Begin…. End” so it looks like one statement, which seems nuts with the only impact being that we now need to refactor and re-test every pipeline written.

User's image

What puzzles me is that the only difference is that the new connector now sends this statement to the database… where previously this was set to 0 so unlimited ?

Seems to be an artificial limit imposed to cause pain and $$ burnt in people time to refactor and re-test.

 User's image

 

 

 

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,681 questions
{count} votes

2 answers

Sort by: Most helpful
  1. Macolino, Nick (Health) 1 Reputation point
    2024-07-17T08:03:20.6733333+00:00

    Good news.

    I raised a ticket with MS and after a couple weeks discussing with the engineers. This limit has been removed in the latest release of the integration runtime.

    So if you upgrade your SHIR to 5.43.8935.2 or above the issue has been resolved.

    Great outcome for us.

    Just means about 10 minutes work in DevOps and replace references and we are back in business.

    Thanks to the MS team for sorting this one out and saving us a lot of time and pain at our end.

    Nick


  2. Smaran Thoomu 15,765 Reputation points Microsoft Vendor
    2024-07-17T08:56:00.08+00:00

    Hi @Macolino, Nick (Health)
    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    Issue: Issue with Snowflake V2

    https://video2.skills-academy.com/en-us/azure/data-factory/connector-snowflake?tabs=data-factory#differences-between-snowflake-and-snowflake-legacy

    How can I get feedback to the engineers that are working on this new connector for ADF for Snowflake.

    This will be enforced come late October this year.

    The highlighted item seems insane to me. Can’t find anywhere on the MS forums to express how crazy this seems and it should be reverted.

    Reason is to get around it all I have to do is wrap my Script Multi Statements in a “Begin…. End” so it looks like one statement, which seems nuts with the only impact being that we now need to refactor and re-test every pipeline written.

    User's image

    What puzzles me is that the only difference is that the new connector now sends this statement to the database… where previously this was set to 0 so unlimited ?

    Seems to be an artificial limit imposed to cause pain and $$ burnt in people time to refactor and re-test.

     User's image

    Solution: Good news.

    I raised a ticket with MS and after a couple weeks discussing with the engineers. This limit has been removed in the latest release of the integration runtime.

    So if you upgrade your SHIR to 5.43.8935.2 or above the issue has been resolved.

    Great outcome for us.

    Just means about 10 minutes work in DevOps and replace references and we are back in business.

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    I hope this helps!

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.


    Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.