ADF Pipeline Parameter Treated as Integer Instead of String When Triggered by Schedule

Uwe Fuchs 125 Reputation points
2024-09-03T22:38:34.4866667+00:00

Hey All,

Hoping you can help with a really strange issue we’re encountering with ADF pipeline parameters and scheduled triggers.

We’ve got a pipeline parameter of type string that defaults to "-1" when nothing is passed through. Everything works as expected when we run the pipeline manually, whether we use the default or explicitly set it to "-1". But when the pipeline is triggered by a schedule, the value "-1" suddenly gets treated as an integer instead of a string, which is causing the switch statement in the pipeline to fail.

We’re planning to add conversions everywhere to make sure it’s handled as a string, but this feels like something that shouldn’t even be necessary. Has anyone run into something like this before? Any idea why this might be happening?

I’ve included some setup details and examples in the screenshots below, showing how the value is interpreted differently in the two scenarios.

Scheduled trigger definition:
User's image

Pipeline runs:
User's image

Pipeline1 Parameter Definition:

User's image

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,566 questions
Accepted answer
  1. phemanth 10,240 Reputation points Microsoft Vendor
    2024-09-04T10:46:02.39+00:00

    @Uwe Fuchs

    Welcome to the Microsoft Q&A and thank you for posting your questions here

    Possible Reasons for the Behavior

    Data Type Conversion: Even though the parameter is defined as a string, there might be a point in your pipeline where the data is being implicitly converted to an integer. This could happen:

    • Within the pipeline itself: A function or transformation you're using might be expecting a number and unintentionally converting it.
    • In the scheduler: It's less likely, but perhaps there's an odd behavior in how the scheduler interacts with parameters and triggers the pipeline.

    Data Flow: If your parameter is used in data flow, the data flow may be causing an implicit conversion of the string to an integer.

    Legacy Code or Configuration: Older versions of ADF might have had inconsistencies with how parameter types were handled.

    Troubleshooting and Solutions

    Inspect Your Pipeline:

    • Examine all operations: Trace the data flow from where the parameter is received to where it's used. Check if any of the transformations, functions, or actions might be expecting a number.
    • Look for explicit conversions: If you have any explicit code that attempts to convert the parameter to an integer, remove or modify it to preserve its string format.
    • Check for implicit conversions: Be cautious of functions that might perform implicit conversions like toInt() or parseFloat().

    Explicit Type Casting:

    • Within your pipeline: Before using the parameter in any action or transformation, explicitly cast it to a string using a function like toString() or convertToString().
    • In the scheduler: It's less likely, but you might need to convert the parameter to a string within the scheduler itself if it's handling it improperly.

    Test Different Parameter Values:

    • Try setting the default value to something that clearly identifies it as a string, like "test" or "-1.0". This might highlight where the issue occurs.

    Hope this helps. Do let us know if you any further queries.

    1 person found this answer helpful.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. phemanth 10,240 Reputation points Microsoft Vendor
    2024-09-05T09:55:16.5766667+00:00

    @Uwe Fuchs

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    Ask:

    Hoping you can help with a really strange issue we’re encountering with ADF pipeline parameters and scheduled triggers.

    We’ve got a pipeline parameter of type string that defaults to "-1" when nothing is passed through. Everything works as expected when we run the pipeline manually, whether we use the default or explicitly set it to "-1". But when the pipeline is triggered by a schedule, the value "-1" suddenly gets treated as an integer instead of a string, which is causing the switch statement in the pipeline to fail.

    We’re planning to add conversions everywhere to make sure it’s handled as a string, but this feels like something that shouldn’t even be necessary. Has anyone run into something like this before? Any idea why this might be happening?

    I’ve included some setup details and examples in the screenshots below, showing how the value is interpreted differently in the two scenarios.

    Scheduled trigger definition: User's image

    Pipeline runs: User's image

    Pipeline1 Parameter Definition:

    User's image

    Solution: as I mentioned in my questions, I did end up just adding the toString() on the switch statement and that has resolved the issue

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.