Passing values from notebook to 'Set Variable' transformation within pipeline.

@Peter Ott

Thanks for reaching out to Microsoft Q&A

It looks like you’re trying to pass values from a Synapse notebook to a ‘Set Variable’ transformation within a pipeline. You have the JSON string ready, but it seems like you’re stuck on the next steps.

Here’s how you can proceed:

1. Capture the JSON Output: Ensure that your notebook’s output is correctly captured by the pipeline activity. The mssparkutils.notebook.exit(result) should return the JSON string as the output.
2. Set Variable Activity: In your pipeline, use the ‘Set Variable’ activity to parse and assign the values from the JSON string to pipeline variables.

Here’s a step-by-step guide:

Step 1: Notebook Activity

Ensure your notebook activity is configured to capture the output. Your code seems correct for generating the JSON string.

import json
variables_dict = {
    "adlspath": adlspath,
    "MDPKeyVault": MDPKeyVault,
    "jdbcHostname": jdbcHostname,
    "jdbcDatabase": jdbcDatabase
}
result = json.dumps(variables_dict)
mssparkutils.notebook.exit(result)

Step 2: Set Variable Activity

Add a ‘Set Variable’ activity to your pipeline.

Link the ‘Set Variable’ activity to the notebook activity.

Configure the ‘Set Variable’ activity:

• Variable Name: Choose the variable you want to set.
• Value: Use the following expression to extract the value from the notebook output:

@json(activity('NotebookActivityName').output.runOutput).adlspath

Example

If your notebook activity is named RunNotebook, and you want to set the adlspath variable, the expression would be:

@json(activity('NotebookActivityName').output.runOutput).adlspath

Repeat this for each variable you need to set.

Hope this helps. Do let us know if you any further queries.