How to view Azure Data Factory SSIS Cluster utilization

Chris Courtney 20

Hi ,

We are running SSIS Integration Runtime in our Data Factory instance to execute our SSIS workloads. For the work loads we have deployed in production I believe the the SSIS Cluster is over spec'ed, we have 2 nodes in a cluster executing SSIS packages.

My question what is the best approach for determining what compute we need for the SSIS Integration Runtime? Is there any documentation detailing how the SSIS packages are executed on each node? Are they load balanced across both available nodes or only executed on another node once max parallel executions is reached etc?

I am unable to find in the logs which SSIS packages are executed on which nodes.

Any help is appreciated.

phemanth 10,325 Reputation points Microsoft Vendor

2023-09-26T09:32:23.73+00:00

@Chris Courtney

Thanks for approaching Microsoft Q&A

However, you can monitor the overall/node-specific properties and statuses of the Azure-SSIS integration runtime using Azure PowerShell, Azure portal, and Azure Monitor.

You can also check the output of the SSIS package execution activity in the Azure Data Factory portal, which includes the execution result, error messages, and operation ID******ctivity runs, debug runs, triggers,

Additionally, you can set the logging level of package execution to Performance to collect duration information for each component in the execution.

if this doesn't help, please do let us know.

Accepted answer

Amira Bedhiafi 24,531 Reputation points

2023-09-25T10:50:47.58+00:00

You can use Azure Monitor and Azure Log Analytics to collect performance and diagnostics data and to determine compute needs. This documentation can help you : https://video2.skills-academy.com/en-us/azure/azure-monitor/

Also, Azure provides cost management and budgeting features that can help you track and manage your spending on SSIS Integration Runtime. https://azure.microsoft.com/fr-fr/products/cost-management/

If you need more : https://video2.skills-academy.com/en-us/sql/integration-services/lift-shift/ssis-azure-lift-shift-ssis-packages-overview?view=sql-server-ver16
Please sign in to rate this answer.

1 person found this answer helpful.
Chris Courtney 20 Reputation points

2023-09-25T14:21:59.7366667+00:00

Thanks for your reply. I cannot find in the Azure Monitor/Log Analytics documentation if there is anyway to see which node is executing the SSIS packages that we have scheduled from Data Factory.

Amira Bedhiafi 24,531 Reputation points

2023-09-26T08:10:36.6666667+00:00

Do your SSIS packages include custom logging that writes the node information (like machine name or IP) to a database or log file when the package starts? I think you should think about having that.

A similar workaround for ADF, you can use its built-in monitoring and management hub where you can check the activity runs, debug runs, triggers, and more. While this won't give you the exact node detail, it can provide more insights into your pipeline and activity runs.

Chris Courtney 20 Reputation points

2023-09-26T08:55:23.3066667+00:00

Thank you for the suggestions, I think getting the Node information from the SSIS Catalog in the DB is a good place to start.

Amira Bedhiafi 24,531 Reputation points

2023-09-26T09:03:19.6333333+00:00

You are always welcome ! Glad to help.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How to view Azure Data Factory SSIS Cluster utilization

0 additional answers

Your answer