Azure Batch
An Azure service that provides cloud-scale job scheduling and compute management.
337 questions
How to modify the start task of an Azure Batch Pool started from HPC Pack 2019
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 60%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECQ%3C/text%3E%3C/svg%3E)
Christopher Quenneville
0
Reputation points
I have an On-Premises HPC Pack 2019 Update 2 head node and Windows workstation nodes environment from which my organisation would like to burst to Linux VMs on Azure Batch.
I have followed correctly this documentation: https://video2.skills-academy.com/en-us/powershell/high-performance-computing/burst-to-azure-batch-with-microsoft-hpc-pack
I am able to start pools but the start task fails every time since HpcBatchAgent.sh included in the automatic deployment fails to install the .NET dependencies to run HpcSync. The script looks for OS releases names and installs the .NET dependencies if it sees "centos" and others. We have to use almalinux now since centos is discontinued, and the script doesn't have "almalinux" listed as an os release to look for and yum install the requisites.
I tried mannually modifying the script to append |"almalinux", and it works. The issue is that pools are created and destroyed from HPC Pack, and we can't establish a workflow in which we would have to make that change mannually everytime.
This issue was fixed for the On-Premises side on HPC Pack 2019 Update 2, now including rockylinux and almalinux. However, it seems that the Azure Batch scripts weren't updated. It seems old, listing ubuntu's latest version as 16 (not 24). I was on the original HPC Pack 2019 and I correctly updated to update 1 and 2, so maybe my issue is there?
Does anyone know how to modify the start task of an Azure Batch Pool started from HPC Pack 2019?
That start task downloads HpcBatchAgent.sh from the web using this link: (https://go.microsoft.com/fwlink/?linkid=852446) (https://download.microsoft.com/download/D/5/6/D56F1BBB-0103-4151-82DC-B54751D447A6/HpcBatchAgent.sh) and if I could just swap that url for my modified version of the script we would be alright.
Thanks a lot in advance.
{count} votes
-
vipullag-MSFT 26,316 Reputation points2024-08-06T07:25:14.0533333+00:00 Hello Christopher Quenneville
Welcome to Microsoft Q&A Platform, thanks for posting your query here.
Based on the details shared, the issue could be with the start task failing due to the HpcBatchAgent.sh script not being able to install the .NET dependencies on almalinux.
To modify the start task of an Azure Batch Pool started from HPC Pack 2019, you can use the Azure Portal or Azure CLI. Here are the steps to modify the start task using the Azure Portal:
- Open the Azure Portal and navigate to your Batch account.
- Click on "Pools" in the left-hand menu and select the pool you want to modify.
- Click on the "Start task" tab.
- Modify the command line and/or resource files as needed.
- Click "Save" to save your changes.
Here are the steps to modify the start task using Azure CLI:
Open Azure CLI and log in to your account.
- Run the following command to get the current start task:
Modify the command line and/or resource files as needed.az batch pool show - Run the following command to update the start task:
Replaceaz batch pool set --pool-id pool-id --start-task-command-line command-line --start-task-resource-files resource-filespool-idwith the ID of your pool,command-linewith the updated command line, andresource-fileswith the updated resource files.
Regarding the HpcBatchAgent.sh script, you can download the script from the link you provided and modify it to include "almalinux" as an OS release to look for and install the .NET dependencies. Once you have modified the script, you can upload it to Azure Storage and update the start task to download the modified script from the new location.
I hope this helps!
-
Christopher Quenneville 0 Reputation points
2024-08-07T11:39:09.34+00:00 Hi vipullag-MSFT, thank you for your response. Unfortunatly, I must be able to change the default template used to create pools from HPC Pack (using the start and stop functions). Doing what you have described will only change the pool template after its creation, and all changes done to it will be permanently lost after "deletion" when using the stop function in HPC Pack
-
vipullag-MSFT 26,316 Reputation points
2024-08-07T14:28:22.37+00:00 Hello Christopher Quenneville
Thanks for responding back on this. Can you please file a support case so that support team will be able to check and assist you on this.
-
Christopher Quenneville 0 Reputation points
2024-08-07T15:47:33.4033333+00:00 Support case request sent, will update here with resolution
-
vipullag-MSFT 26,316 Reputation points
2024-08-08T03:31:50.9733333+00:00 Hello Christopher Quenneville
Please share the resolution, once the issue is resolved.
Sign in to comment
Sign in to answer