Automating execution of ".bat" file using ADF / LogicApp

Nagesh CL 646 Reputation points
2024-10-02T09:34:48.19+00:00

Hello Team,

Currently we are manually executing a .bat file in one of the client VMs and we need to automate the same using ADF / Logic App. The bat file is inside a folder and it executes a .jar file which resides within the same folder. The .jar file internally refers to a config file and few other libraries / executables and all of them reside in the same folder. The folder structure is as below: -

User's image

In ADF, we have a Custom activity. Can this be used to execute the .bat files from ADF ? The Custom activity needs a Batch account. Where should be place these .jar, configs, .bat files ? In the blob account (which is associated to batch account) ? Or should we place it in any of the VMs / Shared Folder and this Custom activity can access the shared folder ? I have seen some tutorials where they execute a simple python script, but in cases where we need all the setup files to be placed at a particular location, how does this work ?

We have flexibility to use either ADF / Logic App to perform this activity. So any help on how to achieve this task is highly appreciable.

Thanks.

Regards,

Nagesh CL

Azure Logic Apps
Azure Logic Apps
An Azure service that automates the access and use of data across clouds without writing code.
3,133 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,679 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Vinodh247 20,476 Reputation points
    2024-10-02T12:38:33.37+00:00

    Hi Nagesh CL,

    Thanks for reaching out to Microsoft Q&A.

    You can try these approaches:

    ADF with Batch Service:

    • Custom Activity: This activity runs code in Azure Batch.
    • Azure Batch:
      • You would need to upload all your required files, including the '.bat', '.jar', and config files, to azure blob storage.

    Steps:

    • Create an Azure Batch Account: Set up a Batch account linked to your ADF.
    • Upload Files: Store the '.bat', '.jar', config, and any other necessary files in Blob Storage.
    • Create a Pool and Job: Use ADF's Custom Activity to create a Batch job. In the job, specify the task to run the '.bat' file.
    • Task Setup: In the Batch task, reference the storage location of your files. The '.bat' file will be executed on Batch compute nodes, which will download the required files from Blob Storage.

    Key Points:

    • The Custom Activity will need an Azure Batch pool to run your tasks.
    • You cannot execute files directly from shared folders without using Azure Batch.
    • Ensure the Batch nodes (VMs) have access to the Blob Storage where your files are stored.

    Logic Apps:

    If your VM is on a public network or reachable by a Logic App workflow, you can execute the '.bat' file remotely using Logic Apps.

    • PowerShell Execution:
      • You can use the Logic Apps PowerShell action to run a PowerShell script that calls your '.bat' file on the client VM.

    The Logic App can either:

    • Invoke a PowerShell script: To remotely execute the '.bat' file on the client VM using a secure connection (like SSH for Linux or WinRM for windows).
    • Use Azure Runbooks: If direct access is not possible, you can trigger an Azure Automation Runbook that runs the '.bat' file on the VM.

    Key Points:

    • For this approach, you need network connectivity between Logic Apps and your VM.
    • You may need to set up authentication mechanisms (like MI, service principals, etc.) to securely execute the script.

    Recommendation:

    • If you already use Azure Batch or are comfortable setting it up, ADF Custom Activity is a scalable and maintainable solution.
    • If you have direct access to the VM and do not want to manage Azure Batch resources, Logic Apps with PowerShell or Azure Automation might be simpler.

    Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.