Is a Azure Batch dedicated node always running?

Erp, Wessel van 21 Reputation points
2022-06-22T16:37:23.827+00:00

Hello,

We use Azure Batch for a very simple task in Azure Data Factory. Recently it occured to us that we have very high VM costs and we're trying to find out why. We know the Azure billing tool might give us more insights, but we're working with a company that is very slow in giving us the appropriate rights. To stop this as soon as possible, we have to find other ways and now we're guessing this might be an issue.

It is the first time that I worked with Azure Batch. It took me some trial and error, but eventually I got Azure Batch to work setting Target dedicated nodes to 1. Here I assumed that we only have to pay for the computing power of this single task, but now I'm afraid this is a wrong assumption and that the VM behind this node is always running and is giving us these high costs.

Am I right about this? And if so, what can I do best? I read something about auto-scale and I would guess that if you program it correctly it shoudl be able to turn on one node when the tasks starts and kills it afterwards.

I would really appreciate if somebody could help me as soon as possible. Thank you in advance!

Best regards

Azure Batch
Azure Batch
An Azure service that provides cloud-scale job scheduling and compute management.
320 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. vipullag-MSFT 25,616 Reputation points
    2022-06-24T05:03:19.063+00:00

    @Erp, Wessel van

    Welcome to Microsoft Q&A Platform, thanks for posting your query here.

    Am I right about this?

    As long as a node is allocated into a pool, it incurring charges. This has nothing to do with the time the task is being executed. Compute time, is essentially the lifetime of the node in the pool. As long as it is allocated to the pool, it is incurring charges.

    214516-image.png

    For Autoscale compute nodes, please check this document.

    Hope that helps.
    If the suggested response helped you resolve your issue, please 'Accept as answer', so that it can help others in the community looking for help on similar topics.

    1 person found this answer helpful.
    0 comments
  2. Xin Xin 26 Reputation points Microsoft Employee
    2022-10-23T09:47:27.957+00:00

    There are two types of pool, fixed and auto-scale. I assume that you use fixed pool and target VM # is 1. Then the VM will be always running and charge cost.
    If you need run task occasionally and on-demand, you can try auto-scale pool, with auto-scale pool, you can setup initial VM # as 0, and do scale-out/in according to active task #.

    You can refer to the document to configure your auto-scale formular:
    https://video2.skills-academy.com/zh-cn/azure/batch/batch-automatic-scaling

    1 person found this answer helpful.
    0 comments