Error when creating batch pool in azure batch, task doesn't finish in azure data factory

Luuk van Vliet 276 Reputation points
2021-01-07T09:24:18.383+00:00

Hello,

I try to set up a batch pool in order to run some python scripts that clean complex datasets orchestrated by azure data factory.
Upon creating a pool I get the following error: AccountVMSeriesCoreQuotaReached.

Ignoring the error and scheduling a task to run the python script in adf leads to the pipeline running for 1 hour without completion. (python script is stored in a blob container)

What am I possibly doing wrong?

54314-image.png

Azure Batch
Azure Batch
An Azure service that provides cloud-scale job scheduling and compute management.
320 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,027 questions
0 comments
Accepted answer
  1. TravisCragg-MSFT 5,681 Reputation points Microsoft Employee
    2021-01-07T19:56:29.167+00:00

    Azure places limits on the amount of vCPUs you can allocate for each VM size. If your subscription does not have any cores, then your batch job will not be able to allocate. Here is a doc that outlines the process to request more cores for a given VM size.

    The default quota limits that you are seeing are placed on your subscription when it is created, and sometimes modified depending upon VM availability. If you need more cores in a particular region and of a specific size, just request it using the process I linked above. Once the cores have been allocated, the batch job should run without an issue.

1 additional answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. LS 6 Reputation points
    2021-01-07T19:03:02.417+00:00

    I am having the exact same issue here. I've checked the quotes in my batch account, and saw all but the Dv3-series have a value of 0. Selecting a Dv3-series in a pool does prevent the AccountVMSeriesCoreQuotaReached-error.

    Why are all these quota values 0?

    How can we use for example A series in a batch job?

    54532-image.png

    1 person found this answer helpful.
    0 comments