Deployment of Azure Container Instance Failed, where did I go wrong?

Rahul Vijan 0 Student Ambassador

{
    "status": "Failed",
    "error": {
        "message": "Pulling image \"waleedka/modern-deep-learning\";Successfully pulled image \"waleedka/modern-deep-learning\" in 2m20.224069124s;Created container test;Started container test;Subscription deployment didn't reach a successful provisioning state after '00:30:00'."
    }
}

Container Specs were 2cores (CPU), 1 K80 GPU, 8GB RAM, EAST US, QUOTA AVAILABILTIY MET
Docker Image: waleedka/modern-deep-learning

I wish to train a model using a container with a GPU, as my student subscription doesn't allow provisioning any GPU (N family) Vms.

Patchfox 3,806 Reputation points

2023-03-24T05:55:53.1733333+00:00
Hi Rahul Vijan, I hope I can help you with this question.

I would recommend executing the following command to get more details about the deployment process.

az container show --resource-group myResourceGroup --name mycontainer

https://video2.skills-academy.com/en-us/azure/container-instances/container-instances-get-logs#get-diagnostic-events

Please post the output here afterward.

If the reply was helpful, please don’t forget to upvote or accept it as an answer, thank you.
Patchfox 3,806 Reputation points

2023-03-25T14:42:16.28+00:00

Hi Rahul Vijan , I quickly want to check, if my answer helped you or if you need further assistance.
deherman-MSFT 35,011 Reputation points Microsoft Employee

2023-03-27T21:21:23.4933333+00:00

@Rahul Vijan Hello,

I hope this has been helpful! We appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community. Thank you for helping to improve Microsoft Q&A!

3 answers

Eddie Neto 1,226 Microsoft Employee

Hi @Rahul Vijan

Thanks for reaching M&A

I´ve performed the same test as you and mine got succussed the only different that I did was to add the the command override "/bin/bash", "-c", "echo hello; sleep 100000"

Hope this helps. Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.

User's image

Rahul Vijan 0 Student Ambassador

Hi @Eddie Neto I did the same, the deployment failed
Status Message:

{
    "status": "Failed",
    "error": {
        "message": "Pulling image \"waleedka/modern-deep-learning\";Successfully pulled image \"waleedka/modern-deep-learning\" in 3m38.604738041s;Created container test2;Started container test2;Subscription deployment didn't reach a successful provisioning state after '00:30:00'."
    }
}




rahul [ ~ ]$ az container show --resource-group test --name test2
{
  "confidentialComputeProperties": null,
  "containers": [
    {
      "command": [
        "/bin/bash",
        "-c",
        "echo hello; sleep 100000"
      ],
      "environmentVariables": [],
      "image": "waleedka/modern-deep-learning",
      "instanceView": {
        "currentState": {
          "detailStatus": "",
          "exitCode": null,
          "finishTime": null,
          "startTime": "2023-04-04T22:12:14+00:00",
          "state": "Running"
        },
        "events": [
          {
            "count": 1,
            "firstTimestamp": "2023-04-04T22:08:34+00:00",
            "lastTimestamp": "2023-04-04T22:08:34+00:00",
            "message": "Pulling image \"waleedka/modern-deep-learning\"",
            "name": "Pulling",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-04-04T22:12:13+00:00",
            "lastTimestamp": "2023-04-04T22:12:13+00:00",
            "message": "Successfully pulled image \"waleedka/modern-deep-learning\" in 3m38.604738041s",
            "name": "Pulled",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-04-04T22:12:13+00:00",
            "lastTimestamp": "2023-04-04T22:12:13+00:00",
            "message": "Created container test2",
            "name": "Created",
            "type": "Normal"
          },
          {
            "count": 1,
            "firstTimestamp": "2023-04-04T22:12:14+00:00",
            "lastTimestamp": "2023-04-04T22:12:14+00:00",
            "message": "Started container test2",
            "name": "Started",
            "type": "Normal"
          }
        ],
        "previousState": null,
        "restartCount": 0
      },
      "livenessProbe": null,
      "name": "test2",
      "ports": [
        {
          "port": 80,
          "protocol": "TCP"
        }
      ],
      "readinessProbe": null,
      "resources": {
        "limits": null,
        "requests": {
          "cpu": 2.0,
          "gpu": {
            "count": 1,
            "sku": "K80"
          },
          "memoryInGb": 8.0
        }
      },
      "volumeMounts": null
    }
  ],
  "diagnostics": null,
  "dnsConfig": null,
  "encryptionProperties": null,
  "extensions": null,
  "id": "/subscriptions/******************************/resourceGroups/test/providers/Microsoft.ContainerInstance/containerGroups/test2",
  "identity": null,
  "imageRegistryCredentials": null,
  "initContainers": [],
  "instanceView": {
    "events": [
      {
        "count": 1,
        "firstTimestamp": "2023-04-04T22:13:01.884570+00:00",
        "lastTimestamp": "2023-04-04T22:13:01.884570+00:00",
        "message": "Subscription deployment didn't reach a successful provisioning state after '00:30:00'.",
        "name": "DeploymentTimeout",
        "type": "Warning"
      }
    ],
    "state": "Running"
  },
  "ipAddress": {
    "autoGeneratedDomainNameLabelScope": "TenantReuse",
    "dnsNameLabel": "test2",
    "fqdn": "test2.fwhvd8auemfec2cm.eastus.azurecontainer.io",
    "ip": "20.81.19.135",
    "ports": [
      {
        "port": 80,
        "protocol": "TCP"
      }
    ],
    "type": "Public"
  },
  "location": "eastus",
  "name": "test2",
  "osType": "Linux",
  "priority": null,
  "provisioningState": "Failed",
  "resourceGroup": "test",
  "restartPolicy": "OnFailure",
  "sku": "Standard",
  "subnetIds": null,
  "tags": {},
  "type": "Microsoft.ContainerInstance/containerGroups",
  "volumes": null,
  "zones": null
}

deherman-MSFT 35,011 Reputation points Microsoft Employee

2023-04-05T16:43:58.5433333+00:00

@Rahul Vijan

This can occur if the container name is being reused. Please try deploying with a different container name to see if it resolves your issue. If you are still facing the same issue please email the following to AzCommunity@microsoft.com and we'll get back to you promptly:

• Subject: "Attn: deherman - "

• Email body: Your Subscription ID

• Email body: A link to this thread so we can validate and expedite the request

If you don't receive a response within 24 hours, please reply to the thread so we can investigate.

Please don’t forget to "Accept the answer" and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment
Gurav, Shankar 0 Reputation points

2024-02-13T12:24:18.0866667+00:00

We faced same problem for deploying containers in private network, after recreating container with new name did not resolved issue. Microsoft support investigated and found that all the remaining IPs in our subnet are currently held, possibly due to known problem with the ACI platform where there was an issue with the leaking Ips. This issue has been fixed for all the newer clusters and new subnets. However, there are some edge cases for some older subnet where already leaked Ips will cause similar behavior.

So redeploying containers on new subnet resolved issue.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Share via

Deployment of Azure Container Instance Failed, where did I go wrong?

3 answers