Azure Container Instance terminated but the container is still transitioning

Luca Maurelli 30

Goodmorning,

I have some ACI resources that, randomly, as far as I can tell, complete with success as shown by logged events but the container is stuck in "the container is still transitioning...". The error seems not to resolve withing some time, e.g. it is stuck since many minutes.

The only way to resolve, to date, is to restart the container, which I don't want to force. I want to be able to start the container again, normally.

To date, I am not aware of any type of log that I can retrieve in order to identify the cause of this behaviour, after reading the Azure documentation. How can I troubleshoot the problem?

I also found out that in the metrics tab of the portal it seems the container is still exchanging network data even if the resource cpu/ram are not anymore used. Maybe it is stuck for this reason:

User's image

Thank you,

Luca

Inkinen Kai 10 Reputation points

2023-09-27T06:59:11.2566667+00:00

We have repeatedly experienced the same problem with some scheduled runs. This seems to happen without any changes from our side. We run a short-lived container on a schedule, and seen it go for several weeks without a problem. Now we've experienced this problem 3 times (on two different ACI setups (1x and 2x)) within a week. This has happened several times before, but at that point we've suspected it's due to some change made by us. Now we're positive that we've changed nothing (happened in both envs over the weekend, when nobody was near a keyboard). This has happened over the last year or so.

As a result of these problems we've deemed ACI not to be fit for production use and we're going to transition away to some other tool.

I'm following this thread, though, to figure out what's the root cause for the problem.
Tim van der Heijden 1 Reputation point

2023-09-29T06:00:00.18+00:00

We experienced the exact same problem, on multiple different tenants even. The setup is a combination of a regular triggered ACI by a Logic App. Out of nowhere the trigger starts failing and restart of the ACI is impossible.

@Eddie Neto If it is any help I can share some of my ACI names and resource group.
Luca Maurelli 30 Reputation points

2023-09-29T06:52:37.9033333+00:00

I redeployed from europe west to north italy with no success. Redeploying to north europe seems stable since last two days.
Juan Pablo Mena 0 Reputation points

2023-10-30T18:38:02.1133333+00:00

We have been experiencing the same problem. Have a short executing time ACI that executes every 6 minutes via a logic app. Since september, we've been having the problem that, at some random moment since the ACI was deployed, sometimes a few hours, sometimes one week, that container is stuck at succesful state but can not be restarted either manually or via the logic app, giving the error "the container is still transitioning". We know we haven't made any change to the image of the container, this error first happened at early september and has become more and more frequent since then. Last week ACI got stuck three times, only fix is to redeploy ACI until it fails again.
Erfan Nariman 436 Reputation points

2023-10-31T15:21:23.35+00:00

@Eddie Neto / Azure team, is it possible to share an update in the ETA and give this more priority? This issue has been going on for more than a month and it's quite an impactful one. We have experienced problems in multiple tenants of clients in West Europe.

Accepted answer

Eddie Neto 1,226 Reputation points Microsoft Employee

2023-09-27T11:07:35.7566667+00:00

Hi @Luca Maurelli

Thanks for reaching Microsoft Q&A.

Could you please share the the ACI name and Resource group name. Do not share your subscription here please.

Waiting for your feedback. Thanks
Please sign in to rate this answer.
Eddie Neto 1,226 Reputation points Microsoft Employee

2023-09-28T15:39:03.2+00:00

Hi @@Luca Maurelli and Inkinen Kai

There is a known issue on going, our team is aware of it and is applying a fix in all regions. For now, as possible mitigation please Stop your ACI and then Start again.

Hope this helps. Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.

Tim van der Heijden 1 Reputation point

2023-09-29T06:02:15.3966667+00:00

Thanks @Eddie Neto I now see your comment on your own post. Any ETA on those fixes (interested in West-Europe)?

Luca Maurelli 30 Reputation points

2023-09-29T06:55:17.2266667+00:00

@Eddie Neto The resource group name is "dockerization_testing_rg", while the ACIs resources are named, for instance, "algex-prod-c1" and "kpiex-prod". I redeployed from west europe to north europe and it seems stable since few days though. Let me know if you resolve. Thank you

Eddie Neto 1,226 Reputation points Microsoft Employee

2023-09-29T08:17:07.7933333+00:00

@Luca Maurelli @Tim van der Heijden @Inkinen Kai

The workaround for now as we still have our team ia taking a special look at "westeurope" region is to deploy your ACI in another region.

Fow now we don't have ETA to be shared, sorry for this inconvenience. I will keep you posted here as soon as we have the fix applied.

Hope this helps to clarify. Please "Accept as Answer" if it helped, so that it can help others in the community looking for help on similar topics.

Juha Vuorinen 0 Reputation points

2023-09-29T10:11:00.7466667+00:00

Unfortunately stop action is disabled on container instances what are in transitioning state.

Luca Maurelli 30 Reputation points

2023-09-29T13:41:19.2566667+00:00

As requested from @Eddie Neto I accepted the aswer to help people with the same problem.

Tim van der Heijden 1 Reputation point

2023-10-16T10:08:05.1133333+00:00

@Eddie Neto Any update on this issue? In some situations we have policies in place where we are only allowed to deploy in westeurope, making the workaround not possible.

Pfefferle Dominik 0 Reputation points

2023-10-17T09:14:27.63+00:00

@Juha Vuorinen Stop Container is disabled in the portal, but "az container stop" worked for me. I created a workaround in my "StartContainer" Azure Function with this command. If the "az container start" fails with this error, im firing a "az container stop"

Luca Maurelli 30 Reputation points

2023-10-24T06:45:11.53+00:00

Unfortunately, @Eddie Neto the solution provided by you, i.e. relocating to another region, stopped working for us around 22/10/2023 around 7 PM. In other words, some resources which stopped worked originally on west europe, and were working properly on north europe, stopped working on north europe also. Do you have some updates on this issue? We are considering stopping using the ACI resources because they are not reliable for our customers for this reason, a feedback about the situation would be highly appreciated. Thank you

Erfan Nariman 436 Reputation points

2023-10-31T15:27:51.7333333+00:00

@Eddie Neto any updates on this issue and an ETA?

Jan Simecka 0 Reputation points

2024-03-05T15:18:26.87+00:00

Hello @Eddie Neto ,

has this issue been fixed also in West Europe region since your last comment?

We had previously faced this issue and for some time now it seems stable, so wanted to confirm we don't have to apply the workaround anymore.
Sign in to comment

Share via

Azure Container Instance terminated but the container is still transitioning

0 additional answers