Old container instances keep running in our linux azure app service after a new version is deployed. How do I get rid of those?

Lafontaine, Mathieu 110 Reputation points
2024-08-25T18:10:38.5533333+00:00

We are deploying an .NET 6 web app to a linux app service with DevOps by publishing a new docker image to container registry and instructing the app service to use the new image. The new version is indeed deployed but there are up to 4 older versions of the app that are somehow still running and processing messages. Our solution consists of a Hangfire server and we noticed that more server showed up in the dashboard and we finally figured out they were older servers that should've been removed when we deployed a new version. These 4 old instances were there more than 18 hours after our deployment. They don't seem to disappear even if we restart our app service. Azure portal App service dashboard only reports a single instance of our app running, but the resource monitor shows higher memory usage and our Hangfire dashboard reports more servers than actually deployed. The only way we got rid of these old instances was by scaling up or down our app service plan so that the underlying machine was swapped. Someone else seems the have reported the same behavior here. The is a major issue for our solution. Something seems completely broken with the linux app service docker image deployment process.

Edit: also, the rogue instances seem to keep executing even if we stop the app service

Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,676 questions
Accepted answer
  1. Saif Badran 81 Reputation points Microsoft Employee
    2024-09-10T08:33:05.55+00:00

    The cause of this issue was found to be due to a recent code bug, a recent upgrade to the container orchestration service on the VMs hosting web apps caused an issue with disposing of old containers during recycling. This meant that sometimes, old containers weren't properly removed. The issue has been fixed and applied globally as of August 27th, and immediately caused duplicate containers to stop spawning.

    If you have already scaled your plan up and down to mitigate the issue, the same problem should not happen again.

    1 person found this answer helpful.
    0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.