Cannot delete event hubs with an active incoming events source

dotnet_guy 10 Reputation points
2024-06-17T21:12:50.2+00:00

Hello,

We enabled Entra ID logs to send to Event Hub, and when we would need to change configuration in our terraform modules, I change it and try to deploy via Azure DevOps pipelines. Everything worked until April end, but just before the Memorial Day weekend, the event hub doesn't delete and gives an error in the pipeline 'Event Hub already exists' while trying to destroy and create. It does not take our custom partition count and it creates a default 4 count. I think this is created automatically due to the incoming events source.

When I remove all the incoming event sources to Event Hub, then I can delete and deploy the hub. Removing all the incoming sources needs a CR from multiple teams, and we are planning to add more in the future and would be a big hassle to move forward. Please let me know if there is a recent change around that or if there is another way I can deploy my hub.

Thanks.

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
585 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
20,311 questions
0 comments

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pinaki Ghatak 2,720 Reputation points Microsoft Employee
    2024-06-18T05:57:58.6566667+00:00

    Hello @dotnet_guy

    Let’s break down the problem and explore potential solutions:

    1. Artifact Already Exists Error: The error message “Event Hub already exists” suggests that the service connection might already be present, and you may not have the appropriate permissions to view or modify it. To address this, follow these steps:
      • Go to Project settings in Azure DevOps.
      • Navigate to Service connections under Pipelines.
      • Check if the “Windows Azure MSDN - Visual Studio Premium” service connection already exists.
    2. Custom Partition Count Issue: If the Event Hub is not taking your custom partition count and defaults to 4, consider the following:
      • Ensure that your Terraform configuration specifies the desired partition count correctly.
      • Verify that there are no conflicting settings or overrides in your pipeline or Terraform files.
      • Check for any recent changes in your Terraform modules or Azure DevOps pipeline configurations.
    3. Incoming Event Sources: Removing all incoming event sources to the Event Hub allows you to delete and deploy it successfully. However, this approach is not ideal due to the hassle of coordinating CRs from multiple teams.
      • Consider reviewing the incoming event sources and their impact on the Event Hub. Is there a way to manage these sources more efficiently?
      • Explore whether you can dynamically adjust the configuration based on the presence of incoming sources.
    4. Recent Changes: It’s essential to check for any recent updates or changes in Azure DevOps, Terraform, or Event Hub behavior. Review release notes, documentation, and community forums to identify any relevant updates or known issues.

    Remember to involve your team members and Azure DevOps experts to troubleshoot further.

    I hope that this response has addressed your query and helped you overcome your challenges. If so, please mark this response as Answered. This will not only acknowledge our efforts, but also assist other community members who may be looking for similar solutions.

  2. PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee
    2024-06-19T04:29:37.95+00:00

    @dotnet_guy - Thanks for the question and using MS Q&A platform.

    It seems like you are facing an issue while deleting an Event Hub that has an active incoming events source. This is expected behavior as you cannot delete an Event Hub that is actively receiving events.

    To delete the Event Hub, you will need to stop all incoming events sources to the Event Hub. Once all the incoming events sources are stopped, you should be able to delete the Event Hub.

    If you are unable to stop all incoming events sources, you can try scaling down the Event Hub to a lower capacity, which may allow you to delete it. You can also try creating a new Event Hub with the desired configuration and migrate your incoming events sources to the new Event Hub.

    Regarding the issue with the partition count, it is possible that the default partition count is being used because the Event Hub is being recreated with the same name. To ensure that the custom partition count is used, you should delete the existing Event Hub and create a new one with the desired partition count.

    Hope this helps. Do let us know if you any further queries.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.