Event Hub Incoming and Outgoing message have a long gap

dotnet_guy 10 Reputation points
2024-07-30T19:16:44.4466667+00:00

Hello,

We have an Azure function triggered by Event Hubs. There is over 7-15min varying gap between each function execution(s). We see about 1k messages coming in constantly but the outgoing spikes every 7-15min are over 8k messages and sometimes around 10k as well. The overall incoming and outgoing messages nearly match as soon as soon as the spike is done. But there are some missing, will tackle that later. There are no errors in App Insights and Log Analytics.

Now I am wondering why the spikes and not have a uniform incoming and outgoing as soon as the messages come in, the memory for the spike is high (~350MB). The function is always on, so no cold start. I did not any additional config, for example to fire the function after a certain time or number of events. We are using 32 partitions and have standard, not premium tier.

Please let me know if I am missing something and how the spikes can be evened out.

Thanks.

Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
4,890 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
627 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Pinaki Ghatak 3,905 Reputation points Microsoft Employee
    2024-07-31T11:38:57.9+00:00

    Hello @dotnet_guy

    One possible reason could be that the function is processing the incoming messages in batches, and the outgoing spikes are the result of the function processing a batch of messages.

    Another possible reason could be that the function is experiencing resource constraints, such as CPU or memory limitations, which are causing delays in processing the incoming messages.

    To investigate further, you can try the following steps:

    1. Check the configuration of your Azure function to ensure that it is not set to fire after a certain time or number of events. You mentioned that you did not configure any additional settings, but it's worth double-checking to make sure.
    2. Check the resource utilization of your Azure function during the spikes in outgoing messages. You mentioned that the memory usage is high during the spikes, but it would be helpful to also check the CPU usage and other resource metrics to see if there are any constraints that could be causing delays in processing the incoming messages.
    3. Check the configuration of your Event Hubs instance to ensure that it is not set to throttle incoming messages. Throttling can cause delays in processing incoming messages, which could result in spikes in outgoing messages.
    4. Consider upgrading to the premium tier of Event Hubs, which provides higher throughput and more predictable performance. The standard tier is designed for moderate workloads, and if you are experiencing spikes in outgoing messages, it's possible that you are hitting the limits of the standard tier.

    I hope that this response has addressed your query and helped you overcome your challenges. If so, please mark this response as Answered. This will not only acknowledge our efforts, but also assist other community members who may be looking for similar solutions.

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.