Eventhub offset inconsistent

Atul Shaurya 0 Reputation points
2023-11-16T15:37:12.7666667+00:00

I am reading events from Azure periodically. I am not using Azure's checkpointing, instaed I keep track of the offset from the most recent read so the next call will start reading from the previously stored offset.

I noticed I can successfully read a few times but then the offset number suddenly goes down.

Is there a number defined in the documentation where offsets roll over? Would using sequence number be a more reliable approach?

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
596 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 85,031 Reputation points Microsoft Employee
    2023-11-17T05:13:03.1166667+00:00

    @Atul Shaurya - Thanks for the question and using MS Q&A platform.

    It sounds like you are experiencing an issue with inconsistent offsets when reading events from Azure Event Hubs. While there is no specific number defined in the documentation where offsets roll over, it is possible that you are encountering a scenario where the offset is being reset due to a failover or other event.

    Using sequence numbers can be a more reliable approach to tracking the position in the event stream, as they are guaranteed to be unique and monotonically increasing. However, it's important to note that sequence numbers are not the same as offsets, and you will need to modify your code to use sequence numbers instead of offsets.

    If you are still experiencing issues with inconsistent offsets, you may want to consider using Azure's checkpointing feature, which allows readers to mark or commit their position within a partition event sequence. Checkpointing can help ensure that you are always reading from the correct position in the event stream, even in the event of a failover or other disruption.

    I hope this helps! Let me know if you have any further questions.

    0 comments