'Microsoft.Streaming.Adapters.Exceptions.EventHubAdapterException' thrown when fetching AVRO events from eventhub

ronnyBravo 20 Reputation points
2024-08-30T10:38:20.8766667+00:00

We have setup a client in our app that is pushing AVRO encoded events to Azure Eventhub. We are using Schema Registry to encode the events.

To use the events in power BI we would like to process incoming events into JSON format and save into blob storage. We are trying to use Stream Analytics job for this. However, in the Stream Analytics the input fails to fetch events and throws error 'Microsoft.Streaming.Adapters.Exceptions.EventHubAdapterException'.

I am not sure what is the issue and how to debug this. The inputs connection to Eventhub is reported as OK and it is configured as follows:

ksnip_20240830-133709

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
646 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
360 questions
0 comments
Accepted answer
  1. phemanth 11,125 Reputation points Microsoft Vendor
    2024-09-02T11:29:34.19+00:00

    @ronnyBravo

    I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

    **Ask :**We have setup a client in our app that is pushing AVRO encoded events to Azure Eventhub. We are using Schema Registry to encode the events.

    To use the events in power BI we would like to process incoming events into JSON format and save into blob storage. We are trying to use Stream Analytics job for this. However, in the Stream Analytics the input fails to fetch events and throws error 'Microsoft.Streaming.Adapters.Exceptions.EventHubAdapterException'.

    I am not sure what is the issue and how to debug this. The inputs connection to Eventhub is reported as OK and it is configured as follows:

    ksnip_20240830-133709

    Solution: I've fgured out what the issue was. As per @phemanth answer it turned out to be authentication issue. We are using RBA and I had not added correct roles for my stream analytics job service principal in eventhub and schema registry.

    If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

    If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

    Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

    0 comments

2 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. phemanth 11,125 Reputation points Microsoft Vendor
    2024-08-30T13:01:33.42+00:00

    @ronnyBravo

    Thanks for the question and using MS Q&A platform.

    I understand you’re encountering a tricky issue with your Stream Analytics job. Here are a few steps you can take to troubleshoot and potentially resolve the EventHubAdapterException:

    Check Consumer Group: Ensure that the consumer group you’re using is not $Default. Creating a new consumer group often resolves this issue.

    Event Hub Configuration: Verify that the Event Hub configuration, including the namespace, name, and consumer group, is correct. Double-check the authentication mode and ensure that the managed identity has the necessary permissions.

    Serialization Format: Confirm that the event serialization format is correctly set to Avro and that the Schema Registry settings are properly configured.

    Input Preview: In the Stream Analytics job, try selecting a time range in the input preview to see if you can fetch any events. Sometimes, not selecting a time range can cause issues.

    Event Size: Ensure that the events being sent to the Event Hub are not exceeding the size limit (256 KB). Larger events can cause errors.

    Restart Job: If the partition count of the Event Hub has changed, you may need to stop and restart the Stream Analytics job.

    Hope this helps. Do let us know if you any further queries.

    1 person found this answer helpful.
    0 comments
  2. ronnyBravo 20 Reputation points
    2024-09-02T06:58:08.6066667+00:00

    I've fgured out what the issue was. As per @phemanth answer it turned out to be authentication issue. We are using RBA and I had not added correct roles for my stream analytics job service principal in eventhub and schema registry.

    Thanks for the help!

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.