How can i capture events in avro from eventhub as per the my specific schema i.e .avsc and not in a manner where i get body as bytes.

Junaid Nazir 5 Reputation points
2024-02-07T07:08:52.3666667+00:00

as i capture events from event hub as acro files i get the data as per below schema { "type" : "record", "name" : "EventData", "namespace" : "Microsoft.ServiceBus.Messaging", "fields" : [ { "name" : "SequenceNumber", "type" : "long" }, { "name" : "Offset", "type" : "string" }, { "name" : "EnqueuedTimeUtc", "type" : "string" }, { "name" : "SystemProperties", "type" : { "type" : "map", "values" : [ "long", "double", "string", "bytes" ] what can i do if i require the data in avros from event hub as per different schema that above. } }, { "name" : "Properties", "type" : { "type" : "map", "values" : [ "long", "double", "string", "bytes", "null" ] } }, { "name" : "Body", "type" : [ "null", "bytes" ] } ] }

Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
2,869 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
591 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 28,606 Reputation points Microsoft Employee
    2024-02-07T21:16:18.6366667+00:00

    Hello https://video2.skills-academy.com/en-us/users/na/?userid=f08f5e59-9067-4e51-ab7b-358fdb37a585

    After the data is captured, you can use Apache Spark to read the Avro files, extract the Body field, and then parse it according to your specific schema.

    Here is a blog post that explains this.

    https://caiomsouza.medium.com/processing-event-hubs-capture-files-avro-format-using-spark-azure-databricks-save-to-parquet-95259001d85f

    https://video2.skills-academy.com/en-us/azure/event-hubs/explore-captured-avro-files

    I hope this helps.

    0 comments