How to add message headers in ADX along with message body?

Chandramohan, Srinivasan 5 Reputation points
2023-12-20T00:13:21.89+00:00

We've configured ADX with EH data connection. We're able to successfully ingest eventhub message body into our ADX database. We also need to add the message headers like correlation id, schema version, message id etc., in ADX database.

How to add message headers in ADX database/table?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
501 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 84,051 Reputation points Microsoft Employee
    2023-12-20T04:55:01.68+00:00

    @Chandramohan, Srinivasan - Thanks for the question and using MS Q&A platform.

    To add message headers along with the message body in Azure Data Explorer (ADX), you can use the ingest inline command. The ingest inline command allows you to specify the message headers and message body as a single JSON object.

    .ingest inline into table MyTable with (format="json")
[
  {
    "CorrelationId": "1234",
    "SchemaVersion": "1.0",
    "MessageId": "5678",
    "MessageBody": {
      "Field1": "Value1",
      "Field2": "Value2"
    }
  }
]

    Here is an example of how you can use the ingest inline command to add message headers and message body to an ADX table:

    In this example, we are ingesting a single message into the MyTable table. The message includes the message headers CorrelationId, SchemaVersion, and MessageId, as well as the message body MessageBody. The message headers and message body are specified as a single JSON object.

    You can modify this example to include your own message headers and message body. You can also use the ingest inline command to ingest multiple messages at once by specifying an array of JSON objects.

    For more details, refer to https://video2.skills-academy.com/en-us/azure/data-explorer/kusto/management/data-ingestion/ingest-inline

    I hope this helps! Let me know if you have any further questions.

    0 comments
  2. Sander van de Velde | MVP 30,711 Reputation points MVP
    2023-12-20T07:36:21.1633333+00:00

    Hello @Chandramohan, Srinivasan,

    welcome to this moderated Azure community forum.

    I expect you to use the ADX database data connection for EventHub.

    To connect you need to provide a mapping that can include the system properties.

    In the wizard, you need to activate the system properties you want to need.

    The properties you mention (correlation id, schema version, message id, etc.) do not seem to be part of the system properties.

    If you have added these yourself, these are called user properties or application properties.

    User properties / application properties are not supported by the connection at this moment.

    Please see also this 'ADX Feedback' and this 'ADX Feedback' pages.

    If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

    0 comments