Hi All, I have a little inquiry regarding ‘<a href="https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp%22https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp%22" rel="ugc nofollow">Receive change notifications through Azure Event Hubs’ . As you know, we load Azure Databricks Diagnostic logs (plus a few system tables via Azure EventGrid) into Azure EventHub, which we then consume into ADX and run KQL queries. This is all fine. What I am trying to achieve here is to enable another internal team to access our Databricks logs directly through Azure EventHub using this ‘Receive change notification’ approach explained in the above article. For this test, I have set up another EventHub and configured it in our Azure Databricks workspace to send Diagnostic logs. Logs are flowing in this EventHub. Please note, this new EventHub has nothing to do with the ADX as it is not part of any ADX Data connection. Now my objective is to enable this ‘other internal team’ to receive notifications whenever there is any new content in this EventHub. Also, they should be able to consume the content directly out of EventHub. I have followed all the steps in this article. What I am slightly confused about is ‘Create the subscription and receive notifications’. Is it correct to understand that: I need to <a href="https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#create-the-subscription%22https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#create-the-subscription%22" rel="ugc nofollow">create a subscription so that this ‘other XYZ team’ can receive notifications whenever there is new content in the EventHub? If true, then I need to offer them a ‘ notificationUrl’ through which they would receive a ‘change notification’ that there is a change. However, I am then confused about the ‘<a href="https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22" rel="ugc nofollow">Receive notifications ’ part. In other words, what is the difference between ‘receiving a change notification’ as part of the create subscription and ‘receive the notifications’ described under <a href="https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22" rel="ugc nofollow">Receive notifications ? If you could briefly explain by giving an example for my use case, it would be helpful. What information do I need to provide to this ‘other internal team’ so that their application can programmatically receive the Databricks diagnostics whenever there is a change? Do I need to provide them with both the Send and Listen EventHub Shared Access Policy ‘Connection string–primary keys’? I need to precisely understand this <a href="https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22https://video2.skills-academy.com/en-us/graph/change-notifications-delivery-event-hubs?tabs=change-notifications-eventhubs-azure-portal%2Cchange-notifications-eventhubs-keyvault%2Chttp#receive-notifications%22" rel="ugc nofollow">article and approach explained here by Microsoft here so that I can take required steps in order to fulfill the objective. Lastly when it comes to this other team building a client to receive/process the event, they simply need to follow this “ receiving events ” after we provide them with the EventHub connection string (Send/Listen both)? Thanks in advance for your support.

Receive change notifications through Azure Event Hubs

Suleman 0

Hi All,

I have a little inquiry regarding ‘Receive change notifications through Azure Event Hubs’.

As you know, we load Azure Databricks Diagnostic logs (plus a few system tables via Azure EventGrid) into Azure EventHub, which we then consume into ADX and run KQL queries. This is all fine.

What I am trying to achieve here is to enable another internal team to access our Databricks logs directly through Azure EventHub using this ‘Receive change notification’ approach explained in the above article. For this test, I have set up another EventHub and configured it in our Azure Databricks workspace to send Diagnostic logs. Logs are flowing in this EventHub. Please note, this new EventHub has nothing to do with the ADX as it is not part of any ADX Data connection.

Now my objective is to enable this ‘other internal team’ to receive notifications whenever there is any new content in this EventHub. Also, they should be able to consume the content directly out of EventHub. I have followed all the steps in this article. What I am slightly confused about is ‘Create the subscription and receive notifications’.

Is it correct to understand that:

I need to create a subscription so that this ‘other XYZ team’ can receive notifications whenever there is new content in the EventHub?
If true, then I need to offer them a ‘notificationUrl’ through which they would receive a ‘change notification’ that there is a change. However, I am then confused about the ‘Receive notifications’ part. In other words, what is the difference between ‘receiving a change notification’ as part of the create subscription and ‘receive the notifications’ described under Receive notifications? If you could briefly explain by giving an example for my use case, it would be helpful.
What information do I need to provide to this ‘other internal team’ so that their application can programmatically receive the Databricks diagnostics whenever there is a change? Do I need to provide them with both the Send and Listen EventHub Shared Access Policy ‘Connection string–primary keys’?

I need to precisely understand this article and approach explained here by Microsoft here so that I can take required steps in order to fulfill the objective.

Lastly when it comes to this other team building a client to receive/process the event, they simply need to follow this “receiving events” after we provide them with the EventHub connection string (Send/Listen both)?

Thanks in advance for your support.

Sander van de Velde | MVP 30,711 Reputation points MVP

2024-06-07T12:12:14.0166667+00:00

Moderated: company name. Please do not share personal or sensitive information on this public forum.
PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee

2024-06-10T07:02:26.0733333+00:00

@Suleman - Thanks for the question and using MS Q&A platform.

As you are using Azure Event Hubs to load Azure Databricks Diagnostic logs and system tables. I'll do my best to help you with your inquiry regarding receiving change notifications through Azure Event Hubs.

Regarding your first question, yes, you need to create a subscription so that the other internal team can receive notifications whenever there is new content in the EventHub. The subscription is created on the Event Grid topic that is associated with the Event Hub. When you create a subscription, you specify a notification endpoint, which is a URL that Event Grid uses to send notifications to your application.

Regarding your second question, "receiving a change notification" as part of the create subscription means that you are setting up the subscription to receive notifications whenever there is a change in the Event Hub. On the other hand, "receive the notifications" described under Receive notifications means that you are actually receiving the notifications that Event Grid sends to your application.

For example, let's say you have set up a subscription to receive notifications whenever there is new content in the Event Hub. When new content is added to the Event Hub, Event Grid sends a notification to the notification endpoint that you specified in the subscription. Your application then receives the notification and can take action based on the contents of the notification.

Regarding your third question, you need to provide the other internal team with the Event Hub connection string that includes both the Send and Listen EventHub Shared Access Policy 'Connection string-primary keys'. This will allow them to programmatically receive the Databricks diagnostics whenever there is a change.

Lastly, when it comes to the other team building a client to receive/process the event, they simply need to follow the "receiving events" after you provide them with the Event Hub connection string (Send/Listen both).

I hope this helps! Let me know if you have any further questions.
Suleman Butt 1 Reputation point

2024-06-10T10:45:16.97+00:00
Many thanks for the reply.

>>``yes, you need to create a subscription so that the other internal team can receive notifications whenever there is new content in the EventHub. The subscription is created on the Event Grid topic that is associated with the Event Hub. When you create a subscription, you specify a notification endpoint, which is a URL that Event Grid uses to send notifications to your application.<<

How can I create a subscription? Or does the subscription creation mean, the creation of:

notificationUrl property that looks like this:

EventHub:https://<azurekeyvaultname>.vault.azure.net/secrets/<secretname>?tenantId=<domainname>

where the <secretname> contains the value for the above EventHub’s “Shared access policies” configured as “Send”? Do I also need the same for "ReceiveNoticiation" URL, where the <secretname> contains the value for the above EventHub’s “Shared access policies” configured as “Listen”.

Is that what Subscription creation mean? If so, then where EventGrid topic is getting created and associated with EventHub. The article does not talk about EventGrid here.

If this is NOT what "Subscription creation" mean then could you please precisely explain how can I create a Subscription for my EventGrid and then what needs to be shared with the the other team?

>>``Regarding your second question, "receiving a change notification" as part of the create subscription means that you are setting up the subscription to receive notifications whenever there is a change in the Event Hub. On the other hand, "receive the notifications" described under Receive notifications means that you are actually receiving the notifications that Event Grid sends to your application.<<

Again I am getting confused, but I hope when you further clarify my initial question regarding how to create "Subscription" then this point will be clarified too.

Does EventGrid send notification to notificationUrl described above as (“Shared access policies” configured as “Send”)? If so, then what role “Shared access policies” configured as “Listen” play and whether that needs to be shared with the other team as well?

Sorry If I am confusing things up, perhaps a short Teams session would be helpful?

Many thanks.
PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee

2024-06-12T05:53:48.92+00:00
@Suleman - To create a subscription for your EventGrid topic associated with the Event Hub, you can follow these steps:

In the Azure portal, navigate to your Event Grid topic.

Select the "Subscriptions" tab.

Select "Add subscription".

In the "Add subscription" pane, specify the following:

"Subscription name": A name for your subscription.

"Endpoint type": Select "Event Hub" from the dropdown.

"Endpoint": The connection string for your Event Hub. This includes the Event Hub namespace, Event Hub name, and the Shared Access Policy key with "Send" permissions.

"Event types": The types of events that you want to receive notifications for. For example, if you want to receive notifications for new Diagnostic logs in the Event Hub, you would select "Microsoft.Databricks/workspaces/diagnosticLogs".

"Filter to specific subject prefix": Optional. You can specify a subject prefix to filter the events further.

Select "Create".

Once you create the subscription, Event Grid sends notifications to the specified endpoint (notificationUrl) whenever an event that matches the specified event types and subject prefix is generated. The notification includes the event data and metadata.

Regarding your question about the Shared Access Policies, the "Send" policy is used to send events to the Event Hub, while the "Listen" policy is used to receive events from the Event Hub. In the subscription endpoint, you need to provide the connection string with the "Send" policy key to send notifications to the Event Hub. The "Listen" policy key is not needed for this scenario.

I hope this helps! Let me know if you have any further questions.
Suleman Butt 1 Reputation point

2024-06-12T13:43:45.9033333+00:00

@PRADEEPCHEEKATLA-MSFT Thanks for your reply.

Just for the sake of clarity or avoid any confusion, let me just reiterate:

Databricks diagnostic logs are getting ingested into Azure EventHub just fine, and I just need to enable this other team access onto these logs inside EventHub through this "receive change notification approach".

Now coming to your suggestion:

>>To create a subscription for your EventGrid topic associated with the Event Hub, you can follow these steps:....

..

I have created an "Event Grid System Topic (test-topic)" and associated(configured) with it the source (Type: Microsoft.Eventhub.Namespaces) as the same EventHub Namespace, which is where I have my Databricks logs already flowing.

Second step is the Event Subscription creation as how you described in your last comment.

Let me explain step by step what i see when i create Event Subscription.

Create Event Subscription (Event grid)

EVENT SUBSCRIPTION DETAILS:

Name: testEventSubscription

Event Schema: Event Grid Schema

TOPIC DETAILS

Pick a topic resource for which events should be pushed to your destination.

Topic Type: Event Hubs Namespace

Source Resource: evh-elsa-dbx-cp-ns-dev

Topic Name: test-topic

EVENT TYPES

Pick which event types get pushed to your destination.

Filter to Event Types: Capture File Created (dropdown with single option only)

ENDPOINT DETAILS

Pick an event handler to receive your events.

Endpoint Type: Event Hub (dropdown with multiple options)

Note: Are you sure this would be event Hub??

If true, should I pick the same Event Hub Namespace and Event Hub as source?

On this above configuration page, there is no place for:

- “”Endpoint": The connection string for your Event Hub. This includes the Event Hub namespace, Event Hub name, and the Shared Access Policy key with "Send" permissions.”

- "Event types": The types of events that you want to receive notifications for. For example, if you want to receive notifications for new Diagnostic logs in the Event Hub, you would select "Microsoft.Databricks/workspaces/diagnosticLogs".

After Event Subscription is created, it shows under my Event Grid System Topic (test-topic), Event Subsections as:

Name: testEventSubscription

Endpoint: EventHub

EventTyopes: Microsoft.EventHub.CaptureFileCreated

Provisioning State: Succeeded

I am not sure if it is correct. What information am I supposed to share with this other team who would like to consume Databricks diagnostic logs which are already flowing into EventHub? Connection string for the same Event Hub with “Send” permission? Which is same as "notificationUrl"?

If it is correct, can you kindly explain this conceptually once again? I am kind of finding it hard to wrap my head around.

Thanks,
PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee

2024-06-14T06:39:15.0933333+00:00
@Suleman - I apologize for any confusion caused. Let me clarify the process for you.

Firstly, you are correct that you need to create an EventGrid subscription to enable the other team to receive notifications whenever there is new content in the EventHub. The subscription is created on the EventGrid topic that is associated with the EventHub.

To create the subscription, you need to provide the following information:

The EventGrid topic that is associated with the EventHub.

The filter criteria for the events that you want to receive notifications for. For example, if you want to receive notifications for new Diagnostic logs in the EventHub, you would select "Microsoft.Databricks/workspaces/diagnosticLogs".

The notification endpoint, which is the URL that EventGrid uses to send notifications to your application. This URL should include the Shared Access Signature (SAS) token for the EventHub 'Send' policy.

Based on your description, it seems that you have created the EventGrid subscription correctly. The subscription is associated with the correct EventGrid topic and EventHub, and the filter criteria is set to receive notifications for new Diagnostic logs in the EventHub.

Regarding the endpoint details, you need to select 'Event Hub' as the endpoint type and provide the connection string for the EventHub 'Send' policy. This connection string should include the Event Hub namespace, Event Hub name, and the Shared Access Policy key with "Send" permissions. This is the same as the 'notificationUrl' that you mentioned earlier.

Once the subscription is created, you need to provide the other team with the following information:

The connection string for the EventHub 'Send' policy.

The filter criteria for the events that they want to receive notifications for. For example, if they want to receive notifications for new Diagnostic logs in the EventHub, they would need to know that the filter criteria is set to "Microsoft.Databricks/workspaces/diagnosticLogs".

The client application that they need to build to listen to the notification endpoint and process the notifications.

I hope this clarifies the process for you. Let me know if you have any further questions.I apologize for any confusion caused. Let me clarify the process for you.

Firstly, you are correct that you need to create an EventGrid subscription to enable the other team to receive notifications whenever there is new content in the EventHub. The subscription is created on the EventGrid topic that is associated with the EventHub.

To create the subscription, you need to provide the following information:

The EventGrid topic that is associated with the EventHub.

The filter criteria for the events that you want to receive notifications for. For example, if you want to receive notifications for new Diagnostic logs in the EventHub, you would select "Microsoft.Databricks/workspaces/diagnosticLogs".

The notification endpoint, which is the URL that EventGrid uses to send notifications to your application. This URL should include the Shared Access Signature (SAS) token for the EventHub 'Send' policy.

Based on your description, it seems that you have created the EventGrid subscription correctly. The subscription is associated with the correct EventGrid topic and EventHub, and the filter criteria is set to receive notifications for new Diagnostic logs in the EventHub.

Regarding the endpoint details, you need to select 'Event Hub' as the endpoint type and provide the connection string for the EventHub 'Send' policy. This connection string should include the Event Hub namespace, Event Hub name, and the Shared Access Policy key with "Send" permissions. This is the same as the 'notificationUrl' that you mentioned earlier.

Once the subscription is created, you need to provide the other team with the following information:

The connection string for the EventHub 'Send' policy.

The filter criteria for the events that they want to receive notifications for. For example, if they want to receive notifications for new Diagnostic logs in the EventHub, they would need to know that the filter criteria is set to "Microsoft.Databricks/workspaces/diagnosticLogs".

The client application that they need to build to listen to the notification endpoint and process the notifications.

I hope this clarifies the process for you. Let me know if you have any further questions.
Suleman 0 Reputation points

2024-06-17T12:04:12.48+00:00

@PRADEEPCHEEKATLA-MSFT ,

Unfortunately, you are not addressing the points for which I am seeking clarifications.

You are suggesting theoretical steps, but in practice, there are no placeholders under "Event Grid System Topic" Event Subscription to configure such things.

I have sent you repeated messages on MS Teams without a response, and I have also tried scheduling a meeting with you without success.

In my opinion, the only practical approach to solve the problem is to have a 30-minute session where we can discuss and configure things together.

Writing instructions here without understanding the problem's context is not very effective so far in my opinion.

If you believe you can really help resolve this issue, I would appreciate having a Teams session with you.

I created a support ticket with MSFT a week ago, but it seems no one knows how this works and no one is willing to join the session.

I appreciate your time writing these messages here, but honestly, it's been going on for some time with little real progress.
PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee

2024-06-18T05:41:42.44+00:00

@Suleman - Could you please share the support ticket number to track and ask the owner to respond asap?
Suleman Butt 1 Reputation point

2024-06-18T06:28:16.95+00:00

@PRADEEPCHEEKATLA-MSFT TrackingID#2406060040007462.

Thanks,
PRADEEPCHEEKATLA-MSFT 83,966 Reputation points Microsoft Employee

2024-06-19T03:58:12.5133333+00:00

@Suleman - Thanks for sharing the support request number.
Once the issue is sorted out with the support, please do share the resolution, which might be beneficial to other community members reading this thread.

Share via

Receive change notifications through Azure Event Hubs