Azure Data Explorer Plotting Old Data

tsun 26

Hi everyone,

I'm currently following this tutorial to set up my data explorer to be able to query data via event grid notifications: https://video2.skills-academy.com/en-us/azure/data-explorer/ingest-data-event-grid.

Setting up a web UI dashboard to plot the newest set of data shouldn't be too difficult since if a new blob gets created, you can just click refresh and the newest set of data should get plotted. However, I was wondering if it would be possible to plot an older set of data stored in the blob container? For example, if I wanted to see data I received yesterday, would it be possible to set it up such that I can just plot that dataset whenever I want without having to create a new tile or something?

I was wondering how I'd be able to go about this? Or is there another product from Azure that would be better suited for this?

Thanks,
Tobias

MartinJaffer-MSFT 26,051 Reputation points

2021-02-04T02:17:40.773+00:00
Hello @tsun and welcome to Microsoft Q&A.

If I understand correctly, all you really want to do is query the data explorer cluster to get the previous day's data, and plot it. Is this correct?

Data Explorer can do rendering (see step 6 of how to query data here).

As to whether there is a better suited product on Azure, that depends on a number of factors. There are many services which can be used to query and / or plot data. You will want to consider:

Latency ( batch jobs on a schedule or have the latest immediately available)

Your confidence in your technical abilities

Volume of data

Other services you want to integrate with
tsun 26 Reputation points

2021-02-04T15:35:08.927+00:00

Hi @MartinJaffer-MSFT

Thank you for the response.

More specifically for the data visualization, I was wondering if it'd be possible to set it up such that a tile can show any day's data through the click of a button on the dashboard interface rather than having to rewrite the KQL query every time you want to plot a different day's data?

At the moment, I'm querying the data through a blob storage. I was looking a little bit into SQL databases and was wondering if this would be better suited for what I'm trying to accomplish?

Some additional info if it's helpful: The data won't be coming in very frequently. A new set of data will be coming in around every 5 minutes or so. The message coming in will be around 500 bytes. Here is how my current architecture is set up: IoT Device-->Azure IoT Hub--> Azure Functions-->Blob Storage-->Event Grid-->Data Explorer.

I was wondering if this helps narrow down what could work for me?

1 answer

MartinJaffer-MSFT 26,051 Reputation points

2021-02-04T21:14:49.163+00:00

I think I found what you are looking for, @tsun .

In the Data Explorer Dashboard feature, there is the possibility to add parameters. The parameters would show up on the dashboard like a button or drop-down. By referencing the parameter in the query behind the panel, you can change the data shown dynamically.

https://video2.skills-academy.com/en-us/azure/data-explorer/dashboard-parameters
https://video2.skills-academy.com/en-us/azure/data-explorer/azure-data-explorer-dashboards#use-parameters
Please sign in to rate this answer.
tsun 26 Reputation points

2021-02-11T21:45:04.693+00:00

Hi @MartinJaffer-MSFT ,

Sorry for the late response. That looks very close to what I'm looking for! Just to make sure I understand it correctly, when the parameter is added in, you would have to go into a tile and there will be a drop down option to change the time range of the data?

I was wondering if we could go a step further than that. Would it be possible to have some sort of drop down menu to set up the time range of the data in the dashboard rather than an individual tile?

MartinJaffer-MSFT 26,051 Reputation points

2021-02-19T17:58:10.077+00:00

@tsun yes that is exactly possible, and happens by default.

The parameter is a drop down, and can be made to select a single value, or a time range.
The parameter is then used to alter any tile which uses the parameter in its query.
Sign in to comment

Share via

Azure Data Explorer Plotting Old Data

1 answer