Azure Data Explorer Replacing old data with new data ingested from blob storage

tsun 26 Reputation points
2021-02-03T21:21:15.093+00:00

Hello everyone,

I'm currently ingesting data from a blob storage into Azure Data Explorer using Event Hubs. I'm trying to plot out some graphs, with each one containing a set of frequency values and power values.

I noticed that whenever a new dataset comes in, when I refresh to see the updated graphs, the new dataset gets appended to the old table. As a result, a graph is essentially trying to plot two sets of data rather than just the new one.

I was wondering if there was a way to make it so that whenever a new dataset gets uploaded into the blob and read into data explorer, the old data in the table gets replaced with the new data?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
502 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. tsun 26 Reputation points
    2021-02-04T20:33:05.22+00:00

    Just wanted to update everyone that I found a simple solution to this issue.

    I simply added a time stamp to each set of data and filtered it such that only the most recent dataset gets plotted.

    1 person found this answer helpful.