How can we achieve real-time import, processing, and export of streaming data by using ADX?

jane 60 Reputation points
2023-11-24T07:52:14.11+00:00

The streaming data comes from Azure iothub, and I would like to process and export each data from iothub in the form of streaming data by using ADX to other containers, such as ADLS, CosmosDB, etc. Due to ASA's lack of flexibility, I am considering replacing ASA with ADX in my project, but I don't know how to address this issue. Could you give me some ideas or provide a simple example? I am looking forward to your reply.

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
524 questions
0 comments
Accepted answer
  1. Sander van de Velde | MVP 32,556 Reputation points MVP
    2023-11-24T11:42:59.14+00:00

    Hello jane,

    Azure Data Explorer is great for ingesting timeseries data and processing in inside the database using table update policies.

    Unfortunately, Azure Data Explorer does not generate events based on changed data.

    You need to use a polling strategy to check for new rows to occur.

    Azure Stream Analytics is in fact a good choice to distribute data to different endpoints/resources.

    I'm not sure what you mean with lack of flexibility.

    Having the ability to join streams and reference data, a n extensive query language, easily extensible with javascript or even machine learning is far from beging not flexible.

    The fact you can put the logic under version control via Visual Studio Code or even test the query real time, is a blizz.

    Just give it some time if you are new to Azure Stream Analytics.

    There are alternatives though.

    Check out Azure Functions (custom code), Logic apps (a no-code experience) or Azure Data Factory as alternatives to Azure Stream Analytics.

    If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.