Separate data in data explorer and use as datastore

yjay 256 Reputation points
2021-04-21T20:19:09.177+00:00

Hello,

We are sending data from IoT Central to Event Hubs and then to Data Explorer, with the hopes of then sending the data to Azure Machine Learning.

In order to send data from Event Hubs to Data Explorer it needs a data ingestion into a table on data explorer.

For this data ingestion, it needs a json mapping.

We could ingest the data, but the message from the iot central data goes to event hubs that goes to data explorer carries the telemetry data as a dynamic type (a json inside a json). 

("telemetry":{"Temp:"37","Vol":"97"})

90018-data.jpg
We want to separate the telemetry data in different columns.

So Temp will have one column and Vol another.

I am wondering how that can be done?

And additionally, since we would like to send the data to ML, can data explorer be used as a datastore in ML?

Thanks!!

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,955 questions
Azure IoT Central
Azure IoT Central
An Azure hosted internet of things (IoT) application platform.
364 questions
Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
530 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
646 questions
0 comments
Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee
    2021-04-22T08:36:44.29+00:00

    Hello @yjay ,

    You can use parse operator - Evaluates a string expression and parses its value into one or more calculated columns. The calculated columns will have nulls, for unsuccessfully parsed strings.

    For more details, refer SO thread addressing similar issue.

    Unfortuantely, Azure Data Explorer is not a supported storage solution with Azure Machine Learning.

    Datastores currently support storing connection information to the storage services listed in the following matrix.

    90159-image.png

    For unsupported storage solutions, and to save data egress cost during ML experiments, move your data to a supported Azure storage solution.

    Reference: Connect to storage services on Azure - Azure Machine Learning.

    Hope this helps. Do let us know if you any further queries.

    ------------

    Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.