Azure Stream analytics map column when writing parquet files

Eran Shemesh 0 Reputation points Microsoft Employee
2023-10-19T10:42:05.63+00:00

I was trying to use the capture feature for eventhub, writing Parquet files to blob storage.
The UI passed me for creating a Azure Stream Analytics.
I have a Json structure in the eventhub, which contains an object field, like the following:

{
	"map-column": {
		"key1": ["value1", "value12"]
		"key2": ["value1", "value2"]
	}
}

Writing it to parquet with the suggested query on Stream Analytics:

WITH [eventhub] AS (SELECT * FROM [eventhub-input]) SELECT * INTO [blob-output] FROM [eventhub]

This creates Parquet files with a struct column, when my intention was to write a map column.

Didn't find a way to change the written schema types.

Does anybody know?

Thanks!

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
591 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
342 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. PRADEEPCHEEKATLA-MSFT 84,456 Reputation points Microsoft Employee
    2023-10-26T09:48:53.6333333+00:00

    @Eran Shemesh - Thanks for the question and using MS Q&A platform.

    Here is the confirmation from internal team: This is not supported as of today.

    Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

    Hope this helps. Do let us know if you any further queries.

    If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

    0 comments