KQL Text Aggregation

Manuprasad M N 40 Reputation points
2023-12-22T10:58:33.6+00:00

In ADX , I have a table with a column named 'Names' that contains values like the following:

Names
["Name1","Name2"]
[""]
["Name1"]
["Name3"]

I need to create a new column that combines the values of each row without duplicates. The desired output is:

Names CombinedNames
["Name1","Name2"] ["Name1","Name2"]
[""] ["Name1","Name2"]
["Name1"] ["Name1","Name2"]
["Name3"] ["Name1","Name2", "Name3"]

How can I achieve this in KQL (Azure)?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
501 questions
0 comments No comments
{count} votes

Accepted answer
  1. Wilko van de Velde 2,226 Reputation points
    2023-12-24T08:41:32.19+00:00

    Hi @Manuprasad M N ,

    To accomplish you need several Kusto functions:

    1. To get the value from the previous row, user the scan operator
    2. Combine 2 arrays, use the set_union operator
    3. To remove the empty string, use my-apply and make_list_if

    This will result in the query below:

    datatable (MyData:dynamic )
    [dynamic(["Name1","Name2"]),
    dynamic([""]),
    dynamic(["Name1"]),
    dynamic(["Name3"])]
    | serialize 
    | scan declare (data_combined_and_filtered: dynamic=dynamic(null)) with 
    (
        //combine and sort the array
        step s1: true => data_combined_and_filtered = array_sort_asc(set_union(MyData, s1.data_combined_and_filtered));
    )
    //remove empty strings
    | mv-apply data_combined_and_filtered on (
        summarize value = make_list_if(data_combined_and_filtered, isnotempty(data_combined_and_filtered))
    )
    

    Hope this will help.

    Kind Regards,

    Wilko


    Please do not forget to "Accept the answer” wherever the information provided helps you, this can be beneficial to other community members. If you have extra questions about this answer, please click "Comment".

    1 person found this answer helpful.
    0 comments No comments

0 additional answers

Sort by: Most helpful