How to ingest data in ADX using python script

Pranav Kumar 0 Reputation points
2024-05-02T09:39:20.6033333+00:00

I am using free trial and trying to ingest data into ADX using python SDK but facing this issue

An error occurred: Principal 'msauser=1:live.com:00037FFE6E855292' is not authorized to perform operation 'Ingest' on 'any database'.

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
501 questions
0 comments No comments
{count} votes

2 answers

Sort by: Most helpful
  1. Sander van de Velde | MVP 30,711 Reputation points MVP
    2024-05-02T10:37:38.6033333+00:00

    Hello @Pranav Kumar,

    welcome to this moderated Azure community forum.

    Although not written in Python, see this guide to ingest data programmatically using a regular Azure Data Explorer.

    Is this what you want to achieve?

    As you can see, your code relies on an identity to access the database.

    If this does not work for your case, I recommend checking out the free ADX streaming ingestion via an Event Hub. Check out this post for details.


    If the response helped, do "Accept Answer". If it doesn't work, please let us know the progress. All community members with similar issues will benefit by doing so. Your contribution is highly appreciated.


  2. Wilko van de Velde 2,226 Reputation points
    2024-05-03T14:17:57.18+00:00

    Hi @Pranav Kumar ,

    Microsoft provides several examples on how to ingest data with Python. For example on this mslearn page: Ingest data using the Azure Data Explorer Python library

    The GitHub repository of the SDK also contains several examples: https://github.com/Azure/azure-kusto-python/blob/master/azure-kusto-ingest/tests/sample.py

    But seeing at your error message, it looks more of an authorization issue. In an Azure Data Explorer cluster, you need to add cluster/database permissions to ingest data to a user. When using a free Azure Data Explorer cluster, this is not possible using the Azure portal. I am not sure if it will work, but you may try to add the permissions by kusto: https://video2.skills-academy.com/en-us/azure/data-explorer/kusto/management/manage-database-security-roles#add-and-drop-security-roles

    For a full overview of the feature comparisons: https://video2.skills-academy.com/en-us/azure/data-explorer/start-for-free#feature-comparison

    Kind Regards,

    Wilko


    Please do not forget to "Accept the answer” wherever the information provided helps you, this can be beneficial to other community members. If you have extra questions about this answer, please click "Comment".

    0 comments No comments