User level auditing in Databricks Azure

Ayushri Jain 176 Reputation points
2021-04-26T09:00:01.47+00:00

We are looking into collecting audit logs for Azure Databricks. We have enabled diagnostic settings for the databricks workspace which sends log data to log analytics, it contains the monitoring audit logs, as in high level - who created which cluster/notebook, who ran which cluster/notebook, who deleted what, who accessed what, etc.
However, we want to get more insights - which user ran which command/query in databricks, basically user level auditing. We have checked the cluster logs-JSON files-eventlog, driver and executor- there is a description column in those json log files but it does not give complete commands/query, it gives first 20-25 characters and then displays "..."

Any other way to get the complete commands/queries?
Thanks in advance!

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,162 questions
Accepted answer
  1. PRADEEPCHEEKATLA-MSFT 88,716 Reputation points Microsoft Employee
    2021-04-27T06:27:00.287+00:00

    Hello @Ayushri Jain ,

    Unfortunately, User level auditing for Azure Databricks is not available for now.

    Note: We don’t have plans for supporting user level auditing at this time.

    I would suggest you to vote up an idea submitted by another Azure customer.

    https://feedback.azure.com/forums/909463-azure-databricks/suggestions/41425570-audit-and-log-notebook-commands

    All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

    Hope this helps. Do let us know if you any further queries.

    ------------

    Please don’t forget to Accept Answer and Up-Vote wherever the information provided helps you, this can be beneficial to other community members.

    0 comments

0 additional answers

Sort by: Most helpful
Most helpful Newest Oldest

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.