An error occurred when using odbc and managed identity to link azure databricks in the. net core

@ZhiHong Zhang (BEYONDSOFT CONSULTING INC) - Thanks for the question and using MS Q&A platform.

It seems like you are facing an issue while using ODBC and managed identity to link Azure Databricks in .NET Core. The error message you are receiving indicates that there is an issue with accessing the Azure Metadata Service.

To use managed identity to authorize access and use Azure Databricks in the app service, you need to follow the steps mentioned in the reference document you shared.

However, I would like to clarify that the reference document you shared is for configuring ODBC authentication for Azure Databricks. If you are not using ODBC, you may not need to follow all the steps mentioned in the document.

To use managed identity to access Azure Databricks, you can follow the steps mentioned in the Azure Databricks documentation. You can use either system-assigned or user-assigned managed identity authentication.

For system-assigned managed identity authentication, you need to retrieve the managed identity information and grant the managed identity the correct permissions in Azure Databricks. In general, you must grant at least the Contributor role to your system-assigned managed identity in Access control (IAM) of Azure Databricks.

For user-assigned managed identity authentication, you need to create one or multiple user-assigned managed identities and grant permission in your Azure Databricks. In general, you must grant at least the Contributor role to your user-assigned managed identity in Access control (IAM) of Azure Databricks.

After you have granted the correct permissions, you can create credentials for each user-assigned managed identity and use them to authenticate your app service to Azure Databricks.

I hope this helps! Let me know if you have any further questions.

As you mentioned above, the basic permission settings have been determined to be set according to the document. After some testing, the same issue was still found.

But starting from step 5 in the document, it is necessary to configure Azure VM, but the current condition is that it is in the web app service, not Azure VM. And the current operating environment is. net core. All ODBC versions in the document must be at least 2.7.7. This is completely certain. When excluding external conditions and running in the code, I even take out Auth-ClientId and repeatedly use the set value. Then you will get this error:

Error [28000] [Simba] [DriverOAuthSupport] (8719) Can't access Azure metadata service, check if you run on VM with MI: OAuthAPIErr

And the speed is very fast, and the error is clearly pointed out as running in the VM. However, we are currently using a web app service and deploying it in a container style. Based on my guess, this authentication method did not recognize MI at all, so it immediately returned an exception.

I don't know if there are any examples that can be provided to me to correctly access Azure databricks and use query statements in the. net core environment of C # code. If there are no examples, would you like to inform me that they are not currently supported. Because there is a reference below indicating that some SDKs do not support it.

https://databricks-bi-artifacts.s3.us-east-2.amazonaws.com/simbaspark-drivers/odbc/2.8.0/SimbaSparkODBC-2.8.0.1002-Debian-64bit.zip

https://video2.skills-academy.com/en-us/azure/databricks/dev-tools/azure-mi-auth

https://video2.skills-academy.com/en-us/azure/databricks/dev-tools/auth/

As you mentioned above, the basic permission settings have been determined to be set according to the document. After some testing, the same issue was still found.

But starting from step 5 in the document, it is necessary to configure Azure VM, but the current condition is that it is in the web app service, not Azure VM. And the current operating environment is. net core. All ODBC versions in the document must be at least 2.7.7. This is completely certain. When excluding external conditions and running in the code, I even take out Auth-ClientId and repeatedly use the set value. Then you will get this error:

Error [28000] [Simba] [DriverOAuthSupport] (8719) Can't access Azure metadata service, check if you run on VM with MI: OAuthAPIErr

And the speed is very fast, and the error is clearly pointed out as running in the VM. However, we are currently using a web app service and deploying it in a container style. Based on my guess, this authentication method did not recognize MI at all, so it immediately returned an exception.

I don't know if there are any examples that can be provided to me to correctly access Azure databricks and use query statements in the. net core environment of C # code. If there are no examples, would you like to inform me that they are not currently supported. Because there is a reference below indicating that some SDKs do not support it.

https://databricks-bi-artifacts.s3.us-east-2.amazonaws.com/simbaspark-drivers/odbc/2.8.0/SimbaSparkODBC-2.8.0.1002-Debian-64bit.zip

https://video2.skills-academy.com/en-us/azure/databricks/dev-tools/azure-mi-auth

https://video2.skills-academy.com/en-us/azure/databricks/dev-tools/auth/

@ZhiHong Zhang (BEYONDSOFT CONSULTING INC) - I agree that this issue looks strange and I wasn't able to reproduce this issue. If you have a support plan could you please file a support ticket for deeper investigation and do share the SR# with us?

@ZhiHong Zhang (BEYONDSOFT CONSULTING INC) - Did you get a chance to open a support ticket?