Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,408 questions
Authentication failed when accessing dataset in compute instance using its MSI via AzureML SDK v2
Shiyao Li
0
Reputation points Microsoft Employee
The dataset can be previewed in AML workspace when the associated datastore is created using identity-based auth with workspace managed identity. I have the same user managed identity assigned to workspace and compute instance. But it only failed in compute instance while using below code snippet to access the dataset.
# Set your subscription, resource group and workspace name:
subscription_id = "xxx"
resource_group = "xxx"
workspace = "xxx"
# connect to the AzureML workspace
ml_client = MLClient(
DefaultAzureCredential(), subscription_id, resource_group, workspace
)
filedataset_asset = ml_client.data.get(name="xxx", version="1")
tbl = mltable.load(f'azureml:/{filedataset_asset.id}')
df = tbl.to_pandas_dataframe()
df
{count} votes
-
Amira Bedhiafi 18,341 Reputation points2024-06-18T13:56:50.44+00:00 Your issue can be due to several reasons :
The user-managed identity is not properly assigned to both the Azure Machine Learning workspace and the compute instance.
The user-managed identity has the necessary roles assigned to access the ADLS, common roles required are
Storage Blob Data ReaderorStorage Blob Data Contributorfor read or read/write access, respectively.Example of how to use
ManagedIdentityCredential():from azure.identity import ManagedIdentityCredential from azure.ai.ml import MLClient # Set your subscription, resource group and workspace name subscription_id = "xxx" resource_group = "xxx" workspace = "xxx" # Use ManagedIdentityCredential for user-managed identity credential = ManagedIdentityCredential(client_id='your_user_managed_identity_client_id') # Connect to the AzureML workspace ml_client = MLClient(credential, subscription_id, resource_group, workspace) # Access the dataset filedataset_asset = ml_client.data.get(name="xxx", version="1") tbl = mltable.load(f'azureml:/{filedataset_asset.id}') # Convert to pandas dataframe df = tbl.to_pandas_dataframe() dfIf you still need help, don't forget to approach Azure support for further assistance.
Sign in to comment