How can I read files stored in Azure file share using databricks. I am able to access container blob storage, but not the files in file share.

Hello @Jeeva , In additional to @Vaibhav Chaudhari response. Before you start loading Azure Files to Azure Databricks, make sure the Azure Storage File module is installed. To install Azure Storage File module, you need to use: pip install azure-storage-file . Once module is installed you follow the stackoverflow thread to load the Azure Files to Azure Databricks. Hope this helps. Do let us know if you any further queries. Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how . Want a reminder to come back and check responses? Here is how to subscribe to a notification .

Below similar question has the answer which uses SAS Key in the URL which can be used to access file share in Databricks https://stackoverflow.com/questions/59928574/load-file-from-azure-files-to-azure-databricks Please don't forget to Accept Answer and Up-vote if the response helped -- Vaibhav

Read files from Azure File share using databricks

Accepted answer

PRADEEPCHEEKATLA-MSFT 89,466 Reputation points Microsoft Employee

2020-10-21T10:28:11.627+00:00
Hello @Jeeva ,

In additional to @Vaibhav Chaudhari response.

Before you start loading Azure Files to Azure Databricks, make sure the Azure Storage File module is installed.

To install Azure Storage File module, you need to use: pip install azure-storage-file

.

Once module is installed you follow the stackoverflow thread to load the Azure Files to Azure Databricks.

Hope this helps. Do let us know if you any further queries.

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.

Want a reminder to come back and check responses? Here is how to subscribe to a notification.
Please sign in to rate this answer.
Jeeva 161 Reputation points

2020-10-21T12:44:22.713+00:00

Pradeep. Thank you for your response.

I did install azure storage before and below is the error I get.

cannot import name 'FileService' from 'azure.storage.fileshare' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-5ff9c3eb-8d7b-48b1-b943-7f26d8bbd316/lib/python3.7/site-packages/azure/storage/fileshare/init.py

cannot import name 'FilePermissions' from 'azure.storage.fileshare' (/local_disk0/.ephemeral_nfs/envs/pythonEnv-5ff9c3eb-8d7b-48b1-b943-7f26d8bbd316/lib/python3.7/site-packages/azure/storage/fileshare/init.py)

PRADEEPCHEEKATLA-MSFT 89,466 Reputation points Microsoft Employee

2020-10-22T07:09:18.307+00:00

Hello @Jeeva ,

In order to investigate further, could you please share the below details.

Azure Databricks Runtime Version?

How you had installed Azure Storage module?

Code snippet which you are trying?

Complete stack trace of the error message along with the screenshot?

PRADEEPCHEEKATLA-MSFT 89,466 Reputation points Microsoft Employee

2020-10-27T04:47:15+00:00

Hello @Jeeva ,

Follow up to see if you have had a chance to see the previous response. We need the following information to understand/investigate this issue further.

Jeeva 161 Reputation points

2020-10-27T14:17:35.427+00:00

Hi Pradeep. Thank you for the follow-up. I figured and the issue is resolved. Thanks for the assistance.

David Browne - msft 3,846 Reputation points

2020-10-27T15:01:04.05+00:00

Note that pip install only loads the package on the driver node, and so must be loaded to pandas first. To make the library available to the Spark worker nodes it needs to be installed as a Databricks Library.

PRADEEPCHEEKATLA-MSFT 89,466 Reputation points Microsoft Employee

2020-10-28T03:28:44.43+00:00

Hello @Jeeva ,

Glad to know that your issue has resolved
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Read files from Azure File share using databricks

1 additional answer

Your answer