Cannot mount Azure VM(windows server) file share on Azure Databricks instance.

Hi,

I wanted to mount windows file share(from Windows server VM in Azure) on a databricks instance. To test it I followed these steps.

1. I created a Windows VM and a Linux VM in azure.
2. I created VNet-peering between these two VMs.
3. After creating the VNet-peering, windows VM was able to ping the private IP address of the Linux VM and vice-versa.
4. I used the command sudo mount -t cifs -o username=XXXX //x.x.x.x/Downloads ./local
5. The mount worked. I could list the files in the remote directory.
6. Similarly, I created a databricks workspace in the same vNet which is being peered with the windows VM virtual network.
7. After creating a cluster, I tried to mount the windows file share from the cluster terminal. The mount is not working.
8. To check the communication between the databricks instance and windows vm, I tried pinging them from each other.
9. Now, ping from databricks instance is reaching the private ip of the windows vm. but windows vm cannot ping databricks private ip. windows vm can ping default gateway of databricks instance. we need some config so that pings from windows vm to databricks private ip.
10. This is the problem, I believe if the ping works, we will be able to mount the file share.

Here are some screenshots:

SS from databricks terminal. Private IP: 10.2.3.4 subnet CIDR: 25

Databricks pinging WindowsVM1

Default gateway of databricks: 10.2.2.4

WindowsVM1 receiving pings from databricks default gateway.

WindowsVM1 sending failed pings to databricks private ip 10.2.3.4

WindowsVM1 sending successful pings to databricks default gateway 10.2.2.4