This issue is resolved now.
This linkudr was very useful in getting the job done.
I mean data exfiltration protection with Azure Databricks and Azure Firewall Premium.
That being said, working with the firewall was never an easy task.
This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
Azure Firewall: Hub & Spoke architecture
Databricks: 7.4 runtime
Azure region: West Us2
I followed this data-exfiltration-protection-with-azure-databricks.html detailed walkthrough guide to setup protect data exfiltration with Azure Firewall and Databricks in our dev environment. After the setup, I was never be able to start my Databricks cluster.
Any help to resolve this error is very much appreciated.
Logs on the cluster
Cluster terminated. Reason: Storage Download Failure
Failed to download bootstrap script from artifact storage. Please check that instances have connectivity to cloud storage.
Instance bootstrap failed command: DownloadBootstrapScript
Failure message: (Base64 encoded) VM extension code: ProvisioningState/succeeded
----REDACTED----
instanceId: InstanceId(3cf2fd8a96e5407cb868d636fdee9d48)
workerEnv: workerenv-979924835046728
Additional details (may be truncated):
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestus2a1.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestus2a3.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: dbartifactsprodwestus2.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestusa14.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: arprodwestusa11.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] Command DownloadBootstrapScript finished. Storage Account: dbartifactsprodwestus.blob.core.windows.net [FAILED] [Will try other storage accounts]
[Bootstrap Event] {FAILED_COMMAND:DownloadBootstrapScript}
[Bootstrap Event] {FAILED_MESSAGE:(Base64 encoded) ----REDACTED----}
[Bootstrap Event] SELF BOOTSTRAP FAILED
[Bootstrap Event] Can reach arprodwestus2a1.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach arprodwestus2a3.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach dbartifactsprodwestus2.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach arprodwestusa14.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach arprodwestusa11.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach dbartifactsprodwestus.blob.core.windows.net: [SUCCEEDED]
[Bootstrap Event] Can reach westus.azuredatabricks.net: [SUCCEEDED]
Firewall Application Rule Collection
Firewall Network Rule Collection
This issue is resolved now.
This linkudr was very useful in getting the job done.
I mean data exfiltration protection with Azure Databricks and Azure Firewall Premium.
That being said, working with the firewall was never an easy task.
@Porsche Me how did you solve this issue. I'm facing the same error and not sure how to move forward