Run fbprophet from ADX without SAS keys

SEYNAEVE Gerdy (SGI/MST) 6 Reputation points
2023-10-25T08:43:35.9733333+00:00

I have been trying our this example - trying to run fbprophet within Adx as documented here :

https://video2.skills-academy.com/en-us/azure/data-explorer/kusto/functions-library/series-fbprophet-forecast-fl?tabs=query-defined&pivots=azuredataexplorer

We are not allowed to use SAS keys to connect different components, and have to rely on managed identities

So I proceeded as follows :

  • enabled the python plugin (3.6.5) on the cluster
  • created a blob storage account & uploaded the fbprophet zip file
  • changed the function to use the storage account see below)
  • Gave the ADX system managed identity access to the storage account (blob storage contributor)
     | evaluate python(typeof(*), code, kwargs,
 external_artifacts=bag_pack('fbprophet.zip', 'https://<mystorageaccountname>.blob.core.windows.net/fbprophet/fbprophet-0.7.1.zip'))
  • Added a callout policy for the managed identity - query in kusto
.alter-merge database db policy managed_identity ```
[
  {
    "ObjectId": "<managed identity of the Adx cluster>",
    "AllowedUsages": "SandboxArtifacts"
  }
]


- I also have the following callout policy 

```javascript
  {
    "CalloutType": "sandbox_artifacts",
    "CalloutUriRegex": "<mystorageaccountname>\\.blob\\.core\\.windows\\.net/fbprophet/",
    "CanCall": true
  },

When I try to run the forecasting algorithm, I get the following error :

Details=Access to persistent storage path 'https://<mystorageaccountname>.blob.core.windows.net/fbprophet/fbprophet-0.7.1.zip' was denied (operation 'BlobPersistentStorageFile.DownloadToFileAsync')

Any ideas if something is still missing ?

Azure Data Explorer
Azure Data Explorer
An Azure data analytics service for real-time analysis on large volumes of data streaming from sources including applications, websites, and internet of things devices.
501 questions

2 answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. QuantumCache 20,186 Reputation points
    2023-10-25T17:17:04.04+00:00

    Hello @SEYNAEVE Gerdy (SGI/MST)

    Check if there are any network restrictions in place that might be blocking the ADX system managed identity from accessing the blob storage account. You can try to access the blob storage account from a different Azure resource (e.g., a VM) using the same managed identity to see if it works?

  2. SEYNAEVE Gerdy (SGI/MST) 6 Reputation points
    2023-12-29T12:46:55.26+00:00

    Finally found the issue : In case of using manage identities, the connectionstring specifically has to mention to use the system managed_identity

    external_artifacts=bag_pack('fbprophet.zip', 'https://<storageaccountname>.blob.core.windows.net/fbprophet/fbprophet-0.7.1.zip;managed_identity=system'))

    0 comments