Failed to execute command group with error An unexpected error occurred while executing command due to: Failed to pull Docker image `mcr.microsoft.com/azureml/runtime/exe/execution-wrapper/installed:eastus-stable` due to: None

Jivjot Singh (IIIT-D) 0 Reputation points
2024-07-29T09:50:59.2433333+00:00

I am creating an azure pipeline through the python sdk v2
The following code is for a small script that i use to test

# test_script.py
import pandas as pd
from datetime import datetime
import os
 
data = {'Timestamp': [datetime.now()]}
df = pd.DataFrame(data)

print("Test script executed successfully.")

output_filename = 'test_output.csv'

from azureml.core import Workspace, Datastore
subscription_id = '-'
resource_group = '-'
workspace_name = '-'
ws = Workspace(subscription_id, resource_group, workspace_name)

datastore = Datastore.get(ws, datastore_name='-')

datastore.upload_files(
    files=[output_filename],
    target_path='test_folder',
    overwrite=True,
    show_progress=True
)
print("Output saved")

Then I use the following code to create a pipeline job

# test_pipeline.py
from azure.identity import DefaultAzureCredential
from azure.ai.ml import MLClient
subscription_id = "-"
resource_group = "-"
workspace_name = "-"
ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group, workspace_name)

from azure.ai.ml.entities import CommandComponent, Environment
command_component = CommandComponent(
    name="testing_pipeline",
    display_name="Pipeline Test",
    description="Adding a test script as a pipeline job",
    version="1",
    command="python test_script.py",
    code="./",
    environment='TestEnvGPU2:0'
)

from azure.ai.ml.dsl import pipeline
@pipeline(default_compute="-")   # Test-Instance-GPU   Test-Instance-CPU
def my_pipeline():
    experiment_job = command_component()
    return {}

pipeline_job = my_pipeline()
submitted_job = ml_client.jobs.create_or_update(pipeline_job)

print(f"Pipeline job submitted. Job ID: {submitted_job.name}")

The same code runs on a CPU instance and creates a pipeline job successfully which i can schedule too. But while running this on a GPU compute instance, the pipeline fails with the following error related to docker image

Failed to execute command group with error An unexpected error occurred while executing command due to: Failed to pull Docker image mcr.microsoft.com/azureml/runtime/exe/execution-wrapper/installed:eastus-stable due to: None

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,833 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.