PublishedPipeline Class

Represents a Pipeline to be submitted without the Python code which constructed it.

In addition, a PublishedPipeline can be used to resubmit a Pipeline with different PipelineParameter values and inputs.

Initialize PublishedPipeline.

:param endpoint The REST endpoint URL to submit pipeline runs for this pipeline. :type endpoint: str :param total_run_steps: The number of steps in this pipeline :type total_run_steps: int :param workspace: The workspace of the published pipeline. :type workspace: azureml.core.Workspace :param continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun

if a step fails, default is false.

Inheritance
azureml.core._portal.HasPipelinePortal
PublishedPipeline

Constructor

PublishedPipeline(name, graph_id, description, version, published_pipeline_id, status, endpoint, total_run_steps, workspace, continue_on_step_failure=None, _pipeline_provider=None, **kwargs)

Parameters

Name Description
name
Required
str

The name of the published pipeline.

graph_id
Required
str

The ID of the graph for this published pipeline.

description
Required
str

The description of the published pipeline.

version
Required
str

The published pipeline version.

published_pipeline_id
Required
str

The ID of the published pipeline.

status
Required
str

The status of the published pipeline ('Active' or 'Disabled').

endpoint
Required
str

The REST endpoint URL to submit runs for this pipeline.

total_run_steps
Required
int

The number of steps in this pipeline.

workspace
Required

The workspace of the published pipeline.

continue_on_step_failure
Required

Whether to continue execution of other steps in the PipelineRun if a step fails. The default is false.

_pipeline_provider
Required
<xref:azureml.pipeline.core._workflow_provider._PublishedPipelineProvider>

The published pipeline provider.

kwargs
Required

Custom keyword arguments, reserved for future development

name
Required
str

The name of the published pipeline.

graph_id
Required
str

The ID of the graph for this published pipeline.

description
Required
str

The description of the published pipeline.

version
Required
str

The published pipeline version.

published_pipeline_id
Required
str

The ID of the published pipeline.

status
Required
str

Status of the published pipeline ('Active' or 'Disabled').

_pipeline_provider
Required
<xref:azureml.pipeline.core._workflow_provider._PublishedPipelineProvider>

The published pipeline provider.

kwargs
Required

Custom keyword arguments, reserved for future development

Remarks

A PublishedPipeline can be created from either a Pipeline or a PipelineRun.

An example to publish from a Pipeline is as follows:


   from azureml.pipeline.core import Pipeline

   pipeline = Pipeline(workspace=ws, steps=steps)
   published_pipeline = pipeline.publish(name="My_New_Pipeline",
                                         description="My New Pipeline Description",
                                         version="1.0",
                                         continue_on_step_failure=True)

To publish from a PipelineRun use:


   from azureml.pipeline.core import PipelineRun

   pipeline_run = PipelineRun(experiment=Experiment(ws, "Pipeline_experiment"), run_id="run_id")
   published_pipeline = pipeline_run.publish_pipeline(name="My_New_Pipeline",
                                                      description="My New Pipeline Description",
                                                      version="1.0",
                                                      continue_on_step_failure=True)

Note: the continue_on_step_failure parameter specifies whether the execution of steps in the Pipeline will continue if one step fails. The default value is False, meaning when one step fails, the Pipeline execution will stop, canceling any running steps.

Submit a PublishedPipeline using submit. When submit is called, a PipelineRun is created which in turn creates StepRun objects for each step in the workflow.

An example to submit a PublishedPipeline is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline)

There are a number of optional settings that can be specified when submitting a PublishedPipeline. These include:

  • continue_on_step_failure: Whether to continue execution of other steps in the PipelineRun if a step fails, optional. Only steps that have no dependency on the output of the failed step will continue execution. If provided, this parameter setting overrides the setting on the Pipeline.

  • pipeline_parameters: Parameters to pipeline execution, dictionary of {name: value}. See PipelineParameter for more details.

  • parent_run_id: You can supply a run ID to set the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to the same experiment as the pipeline being submitted.

An example to submit a PublishedPipeline using these settings is as follows:


   from azureml.pipeline.core import PublishedPipeline

   published_pipeline = PublishedPipeline.get(workspace=ws, id="published_pipeline_id")
   pipeline_run = experiment.submit(published_pipeline,
                                    continue_on_step_failure=True,
                                    pipeline_parameters={"param1": "value1"},
                                    parent_run_id="<run_id>")

All published pipelines have a REST endpoint. With the pipeline endpoint, you can trigger a run of the pipeline from external systems, such as non-Python clients. For information about how to authenticate when calling REST endpoints, see https://aka.ms/pl-restep-auth.

Using the endpoint enables "managed repeatability" in batch scoring and retraining scenarios, for example. For more information, see https://aka.ms/pl-first-pipeline.

Methods

disable

Set the published pipeline to 'Disabled' and unavailable to run.

enable

Set the published pipeline to 'Active' and available to run.

get

Get the published pipeline.

get_all

Get all published pipelines in the current workspace.

DEPRECATED: This method is being deprecated in favor of PublishedPipeline list method.

get_graph

Get the graph of the PublishedPipeline.

get_step_names

Get the list of names of steps in the PublishedPipeline.

list

Get all published pipelines in the current workspace.

save

Save the Pipeline YAML to a file.

Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting.

submit

Submit the published pipeline. This is equivalent to using submit.

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

disable

Set the published pipeline to 'Disabled' and unavailable to run.

disable()

enable

Set the published pipeline to 'Active' and available to run.

enable()

get

Get the published pipeline.

static get(workspace, id, _workflow_provider=None, _service_endpoint=None)

Parameters

Name Description
workspace
Required

The workspace the published pipeline was created in.

id
Required
str

The ID of the published pipeline.

_workflow_provider
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>

The workflow provider.

Default value: None
_service_endpoint
str

The service endpoint.

Default value: None

Returns

Type Description

A PublishedPipeline object.

get_all

Get all published pipelines in the current workspace.

DEPRECATED: This method is being deprecated in favor of PublishedPipeline list method.

static get_all(workspace, active_only=True, _service_endpoint=None)

Parameters

Name Description
workspace
Required

The workspace the published pipeline was created on.

active_only

Whether to return only published pipelines which are currently active.

Default value: True
_service_endpoint
str

The service endpoint.

Default value: None

Returns

Type Description

A list of PublishedPipeline objects.

get_graph

Get the graph of the PublishedPipeline.

get_graph(_workflow_provider=None)

Parameters

Name Description
_workflow_provider
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>

The workflow provider.

Default value: None

Returns

Type Description

The graph.

get_step_names

Get the list of names of steps in the PublishedPipeline.

get_step_names(_workflow_provider=None)

Parameters

Name Description
_workflow_provider
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>

The workflow provider.

Default value: None

Returns

Type Description

The list of the names of steps in the PublishedPipeline.

list

Get all published pipelines in the current workspace.

static list(workspace, active_only=True, _service_endpoint=None)

Parameters

Name Description
workspace
Required

The workspace the published pipeline was created in.

active_only

Whether to return only published pipelines which are currently active.

Default value: True
_service_endpoint
str

The service endpoint.

Default value: None

Returns

Type Description

A list of PublishedPipeline objects.

save

Save the Pipeline YAML to a file.

Currently, only pipelines consisting of ModuleSteps are supported for YAML exporting.

save(path=None, _workflow_provider=None)

Parameters

Name Description
path
str

The path to save the YAML to. If the path is a directory, the Pipeline YAML file is saved at path/pipeline_name.yml. If path is None, the current directory is used.

Default value: None
_workflow_provider
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>

The workflow provider.

Default value: None

Returns

Type Description

submit

Submit the published pipeline. This is equivalent to using submit.

Returns the submitted PipelineRun. Use this object to monitor and view details of the run.

submit(workspace, experiment_name, pipeline_parameters=None, _workflow_provider=None, _service_endpoint=None, parent_run_id=None, continue_on_step_failure=None)

Parameters

Name Description
workspace
Required

The workspace to submit the published pipeline on.

experiment_name
Required
str

The name of the experiment to submit to.

pipeline_parameters

A dictionary of parameters to assign new values {param name, param value}. See PipelineParameter for more details.

Default value: None
_workflow_provider
<xref:azureml.pipeline.core._aeva_provider._AevaWorkflowProvider>

The workflow provider.

Default value: None
_service_endpoint
str

The service endpoint.

Default value: None
parent_run_id
str

Optional run ID to set for the parent run of this pipeline run, which is reflected in RunHistory. The parent run must belong to same experiment as this pipeline is being submitted to.

Default value: None
continue_on_step_failure

Whether to continue execution of other steps in the PipelineRun if a step fails, optional. If provided, will override the setting on the Pipeline.

Default value: None

Returns

Type Description

The submitted pipeline run.

Attributes

continue_on_step_failure

Get the value of the continue_on_step_failure setting.

Returns

Type Description

The value of the continue_on_step_failure setting.

description

Get the description of the published pipeline.

Returns

Type Description
str

The description of the published pipeline.

endpoint

Get the REST endpoint URL for running a published pipeline.

Returns

Type Description
str

The REST endpoint URL for running the published pipeline.

graph_id

Get the ID of the graph for this published pipeline.

Returns

Type Description
str

The ID of the graph.

id

Get the published pipeline ID.

Returns

Type Description
str

The ID of the published pipeline.

name

Get the name of the published pipeline.

Returns

Type Description
str

The published pipeline name.

status

Get the status of the published pipeline.

Returns

Type Description
str

The status of the published pipeline.

total_run_steps

Get the number of steps in the pipeline.

Returns

Type Description
int

The number of steps in the pipeline.

version

Get the version of the published pipeline.

Returns

Type Description
str

The version of the published pipeline.