Configure scaling

Article
08/22/2024

Configure scaling settings to manage the performance and cost of your Managed DevOps Pool. For information on pricing and performance, see Manage cost and performance.

Agent state

Managed DevOps Pools can be configured as stateless or stateful.

Stateless pools - Provide a fresh agent for every job.
Stateful pools - Allow sharing of agents between multiple jobs.

The default setting for a Managed DevOps pool is stateless (Fresh agent every time) but in some cases teams might want to reuse agents in order to reuse the packages or files created during the previous pipeline run. Build workload is a common scenario where teams want to preserve state and reuse agents. You can achieve stateful pools through Managed DevOps Pools while balancing it with security best practices. By default an agent can be reused for a maximum of 7 days but you can configure it to be recycled sooner.

Note

Stateless pools or use of the agent state setting Fresh agent every time are recommended by security experts as a defense against supply chain attacks.

Stateless pools

When a stateless agent is configured, a new agent is procured for each job, and is discarded after the job completes.

Azure portal
ARM template

Screenshot of a stateless agent.

Agents are configured using the agentProfile property in the Managed DevOps Pools resource. In the following example, a Stateless agent is specified.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless"
                }
            }
        }
    ]
}

When Agent state is set to Fresh agent every time, a new agent is procured for each job, and is discarded after the job completes.

Screenshot of a stateful agent.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "maxAgentLifetime": "7.00:00:00",
                    "gracePeriodTimeSpan": "00:30:00",
                    "kind": "Stateful"
                }
            }
        }
    ]
}

When Same agent can be used by multiple builds ("kind": "stateful") is enabled, agents in the pool are considered to be stateful. Stateful pools are configured using the following settings.

Max time to live for standby agents (maxAgetLifetime) configures the maximum duration an agent in a stateful pool can run before it is shut down and discarded. The format for Max time to live for standby agents is dd.hh:mm:ss. The default value of Max time to live for standby agents is set to the maximum allowed duration of seven days (7.00:00:00).
Grace Period (gracePeriodTimeSpan) configures the amount of time an agent in a stateful pool waits for new jobs before shutting down after all current and queued jobs are complete. The format for Grace Period is dd.hh:mm:ss and the default is no grace period.

While agents in stateless pools are shut down and discarded after every job, agents in stateful pools continue running if any of the following conditions are met.

If there is another job queued when the first job completes, Managed DevOps Pools sends that job to the agent that ran the first job instead of shutting it down.
If there is a grace period configured for the pool, agents wait for new jobs for the duration specified by the grace period before shutting down.
If standby agents are enabled, and the agent image meets the criteria of the active provisioning period, the agent continues to run and wait for jobs.

Running agents in stateful pools are shut down and discarded if they run continuously for the duration specified by Max time to live for standby agents, even if the previous conditions are true. For example, if Max time to live for standby agents is configured for three days, and Standby agent mode is set to Manual, All Week Scheme (Machines available 24/7), the agents are restarted after three continuous days of uptime.

Important

Agents in stateful pools can still be shut down and discarded after a job completes if there is no grace period, no active provisioning period for standby agents, and no queued jobs matching the agent. Once an agent is discarded, any state is lost.

Grace period enables the most cost effective way of running stateful pools for pipelines with consistent load and does not require the use of standby agent mode to keep agents online and ready to accept jobs.

Standby agent mode

When you create a pool, Standby agent mode is off by default, and there are no standby agents to immediately assign to your pipelines, which might have to wait a few moments, up to 15 minutes, for an agent to be provisioned on demand. For better performance, enable Standby agent mode and configure a standby agent schedule that provides capacity for your workload.

Off - Standby agent mode is off and agents are provisioned on-demand when jobs are queued.
Manual - Configure a manual standby schedule.
Automatic - Use an automatic standby schedule based on agent usage history and configurable for cost and performance.

Azure portal
ARM template

Screenshot of standby agent mode selection.

Standby agents are configured using the resourcePredictionsProfile section of the agentProfile property. Set "kind": "Manual" to configure a start from scratch, weekday scheme, or all week scheme, and specify the details of the scheme in the resourcePredictions section. Set "kind": "Automatic" to configure automatic standby agents. Omit the ResourcePredictionsProfile section to disable standby agents. See the following sections for details on how to configure each scaling type.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "kind": "Manual"
                    },
                    "resourcePredictions": {...}
                }
            }
        }
    ]
}

Manual

Manual mode is best suited for teams that have knowledge of their CI/CD pipelines usage patterns. If you select the manual option, you need to define your pre-provisioning scheme based on your understanding of when agents in the pool are most likely to get used and how many agents are likely to be used, and specify a provisioning count of agents that meet the projected demand.

You can create your own provisioning schedule or choose from one of the predefined schedules, and you can configure the time zone to use for specifying the schedules. The default value for Pre-provisioning TimeZone is (UTC) Coordinated Universal Time.

Tip

The provisioning count in a scheme can't be greater than the Maximim agents configured in Pool settings.

Every few minutes, Managed DevOps Pools checks the count of active agents running jobs and standby agents waiting for jobs, to ensure that the provisioning count of agents specified by the current provisioning scheme are available. If the current provisioning count is 10, and there are five agents running jobs and two agents on standby, Managed DevOps Pools will start three additional standby agents to bring the total agent count up to 10.

Manual standby agent configuration can be configured in one of the following three ways.

Start from scratch - Configure a set of provisioning periods for standby agents
Weekday Scheme (Machines available during time period every weekday) - Configure a start and end time for standby agents to be available each weekday
All Week Scheme (Machines available 24/7) - Configure a fixed number of standby agents to be continuously available

Each of the pre-provisioning quick starts has the following common settings in addition to the specific settings for that quick start.

Pre-provisioning TimeZone allows you to configure the time zone for the times in your pre-provisioning scheme. The default value for Pre-provisioning TimeZone is (UTC) Coordinated Universal Time.
Standby agent percentage configures the percentage of standby agents you want for each image. You can enter * to ensure all images are provisioned equally, or you can specify an integer from 0 to 100 to represent a percentage. If you specify a percentage, the total for all images must equal 100. If you have a single image, specify * or 100. Standby agent percentage is configured in the images section when using ARM templates. For more information, see Configure images.

Azure portal
ARM template

Screenshot of manual standby mode.

Manual standby agent provisioning is specified in the resourcePredictionsProfile section of agentProfile, and the details are configured in the resourcePredictions section.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "kind": "Manual"
                    },
                    "resourcePredictions": {
                        "timeZone": "Eastern Standard Time",
                        "daysData": [
                            {},
                            {
                                "00:00:00": 1,
                                "04:00:00": 0
                            },
                            {},
                            {},
                            {},
                            {},
                            {}
                        ]
                    }
                }
            }
        }
    ]
}

Specify the desired time zone for your scheme using the timeZone property. The default is UTC. To retrieve a list of time zone names for this property, see TimeZoneInfo.GetSystemTimeZones Method.

The schedule for the standby agents is defined by the daysData list. The daysData list can have either one item or seven items.

A daysData list with seven items maps to the days of the week, starting with Sunday. Each of these seven items can have zero or more "time": count entries, specifying a time in 24 hour format, and a standby agent count. The specified count of standby agents is maintained until the next "time": count entry, which can be on the same day, or on a following day.

A daysData list with a single item defines an All Week scheme, where the single "time": count entry corresponds to the standby agent count for the entire week.

The following example is a manual standby agent scheme, using Eastern Standard Time, with a single agent provisioned Monday through Friday, from 9:00 AM (standby agent count 1) through 5:00 PM (standby agent count 0).

{
    "kind": "Stateless",
    "resourcePredictions": {
        "timeZone": "Eastern Standard Time",
        "daysData": [
            {},
            {
                "09:00:00": 1,
                "17:00:00": 0
            },
            {
                "09:00:00": 1,
                "17:00:00": 0
            },
            {
                "09:00:00": 1,
                "17:00:00": 0
            },
            {
                "09:00:00": 1,
                "17:00:00": 0
            },
            {
                "09:00:00": 1,
                "17:00:00": 0
            },
            {}
        ]
    },
    "resourcePredictionsProfile": {
        "kind": "Manual"
    }
}

A single daysData item contains a dictionary of times and standby agent counts. Each "time" : count entry specifies the number of standby agents to schedule starting at the specified time, in 24 hour format. Consecutive "time" : count entries specify a sequence of scheduled agent counts for that day.

"daysData": [
    {}, # Schedule of standby agent count adjustments for Sunday
    {   # Schedule of standby agent count adjustments for Monday
        "09:00:00": 1, # Adjust standy agent count to 1
        "17:00:00": 0  # Adjust standby agent count to 0
    },
    {  # Schedule of standby agent count adjustments for Tuesday
        "09:00:00": 1,
        "17:00:00": 0
    },
    {  # Schedule of standby agent count adjustments for Wednesday
        "09:00:00": 1,
        "17:00:00": 0
    },
    {  # Schedule of standby agent count adjustments for Thursday
        "09:00:00": 1,
        "17:00:00": 0
    },
    {  # Schedule of standby agent count adjustments for Friday
        "09:00:00": 1,
        "17:00:00": 0
    },
    {} # Schedule of standby agent count adjustments for Saturday
]

Standby agent counts do not automatically reset back to zero at the end of a day or at the end of the week, and specifying an empty daysData item does not disable standby agents for that day. An empty daysData item means that there are no changes to the standby agent count schedule for that day. To set the standby agent to zero starting at a specific time period, you must explicitly provide a "time" : count entry with a count of 0.

Examples

To make no adjustment to the standby agent count specified at the conclusion of the previous day (or week if you are configuring the first period of the week), specify a daysData item with zero entries.

{
}

To schedule a single standby agent to start at 09:00:00 and stop at 17:00:00 (using the time zone specified by the resourcePredictions property), specify the following configuration.

{
    "09:00:00": 1,
    "17:00:00": 0
}

To schedule a single standby agent starting from midnight through 09:00:00, followed by 10 standby agents until 17:00:00, specify the following configuration.

{
    "00:00:00": 1,
    "09:00:00": 10,
    "17:00:00": 0
}

To schedule a standby agent to be available starting at 09:00:00 on the specified day, and stopping at 17:00:00 the following day, use two consecutive daysData items.

{
    "09:00:00": `1`
},
{
    "17:00:00": 0
}

Start From scratch

If you choose to start from scratch, you can add a list of provisioning periods to serve as your provisioning scheme. Each provisioning period consists of a start day, end day, time zone, start time, end time, and a count. Provisioning periods can't overlap each other.

Property	Description
Multi-day	When checked, you can configure both a Start Day and an End Day for your provisioning scheme.
Until next period	When checked, the provisioning period runs from the Start Time until the start of the next provisioning period.
Start Day	The day that the provisioning period starts.
End Day	The day the provisioning period ends. Required if Multi-Day is checked.
Start Time	The time the provisioning period starts.
End Time	The time the provisioning period ends. Required unless Until next period is checked.
Count	The number of standby agents to provision. This number must be greater than zero, and must not be greater than the Maximum agents value configured in Pool settings.

After creating a provisioning period, you can delete or edit the period from the Pre-provisioning scheme list.

The following example configures a manual scheme with 1 agent provisioned on Monday mornings from 12:00 AM to 5:00 AM EST.

Azure portal
ARM template

Screenshot of manual scaling scheme.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "kind": "Manual"
                    },
                    "resourcePredictions": {
                        "timeZone": "Eastern Standard Time",
                        "daysData": [
                            {},
                            {
                                "00:00:00": 1,
                                "04:00:00": 0
                            },
                            {},
                            {},
                            {},
                            {},
                            {}
                        ]
                    }
                }
            }
        }
    ]
}

Weekday scheme

If you choose the weekday scheme, you can specify a start time and end time in which the specified number of standby agents will be on standby each weekday.

Property	Description
Start Time	The time the provisioning period starts.
End Time	The time the provisioning period ends.
Provisioning Count	The number of standby agents to provision. This number must be greater than zero, and must not be greater than the Maximum agents value configured in Pool settings.

The following example configures four agents to be used during working hours with 0 agents during non-working hours and weekends, using Eastern Standard Time.

Azure portal
ARM template

Screenshot of weekday scheme.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "kind": "Manual"
                    },
                    "resourcePredictions": {
                        "timeZone": "Eastern Standard Time",
                        "daysData": [
                            {},
                            {
                                "09:00:00": 4,
                                "17:00:00": 0
                            },
                            {
                                "09:00:00": 4,
                                "17:00:00": 0
                            },
                            {
                                "09:00:00": 4,
                                "17:00:00": 0
                            },
                            {
                                "09:00:00": 4,
                                "17:00:00": 0
                            },
                            {
                                "09:00:00": 4,
                                "17:00:00": 0
                            },
                            {}
                        ]
                    }
                }
            }
        }
    ]
}

All Week Scheme

If you choose the all week scheme, you can specify a number of agents you want available 24/7.

Azure portal
ARM template

Screenshot of all week scheme.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "kind": "Manual"
                    },
                    "resourcePredictions": {
                        "timeZone": "Eastern Standard Time",
                        "daysData": [
                            {
                                "00:00:00": 1
                            }
                        ]
                    }
                }
            }
        }
    ]
}

Automatic

If you don't know your usage patterns and want to rely on automatic forecasting based on past data, choose Automatic. You can balance between cost and agent performance using a slider with the following five options. Managed DevOps Pools runs a query over your past three weeks of historical data (if available), organizing queued sessions of the pool into five minute periods, and assigns the specified percentile (to avoid spikes) to each hour.

Most cost effective (MostCostEffective) - 10th percentile
More cost effective (MoreCostEffective) - 25th percentile
Balanced (default) (Balanced) - 50th percentile
More performance (MorePerformance) - 75th percentile
Best performance (BestPerformance) - 90th percentile

You can view the projected and actual usage for any specific day within the last week, or a seven day average.

Azure portal
ARM template

Screenshot of automatic scaling setting.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "name": "fabrikam-managed-pool",
            "type": "microsoft.devopsinfrastructure/pools",
            "apiVersion": "2024-04-04-preview",
            "location": "eastus",
            "properties": {
                ...
                "agentProfile": {
                    "kind": "Stateless",
                    "resourcePredictionsProfile": {
                        "predictionPreference": "Balanced",
                        "kind": "Automatic"
                    }
                }
            }
        }
    ]
}

Share via