Background removal (version 4.0 preview)

  • Article

Important

This feature is now deprecated. On January 10, 2025, the Azure AI Image Analysis 4.0 Segment API and background removal service will be retired. All requests to this service will fail after this date.

To maintain a smooth operation of your models, install the open-source Florence 2 model and use its Region to segmentation feature, which allows for a similar background removal operation.

The Background removal operation can divide images into multiple segments or regions to help the user identify different objects or parts of the image. Background removal creates an alpha matte that separates the foreground object from the background in an image. This service is currently in preview, and the API may change in the future.

Call the Background removal API

This feature provides two possible outputs based on the customer's needs:

  • The foreground object of the image without the background. This edited image shows the foreground object and makes the background transparent, allowing the foreground to be placed on a new background.
  • An alpha matte that shows the opacity of the detected foreground object. This matte can be used to separate the foreground object from the background for further processing.

Important

Background removal is only available in certain Azure regions. See Region availability.

Background removal examples

The following example images illustrate what the Image Analysis service returns when removing the background of an image and creating an alpha matte.

Original image With background removed Alpha matte
Photo of a city near water. Photo of a city near water; sky is transparent. Alpha matte of a city skyline.
Photo of a group of people using a tablet. Photo of a group of people using a tablet; background is transparent. Alpha matte of a group of people.
Photo of a group of bears in the woods. Photo of a group of bears; background is transparent. Alpha matte of a group of bears.

Limitations

It's important to note the limitations of background removal:

  • Background removal works best for categories such as people and animals, buildings and environmental structures, furniture, vehicles, food, text and graphics, and personal belongings.
  • Objects that aren't prominent in the foreground may not be identified as part of the foreground.
  • Images with thin and detailed structures, like hair or fur, may show some artifacts when overlaid on backgrounds with strong contrast to the original background.
  • The latency of the background removal operation will be higher, up to several seconds, for large images. We suggest you experiment with integrating both modes into your workflow to find the best usage for your needs (for instance, calling background removal on the original image versus calling foreground matting on a downsampled version of the image, then resizing the alpha matte to the original size and applying it to the original image).

Use the API

The background removal feature is available through the Segment API (imageanalysis:segment). See the Background removal how-to guide for more information.

Next steps