Azure AI Video Indexer terminology & concepts

  • Article

Warning

Over the past year, Azure AI Video Indexer (VI) announced the removal of its dependency on Azure Media Services (AMS) due to its retirement. Features adjustments and changes were announced and a migration guide was provided.

The deadline to complete migration was June 30, 2024. VI has extended the update/migrate deadline so you can update your VI account and opt in to the AMS VI asset migration through August 31, 2024.

However, after June 30, if you have not updated your VI account, you won't be able to index new videos nor will you be able to play any videos that have not been migrated. If you update your account after June 30, you can resume indexing immediately but you won't be able to play videos indexed before the account update until they are migrated through the AMS VI migration.

This article gives a brief overview of Azure AI Video Indexer terminology and concepts. Also, review transparency note overview

Artifact files

If you plan to download artifact files, beware of the following warning:

Warning

We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

Confidence scores

The confidence score indicates the confidence in an insight. It's a number between 0.0 and 1.0. The higher the score the greater the confidence in the answer. For example:

"transcript":[
{
  "id":1,
  "text":"Well, good morning everyone and welcome to",
  "confidence":0.8839,
  "speakerId":1,
  "language":"en-US",
  "instances":[
     {
	"adjustedStart":"0:00:10.21",
	"adjustedEnd":"0:00:12.81",
	"start":"0:00:10.21",
	"end":"0:00:12.81"
     }
  ]
},

Content moderation

Use textual and visual content moderation models to keep your users safe from inappropriate content and validate that the content you publish matches your organization's values. You can automatically block certain videos or alert your users about the content. For more information, see Insights: visual and textual content moderation.

Insights

Insights contain an aggregated view of the data: faces, topics, text-based emotion detection. Azure AI Video Indexer analyzes the video and audio content by running 30+ AI models, generating rich insights.

For detailed explanation of insights, see Azure AI Video Indexer insights.

Keyframes

Azure AI Video Indexer selects the frame(s) that best represent each shot. Keyframes are the representative frames selected from the entire video based on aesthetic properties (for example, contrast and stableness). For more information, see Scenes, shots, and keyframes.

Time range vs. adjusted time range

Time range is the time period in the original video. Adjusted time range is the time range relative to the current playlist. Since you can create a playlist from different lines of different videos, you can take a one-hour video and use just one line from it, for example, 10:00-10:15. In that case, you'll have a playlist with one line, where the time range is 10:00-10:15 but the adjusted time range is 00:00-00:15.

Widgets

Azure AI Video Indexer supports embedding widgets in your apps. For more information, see Embed Azure AI Video Indexer widgets in your apps.