Capture game audio, video, screenshots, and metadata
This article describes how to capture game video, audio, and screenshots, and how to submit metadata that the system will embed in captured and broadcast media, allowing your app and others to create dynamic experiences that are synchronized to gameplay events.
There are two different ways that gameplay can be captured in a UWP app. The user can initiate capture using the built-in system UI. Media that is captured using this technique is ingested into the Microsoft gaming ecosystem, can be viewed and shared through first-party experiences such as the Xbox app, and is not directly available to your app or to users. The first sections of this article will show you how to enable and disable system-implemented app capture and how to receive notifications when app capture starts or stops.
The other way to capture media is to use the APIs of the Windows.Media.AppRecording namespace. If capturing is enabled on the device, your app can start capturing gameplay and then, after some time has passed, you can stop the capture, at which point the media is written to a file. If the user has enabled historical capture, then you can also record gameplay that has already occurred by specifying a start time in the past and a duration to record. Both of these techniques produce a video file that can be accessed by your app, and depending on where you choose to save the files, by the user. The middle sections of this article walk you through the implementation of these scenarios.
The Windows.Media.Capture namespace provides APIs for creating metadata that describes the gameplay being captured or broadcast. This can include text or numeric values, with a text label identifying each data item. Metadata can represent an "event" which occurs at a single moment, such as when the user finishes a lap in a racing game, or it can represent a "state" that persists over a span of time, such as the current game map the user is playing in. The metadata is written to a cache that is allocated and managed for your app by the system. The metadata is embedded into broadcast streams and captured video files, including both the built-in system capture or custom app capture techniques. The final sections of this article show you how to write gameplay metadata.
Note
Because the gameplay metadata can be embedded in media files that can potentially be shared over the network, out of the user's control, you should not include personally identifiable information or other potentially sensitive data in the metadata.
Enable and disable system app capture
System app capture is initiated by the user with the built-in system UI. The files are ingested by the Windows gaming ecosystem and is not available to your app or the user, except for through first party experiences like the Xbox app. Your app can disable and enable system-initiated app capture, allowing you to prevent the user from capturing certain content or gameplay.
To enable or disable system app capture, simply call the static method AppCapture.SetAllowedAsync and passing false to disable capture or true to enable capture.
Windows::Media::Capture::AppCapture::SetAllowedAsync(allowed);
Receive notifications when system app capture starts and stops
To receive a notification when system app capture begins or ends, first get an instance of the AppCapture class by calling the factory method GetForCurrentView. Next, register a handler for the CapturingChanged event.
Windows::Media::Capture::AppCapture^ appCapture = Windows::Media::Capture::AppCapture::GetForCurrentView();
appCapture->CapturingChanged +=
ref new TypedEventHandler<Windows::Media::Capture::AppCapture^, Platform::Object^>(this, &App::OnCapturingChanged);
In the handler for the CapturingChanged event, you can check the IsCapturingAudio and the IsCapturingVideo properties to determine if audio or video are being captured respectively. You may want to update your app's UI to indicate the current capturing status.
void App::OnCapturingChanged(Windows::Media::Capture::AppCapture^ sender, Platform::Object^ args)
{
Platform::String^ captureStatusText = "";
if (sender->IsCapturingAudio)
{
captureStatusText += "Capturing audio.";
}
if (sender->IsCapturingVideo)
{
captureStatusText += "Capturing video.";
}
UpdateStatusText(captureStatusText);
}
Add the Windows Desktop Extensions for the UWP to your app
The APIs for recording audio and video and for capturing screenshots directly from your app, found in the Windows.Media.AppRecording namespace, are not included in the Universal API contract. To access the APIs, you must add a reference to the Windows Desktop Extensions for the UWP to your app with the following steps.
- In Visual Studio, in Solution Explorer, expand your UWP project and right-click References and then select Add Reference....
- Expand the Universal Windows node and select Extensions.
- In the list of extensions check the checkbox next to the Windows Desktop Extensions for the UWP entry that matches the target build for your project. For the app broadcast features, the version must be 1709 or greater.
- Click OK.
Get an instance of AppRecordingManager
The AppRecordingManager class is the central API you will use to manage app recording. Get an instance of this class by calling the factory method GetDefault. Before using any of the APIs in the Windows.Media.AppRecording namespace, you should check for their presence on the current device. The APIs are not available on devices running an OS version earlier than Windows 10, version 1709. Rather than check for a specific OS version, use the ApiInformation.IsApiContractPresent method to query for the Windows.Media.AppBroadcasting.AppRecordingContract version 1.0. If this contract is present, then the recording APIs are available on the device. The example code in this article checks for the APIs once and then checks if the AppRecordingManager is null before subsequent operations.
if (Windows::Foundation::Metadata::ApiInformation::IsApiContractPresent(
"Windows.Media.AppRecording.AppRecordingContract", 1, 0))
{
m_appRecordingManager = AppRecordingManager::GetDefault();
}
Determine if your app can currently record
There are several reasons that your app may not currently be able to capture audio or video, including if the current device doesn't meet the hardware requirements for recording or if another app is currently broadcasting. Before initiating a recording, you can check to see if your app is currently able to record. Call the GetStatus method of the AppRecordingManager object and then check the CanRecord property of the returned AppRecordingStatus object. If CanRecord returns false, meaning that your app can't currently record, you can check the Details property to determine the reason. Depending on the reason, you may want to display the status to the user or show instructions for enabling app recording.
bool App::CanRecord()
{
if (m_appRecordingManager == nullptr)
{
return false;
}
AppRecordingStatus^ recordingStatus = m_appRecordingManager->GetStatus();
if (!recordingStatus->CanRecord)
{
AppRecordingStatusDetails^ details = recordingStatus->Details;
if (details->IsAnyAppBroadcasting)
{
UpdateStatusText("Another app is currently broadcasting.");
return false;
}
if (details->IsCaptureResourceUnavailable)
{
UpdateStatusText("The capture resource is currently unavailable.");
return false;
}
if (details->IsGameStreamInProgress)
{
UpdateStatusText("A game stream is currently in progress.");
return false;
}
if (details->IsGpuConstrained)
{
// Typically, this means that the GPU software does not include an H264 encoder
UpdateStatusText("The GPU does not support app recording.");
return false;
}
if (details->IsAppInactive)
{
// Broadcasting can only be started when the application's window is the active window.
UpdateStatusText("The app window to be recorded is not active.");
return false;
}
if (details->IsBlockedForApp)
{
UpdateStatusText("Recording is blocked for this app.");
return false;
}
if (details->IsDisabledByUser)
{
UpdateStatusText("The user has disabled GameBar in Windows Settings.");
return false;
}
if (details->IsDisabledBySystem)
{
UpdateStatusText("Recording is disabled by the system.");
return false;
}
return false;
}
return true;
}
Manually start and stop recording your app to a file
After verifying that your app is able to record, you can start a new recording by calling the StartRecordingToFileAsync method of the AppRecordingManager object.
In the following example, the first then block executes when the asynchronous task fails. The second then block attempts to access the result of the task and, if the result is null, then the task has completed. In both cases, the OnRecordingComplete helper method, shown below, is called to handle the result.
void App::StartRecordToFile(Windows::Storage::StorageFile^ file)
{
if (m_appRecordingManager == nullptr)
{
return;
}
if (!CanRecord())
{
return;
}
// Start a recording operation to record starting from
// now until the operation fails or is cancelled.
m_recordOperation = m_appRecordingManager->StartRecordingToFileAsync(file);
create_task(m_recordOperation).then(
[this](AppRecordingResult^ result)
{
OnRecordingComplete();
}).then([this](task<void> t)
{
try
{
t.get();
}
catch (const task_canceled&)
{
OnRecordingComplete();
}
});
}
When the recording operation completes, check the Succeeded property of the returned AppRecordingResult object to determine if the record operation was successful. If so, you can check the IsFileTruncated property to determine if, for storage reasons, the system was forced to truncate the captured file. You can check the Duration property to discover the actual duration of the recorded file which, if the file is truncated, may be shorter than the duration of the recording operation.
void App::OnRecordingComplete()
{
if (m_recordOperation)
{
auto result = m_recordOperation->GetResults();
if (result->Succeeded)
{
Windows::Foundation::TimeSpan duration = result->Duration;
boolean isTruncated = result->IsFileTruncated;
UpdateStatusText("Recording completed.");
}
else
{
// If the recording failed, ExtendedError
// can be retrieved and used for diagnostic purposes
HResult extendedError = result->ExtendedError;
LogTelemetryMessage("Error during recording: " + extendedError);
}
m_recordOperation = nullptr;
}
}
The following examples show some basic code for starting and stopping the recording operation shown in the previous example.
StorageFolder^ storageFolder = ApplicationData::Current->LocalFolder;
concurrency::create_task(storageFolder->CreateFileAsync("recordtofile_example.mp4", CreationCollisionOption::ReplaceExisting)).then(
[this](StorageFile^ file)
{
StartRecordToFile(file);
});
void App::FinishRecordToFile()
{
m_recordOperation->Cancel();
}
Record a historical time span to a file
If the user has enabled historical recording for your app in the system settings, you can record a time span of gameplay that has previously transpired. A previous example in this article showed how to confirm that your app can currently record gameplay. There is an additional check to determine if historical capture is enabled. Once again, call GetStatus and check the CanRecordTimeSpan property of the returned AppRecordingStatus object. This example also returns the HistoricalBufferDuration property of the AppRecordingStatus which will be used to determine a valid start time for the recording operation.
bool App::CanRecordTimeSpan(TimeSpan &historicalDurationBuffer)
{
if (m_appRecordingManager == nullptr)
{
return false;
}
AppRecordingStatus^ recordingStatus = m_appRecordingManager->GetStatus();
if (recordingStatus->Details->IsTimeSpanRecordingDisabled)
{
UpdateStatusText("Historical time span recording is disabled by the system.");
return false;
}
historicalDurationBuffer = recordingStatus->HistoricalBufferDuration;
return true;
}
To capture a historical timespan, you must specify a start time for the recording and a duration. The start time is provided as a DateTime struct. The start time must be a time before the current time, within the length of the historical recording buffer. For this example, the buffer length is retrieved as part of the check to see if historical recording is enabled, which is shown in the previous code example. The duration of the historical recording is provided as TimeSpan struct, which should also be equal to or smaller than the duration of the historical buffer. Once you have determined your desired start time and duration, call RecordTimeSpanToFileAsync to start the recording operation.
Like recording with manual start and stop, when a historical recording completes, you can check the Succeeded property of the returned AppRecordingResult object to determine if the record operation was successful, and you can check the IsFileTruncated and Duration property to discover the actual duration of the recorded file which, if the file is truncated, may be shorter than the duration of the requested time window.
void App::RecordTimeSpanToFile(Windows::Storage::StorageFile^ file)
{
if (m_appRecordingManager == nullptr)
{
return;
}
if (!CanRecord())
{
return;
}
Windows::Foundation::TimeSpan historicalBufferDuration;
if (!CanRecordTimeSpan(historicalBufferDuration))
{
return;
}
AppRecordingStatus^ recordingStatus = m_appRecordingManager->GetStatus();
Windows::Globalization::Calendar^ calendar = ref new Windows::Globalization::Calendar();
calendar->SetToNow();
Windows::Foundation::DateTime nowTime = calendar->GetDateTime();
int secondsToRecord = min(30, historicalBufferDuration.Duration / 10000000);
calendar->AddSeconds(-1 * secondsToRecord);
Windows::Foundation::DateTime startTime = calendar->GetDateTime();
Windows::Foundation::TimeSpan duration;
duration.Duration = nowTime.UniversalTime - startTime.UniversalTime;
create_task(m_appRecordingManager->RecordTimeSpanToFileAsync(startTime, duration, file)).then(
[this](AppRecordingResult^ result)
{
if (result->Succeeded)
{
Windows::Foundation::TimeSpan duration = result->Duration;
boolean isTruncated = result->IsFileTruncated;
UpdateStatusText("Recording completed.");
}
else
{
// If the recording failed, ExtendedError
// can be retrieved and used for diagnostic purposes
HResult extendedError = result->ExtendedError;
LogTelemetryMessage("Error during recording: " + extendedError);
}
});
}
The following example shows some basic code for initiating the historical record operation shown in the previous example.
StorageFolder^ storageFolder = ApplicationData::Current->LocalFolder;
concurrency::create_task(storageFolder->CreateFileAsync("recordtimespantofile_example.mp4", CreationCollisionOption::ReplaceExisting)).then(
[this](StorageFile^ file)
{
RecordTimeSpanToFile(file);
});
Save screenshot images to files
Your app can initiate a screenshot capture that will save the current contents of the app's window to one image file or to multiple image files with different image encodings. To specify the image encodings you would like to use, create a list of strings where each represents an image type. The properties of the ImageEncodingSubtypes provide the correct string for each supported image type, such as MediaEncodingSubtypes.Png or MediaEncodingSubtypes.JpegXr.
Initiate screen capture by calling the SaveScreenshotToFilesAsync method of the AppRecordingManager object. The first parameter to this method is a StorageFolder where the image files will be saved. The second parameter is a filename prefix to which the system will append the extension for each image type saved, such as ".png".
The third parameter to SaveScreenshotToFilesAsync is necessary for the system to be able to do the proper colorspace conversion if the current window to be captured is displaying HDR content. If HDR content is present, this parameter should be set to AppRecordingSaveScreenshotOption.HdrContentVisible. Otherwise, use AppRecordingSaveScreenshotOption.None. The final parameter to the method is the list of image formats to which the screen should be captured.
When the asynchronous call to SaveScreenshotToFilesAsync completes, it returns a AppRecordingSavedScreenshotInfo object that provides the StorageFile and associated MediaEncodingSubtypes value indicating the image type for each saved image.
void App::SaveScreenShotToFiles(Windows::Storage::StorageFolder^ folder, Platform::String^ filenamePrefix)
{
if (m_appRecordingManager == nullptr)
{
return;
}
Windows::Foundation::Collections::IVectorView<Platform::String^>^ supportedFormats =
m_appRecordingManager->SupportedScreenshotMediaEncodingSubtypes;
Platform::Collections::Vector<Platform::String^>^ requestedFormats =
ref new Platform::Collections::Vector<Platform::String^>();
for (Platform::String^ format : requestedFormats)
{
if (format == Windows::Media::MediaProperties::MediaEncodingSubtypes::Png)
{
requestedFormats->Append(format);
}
else if (format == Windows::Media::MediaProperties::MediaEncodingSubtypes::JpegXr)
{
requestedFormats->Append(format);
}
}
create_task(m_appRecordingManager->SaveScreenshotToFilesAsync(folder, filenamePrefix, AppRecordingSaveScreenshotOption::None,
requestedFormats->GetView())).then(
[this](AppRecordingSaveScreenshotResult^ result)
{
if (result->Succeeded)
{
Windows::Foundation::Collections::IVectorView<AppRecordingSavedScreenshotInfo^>^ returnedScreenshots = result->SavedScreenshotInfos;
for (AppRecordingSavedScreenshotInfo^ screenshotInfo : returnedScreenshots)
{
Windows::Storage::StorageFile^ file = screenshotInfo->File;
Platform::String^ type = screenshotInfo->MediaEncodingSubtype;
}
}
else
{
// If the recording failed, ExtendedError
// can be retrieved and used for diagnostic purposes
HResult extendedError = result->ExtendedError;
LogTelemetryMessage("Error during screenshot: " + extendedError);
}
});
}
The following example shows some basic code for initiating the screenshot operation shown in the previous example.
StorageFolder^ storageFolder = ApplicationData::Current->LocalFolder;
SaveScreenShotToFiles(storageFolder, "screen_capture");
Add game metadata for system and app-initiated capture
The following sections of this article describe how to provide metadata that the system will embed into the MP4 stream of captured or broadcast gameplay. Metadata can be embedded in media that is captured using the built-in system UI and media that is captured by the app with AppRecordingManager. This metadata can be extracted by your app and other apps during media playback in order to provide contextually-aware experiences that are synchronized with the captured or broadcast gameplay.
Get an instance of AppCaptureMetadataWriter
The primary class for managing app capture metadata is AppCaptureMetadataWriter. Before initializing an instance of this class, use the ApiInformation.IsApiContractPresent method to query for the Windows.Media.Capture.AppCaptureMetadataContract version 1.0 to verify that the API is available on the current device.
if (Windows::Foundation::Metadata::ApiInformation::IsApiContractPresent("Windows.Media.Capture.AppCaptureMetadataContract", 1, 0))
{
m_appCaptureMetadataWriter = ref new AppCaptureMetadataWriter();
}
Write metadata to the system cache for your app
Each metadata item has a string label, identifying the metadata item, an associated data value which can be a string, an integer, or a double value, and a value from the AppCaptureMetadataPriority enumeration indicating the relative priority of the data item. A metadata item can either be considered an "event", which occurs at a single point in time, or a "state" which maintains a value over a time window. Metadata is written to a memory cache that is allocated and managed for your app by the system. The system enforces a size limit on the metadata memory cache and, when the limit is reached, will purge data based on the priority with which each metadata item was written. The next section of this article shows how to manage your app's metadata memory allocation.
A typical app may choose to write some metadata at the beginning of the capture session to provide some context for the subsequent data. For this scenario it is recommended that you use instantaneous "event" data. This example calls AddStringEvent, AddDoubleEvent, and AddInt32Event to set instantaneous values for each data type.
void App::StartSession(Platform::String^ sessionId, double averageFps, int resolutionWidth, int resolutionHeight)
{
if (m_appCaptureMetadataWriter != nullptr)
{
m_appCaptureMetadataWriter->AddStringEvent("sessionId", sessionId, AppCaptureMetadataPriority::Informational);
m_appCaptureMetadataWriter->AddDoubleEvent("averageFps", averageFps, AppCaptureMetadataPriority::Informational);
m_appCaptureMetadataWriter->AddInt32Event("resolutionWidth", resolutionWidth, AppCaptureMetadataPriority::Informational);
m_appCaptureMetadataWriter->AddInt32Event("resolutionHeight", resolutionHeight, AppCaptureMetadataPriority::Informational);
}
}
A common scenario for using "state" data that persists over time is to track the game map that the player is currently within. This example calls StartStringState to set the state value.
void App::StartMap(Platform::String^ mapName)
{
m_appCaptureMetadataWriter->StartStringState("map", mapName, AppCaptureMetadataPriority::Important);
}
Call StopState to record that a particular state has ended.
void App::EndMap(Platform::String^ mapName)
{
m_appCaptureMetadataWriter->StopState("map");
}
You can overwrite a state by setting a new value with an existing state label.
void App::LevelUp(int newLevel)
{
m_appCaptureMetadataWriter->StartInt32State("currentLevel", newLevel, AppCaptureMetadataPriority::Important);
}
You can end all currently open states by calling StopAllStates.
void App::RaceComplete()
{
m_appCaptureMetadataWriter->StopAllStates();
}
Manage metadata cache storage limit
The metadata that you write with AppCaptureMetadataWriter is cached by the system until it is written to the associated media stream. The system defines a size limit for each app's metadata cache. Once the cache size limit has been reached, the system will begin purging cached metadata. The system will delete metadata that was written with AppCaptureMetadataPriority.Informational priority value before deleting metadata with the AppCaptureMetadataPriority.Important priority.
At any point, you can check to see the number of bytes available in your app's metadata cache by calling RemainingStorageBytesAvailable. You can choose to set your own app-defined threshold after which you can choose to reduce the amount of metadata that you write to the cache. The following example shows a simple implementation of this pattern.
void App::CheckMetadataStorage()
{
INT64 storageRemaining = m_appCaptureMetadataWriter->RemainingStorageBytesAvailable;
if (storageRemaining < m_myLowStorageLevelInBytes)
{
m_writeLowPriorityMetadata = false;
}
}
void App::ComboExecuted(Platform::String^ comboName)
{
if (m_writeLowPriorityMetadata)
{
m_appCaptureMetadataWriter->AddStringEvent("combo", comboName, AppCaptureMetadataPriority::Informational);
}
}
Receive notifications when the system purges metadata
You can register to receive a notification when the system begins purging metadata for your app by registering a handler for the MetadataPurged event.
if (m_appCaptureMetadataWriter != nullptr)
{
m_appCaptureMetadataWriter->MetadataPurged +=
ref new TypedEventHandler<AppCaptureMetadataWriter^, Platform::Object^>(this, &App::OnMetadataPurged);
}
In the handler for the MetadataPurged event, you can clear up some room in the metadata cache by ending lower-priority states, you can implement app-defined logic for reducing the amount of metadata you write to the cache, or you can do nothing and let the system continue to purge the cache based on the priority with which it was written.
void App::OnMetadataPurged(Windows::Media::Capture::AppCaptureMetadataWriter^ sender, Platform::Object^ args)
{
// Reduce metadata by stopping a low-priority state.
//m_appCaptureMetadataWriter->StopState("map");
// Reduce metadata by stopping all states.
//m_appCaptureMetadataWriter->StopAllStates();
// Change app-specific behavior to write less metadata.
//m_writeLowPriorityMetadata = false;
// Take no action. Let the system purge data as needed. Record event for telemetry.
OutputDebugString(TEXT("Low-priority metadata purged."));
}
Related topics