Live Share media capabilities

Video and audio are instrumental parts of the modern world and workplace. We've heard wide ranging feedback that there's more we can do to increase the quality, accessibility, and license protections of watching videos together in meetings.

The Live Share SDK enables robust media synchronization for any HTML <video> and <audio> element with just a few lines of code. By synchronizing media at the player state and transport controls layer, you can individually attribute views and license, while providing the highest possible quality available through your app.

Install

Live Share media is a JavaScript package published on npm, and you can download through npm or yarn. You must also install its peer dependencies, which include @microsoft/live-share, fluid-framework and @fluidframework/azure-client. If you're using Live Share in your tab application, you must also install @microsoft/teams-js version 2.11.0 or later.

npm install @microsoft/live-share @microsoft/live-share-media fluid-framework @fluidframework/azure-client --save
npm install @microsoft/teams-js --save

OR

To add the latest version of the SDK to your application using Yarn:

yarn add @microsoft/live-share @microsoft/live-share-media fluid-framework @fluidframework/azure-client
yarn add @microsoft/teams-js

Media sync overview

The Live Share SDK has two primary classes related to media synchronization:

Classes Description
LiveMediaSession Custom live object designed to coordinate media transport controls and playback state in independent media streams.
MediaPlayerSynchronizer Synchronizes any object that implements the IMediaPlayer interface -- including HTML5 <video> and <audio> -- using LiveMediaSession.

Example:

<body>
  <video id="player">
    <source src="YOUR_VIDEO_SRC" type="video/mp4" />
  </video>
</body>
import {
  LiveShareClient,
  UserMeetingRole,
  MediaPlayerSynchronizerEvents,
} from "@microsoft/live-share";
import { LiveMediaSession } from "@microsoft/live-share-media";
import { LiveShareHost } from "@microsoft/teams-js";

// Setup the Fluid container
const host = LiveShareHost.create();
const liveShare = new LiveShareClient(host);
const schema = {
  initialObjects: { mediaSession: LiveMediaSession },
};
const { container } = await liveShare.joinContainer(schema);
const { mediaSession } = container.initialObjects;

// Get the player from your document and create synchronizer
const player = document.getElementById("player");
const synchronizer = mediaSession.synchronize(player);

// Listen for groupaction events (optional)
synchronizer.addEventListener(MediaPlayerSynchronizerEvents.groupaction, async (evt) => {
  // See which user made the change (e.g., to display a notification)
  const clientInfo = await synchronizer.mediaSession.getClientInfo(evt.details.clientId);
});

// Define roles you want to allow playback control and start sync
const allowedRoles = [UserMeetingRole.organizer, UserMeetingRole.presenter];
await mediaSession.initialize(allowedRoles);

The LiveMediaSession automatically listens for changes to the group's playback state. MediaPlayerSynchronizer listens to state changes emitted by LiveMediaSession and applies them to the provided IMediaPlayer object, such as an HTML5 <video> or <audio> element. To avoid playback state changes that a user didn't intentionally initiate, such as a buffer event, we must call transport controls through the synchronizer, rather than directly through the player.

Example:

<body>
  <video id="player">
    <source src="YOUR_VIDEO_SRC" type="video/mp4" />
  </video>
  <div class="player-controls">
    <button id="play-button">Play</button>
    <button id="pause-button">Pause</button>
    <button id="restart-button">Restart</button>
    <button id="change-track-button">Change track</button>
  </div>
</body>
// ...

document.getElementById("play-button").onclick = async () => {
  // Plays for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.play();
};

document.getElementById("pause-button").onclick = async () => {
  // Pauses for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.pause();
};

document.getElementById("restart-button").onclick = async () => {
  // Seeks for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.seekTo(0);
};

document.getElementById("change-track-button").onclick = () => {
  // Changes the track for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  synchronizer.setTrack({
    trackIdentifier: "SOME_OTHER_VIDEO_SRC",
  });
};

Note

While you can use the LiveMediaSession object to synchronize media manually, it's generally recommend to use the MediaPlayerSynchronizer. Depending on the player you use in your app, you might need to create a delegate shim to make your web player's interface match the IMediaPlayer interface.

Suspensions and wait points

Screenshot that shows a suspension sync to the presenter.

If you want to temporarily suspend synchronization for the LiveMediaSession object, you can use suspensions. A MediaSessionCoordinatorSuspension object is local by default, which can be helpful in cases where a user might want to catch up on something they missed, take a break, and so on. If the user ends the suspension, synchronization resumes automatically.

// Suspend the media session coordinator
const suspension = mediaSession.coordinator.beginSuspension();

// End the suspension when ready
suspension.end();

When beginning a suspension, you can also include an optional CoordinationWaitPoint parameter, which allows users to define the timestamps in which a suspension should occur for all users. Synchronization doesn't resume until all users end the suspension for that wait point.

Here are a few scenarios where wait points are especially useful:

  • Adding a quiz or survey at certain points in the video.
  • Waiting for everyone to suitably load a video before it starts or while buffering.
  • Allow a presenter to choose points in the video for group discussion.
// Suspend the media session coordinator
const waitPoint = {
  position: 0,
  reason: "ReadyUp", // Optional.
};
const suspension = mediaSession.coordinator.beginSuspension(waitPoint);
// End the suspension when the user readies up
document.getElementById("ready-up-button").onclick = () => {
  // Sync resumes when everyone ends suspension
  suspension.end();
};

Audio ducking

The Live Share SDK supports intelligent audio ducking. You can use the feature in your application by adding the following to your code:

import { meeting } from "@microsoft/teams-js";

// ... set up MediaPlayerSynchronizer

// Register speaking state change handler through Microsoft Teams JavaScript client library (TeamsJS)
let volumeTimer;
meeting.registerSpeakingStateChangeHandler((speakingState) => {
  if (speakingState.isSpeakingDetected && !volumeTimer) {
    // If someone in the meeting starts speaking, periodically
    // lower the volume using your MediaPlayerSynchronizer's
    // VolumeLimiter.
    synchronizer.volumeLimiter?.lowerVolume();
    volumeTimer = setInterval(() => {
      synchronizer.volumeLimiter?.lowerVolume();
    }, 250);
  } else if (volumeTimer) {
    // If everyone in the meeting stops speaking and the
    // interval timer is active, clear the interval.
    clearInterval(volumeTimer);
    volumeTimer = undefined;
  }
});

Additionally, add the following RSC permissions into your app manifest:

{
  // ...rest of your manifest here
  "authorization": {​
    "permissions": {​
      "resourceSpecific": [
        // ...other permissions here​
        {​
          "name": "OnlineMeetingIncomingAudio.Detect.Chat",​
          "type": "Delegated"
        },
        {​
          "name": "OnlineMeetingIncomingAudio.Detect.Group",​
          "type": "Delegated"​
        }​
      ]​
    }​
  }​
}

Note

The registerSpeakingStateChangeHandler API used for audio ducking only works on Microsoft Teams desktop and in scheduled and meet now meeting types.

Code samples

Sample name Description JavaScript TypeScript
React video Basic example showing how the LiveMediaSession object works with HTML5 video. View NA
React media template Enable all connected clients to watch videos together, build a shared playlist, transfer whom is in control, and annotate over the video. View View

Next step

See also