Integrate media capabilities

You can integrate native device capabilities, such as camera and microphone with your Teams app. For integration, you can use Microsoft Teams JavaScript client library that provides the necessary tools for your app to access a user’s device permissions. Use suitable media capability APIs to integrate the device capabilities, such as camera and microphone with the Teams platform within your Microsoft Teams app, and build a richer experience. The media capability is available for Teams web client, desktop, and mobile. To integrate media capabilities, you must update the app manifest file and call the media capability APIs.

For effective integration, you must have a good understanding of code snippets for calling the respective APIs, which allows you to use native media capabilities. It's important to familiarize yourself with the API response errors to handle the errors in your Teams app.

Advantages

The advantage of integrating device capabilities into your Teams apps is that it uses native Teams controls to provide a rich and immersive experience for your users. The following scenarios showcase the advantages of media capabilities:

• Allow the user to capture the rough mockups drawn on a physical whiteboard through their mobile phone and use the captured images as poll options in Teams group chat.

• Allow the user to record audio message and attach it to an incident ticket.

• Allow the user to scan the physical documents from the smartphone to file a car insurance claim.

• Allow the user to record a video at a worksite and upload it for attendance.

Update manifest

Update your Teams app manifest.json file by adding the devicePermissions property and specifying media. It allows your app to ask for requisite permissions from users before they start using the camera to capture the image, open the gallery to select an image to submit as an attachment, or use the microphone to record the conversation. The update for app manifest is as follows:

"devicePermissions": [
    "media",
],

Media capability APIs

The captureImage, selectMedia, getMedia, and viewImages APIs enable you to use native media capabilities as follows:

• Use the native microphone to allow users to record audio (record 10 minutes of conversation) from the device.
• Use native camera control to allow users to capture and attach images and capture videos (record up to five minutes of video) on the go.
• Use native gallery support to allow users to select device images as attachments.
• Use native image viewer control to preview multiple images at one time.
• Support large image transfer (from 1 MB to 50 MB) through the TeamsJS bridge.
• Support advanced image capabilities by allowing users to preview and edit images.
• Scan documents, whiteboard, and business cards through the camera.

The following table lists set of APIs to enable your device's media capabilities:

The following image depicts the web app experience of captureImage API for the image capability on Teams mobile:

• Launch camera and capture the image to share:

  

• Select media from the gallery to share:

  

You can view the web app experience of the selectMedia API for Teams desktop and mobile.

Error handling

Ensure to handle these errors appropriately in your Teams app. The following table lists the error codes and the descriptions under which the errors are generated:

Code snippets

• Call captureImage API for launching and capturing image using camera:

  microsoftTeams.media.captureImage((error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
  // If there's any error, an alert shows the error message/code
  if (error) {
    if (error.message) {
      alert(" ErrorCode: " + error.errorCode + error.message);
    } else {
      alert(" ErrorCode: " + error.errorCode);
    }
  } else if (attachments) {
    let y = attachments[0];
    img.src = ("data:" + y.mimeType + ";base64," + y.preview);
  }
  });
  

• Call selectMedia API for capturing and selecting media from device camera or gallery:

  let imageProp: microsoftTeams.media.ImageProps = {
      sources: [microsoftTeams.media.Source.Camera, microsoftTeams.media.Source.Gallery],
      startMode: microsoftTeams.media.CameraStartMode.Photo,
      ink: false,
      cameraSwitcher: false,
      textSticker: false,
      enableFilter: true,
  };
  let mediaInput: microsoftTeams.media.MediaInputs = {
      mediaType: microsoftTeams.media.MediaType.Image,
      maxMediaCount: 10,
      imageProps: imageProp
  };
  microsoftTeams.media.selectMedia(mediaInput, (error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
      if (error) {
          if (error.message) {
              alert(" ErrorCode: " + error.errorCode + error.message);
          } else {
              alert(" ErrorCode: " + error.errorCode);
          }
      }
      if (attachments) {
          let y = attachments[0];
          img.src = ("data:" + y.mimeType + ";base64," + y.preview);
      }
  });
  

• Call selectMedia API for capturing videos using camera:

  • Capturing videos with fullscreen: true:

    fullscreen: true opens the camera in video recording mode. It provides an option to use both front and rear camera and also provides other attributes as mentioned in the following example:

    
      const defaultLensVideoProps: microsoftTeams.media.VideoProps = {
          sources: [microsoftTeams.media.Source.Camera, microsoftTeams.media.Source.Gallery],
          startMode: microsoftTeams.media.CameraStartMode.Video,
          cameraSwitcher: true,
          maxDuration: 30
     }
      const defaultLensVideoMediaInput: microsoftTeams.media.MediaInputs = {
          mediaType: microsoftTeams.media.MediaType.Video,
          maxMediaCount: 6,
          videoProps: defaultLensVideoProps
     }
    

  • Capturing videos with fullscreen: false:

    fullscreen: false opens the camera in video recording mode and uses the front camera only. Typically fullscreen: false is used when user wants to record video while reading content on the device screen.

    This mode also supports isStopButtonVisible: true that adds a stop button on the screen that allows user to stop the recording. If isStopButtonVisible: false, recording can be stopped either by calling mediaController API or when the recording duration has reached maxDuration time specified.

    Following is an example to stop the recording with maxDuration time specified:

       const defaultNativeVideoProps: microsoftTeams.media.VideoProps = {
          maxDuration: 30,
          isFullScreenMode: false,
          isStopButtonVisible: false,
          videoController: new microsoftTeams.media.VideoController(videoControllerCallback)
      }
       const defaultNativeVideoMediaInput: microsoftTeams.media.MediaInputs = {
          mediaType: microsoftTeams.media.MediaType.Video,
          maxMediaCount: 1,
          videoProps: defaultNativeVideoProps
      }
    

    Following is an example to stop the recording by calling mediaController API:

       const defaultNativeVideoProps: microsoftTeams.media.VideoProps = {
          videoController.stop(),
          isFullScreenMode: false,
          isStopButtonVisible: false,
          videoController: new microsoftTeams.media.VideoController(videoControllerCallback)
      }
       const defaultNativeVideoMediaInput: microsoftTeams.media.MediaInputs = {
          mediaType: microsoftTeams.media.MediaType.Video,
          maxMediaCount: 1,
          videoProps: defaultNativeVideoProps
      }
    

• Call selectMedia API for capturing images and video using camera:

  This API allows users to select between capturing an image or a video.

  
    const defaultVideoAndImageProps: microsoftTeams.media.VideoAndImageProps = {
      sources: [microsoftTeams.media.Source.Camera, microsoftTeams.media.Source.Gallery],
      startMode: microsoftTeams.media.CameraStartMode.Photo,
      ink: true,
      cameraSwitcher: true,
      textSticker: true,
      enableFilter: true,
      maxDuration: 30
    }
  
    const defaultVideoAndImageMediaInput: microsoftTeams.media.MediaInputs = {
      mediaType: microsoftTeams.media.MediaType.VideoAndImage,
      maxMediaCount: 6,
      videoAndImageProps: defaultVideoAndImageProps
    }
  
    let videoControllerCallback: microsoftTeams.media.VideoControllerCallback = {
      onRecordingStarted() {
        console.log('onRecordingStarted Callback Invoked');
      },
    };
  
    microsoftTeams.media.selectMedia(defaultVideoAndImageMediaInput, (error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
      if (error) {
          if (error.message) {
              alert(" ErrorCode: " + error.errorCode + error.message);
          } else {
              alert(" ErrorCode: " + error.errorCode);
          }
      }
  
      var videoElement = document.createElement("video");
      attachments[0].getMedia((error: microsoftTeams.SdkError, blob: Blob) => {
      if (blob) {
          if (blob.type.includes("video")) {
              videoElement.setAttribute("src", URL.createObjectURL(blob));
          }
      }
      if (error) {
          if (error.message) {
              alert(" ErrorCode: " + error.errorCode + error.message);
          } else {
              alert(" ErrorCode: " + error.errorCode);
          }
      }
      });
      });
  
  

• Call getMedia API to retrieve large media in chunks:

  let media: microsoftTeams.media.Media = attachments[0]
  media.getMedia((error: microsoftTeams.SdkError, blob: Blob) => {
      if (blob) {
          if (blob.type.includes("image")) {
              img.src = (URL.createObjectURL(blob));
          }
      }
      if (error) {
          if (error.message) {
              alert(" ErrorCode: " + error.errorCode + error.message);
          } else {
              alert(" ErrorCode: " + error.errorCode);
          }
      }
  });
  

• Call viewImages API by ID, which is returned by selectMedia API:

  // View images by id:
  // Assumption: attachmentArray = select Media API Output
  let uriList = [];
  if (attachmentArray && attachmentArray.length > 0) {
      for (let i = 0; i < attachmentArray.length; i++) {
          let file = attachmentArray[i];
          if (file.mimeType.includes("image")) {
              let imageUri = {
                  value: file.content,
                  type: 1,
              }
              uriList.push(imageUri);
          } else {
              alert("File type is not image");
          }
      }
  }
  if (uriList.length > 0) {
      microsoftTeams.media.viewImages(uriList, (error: microsoftTeams.SdkError) => {
          if (error) {
              if (error.message) {
                  output(" ErrorCode: " + error.errorCode + error.message);
              } else {
                  output(" ErrorCode: " + error.errorCode);
              }
          }
      });
  } else {
      output("Url list is empty");
  }
  

• Call viewImages API by URL:

  // View Images by URL:
  // Assumption 2 urls, url1 and url2
  let uriList = [];
  if (URL1 != null && URL1.length > 0) {
      let imageUri = {
          value: URL1,
          type: 2,
      }
      uriList.push(imageUri);
  }
  if (URL2 != null && URL2.length > 0) {
      let imageUri = {
          value: URL2,
          type: 2,
      }
      uriList.push(imageUri);
  }
  if (uriList.length > 0) {
      microsoftTeams.media.viewImages(uriList, (error: microsoftTeams.SdkError) => {
          if (error) {
              if (error.message) {
                  output(" ErrorCode: " + error.errorCode + error.message);
              } else {
                  output(" ErrorCode: " + error.errorCode);
              }
          }
      });
  } else {
      output("Url list is empty");
  }
  

• Call selectMedia and getMedia APIs for recording audio through microphone:

    let mediaInput: microsoftTeams.media.MediaInputs = {
      mediaType: microsoftTeams.media.MediaType.Audio,
      maxMediaCount: 1,
      };
      microsoftTeams.media.selectMedia(mediaInput, (error: microsoftTeams.SdkError, attachments: microsoftTeams.media.Media[]) => {
          if (error) {
              if (error.message) {
                  alert(" ErrorCode: " + error.errorCode + error.message);
              } else {
                  alert(" ErrorCode: " + error.errorCode);
              }
          }
      // If you want to directly use the audio file (for smaller file sizes (~4MB))    if (attachments) {
      let audioResult = attachments[0];
      var videoElement = document.createElement("video");
      videoElement.setAttribute("src", ("data:" + audioResult.mimeType + ";base64," + audioResult.preview));
      audioResult.getMedia((error: microsoftTeams.SdkError, blob: Blob) => {
      if (blob) {
          if (blob.type.includes("video")) {
              videoElement.setAttribute("src", URL.createObjectURL(blob));
          }
      }
      if (error) {
          if (error.message) {
              alert(" ErrorCode: " + error.errorCode + error.message);
          } else {
              alert(" ErrorCode: " + error.errorCode);
          }
      }
  });
  });
  

File download on Teams mobile

You can configure an app to enable users to download files from the webview to their mobile device.

To enable, follow the steps:

1. Update your Teams app manifest.json file by adding the devicePermissions property and specifying media as shown in the update manifest.

2. Use the following format and add the HTML download attribute to the webpage:

   <a href="path_to_file" download="download">Download</a>
   

See also

• Device capabilities
• Integrate QR or barcode scanner capability in Teams
• Integrate location capabilities in Teams
• Integrate People Picker
• Requirements and considerations for application-hosted media bots
• App manifest schema for Teams
• Plan responsive tabs for Teams mobile