What is the best way to ingest a video stream for emotion detection?

princeofpluto 1 Reputation point
2021-05-21T22:19:16.957+00:00

I need to ingest video streaming data from a mobile application camera. sore the video and perform emotion detection.

AWS has Kinesis Video, which would be the perfect fit. Azure has Event Gate, but doesn't seem to support video.

My client application will have different video events. So I won't have a continuous stream (i.e. user starts and stops recording and a certain time).

I thought about the possibility of creating a Live Event. But how can I have each client event store the video in a separate Asset? And how can I use emotion detection on the video later? I tried creating a Transform/Job but I only got results for face detection (not emotion)

Azure Media Services
Azure Media Services
A group of Azure services that includes encoding, format conversion, on-demand streaming, content protection, and live streaming services.
316 questions
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
2,833 questions
0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ajkuma 26,131 Reputation points Microsoft Employee
    2021-05-24T15:54:19.627+00:00

    @princeofpluto , Apologies for the delay in responding here!

    You may leverage Azure Media Service (AMS) as per your requirement.
    Video Indexer (VI) (https://www.videoindexer.ai) supports emotion detection. Video Indexer is partly built on top of AMS.

    Which offers -Emotion detection: Identifies emotions based on speech (what's being said) and voice tonality (how it's being said). The emotion could be joy, sadness, anger, or fear.

    For more information, visit VI’s portal or the VI developer portal, and test this capability.

    You can also browse videos indexed as to emotional content: sample 1, sample 2, and sample 3.

    Kindly see this doc for more details on this topic:
    Cross-channel emotion analysis in Microsoft Video Indexer

    0 comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.