How to get audio file with the method recognizeOnceAsync of SpeechSDK.SpeechRecognizer?

MarsOnly 5

I'm using the demo code from https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/quickstart/javascript/browser/from-microphone/index.html

like this below

recognizer.recognizing = (s, e) => { console.log(RECOGNIZING: Text=${e.result.text}); }; recognizer.recognized = (s, e) => { if (e.result.reason == sdk.ResultReason.RecognizedSpeech) { console.log(RECOGNIZED: Text=${e.result.text}); } else if (e.result.reason == sdk.ResultReason.NoMatch) { console.log("NOMATCH: Speech could not be recognized."); } }; recognizer.canceled = (s, e) => { console.log(CANCELED: Reason=${e.reason}); if (e.reason == sdk.CancellationReason.Error) { console.log("CANCELED: ErrorCode=${e.errorCode}); console.log("CANCELED: ErrorDetails=${e.errorDetails});
i want to get the audio file from the microphone with the sdk, but i can not find any solution in the document of microsoft-cognitiveservices-speech-sdk.

is there any possible to get the audio file with the sdk?

Azar 21,225 MVP

Hi @MarsOnly

In the JavaScript environment, you can explore the Web Audio API or the MediaRecorder API to capture and save the audio stream from the microphone. These APIs provide functionality to access raw audio data or record audio from the microphone as a file.

// Create a MediaStream object from the microphone navigator.mediaDevices.getUserMedia({ audio: true })   .then(function (stream) {     // Create a MediaRecorder to record the audio stream     const mediaRecorder = new MediaRecorder(stream);     const chunks = [];      // When data is available, save it to the chunks array     mediaRecorder.ondataavailable = function (e) {       chunks.push(e.data);     };      // When recording is stopped, create a new Blob from the collected chunks     mediaRecorder.onstop = function () {       const blob = new Blob(chunks, { type: 'audio/wav' });        // You can now handle the audio file, e.g., upload it or save it locally       // Example: upload the audio file to a server       const formData = new FormData();       formData.append('audio', blob, 'recording.wav');       fetch('/upload', {         method: 'POST',         body: formData,       })       .then(function (response) {         // Handle the server response         console.log('Audio file uploaded successfully');       })       .catch(function (error) {         console.error('Error uploading audio file', error);       });     };      // Start recording     mediaRecorder.start();      // Stop recording after a certain duration or based on user interaction     setTimeout(function () {       mediaRecorder.stop();     }, 5000); // Recording for 5 seconds in this example   })   .catch(function (error) {     console.error('Error accessing microphone', error);   });

By using the MediaRecorder API or similar techniques, you can capture the audio stream from the microphone and save it as a file in various formats, such as WAV or MP3.

If you find this answer useful kindly accept answer, for clarifications comment here

MarsOnly 5 Reputation points

2023-07-10T07:28:32.18+00:00

thanks for your answer, i can use this answer code if i only want to save the audio file from the microphone.

But what i want to do, is transforming the audio input from the microphone to text with the recognizeOnceAsync of SpeechSDK.SpeechRecognizer sdk, and at the same time get the audio file. the microphone input is recognized and controled by the sdk with the code i mentioned. if i use your code , it controled the microphone too, will it
Richard D'Lonesteen 0 Reputation points

2023-10-24T14:12:48.2+00:00

Curious about this too
HarshitShakya-3641 20 Reputation points

2024-02-13T09:16:06.8266667+00:00

Hello @MarsOnly
Have you got the solution for this audio-saved issue? I need to implement exact same functionality as you. If yes, Please share with me.

Thanks

My same query to the community.
https://video2.skills-academy.com/en-gb/answers/questions/1528402/how-to-save-speech

Cesar Bonadio 0

I solved this with the following code:

const getUserMedia = async (constraints) => {
    if (window.navigator.mediaDevices) {
        return window.navigator.mediaDevices.getUserMedia(constraints);
    }
    let legacyApi =
        navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;
    if (legacyApi) {
        return new Promise(function (resolve, reject) {
            legacyApi.bind(window.navigator)(constraints, resolve, reject);
        });
    } else {
        // user api not supported
    }
};

    const startSpeechRecognition = async () => {
  
        sdk.Recognizer.enableTelemetry(false);
        const speechConfig = sdk.SpeechConfig.fromAuthorizationToken(authToken.data.token, "westus")

        const stream = await getUserMedia({ audio: true });
        streamRef.current = stream;
        const audioConfig = sdk.AudioConfig.fromStreamInput(stream); 
        // const audioConfig = sdk.AudioConfig.fromDefaultMicrophoneInput();
    
        const referenceText = "What's the weather like?";
        const enableMiscue = true;
        const enableProsody = true;
        const configJson = {
            GradingSystem: "HundredMark",
            Granularity: "Phoneme",
            Dimension: "Comprehensive",
            ScenarioId: "",  // "" is the default scenario or ask product team for a customized one
            EnableMiscue: enableMiscue,
            EnableProsodyAssessment: enableProsody,
            NBestPhonemeCount: 0  // > 0 to enable "spoken phoneme" mode, 0 to disable
        };
        const pronunciationConfig = new sdk.PronunciationAssessmentConfig(JSON.stringify(configJson));
        pronunciationConfig.referenceText = referenceText;
    
        const speechRecognizer = new sdk.SpeechRecognizer(speechConfig, audioConfig);
   
        pronunciationConfig.applyTo(speechRecognizer);
        
        speechRecognizer.recognizeOnceAsync((result) => {
            if (result.reason === sdk.ResultReason.RecognizedSpeech) {
                console.log(`pronunciation assessment for: ${result.text}`);
                const recognized = result.text;
                setRecognizedText(recognized);
                const pronunciationResult = JSON.parse(result.properties.getProperty(sdk.PropertyId.SpeechServiceResponse_JsonResult));
                console.log(`assessment results:\n${JSON.stringify(pronunciationResult, null, 4)}`);
            } else if (result.reason === sdk.ResultReason.NoMatch) {
                console.log("No speech could be recognized");
            } else if (result.reason === sdk.ResultReason.Canceled) {
                const cancellationDetails = sdk.CancellationDetails.fromResult(result);
                console.log(`Speech Recognition canceled: ${cancellationDetails.reason}`);
                if (cancellationDetails.reason === sdk.CancellationReason.Error) {
                    console.log(`Error details: ${cancellationDetails.errorDetails}`);
                }
            }
            setIsListening(false);
        });

        // start recording
        const mimeTypeCheck = MediaRecorder.isTypeSupported('audio/mp4');
        if (!mimeTypeCheck) {
            console.error('MIME type not supported:', mimeTypeCheck);
            return;
        }
        const options = {
            mimeType: "audio/mp4",
        };
        mediaRecorderRef.current = new MediaRecorder(stream, options);

        mediaRecorderRef.current.ondataavailable = (event) => {
            if (event.data.size > 0) {
                // save data to your array
                //audioChunksRef.current.push(event.data);
            }
        };

        mediaRecorderRef.current.onstop = () => {
            // try {
            //     const localAudioBlob = new Blob(audioChunksRef.current, { type: 'audio/mp4' });
            //     const url = URL.createObjectURL(localAudioBlob);
            //     setAudioUrl(url);
            //     setAudioBlob(localAudioBlob)
            //     audioChunksRef.current = [];
            // } catch (error) {
            //     console.error('Error creating audio URL:', error);
            // }
            console.log('onstop:'); 
        };

        mediaRecorderRef.current.start(100);
    
        setIsListening(true);
    };

The important is to use:

const audioConfig = sdk.AudioConfig.fromStreamInput(stream);      

//in place of 

const audioConfig = sdk.AudioConfig.fromDefaultMicrophoneInput();

Share via

How to get audio file with the method recognizeOnceAsync of SpeechSDK.SpeechRecognizer?