Questions about the simultaneous recording of audio on multiple devices of Azure Kinect DK

Jian Cui 16

Hello everyone,

I purchased two Azure Kinect DK devices, and my purpose is to record the audio of these two devices simultaneously. I use the 'pyaudio' python toolkit to collect audio and use two threads to access the microphones of the two devices simultaneously.

However, by comparing the audio signals collected by the two devices, it is found that the audio signals collected between different devices have a delay of several milliseconds, which cannot meet the synchronization requirements. Does anyone know any way to achieve the effect of complete synchronization for Azure Kinect DK? Or is there a relevant SDK for Azure Kinect DK that can support this function?

Hope to get help, thank you all!

Jian

QuantumCache 20,266 Reputation points

2021-01-04T20:36:02.587+00:00

Hello @Jian Cui

Welcome to Microsoft Q&A Platform and thanks for your query & for showing interest in the Azure Kinect DK product. Community SME's on this topic or our team will review your scenario and circle back at the possible earliest time.
QuantumCache 20,266 Reputation points

2021-01-05T22:06:18.357+00:00

Hello @Jian Cui Could you please confirm the OS of your machine? Windows or Linux?

Also could you please confirm if you are using Azure Cognitive Services? Soon I will be having a quick update on your query.
Jian Cui 16 Reputation points

2021-01-06T01:19:31.473+00:00

Thanks for you reply @QuantumCache

I am using Windows operating system. I didn't use Azure Cognitive Services, because I read about Azure Cognitive Services roughly and I found that it provides some functions like voice-to-text. But I only want to collect the raw seven microphone data of Azure Kinect DK devices and meet the requirements of multi-device synchronization. Do you know that Azure Cognitive Services provides this function? Or do you know that there are some other methods that can achieve this purpose?
QuantumCache 20,266 Reputation points

2021-01-06T16:46:59.237+00:00

Hi @Jian Cui Below is the response from our Product team on Cognitive services.

From Azure Cognitive Services we are supporting a single device only not synchronization between two.

1 answer

QuantumCache 20,266 Reputation points

2021-01-06T16:59:22.68+00:00
Hello @Jian Cui

The Hardware Synchronization feature on Azure Kinect DK is for RGB and Depth Sensors only.

Reference document: Using an external sync trigger

Microphone array operates independently of the HW synchronization feature.

We wanted to know a few more details from your side...

What amount of latency is introduced by ‘pyaudio’ ?

How the starts for both microphones are synchronized in the code?

(We can try putting AKDKs on separate USB controllers to see if there is a change in observed latency).
Please sign in to rate this answer.

1 person found this answer helpful.
Jian Cui 16 Reputation points

2021-01-07T02:06:03.477+00:00

Thanks for you reply @QuantumCache

I think I have solved this problem roughly. I use python and use two threads (threading.Thread) to start two AKDKs at the same time and start recording by pyaudio. But in fact, the two threads did not truly synchronize. I added timestamps to the place where the first frame of audio signal was collected by the two AKDKs, and found that there was a delay of about 20 milliseconds between the two devices. So I manually calibrated this time delay to achieve a rough synchronization.

This is my current idea. Do you have any other solutions? I would really appreciate your other ideas.

By the way, I think it is very inconvenient to collect microphone array data separately. After the audio data are collected, if multiple devices are used at the same time, it is very difficult to synchronize the audio, depth, and RGB data as the threads increase.

QuantumCache 20,266 Reputation points

2021-01-07T20:16:40.923+00:00

Hello @Jian Cui Thanks for the response and here is the Document-Enhancement feedback request which we have submitted to the product team. Thanks for your input in this regard. I have shared your further query & experience with my team, will soon get back.

https://github.com/MicrosoftDocs/azure-docs/issues/68570
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Questions about the simultaneous recording of audio on multiple devices of Azure Kinect DK

1 answer

Your answer