What happens when audio rendering fails?

Skywing sent me an email earlier today asking me essentially "Why doesn't Windows do a better job of handling the case where the default audio device goes away?"[1]

It's a good question, and one that we've spent a lot of time working on for quite some time (this isn't a new issue for Vista, it's been there since day one, not that it actually matters).

For the vast majority of existing users, this isn't a problem - they only have one audio device on their machine, and thus its a moot question - their default device almost never goes away.  On the other hand, looking forwards, it's going to become a more common scenario because of the prevalence of Bluetooth audio hardware and USB headsets (for example, I'm currently listening to Internet radio over a pair of Bluetooth speakers I purchased the other day).

The short answer to Skywing's question is: "It's the responsibility of the application to deal with handling errors".  The audio stack bubbles out the error to the application and lets it figure out how to deal with the problem.

Which then begs the next question: "How do you handle errors so that you can recover from them?"  It turns out that the answer to that question requires a bit of digging into the audio stack.

As I've discussed in the past, there are four major mechanisms for accessing the audio functionality in Windows Vista.  They are:

1. MME - the legacy MME APIs, including waveOutXxx, waveInXxx, and mixerXxxx
2. DirectSound/DirectShow
3. Media Foundation (new in Vista)
4. WASAPI (new in Vista)

For the MME APIs, an application usually accesses the audio stack using the WAVE_MAPPER (or MIDI_MAPPER) pseudo-device.  The nice thing about the WAVE_MAPPER device is that it doesn't matter which device is the current device, it just uses the one chosen as the user's device.  Alternatively, for the MME APIs, you can select a specific device.  For the MME APIs, devices are numbered from 0 to <n>; for appcompat reasons, starting in Windows Server 2003, device 0 is typically the user's default device (there were applications that hard coded device 0 for their output device, which caused issues when you had more than one audio device).

For DirectShow and DirectSound, the call to the DirectSoundCreate API takes a GUID which represents the output or input device, or NULL to represent the default device.  In addition, it also provides a mechanism to address the default voice communications device (DSDEVID_DefaultVoicePlayback).

For Media Foundation and WASAPI, you specify the specific audio endpoint on which you want to render or capture audio in the initialize call (MFCreateAudioRenderer for MF, for WASAPI, you activate an IAudioClient interface on the endpoint).

In either case, once you start streaming to a device, the only mechanism in place when something goes wrong is that an API call fails.  That means that it's up to the application to figure out how to recover from any failure.

For the wave APIs and DSound, you really don't have any choice but to detect the failure close down your local resources and restart streaming - the legacy APIs don't allow you a good mechanism for detecting the cause behind a streaming failure.

For MediaFoundation, MF generates events using its' event generator mechanism to inform an application of interesting events; among the events that can be received is an event indicating that the audio device was removed.  There are other relevant events generated, including events that are generated when the audio service is stopped (which also stops streaming), when the mix format for the audio endpoint is changed, etc.

For WASAPI, a WASAPI client can retrieve the IAudioSessionControl service from an IAudioClient object and can use the IAudioSessionControl::RegisterAudioSessionNotification to register an IAudioSessionEvents interface.  The audio service will call the IAudioSessionEvents::OnSessionDisconnected method when it tears down an audio stream (all these notifications are also passed through to MediaFoundation's event mechanism).

In Windows Vista, there are six different disconnect events generated, and there's a specific set of recovery steps for each of them:

[1] That's not the actual question that he asked, but the answer to his question is included in the answer to my question, so...

Comments

• Anonymous
  October 31, 2007
  As a recent purchaser of a set of bluetooth headphones, I frequently run into the situation of loading

• Anonymous
  October 31, 2007
  "For the vast majority of existing users, this isn't a problem - they only have one audio device on their machine, and thus its a moot question - their default device almost never goes away." This might be one advantage of OEM versions, even though the purchaser purchases them at retail (usually without an option to refuse) when buying a new PC.  With OEM versions, usually the PC's vendor has undertaken some amount of effort to get a more or less working driver installed. But it's still pretty easy to delete that device, accidentally, when doing trial and error to try to debug a problem.  The device might or might not have a driver provided by Microsoft in the Vista DVD, if there's a driver then it might or might not work, and users need to do some trial and error.  For some reason in Vista it's pretty easy to delete Vista's awareness of the device without intending to do so.  In XP it took some deliberate action to delete XP's awareness of a device.  Also in XP it wasn't hard to reverse that action and redetect the device.

• Anonymous
  October 31, 2007
  Norman, I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it. The only scenario where this matters is when you've got an existing audio device, you add a new audio device (bluetooth headphones, for example), then you remove the new device.  What happens at that point is up to the application.

• Anonymous
  October 31, 2007
  Thanks for the answer, Larry :) For purposes of hacking around stupid apps, games in particular, I suppose the most logical place to start is to hook DirectSound and provide a logical abstraction of there never being a loss of the device (when such occurs, recreating the internal, "real" DirectSound objects that the abstraction/hook passes through to for most DSound requests, initializing them as the application initialized the abstraction at start time, and set buffers back up).  I'd imagine there will likely be some glitching in doing all of that, with switching sound buffers midstream without the cooperation of the app, but for the fairly infrequent case of switching audio devices, I'd be willing to live with that.  It's much better than losing sound indefinitely, in any case. Aside from WMP, most of the things that I use on a regular basis which are broken like this are likely to be DirectSound.  I'm hoping that WMP can be fixed at some point and save me from having to do similar hackage with whatever API set it uses internally, though :)

• Anonymous
  October 31, 2007
  "I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it." Taking a glance at a system where that doesn't happen but seeing what menu options pop up, I'll make two vague guesses here.  One is that when the volume mixture applet displayed a device that was disabled due to having no working driver, it was easier to unintentionally delete all knowledge of the device than it was to try to find a working driver.  One is that when the speaker icon had a red X because the sound chip didn't have a working driver, that icon's context menu had something similar.  If I have time to experiment on one PC again then I might be able to reproduce this. I have a third vague guess too.  On a PC that I no longer have, the audio system worked fine with Windows Media Player but not with Windows Media Centre.  The dialogs in Windows Media Centre were more confusing.  There was a statement something like "Please reinstall Windows or reinstall Media Centre" which I actually tried doing several times (as an invited beta tester) but had no effect (as I predicted).  There were some other options which didn't help, and I don't remember if this might have been the PC where I ended up entirely deleting the audio without having that intention.  But I no longer have that PC, and its current owner is using it with its XP licence.

• Anonymous
  November 01, 2007
  The comment has been removed

• Anonymous
  November 02, 2007
  Hi Larry, You said to Norman: "I'm not aware of any way of deleting an audio device that doesn't involve the user using the device manager.  If you are aware of one, I'd love to hear about it." I used the contact button on your blog a few days ago as your blog hadn't dealt with the topic yet about Vista behavior regarding unplugged devices.  It seems that some new device drivers shipped with Dells for Sigmatel based devices don't report any devices at all when you query what's connected with waveInGetNumDevs.  If you plug in a 3.5mm jack, suddenly they do report devices.   I realize waveInGetNumDevs is an old legacy call, but is that behavior expected on Vista? It wouldn't surprise me if these same devices also magically removed the device if you unplugged them.  Unfortunately, these problems don't show up here in testing, only on end-user machines. Anthony Wieser Wieser Software Ltd

• Anonymous
  November 02, 2007
  Anthony, sorry I must have missed that email.  This is absolutely by design, it's a feature called jack detection - essentially an audio device only becomes available when something's plugged into it.  And yes, the wave device will disappear when it's unplugged. But the audio driver isn't removed in that scenario.

• Anonymous
  November 02, 2007
  Thanks for the clarification Larry. I guess I misread the original post when I suggested it was removed.  Is there also a notification when the device is unplugged?  What will my App see if that happens?  

• Anonymous
  November 02, 2007
  Anthony: Yes, there is, but it comes from a different mechanism - MMDeviceAPI fires a device state change notification with "Device Unplugged" when you unplug a device.

• Anonymous
  November 04, 2007
  The comment has been removed

• Anonymous
  November 04, 2007
  The comment has been removed

• Anonymous
  November 10, 2007
  Dean Harding, This issue just reeks of DRM (it checks protected media path). I'm sure Larry or someone else will deny, but he/she better provide a good explanation...

• Anonymous
  November 11, 2007
  Grof: DRM won't affect a rendering stream. Dean: No - the jack detection logic exists because we (and our hardware partners) got a stream of complaints from customers who kept on complaining that they didn't hear any output from their computer when the reason was that they hadn't plugged anything into the jack.   In all honesty, I suspect that the number of people who WANT to be able to play audio without hearing the audio is relatively small, especially when compared to the set of people who want an error to occur when they attempt to play audio through unplugged speakers.

• Anonymous
  November 15, 2007
  Larry: I guess so :-) I guess I could just buy some $2 headphones and leave them plugged in...