Asynchrony in C# 5.0 part Four: It's not magic
Today I want to talk about asynchrony that does not involve any multithreading whatsoever.
People keep on asking me "but how is it possible to have asynchrony without multithreading?" A strange question to ask because you probably already know the answer. Let me turn the question around: how is it possible to have multitasking without multiple CPUs? You can't do two things "at the same time" if there's only one thing doing the work! But you already know the answer to that: multitasking on a single core simply means that the operating system stops one task, saves its continuation somewhere, switches to another task, runs it for a while, saves its continuation, and eventually switches back to continue the first task. Concurrency is an illusion in a single-core system; it is not the case that two things are really happening at the same time. How is it possible for one waiter to serve two tables "at the same time"? It isn't: the tables take turns being served. A skillful waiter makes each guest feel like their needs are met immediately by scheduling the tasks so that no one has to wait.
Asynchrony without multithreading is the same idea. You do a task for a while, and when it yields control, you do another task for a while on that thread. You hope that no one ever has to wait unacceptably long to be served.
Remember a while back I briefly sketched how early versions of Windows implemented multiple processes? Back in the day there was only one thread of control; each process ran for a while and then yielded control back to the operating system. The operating system would then loop around the various processes, giving each one a chance to run. If one of them decided to hog the processor, then the others became non-responsive. It was an entirely cooperative venture.
So let's talk about multi-threading for a bit. Remember a while back, in 2003, I talked a bit about the apartment threading model? The idea here is that writing thread-safe code is expensive and difficult; if you don't have to take on that expense, then don't. If we can guarantee that only "the UI thread" will call a particular control then that control does not have to be safe for use on multiple threads. Most UI components are apartment threaded, and therefore the UI thread acts like Windows 3: everyone has to cooperate, otherwise the UI stops updating.
A surprising number of people have magical beliefs about how exactly applications respond to user inputs in Windows. I assure you that it is not magic. The way that interactive user interfaces are built in Windows is quite straightforward. When something happens, say, a mouse click on a button, the operating system makes a note of it. At some point, a process asks the operating system "did anything interesting happen recently?" and the operating system says "why yes, someone clicked this thing." The process then does whatever action is appropriate for that. What happens is up to the process; it can choose to ignore the click, handle it in its own special way, or tell the operating system "go ahead and do whatever the default is for that kind of event." All this is typically driven by some of the simplest code you'll ever see:
while(GetMessage(&msg, NULL, 0, 0) > 0)
{
TranslateMessage(&msg);
DispatchMessage(&msg);
}
That's it. Somewhere in the heart of every process that has a UI thread is a loop that looks remarkably like this one. One call gets the next message. That message might be at too low a level for you; for example, it might say that a key with a particular keyboard code number was pressed. You might want that translated into "the numlock key was pressed". TranslateMessage does that. There might be some more specific procedure that deals with this message. DispatchMessage passes the message along to the appropriate procedure.
I want to emphasize that this is not magic. It's a while loop. It runs like any other while loop in C that you've ever seen. The loop repeatedly calls three methods, each of which reads or writes a buffer and takes some action before returning. If one of those methods takes a long time to return (typically DispatchMessage is the long-running one of course since it is the one actually doing the work associated with the message) then guess what? The UI doesn't fetch, translate or dispatch notifications from the operating system until such a time as it does return. (Or, unless some other method on the call chain is pumping the message queue, as Raymond points out in the linked article. We'll return to this point below.)
Let's take an even simpler version of our document archiving code from last time:
void FrobAll()
{
for(int i = 0; i < 100; ++i)
Frob(i);
}
Suppose you're running this code as the result of a button click, and a "someone is trying to resize the window" message arrives in the operating system during the first call to Frob. What happens? Nothing, that's what. The message stays in the queue until everything returns control back to that message loop. The message loop isn't running; how could it be? It's just a while loop, and the thread that contains that code is busy Frobbing. The window does not resize until all 100 Frobs are done.
Now suppose you have
async void FrobAll()
{
for(int i = 0; i < 100; ++i)
{
await FrobAsync(i); // somehow get a started task for doing a Frob(i) operation on this thread
}
}
What happens now?
Someone clicks a button. The message for the click is queued up. The message loop dispatches the message and ultimately calls FrobAll.
FrobAll creates a new task with an action.
The task code sends a message to its own thread saying "hey, when you have a minute, call me". It then returns control to FrobAll.
FrobAll creates an awaiter for the task and signs up a continuation for the task.
Control then returns back to the message loop. The message loop sees that there is a message waiting for it: please call me back. So the message loop dispatches the message, and the task starts up the action. It does the first call to Frob.
Now, suppose another message, say, a resize event, occurs at this point. What happens? Nothing. The message loop isn't running. We're busy Frobbing. The message goes in the queue, unprocessed.
The first Frob completes and control returns to the task. It marks itself as completed and sends another message to the message queue: "when you have a minute, please call my continuation". (*)
The task call is done. Control returns to the message loop. It sees that there is a pending window resize message. That is then dispatched.
You see how async makes the UI more responsive without having any threads? Now you only have to wait for one Frob to finish, not for all of them to finish, before the UI responds.
That might still not be good enough, of course. It might be the case that every Frob takes too long. To solve that problem, you could make each call to Frob itself spawn short asynchronous tasks, so that there would be more opportunities for the message loop to run. Or, you really could start the task up on a new thread. (The tricky bit then becomes posting the message to run the continuation of the task to the right message loop on the right thread; that's an advanced topic that I won't cover today.)
Anyway, the message loop dispatches the resize event and then checks its queue again, and sees that it has been asked to call the continuation of the first task. It does so; control branches into the middle of FrobAll and we pick up going around the loop again. The second time through, again we create a new task... and the cycle continues.
The thing I want to emphasize here is that we stayed on one thread the whole time. All we're doing here is breaking up the work into little pieces and sticking the work onto a queue; each piece of work sticks the next piece of work onto the queue. We rely on the fact that there's a message loop somewhere taking work off that queue and performing it.
UPDATE: A number of people have asked me "so does this mean that the Task Asynchrony Pattern only works on UI threads that have message loops? " No. The Task Parallel Library was explicitly designed to solve problems involving concurrency; task asynchrony extends that work. There are mechanisms that allow asynchrony to work in multithreaded environments without message loops that drive user interfaces, like ASP.NET. The intention of this article was to describe how asynchrony works on a UI thread without multithreading, not to say that asynchrony only works on a UI thread without multithreading. I'll talk at a later date about server scenarios where other kinds of "orchestration" code works out which tasks run when.
Extra bonus topic: Old hands at VB know that in order to get UI responsiveness you can use this trick:
Sub FrobAll()
For i = 0 to 99
Call Frob(i)
DoEvents
Next
End Sub
Does that do the same thing as the C# 5 async program above? Did VB6 actually support continuation passing style?
No; this is a much simpler trick. DoEvents does not transfer control back to the original message loop with some sort of "resume here" message like the task awaiting does. Rather, it starts up a second message loop (which, remember, is just a perfectly normal while loop), clears out the backlog of pending messages, and then returns control back to FrobAll. Do you see why this is potentially dangerous?
What if we are in FrobAll as a result of a button click? And what if while frobbing, the user pressed the button again? DoEvents runs another message loop, which clears out the message queue, and now we are running FrobAll within FrobAll; it has become reentrant. And of course, it can happen again, and now we're running a third instance of FrobAll...
Of course, the same is true of task based asynchrony! If you start asynchronously frobbing due to a button click, and there is a second button click while more frobbing work is pending, then you get a second set of tasks created. To prevent this it is probably a good idea to make FrobAll return a Task, and then do something like:
async void Button_OnClick(whatever)
{
button.Disable();
await FrobAll();
button.Enable();
}
so that the button cannot be clicked again while the asynchronous work is still pending.
Next time: Asynchrony is awesome but not a panacea: a real life story about how things can go terribly wrong.
------------
(*) Or, it invokes the continuation right then and there. Whether the continuation is invoked aggressively or is itself simply posted back as more work for the thread to do is user-configurable, but that is an advanced topic that I might not get to.
Comments
Anonymous
November 03, 2010
Perhaps I'm not understanding this right, but based on your post it sounds like await only works in applications that actually HAVE a message loop (i.e. windows applications).Anonymous
November 03, 2010
Awesome....makes a lot more sense now.Anonymous
November 03, 2010
After reading your blog for several years it has become apparent that Frob()ing is a common idiom. Any chance we can see this as a keyword (even contextual) in the next version of C#?Anonymous
November 03, 2010
Am I correct in thinking we're not protected by locks using the new functionality then, unless we don't allow recursive acquires (which is recommended anyway). This post reminds me of an SO question I tried to answer a while ago, where code achieving a similar purpose was put on MSDN: stackoverflow.com/.../dispatcher-vs-multithreading I chose the example of connecting to an FTP site where you really want another thread doing the work, as it usually takes 10s of seconds to time out on something like that.Anonymous
November 03, 2010
How exactly do the tasks created in FrobAll() actually get started? A TaskEx.Run() would seem to be needed instead of new Task().Anonymous
November 04, 2010
@Lenny: applications run in a Context that determines how tasks are queued and run. Your task might be running in a UI context or an ASP.NET context or a multi-threaded context; each of these will queue tasks differently. So you're not limited to applications with a message loop. Look at the SynchronizationContext and TaskScheduler classes for more details.Anonymous
November 04, 2010
I think you plagiarized my blog's title: "It's Not Magic." :) I am also confused on how the Windows message loop is suddenly aware of the fact that the task needs to continue running. Is this predicated on the assumption that a specific task Context is posting and intercepting specific messages in the message loop? I'm not too up-to-speed on all the .NET 4.0 stuff and the task framework.Anonymous
November 04, 2010
Does this mean async and await only works in GUI applications, and more specifically, on threads that have such a message loop (so async won't work on console applications)?Anonymous
November 04, 2010
It'll also work on non-GUI threads; but it will work differently there: since there's no message loop to post to, the task will be enqueued as work item in the thread pool. This means that any thread from the pool may run the continuation, so each 'await' might jump from one thread to the next. Of course, if you really want to stay on a single thread in a console application, you still have the option to write your own message loop and SynchronizationContext implementation.Anonymous
November 04, 2010
To continue with my previous comment, if we take your document download and archive concept and apply it here, won't we potentially get lengthy hangs if we struggle to connect to the download service (as far as I know, that's a single operation which can't be broken up into smaller tasks)? Assuming we're writing a Windows Forms app here, how does the new async stuff work regarding lengthy operations that can't be broken up? Do I just go back and do things the old way?Anonymous
November 04, 2010
Alex: WinSock has async versions of probably every network call you would make, so nothing has to be done on a new thread to avoid blocking the UI. However, as Raymond Chen pointed out recently, there's no way to make the CreateFile call async (you need a file handle to associate with an IOCP but you don't get a handle until CreateFile returns). Thus, anything that calls CreateFile (and many APIs call CreateFile internally) has to be run on another thread to avoid blocking the UI. CreateFile can block on accessing a network server, spinning up a HD, seeking a CD-ROM, or any number of other possibilities.Anonymous
November 04, 2010
@Daniel Grunwald: So await does require separate threads if there's no message loop available? That doesn't jive with what microsoft have been telling us throughout - 'async doesn't need threading!'Anonymous
November 04, 2010
Maybe I am missing something, but I was a bit confused that the Task.Start() would schedule the Task to be executed on the UI thread. I pasted this code in a Windows Forms app with the following implementation of Frob. private void Frob(int i) { button1.Text = "Demo Frob:" + i; } This throws an InvalidOperationException: "Cross-thread operation not valid: Control 'button1' accessed from a thread other than the thread it was created on." I'm not sure about other UI frameworks like Silverlight, but for Windows Forms you will need to explicitly pass a TaskScheduler to make sure the Task gets scheduled on the UI thread (In Windows Forms): task.Start(TaskScheduler.FromCurrentSynchronizationContext());Anonymous
November 04, 2010
The comment has been removedAnonymous
November 04, 2010
@James Hart: Task<T> isn't really to do with threads. It's to do with "a task which may complete at some point". Maybe that will involve other threads - maybe it won't. Just because it's part of the TPL doesn't mean it's explicitly about threads. I was originally concerned about the use of Task<T> as well, but I think it's neutral enough to be okay.Anonymous
November 04, 2010
The comment has been removedAnonymous
November 04, 2010
The comment has been removedAnonymous
November 04, 2010
Eric, could you further elaborate on the pros&cons of doing UI asynchrony either as continuations or as DoEvents? Your button-disabling example isn't solved any differently with DoEvents.Anonymous
November 05, 2010
The comment has been removedAnonymous
November 05, 2010
@Apollonius Good luck with task compositions using yields...Anonymous
November 05, 2010
The comment has been removedAnonymous
November 05, 2010
Yes, you are missing something. await taskExpression will evaluate taskExpression just as yield return taskExpression does; the whole point of both is that taskExpression evaluates to a Task, i.e. a promise for a value, not the value itself. Note how yielding the task will make AsyncByHand sign up the next iterator round (i.e. everything left to do) as a continuation of the task. This behaviour is identical to that of await! Regarding Focus' comment: You apparently don't understand that task compositions do not really have much to do with the language feature. Task composition is just the process of creating a new task from old tasks, which is already possible today (see System.Threading.Tasks.TaskFactory.ContinueWhenAll/ContinueWhenAny). There is no reason why this wouldn't work with my solution.Anonymous
November 05, 2010
The comment has been removedAnonymous
November 05, 2010
What I provided is not an "example". It is a rough sketch of how to translate the "await" feature to an iterator. As such, it is "just async, not multithreaded", whatever you mean by that. To your second point: I acknowledge that there are different opinions on this one, but I think that once you take out the body of AsyncByHand - which, as I said, could be refactored into a library method - you end up with almost the same code as with await. I am simply doubting that introducing a fairly large changes - new keywords in the language, new rewriting passes in the compiler - is worth the benefits which I personally consider quite small. (I know, I keep repeating myself.) But note also that I'm not so much restating what Eric said but rather contradicting him in a way; the C# team apparently seeks the easiest possible syntax as the key to promoting asynchronicity, while I am actually content with a fairly easy syntax and think awareness is more important. (Apologies to Eric if I have incorrectly stated "his opinion" on this stance, and for highjacking the comment discussion on his blog) Honestly, this is just my opinion; it's not binding for anyone, just a counterweight to the "Huh, that's awesome" comments posted earlier.Anonymous
November 07, 2010
The comment has been removedAnonymous
November 07, 2010
Is it possible to specificically start the async task in a new thread ? something like- awaitNewThread FrobAsync(i); instead of - await FrobAsync(i);Anonymous
November 25, 2010
On a side note regarding your example of disabling the button prior to starting FrobAll, what happens if there are already multiple button click messages in the queue before the first button click is processed. Will disabling the button cause any pending messages for that button in the queue to be discarded (in GetMessage or DispatchMessage maybe) or will they be delivered?Anonymous
December 30, 2010
The comment has been removedAnonymous
January 28, 2012
The async void FrobAll seems to be awaited in Button_OnClick, if I understand it right. But in the Visual Studio 11 preview, async void methods really do return void (not Task). So you can't await them. I'm pretty sure it must have worked in the original CTP because I wrote a blog post that mused on taking a different approach to the problem of ensuring two button clicks don't launch two simultaneous "Frobs": smellegantcode.wordpress.com/.../c-5-0-asyncawait-and-gui-events Of course it could be fixed by changing your async void method to return Task<bool> or something. But would I be right if I took a strong hint from this change: that side-effecting asyncs should not be freely composed in the same way as value-returning asyncs? Is there something dangerous about doing this?Anonymous
January 28, 2012
I see that changing "async void FrobAll()" to "async Task FrobAll()" restores the ability to await, so I guess it's not frowned upon after all.Anonymous
October 31, 2012
Makes a lot sens now!