Asynchrony in C# 5, Part One
The designers of C# 2.0 realized that writing iterator logic was painful. So they added iterator blocks. That way the compiler could figure out how to build a state machine that could store the continuation - the “what comes next” - in state somewhere, hidden behind the scenes, so that you don’t have to write that code.
They also realized that writing little methods that make use of local variables was painful. So they added anonymous methods. That way the compiler could figure out how to hoist the locals to a closure class, so that you don't have to write that code.
The designers of C# 3.0 realized that writing code that sorts, filters, joins, groups and summarizes complex data sets was painful. So they added query comprehensions and all the rest of the LINQ features. That way the compiler could figure out how to do the right object model calls to build the query, the expression trees, and so on.
The designers of C# 4.0 realized that interoperating with modern and legacy dynamic object models was painful. So they added the dynamic type. That way the compiler could figure out how to generate the code at compile time that does the analysis in the Dynamic Language Runtime at runtime.
The designers of C# 5.0 realized that writing asynchronous code is painful, in so many ways. Asynchronous code is hard to reason about, and as we've seen, the transformation into a continuation is complex and leads to code replete with mechanisms that obscure the meaning of the code.
This shall not stand.
I am pleased to announce that there will be a C# 5.0 (*), and that in C# 5.0 you’ll be able to take this synchronous code:
void ArchiveDocuments(List<Url> urls)
{
for(int i = 0; i < urls.Count; ++i)
Archive(Fetch(urls[i]));
}
and, given reasonable implementations of the FetchAsync and ArchiveAsync methods, transform it into this code to achieve the goal of sharing wait times as described yesterday:
async void ArchiveDocuments(List<Url> urls)
{
Task archive = null;
for(int i = 0; i < urls.Count; ++i)
{
var document = await FetchAsync(urls[i]);
if (archive != null)
await archive;
archive = ArchiveAsync(document);
}
}
Where is the state machine code, the lambdas, the continuations, the checks to see if the task is already complete? They’re all still there. Let the compiler generate all that stuff for you, just like you let the compiler generate code for iterator blocks, closures, expression trees, query comprehensions and dynamic calls. The C# 5.0 code above is essentially a syntactic sugar for the code I presented yesterday. That's a pretty sweet sugar!
I want to be quite clear on this point: the action of the code above will be logically the same as the action of yesterday's code. Whenever a task is "awaited", the remainder of the current method is signed up as a continuation of the task, and then control immediately returns to the caller. When the task completes, the continuation is invoked and the method starts up where it was before.
If I’ve timed the posting of this article correctly then Anders is announcing this new language feature at the PDC right about… now. You can watch it here.
We are as of right now making a Community Technology Preview edition of the prototype C# 5.0 compiler available. The prototype compiler will be of prototype quality; expect features to be rough around the edges still. The idea is to give you a chance to try it and see what you think so that we can get early feedback.
I'll be posting more about this feature tomorrow and for quite some time after that; if you can't wait, or want to get your hands on the prototype, you can get lots more information and fresh tasty compiler bits from msdn.com/vstudio/async.
And of course, in keeping with our co-evolution strategy, there will be a Visual Basic 11 and it will also feature task-based asynchrony. Check out the VB team blog for details, or read all about this feature at my colleague Lucian's blog. (Lucian did much of the design and prototyping of this feature for both C# and VB; he, not I, is the expert on this so if you have deep questions, you might want to ask him, not me.)
Tomorrow: await? async? Task? what about AsyncThingy<T>? Tell me more!
(*) We are absolutely positively not announcing any dates or ship vehicles at this time, so don't even ask. Even if I knew, which I don't, and even if my knowledge had the faintest chance of being accurate, which it doesn't, I still wouldn't tell you.
Comments
Anonymous
October 28, 2010
listening to Future of c# and heard the word continuation. let's play drinking game... :DAnonymous
October 28, 2010
Cool :) Was this influenced by Delphi Prism's async support, or is it just a coincidence? In any case, that is great syntactic sugar as async operations require a lot of ceremony and plumbing around them.Anonymous
October 28, 2010
No more "hypothetical future versions" hooray! This looks pretty amazing although you spoiled part of it with your CPS run up to PDC. I'm going to listen to Anders finish explaining this and consider the implications.Anonymous
October 28, 2010
Just saw it. Its greatly simplifies Threading. cool..Anonymous
October 28, 2010
Just the other day I was thinking about some Technical DSL to come composing Tasks easier, and now look at that. That looks neat and great...but it ain't threading, or is it? Just saw Anders' session and there seems to be no need to dispatch on the UI thread when the continuations kick in. Or was the example just well balanced to avoid that? There's still a lot for me to understand but it looks veeery neat and useful.Anonymous
October 28, 2010
It certainly looks very, very similar to the (already shipping in Dev 10) F# async support - the big difference being the use of Task<T> rather than Async<T>. I can certainly see the merits of using the existing Task<T> type, but it does raise the question of interop. Will it become F#'s problem to adapt to match?Anonymous
October 28, 2010
This is.... amazing. I think it might even be more beneficial than Linq. Linq made things we already did prettier. This stuff is going to help us do things we didn't even consider doing, because they were just too painful. And all of that for the price of just two keywords! Hats off.Anonymous
October 28, 2010
The comment has been removedAnonymous
October 28, 2010
@Frank: it wouldn't be magic if it didn't take care of dispatching continuations on the right thread for you, wouldn't it :)Anonymous
October 28, 2010
I know that you guys aren't trying to copy F# but don't you think a more general syntax for monads would allow more flexibility?Anonymous
October 28, 2010
This seems pretty nice and it abstracts some of the tedium of handling continuations from the programmer. However I'm curious about further applications and how perhaps it could feed into the Reactive framework. Tasks are great for single results, but what about series of results? I'd love to see it tie in with IObservable<T> and provide "asynchrony" across iterators. For example, imagine the following: public async IObservable<int> CountSlowly(int start, int end, int delay) { for(int i = start; i < end; i++) { Thread.Sleep(1000); yield return i; } } Then there is the whole concept of synchronization. The "await" keyword makes simple linear continuations nice and easy, but what about waiting across multiple sets of tasks grouped into levels of priority? I was a huge fan of the Cw MSR project and the "Polyphonic C#" project that inherited from and I thought that the language semantics introduced there were of the both elegant method I've ever seen for complex synchronization scenarios. The Reactive framework does introduce joins over observables which provides the functionality but, as Anders put it, you have to twist you coding inside-out with lambdas to utilize. So I think that the new enhancements are great and work well with the existing .NET 4.0 TPL concepts, but I hope that the language designers also look at Reactive and see the huge amount of potential that could be realized in also marrying those concepts. I can only imagine that by the time that C# 5.0 is anywhere near shipping that Reactive will be similarly ready for prime time as a part of .NET Framework 5.0.Anonymous
October 28, 2010
How related async, await with Rx? Are they both solving the same async problems?Anonymous
October 28, 2010
Isn't the term "ship vehicle" redundant? I don't think I've ever seen a ship that wasn't also a vehicle. Can a ship be anything other than a boat or rocket? Hopefully C# 5 will be of the latter variety.Anonymous
October 28, 2010
@guest According to Eric, you can 'await' any type with a GetAwaiter() method (or extension method) matching the required pattern. Today's Rx drop includes an Observable.GetAwaiter() method, so you should be able to 'await' any IObservable<T>. However, you can only declare 'async' methods which return void, Task, or Task<T>. In terms of interoperability potential, I think this is about the best we could have hoped for.Anonymous
October 28, 2010
I wonder decision was made to require async keyword for a method? Compiler slices methods with yield returns in the similar manner without any additional keywords.Anonymous
October 28, 2010
@Iso Su: Anders mentioned in his talk that they were capable of transforming the code without the async keyword but they decided to include it so that it is clear a method is asynchronous (and can return before its work is done).Anonymous
October 28, 2010
This is pretty cool. I was just contemplating how one would make use of this feature in order to create a function which is triggered on completion of the async event. Then I realized one could easily enough just make an async function which awaits completion of another async function and then performs whatever actions are necessary.Anonymous
October 28, 2010
Awesome ! After yesterday's post, I expected something like that, but I had no idea how it would look like... I assume the FetchAsync method just starts a new Task<Document> and returns it ?Anonymous
October 28, 2010
How do Reactive Extensions fit in this picture? Are they going to become useless, coexist, or become even more relevant with C# 5.0?Anonymous
October 28, 2010
@Flavien, watch Bart De Smet's session on the PDC website (LINQ Take Two), he explains how Rx can take advantage of this new feature. But the video of Anders's session doesn't seem to be available yet, even though it was planned at the same time as Bart's...Anonymous
October 28, 2010
@ChaosPandion A general syntax for monads ... like LINQ you mean?Anonymous
October 28, 2010
Will this link in with LINQ? One of the biggest challenges in my recent WPF/LINQ to SQL applications was making LINQ calls asynchronous to avoid horrendous roundtrip times. LINQ to SQL doesn't like to be used from multiple threads.Anonymous
October 28, 2010
So all of that talk about CPS and how it reifies and generalizes control flow so you can write whatever control flow you want... none of that is actually in C# 5? Just the ability to make stuff async? Don't get me wrong, I know making stuff async is a Big Friggin Deal and this is a really cool feature. I just was expecting something different after the whole CPS discussion. The "implementing try-catch via CPS" had me expecting the ability to implement my own control flow structures, like super-"using" that calls .Success() on normal completion and .Fail(ex) on throw before calling .Dispose() in the finally block. Or making it possible to change my Update function which currently reads like: Update(update => { update.DoOneThing(); update.DoAnotherThing(); }); with less punctuation and avoiding the problems with definite assignment if I wanted to assign a value inside the block or return directly from it to the outer caller. Am I missing a way that async / await can accommodate this sort of thing, is there more yet to be announced, or is that just not part of C#5?Anonymous
October 28, 2010
I hope this quickly finds its way into Silverlight and Windows Phone 7 to simplify all the async calls we do.Anonymous
October 28, 2010
The comment has been removedAnonymous
October 28, 2010
can you use await and yield in the same enumerator? how about try/catch ?Anonymous
October 28, 2010
The comment has been removedAnonymous
October 28, 2010
I've looked at the 'await' support in the reactive framework now; it simply subscribes and waits until all elements were observed, then returns those as a List<T>. Anders' PDC demo seems to have been carefully constructed to avoid this problem. It has UI and data access intermixed, and they cannot be cleanly separated without having to fall back to 'turn yourself inside out' when you consume the incoming data stream (IObservable). Using 'await' with the observable would destroy the usability of the app, as no elements would be visible until the whole set is loaded.Anonymous
October 28, 2010
Asynchrony. Sounds very odd. Shouldn't it be Asynchronicity?Anonymous
October 28, 2010
As mentioned above: What about exception handling? Could I do something like this? (and what would the semantics be?) async void ArchiveDocuments(List<Url> urls) { try { Task archive = null; for(int i = 0; i < urls.Count; ++i) { var document = await FetchAsync(urls[i]); if (archive != null) await archive; archive = ArchiveAsync(document); } } catch (MyFetchException e) { ... } } Because ... without exception handling IO stuff is still going to be painfull (albeight a bit less than we are used to).Anonymous
October 28, 2010
As mentioned above: What about exception handling? Could I do something like this? (and what would the semantics be?) async void ArchiveDocuments(List<Url> urls) { try { Task archive = null; for(int i = 0; i < urls.Count; ++i) { var document = await FetchAsync(urls[i]); if (archive != null) await archive; archive = ArchiveAsync(document); } } catch (MyFetchException e) { ... } } Because ... without exception handling IO stuff is still going to be painfull (albeight a bit less than we are used to).Anonymous
October 28, 2010
@Michael, yes, you can do exception handling of course. Watch Anders' presentation at PDC, he shows an exampleAnonymous
October 28, 2010
The comment has been removedAnonymous
October 28, 2010
The comment has been removedAnonymous
October 28, 2010
@Szindbad: but the URLs aren't independent in his example; he wanted to run the next fetch operation concurrently to the archive operation, but not run multiple fetch operations concurrently. If parallelism between independent operations was the goal, C# 4.0 can already do it easily without using "async" at all: Parallel.ForEach(urls, url => Archive(Fetch(url)); @W:
- AFAIK it can throw only when someone calls cancellationToken.ThrowIfCancellationRequested(). That means even "await" itself doesn't throw (but it can propagate exceptions thrown by the task it was waiting for).
- That's what I was complaining about: it simply doesn't work. async and yield are similar, yet can't be mixed. The example you gave CANNOT be written using the async construct unless you jump through some major hoops. Apparently the plan is to have the System.Reactive library jump through those hoops for you, but I'd prefer if C# allowed mixing of async and yield.
Anonymous
October 28, 2010
The last week was like the week before Christmas. And today Santa delivered his presents.Anonymous
October 29, 2010
I it possible to "await" an event?Anonymous
October 29, 2010
Re: "The designers of". I do not like that. Always thought they had realized a bit more than you mention, in all cases. Re: "I am pleased to announce that there will be a C# 5.0". I am very pleased to here that. Re: "transform it into this code". Nice sugar, but the archive variable is not intuitive. If it is set to null and not used anywhere, how can one expect it is not null? I guess it is already too late to move or use it, but still... I would also add more sugar so that created async tasks can be killed easily. What if they are processor hungry and I have lost interest in the result?Anonymous
October 29, 2010
Re-read the article and understood how archive can be not null. Great sugar!Anonymous
October 29, 2010
Wow - just like tasks and rendezvous points that Ada had in the 1980'sAnonymous
October 29, 2010
That is so ugly. Look at Google Go's approach instead.Anonymous
October 29, 2010
I know you're not announcing dates, but you included one more bit of information that Anders and others didn't - you mentioned "C# 5". I was starting to have some hope that this feature might ship as an official "go live" update, but I'd imagine that C# 5 would be further out and would include more changes. This change simplifies Silverlight development so much and seems to work well, so it'd be a shame not to be able to use it for a long time because we're waiting for the rest of C# 5.Anonymous
October 29, 2010
Try Nemerle. It's much better than thisAnonymous
October 29, 2010
I think yield task would make it clear that a Task is coming from the statement.Anonymous
October 29, 2010
Looks a lot like F# asynchronous workflows :)Anonymous
October 29, 2010
The comment has been removedAnonymous
October 29, 2010
@Bohdan: async and await themselves deal with tasks, not threads. How a particular task executes is up to that task's implementation. Some tasks spin up threads (and, again, it's up to them where the threads come from, and how many). I/O stuff would probably use Win32 APIs with callbacks, so no threads involved at all.Anonymous
October 29, 2010
@Bohdan Trotsenko "How many threads will do the job" Needn't be multiple threads involved at all. A GUI-based app, single threaded, could have a message-loop timer event, and expose it as a Task (or syntactically Task-like object) that finishes when the time period ends. In an async method you'd say: // do stuff await Timer.AsTask(5); // do more stuff AsTask would return the Task, the compiler would turn the async method into a restartable state machine, and it would be equivalent to enlisting on the boring old GUI timer event a handler lambda that would do-more-stuff.Anonymous
October 30, 2010
Can't wait...Anonymous
October 31, 2010
Nice! All these years of inability to easily update GUI... Gone!!!!!Anonymous
November 02, 2010
blogs.msdn.com/.../asynchrony-in-c-5-part-one.aspx blogs.msdn.com/.../asynchrony-in-c-5-part-one.aspx blogs.msdn.com/.../asynchrony-in-c-5-part-one.aspxAnonymous
November 23, 2010
I like it! I don't care what other people are saying about redundancy.. blah blah.. I think it's cool!Anonymous
December 07, 2010
This is a great feature and makes me jealous that I don't write in C# all the time. One question - what happens when debugging? When stopped, what would the call stack show - the code as written or the code as generated?Anonymous
December 10, 2010
Yes! This makes me so happy. In 2007 I posted this question - www.msdotnet.org/interrupting-flow-of-a-function-and-or-yielding-control-t355895.html - and a few people from the CLR team have either participated or seen this. The kind of things that we were trying to do are now really possible without having to host CLR :)Anonymous
January 13, 2011
I don't want asynchrousity. i wont more generous programming! Whati talking about? Consider this please: class A<B> where B has { [static ]void SuperFunc(); } {////} A wil accept any type that has public void SuperFunc(); method. if static specified, look for public static coid SuperMethod() method, for instance method otherwise. class A<B> where B like [static] ISuperInterface {///} A will accept any type that has at least all methods and properties of ISuperInterface as public members. if static is specified, require fur these members as static, and so on. If B implemnts ISuperInterface in a usual way, it is acceplable too. Lets declare some operators: public static abstract class MathBase<T> { public abstract static T operator+(T a, T b) {,,,,}; public abstract static T operator-(T a, T b) {,,,,}; public abstract static T operator*(T a, T b) {,,,,}; public abstract static T operator/(T a, T b) {,,,,}; } Lets make it more specific: public virtual static class SuperMath : MathBase<SuperNumber> { public override static SuperNumber operator+(SuperNumber a, SuperNumber b) {,,,,}; public override static SuperNumber operator-(SuperNumber a, SuperNumber b) {,,,,}; public override static SuperNumber operator*(SuperNumber a, SuperNumber b) {,,,,}; public override static SuperNumber operator/(SuperNumber a, SuperNumber b) {,,,,}; } And so on.... Or let we allow this: public staic delegate T Sum<T>(this IEnumerable<T> items) where T has { static T operator+(T a, T b) } {...} and now we can sum anithing that has the addition operator. I sure it will open the new era of generic rogramming,,, ir at least can..Anonymous
February 16, 2011
strange how i was just this morning thinking about how this is still the one painful part of .net/c#... this will be such a hugely needed improvement.Anonymous
July 07, 2011
what if some Fetch call never returns?Anonymous
September 07, 2011
Is alredy out some version (beta, alpha) of the .Net Framework 5 ?Anonymous
October 14, 2011
The comment has been removedAnonymous
October 14, 2011
(well, I forgot checking for the empty list, my bad)Anonymous
November 19, 2011
Pretty cool feature. It keeps everything under the wraps but still enriches the language.