Async Mashups using ASP.NET Web API
This blog shows an example of how to build an ASP.NET Web API ApiController that asynchronously talks to multiple other Web APIs in parallel without blocking a thread on the server. Btw, if you have detected a certain theme in these blogs around using Tasks with ASP.NET Web API then you are indeed on to something :)
Asynchronous programming is an important part of building scalable, robust, and responsive Web applications regardless of whether on client side or server side. Asynchronous programming has traditionally been very complicated leaving it to only the most dedicated to implement it but with the new Task model even complex patterns such as dealing with multiple asynchronous requests in parallel are manageable without braking too much of a sweat. The sample is written using Visual Studio 11 Beta but can be modified to run on Visual Studio 2010 as well.
Scenario
In the scenario we have an ApiController that exposes a query operation with a single token (in the sequence diagram below the sample query token is “microsoft”). The ApiController then turns around and issues a query to digg and delicious respectively for stories on the query token. When both results are complete, the results are accumulated into a single response that is then sent back to the client. The goal is to do the entire ApiController request asynchronously and to issue the queries to digg and delicious in parallel so that we don’t every block a thread and optimize network utilization.
The response we send back to the client is a collection of “stories” that simply look like this:
1: public class Story
2: {
3: // The source (either digg or delicious)
4: public string Source { get; set; }
5:
6: // The description of the story
7: public string Description { get; set; }
8: }
Creating the ApiController
Writing the controller involves three things: two helpers for processing the requests for digg and delicious respectively and then the controller to put it all together.
Executing digg Query
First we write a helper for processing the digg query. Here we submit a request, wait for the response, and then process it as JsonValue to build a set of Story instances as defined in the scenario above. As there is no instance state involved the query processing can be done as a static method.
1: private static async Task<List<Story>> ExecuteDiggQuery(string queryToken)
2: {
3: List<Story> result = new List<Story>();
4:
5: // URI query for a basic digg query -- see https://developers.digg.com/documentation
6: string query = string.Format("https://services.digg.com/2.0/search.search?query={0}", queryToken);
7:
8: // Submit async request
9: HttpResponseMessage diggResponse = await _client.GetAsync(query);
10:
11: // Read result using JsonValue and process the stories
12: if (diggResponse.IsSuccessStatusCode)
13: {
14: JsonValue diggResult = await diggResponse.Content.ReadAsAsync<JsonValue>();
15: foreach (var story in diggResult["stories"] as JsonArray)
16: {
17: result.Add(new Story
18: {
19: Source = "digg",
20: Description = story["title"].ReadAs<string>()
21: });
22: }
23: }
24:
25: return result;
26: }
Executing delicious Query
Then we write a similar helper for processing the delicious query. It follows the exact same pattern and also uses JsonValue to read the response and create a set of Story instances. Again, as there is no instance state involved the query processing can be done as a static method.
1: private static async Task<List<Story>> ExecuteDeliciousQuery(string queryToken)
2: {
3: List<Story> result = new List<Story>();
4:
5: // URI query for a basic delicious query -- see https://delicious.com/developers
6: string query = string.Format("https://feeds.delicious.com/v2/json/tag/{0}", queryToken);
7:
8: // Submit async request
9: HttpResponseMessage deliciousResponse = await _client.GetAsync(query);
10:
11: // Read result using JsonValue and process the stories
12: if (deliciousResponse.IsSuccessStatusCode)
13: {
14: JsonArray deliciousResult = await deliciousResponse.Content.ReadAsAsync<JsonArray>();
15: foreach (var story in deliciousResult)
16: {
17: result.Add(new Story
18: {
19: Source = "delicious",
20: Description = story["d"].ReadAs<string>()
21: });
22: }
23: }
24:
25: return result;
26: }
Writing the Controller
Now we can write the actual ApiController. First we look for a valid query token (we don’t want it to contain any ‘&’ characters). Then we kick off the two helpers in parallel and wait for them to complete (but without blocking a thread using the Task.WhenAll construct). Finally we aggregate the two results and return them to the client.
We of course use HttpClient to submit requests to dig and delicious but as HttpClient can handler requests submitted from multiple threads simultaneously we only need one instance for all requests. This means that the same HttpClient instance is reused for all requests across all ApiController instances.
1: public async Task<List<Story>> GetContent(string topic)
2: {
3: List<Story> result = new List<Story>();
4:
5: // Check that we have a topic or return empty list
6: if (topic == null)
7: {
8: return result;
9: }
10:
11: // Isolate topic to ensure we have a single term
12: string queryToken = topic.Split(new char[] { '&' }).FirstOrDefault();
13:
14: // Submit async query requests and process responses in parallel
15: List<Story>[] queryResults = await Task.WhenAll(
16: ExecuteDiggQuery(queryToken),
17: ExecuteDeliciousQuery(queryToken));
18:
19: // Aggregate results from digg and delicious
20: foreach (List<Story> queryResult in queryResults)
21: {
22: result.AddRange(queryResult);
23: }
24:
25: return result;
26: }
That’s all we need for the controller – next is just to host it and then run it.
Hosting the Controller
As usual we use a simple command line program for hosting the ApiController and it follows the usual pattern seen in the other blogs:
1: static void Main(string[] args)
2: {
3: var baseAddress = "https://localhost:8080/";
4: HttpSelfHostServer server = null;
5:
6: try
7: {
8: // Create configuration
9: var config = new HttpSelfHostConfiguration(baseAddress);
10:
11: // Add a route
12: config.Routes.MapHttpRoute(
13: name: "default",
14: routeTemplate: "api/{controller}/{id}",
15: defaults: new { controller = "Home", id = RouteParameter.Optional });
16:
17: // Create server
18: server = new HttpSelfHostServer(config);
19:
20: // Start server
21: server.OpenAsync().Wait();
22:
23: Console.WriteLine("Hit ENTER to exit");
24: Console.ReadLine();
25: }
26: finally
27: {
28: if (server != null)
29: {
30: server.CloseAsync().Wait();
31: }
32: }
33: }
Trying it Out
To try it out it is useful to use Fiddler as it allows you to see both the incoming and outgoing requests. In the screen capture below we use fiddler to directly compose a request to our localhost ApiController. To the left we can see the two requests to digg and delicious respectively and in the right lower corner you can see the result with parts coming from digg and parts coming from delicious.
Have fun!
Henrik
del.icio.us Tags: asp.net,webapi,mvc,rest,http,httpclient
Comments
Anonymous
March 03, 2012
It seems the generic ReadAsAsync() method is not present in the .Net 4.5 Beta version of System.Net.Http. Why is there a difference between that and the version in ASP.NET Web API?Anonymous
March 04, 2012
It is part of the System.Net.Http.Formatting packge which is also available on NuGet, see nuget.org/.../System.Net.Http.FormattingAnonymous
March 04, 2012
You can see what NuGet packages are part of ASP.NET Web API on the blog "ASP.NET Web API and HttpClient Available on NuGet" at blogs.msdn.com/.../asp-net-web-api-and-httpclient-available-on-nuget.aspxAnonymous
March 21, 2012
Hi Henrik. My question is: the async operations (above) are using ASP.Net worker threads ,IOCP or other threads? and what is the difference with the AsyncController? Thank you in advance.Anonymous
March 26, 2012
How does it dispatch to the GetContent method of the controller?Anonymous
April 14, 2012
Do I need to do anything to host this controller in IIS 7? What is the purpose of server.OpenAsync().Wait(); ?Anonymous
April 15, 2012
Stelios, By default threads used by Task-oriented programming comes out of the threadpool.The ApiController is different from the AsyncController in that the former is used to write Web APIs and the latter is used to write MVC controllers. The ApiController supports Task as a first-class concept so it doesn't need an "async" equivalent -- it already is async. For more information on ASP.NET Web API you can go to http://www.asp.net/web-api Thanks, Henrik