Push and Pull Streams using HttpClient

One of the common discussions to have around network APIs is whether streams should be push or pull. That is, do you push content to the network, or does the infrastructure pull it from you and copy it to the network? In general there is no “right” way of determining this, it is a matter of taste and what fits more naturally in the object model in question.

HttpContent is the common abstraction for any kind of content in HttpClient regardless of whether it be as part of an HttpRequestMessage or an HttpResponseMessage. Luckily it supports both patterns.

The pull pattern is supported by StreamContent which is one of the default implementations of HttpContent. Here the input stream is read (pulled) by the infrastructure and copied to the network. This works great in many cases such as file streams and the like where the content easily can be read.

To support a push model we similarly use a special HttpContent implementation. There isn’t one in the current bits but we can easily create one to support push. The trick is that it doesn’t take an input stream but rather an Action<Stream> as input.  When it is time the Action delegate is called so that it can write whatever it wants to the output stream and so there you have a push model.

    1: public class ActionOfStreamContent : HttpContent
    2: {
    3:     private readonly Action<Stream> _actionOfStream;
    4:  
    5:     public ActionOfStreamContent(Action<Stream> actionOfStream)
    6:     {
    7:         if (actionOfStream == null)
    8:         {
    9:             throw new ArgumentNullException("actionOfStream");
   10:         }
   11:  
   12:         _actionOfStream = actionOfStream;
   13:     }
   14:  
   15:     protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
   16:     {
   17:         return Task.Factory.StartNew(
   18:             (obj) =>
   19:             {
   20:                 Stream target = obj as Stream;
   21:                 _actionOfStream(target);
   22:             },
   23:             stream);
   24:     }
   25:  
   26:     protected override bool TryComputeLength(out long length)
   27:     {
   28:         // We can't know how much the Action<Stream> is going to write
   29:         length = -1;
   30:         return false;
   31:     }
   32: }

Once we have the ActionOfStream content then we can use it to write content that requires a push model such as XDocument.Save which writes content directly to an output stream with no support for a pull model:

    1: static void Main(string[] args)
    2: {
    3:     // Create an HttpClient instance
    4:     HttpClient client = new HttpClient();
    5:  
    6:     // Send a request asynchronously continue when complete
    7:     XDocument xDoc = XDocument.Load("Sample.xml", LoadOptions.None);
    8:     ActionOfStreamContent content = new ActionOfStreamContent((stream) => xDoc.Save(stream));
    9:  
   10:     client.PostAsync(_address, content).ContinueWith(
   11:         (requestTask) =>
   12:         {
   13:             // Get HTTP response from completed task.
   14:             HttpResponseMessage response = requestTask.Result;
   15:  
   16:             // Check that response was successful or throw exception
   17:             response.EnsureSuccessStatusCode();
   18:  
   19:         });
   20:  
   21:     Console.WriteLine("Hit ENTER to exit...");
   22:     Console.ReadLine();
   23: }

Have fun!

Henrik

del.icio.us Tags: asp.net,webapi,mvc,web,rest,httpclient

Comments

• Anonymous
  February 26, 2012
  Alletiders, lige hvad jeg har været på jagt efter et stykke tid....

• Anonymous
  February 27, 2012
  I tried posting an alternative of this that implemented a "firehose" approach, aka a streaming API, where new chunks could be added until the connection was explicitly closed, but it seems that the spam filtering ate it. If I would like to share this code, where should I post this? Is there any official guidance as how to best achieve this? Will there be code forthcoming? My basic approach was this: I saved the stream in SerializeToStreamAsync, returning a task from a TaskCompletionSource and having as many Action<Stream> as you'd like copy what they want onto the stream. When I was done (or as soon as any of the stream actions hit an exception, or when a passed-in cancellation token was cancelled), I set the result or exception of the TaskCompletionSource task, letting the infrastructure know that the long-lived request had ended.

• Anonymous
  March 26, 2012
  JesparEC brings up a good concept - how would we use web API to have a firehose approach where we can stream say a set of DataRow's or file chunks? For example, if you could provide some detail on how to open a file, read in each chunk from the file (say 1MB of bytes at a time), manipulate that chunk (be it custom serialization, compression or anything else) and then writing that out to the HTTP Response body. There are examples doing this for WCF REST (i.e. using the WCF REST Starter Kit using methods that return Streams and using the AdapterStream async call) but reading your blogs and the other Web API tutorials, it is not apparent to me how to achieve the same thing in Web API.

• Anonymous
  June 06, 2012
  Does this code need to be updated for the Web API RC?