Large Request Payload Support in WinHttp for Windows Vista

  • Article

If you have used WinHttp to upload data to a server, you probably noticed that the total payload length parameter (dwTotalLength in WinHttpSendRequest) is of type DWORD, which is a 32 bit unsigned number. This limits the WinHttp apps to uploading 4GB of data, which could become a limitation in certain scenarios.
A natural way to work around this limitation is to try to "chunked" encode the request payload, but there was a problem with WinHttp always appending a Content-Length header even if the app specifies a Transfer-Encoding: Chunked header, which causes many servers to reject the request.

Support > 4GB uploads using "Content-Length" in Windows Vista

In Windows Vista, WinHttp is getting an update in this category. First, applications can specify a Content-Length > 4GB. To do that, they have to manually append a Content-Length header with the desired value. WinHttp will monitor for this header and if it is present and its value is bigger than 2^32 it will use it as an indicator of the total request payload size and will ignore the dwTotalLength parameter. WinHttp will internally store the header value in a ULONGLONG, which is a 64 bit unsigned variable.

Support for "chunked" encoding the request payload in Windows Vista

Another WinHttp update for Windows Vista is that applications now can chunk encode the request data. To do that applications have to specify a non- "identity" "Transfer-Encoding" header (say "Transfer-Encoding: Chunked"), in which case WinHttp ignores the dwTotalLength parameter of WinHttpSendRequest API. In this mode of operation applications must not specify any optional data via WinHttpSendRequest call; attempting to do so will result in ERROR_WINHTTP_INVALID_PARAMETER. All entity body posting must be done via WinHttpWriteData API. Applications should also generate the last zero length chunk and send it via WinHttpWriteData API to terminate the upload operation since WinHttp is oblivious to how you are sending the entity body data. Calling WinHttpReceiveResponse tells WinHttp that you are done sending data. For more about chunk encoding please read the corresponding section in the HTTP/1.1 RFC.

There are a couple important notes for both styles of usage. First, applications need to be prepared to handle the ERROR_WINHTTP_RESEND_REQUEST return code in case WinHttp needs to resend the request (due to redirects and / or auth challenges). Since you will be using multiple WinHttpSendRequest and WinHttpWriteData calls, WinHttp won't be able to resend the entire request on your behalf like you might be used to when only calling WinHttpSendRequest. You should also be careful that WinHttpReceiveResponse can also pass back the resend return code. Second, in the case a POST request is redirected to a GET request (e.g. 302 response), WinHttp will automatically remove or reset the application specified "Transfer-Encoding" and/or Content-Length headers.

As always, we want your feedback. Please let us know what you think about this new functionality and whether you want to see some code samples on how to upload large payloads using WinHttp.

-Nesho Neshev

Comments

  • Anonymous
    August 10, 2006
    Glad to see support for transfer-encoding: chunked. For those who must use older winhttp revs to get this functionality, an Aug 22 2005 post by Stephen Sulzer: http://groups.google.com/group/microsoft.public.winhttp/browse_thread/thread/24600c14ae603ccf/49ddbd64e8c355ef?lnk=gst&q=chunked&rnum=2#49ddbd64e8c355ef documents a valid workaround. In addition to Stephen's instructions, if you're in async mode you need to set your completion event to end WinHttpSendRequest upon receipt of WINHTTP_CALLBACK_STATUS_INTERMEDIATE_RESPONSE, then call WinHttpWriteDate repeatedly, and then call WinHttpReceiveResponse.

  • Anonymous
    August 18, 2009
    In case anyone reads this, Fc2 PC uses winhttp 5.1 and to support transfer-encoding : chunked uploads on Windows 7 we had to set the chunked transfer header before AND after the sendrequest in the callback as per above (for compatibility across both XP and Win 7 using only one executable, i.e. no IFDEFs). AFAIK Vista worked by only adding the chunked header once in the callback, after the sendrequest was made. But I could be mistaken about this. In any case, this blog made chunked uploads possible using winhttp on XP. Thanks! The sample winhttp code for the autodetect checkmark in IE seems to crash in win7 now, though. It worked fine in XP. (we disabled it because there were huge delays on each url checking the .pac file)

  • Anonymous
    August 19, 2009
    Can you please give a pointer to sample code that you are having problems with? Thanks!