Streaming Feeds Sample

Download sample

This sample demonstrates how to manage syndication feeds that contain large numbers of items. On the server, the sample demonstrates how to delay the creation of individual SyndicationItem objects within the feed until immediately before the item is written to the network stream.

On the client, the sample shows how a custom syndication feed formatter can be used to read individual items from the network stream so that the feed being read is never fully buffered into memory.

To best demonstrate the streaming capability of the syndication API, this sample uses a somewhat unlikely scenario in which the server exposes a feed that contains an infinite number of items. In this case, the server continues generating new items into the feed until it determines that the client has read a specified number of items from the feed (by default, 10). For simplicity, both the client and the server are implemented in the same process and use a shared ItemCounter object to keep track of how many items the client has produced. The ItemCounter type exists only for the purpose of allowing the sample scenario to terminate cleanly, and is not a core element of the pattern being demonstrated.

The demonstration makes use of Visual C# iterators (using the yield return keyword construct). For more information about iterators, see the "Using Iterators" topic on MSDN.

Service

The service implements a basic WebGetAttribute contract that consists of one operation, as shown in the following code.

[ServiceContract]
interface IStreamingFeedService
{
    [WebGet]
    [OperationContract]
    Atom10FeedFormatter StreamedFeed();
}

The service implements this contract by using an ItemGenerator class to create a potentially infinite stream of SyndicationItem instances using an iterator, as shown in the following code.

class ItemGenerator
{
    public IEnumerable<SyndicationItem> GenerateItems()
    {
        while (counter.GetCount() < maxItemsRead)
        {
            itemsReturned++;
            yield return CreateNextItem();
        }
        
    }
    ...
}

When the service implementation creates the feed, the output of ItemGenerator.GenerateItems() is used instead of a buffered collection of items.

public Atom10FeedFormatter StreamedFeed()
{
    SyndicationFeed feed = new SyndicationFeed("Streamed feed", "Feed to test streaming", null);
    //Generate an infinite stream of items. Both the client and the service share
    //a reference to the ItemCounter, which allows the sample to terminate
    //execution after the client has read 10 items from the stream
    ItemGenerator itemGenerator = new ItemGenerator(this.counter, 10);

    feed.Items = itemGenerator.GenerateItems();
    return feed.GetAtom10Formatter();
}

As a result, the item stream is never fully buffered into memory. You can observe this behavior by setting a breakpoint on the yield return statement inside of the ItemGenerator.GenerateItems() method and noting that this breakpoint is encountered for the first time after the service has returned the result of the StreamedFeed() method.

Client

The client in this sample uses a custom SyndicationFeedFormatter implementation that delays the materialization of individual items in the feed instead of buffering them into memory. The custom StreamedAtom10FeedFormatter instance is used as follows.

XmlReader reader = XmlReader.Create("https://localhost:8000/Service/Feeds/StreamedFeed");
StreamedAtom10FeedFormatter formatter = new StreamedAtom10FeedFormatter(counter);

SyndicationFeed feed = formatter.ReadFrom(reader);

Normally, a call to ReadFrom does not return until the entire contents of the feed have been read from the network and buffered into memory. However, the StreamedAtom10FeedFormatter object overrides ReadItems to return an iterator instead of a buffered collection, as shown in the following code.

protected override IEnumerable<SyndicationItem> ReadItems(XmlReader reader, SyndicationFeed feed, out bool areAllItemsRead)
{
    areAllItemsRead = false;
    return DelayReadItems(reader, feed);
}

private IEnumerable<SyndicationItem> DelayReadItems(XmlReader reader, SyndicationFeed feed)
{
    while (reader.IsStartElement("entry", "http://www.w3.org/2005/Atom"))
    {
        yield return this.ReadItem(reader, feed);
    }

    reader.ReadEndElement();
}

As a result, each item is not read from the network until the client application traversing the results of ReadItems() is ready to use it. You can observe this behavior by setting a breakpoint on the yield return statement inside of StreamedAtom10FeedFormatter.DelayReadItems() and noticing that this breakpoint is encountered for the first time after the call to ReadFrom() completes.

The following instructions show how to build and run the sample. Note that although the server stops generating items after the client has read 10 items, the output shows that the client reads significantly more than 10 items. This is because the networking binding used by the sample transmits data in four-kilobyte (KB) segments. As such, the client receives 4KB of item data before it has the opportunity to read even one item. This is normal behavior (sending streamed HTTP data in reasonably-sized segments increases performance).

To set up, build, and run the sample

1. Ensure that you have performed the One-Time Set Up Procedure for the Windows Communication Foundation Samples.

2. To build the C# or Visual Basic .NET edition of the solution, follow the instructions in Building the Windows Communication Foundation Samples.

3. To run the sample in a single- or cross-machine configuration, follow the instructions in Running the Windows Communication Foundation Samples.

See Also

Other Resources

Stand-Alone Diagnostics Feed Sample

© 2007 Microsoft Corporation. All rights reserved.