Streaming Provider (WCF Data Services)

A data service can expose large object binary data. This binary data might represent video and audio streams, images, document files, or other types of binary media. When an entity in the data model includes one or more binary properties, the data service returns this binary data encoded as base-64 inside the entry in the response feed. Because loading and serializing large binary data in this manner can affect performance, the Open Data Protocol (OData) defines a mechanism for retrieving binary data independent of the entity to which it belongs. This is accomplished by separating the binary data from the entity into one or more data streams.

The OData protocol supports the following two mechanisms for exposing binary data as streams related to an entity:

• Media resource/media link entry

  The Atom Publishing Protocol (AtomPub) defines a mechanism to associate binary data as a media resource with an entry in a data feed, which is called the media link entry. There can only be one media resource defined for a given media link entry. A media resource can be considered as the default stream for an entity. OData inherits this streaming behavior from AtomPub.

• Named resource stream

  Starting with version 3 of OData, an entity can have multiple related resource streams, which are accessed by name. This mechanism does not depend on AtomPub, so it is possible for an entity to have named resource streams and not be a media link entry. It is also possible for a media link entry to have named streams. For more information, see the post Named Resource Streams.

With WCF Data Services, you define binary resource streams by implementing streaming data providers. The streaming provider implementations supply the data service with the streams associated with a specific entity as Stream objects.

Configuring a data service to support the streaming of binary data requires the following steps:

1. Attribute the entity in the data model that has related resource streams. Each data provider has its own requirements, and each kind of binary resource stream is defined in the data model by a different mechanism.

2. Implement the following stream provider interfaces:

   • IDataServiceStreamProvider – required to support both kinds of binary resource streams.

   • IDataServiceStreamProvider2 – required only for named resource streams.

3. Define a data service that implements the IServiceProvider interface. The data service uses the GetService implementation to access the streaming data provider implementation. This method returns the appropriate streaming provider implementation.

4. Enable large message streams in the Web application configuration.

5. Enable access to binary resources on the server or in a data source.

The examples in this topic are based on a sample streaming photo service, which is discussed in depth in the post Data Services Streaming Provider Series: Implementing a Streaming Provider (Part 1). The source code for this sample service is available on the Streaming Photo Data Service Sample page on MSDN Code Gallery.

Defining Resource Streams in the Data Model

The way in which a binary resource stream is defined in the data model depends both on whether the stream is a media resource or named resource stream, as well as the data source provider being used. The following example shows the definition of a media resource in the metadata returned by the data service, where PhotoInfo is a media link entry that also has a named resource stream (Thumbnail) defined:

<EntityType Name="PhotoInfo" m:HasStream="true">
  <Key>
    <PropertyRef Name="PhotoId" />
  </Key>
  <Property Name="PhotoId" Type="Edm.Int32" Nullable="false" 
            p9:StoreGeneratedPattern="Identity" 
            xmlns:p9="https://schemas.microsoft.com/ado/2009/02/edm/annotation" />
  <Property Name="FileName" Type="Edm.String" Nullable="false" />
  <Property Name="FileSize" Type="Edm.Int32" Nullable="true" />
  <Property Name="DateTaken" Type="Edm.DateTime" Nullable="true" />
  <Property Name="TakenBy" Type="Edm.String" Nullable="true" />
  <Property Name="DateAdded" Type="Edm.DateTime" Nullable="false" />
  <Property Name="Exposure" Type="PhotoData.Exposure" Nullable="false" />
  <Property Name="Dimensions" Type="PhotoData.Dimensions" Nullable="false" />
  <Property Name="DateModified" Type="Edm.DateTime" Nullable="false" />
  <Property Name="Comments" Type="Edm.String" Nullable="true" MaxLength="Max" 
            Unicode="true" FixedLength="false" />
  <Property Name="ContentType" Type="Edm.String" Nullable="true" MaxLength="50" 
            Unicode="true" FixedLength="false" />
  <Property Name="Thumbnail" Type="Edm.Stream" Nullable="false" />
</EntityType>

Entity Framework Provider

When using the Entity Framework provider, you define a binary resource stream in the data model itself in one of the following ways, depending on the kind of stream:

• Media resource stream:

  To indicate that an entity is a media link entry that has an associated media resource, add the HasStream attribute to the entity type definition in the conceptual model. You must also add a reference to the namespace xmlns:m=https://schemas.microsoft.com/ado/2007/08/dataservices/metadata to either the media link entry or one of its parent elements in the data model.

• Named resource stream:

  A named stream that belongs to a media link entry is defined as a property of type Stream in the conceptual model.

  Important

  The Stream data type is supported by the Entity Data Model (EDM), starting with version 2.2. However, in the .NET Framework 4 the Entity Framework does not support this version of the EDM. As a workaround, you can instead define a named resource stream in the data model by applying the NamedStreamAttribute to the class in the data model object layer that represents the entity. In this attribute, the name parameter is the name of the stream. We recommend that you add this attribute to the entity class by creating a partial class definition that is maintained in a separate code file; otherwise this customization is overwritten when the entity data classes are regenerated by the Entity Framework. As with the reflection provider, the data service generates a property of the supplied name of type Stream in the data model.

For an example of exposing a media resource by using the Entity Framework provider, see the post Data Services Streaming Provider Series: Implementing a Streaming Provider (Part 1).

Reflection Provider

When using the reflection provider, you identify that a binary resource stream belongs to an entity type by applying attributes in one of the following ways to the class that is the entity type, depending on the kind of stream:

• Media resource stream:

  Apply HasStreamAttribute to define a media resource (default) stream that belongs to the type, which is a media link entry.

• Named resource stream:

  Apply the NamedStreamAttribute to define a named resource stream that belongs the entity type, where the supplied name parameter is the name of the stream.

Custom Data Service Provider

When using custom service providers, you implement the IDataServiceMetadataProvider interface to define the metadata for your data service. For more information, see Custom Data Service Providers (WCF Data Services). You indicate that a binary resource stream belongs to a ResourceType in one of the following ways:

• Media resource stream:

  Set the IsMediaLinkEntry property to true on the ResourceType that represents the entity type, which is a media link entry.

• Named resource stream:

  Add a ResourceProperty to the ResourceType for each named stream. A ResourceProperty that represents a named stream must have a ResourcePropertyKind value of Stream and the ResourceType of the property must be based on the Stream type. Use the GetPrimitiveResourceType(Type) method on ResourceType to create a ResourceType based on the Stream primitive type.

Implementing the Stream Provider Interfaces

To create a data service that supports binary data streams, you must implement at least the IDataServiceStreamProvider interface. This implementation enables the data service to return binary data as a stream to the client and consume binary data as a stream sent from the client. To support named streams, you must also implement the IDataServiceStreamProvider2 interface. The data service creates an instance of the appropriate interface whenever it needs to access binary data as a stream.

IDataServiceStreamProvider

The IDataServiceStreamProvider interface specifies the following members:

IDataServiceStreamProvider2

The IDataServiceStreamProvider2 interface specifies the following members, which overload members of IDataServiceStreamProvider to take a ResourceProperty parameter that is a named resource stream:

Creating the Streaming Data Service

To provide the WCF Data Services runtime with access to the IDataServiceStreamProvider and IDataServiceStreamProvider2 implementations, the data service that you create must also implement the IServiceProvider interface. The following example shows how to implement the GetService method to return an instance of the PhotoServiceStreamProvider class that implements IDataServiceStreamProvider and IDataServiceStreamProvider2.

Partial Public Class PhotoData
    Inherits DataService(Of PhotoDataContainer)
    Implements IServiceProvider

    ' This method is called only once to initialize service-wide policies.
    Public Shared Sub InitializeService(ByVal config As DataServiceConfiguration)
        config.SetEntitySetAccessRule("PhotoInfo", _
            EntitySetRights.ReadMultiple Or _
            EntitySetRights.ReadSingle Or _
            EntitySetRights.AllWrite)

        ' Named streams require version 3 of the OData protocol.
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3
    End Sub
#Region "IServiceProvider Members"
    Public Function GetService(ByVal serviceType As Type) As Object _
    Implements IServiceProvider.GetService
        If serviceType Is GetType(IDataServiceStreamProvider) _
            Or serviceType Is GetType(IDataServiceStreamProvider2) Then
            Return New PhotoServiceStreamProvider(Me.CurrentDataSource)
        End If
        Return Nothing
    End Function
#End Region
End Class

public partial class PhotoData : DataService<PhotoDataContainer>, IServiceProvider
{
    // This method is called only once to initialize service-wide policies.
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("PhotoInfo",
            EntitySetRights.ReadMultiple |
            EntitySetRights.ReadSingle |
            EntitySetRights.AllWrite);

        // Named resource streams require version 3 of the OData protocol.
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3;
    }
    public object GetService(Type serviceType)
    {
        if (serviceType == typeof(IDataServiceStreamProvider2))
        {
            // Return the stream provider to the data service.
            return new PhotoServiceStreamProvider(this.CurrentDataSource);
        }

        return null;
    }
}

For general information about how to create a data service, see Configuring the Data Service (WCF Data Services).

Enabling Large Binary Streams in the Hosting Environment

When you create a data service in an ASP.NET Web application, Windows Communication Foundation (WCF) is used to provide the HTTP protocol implementation. By default, WCF limits the size of HTTP messages to only 65K bytes. To be able to stream large binary data to and from the data service, you must also configure the Web application to enable large binary files and to use streams for transfer. To do this, add the following in the <configuration /> element of the application's Web.config file:

    <system.serviceModel>
        <serviceHostingEnvironment aspNetCompatibilityEnabled="true"/>
        <services>
            <!-- The name of the service -->
            <service name="PhotoService.PhotoData">
                <!--you can leave the address blank or specify your end point URI-->
                <endpoint binding="webHttpBinding" bindingConfiguration="higherMessageSize" 
                  contract="System.Data.Services.IRequestHandler"></endpoint>
            </service>
        </services>
        <bindings>
            <webHttpBinding>
                <!-- configure the maxReceivedMessageSize value to suit the max size of 
   the request (in bytes) you want the service to receive-->
                <binding name="higherMessageSize" transferMode="Streamed"  
                 maxReceivedMessageSize="2147483647"/>
            </webHttpBinding>
        </bindings>
    </system.serviceModel>

For more information, see Streaming Message Transfer and Transport Quotas.

By default, Internet Information Services (IIS) also limits the size of requests to 4MB. To enable your data service to receive streams larger than 4MB when running on IIS, you must also set the maxRequestLength attribute of the httpRuntime Element in the <system.web /> configuration section, as shown in the following example:

 <system.web>
        <!-- maxRequestLength (in KB): default=4000 (4MB); max size=2048MB. -->
        <httpRuntime maxRequestLength="2000000"/>
  </system.web>

Using Data Streams in a Client Application

You can access a binary data stream directly by using one of the following URIs:

• Media resource stream:

  https://localhost/PhotoService/PhotoData.svc/PhotoInfo(1)/$value
  

• Named resource stream:

  https://localhost/PhotoService/PhotoData.svc/PhotoInfo(1)/Thumbnail
  

The WCF Data Services client library enables you to both retrieve and update these exposed resources as binary streams on the client. For more information, see Working with Binary Data (WCF Data Services).

Considerations for Working with a Streaming Provider

The following are things to consider when you implement a streaming provider and when you access media resources from a data service.

• You must implement the IDataServiceStreamProvider interface when supporting either media resource or named resource streams. To support named resource streams, you must also implement the IDataServiceStreamProvider2 interface.

• When you define a named resource stream that belongs to a media link entry, the entity defined in the data model should not include a property with the same name as the stream.

• MERGE requests are not supported for media resources. Use a PUT request to change the media resource of an existing entity.

• A POST request cannot be used to create a new media link entry. Instead, you must issue a POST request to create a new media resource, and the data service creates a new media link entry with default values. This new entity can be updated by a subsequent MERGE or PUT request. You may also consider caching the entity and make updates in the disposer, such as setting the property value to the value of the Slug header in the POST request.

• When a POST request is received, the data service calls GetWriteStream to create the media resource before it calls SaveChanges to create the media link entry.

• An implementation of GetWriteStream should not return a MemoryStream object. When you use this kind of stream, memory resource issues will occur when the service receives very large data streams.

• The following are things to consider when storing media resources in a database:

  • A binary property that is a media resource should not be included in the data model. All properties exposed in a data model are returned in the entry in a response feed.

  • To improve performance with a large binary stream, we recommend that you create a custom stream class to store binary data in the database. This class is returned by your GetWriteStream implementation and sends the binary data to the database in chunks. For a SQL Server database, we recommend that you use a FILESTREAM to stream data into the database when the binary data is larger than 1MB.

  • Ensure that your database is designed to store the binary large streams that are to be received by your data service.

  • When a client sends a POST request to insert a media link entry with a media resource in a single request, GetWriteStream is called to obtain the stream before the data service inserts the new entity into the database. A streaming provider implementation must be able to handle this data service behavior. Consider using a separate data table to store the binary data or store the data stream in a file until after the entity has been inserted into the database.

• When you implement the DeleteStream, GetReadStream, or GetWriteStream methods, you must use the eTag and Content-Type values that are supplied as method parameters. Do not set eTag or Content-Type headers in your IDataServiceStreamProvider provider implementation.

• By default, the client sends large binary streams by using a chunked HTTP Transfer-Encoding. Because the ASP.NET Development Server does not support this kind of encoding, you cannot use this Web server to host a streaming data service that must accept large binary streams. For more information on ASP.NET Development Server, see Web Servers in Visual Web Developer.

Versioning Requirements

The streaming provider has the following OData protocol versioning requirements:

• The streaming provider requires that the data service support version 2.0 of the OData protocol and later versions.

• Support for named streams requires that both the client and data service support version 3.0 of the OData protocol and later versions.

For more information, see Data Service Versioning (WCF Data Services).

See Also

Concepts

Data Service Providers (WCF Data Services)

Custom Data Service Providers (WCF Data Services)

Working with Binary Data (WCF Data Services)