Simulated Covariance for .NET Generics

  • Article

I just wrote this pattern, but I am not sure if I should add it officially to the Framework Design Guidelines. It seems like a bit of a corner case scenario, though I do get questions about it from time to time. Anyway, let me know what you think. 

 

Different constructed types don’t have a common root type. For example, there would not be a common representation of IEnumerable<string> and IEnumerable<object> if not for a pattern implemented by IEnumerable<T> called Simulated Covariance. This post describes the details of the pattern.

Generics is a very powerful type system feature added to the .NET Framework 2.0. It allows creation of so called parameterized types. For example, List<T> is such a type and it represents a list of objects of type T. The T is specified at the time when the instance of the list is created.

List<string> names = new List<string>();

names.Add(“John Smith”);

names.Add(“Mary Johnson”);

 

Such Generic data structures have many benefits over their non-Generic counterparts. But they also have some, sometimes surprising, limitations. For example, some users expect that a List<string> can be cast to List<object>, just as a String can be cast to Object. But unfortunately, the following code won’t even compile.

List<string> names = new List<string>();

List<object> objects = names; // this won’t compile

There is a very good reason for this limitation, and that is to allow for full strong typing. For example, if you could cast List<string> to a List<object> the following incorrect code would compile, but the program would fail at runtime.

static void Main(){

List<string> names = new List<string>();

// this of course does not compile, but if it did

// the whole program would compile, but would be incorrect as it

// attempts to add arbitrary objects to a list of strings.

AddObjects((List<object>)names);

string name = names[0]; // how could this work?

}

// this would (and does) compile just fine.

static void AddObjects(List<object> list){

   list.Add(new object()); // it’s a list of strings, really. Should we throw?

   list.Add(new Button());

}

Unfortunately this limitation can also be undesired in some scenarios. For example, there is nothing wrong with casting a List<string> to IEnumerable<object>, like in the following example.

List<string> names = new List<string>();

IEnumerable<object> objects = names; // this won’t compile

foreach(object obj in objects){

   Console.WriteLine(obj.ToString());

}

In general, having a way to represent “any list” (or in general “any instance of this generic type”) is very useful.

// what type should ??? be?

static void PrintItems(??? anyList){

   foreach(object obj in anyList){

   Console.WriteLine(obj.ToString());

   }

}

Unfortunately, unless List<T> implemented a pattern that will be described in a moment, the only common representation of all List<T> instances would be System.Object. But System.Object is too limiting and would not allow PrintItems method to enumerate items in the list.

The reason that casting to IEnumerable<object> is just fine, but casting to List<object> can cause all sorts of problems is that in case of IEnumerable<object>, the object appears only in the output position (the return type of GetEnumerator is IEnumerator<object>). In case of List<object>, the object represents both output and input types. For example, object is the type of the input to the Add method.

// T does not appear as input to any members or dependencies of this interface

public interface IEnumerable<T> {

   IEnumerator<T> GetEnumerator();

}

public interface IEnumerator<T> {

   T Current { get; }

   bool MoveNext();

}

// T does appear as input to members of List<T>

public class List<T> {

   public void Add(T item); // T is an input here

   public T this[int index]{

       get;

       set; // T is actually an input here

}

}

In other words, we say that in IEnumerable<T>, the T is at covariant positions (outputs). In List<T>, the T is at covariant and contravariant (inputs) positions.

To solve the problem of not having a common type representing the root of all constructions of a generic type, you can implement what’s called the Simulated Covariance Pattern.

Given a generic type (class or interface) and its dependencies

public class Foo<T> {

   public T Property1 { get; }

   public T Property2 { set; }

   public T Property3 { get; set; }

   public void Method1(T arg1);

public T Method2();

   public T Method3(T arg);

   public Type1<T> GetMethod1();

public Type2<T> GetMethod2();

}

public class Type1<T> {

   public T Property { get; }

}

public class Type2<T> {

   public T Property { get; set; }

}

Create a new interface (Root Type) with all members containing Ts at contravariant positions removed. In addition, feel free to remove all members that might not make sense in the context of the trimmed down type.

 

public interface IFoo<T> {

    T Property1 { get; }

    T Property3 { get; } // setter removed

    T Method2();

    Type1<T> GetMethod1();

    IType2<T> GetMethod2(); // note that the return type changed

}

public interface IType2<T> {

    T Property { get; } // setter removed

}

The generic type should then implement the interface explicitly and “add back” the strongly typed members (using T instead of object) to its public API surface.

 

public class Foo<T> : IFoo<object> {

    public T Property1 { get; }

    public T Property2 { set; }

    public T Property3 { get; set;}

    public void Method1(T arg1);

    public T Method2();

    public T Method3(T arg);

    public Type1<T> GetMethod1();

    public Type2<T> GetMethod2();

    object IFoo<object>.Property1 { get; }

    object IFoo<object>.Property3 { get; }

    object IFoo<object>.Method2() { return null; }

    Type1<object> IFoo<object>.GetMethod1();

    IType2<object> IFoo<object>.GetMethod2();

}

public class Type2<T> : IType2<object> {

    public T Property { get; set; }

    object IType2<object>.Property { get; }

}

Now, all constructed instantiation of Foo<T> have a common root type IFoo<object>.

 

var foos = new List<IFoo<object>>();

foos.Add(new Foo<int>());

foos.Add(new Foo<string>());

…

foreach(IFoo<object> foo in foos){

   Console.WriteLine(foo.Property1);

   Console.WriteLine(foo.GetMethod2().Property);

}

þ CONSIDER using the Simulated Covariance Pattern, if there is a need to have a representation for all instantiations of a generic type.

The pattern should not be used frivolously as it results in additional types in the library and can makes the existing types more complex.

 

þ DO ensure that the implementation of the root’s members is equivalent to the implementation of the corresponding generic type members.

There should not be an observable difference between calling a member on the root type and calling the corresponding member on the generic type. In many cases the members of the root are implemented by calling members on the generic type.

public class Foo<T> : IFoo<object> {

   

   public T Property3 { get { ... } set { ... } }

   object IFoo<object>.Property3 { get { return Property3; } }

…

}

þ CONSIDER using an abstract class instead of an interface to represent the root.

This might sometimes be a better option as interfaces are more difficult to evolve (see section X). On the other hand there are some problem with using abstract classes for the root. Abstract class members cannot be implemented explicitly and the subtypes need to use the new modifier. This makes it tricky to implement the root’s members by delegating to the generic type members.

þ CONSIDER using a non-generic root type, if such type is already available.

For example, List<T> implements IEnumerable for the purpose of simulating covariance.

Comments

  • Anonymous
    April 02, 2008
    Interesting, thanks! One typo: // this of course does not complie                                     ^^^^^ Also, the article says that "in case of IEnumerable<object>, the object appears only in the output position" and then "in IEnumerable<T>, the T is at covariant positions (inputs). In List<T>, the T is at covariant and contravariant (output) positions." It appears as though inputs and outputs were reversed.

  • Anonymous
    April 02, 2008
    I have encountered this situation in the past, and look forward to having your perspective to inform my design in the future. Thanks for sharing!

  • Anonymous
    April 02, 2008
    It's a good idea to document this pattern in the FxDG, as I independently reinvented this pattern (poorly) while helping someone on IRC recently... What should also be mentioned, perhaps in a sidebar, are ways to avoid this.  In particular, the C# Annotated Standard section 25.7 has a "Using constraints to simulate Java wildcard types" sidebar, ways to (ab)use the C# constraint mechanism to support collections of types rooted at some base type. Thus, if PrintItems() can be made generic, it can be written as:  static void PrintItems<T> (IEnumerable<T> anyList)  {    foreach (object obj in anyList)      Console.WriteLine (obj.ToString ());  } This suffers in that we must use generic methods to do this (compare to your original example where PrintItems() wasn't generic), but this is something that should be more widely known, and documenting it in the FxDG would be useful. See also: http://www.jprl.com/Blog/archive/development/2007/Aug-31.html#jcs-csharp-wildcards

  • Jon

  • Anonymous
    April 02, 2008
    Bill, thanks for pointing out the errors. I updated the text.

  • Anonymous
    April 02, 2008
    Jon, great point! I should have included it in the guideline. But, in addition of some perf overhead related to JITing separate methods for each T, you cannot use the trick for properties, ... and databinging APIs (properties) actually do need a single root.

  • Anonymous
    April 02, 2008
    It looks good. However, the .NET guideline doc is getting longer and longer. If it is too long and takes too much time for everyone to understand, it will not be so practical. I like the way C# enforce many usage by the language itself, instead of having to remember tons of patterns do the basic things right like in Java. My wish is that you great guys can solve that in the compiler one day. But for now, the pattern is a good walkaround.

  • Anonymous
    April 02, 2008
    Looks like a useful pattern, but your particular motivating example is slightly shot down by your own final point When you say in the example "// what type should ??? be?", the answer is given by "CONSIDER using a non-generic root type, if such type is already available. For example, List<T> implements IEnumerable for the purpose of simulating covariance." ??? should be IEnumerable. End of story in the context of that example.

  • Anonymous
    April 02, 2008
    "CONSIDER using a non-generic root type, if such type is already available." Why would you restrict this recommendation to available non-generic interfaces?  IFoo<Object> doesn't look right to me.  Using Object as a type parameter clashes with the concept of .NET generics since every type is always an Object anyway. Using <Object> would make sense if you could create a universally compatible "base type" from the existing generic type, but as you point out that's not possible -- you have to define a new interface anyway.  And once you do that, why not drop the useless type argument altogether and just say IFoo? That's how IEnumerable works for generic collections, and keeping that pattern seems preferable IMO.

  • Anonymous
    April 02, 2008
    Thanks for sharing - this is a great pattern which I have already found useful. Would have been a great addition to the book.

  • Anonymous
    April 03, 2008
    The comment has been removed

  • Anonymous
    April 03, 2008
    Definitely worthy for the book!!!

  • Anonymous
    April 04, 2008
    Larry, thanks for the feedback. I will try to use another example. I agree the list root might be confusing. But, it is a real propblem that we talked at one of the BCL meetings dseveral years ago and decided to have IEnumerable<T> extend IEnumerable (which was not the design early on). If we did not make the decision, you would not be able to use IEnumerable as the root for all collections.

  • Anonymous
    April 04, 2008
    Chris, the reason I think generic roots are preferable to non-generic roots (unless a non-generic root already exists) is that the CLI standard (and the current CLR implementation) already support variance annotations. If/Once main .NET languages start to support it, IFoo<T> will be much more useful than IFoo.

  • Anonymous
    April 06, 2008
    The comment has been removed

  • Anonymous
    April 07, 2008
    Thanks Nick! I fixed it.

  • Anonymous
    April 17, 2008
    Did you just write this pattern b/c you just happened to think it up recently, or is this a pattern that's used throughout the framework or other substantial projects that you want to officially codify by adding it to the design guidelines?  In the case of the former, I'd prefer to keep the guidelines free of ad-hoc contributions that, while interesting, have not yet stood the test of time (that's makes it suitable for a blog entry but not for an official publication); in the case of the latter -- seems like it would be a decent addition -- particularly if you could cite examples of places/types in the framework or other major projects that implement the pattern.

  • Anonymous
    May 13, 2008
    Just lately there seems to be a veritable feast of new blog stuff about VSX (if you don't mind me switching

  • Anonymous
    May 23, 2008
    Krzysztof, it's an excellent point, about the potential harm of generalizing a modifiable collection. However, if it were possible to generalize at (RO) interface level, that would go a ways toward supporting the "program-to-interface" practice. Here's what I mean: public interface IDisplayable {    string Text { get; }    string Data { get; } } public class Base : IDisplayable {    public string Text { get { return Name; } }    public string Data { get { return Id.ToString(); } }    public string Name { get; set; }    public Guid Id { get; set; } } class Sub : Base {    ... } public IList<IDisplayable> GetData() {    return [...].ToList<Base>(); } or: public void FillGrid(IList<Base> source); public void ShowData() {    FillGrid([...].ToList<Sub>()); } In the first example, we have a syntactically RO interface -- no chance of danger at all! The second example, admittedly, is iffier: from the reading of it, we assume that there's no updates to the list, though nothing prevents us from sneaking in some silliness. Perhaps, the second example ought to be handled with an RO interface, as well. In any case, my point is that, if we can assure the RO quality of the generalized collection, the language should be happy to support our desire to generalize, no? Cheers, Victor.

  • Anonymous
    July 10, 2010
    How can I sum two generic types together? public T func<T>(T num1,T num2) { return num1 + num2; }

  • Anonymous
    June 16, 2011
    Mahmound, there is not a clean solution to your scenario today. We have attempted to solve it in a couple of recent Framework releases and it turned out to be more difficult than it appears on the surface. The best workaround today, is to: a) create an abstraction Calculator<T> with members for basic math operations, b) make your Func<T> take these 2 number and an instance of Calculator<T>.