Oops, we did it again

  • Article

A new update to LINQ preview is available at MSDN site.

What's the coolest new feature?  IMHO, its IQueryable<T>. 

 DLINQ's query mechanism has been generalized and available for all to use as part of System.Query.  It implements the Standard Query Operators for you using expression nodes to represent the query. Your queries can now be truly polymorphic, written over a common abstraction and translated into the target environment only when you need it to.

  public int CustomersInLondon(IQueryable<Customer> customers) {

        int count = (from c in customers

                     where c.City == "London"

                     select c).Count();

        return count;

    }

Now you can define a function like this and it can operate on either an in memory collection or a remote DLINQ collection (or you own IQueryable for that matter.)  The query is then either run entirely locally or remotely depending on the target. 

If its a DLINQ query a count query is sent to the database.

SELECT COUNT(*) AS [value]

FROM [Customers] AS [t0]

WHERE [t0].[City] = @p0

If its a normal CLR collection, the query is executed locally, using the System.Query.Sequence classes definitions of the standard query operators.  All you need to do is turn your IEnumerable<Customer> into IQueryable<Customer>.  This is accomplished easily with a built-in ToQueryable() method.

  List<Customer> customers = ...;

  CustomersInLondon(customers.ToQueryable());

Wow!  That was easy.  But, how is this done?  How can you possible turn my List<T> into some queryable thingamabob?

Good question.  Glad you asked.

Check out this little gem: 

  Expression<Func<Customer,bool>> predicate = c => c.City == "London";

  Func<Customer,bool> d = predicate.Compile();

 

Now you can compile lambda expressions directly into IL at runtime!

ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly, the operators rebound to refer to System.Query.Sequence, and the resulting code is compiled using the built-in expression compiler.  That code is then invoked producing your results.

Amazing, but true.

But wait, there is more!  Yes, it is true, not only can you slice, but you can also dice!

That's not the end of IQueryable<T>'s little bag of tricks.  What else could there be? How about dynamic queries, ones you can build up at runtime? Yes? No?  Say it isn't so!

It's true.  IQueryable<T> is fully dynamic.  Sort of.  Each IQueryable<T> has a method CreateQuery<S>() that creates a new IQueryable<S> (of the same concrete type) with a single Expression tree argument.  That is, if I have an expression tree representing the code that defines a query, I can make an IQueryable<S> out of an old IQueryable<T>.

That's great you say, but you probably don't statically know what the 'T' and 'S' are in your program if you are building dynamic queries at runtime.  True, true, we've thought of that too. You see, IQueryable<T> has a smaller cousin, IQueryable, just like IEnumerable<T> has its little friend IEnumerable.  IQueryable is non-generic, but its ever so much a query as IQueryable<T>.  In fact, it has its own CreateQuery() method that builds a new IQueryable so you can do all this query building without all that static info getting in your way.

So how do you build the expression trees that make up a query.  Check out the System.Query.QueryExpression class.  It is a collection of static methods that correspond to the standard query operator pattern, but are more loosely typed, taking arguments that are Expression's instead of Expression<T>'s.  These methods do all the work necessary to build up the call nodes that represent calls to the more fully typed IQueryable<T> query operator methods. 

Here's how you can dynamic build up the query in the prior example.

IQueryable q = ...;

ParameterExpression p = Expression.Parameter(typeof(Customer), "c");

Expression body = Expression.EQ(

    Expression.Property(p, typeof(Customer).GetProperty("City")),

    Expression.Constant("London")

    );

LambdaExpression predicate = QueryExpression.Lambda(body, p);

Expression where = QueryExpression.Where(q.Expression, predicate);

q = q.CreateQuery(where);

 

You can then call GetEnumerator() on 'q' to execute the query.

Too hard you say?  Do I have to write all that code?  What if I just want my users to type in a little filter expression.  Do I have to turn that into all those nodes?

Ah, well, we've thought of that too.  Try this out. 

IQueryable q = ...;

ParameterExpression p = Expression.Parameter(typeof(Customer), "c");

LambdaExpression predicate =
QueryExpression.Lambda("c.City = 'London'", p);

Expression where = QueryExpression.Where(q.Expression, predicate);

q = q.CreateQuery(where);

Or if you don't want them to have to type silly dot's, just don't give the parameter a name.

IQueryable q = ...;

ParameterExpression p = Expression.Parameter(typeof(Customer), "");

LambdaExpression predicate =
QueryExpression.Lambda("City = 'London'", p);

Expression where = QueryExpression.Where(q.Expression, predicate);

q = q.CreateQuery(where);

 

Viola!  Instant filter. Instant dynamic query. Run it here. Run it there. Why, you can run it everywhere.

But that's enough from me. Why are you wasting time reading this when you could be downloading the preview and trying it out for yourself?

Matt

Comments

  • Anonymous
    May 10, 2006
    Matt,

    this stuff is so cool. linq rocks...

    WM_THX
    thomas woelfer

  • Anonymous
    May 10, 2006
    OMG! This is so cool. I don't know what it is, but it is so cool.  

  • Anonymous
    May 10, 2006
    Show me how that last one can be parameterized to avoid injection issues...

  • Anonymous
    May 10, 2006
    You wouldn't use that text based filter if you were assembling user input with other pieces, you'd use the node construction pattern. The text based filter would only be used if the entire predicate is supplied by the user.  The user cannot cause injection because this expression is parsed, turned into expression nodes that must statically check as a correct predicate. DLINQ automatically converts the user supplied literal into an ADO.Net parameter, so there is no injection on that end either.

  • Anonymous
    May 11, 2006
    I mentioned this in my last blog entry, but thought I should make it more discoverable. We've released...

  • Anonymous
    May 11, 2006
    I mentioned this in my last blog entry, but thought I should make it more discoverable. We've released...

  • Anonymous
    May 11, 2006
    A new LINQ CTP is available. Looking for a reason to install it? See&amp;nbsp;this.

  • Anonymous
    May 11, 2006
    The comment has been removed

  • Anonymous
    May 11, 2006
    It would seem that you don't need IEnumerable<T> operators at all anymore.  However, you still need them as actual methods to invoke when the IQueryable<T> is executed locally, and its has additional overhead due to the expression tree representation and the runtime compilation.

  • Anonymous
    May 11, 2006

    There's an update to the LINQ extensions for the .NET framework available on MSDN. Check it out on...

  • Anonymous
    May 12, 2006
    See (scroll down to the bottom for better samples)

    http://www.entityspaces.net/portal/Documentation/QueryAPISamples/tabid/80/Default.aspx

    Anyway, LINQ looks very cool Matt, what level are you involved with it?  

  • Anonymous
    May 12, 2006
    AggregateTestCollection aggTestColl = new AggregateTestCollection();
    aggTestColl.Query.es.CountAll = true;
    aggTestColl.Query
       .Select (aggTestColl.Query.IsActive,
                aggTestColl.Query.DepartmentID)
       .Where  (aggTestColl.Query.IsActive.Equal(true))
       .GroupBy(aggTestColl.Query.IsActive,
                aggTestColl.Query.DepartmentID)
       .OrderBy(aggTestColl.Query.DepartmentID.Ascending,             aggTestColl.Query.IsActive.Ascending);
    aggTestColl.Query.es.WithRollup = true;
    aggTestColl.Query.Load();

    Yields
    ==============
    SELECT [IsActive],[DepartmentID] ,COUNT(*) AS 'Count' FROM [AggregateTest]
    WHERE ([IsActive] = @IsActive1 )
    GROUP BY [IsActive],[DepartmentID] WITH ROLLUP
    ORDER BY [DepartmentID] ASC,[IsActive] ASC

    Same binary code runs on all databases.

  • Anonymous
    May 12, 2006
    Mike, you could certainly add LINQ to your product. It would be relatively simple to translate IQueryable<T> expression trees into your query API, so it would simply bolt on top of what you have now.

    By implementing IQueryable<AggregateTest> on your AggregateTestCollection class your users could write LINQ queries.

  • Anonymous
    May 12, 2006
    That's exactly what we were thinking. We provide a whole other superset of functionality but thought exactly as you suggested, implement the IQueryable<> syntax. We're releasing 1.4 this weekend which adds MySQL and some other cools stuff. Then 1.5 adds full hierarchical support. Of course we generate it all from your db schema. What's cool is that our NUnit test suite runs the exact same binary code against Oracle, Access, Microosft SQL, and MySQL the only difference being a connection string (of course, you have to use a schema that works in all databases but it's very forgiving). Anyway, we're thinking along the same lines ...

  • Anonymous
    May 12, 2006
    Добавлены новые возможности: Поддержка запросов времени выполнения, соединений т

  • Anonymous
    May 12, 2006
    One of the biggest Linq improvements from the last CTP is IQueryable&amp;lt;T&amp;gt; which allows polymorphic...

  • Anonymous
    May 14, 2006
    Qualche mese fa scrivevo qualche commento relativamente al ruolo di LINQ. Il recente rilascio di una...

  • Anonymous
    May 17, 2006
    PingBack from http://mdavey.wordpress.com/2006/05/11/linq-may-ctp/

  • Anonymous
    May 21, 2006
    After false steps of ObjectSpaces and non-existent O/R mapping tools from Microsoft, the LINQ family...

  • Anonymous
    May 21, 2006
    After false steps of ObjectSpaces and non-existent O/R mapping tools from Microsoft, the LINQ family...

  • Anonymous
    May 22, 2006
    I do a good amount of work in the  multidimentional (MDX) query space.  Will LINQ be extended for that paradigm also?

  • Anonymous
    May 31, 2006
    PingBack from <a href="http://blog.genom-e.com/default,date,2006-05-31.aspx">http://blog.genom-e.com/default,date,2006-05-31.aspx</a>

  • Anonymous
    June 02, 2006
    We are wellocme to it's configuration.

  • Anonymous
    June 04, 2006
    This is soo cool.
    We are currently implementing Linq support for our opensource mapper (NPersist)

    http://blogs.wdevs.com/phirephly/archive/2006/06/05/13420.aspx#13422

    it works lovely..
    kudos to the Linq team :-)

  • Anonymous
    June 17, 2006
    The documents on ADO.NET vNext that were previously pulled from MSDN have been republished by Microsoft.I...

  • Anonymous
    October 30, 2006
    This is the first in a series of posts on C# and LINQ. These posts will describe a natural, easy to understand

  • Anonymous
    November 22, 2006
    By Matt Duffin. So, you’re creating an application… no surprise there. As part of that application, you

  • Anonymous
    April 03, 2007
    The comment has been removed