Mutation of a predicate expression tree to specify a different type

Question

Introduction

In the application I'm working on now, there are two types of each business object: type "ActiveRecord" and view "DataContract". So, for example, it would be:

namespace ActiveRecord { class Widget { public int Id { get; set; } } } namespace DataContract { class Widget { public int Id { get; set; } } } 

The database access level provides translation between families: you can tell him about the DataContract.Widget update, and he will magically create ActiveRecord.Widget with the same property values ​​and save them.

The problem occurred while trying to reorganize this level of access to the database.

Problem

I want to add the following methods to the database access level:

 // Widget is DataContract.Widget interface IDbAccessLayer { IEnumerable<Widget> GetMany(Expression<Func<Widget, bool>> predicate); } 

The above is a simple general purpose get method with a custom predicate. The only thing interesting is that I pass an expression tree instead of a lambda, because inside IDbAccessLayer I request IQueryable<ActiveRecord.Widget> ; to do this efficiently (think about LINQ to SQL) I need to pass an expression tree so this method asks for exactly that.

Trap: the parameter must be magically converted from Expression<Func<DataContract.Widget, bool>> to Expression<Func<ActiveRecord.Widget, bool>> .

Attempt to solve

What I would like to do inside GetMany :

 IEnumerable<DataContract.Widget> GetMany( Expression<Func<DataContract.Widget, bool>> predicate) { var lambda = Expression.Lambda<Func<ActiveRecord.Widget, bool>>( predicate.Body, predicate.Parameters); // use lambda to query ActiveRecord.Widget and return some value } 

This will not work, because in a typical scenario, for example, if:

 predicate == w => w.Id == 0; 

... the expression tree contains an instance of MemberAccessExpression , which has a property of type MemberInfo that describes DataContract.Widget.Id . There are also instances of ParameterExpression both in the expression tree and in its set of parameters ( predicate.Parameters ) that describe DataContract.Widget ; all this will lead to errors, since the request body does not contain this kind of widget, but rather ActiveRecord.Widget .

After some searching, I found System.Linq.Expressions.ExpressionVisitor (its source can be found here in the context of practical use), which offers a convenient way to modify the expression tree. In .NET 4, this class is included out of the box.

Armed with this, I introduced the visitor. This simple visitor only cares about changing types in member access and parameter expressions, but this is enough to work with the predicate w => w.Id == 0 .

 internal class Visitor : ExpressionVisitor { private readonly Func<Type, Type> typeConverter; public Visitor(Func<Type, Type> typeConverter) { this.typeConverter = typeConverter; } protected override Expression VisitMember(MemberExpression node) { var dataContractType = node.Member.ReflectedType; var activeRecordType = this.typeConverter(dataContractType); var converted = Expression.MakeMemberAccess( base.Visit(node.Expression), activeRecordType.GetProperty(node.Member.Name)); return converted; } protected override Expression VisitParameter(ParameterExpression node) { var dataContractType = node.Type; var activeRecordType = this.typeConverter(dataContractType); return Expression.Parameter(activeRecordType, node.Name); } } 

With this visitor GetMany becomes:

 IEnumerable<DataContract.Widget> GetMany( Expression<Func<DataContract.Widget, bool>> predicate) { var visitor = new Visitor(...); var lambda = Expression.Lambda<Func<ActiveRecord.Widget, bool>>( visitor.Visit(predicate.Body), predicate.Parameters.Select(p => visitor.Visit(p)); var widgets = ActiveRecord.Widget.Repository().Where(lambda); // This is just for reference, see below Expression<Func<ActiveRecord.Widget, bool>> referenceLambda = w => w.Id == 0; // Here we 'd convert the widgets to instances of DataContract.Widget and // return them -- this has nothing to do with the question though. } 

results

The good news is that lambda built just fine. The bad news is that it does not work; it exploded me when I try to use it, and exception messages really don't help at all.

I studied lambda, which produces my code, and hard-coded lambda with the same expression; they look exactly the same. I spent hours in the debugger trying to find some difference, but I can’t.

When the predicate w => w.Id == 0 , lambda looks exactly like referenceLambda . But the latter works, for example, IQueryable<T>.Where , and the first does not; I tried this in the nearest debugger window.

I should also mention that when the predicate w => true , everything works fine. Therefore, I suppose that I am not doing enough work with the visitor, but I can no longer find to follow.

Final decision

After accepting the correct answers to the problem (two of them below, one short, one with code), the problem was solved; I added the code along with a few important notes in a separate answer so that this long question does not get any longer.

Thank you all for your answers and comments!

Answer 1

It seems you are generating the parameter expression twice in VisitMember () here:

 var converted = Expression.MakeMemberAccess( base.Visit(node.Expression), activeRecordType.GetProperty(node.Member.Name)); 

... since base.Visit () gets into VisParameter, I think in GetMany () itself:

 var lambda = Expression.Lambda<Func<ActiveRecord.Widget, bool>>( visitor.Visit(predicate.Body), predicate.Parameters.Select(p => visitor.Visit(p)); 

If you use the ParameterExpression expression in the body, it must be the same instance (and not just the same type and name) as the one declared for Lambda. I used to have problems with a similar scenario, although I think the result was that I just could not create an expression, it would just throw an exception. In any case, you can try reusing the instance of the parameter if that helps.

Answer 2

It turned out that the tricky part is simply that the ParameterExpression instances that exist in the expression tree of the new lambda must be the same instances that are passed in the IEnumerable<ParameterExpression> Expression.Lambda IEnumerable<ParameterExpression> .

Note that inside TransformPredicateLambda I provide t => typeof(TNewTarget) as a type converter function; that, since in this particular case we can assume that all parameters and membership calls will be of one particular type. More complex scenarios may require additional logic.

The code:

 internal class DbAccessLayer { private static Expression<Func<TNewTarget, bool>> TransformPredicateLambda<TOldTarget, TNewTarget>( Expression<Func<TOldTarget, bool>> predicate) { var lambda = (LambdaExpression) predicate; if (lambda == null) { throw new NotSupportedException(); } var mutator = new ExpressionTargetTypeMutator(t => typeof(TNewTarget)); var explorer = new ExpressionTreeExplorer(); var converted = mutator.Visit(predicate.Body); return Expression.Lambda<Func<TNewTarget, bool>>( converted, lambda.Name, lambda.TailCall, explorer.Explore(converted).OfType<ParameterExpression>()); } private class ExpressionTargetTypeMutator : ExpressionVisitor { private readonly Func<Type, Type> typeConverter; public ExpressionTargetTypeMutator(Func<Type, Type> typeConverter) { this.typeConverter = typeConverter; } protected override Expression VisitMember(MemberExpression node) { var dataContractType = node.Member.ReflectedType; var activeRecordType = this.typeConverter(dataContractType); var converted = Expression.MakeMemberAccess( base.Visit(node.Expression), activeRecordType.GetProperty(node.Member.Name)); return converted; } protected override Expression VisitParameter(ParameterExpression node) { var dataContractType = node.Type; var activeRecordType = this.typeConverter(dataContractType); return Expression.Parameter(activeRecordType, node.Name); } } } /// <summary> /// Utility class for the traversal of expression trees. /// </summary> public class ExpressionTreeExplorer { private readonly Visitor visitor = new Visitor(); /// <summary> /// Returns the enumerable collection of expressions that comprise /// the expression tree rooted at the specified node. /// </summary> /// <param name="node">The node.</param> /// <returns> /// The enumerable collection of expressions that comprise the expression tree. /// </returns> public IEnumerable<Expression> Explore(Expression node) { return this.visitor.Explore(node); } private class Visitor : ExpressionVisitor { private readonly List<Expression> expressions = new List<Expression>(); protected override Expression VisitBinary(BinaryExpression node) { this.expressions.Add(node); return base.VisitBinary(node); } protected override Expression VisitBlock(BlockExpression node) { this.expressions.Add(node); return base.VisitBlock(node); } protected override Expression VisitConditional(ConditionalExpression node) { this.expressions.Add(node); return base.VisitConditional(node); } protected override Expression VisitConstant(ConstantExpression node) { this.expressions.Add(node); return base.VisitConstant(node); } protected override Expression VisitDebugInfo(DebugInfoExpression node) { this.expressions.Add(node); return base.VisitDebugInfo(node); } protected override Expression VisitDefault(DefaultExpression node) { this.expressions.Add(node); return base.VisitDefault(node); } protected override Expression VisitDynamic(DynamicExpression node) { this.expressions.Add(node); return base.VisitDynamic(node); } protected override Expression VisitExtension(Expression node) { this.expressions.Add(node); return base.VisitExtension(node); } protected override Expression VisitGoto(GotoExpression node) { this.expressions.Add(node); return base.VisitGoto(node); } protected override Expression VisitIndex(IndexExpression node) { this.expressions.Add(node); return base.VisitIndex(node); } protected override Expression VisitInvocation(InvocationExpression node) { this.expressions.Add(node); return base.VisitInvocation(node); } protected override Expression VisitLabel(LabelExpression node) { this.expressions.Add(node); return base.VisitLabel(node); } protected override Expression VisitLambda<T>(Expression<T> node) { this.expressions.Add(node); return base.VisitLambda(node); } protected override Expression VisitListInit(ListInitExpression node) { this.expressions.Add(node); return base.VisitListInit(node); } protected override Expression VisitLoop(LoopExpression node) { this.expressions.Add(node); return base.VisitLoop(node); } protected override Expression VisitMember(MemberExpression node) { this.expressions.Add(node); return base.VisitMember(node); } protected override Expression VisitMemberInit(MemberInitExpression node) { this.expressions.Add(node); return base.VisitMemberInit(node); } protected override Expression VisitMethodCall(MethodCallExpression node) { this.expressions.Add(node); return base.VisitMethodCall(node); } protected override Expression VisitNew(NewExpression node) { this.expressions.Add(node); return base.VisitNew(node); } protected override Expression VisitNewArray(NewArrayExpression node) { this.expressions.Add(node); return base.VisitNewArray(node); } protected override Expression VisitParameter(ParameterExpression node) { this.expressions.Add(node); return base.VisitParameter(node); } protected override Expression VisitRuntimeVariables(RuntimeVariablesExpression node) { this.expressions.Add(node); return base.VisitRuntimeVariables(node); } protected override Expression VisitSwitch(SwitchExpression node) { this.expressions.Add(node); return base.VisitSwitch(node); } protected override Expression VisitTry(TryExpression node) { this.expressions.Add(node); return base.VisitTry(node); } protected override Expression VisitTypeBinary(TypeBinaryExpression node) { this.expressions.Add(node); return base.VisitTypeBinary(node); } protected override Expression VisitUnary(UnaryExpression node) { this.expressions.Add(node); return base.VisitUnary(node); } public IEnumerable<Expression> Explore(Expression node) { this.expressions.Clear(); this.Visit(node); return expressions.ToArray(); } } } 

Answer 3

I tried a simple (not complete) implementation to mutate the expression p => p.Id == 15 (code below). There is one class called "CrossMapping" that defines the mapping between original and "new" types and type members.

There are several labels named Mutate_XY_Expression for each type of expression that creates a new mutated expression. Method inputs require an express expression ( MemberExpression originalExpression ) as an expression model, a list expression, or parameters ( IList<ParameterExpression> parameterExpressions ), which are defined parameters using the parent expression and must be used by the parent body and the mapping object ( CrossMapping mapping ), which defines the mapping between types and members.

For a full implementation, you may need more information from the parent expression than parameters. But the template should be the same.

The sample does not implement the visitor pattern, as you know - this is because of simplicity. But there is no obstacle to their transformation.

Hope this helps.

Code (C # 4.0):

 using System; using System.Collections.Generic; using System.Linq; using System.Reflection; using System.Linq.Expressions; namespace ConsoleApplication1 { public class Product1 { public int Id { get; set; } public string Name { get; set; } public decimal Weight { get; set; } } public class Product2 { public int Id { get; set; } public string Name { get; set; } public decimal Weight { get; set; } } class Program { static void Main( string[] args ) { // list of products typed as Product1 var lst1 = new List<Product1> { new Product1{ Id = 1, Name = "One" }, new Product1{ Id = 15, Name = "Fifteen" }, new Product1{ Id = 9, Name = "Nine" } }; // the expression for filtering products // typed as Product1 Expression<Func<Product1, bool>> q1; q1 = p => p.Id == 15; // list of products typed as Product2 var lst2 = new List<Product2> { new Product2{ Id = 1, Name = "One" }, new Product2{ Id = 15, Name = "Fifteen" }, new Product2{ Id = 9, Name = "Nine" } }; // type of Product1 var tp1 = typeof( Product1 ); // property info of "Id" property from type Product1 var tp1Id = tp1.GetProperty( "Id", BindingFlags.Public | BindingFlags.Instance ); // delegate type for predicating for Product1 var tp1FuncBool = typeof( Func<,> ).MakeGenericType( tp1, typeof( bool ) ); // type of Product2 var tp2 = typeof( Product2 ); // property info of "Id" property from type Product2 var tp21Id = tp2.GetProperty( "Id", BindingFlags.Public | BindingFlags.Instance ); // delegate type for predicating for Product2 var tp2FuncBool = typeof( Func<,> ).MakeGenericType( tp2, typeof( bool ) ); // mapping object for types and type members var cm1 = new CrossMapping { TypeMapping = { // Product1 -> Product2 { tp1, tp2 }, // Func<Product1, bool> -> Func<Product2, bool> { tp1FuncBool, tp2FuncBool } }, MemberMapping = { // Product1.Id -> Product2.Id { tp1Id, tp21Id } } }; // mutate express from Product1 "enviroment" to Product2 "enviroment" var cq1_2 = MutateExpression( q1, cm1 ); // compile lambda to delegate var dlg1_2 = ((LambdaExpression)cq1_2).Compile(); // executing delegate var rslt1_2 = lst2.Where( (Func<Product2, bool>)dlg1_2 ).ToList(); return; } class CrossMapping { public IDictionary<Type, Type> TypeMapping { get; private set; } public IDictionary<MemberInfo, MemberInfo> MemberMapping { get; private set; } public CrossMapping() { this.TypeMapping = new Dictionary<Type, Type>(); this.MemberMapping = new Dictionary<MemberInfo, MemberInfo>(); } } static Expression MutateExpression( Expression originalExpression, CrossMapping mapping ) { var ret = MutateExpression( originalExpression: originalExpression, parameterExpressions: null, mapping: mapping ); return ret; } static Expression MutateExpression( Expression originalExpression, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { Expression ret; if ( null == originalExpression ) { ret = null; } else if ( originalExpression is LambdaExpression ) { ret = MutateLambdaExpression( (LambdaExpression)originalExpression, parameterExpressions, mapping ); } else if ( originalExpression is BinaryExpression ) { ret = MutateBinaryExpression( (BinaryExpression)originalExpression, parameterExpressions, mapping ); } else if ( originalExpression is ParameterExpression ) { ret = MutateParameterExpression( (ParameterExpression)originalExpression, parameterExpressions, mapping ); } else if ( originalExpression is MemberExpression ) { ret = MutateMemberExpression( (MemberExpression)originalExpression, parameterExpressions, mapping ); } else if ( originalExpression is ConstantExpression ) { ret = MutateConstantExpression( (ConstantExpression)originalExpression, parameterExpressions, mapping ); } else { throw new NotImplementedException(); } return ret; } static Type MutateType( Type originalType, IDictionary<Type, Type> typeMapping ) { if ( null == originalType ) { return null; } Type ret; typeMapping.TryGetValue( originalType, out ret ); if ( null == ret ) { ret = originalType; } return ret; } static MemberInfo MutateMember( MemberInfo originalMember, IDictionary<MemberInfo, MemberInfo> memberMapping ) { if ( null == originalMember ) { return null; } MemberInfo ret; memberMapping.TryGetValue( originalMember, out ret ); if ( null == ret ) { ret = originalMember; } return ret; } static LambdaExpression MutateLambdaExpression( LambdaExpression originalExpression, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { if ( null == originalExpression ) { return null; } var newParameters = (from p in originalExpression.Parameters let np = MutateParameterExpression( p, parameterExpressions, mapping ) select np).ToArray(); var newBody = MutateExpression( originalExpression.Body, newParameters, mapping ); var newType = MutateType( originalExpression.Type, mapping.TypeMapping ); var ret = Expression.Lambda( delegateType: newType, body: newBody, name: originalExpression.Name, tailCall: originalExpression.TailCall, parameters: newParameters ); return ret; } static BinaryExpression MutateBinaryExpression( BinaryExpression originalExpression, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { if ( null == originalExpression ) { return null; } var newExprConversion = MutateExpression( originalExpression.Conversion, parameterExpressions, mapping ); var newExprLambdaConversion = (LambdaExpression)newExprConversion; var newExprLeft = MutateExpression( originalExpression.Left, parameterExpressions, mapping ); var newExprRigth = MutateExpression( originalExpression.Right, parameterExpressions, mapping ); var newType = MutateType( originalExpression.Type, mapping.TypeMapping ); var newMember = MutateMember( originalExpression.Method, mapping.MemberMapping); var newMethod = (MethodInfo)newMember; var ret = Expression.MakeBinary( binaryType: originalExpression.NodeType, left: newExprLeft, right: newExprRigth, liftToNull: originalExpression.IsLiftedToNull, method: newMethod, conversion: newExprLambdaConversion ); return ret; } static ParameterExpression MutateParameterExpression( ParameterExpression originalExpresion, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { if ( null == originalExpresion ) { return null; } ParameterExpression ret = null; if ( null != parameterExpressions ) { ret = (from p in parameterExpressions where p.Name == originalExpresion.Name select p).FirstOrDefault(); } if ( null == ret ) { var newType = MutateType( originalExpresion.Type, mapping.TypeMapping ); ret = Expression.Parameter( newType, originalExpresion.Name ); } return ret; } static MemberExpression MutateMemberExpression( MemberExpression originalExpression, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { if ( null == originalExpression ) { return null; } var newExpression = MutateExpression( originalExpression.Expression, parameterExpressions, mapping ); var newMember = MutateMember( originalExpression.Member, mapping.MemberMapping ); var ret = Expression.MakeMemberAccess( expression: newExpression, member: newMember ); return ret; } static ConstantExpression MutateConstantExpression( ConstantExpression originalExpression, IList<ParameterExpression> parameterExpressions, CrossMapping mapping ) { if ( null == originalExpression ) { return null; } var newType = MutateType( originalExpression.Type, mapping.TypeMapping ); var newValue = originalExpression.Value; var ret = Expression.Constant( value: newValue, type: newType ); return ret; } } } 

Answer 4

John's good answer is above, so I expanded it to handle method calls, constant expressions, etc., so now it also works for expressions like:

 x => x.SubObjects .AsQueryable() .SelectMany(y => y.GrandChildObjects) .Any(z => z.Value == 3) 

I also retired from ExpressionTreeExplorer , since the only thing we need is ParameterExpressions.

Here is the code (Update: clear the cache when the conversion is completed)

 public class ExpressionTargetTypeMutator : ExpressionVisitor { private readonly Func<Type, Type> typeConverter; private readonly Dictionary<Expression, Expression> _convertedExpressions = new Dictionary<Expression, Expression>(); public ExpressionTargetTypeMutator(Func<Type, Type> typeConverter) { this.typeConverter = typeConverter; } // Clear the ParameterExpression cache between calls to Visit. // Not thread safe, but you can probably fix it easily. public override Expression Visit(Expression node) { bool outermostCall = false; if (false == _isVisiting) { this._isVisiting = true; outermostCall = true; } try { return base.Visit(node); } finally { if (outermostCall) { this._isVisiting = false; _convertedExpressions.Clear(); } } } protected override Expression VisitMember(MemberExpression node) { var sourceType = node.Member.ReflectedType; var targetType = this.typeConverter(sourceType); var converted = Expression.MakeMemberAccess( base.Visit(node.Expression), targetType.GetProperty(node.Member.Name)); return converted; } protected override Expression VisitParameter(ParameterExpression node) { Expression converted; if (false == _convertedExpressions.TryGetValue(node, out converted)) { var sourceType = node.Type; var targetType = this.typeConverter(sourceType); converted = Expression.Parameter(targetType, node.Name); _convertedExpressions.Add(node, converted); } return converted; } protected override Expression VisitMethodCall(MethodCallExpression node) { if (node.Method.IsGenericMethod) { var convertedTypeArguments = node.Method.GetGenericArguments() .Select(this.typeConverter) .ToArray(); var genericMethodDefinition = node.Method.GetGenericMethodDefinition(); var newMethod = genericMethodDefinition.MakeGenericMethod(convertedTypeArguments); return Expression.Call(newMethod, node.Arguments.Select(this.Visit)); } return base.VisitMethodCall(node); } protected override Expression VisitConstant(ConstantExpression node) { var valueExpression = node.Value as Expression; if (null != valueExpression) { return Expression.Constant(this.Visit(valueExpression)); } return base.VisitConstant(node); } protected override Expression VisitLambda<T>(Expression<T> node) { return Expression.Lambda(this.Visit(node.Body), node.Name, node.TailCall, node.Parameters.Select(x => (ParameterExpression)this.VisitParameter(x))); } } 

Answer 5

Doesn't ExecuteTypedList do what you want to do? SubSonic will populate your DTO / POCOs. From Rob Connery's blog:

ExecuteTypedList <> tries to match returned column names to passed type property names. In this example, they exactly coincide - and this is not so completely real world. You can get around this by overlaying columns - just like with SQL, call:

 return Northwind.DB.Select("ProductID as 'ID'", "ProductName as 'Name'", "UnitPrice as 'Price'") .From<Northwind.Product>().ExecuteTypedList<Product>(); 

Here's a link to Rob Writing Split, Testable Code with SubSonic 2.1

Answer 6

I think Linq-To-Sql will create the desired SQL if you execute your queries correctly. In this case, using IQueryable and deferred execution, you may not return all ActiveRecord.Widget entries.

 IEnumerable<DataContract.Widget> GetMany( Func<DataContract.Widget, bool> predicate) { // get Widgets IQueryable<DataContract.Widget> qry = dc.Widgets.Select(w => TODO: CONVERT_TO_DataContract.Widget); return qry.Where(predicate); }