Tuesday, April 10, 2007 3:14 PM bart

The IQueryable tales - LINQ to LDAP - Part 4: Parsing and executing queries

Introduction

Welcome back to the LINQ-to-LDAP series. So far, we've been discussing:

In the previous post, we entered the domain of implementing a custom query provider for LINQ. More specifically, we did discuss the need for entities and came up with a class definition that maps an object from the underlying data source, which is Active Directory in our case. Such an entity has the following shape:

An entity type for user objects in Active Directory - Copy Code
1 [DirectorySchema("user")] 2 public class User 3 { 4 [DirectoryAttribute("givenName")] 5 public string FirstName { get; set; } 6 7 [DirectoryAttribute("sn")] 8 public string LastName { get; set; } 9 10 [DirectoryAttribute("physicalDeliveryOfficeName")] 11 public string Office { get; set; } 12 }

Today we (finally) take it another stpe forward, focusing on the LINQ-to-LDAP translation when writing queries like:

A LINQ query against Active Directory - Copy Code
1 DirectorySource<User> users = new DirectorySource<User>(...); 2 var res = from usr in users 3 where usr.Office == "Building 10 - 1.25" 4 select usr.FirstName + " " + usr.LastName;

In technical terms, we'll make a first implementation of IQueryable<T>: DirectorySource<T>.

 

Back to the attributes

In the previous post, two custom attributes were introduced to perform the mapping between entities and the underlying directory objects. Before moving on, I want to extend those a little more to support more complex data types. For instance, attributes like pwdLastSet are retrieved by DirectoryEntry's Properties collections as a COM-object. We could try to do a conversion to the more convenient DateTime BCL type ourselves, but instead a COM library called "Active DS Type Library" proves handy since it exposes properties in the right .NET type. To use this library, add a reference to the Active DS Type Library via Add Reference..., tab COM.

When you take a look inside the library using the Object Browser, you'll find a series of interfaces starting with IADs. One of these is IADsUser:

As you can see, the PasswordLastChanges property is of type System.DateType, allowing us to use it right away without having to mess around with COM-to-.NET conversions for complex types. You might wonder why we don't just use this class for all our entity stuff. The answer is that we want to provide the end-users with the maximum level of flexibility. Therefore, we'll both support "LDAP attribute names" as well as property names from the IADs* interfaces. So, how can we reflect these feature requests in the custom attribute definitions? The answer is below.

Note: In future posts, we'll talk even more about entities, when we want to support updates too. In order to make this possible, we'll have to track changes to retrieved objects in order to feed these back when making an Update call to the DirectorySource<T> data source. However, some properties don't support updating because of their read-only nature (PasswordLastChanged is one like this). Therefore, the entity type will need to reflect this.

Mapping custom attributes for entity objects - Copy Code
1 /// <summary> 2 /// Specifies the directory schema to query. 3 /// </summary> 4 [AttributeUsage(AttributeTargets.Class, AllowMultiple = false)] 5 public class DirectorySchemaAttribute : Attribute 6 { 7 private string schema; 8 private Type helper; 9 10 /// <summary> 11 /// Creates a new schema indicator attribute. 12 /// </summary> 13 /// <param name="schema">Name of the schema to query for.</param> 14 public DirectorySchemaAttribute(string schema) 15 { 16 this.schema = schema; 17 } 18 19 /// <summary> 20 /// Creates a new schema indicator attribute. 21 /// </summary> 22 /// <param name="schema">Name of the schema to query for.</param> 23 /// <param name="activeDsHelperType">Helper type for Active DS object properties.</param> 24 public DirectorySchemaAttribute(string schema, Type activeDsHelperType) 25 { 26 this.schema = schema; 27 this.helper = activeDsHelperType; 28 } 29 30 /// <summary> 31 /// Name of the schema to query for. 32 /// </summary> 33 public string Schema 34 { 35 get { return schema; } 36 set { schema = value; } 37 } 38 39 /// <summary> 40 /// Helper type for Active DS object properties. 41 /// </summary> 42 public Type ActiveDsHelperType 43 { 44 get { return helper; } 45 set { helper = value; } 46 } 47 } 48 49 /// <summary> 50 /// Specifies the underlying attribute to query for in the directory. 51 /// </summary> 52 [AttributeUsage(AttributeTargets.Property, AllowMultiple = false)] 53 public class DirectoryAttributeAttribute : Attribute 54 { 55 private string attribute; 56 private DirectoryAttributeType type; 57 58 /// <summary> 59 /// Creates a new attribute binding attribute for a entity class field or property. 60 /// </summary> 61 /// <param name="attribute">Name of the attribute to query for.</param> 62 public DirectoryAttributeAttribute(string attribute) 63 { 64 this.attribute = attribute; 65 this.type = DirectoryAttributeType.Ldap; 66 } 67 68 /// <summary> 69 /// Creates a new attribute binding attribute for a entity class field or property. 70 /// </summary> 71 /// <param name="attribute">Name of the attribute to query for.</param> 72 /// <param name="type">Type of the underlying query source to get the attribute from.</param> 73 public DirectoryAttributeAttribute(string attribute, DirectoryAttributeType type) 74 { 75 this.attribute = attribute; 76 this.type = type; 77 } 78 79 /// <summary> 80 /// Name of the attribute to query for. 81 /// </summary> 82 public string Attribute 83 { 84 get { return attribute; } 85 set { attribute = value; } 86 } 87 88 /// <summary> 89 /// Type of the underlying query source to get the attribute from. 90 /// </summary> 91 public DirectoryAttributeType Type 92 { 93 get { return type; } 94 set { type = value; } 95 } 96 } 97 98 /// <summary> 99 /// Type of the query source to perform queries with. 100 /// </summary> 101 public enum DirectoryAttributeType 102 { 103 /// <summary> 104 /// Default value. Uses the Properties collection of DirectoryEntry to get data from. 105 /// </summary> 106 Ldap, 107 108 /// <summary> 109 /// Uses Active DS Helper IADs* objects to get data from. 110 /// </summary> 111 ActiveDs 112 }

The custom attribute definition from above allows us to write things like this:

An entity with complex mapping - Copy Code
1 [DirectorySchema("user", typeof(IADsUser))] 2 public class User 3 { 4 [DirectoryAttribute("givenName")] 5 public string FirstName { get; set; } 6 7 [DirectoryAttribute("sn")] 8 public string LastName { get; set; } 9 10 [DirectoryAttribute("physicalDeliveryOfficeName")] 11 public string Office { get; set; } 12 13 [DirectoryAttribute("PasswordLastChanged", DirectoryAttributeType.ActiveDs)] 14 public DateTime PasswordLastSet { get; set; } 15 }

In this sample, a property has been defined (line 13-14) that relies on the ActiveDs property name instead of the LDAP property name, which has been indicated using DirectoryAttributeType.ActiveDs. Notice that the PasswordLastSet property has both a getter and setter accessor defined although it is read-only in the underlying IADsUser type definition; the setter is required for our framework to assign the retrieved value to it. In order for the framework to find the corresponding IADs* type for the entity object, an additional parameter has been added to the DirectorySchema attribute in line 1 (typeof(IADsUser)). This set of information will be enough to compose the query and get the values back via the desired "channel".

Furthermore, we'll also allow entity types to omit the DirectoryAttribute decorations if no renaming has to happen:

Copy Code
1 [DirectorySchema("user")] 2 public class User 3 { 4 public string GivenName { get; set; } 5 public string Sn { get; set; } 6 }

In here, the GivenName and Sn properties have the same name as their LDAP attribute name equivalent (which is case-insensitive by the way), so the system can figure out the mapping on its own. We'll support such an automatic mapping only for LDAP attribute names, not for IADs* property names.

 

The skeleton

On to the real work now; the DirectorySource<T> implementation. Let's start by revisiting our IQueryable<T> skeleton implementation. Since we won't support sorting right now, we won't derive from IOrderedQueryable<T>. An implementation of sorting is possible though, using a IQueryable<T>-to-IEnumerable<T> translation, followed by LINQ-to-Objects sorting, when yielding the results back. This discussion would lead us too far, so let's move on with the basics for now:

Basic skeleton of IQueryable<T> implementation - Copy Code
1 /// <summary> 2 /// Represents an LDAP data source. Allows for querying the LDAP data source via LINQ. 3 /// </summary> 4 /// <typeparam name="T">Entity type in the underlying source.</typeparam> 5 public class DirectorySource<T> : IQueryable<T> 6 { 7 /// <summary> 8 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 9 /// </summary> 10 /// <param name="expression">Expression representing the LDAP query.</param> 11 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 12 public IQueryable CreateQuery(Expression expression) 13 { 14 return CreateQuery<T>(expression); 15 } 16 17 /// <summary> 18 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 19 /// </summary> 20 /// <param name="expression">Expression representing the LDAP query.</param> 21 /// <typeparam name="TElement">The type of the elements of the IQueryable that is returned.</typeparam> 22 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 23 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 24 { 25 // 26 // TODO 27 // 28 } 29 30 #region Execution (not implemented) 31 32 public object Execute(Expression expression) 33 { 34 throw new NotImplementedException(); 35 } 36 37 public TResult Execute<TResult>(Expression expression) 38 { 39 throw new NotImplementedException(); 40 } 41 42 #endregion 43 44 #region Enumeration 45 46 IEnumerator IEnumerable.GetEnumerator() 47 { 48 return GetEnumerator(); 49 } 50 51 public IEnumerator<T> GetEnumerator() 52 { 53 // 54 // TODO 55 // 56 } 57 58 #endregion 59 60 public Type ElementType 61 { 62 get { return typeof(T); } 63 } 64 65 public Expression Expression 66 { 67 get { return Expression.Constant(this); } 68 } 69 }

This basic skeleton is based on the one shown in Part 2 of this series.

A first concern is the constructor, which should collect enough information to be able sending the query to AD. In order to allow a maximum level of flexibility, we're going to support passing in a DirectoryEntry representing the node to start the search from (remember that directory services implementations are tree based, in which distinguished names - aka DNs - are put together by various parts representing domain names, OUs, etc) together with a SearchScope. The latter argument is required to allows users to specify how the search has to be performed; it can be a base-level search (in which only the current node is searched, so at maximum one result - the node itself - can be returned; maybe not that useful) or a subtree search (the whole subtree below the current node) or a "one level" deep search (only searching the childs of the current node). The constructor is defined below:

Constructor for DirectorySource<T> - Copy Code
1 /// <summary> 2 /// Creates a new data source instance for the given directory search root and with a given search scope. 3 /// </summary> 4 /// <param name="searchRoot">Root location in the directory to start all searches from.</param> 5 /// <param name="searchScope">Search scope for all queries performed through this data source.</param> 6 public DirectorySource(DirectoryEntry searchRoot, SearchScope searchScope) 7 { 8 this.searchRoot = searchRoot; 9 this.searchScope = searchScope; 10 }

This requires two member attributes:

Private members related to the constructor's parameterization - Copy Code
1 #region Directory information 2 3 private DirectoryEntry searchRoot; 4 private SearchScope searchScope; 5 6 #endregion

As another part of the "skeleton", let's add a property to support logging, in a similar fashion as the LINQ-to-SQL API does:

Logger support - Copy Code
1 private TextWriter logger; 2 3 /// <summary> 4 /// Used to configure a logger to print diagnostic information about the query performed. 5 /// </summary> 6 public TextWriter Log 7 { 8 get { return logger; } 9 set { logger = value; } 10 }

 

Implementing the parsing logic

Next, we move on to the CreateQuery<TElement> implementation itself. In part 2 of this series, we've shown the rationale behind CreateQuery and how several query-related operations result in a chain of CreateQuery calls to be made. The signature of CreateQuery is shown below as a quick refresh:

// Summary: // Constructs an System.Linq.IQueryable<T> object that can evaluate the query // represented by the specified expression tree. // // Parameters: // expression: // The System.Linq.Expressions.Expression representing the query to be encompassed. // // Returns: // An System.Linq.IQueryable<T> that can evaluate the query represented by the // specified expression tree. IQueryable<TElement> CreateQuery<TElement>(Expression expression);

This method is part of IQueryable<T> and transforms it into a new IQueryable<TElement>. For example, when doing filtering with a Where clause, T and TElement will be the same because of the Queryable.Where's signature:

// // Summary: // Filters a sequence of values based on a predicate. // // Parameters: // predicate: // An System.Linq.Expressions.Expression<TDelegate> that represents a function // that takes a value of type TSource and returns a System.Boolean to use to // test each element. // // source: // A System.Linq.IQueryable<T> to filter. // // Returns: // An System.Linq.IQueryable<T> that contains elements from the input sequence // which satisfy the condition specified by predicate. public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate);

However, when doing a projection, T and TElement will differ, as illustrated below by inspection of the Select method:

// // Summary: // Projects each element of a sequence into a new form. // // Parameters: // source: // A sequence of values to invoke a selector function on. // // selector: // An System.Linq.Expressions.Expression<TDelegate> that represents a generic // function to apply to each element. // // Returns: // An System.Linq.IQueryable<T> whose elements are the result of invoking a // selector function on each element of source. public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector);

During the chain of CreateQuery calls we need to gather enough information to carry out the query when asked to do so, in the GetEnumerator method. Each time CreateQuery is called, we'll create a new instance of the DirectorySource<TElement> class that will gather additional information about the query. Enough theory, let's introduce a few private class members that carry query information:

Query information members - Copy Code
1 #region Query information 2 3 private string query; 4 5 #endregion 6 7 #region Projection information 8 9 private HashSet<string> properties = new HashSet<string>(); 10 private Delegate project; 11 12 #endregion 13 14 private Type originalType = typeof(T);

The query member will capture the LDAP filter expression that needs to be sent to the AD server to perform the query. To optimize the query, we'll collect all of the AD object attributes that will make up the propertiesToLoad parameter to the DirectorySearcher type, in the properties set. Furthermore, a delegate to dynamically compiled code for the projection is kept in project and finally we keep track of the original type of the query objects, as we'll discuss later. We'll start with the CreateQuery method definition now:

CreateQuery implementation - Copy Code
1 /// <summary> 2 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 3 /// </summary> 4 /// <param name="expression">Expression representing the LDAP query.</param> 5 /// <typeparam name="TElement">The type of the elements of the IQueryable that is returned.</typeparam> 6 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 7 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 8 { 9 // 10 // Create a new queryable object based on the current one. A copy is needed to guarantee uniqueness across multiple queries. 11 // 12 DirectorySource<TElement> d = new DirectorySource<TElement>(searchRoot, searchScope); 13 d.query = query; 14 d.project = project; 15 d.properties = properties; 16 d.originalType = originalType; 17 d.logger = logger; 18 19 // 20 // We expect a method call expression for the chain of LINQ query operator method calls. 21 // 22 MethodCallExpression call = expression as MethodCallExpression; 23 if (call != null) 24 { 25 // 26 // First parameter to the method call represents the (unary) lambda in LINQ style. 27 // E.g. (user => user.Name == "Bart") for a Where clause 28 // (user => new { user.Name }) for a Select clause 29 // 30 switch (call.Method.Name) 31 { 32 // 33 // Builds the query LDAP expression. 34 // 35 case "Where": 36 d.BuildQuery(((UnaryExpression)call.Arguments[1]).Operand as LambdaExpression); 37 break; 38 // 39 // Builds the projection and filters the required properties. 40 // 41 case "Select": 42 d.BuildProjection(((UnaryExpression)call.Arguments[1]).Operand as LambdaExpression); 43 break; 44 } 45 } 46 47 return d; 48 }

A quick analysis:

  • On lines 12 to 17, we create a new instance of DirectorySource for the TElement "output".
  • As discussed in Part 2, all of the CreateQuery calls originate from the Queryable.* methods that add themselves as method call expressions to the expression tree. On line 22 we obtain that call expression.
  • Next, we switch on the method call being made, which we expect to be either "Where" or "Select" for basic queries. If you want to provide some way to support sorting, you could also capture method calls to OrderBy, ThenBy, OrderByDescending and ThenByDescending. Similarly, you could intercept all method call expressions for all of the Queryable-methods. Finally, calls are made to BuildQuery or BuildProjection on the newly created object, as explained below.

 

Translating filters to LDAP query expressions

Now, let's move our focus on the query parsing itself, i.e. the Select case. The BuildQuery method takes care of this:

BuildQuery method - Copy Code
1 /// <summary> 2 /// Helper method to build the LDAP query. 3 /// </summary> 4 /// <param name="q">Lambda expression to be translated to LDAP.</param> 5 private void BuildQuery(LambdaExpression q) 6 { 7 StringBuilder sb = new StringBuilder(); 8 9 // 10 // Recursive tree traversal to build the LDAP query (prefix notation). 11 // 12 ParseQuery(q.Body, sb); 13 14 query = sb.ToString(); 15 }

This method prepares an "accumulator" object in the form of a string builder. The ParseQuery method will build the query recursively, spitting its output in the StringBuilder instance, and the result is finally kept in the query private member of the object, ready for comsumption at a later stage when GetEnumerator is called (see further). Here's ParseQuery:

Recursive query parsing - Copy Code
1 /// <summary> 2 /// Recursive helper method for query parsing based on the given expression tree. 3 /// </summary> 4 /// <param name="e">Expression tree to be translated to LDAP.</param> 5 /// <param name="sb">Accummulative query string used in recursion.</param> 6 private void ParseQuery(Expression e, StringBuilder sb) 7 { 8 sb.Append("("); 9 // 10 // Support for boolean operators & and |. Support for "raw" conditions (like equality). 11 // 12 if (e is BinaryExpression) 13 { 14 BinaryExpression c = e as BinaryExpression; 15 switch (c.NodeType) 16 { 17 case ExpressionType.AndAlso: 18 sb.Append("&"); 19 ParseQuery(c.Left, sb); 20 ParseQuery(c.Right, sb); 21 break; 22 case ExpressionType.OrElse: 23 sb.Append("|"); 24 ParseQuery(c.Left, sb); 25 ParseQuery(c.Right, sb); 26 break; 27 default: //E.g. Equal, NotEqual, GreaterThan 28 sb.Append(GetCondition(c)); 29 break; 30 } 31 } 32 // 33 // Support for boolean negation. 34 // 35 else if (e is UnaryExpression) 36 { 37 UnaryExpression c = e as UnaryExpression; 38 if (c.NodeType == ExpressionType.Not) 39 { 40 sb.Append("!"); 41 ParseQuery(c.Operand, sb); 42 } 43 else 44 throw new NotSupportedException("Unsupported query operator detected: " + c.NodeType); 45 } 46 // 47 // Support for string operations. 48 // 49 else if (e is MethodCallExpression) 50 { 51 MethodCallExpression m = e as MethodCallExpression; 52 MemberExpression o = (m.Object as MemberExpression); 53 if (m.Method.DeclaringType == typeof(string)) 54 { 55 switch (m.Method.Name) 56 { 57 case "Contains": 58 { 59 ConstantExpression c = m.Arguments[0] as ConstantExpression; 60 sb.AppendFormat("{0}=*{1}*", GetFieldName(o.Member), c.Value); 61 break; 62 } 63 case "StartsWith": 64 { 65 ConstantExpression c = m.Arguments[0] as ConstantExpression; 66 sb.AppendFormat("{0}={1}*", GetFieldName(o.Member), c.Value); 67 break; 68 } 69 case "EndsWith": 70 { 71 ConstantExpression c = m.Arguments[0] as ConstantExpression; 72 sb.AppendFormat("{0}=*{1}", GetFieldName(o.Member), c.Value); 73 break; 74 } 75 default: 76 throw new NotSupportedException("Unsupported string filtering query expression detected. Cannot translate to LDAP equivalent."); 77 } 78 } 79 else 80 throw new NotSupportedException("Unsupported query expression detected. Cannot translate to LDAP equivalent."); 81 } 82 else 83 throw new NotSupportedException("Unsupported query expression detected. Cannot translate to LDAP equivalent."); 84 sb.Append(")"); 85 }

This method requires a bit more explanation:

  • On line 1 and 84, the enclosing braces for the query expression are added; as you saw in Part 3, LDAP queries have a prefix notiation in which each part of the query is enclosed in parentheses.
  • Lines 12-31 covers the case of a binary expression, which can be quite a lot of things. Two common cases are the && operator and the || operator, which are translated into the LDAP equivalents & and | respectively, followed by a recursive call for both sides of the expression (left and right). The final case of the switch-operator covers the remainder of cases, such as usr.Name == "Bart", which is also a binary operation. We'll discuss this parsing later, when talking about GetCondition.
  • Lines 35-45 covers unary expressions, for which we'll only support the negation operator. In such a case, the ! LDAP operator is sent to the output, followed by recursive parsing of the operand.
  • Finally, in lines 49-81, we move on to the method call case. A typical supported example is usr.Name.StartsWith("A") which translates nicely into a A* LDAP query equivalent. We'll only support such operations when applied to strings and only for Contains, StartsWith and EndsWith. This code isn't perfect, since we assume the first parameter to be a constant expression, but it could be more complex too (for example when writing something like usr.Name.Contains(DoSomething(123))). This would lead us too far however, but feel free to change to the code to support this (tip: see further, when dealing with projections where we'll perform dynamic compilation of portiong of an expression tree). In here, we call a method GetFieldName that is responsible to find the underlying name for an entity object's member (e.g. in usr.FirstName.Contains("A"), the FirstName property could be translated into givenName which is the corresponding LDAP name). Note: IADs* string properties can't be used via this approach because we perform a translation to LDAP names directly. More generally, none of the IADs* properties can be used in the filter expression since these are only available when the query has executed. You could provide additional logic to throw an exception when such a case is encountered (by using custom attribute inspection in a similar way as done further in this post).

Now, we'll move on to the GetCondition method that's responsible to parse the "atoms" of query expressions:

GetCondition method - Copy Code
1 /// <summary> 2 /// Helper expression to translate conditions to LDAP filters. 3 /// </summary> 4 /// <param name="e">Conditional expression to be translated to an LDAP filter.</param> 5 /// <returns>String representing the condition in LDAP.</returns> 6 private string GetCondition(BinaryExpression e) 7 { 8 string val, attrib; 9 10 bool neg; 11 12 // 13 // Find the order of the operands in the binary expression. At least one should refer to the entity type. 14 // 15 if (e.Left is MemberExpression && ((MemberExpression)e.Left).Member.DeclaringType == originalType) 16 { 17 neg = false; 18 19 attrib = GetFieldName(((MemberExpression)e.Left).Member); 20 val = Expression.Lambda(e.Right).Compile().DynamicInvoke().ToString(); 21 } 22 else if (e.Right is MemberExpression && ((MemberExpression)e.Right).Member.DeclaringType == originalType) 23 { 24 neg = true; 25 26 attrib = GetFieldName(((MemberExpression)e.Right).Member); 27 val = Expression.Lambda(e.Left).Compile().DynamicInvoke().ToString(); 28 } 29 else 30 throw new NotSupportedException("A filtering expression should contain an entity member selection expression."); 31 32 // 33 // Normalize some common characters that cannot be used in LDAP filters. 34 // 35 val = val.ToString().Replace("(", "0x28").Replace(")", "0x29").Replace(@"\", "0x5c"); 36 37 // 38 // Determine the operator and swap the operandi if necessary (LDAP requires a field=value order). 39 // 40 switch (e.NodeType) 41 { 42 case ExpressionType.Equal: 43 return String.Format("{0}={1}", attrib, val); 44 case ExpressionType.NotEqual: 45 return String.Format("!({0}={1})", attrib, val); 46 case ExpressionType.GreaterThanOrEqual: 47 if (!neg) 48 return String.Format("{0}>={1}", attrib, val); 49 else 50 return String.Format("{0}<={1}", attrib, val); 51 case ExpressionType.GreaterThan: 52 if (!neg) 53 return String.Format("&({0}>={1})(!({0}={1}))", attrib, val); 54 else 55 return String.Format("&({0}<={1})(!({0}={1}))", attrib, val); 56 case ExpressionType.LessThanOrEqual: 57 if (!neg) 58 return String.Format("{0}<={1}", attrib, val); 59 else 60 return String.Format("{0}>={1}", attrib, val); 61 case ExpressionType.LessThan: 62 if (!neg) 63 return String.Format("&({0}<={1})(!({0}={1}))", attrib, val); 64 else 65 return String.Format("&({0}>={1})(!({0}={1}))", attrib, val); 66 default: 67 throw new NotSupportedException("Unsupported filtering operator detected: " + e.NodeType); 68 } 69 }

A quick analysis... Most query expressions are of the format <entity>.<property> <operator> <expression> or <expression> <operator> <entity>.<property>. LDAP only supports the former order, so we need to take care of this, which is done in lines 15-28. A few important remarks though:

  • We don't support expressions in which the entity member selection is (deeply) nested, like DoIt(usr.Name) == "Bart".
  • We do support a more complex expression for the "value" however, by means of dynamic compilation as shown in lines 20 and 27. However, when such an expression contains an entity member selection like usr.Name, things won't work any longer (at runtime).

In line 35, a few characters are escaped in order to make the query expression valid. One escape isn't included however, i.e. the one required for the * character. This allows complex queries to be created right away, like this: usr.Name == "B*rt * Smet". In an asterisk should be treated as a constant, one should escape it manually as 0x2a (see RFC 2254 page 4). A similar remark holds for the NUL value (0x00).

Finally, the condition is translated to the LDAP filter equivalent, supporting operators =, >= and <=, based on the expression tree node's operator (NodeType). Notice that if-constructs are required for the "reverse expression" case from lines 24-27, where the operator should be reversed (as in lines 48, 53, 58 and 63). This implementation is very naive though, since LDAP operators <= and >= are used for lexographical comparisons (see How Active Directory Works) and we allow a much broader domain of applicability. This too is kind of a domain mismatch between OO and LDAP, as mentioned in Part 3. We won't elaborate on this; you might think of better alternatives to deal with this operator meaning mismatch (maybe by just rejecting the use of these operators via LINQ?). Also, notice that strings in .NET don't have support for <= or >= directly and a translation from CompareTo method calls to <= and >= LDAP operators would be required.

To finish the "Where"-case of the parsing, take a look at GetFieldName:

GetFieldName method - Copy Code
1 private string GetFieldName(MemberInfo member) 2 { 3 DirectoryAttributeAttribute[] da = member.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 4 if (da != null && da.Length != 0) 5 { 6 if (da[0].Type == DirectoryAttributeType.ActiveDs) 7 throw new InvalidOperationException("Can't execute query filters for IADs* properties."); 8 else 9 return da[0].Attribute; 10 } 11 else 12 return member.Name; 13 }

In here, we check whether or not a DirectoryAttributeAttribute custom attribute has been applied to the specified field. If that's the case, we should make sure that no ActiveDs (IADs*) mapping is made since these cannot be supported in LDAP queries directly (as explained earlier). Finally, the name of the underlying LDAP attribute is retrieved, either by means of the custom attribute's Attribute field or by copying the name of the member directly in case no mapping has been specified.

 

Supporting projections

On to the "Select"-case now, as implemented in BuildProjection:

Projection support - Copy Code
1 /// <summary> 2 /// Helper method for projection clauses (Select). 3 /// </summary> 4 /// <param name="p">Lambda expression representing the projection.</param> 5 private void BuildProjection(LambdaExpression p) 6 { 7 // 8 // Store projection information including the compiled lambda for subsequent execution 9 // and a minimal set of properties to be retrieved (improves efficiency of queries). 10 // 11 project = p.Compile(); 12 13 // 14 // Original type is kept for reflection during querying. 15 // 16 originalType = p.Parameters[0].Type; 17 18 // 19 // Support for (anonymous) type initialization based on "member init expressions". 20 // 21 MemberInitExpression mi = p.Body as MemberInitExpression; 22 if (mi != null) 23 foreach (MemberAssignment b in mi.Bindings) 24 FindProperties(b.Expression); 25 // 26 // Support for identity projections (e.g. user => user), getting all properties back. 27 // 28 else 29 foreach (PropertyInfo i in originalType.GetProperties()) 30 properties.Add(i.Name); 31 }

As the matter in fact, we're very lazy this time. Since LDAP queries don't support projections, the query value doesn't need to be altered at all. Therefore, we compile the lambda expression and store it in the project delegate member variable. This will allow us to execute the projection in all its glory (and possible complexity, e.g. with member assignments using method calls to transform retrieved results before assignment) when processing the results in the GetEnumerator method. Next, the original type is kept in originalType (which reflects the one and only parameter to the projection lambda expression). This will prove useful for quite a bit of things down the drain. Finally, we switch between the member initialization case (new { <member> = <value> } constructs) and an identity projection (p => p), finally resulting in a series of properties that has to be retrieved (this allows us to request only the values of the attributes required for the projection logic, causing more efficient processing inside GetEnumerator as shown later). In the first case, using a member initialization expression, we need to traverse the entire expression tree for all possible "member-grabbing" references to the entity object (e.g. new { First = usr.FirstName, Nick = GetNickName(usr.FirstName + usr.LastName), Stats = new { usr.LogonCount } }), as done in FindProperties:

FindProperties uses recursion to find all entity type property references used in projection - Copy Code
1 /// <summary> 2 /// Recursive helper method to finds all required properties for projection. 3 /// </summary> 4 /// <param name="e">Expression to detect property uses for.</param> 5 private void FindProperties(Expression e) 6 { 7 // 8 // Record member accesses to properties or fields from the entity. 9 // 10 if (e.NodeType == ExpressionType.MemberAccess) 11 { 12 MemberExpression me = e as MemberExpression; 13 if (me.Member.DeclaringType == originalType) 14 { 15 DirectoryAttributeAttribute[] da = me.Member.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 16 if (da != null && da.Length != 0) 17 { 18 if (da[0].Type != DirectoryAttributeType.ActiveDs) 19 properties.Add(me.Member.Name); 20 } 21 else 22 properties.Add(me.Member.Name); 23 } 24 } 25 else 26 { 27 if (e is BinaryExpression) 28 { 29 BinaryExpression b = e as BinaryExpression; 30 FindProperties(b.Left); 31 FindProperties(b.Right); 32 } 33 else if (e is UnaryExpression) 34 { 35 UnaryExpression u = e as UnaryExpression; 36 FindProperties(u.Operand); 37 } 38 else if (e is ConditionalExpression) 39 { 40 ConditionalExpression c = e as ConditionalExpression; 41 FindProperties(c.IfFalse); 42 FindProperties(c.IfTrue); 43 FindProperties(c.Test); 44 } 45 else if (e is InvocationExpression) 46 { 47 InvocationExpression i = e as InvocationExpression; 48 FindProperties(i.Expression); 49 foreach (Expression ex in i.Arguments) 50 FindProperties(ex); 51 } 52 else if (e is LambdaExpression) 53 { 54 LambdaExpression l = e as LambdaExpression; 55 FindProperties(l.Body); 56 foreach (Expression ex in l.Parameters) 57 FindProperties(ex); 58 } 59 else if (e is ListInitExpression) 60 { 61 ListInitExpression li = e as ListInitExpression; 62 FindProperties(li.NewExpression); 63 foreach (Expression ex in li.BLOCKED EXPRESSION 64 FindProperties(ex); 65 } 66 else if (e is MemberInitExpression) 67 { 68 MemberInitExpression mi = e as MemberInitExpression; 69 FindProperties(mi.NewExpression); 70 foreach (MemberAssignment b in mi.Bindings) 71 FindProperties(b.Expression); 72 } 73 else if (e is MethodCallExpression) 74 { 75 MethodCallExpression mc = e as MethodCallExpression; 76 FindProperties(mc.Object); 77 foreach (Expression ex in mc.Arguments) 78 FindProperties(ex); 79 } 80 else if (e is NewExpression) 81 { 82 NewExpression n = e as NewExpression; 83 foreach (Expression ex in n.Arguments) 84 FindProperties(ex); 85 } 86 else if (e is NewArrayExpression) 87 { 88 NewArrayExpression na = e as NewArrayExpression; 89 foreach (Expression ex in na.BLOCKED EXPRESSION 90 FindProperties(ex); 91 } 92 else if (e is TypeBinaryExpression) 93 { 94 TypeBinaryExpression tb = e as TypeBinaryExpression; 95 FindProperties(tb.Expression); 96 } 97 } 98 }

Also, we'll only capture properties that are mapped to LDAP attributes, and not those mapped to IADs* objects. The recursion cases shown above should be pretty complete. All of this plumbing finally results in a HashSet of strings representing all the LDAP properties that should be retrieved.

 

Executing the query and yielding the results

Now that we have all the information required to carry out the query, we're ready to get called via GetEnumerator. In here, we'll execute the query, get the results back and perform the projections by means of the kept delegate project. Here's the code for GetEnumerator:

Executing the query and getting the results back - Copy Code
1 public IEnumerator<T> GetEnumerator() 2 { 3 return GetResults(); 4 } 5 6 private IEnumerator<T> GetResults() 7 { 8 DirectorySchemaAttribute[] attr = (DirectorySchemaAttribute[])originalType.GetCustomAttributes(typeof(DirectorySchemaAttribute), false); 9 if (attr == null || attr.Length == 0) 10 throw new InvalidOperationException("Missing schema mapping attribute."); 11 12 DirectoryEntry root = searchRoot; 13 string q = String.Format("(&(objectClass={0}){1})", attr[0].Schema, query); 14 DirectorySearcher s = new DirectorySearcher(root, q, properties.ToArray(), searchScope); 15 16 if (logger != null) 17 Log.WriteLine(q); 18 19 Type helper = attr[0].ActiveDsHelperType; 20 21 foreach (SearchResult sr in s.FindAll()) 22 { 23 DirectoryEntry e = sr.GetDirectoryEntry(); 24 25 object result = Activator.CreateInstance(project == null ? typeof(T) : originalType); 26 27 if (project == null) 28 { 29 foreach (PropertyInfo p in typeof(T).GetProperties()) 30 AssignResultProperty(helper, e, result, p.Name); 31 32 yield return (T)result; 33 } 34 else 35 { 36 foreach (string prop in properties) 37 AssignResultProperty(helper, e, result, prop); 38 39 yield return (T)project.DynamicInvoke(result); 40 } 41 } 42 }

First, on lines 8 to 13, we modify the query with another filtering expression in order to get only the only objects back that match the specified schema in the DirectorySchemaAttibute custom attribute (which is required) on the entity type. Finally, we're ready to prepare the query in line 14 and launch it in line 21. Finally, we iterate over the results and carry out a projection in case it's required. In here, an additional helper method AssignResultProperty is used to assign the properties in the resulting objects.

AssignResultProperty helper method - Copy Code
1 private void AssignResultProperty(Type helper, DirectoryEntry e, object result, string prop) 2 { 3 PropertyInfo i = originalType.GetProperty(prop); 4 DirectoryAttributeAttribute[] da = i.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 5 if (da != null && da.Length != 0) 6 { 7 if (da[0].Type == DirectoryAttributeType.ActiveDs) 8 { 9 PropertyInfo p = helper.GetProperty(da[0].Attribute, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase); 10 try 11 { 12 i.SetValue(result, p.GetValue(e.NativeObject, null), null); 13 } 14 catch (TargetInvocationException) { } 15 } 16 else 17 { 18 PropertyValueCollection pvc = e.Properties[da[0].Attribute]; 19 if (i.PropertyType.IsArray) 20 { 21 Array o = Array.CreateInstance(i.PropertyType.GetElementType(), pvc.Count); 22 23 int j = 0; 24 foreach (object oo in pvc) 25 o.SetValue(oo, j++); 26 27 i.SetValue(result, o, null); 28 } 29 else 30 if (pvc.Count == 1) 31 i.SetValue(result, pvc[0], null); 32 } 33 } 34 else 35 { 36 PropertyValueCollection pvc = e.Properties[prop]; 37 if (pvc.Count == 1) 38 i.SetValue(result, pvc[0], null); 39 } 40 } 41

This code uses quite a bit of reflection stuff to get the value of the properties (either from the LDAP properties through the DirectoryEntry instance or via the underlying IADs* type (e.NativeObject in line 12). In case an entity type member is an array, the code in lines 21 to 27 takes care of this. Anyhow, all cases above ultimately call SetValue for a property of the original type in order to supply its value.

 

Testing it

Now we're ready to execute queries against AD. You can download the resulting code over here. Notice that this isn't production quality code and is only meant as a sample. A few sample of queries are shown below:

A simple sample of LINQ-to-LDAP - Copy Code
1 using System; 2 using System.Linq; 3 using BdsSoft.DirectoryServices.Linq; 4 using System.DirectoryServices; 5 using ActiveDs; 6 7 namespace Demo 8 { 9 class Program 10 { 11 static void Main(string[] args) 12 { 13 var users = new DirectorySource<User>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); 14 users.Log = Console.Out; 15 var groups = new DirectorySource<Group>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); 16 groups.Log = Console.Out; 17 18 // 19 // Simple query with all-property projection (usr => usr). 20 // I.e. users.Select(usr => usr); 21 // 22 var res1 = from usr in users 23 select usr; 24 25 Console.WriteLine("QUERY 1\n======="); 26 foreach (var w in res1) 27 Console.WriteLine("{0}: {1} {2}", w.Name, w.Description, w.PasswordLastSet); 28 Console.WriteLine(); 29 30 // 31 // Query with selection criterion. 32 // I.e. users.Where(usr => usr.Name == "A*").Select(usr => usr); 33 // 34 var res2 = from usr in users 35 where usr.Name == "A*" 36 select usr; 37 38 Console.WriteLine("QUERY 2\n======="); 39 foreach (var w in res2) 40 Console.WriteLine("{0}'s full name is {1}", w.Name, w.Dn); 41 Console.WriteLine(); 42 43 // 44 // Query with selection criterion using a local variable and calling a method. 45 // Uses complicated string matching and projection using an anonymous type and a local variable. 46 // I.e. users.Where(usr => usr.Name == GetQueryStartWith("A") 47 // && usr.Description.Contains("Built-in")) 48 // .Select(usr => new { usr.Name, usr.Description, usr.Groups }); 49 // 50 var res3 = from usr in users 51 where usr.Name == GetQueryStartWith("A") && usr.Description.Contains("Built-in") 52 select new { usr.Name, usr.Description, usr.Groups, usr.LogonCount }; 53 54 Console.WriteLine("QUERY 3\n======="); 55 foreach (var w in res3) 56 { 57 Console.WriteLine("{0} has logged on {2} times and belongs to {1} groups:", w.Name, w.Groups.Length, w.LogonCount); 58 foreach (string group in w.Groups) 59 Console.WriteLine("- {0}", group); 60 } 61 Console.WriteLine(); 62 63 // 64 // Query with selection criterion using a local variable involved in a larger expression. 65 // Uses nested objects with anonymous types and expressions in initialization logic. 66 // Illustrates the LDAP query construction based on tree traversal. 67 // I.e. users.Where(usr => usr.Name.StartsWith("A") && usr.LogonCount > 1 * n) || usr.Name == "Guest") 68 // .Select(usr => new { usr.Name, usr.Description, usr.Dn, usr.PasswordLastSet, 69 // Stats = new { usr.PasswordLastSet, usr.LogonCount, TwiceLogonCount = usr.LogonCount * 2 } }); 70 // 71 var res4 = from usr in users 72 where (usr.Name.StartsWith("A") && usr.Name.EndsWith("strator")) || usr.Name == "Guest" 73 select new { usr.Name, usr.Description, usr.Dn, Stats = new { usr.PasswordLastSet, usr.LogonCount, TwiceLogonCount = usr.LogonCount * 2 } }; 74 75 Console.WriteLine("QUERY 4\n======="); 76 foreach (var w in res4) 77 Console.WriteLine("{0} has been logged on {1} times; password last set on {2}", w.Name, w.Stats.TwiceLogonCount - w.Stats.LogonCount, w.Stats.PasswordLastSet); 78 Console.WriteLine(); 79 80 // 81 // Query with sorting (not supported currently). 82 // I.e. users.OrderBy(usr => usr.Name).Select(usr => usr); 83 // 84 var res5 = (from usr in users 85 //orderby usr.Name ascending //not supported in LDAP; alternative in-memory sort 86 select usr).AsEnumerable().OrderBy(usr => usr.Name); 87 88 Console.WriteLine("QUERY 5\n======="); 89 foreach (var w in res5) 90 Console.WriteLine("{0}: {1}", w.Name, w.Description); 91 Console.WriteLine(); 92 93 // 94 // Query against groups in AD. 95 // I.e. groups.Where(grp => grp.Name.EndsWith("ators")).Select(grp => new { grp.Name, MemberCount = grp.Members.Length }); 96 // 97 var res6 = from grp in groups 98 where grp.Name.EndsWith("ators") 99 select new { grp.Name, MemberCount = grp.Members.Length }; 100 101 Console.WriteLine("QUERY 6\n======="); 102 foreach (var w in res6) 103 Console.WriteLine("{0} has {1} members", w.Name, w.MemberCount); 104 Console.WriteLine(); 105 } 106 107 static string GetQueryStartWith(string s) 108 { 109 return s + "*"; 110 } 111 } 112 113 [DirectorySchema("user", typeof(IADsUser))] 114 class User 115 { 116 private string name; 117 118 public string Name 119 { 120 get { return name; } 121 set { name = value; } 122 } 123 124 private string description; 125 126 public string Description 127 { 128 get { return description; } 129 set { description = value; } 130 } 131 132 private int logonCount; 133 134 public int LogonCount 135 { 136 get { return logonCount; } 137 set { logonCount = value; } 138 } 139 140 private DateTime pwdLastSet; 141 142 [DirectoryAttribute("PasswordLastChanged", DirectoryAttributeType.ActiveDs)] 143 public DateTime PasswordLastSet 144 { 145 get { return pwdLastSet; } 146 set { pwdLastSet = value; } 147 } 148 149 private string distinguishedName; 150 151 [DirectoryAttribute("distinguishedName")] 152 public string Dn 153 { 154 get { return distinguishedName; } 155 set { distinguishedName = value; } 156 } 157 158 private string[] memberOf; 159 160 [DirectoryAttribute("memberOf")] 161 public string[] Groups 162 { 163 get { return memberOf; } 164 set { memberOf = value; } 165 } 166 167 } 168 169 [DirectorySchema("group")] 170 class Group 171 { 172 private string name; 173 174 public string Name 175 { 176 get { return name; } 177 set { name = value; } 178 } 179 180 private string[] members; 181 182 [DirectoryAttribute("member")] 183 public string[] Members 184 { 185 get { return members; } 186 set { members = value; } 187 } 188 } 189 }

The resulting output is shown below - We did it!

In the next post, we'll deal with updates to Active Directory through entities. As an example, we'll be able to do the following:

// // Query with update functionality using an entity MyUser : DirectoryEntity. // string oldOffice = "Test"; string newOffice = "Demo"; var res7 = from usr in myusers where usr.Office == oldOffice select usr; Console.WriteLine("QUERY 7\n======="); foreach (var u in res7) { Console.WriteLine("{0} {1} works in {2}", u.FirstName, u.LastName, u.Office); u.Office = newOffice; } Console.WriteLine(); Console.WriteLine("Moving people to new office {0}...\n", newOffice); myusers.Update();

or even

// // Query with method call functionality using an entity MyUser : DirectoryEntity and a method SetPassword. // var res8 = from usr in myusers select usr; string newPassword = "Hello W0rld!"; Console.WriteLine("QUERY 8\n======="); foreach (var u in res8) { Console.WriteLine("Setting the password of {0} {1}...", u.FirstName, u.LastName); u.SetPassword(newPassword); }

Stay tuned!

Read on ... The IQueryable tales - LINQ to LDAP - Part 5: Supporting updates

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Filed under: ,

Comments

# The IQueryable tales - LINQ to LDAP - Part 3: Why do we need entities?

Tuesday, April 10, 2007 3:19 PM by B# .NET Blog

Introduction Welcome back to the LINQ-to-LDAP series. So far, we've been discussing: Part 0: Introduction

# The IQueryable tales - LINQ to LDAP - Part 5: Supporting updates

Tuesday, April 10, 2007 6:29 PM by B# .NET Blog

Introduction Welcome back to the LINQ-to-LDAP series. So far, we've been discussing: Part 0: Introduction

# New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 10:23 PM by ScottGu's Blog

Last month I started a series of posts covering some of the new VB and C# language features that are

# LINQ to LDAP - Implementation Details

Wednesday, April 11, 2007 12:02 PM by ((Research + Development) - Sleep) > 24

Today, Sam Gentile noted a series of posts by Bart De Smet describing (in great detail) how LINQ queries

# re: The IQueryable tales - LINQ to LDAP - Part 4: Parsing and executing queries

Wednesday, April 11, 2007 12:47 PM by SvetMy

Hi Bart!

How about transactional updates?

THanks,

Slava

# Coming soon - The return of IQueryable<T> - LINQ to SharePoint

Friday, April 13, 2007 4:09 PM by B# .NET Blog

Earlier this week, "The IQueryable Tales" were published on my blog, with great success. This series

# Building Custom LINQ Enabled Data Providers using IQueryable<T>

Friday, April 20, 2007 6:16 AM by Tom's MSDN Belux Corner

Bart De Smet wrote a hands-on tutorial that explains quite in-depth how one can build a custom data provider

# New "Orcas" Language Feature: Lambda Expressions

Tuesday, May 08, 2007 1:22 AM by Tyrannosaurus Rex

Last month I started a series of posts covering some of the new VB and C# language features that are

# Community Convergence XXVII

Sunday, May 13, 2007 11:13 PM by Charlie Calvert's Community Blog

Welcome to the 27th Community Convergence. I use this column to keep you informed of events in the C#

# The IQueryable tales - LINQ to LDAP - Part 5: Supporting updates - B# .NET Blog

Pingback from  The IQueryable tales - LINQ to LDAP - Part 5: Supporting updates - B# .NET Blog

# LINQ to Active Directory (formerly known as LINQ to LDAP) is here

Sunday, November 25, 2007 11:23 PM by B# .NET Blog

Within a few seconds from now I&#39;ll hit the &quot;Publish This Project&quot; button in CodePlex. Back

# LINQ to Active Directory (formerly known as LINQ to LDAP) is here

Sunday, November 25, 2007 11:24 PM by Elan Hasson's Favorite Blogs

Within a few seconds from now I&#39;ll hit the &quot;Publish This Project&quot; button in CodePlex. Back

# LINQ to Active Directory (formerly known as LINQ to LDAP) is here

Friday, December 21, 2007 7:44 AM by Developer Blogs

Within a few seconds from now I&#39;ll hit the &quot;Publish This Project&quot; button in CodePlex. Back