April 2007 - Posts

While preparing a little "Orcas" demo, I ended up with the System.IO.Pipes namespace that provides support for named pipes. In the past, I've been doing some named pipe work using unmanaged code and using interop, but in Orcas it becomes as simple as any other kind of I/O (if you know some basic concepts of named pipes though).

Below is a very little sample that shows how easy it is to use named pipes; it's part of a bigger demo that shows how to use pipes for (secure, using access control which is supported on named pipes too) communication between a service agent (in the taskbar notification area) and a Windows Service running in the background, even with cross-machine support. Here's the code of the "reduced" demo:

Named pipes for dummies - Copy Code
1 using System; 2 using System.IO; 3 using System.IO.Pipes; 4 using System.Threading; 5 6 class Program 7 { 8 static string NAME = "Demo"; 9 static ManualResetEvent evt = new ManualResetEvent(false); 10 11 static void Server() 12 { 13 using (NamedPipeServerStream server = new NamedPipeServerStream(NAME, PipeDirection.InOut)) 14 { 15 evt.Set(); 16 WriteGreen("Waiting for connection ..."); 17 server.WaitForConnection(); 18 WriteGreen("Connection established."); 19 20 using (StreamReader sr = new StreamReader(server)) 21 { 22 using (StreamWriter sw = new StreamWriter(server)) 23 { 24 int i = 0; 25 while (true) 26 { 27 i = int.Parse(sr.ReadLine()); 28 string s = Convert.ToString(i, 2); 29 WriteGreen("SERVER: Received {0}, sent {1}.", i, s); 30 sw.WriteLine(s); 31 sw.Flush(); 32 Thread.Sleep(1000); 33 } 34 } 35 } 36 } 37 } 38 39 static void Main(string[] args) 40 { 41 new Thread(Server).Start(); 42 evt.WaitOne(); 43 44 using (NamedPipeClientStream client = new NamedPipeClientStream(".", NAME, PipeDirection.InOut)) 45 { 46 WriteRed("Connecting to server ..."); 47 client.Connect(); 48 WriteRed("Connected."); 49 50 using (StreamWriter sw = new StreamWriter(client)) 51 { 52 using (StreamReader sr = new StreamReader(client)) 53 { 54 Random rand = new Random(); 55 56 while (true) 57 { 58 int i = rand.Next(1000); 59 sw.WriteLine(i); 60 sw.Flush(); 61 string s = sr.ReadLine(); 62 WriteRed("CLIENT: Sent {0}, received {1}.", i, s); 63 } 64 } 65 } 66 } 67 } 68 69 static void WriteRed(string msg, params object[] p) 70 { 71 Console.ForegroundColor = ConsoleColor.Red; 72 Console.WriteLine(msg, p); 73 Console.ResetColor(); 74 } 75 76 static void WriteGreen(string msg, params object[] p) 77 { 78 Console.ForegroundColor = ConsoleColor.Green; 79 Console.WriteLine(msg, p); 80 Console.ResetColor(); 81 } 82 }

And here's the output:

Another reason to take a closer look at Orcas... Beta coming soon!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

 

Introduction

Welcome back to the LINQ-to-LDAP series. So far, we've been discussing:

In the previous post, we managed to translate a query expression from the LINQ domain to the LDAP domain, allowing retrieval of directory objects in a quite flexible way. One piece is missing however: updating the retrieved entity objects and feeding updates back to Active Directory. In this post, we'll show how this can be made possible.

 

Revised entities

Essentially, when retrieving objects through LINQ-to-LDAP, we can support updates to entities if a few conditions are met:

  1. The objects returned should be the entities themselves; we won't support the case where projections are made. Or, in other words, the query should end with a plain vanilla "select <dummy>" clause.
  2. Entity objects should be "improved" to make them more intelligent, allowing to track changes (i.e. property setter calls).

Luckily, .NET's component model comes to our rescue by providing a concept of "property changes", in the INotifyPropertyChanged interface that's defined as follows:

namespace System.ComponentModel { // Summary: // Notifies clients that a property value has changed. public interface INotifyPropertyChanged { // Summary: // Occurs when a property value changes. event PropertyChangedEventHandler PropertyChanged; } }

Based on this interface, we'll create a so-called DirectoryEntity class that acts as the root class for entities that require update support:

DirectoryEntity class to provide update support - Copy Code
1 public class DirectoryEntity : INotifyPropertyChanged 2 { 3 private DirectoryEntry directoryEntry; 4 5 protected internal DirectoryEntry DirectoryEntry 6 { 7 get { return directoryEntry; } 8 set { directoryEntry = value; } 9 } 10 11 public event PropertyChangedEventHandler PropertyChanged; 12 13 protected void OnPropertyChanged(string propertyName) 14 { 15 if (PropertyChanged != null) 16 PropertyChanged(this, new PropertyChangedEventArgs(propertyName)); 17 } 18 }

This class provides an event PropertyChanged (through the INotifyPropertyChanged interface) to notify whenever a property is changed and which will be called through the OnPropertyChanged method. Beside of this, a DirectoryEntry instance is kept, in order to be able feeding back changes to AD.

Next, we'll revise the entities like this:

Entity type for user objects with update support - Copy Code
1 [DirectorySchema("user")] 2 class MyUser : DirectoryEntity 3 { 4 private DateTime expiration; 5 6 [DirectoryAttribute("AccountExpirationDate", DirectoryAttributeType.ActiveDs)] 7 public DateTime AccountExpirationDate 8 { 9 get { return expiration; } 10 set 11 { 12 if (expiration != value) 13 { 14 expiration = value; 15 OnPropertyChanged("AccountExpirationDate"); 16 } 17 } 18 } 19 20 private string first; 21 22 [DirectoryAttribute("givenName")] 23 public string FirstName 24 { 25 get { return first; } 26 set 27 { 28 if (first != value) 29 { 30 first = value; 31 OnPropertyChanged("FirstName"); 32 } 33 } 34 } 35 36 private string last; 37 38 [DirectoryAttribute("sn")] 39 public string LastName 40 { 41 get { return last; } 42 set 43 { 44 if (last != value) 45 { 46 last = value; 47 OnPropertyChanged("LastName"); 48 } 49 } 50 } 51 52 private string office; 53 54 [DirectoryAttribute("physicalDeliveryOfficeName")] 55 public string Office 56 { 57 get { return office; } 58 set 59 { 60 if (office != value) 61 { 62 office = value; 63 OnPropertyChanged("Office"); 64 } 65 } 66 } 67 68 private string accoutName; 69 70 [DirectoryAttribute("sAMAccountName")] 71 public string AccountName 72 { 73 get { return accoutName; } 74 set 75 { 76 if (accoutName != value) 77 { 78 accoutName = value; 79 OnPropertyChanged("AccountName"); 80 } 81 } 82 } 83 84 public bool SetPassword(string password) 85 { 86 return this.DirectoryEntry.Invoke("SetPassword", new object[] { password }) == null; 87 } 88 }

Now, we're out of Automatic Property luck, so we'll write complete property definitions together with a private field. Notice the major change being the presence of OnPropertyChanged calls in the property setters. Also, the class derives from DirectoryEntity and it also contains a method that allows invoking operations on the object through the corresponding DirectoryEntry object, for example to set a password (SetPassword).

To make the creation of these entity properties easier, you could create your own code snippet:

Code snippet for entity properties with change notification support - Copy Code
<?xml version="1.0" encoding="utf-8" ?> <CodeSnippets xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet"> <CodeSnippet Format="1.0.0"> <Header> <Title>entityprop</Title> <Shortcut>entityprop</Shortcut> <Description>Code snippet for entity property creation with change notification support</Description> <Author>Bart De Smet</Author> <SnippetTypes> <SnippetType>Expansion</SnippetType> </SnippetTypes> </Header> <Snippet> <Declarations> <Literal> <ID>type</ID> <ToolTip>Property type</ToolTip> <Default>int</Default> </Literal> <Literal> <ID>property</ID> <ToolTip>Property name</ToolTip> <Default>MyProperty</Default> </Literal> <Literal> <ID>field</ID> <ToolTip>The variable backing this property</ToolTip> <Default>myVar</Default> </Literal> </Declarations> <Code Language="csharp"><![CDATA[private $type$ $field$; public $type$ $property$ { get { return $field$;} set { if ($field$ != value) { $field$ = value; OnPropertyChanged("$property$"); } } } $end$]]> </Code> </Snippet> </CodeSnippet> </CodeSnippets>

Store it in your %profile%\My Documents\Visual Studio Codename Orcas\Code Snippets\Visual C#\My Code Snippets folder with the name entityprop.snippet:

and you'll be able to write entity properties easily by writing entityprop and pressing TAB twice:

 

Making DirectorySource<T> updatable

Now it's time to make DirectorySource<T> update-aware. We'll start by adding the following method:

Update method - Copy Code
1 private Dictionary<object, HashSet<string>> updates = new Dictionary<object, HashSet<string>>(); 2 3 public void Update() 4 { 5 Type t = typeof(T); 6 DirectorySchemaAttribute[] attr = (DirectorySchemaAttribute[])t.GetCustomAttributes(typeof(DirectorySchemaAttribute), false); 7 8 foreach (var e in updates) 9 { 10 if (e.Key is T && e.Key is DirectoryEntity) 11 { 12 DirectoryEntry entry = ((DirectoryEntity)e.Key).DirectoryEntry; 13 foreach (string property in e.Value) 14 { 15 PropertyInfo i = t.GetProperty(property); 16 17 DirectoryAttributeAttribute[] da = i.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 18 if (da != null && da.Length != 0 && da[0] != null) 19 { 20 if (da[0].Type == DirectoryAttributeType.ActiveDs) 21 { 22 if (attr != null && attr.Length != 0) 23 attr[0].ActiveDsHelperType.GetProperty(da[0].Attribute).SetValue(entry.NativeObject, i.GetValue(e.Key, null), null); 24 else 25 throw new InvalidOperationException("Missing schema mapping attribute for updates through ADSI."); 26 } 27 else 28 entry.Properties[da[0].Attribute].Value = i.GetValue(e.Key, null); 29 } 30 else 31 entry.Properties[i.Name].Value = i.GetValue(e.Key, null); 32 } 33 entry.CommitChanges(); 34 } 35 else 36 throw new InvalidOperationException("Can't apply update because updates type doesn't match original entity type."); 37 } 38 39 updates.Clear(); 40 }

The private member update on line 1 will hold mappings between entity objects (represented as "object") and a list of property names that were updated since results were retrieved. The code for the Update method does the following for each update (line 8 to 37):

  • Retrieves information about the property for the given property name on line 15.
  • Inspects the DirectoryAttributeAttribute custom attribute (if present) to find out which value to set. Ultimately, either the IADs* native object will be touched (line 23) or the LDAP property will be updated directly (line 28 and line 31).
  • Finally, changes are committed on line 33.

This logic will only process DirectoryEntity objects (see line 10).

Note: This centralized approach to updates could be extended with update support on the entities themselves. Our approach is to "batch" together updates, however only on the conceptual level since LDAP only supports "atomic" updates of individual objects (on the DirectoryEntry level in managed code terms). Nevertheless, allowing updates to be made on the entity level itself would be a nice addition (guidelines: register for the update notification event in the DirectoryEntity class itself and provide an "update" collector with a local HashSet to keep the updated fields; finally, add an Update method that is more or less equivalent to the one above).

Next, CreateQuery needs a change to support updates as shown in lines 13-14 below:

Updates to CreateQuery - Copy Code
1 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 2 { 3 // 4 // Create a new queryable object based on the current one. A copy is needed to guarantee uniqueness across multiple queries. 5 // 6 DirectorySource<TElement> d = new DirectorySource<TElement>(searchRoot, searchScope); 7 d.query = query; 8 d.project = project; 9 d.properties = properties; 10 d.originalType = originalType; 11 d.logger = logger; 12 13 // *** UPDATE *** 14 d.updates = updates; 15 16 // 17 // We expect a method call expression for the chain of LINQ query operator method calls. 18 // 19 MethodCallExpression call = expression as MethodCallExpression;

This change takes care of passing the update dictionary reference. This is important because updates will be applied via the original "directory source" while the update notifications will be added via other instances of the DirectorySource possible (because of the way the CreateQuery method works, i.e. returning new instances of the type all the time). In a similar way, we only want to keep one update dictionary that collects all updates made by possibly more than one query (e.g. you change a user's Office in a first loop over the query results for users in specific offices and next you change the account expiration day for another set of entities that were retrieved with another query from the same directory source).

The next change needs to be applied in GetResults's main loop as shown below:

Updates to GetResults main loop - Copy Code
1 foreach (SearchResult sr in s.FindAll()) 2 { 3 DirectoryEntry e = sr.GetDirectoryEntry(); 4 5 object result = Activator.CreateInstance(project == null ? typeof(T) : originalType); 6 7 /// *** UPDATE *** 8 DirectoryEntity entity = result as DirectoryEntity; 9 if (entity != null) 10 entity.DirectoryEntry = e; 11 12 if (project == null) 13 { 14 foreach (PropertyInfo p in typeof(T).GetProperties()) 15 AssignResultProperty(helper, e, result, p.Name); 16 17 /// *** UPDATE *** 18 if (entity != null) 19 entity.PropertyChanged += new PropertyChangedEventHandler(UpdateNotification); 20 21 yield return (T)result; 22 } 23 else 24 { 25 foreach (string prop in properties) 26 AssignResultProperty(helper, e, result, prop); 27 28 /// *** UPDATE *** 29 if (entity != null) 30 entity.PropertyChanged += new PropertyChangedEventHandler(UpdateNotification); 31 32 yield return (T)project.DynamicInvoke(result); 33 } 34 }

This code first checks whether or not the result object is a DirectoryEntity, thus allows updates to be made. If so, an event handler for PropertyChanged is hooked up in line 19 or line 30. The UpdateNotification method is defined like this:

UpdateNotification event handler - Copy Code
1 void UpdateNotification(object sender, PropertyChangedEventArgs e) 2 { 3 T source = (T)sender; 4 5 if (!updates.ContainsKey(source)) 6 updates.Add(source, new HashSet<string>()); 7 8 updates[source].Add(e.PropertyName); 9 }
This is it!

 

A few samples

Let's show a few queries with updating logic to show the power of this implementation. The first sample shows how to move people to other offices:

Move users to other office
var myusers = new DirectorySource<MyUser>(new DirectoryEntry("LDAP://localhost/OU=Demo,DC=linqdemo,DC=local"), SearchScope.Subtree); // // Query with update functionality using an entity MyUser : DirectoryEntity. // string oldOffice = "Test"; string newOffice = "Demo"; var res7 = from usr in myusers where usr.Office == oldOffice select usr; Console.WriteLine("QUERY 7\n======="); foreach (var u in res7) { Console.WriteLine("{0} {1} works in {2}", u.FirstName, u.LastName, u.Office); u.Office = newOffice; } Console.WriteLine(); Console.WriteLine("Moving people to new office {0}...\n", newOffice); myusers.Update(); int k = 0; foreach (var u in res7) //should be empty now Console.WriteLine("{0} {1} still works in {2}", u.FirstName, u.LastName, u.Office, k++); if (k == 0) Console.WriteLine("No results returned."); //expected case Console.WriteLine();

The result is illustrated below (before, after):

   

Notice that line 25 re-executes the query, which shows the behavior of the lazy execution model. You could implement a slightly other behavior by caching the results obtained in the first iteration over the result set, in order to speed up subsequent iterations. Of course, you'll need support to invalidate the cache in such a case, either by a manual method call or by getting update notifications back from the server in the background (like SQL Server 2005 can do with update notifications and database dependencies).

If you change the query like this:

var res7 = from usr in myusers where usr.Office == oldOffice select new { usr.FirstName, usr.LastName, usr.Office };

update code won't work anymore, since we're not touching the entity objects but instances of "some anonymous type" generated by the compiler when it encountered the new { ... } anonymous type syntax. The code doesn't throw an exception though; you should simply know that this won't update the data source. Therefore, only projections of type "select <dummy>" will yield updatable entity objects.

Another sample is to call SetPassword for every user. The code below does this and checks for validity of the new password to verify the result:

Changing user passwords - Copy Code
1 var res8 = from usr in myusers 2 select usr; 3 4 string newPassword = "Hello W0rld!"; 5 6 Console.WriteLine("QUERY 8\n======="); 7 foreach (var u in res8) 8 { 9 Console.Write("Setting the password of {0} {1}... ", u.FirstName, u.LastName); 10 if (u.SetPassword(newPassword)) 11 { 12 Console.WriteLine("Done."); 13 Console.Write("Validating password... "); 14 try 15 { 16 new DirectoryEntry("LDAP://localhost", u.AccountName, newPassword).RefreshCache(); 17 Console.WriteLine("Successful."); 18 } 19 catch (DirectoryServicesCOMException ex) 20 { 21 Console.ForegroundColor = ConsoleColor.Red; 22 Console.WriteLine(ex.Message); 23 Console.ResetColor(); 24 } 25 } 26 else 27 Console.WriteLine("Failed."); 28 } 29 Console.WriteLine(); 30 31 myusers.Update();

Together with a random password generator you could use this code fragment to reset a set of user passwords. The RefreshCache call on line 16 is a simple trick to make sure that a request is made to the Active Directory domain controller, in order to force an exception to be thrown when the password is invalid. Fortunately, the code runs correctly :-).

Finally, a little sample to show that updating through ADSI (IADs*) properties works fine too:

Expiring accounts - Copy Code
1 var res9 = from usr in myusers 2 select usr; 3 4 Console.WriteLine("QUERY 9\n======="); 5 foreach (var u in res9) 6 u.AccountExpirationDate = DateTime.Now.AddDays(30); 7 Console.WriteLine(); 8 9 myusers.Update();

With the following result:

   

Have fun! You can download the resulting code over here. Notice that this isn't production quality code and is only meant as a sample. Verify all updates before executing the code or you could harm your domain infrastructure.

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

Welcome back to the LINQ-to-LDAP series. So far, we've been discussing:

In the previous post, we entered the domain of implementing a custom query provider for LINQ. More specifically, we did discuss the need for entities and came up with a class definition that maps an object from the underlying data source, which is Active Directory in our case. Such an entity has the following shape:

An entity type for user objects in Active Directory - Copy Code
1 [DirectorySchema("user")] 2 public class User 3 { 4 [DirectoryAttribute("givenName")] 5 public string FirstName { get; set; } 6 7 [DirectoryAttribute("sn")] 8 public string LastName { get; set; } 9 10 [DirectoryAttribute("physicalDeliveryOfficeName")] 11 public string Office { get; set; } 12 }

Today we (finally) take it another stpe forward, focusing on the LINQ-to-LDAP translation when writing queries like:

A LINQ query against Active Directory - Copy Code
1 DirectorySource<User> users = new DirectorySource<User>(...); 2 var res = from usr in users 3 where usr.Office == "Building 10 - 1.25" 4 select usr.FirstName + " " + usr.LastName;

In technical terms, we'll make a first implementation of IQueryable<T>: DirectorySource<T>.

 

Back to the attributes

In the previous post, two custom attributes were introduced to perform the mapping between entities and the underlying directory objects. Before moving on, I want to extend those a little more to support more complex data types. For instance, attributes like pwdLastSet are retrieved by DirectoryEntry's Properties collections as a COM-object. We could try to do a conversion to the more convenient DateTime BCL type ourselves, but instead a COM library called "Active DS Type Library" proves handy since it exposes properties in the right .NET type. To use this library, add a reference to the Active DS Type Library via Add Reference..., tab COM.

When you take a look inside the library using the Object Browser, you'll find a series of interfaces starting with IADs. One of these is IADsUser:

As you can see, the PasswordLastChanges property is of type System.DateType, allowing us to use it right away without having to mess around with COM-to-.NET conversions for complex types. You might wonder why we don't just use this class for all our entity stuff. The answer is that we want to provide the end-users with the maximum level of flexibility. Therefore, we'll both support "LDAP attribute names" as well as property names from the IADs* interfaces. So, how can we reflect these feature requests in the custom attribute definitions? The answer is below.

Note: In future posts, we'll talk even more about entities, when we want to support updates too. In order to make this possible, we'll have to track changes to retrieved objects in order to feed these back when making an Update call to the DirectorySource<T> data source. However, some properties don't support updating because of their read-only nature (PasswordLastChanged is one like this). Therefore, the entity type will need to reflect this.

Mapping custom attributes for entity objects - Copy Code
1 /// <summary> 2 /// Specifies the directory schema to query. 3 /// </summary> 4 [AttributeUsage(AttributeTargets.Class, AllowMultiple = false)] 5 public class DirectorySchemaAttribute : Attribute 6 { 7 private string schema; 8 private Type helper; 9 10 /// <summary> 11 /// Creates a new schema indicator attribute. 12 /// </summary> 13 /// <param name="schema">Name of the schema to query for.</param> 14 public DirectorySchemaAttribute(string schema) 15 { 16 this.schema = schema; 17 } 18 19 /// <summary> 20 /// Creates a new schema indicator attribute. 21 /// </summary> 22 /// <param name="schema">Name of the schema to query for.</param> 23 /// <param name="activeDsHelperType">Helper type for Active DS object properties.</param> 24 public DirectorySchemaAttribute(string schema, Type activeDsHelperType) 25 { 26 this.schema = schema; 27 this.helper = activeDsHelperType; 28 } 29 30 /// <summary> 31 /// Name of the schema to query for. 32 /// </summary> 33 public string Schema 34 { 35 get { return schema; } 36 set { schema = value; } 37 } 38 39 /// <summary> 40 /// Helper type for Active DS object properties. 41 /// </summary> 42 public Type ActiveDsHelperType 43 { 44 get { return helper; } 45 set { helper = value; } 46 } 47 } 48 49 /// <summary> 50 /// Specifies the underlying attribute to query for in the directory. 51 /// </summary> 52 [AttributeUsage(AttributeTargets.Property, AllowMultiple = false)] 53 public class DirectoryAttributeAttribute : Attribute 54 { 55 private string attribute; 56 private DirectoryAttributeType type; 57 58 /// <summary> 59 /// Creates a new attribute binding attribute for a entity class field or property. 60 /// </summary> 61 /// <param name="attribute">Name of the attribute to query for.</param> 62 public DirectoryAttributeAttribute(string attribute) 63 { 64 this.attribute = attribute; 65 this.type = DirectoryAttributeType.Ldap; 66 } 67 68 /// <summary> 69 /// Creates a new attribute binding attribute for a entity class field or property. 70 /// </summary> 71 /// <param name="attribute">Name of the attribute to query for.</param> 72 /// <param name="type">Type of the underlying query source to get the attribute from.</param> 73 public DirectoryAttributeAttribute(string attribute, DirectoryAttributeType type) 74 { 75 this.attribute = attribute; 76 this.type = type; 77 } 78 79 /// <summary> 80 /// Name of the attribute to query for. 81 /// </summary> 82 public string Attribute 83 { 84 get { return attribute; } 85 set { attribute = value; } 86 } 87 88 /// <summary> 89 /// Type of the underlying query source to get the attribute from. 90 /// </summary> 91 public DirectoryAttributeType Type 92 { 93 get { return type; } 94 set { type = value; } 95 } 96 } 97 98 /// <summary> 99 /// Type of the query source to perform queries with. 100 /// </summary> 101 public enum DirectoryAttributeType 102 { 103 /// <summary> 104 /// Default value. Uses the Properties collection of DirectoryEntry to get data from. 105 /// </summary> 106 Ldap, 107 108 /// <summary> 109 /// Uses Active DS Helper IADs* objects to get data from. 110 /// </summary> 111 ActiveDs 112 }

The custom attribute definition from above allows us to write things like this:

An entity with complex mapping - Copy Code
1 [DirectorySchema("user", typeof(IADsUser))] 2 public class User 3 { 4 [DirectoryAttribute("givenName")] 5 public string FirstName { get; set; } 6 7 [DirectoryAttribute("sn")] 8 public string LastName { get; set; } 9 10 [DirectoryAttribute("physicalDeliveryOfficeName")] 11 public string Office { get; set; } 12 13 [DirectoryAttribute("PasswordLastChanged", DirectoryAttributeType.ActiveDs)] 14 public DateTime PasswordLastSet { get; set; } 15 }

In this sample, a property has been defined (line 13-14) that relies on the ActiveDs property name instead of the LDAP property name, which has been indicated using DirectoryAttributeType.ActiveDs. Notice that the PasswordLastSet property has both a getter and setter accessor defined although it is read-only in the underlying IADsUser type definition; the setter is required for our framework to assign the retrieved value to it. In order for the framework to find the corresponding IADs* type for the entity object, an additional parameter has been added to the DirectorySchema attribute in line 1 (typeof(IADsUser)). This set of information will be enough to compose the query and get the values back via the desired "channel".

Furthermore, we'll also allow entity types to omit the DirectoryAttribute decorations if no renaming has to happen:

Copy Code
1 [DirectorySchema("user")] 2 public class User 3 { 4 public string GivenName { get; set; } 5 public string Sn { get; set; } 6 }

In here, the GivenName and Sn properties have the same name as their LDAP attribute name equivalent (which is case-insensitive by the way), so the system can figure out the mapping on its own. We'll support such an automatic mapping only for LDAP attribute names, not for IADs* property names.

 

The skeleton

On to the real work now; the DirectorySource<T> implementation. Let's start by revisiting our IQueryable<T> skeleton implementation. Since we won't support sorting right now, we won't derive from IOrderedQueryable<T>. An implementation of sorting is possible though, using a IQueryable<T>-to-IEnumerable<T> translation, followed by LINQ-to-Objects sorting, when yielding the results back. This discussion would lead us too far, so let's move on with the basics for now:

Basic skeleton of IQueryable<T> implementation - Copy Code
1 /// <summary> 2 /// Represents an LDAP data source. Allows for querying the LDAP data source via LINQ. 3 /// </summary> 4 /// <typeparam name="T">Entity type in the underlying source.</typeparam> 5 public class DirectorySource<T> : IQueryable<T> 6 { 7 /// <summary> 8 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 9 /// </summary> 10 /// <param name="expression">Expression representing the LDAP query.</param> 11 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 12 public IQueryable CreateQuery(Expression expression) 13 { 14 return CreateQuery<T>(expression); 15 } 16 17 /// <summary> 18 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 19 /// </summary> 20 /// <param name="expression">Expression representing the LDAP query.</param> 21 /// <typeparam name="TElement">The type of the elements of the IQueryable that is returned.</typeparam> 22 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 23 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 24 { 25 // 26 // TODO 27 // 28 } 29 30 #region Execution (not implemented) 31 32 public object Execute(Expression expression) 33 { 34 throw new NotImplementedException(); 35 } 36 37 public TResult Execute<TResult>(Expression expression) 38 { 39 throw new NotImplementedException(); 40 } 41 42 #endregion 43 44 #region Enumeration 45 46 IEnumerator IEnumerable.GetEnumerator() 47 { 48 return GetEnumerator(); 49 } 50 51 public IEnumerator<T> GetEnumerator() 52 { 53 // 54 // TODO 55 // 56 } 57 58 #endregion 59 60 public Type ElementType 61 { 62 get { return typeof(T); } 63 } 64 65 public Expression Expression 66 { 67 get { return Expression.Constant(this); } 68 } 69 }

This basic skeleton is based on the one shown in Part 2 of this series.

A first concern is the constructor, which should collect enough information to be able sending the query to AD. In order to allow a maximum level of flexibility, we're going to support passing in a DirectoryEntry representing the node to start the search from (remember that directory services implementations are tree based, in which distinguished names - aka DNs - are put together by various parts representing domain names, OUs, etc) together with a SearchScope. The latter argument is required to allows users to specify how the search has to be performed; it can be a base-level search (in which only the current node is searched, so at maximum one result - the node itself - can be returned; maybe not that useful) or a subtree search (the whole subtree below the current node) or a "one level" deep search (only searching the childs of the current node). The constructor is defined below:

Constructor for DirectorySource<T> - Copy Code
1 /// <summary> 2 /// Creates a new data source instance for the given directory search root and with a given search scope. 3 /// </summary> 4 /// <param name="searchRoot">Root location in the directory to start all searches from.</param> 5 /// <param name="searchScope">Search scope for all queries performed through this data source.</param> 6 public DirectorySource(DirectoryEntry searchRoot, SearchScope searchScope) 7 { 8 this.searchRoot = searchRoot; 9 this.searchScope = searchScope; 10 }

This requires two member attributes:

Private members related to the constructor's parameterization - Copy Code
1 #region Directory information 2 3 private DirectoryEntry searchRoot; 4 private SearchScope searchScope; 5 6 #endregion

As another part of the "skeleton", let's add a property to support logging, in a similar fashion as the LINQ-to-SQL API does:

Logger support - Copy Code
1 private TextWriter logger; 2 3 /// <summary> 4 /// Used to configure a logger to print diagnostic information about the query performed. 5 /// </summary> 6 public TextWriter Log 7 { 8 get { return logger; } 9 set { logger = value; } 10 }

 

Implementing the parsing logic

Next, we move on to the CreateQuery<TElement> implementation itself. In part 2 of this series, we've shown the rationale behind CreateQuery and how several query-related operations result in a chain of CreateQuery calls to be made. The signature of CreateQuery is shown below as a quick refresh:

// Summary: // Constructs an System.Linq.IQueryable<T> object that can evaluate the query // represented by the specified expression tree. // // Parameters: // expression: // The System.Linq.Expressions.Expression representing the query to be encompassed. // // Returns: // An System.Linq.IQueryable<T> that can evaluate the query represented by the // specified expression tree. IQueryable<TElement> CreateQuery<TElement>(Expression expression);

This method is part of IQueryable<T> and transforms it into a new IQueryable<TElement>. For example, when doing filtering with a Where clause, T and TElement will be the same because of the Queryable.Where's signature:

// // Summary: // Filters a sequence of values based on a predicate. // // Parameters: // predicate: // An System.Linq.Expressions.Expression<TDelegate> that represents a function // that takes a value of type TSource and returns a System.Boolean to use to // test each element. // // source: // A System.Linq.IQueryable<T> to filter. // // Returns: // An System.Linq.IQueryable<T> that contains elements from the input sequence // which satisfy the condition specified by predicate. public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate);

However, when doing a projection, T and TElement will differ, as illustrated below by inspection of the Select method:

// // Summary: // Projects each element of a sequence into a new form. // // Parameters: // source: // A sequence of values to invoke a selector function on. // // selector: // An System.Linq.Expressions.Expression<TDelegate> that represents a generic // function to apply to each element. // // Returns: // An System.Linq.IQueryable<T> whose elements are the result of invoking a // selector function on each element of source. public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector);

During the chain of CreateQuery calls we need to gather enough information to carry out the query when asked to do so, in the GetEnumerator method. Each time CreateQuery is called, we'll create a new instance of the DirectorySource<TElement> class that will gather additional information about the query. Enough theory, let's introduce a few private class members that carry query information:

Query information members - Copy Code
1 #region Query information 2 3 private string query; 4 5 #endregion 6 7 #region Projection information 8 9 private HashSet<string> properties = new HashSet<string>(); 10 private Delegate project; 11 12 #endregion 13 14 private Type originalType = typeof(T);

The query member will capture the LDAP filter expression that needs to be sent to the AD server to perform the query. To optimize the query, we'll collect all of the AD object attributes that will make up the propertiesToLoad parameter to the DirectorySearcher type, in the properties set. Furthermore, a delegate to dynamically compiled code for the projection is kept in project and finally we keep track of the original type of the query objects, as we'll discuss later. We'll start with the CreateQuery method definition now:

CreateQuery implementation - Copy Code
1 /// <summary> 2 /// Constructs an IQueryable object that can evaluate the query represented by the specified expression tree. 3 /// </summary> 4 /// <param name="expression">Expression representing the LDAP query.</param> 5 /// <typeparam name="TElement">The type of the elements of the IQueryable that is returned.</typeparam> 6 /// <returns>IQueryable object that can evaluate the query represented by the specified expression tree.</returns> 7 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 8 { 9 // 10 // Create a new queryable object based on the current one. A copy is needed to guarantee uniqueness across multiple queries. 11 // 12 DirectorySource<TElement> d = new DirectorySource<TElement>(searchRoot, searchScope); 13 d.query = query; 14 d.project = project; 15 d.properties = properties; 16 d.originalType = originalType; 17 d.logger = logger; 18 19 // 20 // We expect a method call expression for the chain of LINQ query operator method calls. 21 // 22 MethodCallExpression call = expression as MethodCallExpression; 23 if (call != null) 24 { 25 // 26 // First parameter to the method call represents the (unary) lambda in LINQ style. 27 // E.g. (user => user.Name == "Bart") for a Where clause 28 // (user => new { user.Name }) for a Select clause 29 // 30 switch (call.Method.Name) 31 { 32 // 33 // Builds the query LDAP expression. 34 // 35 case "Where": 36 d.BuildQuery(((UnaryExpression)call.Arguments[1]).Operand as LambdaExpression); 37 break; 38 // 39 // Builds the projection and filters the required properties. 40 // 41 case "Select": 42 d.BuildProjection(((UnaryExpression)call.Arguments[1]).Operand as LambdaExpression); 43 break; 44 } 45 } 46 47 return d; 48 }

A quick analysis:

  • On lines 12 to 17, we create a new instance of DirectorySource for the TElement "output".
  • As discussed in Part 2, all of the CreateQuery calls originate from the Queryable.* methods that add themselves as method call expressions to the expression tree. On line 22 we obtain that call expression.
  • Next, we switch on the method call being made, which we expect to be either "Where" or "Select" for basic queries. If you want to provide some way to support sorting, you could also capture method calls to OrderBy, ThenBy, OrderByDescending and ThenByDescending. Similarly, you could intercept all method call expressions for all of the Queryable-methods. Finally, calls are made to BuildQuery or BuildProjection on the newly created object, as explained below.

 

Translating filters to LDAP query expressions

Now, let's move our focus on the query parsing itself, i.e. the Select case. The BuildQuery method takes care of this:

BuildQuery method - Copy Code
1 /// <summary> 2 /// Helper method to build the LDAP query. 3 /// </summary> 4 /// <param name="q">Lambda expression to be translated to LDAP.</param> 5 private void BuildQuery(LambdaExpression q) 6 { 7 StringBuilder sb = new StringBuilder(); 8 9 // 10 // Recursive tree traversal to build the LDAP query (prefix notation). 11 // 12 ParseQuery(q.Body, sb); 13 14 query = sb.ToString(); 15 }

This method prepares an "accumulator" object in the form of a string builder. The ParseQuery method will build the query recursively, spitting its output in the StringBuilder instance, and the result is finally kept in the query private member of the object, ready for comsumption at a later stage when GetEnumerator is called (see further). Here's ParseQuery:

Recursive query parsing - Copy Code
1 /// <summary> 2 /// Recursive helper method for query parsing based on the given expression tree. 3 /// </summary> 4 /// <param name="e">Expression tree to be translated to LDAP.</param> 5 /// <param name="sb">Accummulative query string used in recursion.</param> 6 private void ParseQuery(Expression e, StringBuilder sb) 7 { 8 sb.Append("("); 9 // 10 // Support for boolean operators & and |. Support for "raw" conditions (like equality). 11 // 12 if (e is BinaryExpression) 13 { 14 BinaryExpression c = e as BinaryExpression; 15 switch (c.NodeType) 16 { 17 case ExpressionType.AndAlso: 18 sb.Append("&"); 19 ParseQuery(c.Left, sb); 20 ParseQuery(c.Right, sb); 21 break; 22 case ExpressionType.OrElse: 23 sb.Append("|"); 24 ParseQuery(c.Left, sb); 25 ParseQuery(c.Right, sb); 26 break; 27 default: //E.g. Equal, NotEqual, GreaterThan 28 sb.Append(GetCondition(c)); 29 break; 30 } 31 } 32 // 33 // Support for boolean negation. 34 // 35 else if (e is UnaryExpression) 36 { 37 UnaryExpression c = e as UnaryExpression; 38 if (c.NodeType == ExpressionType.Not) 39 { 40 sb.Append("!"); 41 ParseQuery(c.Operand, sb); 42 } 43 else 44 throw new NotSupportedException("Unsupported query operator detected: " + c.NodeType); 45 } 46 // 47 // Support for string operations. 48 // 49 else if (e is MethodCallExpression) 50 { 51 MethodCallExpression m = e as MethodCallExpression; 52 MemberExpression o = (m.Object as MemberExpression); 53 if (m.Method.DeclaringType == typeof(string)) 54 { 55 switch (m.Method.Name) 56 { 57 case "Contains": 58 { 59 ConstantExpression c = m.Arguments[0] as ConstantExpression; 60 sb.AppendFormat("{0}=*{1}*", GetFieldName(o.Member), c.Value); 61 break; 62 } 63 case "StartsWith": 64 { 65 ConstantExpression c = m.Arguments[0] as ConstantExpression; 66 sb.AppendFormat("{0}={1}*", GetFieldName(o.Member), c.Value); 67 break; 68 } 69 case "EndsWith": 70 { 71 ConstantExpression c = m.Arguments[0] as ConstantExpression; 72 sb.AppendFormat("{0}=*{1}", GetFieldName(o.Member), c.Value); 73 break; 74 } 75 default: 76 throw new NotSupportedException("Unsupported string filtering query expression detected. Cannot translate to LDAP equivalent."); 77 } 78 } 79 else 80 throw new NotSupportedException("Unsupported query expression detected. Cannot translate to LDAP equivalent."); 81 } 82 else 83 throw new NotSupportedException("Unsupported query expression detected. Cannot translate to LDAP equivalent."); 84 sb.Append(")"); 85 }

This method requires a bit more explanation:

  • On line 1 and 84, the enclosing braces for the query expression are added; as you saw in Part 3, LDAP queries have a prefix notiation in which each part of the query is enclosed in parentheses.
  • Lines 12-31 covers the case of a binary expression, which can be quite a lot of things. Two common cases are the && operator and the || operator, which are translated into the LDAP equivalents & and | respectively, followed by a recursive call for both sides of the expression (left and right). The final case of the switch-operator covers the remainder of cases, such as usr.Name == "Bart", which is also a binary operation. We'll discuss this parsing later, when talking about GetCondition.
  • Lines 35-45 covers unary expressions, for which we'll only support the negation operator. In such a case, the ! LDAP operator is sent to the output, followed by recursive parsing of the operand.
  • Finally, in lines 49-81, we move on to the method call case. A typical supported example is usr.Name.StartsWith("A") which translates nicely into a A* LDAP query equivalent. We'll only support such operations when applied to strings and only for Contains, StartsWith and EndsWith. This code isn't perfect, since we assume the first parameter to be a constant expression, but it could be more complex too (for example when writing something like usr.Name.Contains(DoSomething(123))). This would lead us too far however, but feel free to change to the code to support this (tip: see further, when dealing with projections where we'll perform dynamic compilation of portiong of an expression tree). In here, we call a method GetFieldName that is responsible to find the underlying name for an entity object's member (e.g. in usr.FirstName.Contains("A"), the FirstName property could be translated into givenName which is the corresponding LDAP name). Note: IADs* string properties can't be used via this approach because we perform a translation to LDAP names directly. More generally, none of the IADs* properties can be used in the filter expression since these are only available when the query has executed. You could provide additional logic to throw an exception when such a case is encountered (by using custom attribute inspection in a similar way as done further in this post).

Now, we'll move on to the GetCondition method that's responsible to parse the "atoms" of query expressions:

GetCondition method - Copy Code
1 /// <summary> 2 /// Helper expression to translate conditions to LDAP filters. 3 /// </summary> 4 /// <param name="e">Conditional expression to be translated to an LDAP filter.</param> 5 /// <returns>String representing the condition in LDAP.</returns> 6 private string GetCondition(BinaryExpression e) 7 { 8 string val, attrib; 9 10 bool neg; 11 12 // 13 // Find the order of the operands in the binary expression. At least one should refer to the entity type. 14 // 15 if (e.Left is MemberExpression && ((MemberExpression)e.Left).Member.DeclaringType == originalType) 16 { 17 neg = false; 18 19 attrib = GetFieldName(((MemberExpression)e.Left).Member); 20 val = Expression.Lambda(e.Right).Compile().DynamicInvoke().ToString(); 21 } 22 else if (e.Right is MemberExpression && ((MemberExpression)e.Right).Member.DeclaringType == originalType) 23 { 24 neg = true; 25 26 attrib = GetFieldName(((MemberExpression)e.Right).Member); 27 val = Expression.Lambda(e.Left).Compile().DynamicInvoke().ToString(); 28 } 29 else 30 throw new NotSupportedException("A filtering expression should contain an entity member selection expression."); 31 32 // 33 // Normalize some common characters that cannot be used in LDAP filters. 34 // 35 val = val.ToString().Replace("(", "0x28").Replace(")", "0x29").Replace(@"\", "0x5c"); 36 37 // 38 // Determine the operator and swap the operandi if necessary (LDAP requires a field=value order). 39 // 40 switch (e.NodeType) 41 { 42 case ExpressionType.Equal: 43 return String.Format("{0}={1}", attrib, val); 44 case ExpressionType.NotEqual: 45 return String.Format("!({0}={1})", attrib, val); 46 case ExpressionType.GreaterThanOrEqual: 47 if (!neg) 48 return String.Format("{0}>={1}", attrib, val); 49 else 50 return String.Format("{0}<={1}", attrib, val); 51 case ExpressionType.GreaterThan: 52 if (!neg) 53 return String.Format("&({0}>={1})(!({0}={1}))", attrib, val); 54 else 55 return String.Format("&({0}<={1})(!({0}={1}))", attrib, val); 56 case ExpressionType.LessThanOrEqual: 57 if (!neg) 58 return String.Format("{0}<={1}", attrib, val); 59 else 60 return String.Format("{0}>={1}", attrib, val); 61 case ExpressionType.LessThan: 62 if (!neg) 63 return String.Format("&({0}<={1})(!({0}={1}))", attrib, val); 64 else 65 return String.Format("&({0}>={1})(!({0}={1}))", attrib, val); 66 default: 67 throw new NotSupportedException("Unsupported filtering operator detected: " + e.NodeType); 68 } 69 }

A quick analysis... Most query expressions are of the format <entity>.<property> <operator> <expression> or <expression> <operator> <entity>.<property>. LDAP only supports the former order, so we need to take care of this, which is done in lines 15-28. A few important remarks though:

  • We don't support expressions in which the entity member selection is (deeply) nested, like DoIt(usr.Name) == "Bart".
  • We do support a more complex expression for the "value" however, by means of dynamic compilation as shown in lines 20 and 27. However, when such an expression contains an entity member selection like usr.Name, things won't work any longer (at runtime).

In line 35, a few characters are escaped in order to make the query expression valid. One escape isn't included however, i.e. the one required for the * character. This allows complex queries to be created right away, like this: usr.Name == "B*rt * Smet". In an asterisk should be treated as a constant, one should escape it manually as 0x2a (see RFC 2254 page 4). A similar remark holds for the NUL value (0x00).

Finally, the condition is translated to the LDAP filter equivalent, supporting operators =, >= and <=, based on the expression tree node's operator (NodeType). Notice that if-constructs are required for the "reverse expression" case from lines 24-27, where the operator should be reversed (as in lines 48, 53, 58 and 63). This implementation is very naive though, since LDAP operators <= and >= are used for lexographical comparisons (see How Active Directory Works) and we allow a much broader domain of applicability. This too is kind of a domain mismatch between OO and LDAP, as mentioned in Part 3. We won't elaborate on this; you might think of better alternatives to deal with this operator meaning mismatch (maybe by just rejecting the use of these operators via LINQ?). Also, notice that strings in .NET don't have support for <= or >= directly and a translation from CompareTo method calls to <= and >= LDAP operators would be required.

To finish the "Where"-case of the parsing, take a look at GetFieldName:

GetFieldName method - Copy Code
1 private string GetFieldName(MemberInfo member) 2 { 3 DirectoryAttributeAttribute[] da = member.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 4 if (da != null && da.Length != 0) 5 { 6 if (da[0].Type == DirectoryAttributeType.ActiveDs) 7 throw new InvalidOperationException("Can't execute query filters for IADs* properties."); 8 else 9 return da[0].Attribute; 10 } 11 else 12 return member.Name; 13 }

In here, we check whether or not a DirectoryAttributeAttribute custom attribute has been applied to the specified field. If that's the case, we should make sure that no ActiveDs (IADs*) mapping is made since these cannot be supported in LDAP queries directly (as explained earlier). Finally, the name of the underlying LDAP attribute is retrieved, either by means of the custom attribute's Attribute field or by copying the name of the member directly in case no mapping has been specified.

 

Supporting projections

On to the "Select"-case now, as implemented in BuildProjection:

Projection support - Copy Code
1 /// <summary> 2 /// Helper method for projection clauses (Select). 3 /// </summary> 4 /// <param name="p">Lambda expression representing the projection.</param> 5 private void BuildProjection(LambdaExpression p) 6 { 7 // 8 // Store projection information including the compiled lambda for subsequent execution 9 // and a minimal set of properties to be retrieved (improves efficiency of queries). 10 // 11 project = p.Compile(); 12 13 // 14 // Original type is kept for reflection during querying. 15 // 16 originalType = p.Parameters[0].Type; 17 18 // 19 // Support for (anonymous) type initialization based on "member init expressions". 20 // 21 MemberInitExpression mi = p.Body as MemberInitExpression; 22 if (mi != null) 23 foreach (MemberAssignment b in mi.Bindings) 24 FindProperties(b.Expression); 25 // 26 // Support for identity projections (e.g. user => user), getting all properties back. 27 // 28 else 29 foreach (PropertyInfo i in originalType.GetProperties()) 30 properties.Add(i.Name); 31 }

As the matter in fact, we're very lazy this time. Since LDAP queries don't support projections, the query value doesn't need to be altered at all. Therefore, we compile the lambda expression and store it in the project delegate member variable. This will allow us to execute the projection in all its glory (and possible complexity, e.g. with member assignments using method calls to transform retrieved results before assignment) when processing the results in the GetEnumerator method. Next, the original type is kept in originalType (which reflects the one and only parameter to the projection lambda expression). This will prove useful for quite a bit of things down the drain. Finally, we switch between the member initialization case (new { <member> = <value> } constructs) and an identity projection (p => p), finally resulting in a series of properties that has to be retrieved (this allows us to request only the values of the attributes required for the projection logic, causing more efficient processing inside GetEnumerator as shown later). In the first case, using a member initialization expression, we need to traverse the entire expression tree for all possible "member-grabbing" references to the entity object (e.g. new { First = usr.FirstName, Nick = GetNickName(usr.FirstName + usr.LastName), Stats = new { usr.LogonCount } }), as done in FindProperties:

FindProperties uses recursion to find all entity type property references used in projection - Copy Code
1 /// <summary> 2 /// Recursive helper method to finds all required properties for projection. 3 /// </summary> 4 /// <param name="e">Expression to detect property uses for.</param> 5 private void FindProperties(Expression e) 6 { 7 // 8 // Record member accesses to properties or fields from the entity. 9 // 10 if (e.NodeType == ExpressionType.MemberAccess) 11 { 12 MemberExpression me = e as MemberExpression; 13 if (me.Member.DeclaringType == originalType) 14 { 15 DirectoryAttributeAttribute[] da = me.Member.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 16 if (da != null && da.Length != 0) 17 { 18 if (da[0].Type != DirectoryAttributeType.ActiveDs) 19 properties.Add(me.Member.Name); 20 } 21 else 22 properties.Add(me.Member.Name); 23 } 24 } 25 else 26 { 27 if (e is BinaryExpression) 28 { 29 BinaryExpression b = e as BinaryExpression; 30 FindProperties(b.Left); 31 FindProperties(b.Right); 32 } 33 else if (e is UnaryExpression) 34 { 35 UnaryExpression u = e as UnaryExpression; 36 FindProperties(u.Operand); 37 } 38 else if (e is ConditionalExpression) 39 { 40 ConditionalExpression c = e as ConditionalExpression; 41 FindProperties(c.IfFalse); 42 FindProperties(c.IfTrue); 43 FindProperties(c.Test); 44 } 45 else if (e is InvocationExpression) 46 { 47 InvocationExpression i = e as InvocationExpression; 48 FindProperties(i.Expression); 49 foreach (Expression ex in i.Arguments) 50 FindProperties(ex); 51 } 52 else if (e is LambdaExpression) 53 { 54 LambdaExpression l = e as LambdaExpression; 55 FindProperties(l.Body); 56 foreach (Expression ex in l.Parameters) 57 FindProperties(ex); 58 } 59 else if (e is ListInitExpression) 60 { 61 ListInitExpression li = e as ListInitExpression; 62 FindProperties(li.NewExpression); 63 foreach (Expression ex in li.BLOCKED EXPRESSION 64 FindProperties(ex); 65 } 66 else if (e is MemberInitExpression) 67 { 68 MemberInitExpression mi = e as MemberInitExpression; 69 FindProperties(mi.NewExpression); 70 foreach (MemberAssignment b in mi.Bindings) 71 FindProperties(b.Expression); 72 } 73 else if (e is MethodCallExpression) 74 { 75 MethodCallExpression mc = e as MethodCallExpression; 76 FindProperties(mc.Object); 77 foreach (Expression ex in mc.Arguments) 78 FindProperties(ex); 79 } 80 else if (e is NewExpression) 81 { 82 NewExpression n = e as NewExpression; 83 foreach (Expression ex in n.Arguments) 84 FindProperties(ex); 85 } 86 else if (e is NewArrayExpression) 87 { 88 NewArrayExpression na = e as NewArrayExpression; 89 foreach (Expression ex in na.BLOCKED EXPRESSION 90 FindProperties(ex); 91 } 92 else if (e is TypeBinaryExpression) 93 { 94 TypeBinaryExpression tb = e as TypeBinaryExpression; 95 FindProperties(tb.Expression); 96 } 97 } 98 }

Also, we'll only capture properties that are mapped to LDAP attributes, and not those mapped to IADs* objects. The recursion cases shown above should be pretty complete. All of this plumbing finally results in a HashSet of strings representing all the LDAP properties that should be retrieved.

 

Executing the query and yielding the results

Now that we have all the information required to carry out the query, we're ready to get called via GetEnumerator. In here, we'll execute the query, get the results back and perform the projections by means of the kept delegate project. Here's the code for GetEnumerator:

Executing the query and getting the results back - Copy Code
1 public IEnumerator<T> GetEnumerator() 2 { 3 return GetResults(); 4 } 5 6 private IEnumerator<T> GetResults() 7 { 8 DirectorySchemaAttribute[] attr = (DirectorySchemaAttribute[])originalType.GetCustomAttributes(typeof(DirectorySchemaAttribute), false); 9 if (attr == null || attr.Length == 0) 10 throw new InvalidOperationException("Missing schema mapping attribute."); 11 12 DirectoryEntry root = searchRoot; 13 string q = String.Format("(&(objectClass={0}){1})", attr[0].Schema, query); 14 DirectorySearcher s = new DirectorySearcher(root, q, properties.ToArray(), searchScope); 15 16 if (logger != null) 17 Log.WriteLine(q); 18 19 Type helper = attr[0].ActiveDsHelperType; 20 21 foreach (SearchResult sr in s.FindAll()) 22 { 23 DirectoryEntry e = sr.GetDirectoryEntry(); 24 25 object result = Activator.CreateInstance(project == null ? typeof(T) : originalType); 26 27 if (project == null) 28 { 29 foreach (PropertyInfo p in typeof(T).GetProperties()) 30 AssignResultProperty(helper, e, result, p.Name); 31 32 yield return (T)result; 33 } 34 else 35 { 36 foreach (string prop in properties) 37 AssignResultProperty(helper, e, result, prop); 38 39 yield return (T)project.DynamicInvoke(result); 40 } 41 } 42 }

First, on lines 8 to 13, we modify the query with another filtering expression in order to get only the only objects back that match the specified schema in the DirectorySchemaAttibute custom attribute (which is required) on the entity type. Finally, we're ready to prepare the query in line 14 and launch it in line 21. Finally, we iterate over the results and carry out a projection in case it's required. In here, an additional helper method AssignResultProperty is used to assign the properties in the resulting objects.

AssignResultProperty helper method - Copy Code
1 private void AssignResultProperty(Type helper, DirectoryEntry e, object result, string prop) 2 { 3 PropertyInfo i = originalType.GetProperty(prop); 4 DirectoryAttributeAttribute[] da = i.GetCustomAttributes(typeof(DirectoryAttributeAttribute), false) as DirectoryAttributeAttribute[]; 5 if (da != null && da.Length != 0) 6 { 7 if (da[0].Type == DirectoryAttributeType.ActiveDs) 8 { 9 PropertyInfo p = helper.GetProperty(da[0].Attribute, BindingFlags.Public | BindingFlags.Instance | BindingFlags.IgnoreCase); 10 try 11 { 12 i.SetValue(result, p.GetValue(e.NativeObject, null), null); 13 } 14 catch (TargetInvocationException) { } 15 } 16 else 17 { 18 PropertyValueCollection pvc = e.Properties[da[0].Attribute]; 19 if (i.PropertyType.IsArray) 20 { 21 Array o = Array.CreateInstance(i.PropertyType.GetElementType(), pvc.Count); 22 23 int j = 0; 24 foreach (object oo in pvc) 25 o.SetValue(oo, j++); 26 27 i.SetValue(result, o, null); 28 } 29 else 30 if (pvc.Count == 1) 31 i.SetValue(result, pvc[0], null); 32 } 33 } 34 else 35 { 36 PropertyValueCollection pvc = e.Properties[prop]; 37 if (pvc.Count == 1) 38 i.SetValue(result, pvc[0], null); 39 } 40 } 41

This code uses quite a bit of reflection stuff to get the value of the properties (either from the LDAP properties through the DirectoryEntry instance or via the underlying IADs* type (e.NativeObject in line 12). In case an entity type member is an array, the code in lines 21 to 27 takes care of this. Anyhow, all cases above ultimately call SetValue for a property of the original type in order to supply its value.

 

Testing it

Now we're ready to execute queries against AD. You can download the resulting code over here. Notice that this isn't production quality code and is only meant as a sample. A few sample of queries are shown below:

A simple sample of LINQ-to-LDAP - Copy Code
1 using System; 2 using System.Linq; 3 using BdsSoft.DirectoryServices.Linq; 4 using System.DirectoryServices; 5 using ActiveDs; 6 7 namespace Demo 8 { 9 class Program 10 { 11 static void Main(string[] args) 12 { 13 var users = new DirectorySource<User>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); 14 users.Log = Console.Out; 15 var groups = new DirectorySource<Group>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); 16 groups.Log = Console.Out; 17 18 // 19 // Simple query with all-property projection (usr => usr). 20 // I.e. users.Select(usr => usr); 21 // 22 var res1 = from usr in users 23 select usr; 24 25 Console.WriteLine("QUERY 1\n======="); 26 foreach (var w in res1) 27 Console.WriteLine("{0}: {1} {2}", w.Name, w.Description, w.PasswordLastSet); 28 Console.WriteLine(); 29 30 // 31 // Query with selection criterion. 32 // I.e. users.Where(usr => usr.Name == "A*").Select(usr => usr); 33 // 34 var res2 = from usr in users 35 where usr.Name == "A*" 36 select usr; 37 38 Console.WriteLine("QUERY 2\n======="); 39 foreach (var w in res2) 40 Console.WriteLine("{0}'s full name is {1}", w.Name, w.Dn); 41 Console.WriteLine(); 42 43 // 44 // Query with selection criterion using a local variable and calling a method. 45 // Uses complicated string matching and projection using an anonymous type and a local variable. 46 // I.e. users.Where(usr => usr.Name == GetQueryStartWith("A") 47 // && usr.Description.Contains("Built-in")) 48 // .Select(usr => new { usr.Name, usr.Description, usr.Groups }); 49 // 50 var res3 = from usr in users 51 where usr.Name == GetQueryStartWith("A") && usr.Description.Contains("Built-in") 52 select new { usr.Name, usr.Description, usr.Groups, usr.LogonCount }; 53 54 Console.WriteLine("QUERY 3\n======="); 55 foreach (var w in res3) 56 { 57 Console.WriteLine("{0} has logged on {2} times and belongs to {1} groups:", w.Name, w.Groups.Length, w.LogonCount); 58 foreach (string group in w.Groups) 59 Console.WriteLine("- {0}", group); 60 } 61 Console.WriteLine(); 62 63 // 64 // Query with selection criterion using a local variable involved in a larger expression. 65 // Uses nested objects with anonymous types and expressions in initialization logic. 66 // Illustrates the LDAP query construction based on tree traversal. 67 // I.e. users.Where(usr => usr.Name.StartsWith("A") && usr.LogonCount > 1 * n) || usr.Name == "Guest") 68 // .Select(usr => new { usr.Name, usr.Description, usr.Dn, usr.PasswordLastSet, 69 // Stats = new { usr.PasswordLastSet, usr.LogonCount, TwiceLogonCount = usr.LogonCount * 2 } }); 70 // 71 var res4 = from usr in users 72 where (usr.Name.StartsWith("A") && usr.Name.EndsWith("strator")) || usr.Name == "Guest" 73 select new { usr.Name, usr.Description, usr.Dn, Stats = new { usr.PasswordLastSet, usr.LogonCount, TwiceLogonCount = usr.LogonCount * 2 } }; 74 75 Console.WriteLine("QUERY 4\n======="); 76 foreach (var w in res4) 77 Console.WriteLine("{0} has been logged on {1} times; password last set on {2}", w.Name, w.Stats.TwiceLogonCount - w.Stats.LogonCount, w.Stats.PasswordLastSet); 78 Console.WriteLine(); 79 80 // 81 // Query with sorting (not supported currently). 82 // I.e. users.OrderBy(usr => usr.Name).Select(usr => usr); 83 // 84 var res5 = (from usr in users 85 //orderby usr.Name ascending //not supported in LDAP; alternative in-memory sort 86 select usr).AsEnumerable().OrderBy(usr => usr.Name); 87 88 Console.WriteLine("QUERY 5\n======="); 89 foreach (var w in res5) 90 Console.WriteLine("{0}: {1}", w.Name, w.Description); 91 Console.WriteLine(); 92 93 // 94 // Query against groups in AD. 95 // I.e. groups.Where(grp => grp.Name.EndsWith("ators")).Select(grp => new { grp.Name, MemberCount = grp.Members.Length }); 96 // 97 var res6 = from grp in groups 98 where grp.Name.EndsWith("ators") 99 select new { grp.Name, MemberCount = grp.Members.Length }; 100 101 Console.WriteLine("QUERY 6\n======="); 102 foreach (var w in res6) 103 Console.WriteLine("{0} has {1} members", w.Name, w.MemberCount); 104 Console.WriteLine(); 105 } 106 107 static string GetQueryStartWith(string s) 108 { 109 return s + "*"; 110 } 111 } 112 113 [DirectorySchema("user", typeof(IADsUser))] 114 class User 115 { 116 private string name; 117 118 public string Name 119 { 120 get { return name; } 121 set { name = value; } 122 } 123 124 private string description; 125 126 public string Description 127 { 128 get { return description; } 129 set { description = value; } 130 } 131 132 private int logonCount; 133 134 public int LogonCount 135 { 136 get { return logonCount; } 137 set { logonCount = value; } 138 } 139 140 private DateTime pwdLastSet; 141 142 [DirectoryAttribute("PasswordLastChanged", DirectoryAttributeType.ActiveDs)] 143 public DateTime PasswordLastSet 144 { 145 get { return pwdLastSet; } 146 set { pwdLastSet = value; } 147 } 148 149 private string distinguishedName; 150 151 [DirectoryAttribute("distinguishedName")] 152 public string Dn 153 { 154 get { return distinguishedName; } 155 set { distinguishedName = value; } 156 } 157 158 private string[] memberOf; 159 160 [DirectoryAttribute("memberOf")] 161 public string[] Groups 162 { 163 get { return memberOf; } 164 set { memberOf = value; } 165 } 166 167 } 168 169 [DirectorySchema("group")] 170 class Group 171 { 172 private string name; 173 174 public string Name 175 { 176 get { return name; } 177 set { name = value; } 178 } 179 180 private string[] members; 181 182 [DirectoryAttribute("member")] 183 public string[] Members 184 { 185 get { return members; } 186 set { members = value; } 187 } 188 } 189 }

The resulting output is shown below - We did it!

In the next post, we'll deal with updates to Active Directory through entities. As an example, we'll be able to do the following:

// // Query with update functionality using an entity MyUser : DirectoryEntity. // string oldOffice = "Test"; string newOffice = "Demo"; var res7 = from usr in myusers where usr.Office == oldOffice select usr; Console.WriteLine("QUERY 7\n======="); foreach (var u in res7) { Console.WriteLine("{0} {1} works in {2}", u.FirstName, u.LastName, u.Office); u.Office = newOffice; } Console.WriteLine(); Console.WriteLine("Moving people to new office {0}...\n", newOffice); myusers.Update();

or even

// // Query with method call functionality using an entity MyUser : DirectoryEntity and a method SetPassword. // var res8 = from usr in myusers select usr; string newPassword = "Hello W0rld!"; Console.WriteLine("QUERY 8\n======="); foreach (var u in res8) { Console.WriteLine("Setting the password of {0} {1}...", u.FirstName, u.LastName); u.SetPassword(newPassword); }

Stay tuned!

Read on ... The IQueryable tales - LINQ to LDAP - Part 5: Supporting updates

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

Welcome back to the LINQ-to-LDAP series. So far, we've been discussing:

In the previous post, we discussed quite a bit pieces of the LINQ puzzle, focusing in much detail on expression trees and the role of IQueryable<T> as a query provider's opportunity to parse an expression tree. As you saw, the CreateQuery method plays a central role in this story. In this and future posts, we'll zoom in to this CreateQuery method by creating a first (simple) implementation that makes a translation to LDAP queries. But before we can do so, a bit of preparation needs to be done: enter the world of LDAP query syntax and entity types.

 

Refreshing LDAP queries

In order to be on the same page, let's refresh our LDAP query knowledge. In RFC terms, "LDAP queries" are known as "search filters", which are specified in RFC 2254 entitled "The String Representation of LDAP Search Filters". Although it has been obsoleted by RFC 4515: "Lightweight Directory Access Protocol (LDAP): String Representation of Search Filters" in June 2006, we'll stick with RFC 2254 as implemented by Active Directory (see Active Directory's LDAP Compliance, How Active Directory Searches Work and Search Filter Syntax).

The ABNF grammar of search filters is displayed below, in a slightly restructured format for easier comsumption:

filter = "(" filtercomp ")"
filterlist = 1*filter

filtercomp = and / or / not / item

  • and = "&" filterlist
  • or = "|" filterlist
  • not = "!" filter
  • item = simple / present / substring / extensible
    • simple = attr filtertype value
      • attr = AttributeDescription from Section 4.1.5 of RFC 2251
      • filtertype = equal / approx / greater / less
        • equal = "="
        • approx = "~="
        • greater = ">="
        • less = "<="
      • value = AttributeValue from Section 4.1.6 of RFC 2251
    • present = attr "=*"
      • attr = AttributeDescription from Section 4.1.5 of RFC 2251
    • substring = attr "=" [initial] any [final]
      • initial = value
        • value = AttributeValue from Section 4.1.6 of RFC 2251
      • any = "*" *(value "*")
        • value = AttributeValue from Section 4.1.6 of RFC 2251
      • final = value
        • value = AttributeValue from Section 4.1.6 of RFC 2251
    • extensible = attr [":dn"] [":" matchingrule] ":=" value / [":dn"] ":" matchingrule ":=" value
      • attr = AttributeDescription from Section 4.1.5 of RFC 2251
      • matchingrule = MatchingRuleId from Section 4.1.9 of RFC 2251
      • value = AttributeValue from Section 4.1.6 of RFC 2251

An example of a production is the following:

  • filter = "(" filtercomp ")"
    • filtercomp = item
      • item = simple
        • simple = attr filtertype value
          • attr = givenName
          • filtertype = equal
            • equal = "="
          • value = Bart

resulting in

   givenName=Bart

Other samples of queries are listed below:

  • (cn=Babs Jensen)
  • (!(cn=Tim Howes))
  • (&(objectClass=Person)(|(sn=Jensen)(cn=Babs J*)))
  • (o=univ*of*mich*)

LDAP queries have a prefix-operator notation; humans (and therefore C# programmers) are more familiar with infix notation. These queries are equivalent to the following in C#-ish style:

  • cn == "Babs Jensen"
  • cn != "Tim Howes"
  • objectClass == "Person" && (sn == "Jensen" || cn == "Babs J*")
  • o == "univ*of*mich*"

However, prefix notation is slightly easier to deal with during parsing. Take a look at my DynCalc post series for more information, especially on Infix to Postfix (the inverse of prefix).

For sake of simplicity we trim the grammar down to the following:

filter = "(" filtercomp ")"
filterlist = 1*filter

filtercomp = and / or / not / item

  • and = "&" filterlist
  • or = "|" filterlist
  • not = "!" filter
  • item = simple / present / substring
    • simple = attr filtertype value
      • attr = AttributeDescription from Section 4.1.5 of RFC 2251
      • filtertype = equal / greater / less
        • equal = "="
        • greater = ">="
        • less = "<="
      • value = AttributeValue from Section 4.1.6 of RFC 2251
    • present = attr "=*"
      • attr = AttributeDescription from Section 4.1.5 of RFC 2251
    • substring = attr "=" [initial] any [final]
      • initial = value
        • value = AttributeValue from Section 4.1.6 of RFC 2251
      • any = "*" *(value "*")
        • value = AttributeValue from Section 4.1.6 of RFC 2251
      • final = value
        • value = AttributeValue from Section 4.1.6 of RFC 2251

 

The curse of the Impedance Mismatch

LINQ is all about overcoming the impedance mismatch that exists between various data models, such as hierarchical (XML) or relational (SQL) versus objects (OO). Again, with LDAP, we're faced with such a mismatch of data model representations, in this case directory objects (AD) versus objects (OO). As the matter in fact, objects kept in directory services like AD (Active Directory) or ADAM (Active Directory/Application Mode) have a strong-typed fashion when exposed to the outside world; for example take a look at IADsUser. Beside of a series of properties (like FirstName) those objects also support operations via methods (like SetPassword). This brings us to the first question: "How to map .NET objects to query objects?".

Recall that our LINQ queries will be executed against an object that implements IQueryable<T>. In our case, T has to be some kind of "entity class" (senso lato). The properties of this particular object will play a prominent role because the way the compiler works; as the matter in fact LINQ enforces strong-typing and type safety inside the query. An example to clarify things:

1 class User 2 { 3 public string FirstName { get; set; } 4 public string LastName { get; set; } 5 public int Age { get; set; } 6 } 7 8 class Program 9 { 10 static void Main() 11 { 12 IQueryable<User> src = ...; 13 var res = from usr in src 14 where usr.FirstName.StartsWith("B") && usr.Age >= 24 15 select usr.FirstName + " " + usr.LastName; 16 } 17 }

An example of type safety in the sample above is the variable usr which is of type User throughout the whole query. Because of this, the use of properties can be validated, as well as their types (e.g. Age is an Int32, FirstName is a string). Therefore, we need a good mapping mechanism by means of "entity types" that represent the objects we're querying for.

Things get a little more complicated though: "What about query operators?" The query from the fragment above doesn't translate very well to LDAP. If you've read the How Active Directory Searches Work article in detail, you've noticed that LDAP operators for <= and >= perform lexicographical comparisons. For simplicity's sake however, we won't pay much attention to this caveat. Similarly, we won't provider support for the ~= operator. Furthermore, LDAP is a very very basic language that is far from complete from a query's perspective; therefore we won't be able to translate operations like sorting, grouping, etc into LDAP. Queries like the one shown below:

1 DemoDataContext ctx = new DemoDataContext(); 2 ctx.Log = Console.Out; 3 var res = (from usr in ctx.Users 4 where usr.Age >= 24 5 orderby usr.Name descending 6 select usr.Name).Skip(10).Take(5); 7 foreach (var u in res) 8 ;

translate easily into SQL statements, as illustrated below:

but there's no counterpart in LDAP. A solution to this is to split queries into a piece that does get translated into LDAP while remaining pieces are executed on the client machine as LINQ-to-Objects queries using System.Linq.Enumerable. Such conversions are made possible using the AsEnumerable and AsQueryable methods respectively:

1 DemoDataContext ctx = new DemoDataContext(); 2 ctx.Log = Console.Out; 3 var res = (from usr in ctx.Users 4 where usr.Age >= 24 5 orderby usr.Name descending 6 select usr.Name).AsEnumerable().Skip(10).Take(5); 7 foreach (var u in res) 8 ;

In here, the first part of the query is represented as an expression tree, while the last part with the Skip(10).Take(5) calls is applied on an IEnumerable<string> object, resulting in direct compilation to executable code. The resulting query sent to the database looks like this:

Notice that the SQL plumbing around "paging" with Skip-Take isn't present. Note: if you'd only use a Take(n) method call, you'd get a SELECT TOP n * FROM ... SQL query:

For example, if the underlying datasource (like AD) doesn't support sorting, one could overcome this issue using the following query:

1 var res = (from usr in users 2 where usr.Name.StartsWith("B") 3 select usr.Name).AsEnumerable().OrderByDescending(name => name);

It's less elegant, but the piece between parentheses can be translated into LDAP without any problems (something like (givenName=B*) would be a good translation, see further), while the rest of the query is left to LINQ-to-Objects via the AsEnumerable() call. Notice that the projection in the last line results in an IQueryable<string>, therefore AsEnumerable will return an IEnumerable<string> which on its turn acts as a datasource for LINQ-to-Objects. With this source, we have only string instances left, thus the lambda expression for OrderByDescending needs to be name => name (or s => s or ...).

Another query is shown below and was split into pieces, making the distinction between the IQueryable and IEnumerable portions more sharp:

1 var t = (from usr in users 2 where usr.Name.StartsWith("B") 3 select usr).AsEnumerable(); 4 var res = (from usr in t 5 where usr.Age >= 24 6 orderby usr.Age descending 7 select usr.Name);

We also go rid of the explicit OrderByDescending call by using a second portion of LINQ syntax. One could write this in one statement too:

1 var res = from usr in 2 (from usr in users 3 where usr.Name.StartsWith("B") 4 select usr).AsEnumerable() 5 where usr.Age >= 24 6 orderby usr.Age descending 7 select usr.Name;

As the matter in fact, your IQueryable<T> implementation could be smart enough to track all the things it can't do directly in LDAP (for instance the sorting operations) and send off an LDAP query to AD with the maximum amount of things it can do using LDAP. Once it has fetched the results, all the things it wasn't able to do could be performed by compiling the remainder of the query to Enumerable.* calls (LINQ-to-Objects) and sending the results from the LDAP query through this in-memory query pipeline automatically. This would drive us too far from home, so we'll stick with a simple implementation.

Another question we need to answer is "How method calls are mapped into queries?". Method calls go much beyond the ones you see in the query language itself, such as OrderBy, GroupBy, ThenBy, Take, Skip, ... Other much useful ones are applied on the (common) data types of a query themselves, such as the System.String methods. Consider a LINQ-to-SQL query like the one below:

var res = from usr in ctx.Users where usr.Name.Substring(1, 3).ToLower().Replace("a", "b").CompareTo("c") > 0 && usr.Age + 1 >= 24 && (DateTime.Now - usr.Birthday).TotalDays % 100 == 0 select usr;

agains a table defined like this:

with ID = int, Name = nvarchar(50), Age = int, Birthday = datetime. If you can predict the SQL query that was generated by the LINQ-to-SQL implementation, you deserve a statue in the "LINQ Hall of Fame". Indeed, everything happens on the server using this autogenerated (at runtime!) query:

SELECT [t0].[ID], [t0].[Name], [t0].[Age], [t0].[Birthday]
FROM [dbo].[Users] AS [t0]
WHERE (REPLACE(LOWER(SUBSTRING([t0].[Name], @p0 + 1, @p1)), @p2, @p3) > @p4) AND (([t0].[Age] + @p5) >= @p6) AND ((((CONVERT(Float,CONVERTBigInt,(((CONVERT(BigInt,DATEDIFF(DAY, [t0].[Birthday], @p7))) * 86400000) + DATEDIFF(MILLISECOND, DATEADD(DAY, DATEDIFF(DAY, [t0].Birthday], @p7), [t0].[Birthday]), @p7)) * 10000))) / 864000000000) % @p8) = @p9)
-- @p0: Input Int (Size = 0; Prec = 0; Scale = 0) NOT NULL [1]
-- @p1: Input Int (Size = 0; Prec = 0; Scale = 0) NOT NULL [3]
-- @p2: Input NVarChar (Size = 1; Prec = 0; Scale = 0) NOT NULL [a]
-- @p3: Input NVarChar (Size = 1; Prec = 0; Scale = 0) NOT NULL [b]
-- @p4: Input NVarChar (Size = 1; Prec = 0; Scale = 0) NOT NULL [c]
-- @p5: Input Int (Size = 0; Prec = 0; Scale = 0) NOT NULL [1]
-- @p6: Input Int (Size = 0; Prec = 0; Scale = 0) NOT NULL [24]
-- @p7: Input DateTime (Size = 0; Prec = 0; Scale = 0) NOT NULL [4/7/2007 10:45:17 AM]
-- @p8: Input Float (Size = 0; Prec = 0; Scale = 0) NOT NULL [100]
-- @p9: Input Float (Size = 0; Prec = 0; Scale = 0) NOT NULL [0]

So, if you want to create a damn good query provider to some underlying data source, that exploits all of that datasource's querying capabilities you're up for a lot of work. In the sample above, the SQL functions REPLACE, LOWER, SUBSTRING, CONVERT, DATEDIFF, DATEADD were used which only represent a small subset of the available functions. Notice that DateTime.Now was treated as a constant that was supplied as a parameter (@p7) in order to overcome clock differences and skews between the machine running the query and the machine executing it (SQL Server). Even things like Math.Sin are translated into corresponding SQL Server functions!

Maybe it's a good exercise to try to get LINQ-to-SQL on its knees by writing overly complex queries that use a bunch of methods from various BCL types :-). That said, there are things that LINQ-to-SQL doesn't know how to deal with. I cheated a bit by extending System.String with my own extension method:

A simple System.String extension method - Copy Code
1 static class Ext 2 { 3 public static string Reverse(this string s) 4 { 5 char[] sa = s.ToCharArray(); 6 Array.Reverse(sa); 7 return new string(sa); 8 } 9 }

When launching a query like the following one:

Invalid LINQ-to-SQL query - Copy Code
1 var res = from usr in ctx.Users 2 where usr.Name.Reverse() == "traB" 3 select usr; 4 5 foreach (var u in res) 6 ;

LINQ-to-SQL kindly admits its own mortality by throwing a NotSupportedException:

Unhandled Exception: System.NotSupportedException: Method 'System.String Reverse(System.String)' has no supported translation to SQL.

It's clear that we won't focus on supporting all sorts of exotic methods. We do want to support a few common ones though, especially those that prove useful in writing (implicit) wildcard queries. Three typical ones are listed below:

A few queries with "implicit wildcards" - Copy Code
1 var res1 = from usr in users where usr.Name.StartsWith("B") select usr; 2 var res2 = from usr in users where usr.Name.EndsWith("t") select usr; 3 var res3 = from usr in users where usr.Name.Contains("ar") select usr;

These get translated into the following LDAP queries respectively:

1 (&(objectClass=user)(givenName=B*))
2 (&(objectClass=user)(givenName=*t))
3 (&(objectClass=user)(givenName=*ar*))

Careful readers with eye for detail will notice that the (objectClass=user) portion comes out of the blue all of a sudden. Furthermore, the usr.Name property has been translated into givenName too. These differences will become clear when talking about "entity objects" and mapping schemes.

To wrap up, one should be aware of the limitations of our implementation:

  • No support for advanced query operations since LDAP doesn't support these. Simple from ... where ... select ... statements are supported though.
  • No support for the ~= operator.
  • Limited string operation support to allow implicit wildcard queries.
  • Semantics of >= and <= might be counterintuitive, causing a mismatch between OO and LDAP.

 

We need entities!

In the previous paragraph, we touched the question of entity types already: "How to map .NET objects to query objects?". End-users only care about one thing: an easy way to write queries that are (strongly-typed and) integrated with the language, in the LINQ philosophy. To do this, we need a type to be passed in as a type parameter (T) to some IQueryable<T> implementation. To set your mind, think back of LINQ-to-SQL:

  1. In Visual Studio "Orcas", a LINQ to SQL file is made:


  2. Next, tables and/or sprocs are dragged-and-dropped from the Server Explorer to the Object Relational Designer surface:


  3. Finally, queries can be written like this:

    Copy Code
    1 NorthwindDataContext ctx = new NorthwindDataContext(); 2 3 var res = from usr in ctx.Users 4 where usr.Name == "Bart" 5 select usr;

Behind the scenes, the designer created the NorthwindDataContext class that has the following signature:

public partial class NorthwindDataContext : global::System.Data.Linq.DataContext {


In here, we have a property called Users, declared like this:

public global::System.Data.Linq.Table<User> Users { get { return this.GetTable<User>(); } }


In here, User is the entity type, representing a row from the source table:

Finally, the Table<User> represents the queryable table and indeed, it implements IQueryable<User>:

public sealed class Table<T> : IQueryable<T>, IEnumerable<T>, ITable, IQueryable, IEnumerable, IListSource

Back to LINQ-to-LDAP now. First, we'll also need a class that plays a similar role to Table<T>. It needs to be IQueryable<T> and it should represent a data source coming from AD. In all of my creativity, I came up with the name DirectorySource<T>:

 

Next, the to-be-queried object types need to be represented in some way or another, by means of entities. These are simple class definitions that need to carry enough information too cook up an LDAP query. This is where things get a little tricky. Why? Let's take a look at some of the contents of Active Directory by means of the Windows Server 2003 Support Tool called ldp.exe:

This output shows the details for a user called "Bart De Smet" in the "Demo" organizational unit (OU) in my domain linqdemo.local. All of the lines in the right-hand side pane represent individual properties. For example, my sn is "Bart De Smet" while my physicalDeliveryOfficeName is "Test". These are just two samples of the naming schema employed by AD (and other directory services out there in the wild). Instead, I'd like to talk to AD with more familiar terms like LastName (instead of sn) and Office (instead of physicalDeliveryOfficeName). Therefore, we need some mapping mechanism that makes our entity class "natural" to the users, like this:

A first attempted entity type - Copy Code
1 public class User 2 { 3 public string FirstName { get; set; } 4 public string LastName { get; set; } 5 public string Office { get; set; } 6 }

so that I can write queries like this:

Natural feeling query against AD - Copy Code
1 DirectorySource<User> users = new DirectorySource<User>(...); 2 var res = from usr in users 3 where usr.Office == "Building 10 - 1.25" 4 select usr.FirstName + " " + usr.LastName;

In order to express this mapping, metadata turns out handy. Therefore, we'll create a custom attribute that can be applied to properties (I won't allow public fields since automatic properties make the creation of properties in C# 3.0 plain easy) and that contains information about the mapping to the underlying "internal name" used in Active Directory. Let's just show what I mean based on a revised entity type definition:

A better entity type with mappings - Copy Code
1 public class User 2 { 3 [DirectoryAttribute("givenName")] 4 public string FirstName { get; set; } 5 6 [DirectoryAttribute("sn")] 7 public string LastName { get; set; } 8 9 [DirectoryAttribute("physicalDeliveryOfficeName")] 10 public string Office { get; set; } 11 } 12

Using (an optional) attribute, we've added information to the properties needed to do the mapping. The class itself hasn't changed, so the query mentioned above remains perfectly valid. There's still one other thing we need to add to the class's metadata however: the underlying directory object type (or class schema) that we're going to query. To add this piece of information, we'll apply an attribute on the class level, like this:

An outstanding entity type - Copy Code
1 [DirectorySchema("user")] 2 public class User 3 { 4 [DirectoryAttribute("givenName")] 5 public string FirstName { get; set; } 6 7 [DirectoryAttribute("sn")] 8 public string LastName { get; set; } 9 10 [DirectoryAttribute("physicalDeliveryOfficeName")] 11 public string Office { get; set; } 12 }

Now take a look back at our original query. Based on the the custom attributes metadata, our DirectorySource<T> class is capable of making the translation into LDAP. By now you should be able to see the translation by visual inspection; here it is:

(&(objectClass=user)(physicalDeliveryOfficeName=Building 10 - 1.25))

The purple portion originates from the DirectorySchema attribute value, while the green portion originates from the DirectoryAttribute applied to the Office property on lines 10-11.

Note: You might wonder how you can easily grab all of the properties from the AD schema you're querying for. This is a more than valid concern indeed. One way is to use the ldp.exe tool to inspect current entries in the system. However, it does only display the attributes that are supplied for a given directory entry (for example the properties set for an individual user account). If you want to see the schema itself, you can use a tool called ADSI Edit that comes with Windows Server 2003 and is available through the MMC snap-ins:

If you configure it properly

you'll be able to inspect the schema in detail. Below you can see a screenshot for the User class schema:

If you open it, you can find a bunch of interesting information. For example, there's an entry called systemMayContain with optional attributes (contrast to systemMustContain):

Another property that will interest you is the subClassOf property that reveals the class hierarchy:

The hierarchy for user is: User <-- Organizational-Person <-- Person <-- Top. Top is the root type (like IUnknown or System.Object if you want). This way, you'll find that the physicalDeliveryOffice attribute is used in Organizational-Person. Furthermore, each attribute is also defined in the schema in order to get the required type information, a "required" indicator (nullability), etc. For example, you'll find an attribute called Physical-Delivery-Office-Name:

I agree this is a tiresome job to do; that's why I've created a little tool called AdMetal that allows to export the schema and walk the inheritance tree automatically. Output for the Organizational-Person object is shown in the screenshot below:

You can download an early alpha of this tool over here. Ultimately, this tool could become capable of much more, like automatic code generation for entity types, much like the O/R designer for LINQ-to-SQL. Time will tell... But for now, consider it as a handy tool to query the schema easily. The names between parentheses are the ones you need in the DirectoryAttribute and DirectorySchema attributes, aka the "LDAP display names".

The DirectoryAttributes are not just there to serve the predicate-formulating phase of the LDAP translation however. You might have noticed that LDAP doesn't have a projection portion; indeed LDAP queries are nothing more than filter predicates ("where" clauses). However, the libraries on top of it have support for some kind of projection. I'm referring to the System.DirectoryServices.DirectoryEntry class that has a Properties collection, as well as - more importantly - the System.DirectoryServices.DirectorySearcher class that is used to process queries. In order to execute our LDAP query, we'd write the following piece of code:

A classic LDAP query from C# - Copy Code
1 IEnumerable<string> GetResults() 2 { 3 DirectoryEntry root = new DirectoryEntry("LDAP://localhost/DC=linqdemo,DC=local"); 4 string query = "(&(objectClass=user)(physicalDeliveryOffice=Building 10 - 1.25))"; 5 string[] properties = new string[] { "givenName", "sn" }; 6 7 DirectorySearcher search = new DirectorySearcher(root, query, properties, SearchScope.Subtree); 8 SearchResultCollection res = search.FindAll(); 9 10 foreach (SearchResult sr in res) 11 { 12 DirectoryEntry e = sr.GetDirectoryEntry(); 13 yield return (string)e.Properties["givenName"][0] + " " + (string)e.Properties["sn"][0]; 14 } 15 }

This is the mechanical equivalent of our wanna-be-LINQ-query. Take a look at line 5, where the desired properties (the "projection") are listed as the propertiesToLoad (see constructor information). These values come back in line 13 where the projection really happens (in here using manual coding though). Based on this observation, you can conclude that our IQueryable<T> implementation should figure out the "properties to load" required to do the projection. Think about this for a while; it might seem trivial but it certainly isn't. Consider the following query to get this insight:

Wanna see a complex query? - Copy Code
1 var res = from usr in users 2 where usr.Name.StartsWith("A") 3 select new { Nick = GetNickName(usr.GivenName + " " + usr.LastName), 4 Stats = new { TwiceLogonCount = usr.LogonCount * 2, usr.Office.ToLower() } };

Don't ask why you'd write such a query, but be assured that your customers will try it! In here, the projection on line 3 is represented by a quite complicated expression tree (I'm not going to show it here) where the various projected properties live somewhere deep in the tree. So, we'll need to traverse the "projection expression tree" to find all of those properties that we need to load. This way, we'll make our implementation more efficient than if we would just load all of the properties based on the original entity type class (which would be equivalent to a SELECT * FROM ... query in SQL while you only need the value of one single column for instance).

Notice that the System.DirectoryServices.DirectorySearcher class has quite a lot of interesting features that might prove useful when creating a really good LINQ-to-LDAP implementation:

  • SizeLimit allows to limit the number of results returned; this looks a bit like "SELECT TOP ..." or the Take-method from LINQ.
  • PageSize could help to do paged searchs (a bit like Skip+Take) but it could be used to make queries more efficient too. After all, the query only starts executing when you iterate over the resulting IQuerable<...> object, in which you can use iterators to return results element-per-element. Using paging, you could load say 10 results at a time and while the iterator is going over the current set of 10 results, you could start loading the next 10 results behind the scenes (read: on a background thread). This comes in the neighborhood of PLINQ (although a typical PLINQ-scenario consists of grabbing data from multiple data sources in parallel before feeding the results into a join operation).

To complete this entity discussion, here are the definitions for the custom attributes:

Entity-supporting custom attributes - Copy Code
1 [AttributeUsage(AttributeTargets.Class, AllowMultiple = false)] 2 class DirectorySchemaAttribute : Attribute 3 { 4 public DirectorySchemaAttribute(string schema) 5 { 6 Schema = schema; 7 } 8 9 public string Schema { get; set; } 10 } 11 12 [AttributeUsage(AttributeTargets.Property, AllowMultiple = false)] 13 class DirectoryAttributeAttribute : Attribute 14 { 15 private string attribute; 16 17 public DirectoryAttributeAttribute(string attribute) 18 { 19 Attribute = attribute; 20 } 21 22 public string Attribute { get; set; } 23 }

The creation of custom attributes shouldn't be too much of a problem I guess; if it is, more info can be found on MSDN. We'll extend these custom attributes a little more in future posts since there are still a few boobytraps in the AD jungle that will come and get us :-). For now, we're satisfied with this simple definition though.

 

What's next?

In the next post, we'll start the real tree parsing work inside the CreateQuery method. Or, in other words, we'll start the implementation effort for DirectorySource<T> itself. Stay tuned!

Read on ... The IQueryable tales - LINQ to LDAP - Part 4: Parsing and executing queries

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

Welcome back to the LINQ-to-LDAP series. So far, we've been discussing:

In the previous post, we put ourselves at the side of the compiler either being faced with an IEnumerable<T> or IQueryable<T> object on which a query is to be performed. As you've seen, both cases result in different types of code generation: executable IL code and expression tree representations respectively. In this post, we zoom in to the IQueryable<T> case. More specifically, we'll move focus to the implementation of a query provider that needs to do translation of LINQ queries (fed in through an expression tree) to some specific query language, in our case LDAP. Or, in more techy terms, how to implement IQueryable<T>.

 

Getting started - playground definition

To get started, create a new console application project in C# 3.0 via Visual Studio "Orcas" (alternatively, you can clean the project from the previous post). In the Program.cs file, we'll add a new type called MyDataSource with generic parameter T, that implements IQueryable<T>:

Just implement the interface trivally using the implementation option provided by the "smart tip" (or press SHIFT-ALT-F10). Next, add a simple type called User, defined as follows:

User "entity object" - Copy Code
1 class User 2 { 3 public string Name { get; set; } 4 public int Age { get; set; } 5 }

Again as in the previous post, we're using the C# 3.0 automatic property syntax. Now it's time to move our focus to writing queries, so enter the following query in Main:

A simple query - Copy Code
1 var src = new MyDataSource<User>(); 2 3 var res = from o in src 4 where o.Age >= 24 5 select o.Name;

 

Expression trees in depth

Based on the discussion from the previous post, you should know that the "simple query" code translates into an expression tree because the data source is an IQueryable<T>, in this case IQueryable<User>. Indeed, if you inspect the assembly in IL code, you'll see the code that generates the expression tree being present in the Main method:

.method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 165 (0xa5) .maxstack 5 .locals init ([0] class LINQtoLDAP.MyDataSource`1<class LINQtoLDAP.User> src, [1] class [System.Core]System.Linq.IQueryable`1<string> res, [2] class [System.Core]System.Linq.Expressions.ParameterExpression CS$0$0000, [3] class [System.Core]System.Linq.Expressions.ParameterExpression[] CS$0$0001) IL_0000: nop IL_0001: newobj instance void class LINQtoLDAP.MyDataSource`1<class LINQtoLDAP.User>::.ctor() IL_0006: stloc.0 IL_0007: ldloc.0 IL_0008: ldtoken LINQtoLDAP.User IL_000d: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_0012: ldstr "o" IL_0017: call class [System.Core]System.Linq.Expressions.ParameterExpression [System.Core]System.Linq.Expressions.Expression::Parameter(class [mscorlib]System.Type, string) IL_001c: stloc.2 IL_001d: ldloc.2 IL_001e: ldtoken method instance int32 LINQtoLDAP.User::get_Age() IL_0023: call class [mscorlib]System.Reflection.MethodBase [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype [mscorlib]System.RuntimeMethodHandle) IL_0028: castclass [mscorlib]System.Reflection.MethodInfo IL_002d: call class [System.Core]System.Linq.Expressions.MemberExpression [System.Core]System.Linq.Expressions.Expression::Property(class [System.Core]System.Linq.Expressions.Expression, class [mscorlib]System.Reflection.MethodInfo) IL_0032: ldc.i4.s 24 IL_0034: box [mscorlib]System.Int32 IL_0039: ldtoken [mscorlib]System.Int32 IL_003e: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_0043: call class [System.Core]System.Linq.Expressions.ConstantExpression [System.Core]System.Linq.Expressions.Expression::Constant(object, class [mscorlib]System.Type) IL_0048: call class [System.Core]System.Linq.Expressions.BinaryExpression [System.Core]System.Linq.Expressions.Expression::GreaterThanOrEqual(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.Expression) IL_004d: ldc.i4.1 IL_004e: newarr [System.Core]System.Linq.Expressions.ParameterExpression IL_0053: stloc.3 IL_0054: ldloc.3 IL_0055: ldc.i4.0 IL_0056: ldloc.2 IL_0057: stelem.ref IL_0058: ldloc.3 IL_0059: call class [System.Core]System.Linq.Expressions.Expression`1<!!0> [System.Core]System.Linq.Expressions.Expression::Lambda<class [System.Core]System.Linq.Func`2<class LINQtoLDAP.User,bool>>(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.ParameterExpression[]) IL_005e: call class [System.Core]System.Linq.IQueryable`1<!!0> [System.Core]System.Linq.Queryable::Where<class LINQtoLDAP.User>(class [System.Core]System.Linq.IQueryable`1<!!0>, class [System.Core]System.Linq.Expressions.Expression`1<class [System.Core]System.Linq.Func`2<!!0,bool>>) IL_0063: ldtoken LINQtoLDAP.User IL_0068: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_006d: ldstr "o" IL_0072: call class [System.Core]System.Linq.Expressions.ParameterExpression [System.Core]System.Linq.Expressions.Expression::Parameter(class [mscorlib]System.Type, string) IL_0077: stloc.2 IL_0078: ldloc.2 IL_0079: ldtoken method instance string LINQtoLDAP.User::get_Name() IL_007e: call class [mscorlib]System.Reflection.MethodBase [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype [mscorlib]System.RuntimeMethodHandle) IL_0083: castclass [mscorlib]System.Reflection.MethodInfo IL_0088: call class [System.Core]System.Linq.Expressions.MemberExpression [System.Core]System.Linq.Expressions.Expression::Property(class [System.Core]System.Linq.Expressions.Expression, class [mscorlib]System.Reflection.MethodInfo) IL_008d: ldc.i4.1 IL_008e: newarr [System.Core]System.Linq.Expressions.ParameterExpression IL_0093: stloc.3 IL_0094: ldloc.3 IL_0095: ldc.i4.0 IL_0096: ldloc.2 IL_0097: stelem.ref IL_0098: ldloc.3 IL_0099: call class [System.Core]System.Linq.Expressions.Expression`1<!!0> [System.Core]System.Linq.Expressions.Expression::Lambda<class [System.Core]System.Linq.Func`2<class LINQtoLDAP.User,string>>(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.ParameterExpression[]) IL_009e: call class [System.Core]System.Linq.IQueryable`1<!!1> [System.Core]System.Linq.Queryable::Select<class LINQtoLDAP.User,string>(class [System.Core]System.Linq.IQueryable`1<!!0>, class [System.Core]System.Linq.Expressions.Expression`1<class [System.Core]System.Linq.Func`2<!!0,!!1>>) IL_00a3: stloc.1 IL_00a4: ret } // end of method Program::Main

Two lines in this whole chunk of code are of interest to us for the moment: take a look at lines IL_005e and IL_009e. In here the static class System.Linq.Queryable's extension methods Where and Select are called. Based on this observation, we could rewrite the query code like this:

Resolving extension method calls - Copy Code
1 var res = Queryable.Select( 2 Queryable.Where( 3 src, 4 o => o.Age >= 24), 5 o => o.Name);

In here, I've rewritten the clauses of the query as lambda expressions with dummy variable o. This piece of code is still functionally the same as the original fragment. A question that might pop up in your mind is: "How does the compiler know to translate the lambda expressions into expression trees rather than anonymous methods?". The answer is outlined below in two simple methods:

Code versus data - Copy Code
1 static void LambdaAsCode() 2 { 3 Func<int, int, int> product = (a, b) => a * b; 4 } 5 6 static void LambdaAsData() 7 { 8 Expression<Func<int, int, int>> product = (a, b) => a * b; 9 }

When you take a look at the generated IL-code for both methods, you'll see this:

.field private static class [System.Core]System.Linq.Func`3<int32,int32,int32> '<>9__CachedAnonymousMethodDelegate5' .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 ) .method private hidebysig static void LambdaAsCode() cil managed { // Code size 34 (0x22) .maxstack 3 .locals init ([0] class [System.Core]System.Linq.Func`3<int32,int32,int32> product) IL_0000: nop IL_0001: ldsfld class [System.Core]System.Linq.Func`3<int32,int32,int32> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_0006: brtrue.s IL_001b IL_0008: ldnull IL_0009: ldftn int32 LINQtoLDAP.Program::'<LambdaAsCode>b__4'(int32, int32) IL_000f: newobj instance void class [System.Core]System.Linq.Func`3<int32,int32,int32>::.ctor(object, native int) IL_0014: stsfld class [System.Core]System.Linq.Func`3<int32,int32,int32> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_0019: br.s IL_001b IL_001b: ldsfld class [System.Core]System.Linq.Func`3<int32,int32,int32> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_0020: stloc.0 IL_0021: ret } // end of method Program::LambdaAsCode .method private hidebysig static int32 '<LambdaAsCode>b__4'(int32 a, int32 b) cil managed { .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 ) // Code size 8 (0x8) .maxstack 2 .locals init ([0] int32 CS$1$0000) IL_0000: ldarg.0 IL_0001: ldarg.1 IL_0002: mul IL_0003: stloc.0 IL_0004: br.s IL_0006 IL_0006: ldloc.0 IL_0007: ret } // end of method Program::'<LambdaAsCode>b__4' .method private hidebysig static void LambdaAsData() cil managed { // Code size 73 (0x49) .maxstack 4 .locals init ([0] class [System.Core]System.Linq.Expressions.Expression`1<class [System.Core]System.Linq.Func`3<int32,int32,int32>> product, [1] class [System.Core]System.Linq.Expressions.ParameterExpression CS$0$0000, [2] class [System.Core]System.Linq.Expressions.ParameterExpression CS$0$0001, [3] class [System.Core]System.Linq.Expressions.ParameterExpression[] CS$0$0002) IL_0000: nop IL_0001: ldtoken [mscorlib]System.Int32 IL_0006: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_000b: ldstr "a" IL_0010: call class [System.Core]System.Linq.Expressions.ParameterExpression [System.Core]System.Linq.Expressions.Expression::Parameter(class [mscorlib]System.Type, string) IL_0015: stloc.1 IL_0016: ldtoken [mscorlib]System.Int32 IL_001b: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_0020: ldstr "b" IL_0025: call class [System.Core]System.Linq.Expressions.ParameterExpression [System.Core]System.Linq.Expressions.Expression::Parameter(class [mscorlib]System.Type, string) IL_002a: stloc.2 IL_002b: ldloc.1 IL_002c: ldloc.2 IL_002d: call class [System.Core]System.Linq.Expressions.BinaryExpression [System.Core]System.Linq.Expressions.Expression::Multiply(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.Expression) IL_0032: ldc.i4.2 IL_0033: newarr [System.Core]System.Linq.Expressions.ParameterExpression IL_0038: stloc.3 IL_0039: ldloc.3 IL_003a: ldc.i4.0 IL_003b: ldloc.1 IL_003c: stelem.ref IL_003d: ldloc.3 IL_003e: ldc.i4.1 IL_003f: ldloc.2 IL_0040: stelem.ref IL_0041: ldloc.3 IL_0042: call class [System.Core]System.Linq.Expressions.Expression`1<!!0> [System.Core]System.Linq.Expressions.Expression::Lambda<class [System.Core]System.Linq.Func`3<int32,int32,int32>>(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.ParameterExpression[]) IL_0047: stloc.0 IL_0048: ret } // end of method Program::LambdaAsData

The difference is pretty simple: when the compiler encounters a lambda expression being assigned to an Expression-type variable, it generates an expression tree (e.g. LambdaAsData which contains an expression tree with two parameters and one Multiply node); otherwise, it will generate IL-code (e.g. <LambdaAsCode>b__4 containing the multiplication code).

In our simple code snippet

1 var res = Queryable.Select( 2 Queryable.Where( 3 src, 4 o => o.Age >= 24), 5 o => o.Name);

the two methods Select and Where take an Expression as their second parameter, telling the compiler they want the lambda expression in an expression tree format. For fun and to explain things in depth, let's play the human compiler (or a reverse MSIL-to-C# compiler) that translates both lambda expressions in their equivalent expression tree generation code:

Manual expression tree creation - Copy Code
1 ParameterExpression o = Expression.Parameter(typeof(User), "o"); 2 3 /* o => o.Age >= 24 */ 4 Expression<Func<User, bool>> predicate = 5 Expression.Lambda<Func<User,bool>>( 6 /* >= */ 7 Expression.GreaterThanOrEqual( 8 /* o.Age */ 9 Expression.Property(o, typeof(User).GetProperty("Age")), 10 /* 24 */ 11 Expression.Constant(24) 12 ), 13 /* o */ 14 new ParameterExpression[] { o } 15 ); 16 17 /* o => o.Name */ 18 Expression<Func<User, string>> projection = 19 Expression.Lambda<Func<User, string>>( 20 /* o.Name */ 21 Expression.Property(o, typeof(User).GetProperty("Name")), 22 /* o */ 23 new ParameterExpression[] { o } 24 ); 25 26 var res = Queryable.Select( 27 Queryable.Where( 28 src, 29 predicate), 30 projection);

Again, the code is still functionally the same. Recall our original query definition:

Original query - Copy Code
1 var res = from o in src 2 where o.Age >= 24 3 select o.Name;

Yes, we're still talking about the same thing :-). However, one thing we didn't try yet is to execute this code... On to IQueryable<T> (finally).

 

What IQueryable<T> does - A quick analysis

So, execute the code in the debugger with a magic press of F5 :-). Soon, the program will crash:

Watch the Call Stack window to see how we reached the Expression property of our MyDataSource<T> class that implements IQueryable<T>. Notice I did put the "Show External Code" option of the Call Stack window on:

The second line of the Call Stack is of interest to us and shows that the call to get_Expression originates from Queryable.Where. Recall lines 26 to 30 from our "Manual expression tree creation" code snippet:

1 var res = Queryable.Select( 2 Queryable.Where( 3 src, 4 predicate), 5 projection);

which are equivalent to the following:

Nested call unwinding - Copy Code
1 IQueryable<User> filtered = Queryable.Where(src, predicate); 2 IQueryable<string> projected = Queryable.Select(filtered, projection); 3 4 var res = projected;

Notice that I did make the type of the query's result explicit in line 2; indeed, the projection takes a User object and returns a string, resulting in an IQueryable<string> result type. This reflects the projection lambda expression o => o.Name where o is of type User and o.Name is of type string.

Line 1 of the fragment above explains what's happening right now. Inside the Where method, the first parameter's (which is src, or our MyDataSource<User> instance) Expression property getter is called. Basically, this is LINQ calling our data source with the words: "Dear data source, can you be so kind to represent yourself as an expression?". A good implementation for this property in order to move further in our analysis is shown below:

Copy Code
1 public Expression Expression 2 { 3 get 4 { 5 return Expression.Constant(this); 6 } 7 }

Or, in human words, the query provider responds to LINQ with the words: "Sure, here I am: this constant expression is me.". Restart the application in the debugger; again the app isn't completely happy yet:

As a second step, the Where method calls into our IQueryable<T>'s CreateQuery method. Observe the parameter to the call, this is the predicate from line 1 in the "Nested call unwinding" snippet, wrapped inside a MethodCallExpression to the Where method. Essentially, Queryable.Where has added itself as an expression parent node of type MethodCallExpression with two child nodes:

  • The expression retrieved by calling the src's Expression property from above;
  • the predicate expression that was passed in as the second parameter.

To put it in code, inside Queryable.Where something like this has happened:

public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate) { MethodCallExpression mc = Expression.Call(typeof(Queryable).GetMethod("Where"), source.Expression, predicate); return source.CreateQuery<TSource>(mc); }

In simple words, Queryable's (extension) methods lift themselves in the expression tree, making a MethodCallExpression to themselves with all arguments required, the first of which points to the first parameter (through a call to the Expression property getter of that first parameter).

What does all this magic mean to us? The answer: It's in this place - CreateQuery - that we'll hook up code to parse the where predicate filter clause, resulting in an LDAP query after all (see upcoming posts for details on this). As an implementation, we'll provide some inspection logic that prints a bit of information about the CreateQuery call and its parameters:

Simple CreateQuery implementation
1 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 2 { 3 Console.WriteLine("MyDataSource<{0}> was asked to return IQueryable<{1}>", typeof(T).Name, typeof(TElement).Name); 4 MethodCallExpression e = (MethodCallExpression)expression; 5 6 Console.WriteLine("-> Expression: {0}", e.Method.Name); 7 foreach (Expression arg in e.Arguments) 8 if (arg is ConstantExpression) 9 Console.WriteLine(" - {0}", (arg as ConstantExpression).Value); 10 else 11 Console.WriteLine(" - {0}", arg); 12 13 Console.WriteLine(); 14 15 return new MyDataSource<TElement>(); 16 }

In here, we print information about the generic parameter T type of the current MyDataSource instance, together with the requested TElement return type. These two can vary since methods like Select take in objects of some type and return another type (think of the projection o => o.Name that translates a User into a string object). In line 4, we assume the expression is a MethodCallExpression, which will be true as explained above (the "lifting" performed by Queryable's methods, such as Where and Select). Next, the arguments are printed in lines 7 to 11 (this code could be "simplified" because Where and Select only have two arguments, but let's keep it a bit generic...). Finally, on line 15, we return what we were asked for (an IQueryable<TElement> object with the desired element type of TElement), just to continue execution.

Now think back of this piece of code:

1 IQueryable<User> filtered = Queryable.Where(src, predicate); 2 IQueryable<string> projected = Queryable.Select(filtered, projection); 3 4 var res = projected;

and imagine what will happen. Indeed, first Where is executed, causing a call to CreateQuery on src, returning an object of type IQuerable<User> that's assigned to the filtered variable. Match this with the signature of Queryable.Where to confirm the generic type parameters match:

public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate);

In here TSource is equal to User, thus filtered will return an IQueryable<User>. This result is fed into the call to Select which has the following signature:

public static IQueryable<TResult> Select<TSource, TResult>(this IQueryable<TSource> source, Expression<Func<TSource, TResult>> selector);

Since the selector (projection) is of type Func<User, string> (o => o.Name), TResult equals to string and the result is an IQueryable<string>. Here's the output of our analysis application:

 

If you revert the Main's code from the long an nasty manual expression generating code and the Where-Select call sequence to the following:

With LINQ query syntax - Copy Code
1 static void Main(string[] args) 2 { 3 var src = new MyDataSource<User>(); 4 5 var res = from o in src 6 where o.Age >= 24 7 select o.Name; 8 }

you should still see the same result when executing the code. In the whole description above, you learned how to translate the LINQ query into an equivalent expression tree representation and how the Queryable.Where and Queryable.Select methods work to complete that expression tree's definition. The result is an IQueryable<string> object in variable res that has captured enough information about the query to start execution, triggered by enumeration as explained in the next section.

 

An IQueryable<T> implementation skeleton

IQueryable<T> is more than the Expression property and the CreateQuery<TElement> method however. It's also about enumeration of the results. Here's the metadata-generated definition of the interface:

// Summary: // Provides functionality to evaluate queries against a specific data source // wherein the type of the data is known. public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable { // Summary: // Constructs an System.Linq.IQueryable<T> object that can evaluate the query // represented by the specified expression tree. // // Parameters: // expression: // The System.Linq.Expressions.Expression representing the query to be encompassed. // // Returns: // An System.Linq.IQueryable<T> that can evaluate the query represented by the // specified expression tree. IQueryable<TElement> CreateQuery<TElement>(Expression expression); // // Summary: // Executes the query represented by the specified expression tree. // // Parameters: // expression: // The System.Linq.Expressions.Expression that represents the query to be executed. // // Returns: // A value of type TResult representing the result of the specified query. TResult Execute<TResult>(Expression expression); }

In our journey, we'll focus on the CreateQuery method that gets invoked automatically by the Queryable.* extension methods when creating a query object. This is where we need to prepare the query for subsequent execution, which gets triggered by the enumeration. Previously, on line 15 of our "Simple CreateQuery implementation", we did a simple return of

return new MyDataSource<TElement>();

In a real implementation however, we'll need to carry the information-gained-so-far down the drain till we reach the root level of the query. In our simple sample from above, we created a pipeline like this:

    src (MyDataSource<User>) --(Where)--> filtered (MyDataSource<User>) --(Select)--> projected (MyDataSource<string>)

When enumeration happens on the projected variable, like this:

1 IQueryable<User> filtered = Queryable.Where(src, predicate); 2 IQueryable<string> projected = Queryable.Select(filtered, projection); 3 4 foreach (var item in projected) 5 ;

or, equivalently,

1 var res = from o in src 2 where o.Age >= 24 3 select o.Name; 4 5 foreach (var item in res) 6 ;

the query needs to be launched (and not any earlier than the enumeration!). Of course, this implies that the projected variable needs to carry enough information about what happened before it in the pipeline, up to the src variable. Or: projected needs to carry the information gathered in filtered. When additional steps are involved, like sorting, similar things have to happen. Essentially, the pipeline-based construction of IQueryable<T> instances acts as an accumulator that brings together all the pieces of the query in order to be executed. Our implementation of LINQ-to-LDAP will gather the following information:

  • the LDAP query during the "Where" stage;
  • the projected properties (that need to be retrieved from AD) during the "Select" stage.

An example of a more complex query is shown below:

1 var res = from o in src 2 where o.Age >= 24 3 orderby o.Name ascending 4 select o.Name; 5 6 foreach (var item in res) 7 ;

This translates into the following composition:

You'll need to change IQueryable<T> by IOrderedQueryable<T> to make this work. In such a case, the pipeline

    src (MyDataSource<User>) --(Where)--> filtered (MyDataSource<User>) --(OrderBy)--> sorted (MyDataSource<User>) --(Select)--> projected (MyDataSource<string>)

needs to carry information about the sorting operation too, all the way down to the resulting projected IQuerable<T> variable. Feel free to play around with the query to see the result of additional calls, for example by feeding the following into our dummy IQueryable<T> implementation:

Complex query - Copy Code
1 var res = (from o in src 2 where o.Age >= 24 3 group o by o.Age 4 into g 5 select g).Skip(10).Take(5);

Back to the enumeration story for now. As we said, every IQueryable<T> implements IEnumerable<T> which means that a foreach loop can be used directly to iterate over the results. It's only when iteration is started - thus when a call to GetEnumerator is made - that the query should be launched and results should be obtained, possibly in a lazy fashion that gets a few results at a time and uses C# iterators to "yield return" results.

All of this discussion leads to a simple implementation for IOrderedQueryable<T> (or IQueryable<T> if you don't support sorting, both interfaces are the same) that can be your starting point for future investigations (and implementations):

A dummy implemention for IOrderedQueryable<T> - Copy Code
1 class DummyQueryable<T> : IOrderedQueryable<T> 2 { 3 public IQueryable<TElement> CreateQuery<TElement>(Expression expression) 4 { 5 return new MyDataSource<TElement>(); 6 } 7 8 public IQueryable CreateQuery(Expression expression) 9 { 10 return CreateQuery(expression); 11 } 12 13 public TResult Execute<TResult>(Expression expression) 14 { 15 throw new Exception("The method or operation is not implemented."); 16 } 17 18 public object Execute(Expression expression) 19 { 20 throw new Exception("The method or operation is not implemented."); 21 } 22 23 public IEnumerator<T> GetEnumerator() 24 { 25 yield break; 26 } 27 28 IEnumerator IEnumerable.GetEnumerator() 29 { 30 return GetEnumerator(); 31 } 32 33 public Type ElementType 34 { 35 get { return typeof(T); } 36 } 37 38 public Expression Expression 39 { 40 get { return Expression.Constant(this); } 41 } 42 }

You can run any LINQ query against it and you should never get an exception back (neither should you get any useful results of course :-)). By setting breakpoints in CreateQuery<TElement> you can inspect the expression tree that's passed in. Notice we didn't implement Execute which isn't required strictly spoken. This method can be used to take in a whole expression tree (representing a query) and ask the data source to execute it; normally you won't need it for simple implementations.

 

Coming next on the IQueryable tales...

So, that's it for today... In subsequent posts we'll talk about the creation of mapping entity objects (for example to represent users in AD), tree parsing, update support, etc. Next time, we'll start by making a more useful implementation of CreateQuery that's capable of taking in an expression tree and translating it into the corresponding LDAP query. Here's a little example of it:

var res = from usr in users where usr.Name == GetQueryStartWith("A") && usr.LogonCount >= 10 && usr.Description.Contains("Built-in") select new { usr.Name, usr.Description, usr.Groups, MaxLogonCount = n };

becomes

    (&(objectClass=user)(&(&(Name=A*)(LogonCount>=10))(Description=*Built-in*)))

Get ready for a rollercoaster ride - it's going to be heavy!

Read on ... The IQueryable tales - LINQ to LDAP - Part 3: Why do we need entities?

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

Welcome to the first real part of our LINQ-to-LDAP series. So far, we've been discussing:

In this post, a few key concepts of LINQ concerning the way queries are compiled are discussed. More specifically, we'll talk about IEnumerable<T>, IQueryable<T> and expression trees from a 10,000 feet view. In subsequent posts, we'll dig into more details on the latter portion around IQueryable<T> and expression tree parsing to end up with a nice and smooth translation of LINQ queries to LDAP queries.

To get started, open up Visual Studio 2005 "Orcas" March 2007 CTP and create a new Console Application project. Give it whatever name you see fit, e.g. LINQtoLDAP. For the sake of the demo, we're going to do something rather extraordinary. Instead of running the application, we'll make ourselves happy with the compilation result only, in order to do some ILDASM inspection.

Start by creating a simple "entity" object, which is a class defined like this:

A simple entity object - Copy Code
1 class MyObject 2 { 3 public string Name { get; set; } 4 }

Notice the use of the C# 3.0 automatic properties feature.

 

IEnumerable<T>

In the first demo, we'll focus on the LINQ-to-Objects "query provider". When the compiler encounters a class that implements IEnumerable<T>, the System.Linq.Enumerable extension methods come into play to support query operations. An example will make much clear; change the Main method as follows:

An IEnumerable query - Copy Code
1 static void Main(string[] args) 2 { 3 var lst = new List<MyObject>(); 4 var res = from o in lst 5 where o.Name == "Bart" 6 select o; 7 }

The LINQ query that spans lines 4 to 6 gets translated into the following by the compiler:

static void Main(string[] args) { var lst = new List<MyObject>(); var res = lst.Where(o => o.Name == "Bart").Select(o => o); }

In here, Where and Select are so-called extension methods. In reality the code above translates into:

static void Main(string[] args) { var lst = new List<MyObject>(); var res = System.Linq.Enumerable.Select(System.Linq.Enumerable.Where(lst, o => o.Name == "Bart"), o => o); }

using static method calls. What remains are the lambda expressions that get translated into anonymous methods via delegates:

static void Main(string[] args) { var lst = new List<MyObject>(); var res = System.Linq.Enumerable.Select(System.Linq.Enumerable.Where(lst, delegate(MyObject o) { return o.Name == "Bart"; }), delegate (MyObject o) { return o; }); }

Finally, the local type inference (keyword "var") is resolved, which is pretty simple in this case because we didn't do any projections using anonymous types:

An IEnumerable query's C# 2.0 equivalent - Copy Code
1 static void Main(string[] args) 2 { 3 List<MyObject> lst = new List<MyObject>(); 4 IEnumerable<MyObject> res = System.Linq.Enumerable.Select(System.Linq.Enumerable.Where(lst, delegate(MyObject o) { return o.Name == "Bart"; }), delegate (MyObject o) { return o; }); 5 } 6

As you can see, the syntactical sugar of C# 3.0 makes the query much more like a query. The extension methods Where and Select are implemented as enumerators, making the whole thing lazy evaluated (deferred execution). A sample implementation of both is shown below (you can take this piece of code and compile it in Visual Studio 2005; it's C# 3.0 free):

LINQ in C# 2.0 style - Copy Code
1 using System; 2 using System.Collections.Generic; 3 4 namespace LINQtoLDAP 5 { 6 class Program 7 { 8 static void Main(string[] args) 9 { 10 List<MyObject> lst = new List<MyObject>(); 11 IEnumerable<MyObject> res = Select(Where(lst, delegate(MyObject o) { return o.Name == "Bart"; }), delegate(MyObject o) { return o; }); 12 } 13 14 public delegate TResult Func<TArg0, TResult>(TArg0 arg0); 15 16 public static IEnumerable<TSource> Where<TSource>(IEnumerable<TSource> source, Func<TSource, bool> predicate) 17 { 18 foreach (TSource item in source) 19 if (predicate(item)) 20 yield return item; 21 } 22 23 public static IEnumerable<TResult> Select<TSource, TResult>(IEnumerable<TSource> source, Func<TSource, TResult> selector) 24 { 25 foreach (TSource item in source) 26 yield return selector(item); 27 } 28 } 29 30 class MyObject 31 { 32 private string name; 33 public string Name { get { return name; } set { name = value; } } 34 } 35 }

Basically, we've created a pipeline of enumerators that feed their output in the next one's input. It's only when a foreach loop iterates over the output of the pipeline that the whole thing starts to work; therefore, no more processing is done than strictly required. Notice that if we would provide e.g. a sorting enumerator however, we'd need to suck the whole result from the predecessor in the pipeline in order to be able to sort it to the output. Therefore, laziness has its restrictions too.

Now, let's revisit our original C# 3.0 stylish query definition:

C# 3.0 LINQ style query
static void Main(string[] args) { var lst = new List<MyObject>(); var res = from o in lst where o.Name == "Bart" select o; }

Make sure to import the System.Linq namespace again, which is required to resolve the extension method calls. Compile the application and open up a "Visual Studio Codename Orcas command prompt". Navigate to the bin\Debug output folder of your newly created project and run ildasm.exe against the .exe assembly file. You should see the following output:

We're not going to focus on the MyObject class definition which is just some automatic property magic. We're much more interested in the Program class however. So, open up the Main : void(string[]) method right now. Here's the result:

Main method IL
.method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 46 (0x2e) .maxstack 4 .locals init ([0] class [mscorlib]System.Collections.Generic.List`1<class LINQtoLDAP.MyObject> lst, [1] class [mscorlib]System.Collections.Generic.IEnumerable`1<class LINQtoLDAP.MyObject> res) IL_0000: nop IL_0001: newobj instance void class [mscorlib]System.Collections.Generic.List`1<class LINQtoLDAP.MyObject>::.ctor() IL_0006: stloc.0 IL_0007: ldloc.0 IL_0008: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate2' IL_000d: brtrue.s IL_0022 IL_000f: ldnull IL_0010: ldftn bool LINQtoLDAP.Program::'<Main>b__1'(class LINQtoLDAP.MyObject) IL_0016: newobj instance void class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool>::.ctor(object, native int) IL_001b: stsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate2' IL_0020: br.s IL_0022 IL_0022: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate2' IL_0027: call class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0> [System.Core]System.Linq.Enumerable::Where<class LINQtoLDAP.MyObject>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0>, class [System.Core]System.Linq.Func`2<!!0,bool>) IL_002c: stloc.1 IL_002d: ret } // end of method Program::Main

Time to reconstruct this piece of code back to C# terms. Don't worry about the IL_0000 line which happens to be debugger supporting. In a release build, you'd see slightly different code. A quick analysis:

  • IL_0001 creates the generic List<MyObject> object, as defined in line 3 of the original source.
  • IL_0008 pushes the static field <>9__CachedAnonymousMethodDelegate2 on the stack. This is an anonymous delegate, which will be discussed in a minute. It will be null however, causing IL_000d to fall through to IL_000f.
  • IL_0010 gets a function pointer to the static method <Main>b__1, an anonymous method, as discussed below.
  • IL_0016 finally creates the delegate object that used to be null previously (IL_0008 and IL_000d decided this). As you can see, it's of type System.Linq.Func<MyObject, bool> which is the signature for a predicate (see line 16 of the C# 2.0 style LINQ query code). IL_001b stores the delegate as a static field (as used in IL_0008).
  • IL_0022 is where we end up finally. Here's the predicate delegate is retrieved again.
  • IL_0027 now makes a call to System.Linq.Enumerable.Where passing in two parameters from the stack; the first was loaded on the stack in line IL_0007; the second one is the predicate loaded in line IL_0022.
  • IL_002c stores the result, which is of type System.Collections.Generic.IEnumerable<MyObject> in a local variable, ready to be returned to the caller in line IL_002d.

To clarify things, a little stack transition diagram:

  • IL_0000: {} -> {}
  • IL_0001: {} -> { System.Collections.Generic.List<MyObject> }
  • IL_0006: { System.Collections.Generic.List<MyObject> } -> {}
  • IL_0007: {} -> { System.Collections.Generic.List<MyObject> }
  • IL_0008: { System.Collections.Generic.List<MyObject> } -> { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> }
  • IL_000d: { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> } -> { System.Collections.Generic.List<MyObject> }
    • False
      • IL_000f: { System.Collections.Generic.List<MyObject> } -> { System.Collections.Generic.List<MyObject>, null }
      • IL_0010: { System.Collections.Generic.List<MyObject>, null } -> { System.Collections.Generic.List<MyObject>, null, native int } (top of stack is function pointer)
      • IL_0016: { System.Collections.Generic.List<MyObject>, null, native int } -> { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> }
      • IL_001b: { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> } -> { System.Collections.Generic.List<MyObject> }
      • IL_0020: { System.Collections.Generic.List<MyObject> } -> { System.Collections.Generic.List<MyObject> }
      • (Fall through to IL_0022)
  • IL_0022: { System.Collections.Generic.List<MyObject> } -> { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> }
  • IL_0027: { System.Collections.Generic.List<MyObject>, System.Linq.Func<MyObject,bool> } -> { System.Collections.Generic.IEnumerable<MyObject> }
  • IL_002c: { System.Collections.Generic.IEnumerable<MyObject> } -> {}
  • IL_002d: {} -> {} (= void)

Now let's take a look at the anonymous method definition in <Main>b__1:

.method private hidebysig static bool '<Main>b__1'(class LINQtoLDAP.MyObject o) cil managed { .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 ) // Code size 21 (0x15) .maxstack 2 .locals init ([0] bool CS$1$0000) IL_0000: ldarg.0 IL_0001: callvirt instance string LINQtoLDAP.MyObject::get_Name() IL_0006: ldstr "Bart" IL_000b: call bool [mscorlib]System.String::op_Equality(string, string) IL_0010: stloc.0 IL_0011: br.s IL_0013 IL_0013: ldloc.0 IL_0014: ret } // end of method Program::'<Main>b__1'

This is our predicate function that takes in a MyObject as its parameter and returns true or false depending on the condition (o.Name == "Bart"). The code is straightforward to understand and consist of a property getter call (IL_0001) and an equality test (IL_000b). As an exercise you can try to reconstruct the stack transition diagram for this one too :-).

One remarkable thing is that the Select method doesn't appear in here at all; this is a little optimization that eliminates identity projections (o => o). If we'd perform a projection using an anonymous type, things would look different:

Query with projection - Copy Code
1 static void Main(string[] args) 2 { 3 var lst = new List<MyObject>(); 4 var res = from o in lst 5 where o.Name == "Bart" 6 select new { CapitalizedName = o.Name.ToUpper() }; 7 }

The result is this:

At the bottom, there's an anonymous type with the easy-to-remember (not!) name <>f__AnonymousType0`1<'<>__AnonymousTypeTypeParameter1'>. Feel free to inspect it in more detail; you'll see that a property CapitalizedName was created as defined in line 6 of the original source. Much more magic happens inside e.g. the Equals method (compares all fields for equality), the GetHashCode method and the ToString method (yes, it has a nice string representation that's built automagically using a StringBuilder object). However, we shift our focus on the projection operation itself, which is represented as a delegate of type System.Linq.Func<MyObject, the anonymous type with the nasty name>:

.field private static class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>> '<>9__CachedAnonymousMethodDelegate5' .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 )

Our projection logic lives in the <Main>b__3 method:

.method private hidebysig static class '<>f__AnonymousType0`1'<string> '<Main>b__3'(class LINQtoLDAP.MyObject o) cil managed { .custom instance void [mscorlib]System.Runtime.CompilerServices.CompilerGeneratedAttribute::.ctor() = ( 01 00 00 00 ) // Code size 30 (0x1e) .maxstack 2 .locals init ([0] class '<>f__AnonymousType0`1'<string> '<>g__initLocal1', [1] class '<>f__AnonymousType0`1'<string> CS$1$0000) IL_0000: newobj instance void class '<>f__AnonymousType0`1'<string>::.ctor() IL_0005: stloc.0 IL_0006: ldloc.0 IL_0007: ldarg.0 IL_0008: callvirt instance string LINQtoLDAP.MyObject::get_Name() IL_000d: callvirt instance string [mscorlib]System.String::ToUpper() IL_0012: callvirt instance void class '<>f__AnonymousType0`1'<string>::set_CapitalizedName(!0) IL_0017: nop IL_0018: ldloc.0 IL_0019: stloc.1 IL_001a: br.s IL_001c IL_001c: ldloc.1 IL_001d: ret } // end of method Program::'<Main>b__3'

Again, little surprising. Now our Main methods looks like this:

.method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 82 (0x52) .maxstack 4 .locals init ([0] class [mscorlib]System.Collections.Generic.List`1<class LINQtoLDAP.MyObject> lst, [1] class [mscorlib]System.Collections.Generic.IEnumerable`1<class '<>f__AnonymousType0`1'<string>> res) IL_0000: nop IL_0001: newobj instance void class [mscorlib]System.Collections.Generic.List`1<class LINQtoLDAP.MyObject>::.ctor() IL_0006: stloc.0 IL_0007: ldloc.0 IL_0008: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate4' IL_000d: brtrue.s IL_0022 IL_000f: ldnull IL_0010: ldftn bool LINQtoLDAP.Program::'<Main>b__2'(class LINQtoLDAP.MyObject) IL_0016: newobj instance void class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool>::.ctor(object, native int) IL_001b: stsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate4' IL_0020: br.s IL_0022 IL_0022: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,bool> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate4' IL_0027: call class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0> [System.Core]System.Linq.Enumerable::Where<class LINQtoLDAP.MyObject>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0>, class [System.Core]System.Linq.Func`2<!!0,bool>) IL_002c: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_0031: brtrue.s IL_0046 IL_0033: ldnull IL_0034: ldftn class '<>f__AnonymousType0`1'<string> LINQtoLDAP.Program::'<Main>b__3'(class LINQtoLDAP.MyObject) IL_003a: newobj instance void class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>>::.ctor(object, native int) IL_003f: stsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_0044: br.s IL_0046 IL_0046: ldsfld class [System.Core]System.Linq.Func`2<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>> LINQtoLDAP.Program::'<>9__CachedAnonymousMethodDelegate5' IL_004b: call class [mscorlib]System.Collections.Generic.IEnumerable`1<!!1> [System.Core]System.Linq.Enumerable::Select<class LINQtoLDAP.MyObject,class '<>f__AnonymousType0`1'<string>>(class [mscorlib]System.Collections.Generic.IEnumerable`1<!!0>, class [System.Core]System.Linq.Func`2<!!0,!!1>) IL_0050: stloc.1 IL_0051: ret } // end of method Program::Main

The new stuff starts on line IL_002c where similar logic as for the Where-method call is present, this time to serve the Select clause. Based on this analysis, you can conclude that LINQ queries against IEnumerable<T> implementations (like List<MyObject>) are completely transformed into corresponding IL code. Feel free to make queries more complicated (orderby, groupby, join, ...) and take the challenge to parse the resulting IL code :-).

If you want to learn more about the extension standard query operator methods that are implemented in System.Linq.Enumerable, download my LINQSQO project that contains a full custom implementation of all these operators (it can even be compiled using C# 2.0 in Visual Studio 2005!).

 

IQueryable<T>

Now, let's swap to the big brother of IEnumerable<T>: enter LINQ's IQuerable<T> interface. The definition is displayed below:

Generic IQueryable definition - Copy Code
1 using System.Collections; 2 using System.Collections.Generic; 3 using System.Linq.Expressions; 4 5 namespace System.Linq 6 { 7 // Summary: 8 // Provides functionality to evaluate queries against a specific data source 9 // wherein the type of the data is known. 10 public interface IQueryable<T> : IEnumerable<T>, IQueryable, IEnumerable 11 { 12 // Summary: 13 // Constructs an System.Linq.IQueryable<T> object that can evaluate the query 14 // represented by the specified expression tree. 15 // 16 // Parameters: 17 // expression: 18 // The System.Linq.Expressions.Expression representing the query to be encompassed. 19 // 20 // Returns: 21 // An System.Linq.IQueryable<T> that can evaluate the query represented by the 22 // specified expression tree. 23 IQueryable<TElement> CreateQuery<TElement>(Expression expression); 24 // 25 // Summary: 26 // Executes the query represented by the specified expression tree. 27 // 28 // Parameters: 29 // expression: 30 // The System.Linq.Expressions.Expression that represents the query to be executed. 31 // 32 // Returns: 33 // A value of type TResult representing the result of the specified query. 34 TResult Execute<TResult>(Expression expression); 35 } 36 }

Notice that this interface extends on IEnumerable<T>; that is, every IQueryable<T> is also IEnumerable<T>, meaning that you can iterate over it. As usual, the generic interface has a non-generic counterpart IQueryable, just like IEnumerable<T> has a non-generic variant of IEnumerable. The main difference between pure IEnumerable<T>'s and IQueryable<T>'s is the fact that the latter ones are based on expressions. Essentially, such an expression is a data-based representation of a piece of code (in this case a query, hence the name IQueryable) that can be parsed at runtime in order to be translated into a domain-specific query language like LDAP. This is done via the CreateQuery method that takes an expression and returns another IQueryable<TElement> object (where T and TElement can be possibly different, e.g. to serve projections). This result is IQueryable<>, thus it's IEnumerable<> too, meaning that we can iterate over it. However, the results won't be created by means on System.Linq.Enumerable extension methods this time; instead, the underlying implementation of the IQuerable<> implementor will perform lots of plumbing to translate the given expression in a suitable query expression that's sent to the underlying data source, getting the results back. We won't focus on Execute for now.

As an example, consider LINQ-to-SQL. Go to our project in Visual Studio "Orcas" and add a new item to the project - a LINQ to SQL File:

In Server Explorer, make a connection to a database that contains some table (note: the March 07 "Orcas" CTP VPC contains a SQLEXPRESS instance with default databases; feel free to add your own database in there for testing). On my machine, I've created a simple Users table in the msdb database. Drag-and-drop the table to the Object Relational Designer surface of Orcas:

Now, in the Main method, add the following:

LINQ-to-SQL sample - Copy Code
1 static void Main(string[] args) 2 { 3 DemoDataContext ctx = new DemoDataContext(); 4 ctx.Log = Console.Out; 5 6 var res = from usr in ctx.Users 7 where usr.Name.Contains("Bart") 8 select usr; 9 10 foreach (var user in res) 11 Console.WriteLine(user.ID + " - " + user.Name); 12 }

The LINQ query doesn't reveal we're talking to another type of data source (i.e. SQL Server). However, behind the scenes all sorts of magic happens to translate the query to an appropriate SQL statement. The Log property in line 4 can be used to display what's happening at runtime:

 

Indeed, the LINQ query was translated into a SQL statement at runtime (in contrast to lots of O/R mapping tools that perform this task at design time). As the matter in fact, the C# compiler encountered an IQueryable<> object this time ... ctx.Users. When you go to the definition of this property, you'll see something like this inside the DemoDataContext class:

public global::System.Data.Linq.Table<User> Users { get { return this.GetTable<User>(); } }

The GetTable<T> method is derived from System.Linq.Data.DataContext and returns a Table<T>. In our case, T equals User, as defined below:

[global::System.Data.Linq.Table(Name="dbo.Users")] public partial class User : global::System.Data.Linq.INotifyPropertyChanging, global::System.ComponentModel.INotifyPropertyChanged { private int _ID; private string _Name; [global::System.Diagnostics.DebuggerNonUserCodeAttribute()] public User() { this._ID = default(int); } [global::System.Data.Linq.Column(Storage="_ID", Name="ID", DBType="Int NOT NULL IDENTITY", IsPrimaryKey=true, IsDBGenerated=true, CanBeNull=false)] public int ID { get { return this._ID; } } [global::System.Data.Linq.Column(Storage="_Name", Name="Name", DBType="NVarChar(50) NOT NULL", CanBeNull=false)] public string Name { get { return this._Name; } set { if ((this._Name != value)) { this.OnPropertyChanging("Name"); this._Name = value; this.OnPropertyChanged("Name"); } } } public event global::System.ComponentModel.PropertyChangedEventHandler PropertyChanging; public event global::System.ComponentModel.PropertyChangedEventHandler PropertyChanged; [global::System.Diagnostics.DebuggerNonUserCodeAttribute()] protected void OnPropertyChanging(string propertyName) { if ((this.PropertyChanging != null)) { this.PropertyChanging(this, new global::System.ComponentModel.PropertyChangedEventArgs(propertyName)); } } [global::System.Diagnostics.DebuggerNonUserCodeAttribute()] protected void OnPropertyChanged(string propertyName) { if ((this.PropertyChanged != null)) { this.PropertyChanged(this, new global::System.ComponentModel.PropertyChangedEventArgs(propertyName)); } } }

This mapping was done by the designer and contains all the information needed to construct SQL statements for querying and updating at runtime, based on various attributes like TableAttribute and ColumnAttribute. We'll come back to this in one of the subsequent posts, when talking about "entity class" generation (in our case for Active Directory objects). However, let's revisit System.Data.Linq.Table<T> for a minute and generate its definition using the IDE:

Just focus on the declaration:

// Summary: // Represents a table for a particular type in the underlying database. public sealed class Table<T> : IQueryable<T>, IEnumerable<T>, ITable, IQueryable, IEnumerable, IListSource {

Indeed, implementing IQueryable<T>! Right, time for some IL. Strip the Main method definition to the following (unless you want to see more more IL than necessary for the demo :-)):

LINQ-to-SQL simplified - Copy Code
1 static void Main(string[] args) 2 { 3 DemoDataContext ctx = new DemoDataContext(); 4 5 var res = from usr in ctx.Users 6 where usr.Name.Contains("Bart") 7 select usr; 8 }

Compile and go back to the command prompt to open the assembly with ILDASM. Again, open the Main method; notice that all anonymous methods and types are gone. Here's Main's IL code:

.method private hidebysig static void Main(string[] args) cil managed { .entrypoint // Code size 133 (0x85) .maxstack 7 .locals init ([0] class LINQtoLDAP.DemoDataContext ctx, [1] class [System.Core]System.Linq.IQueryable`1<class LINQtoLDAP.User> res, [2] class [System.Core]System.Linq.Expressions.ParameterExpression CS$0$0000, [3] class [System.Core]System.Linq.Expressions.Expression[] CS$0$0001, [4] class [System.Core]System.Linq.Expressions.ParameterExpression[] CS$0$0002) IL_0000: nop IL_0001: newobj instance void LINQtoLDAP.DemoDataContext::.ctor() IL_0006: stloc.0 IL_0007: ldloc.0 IL_0008: callvirt instance class [System.Data.Linq]System.Data.Linq.Table`1<class LINQtoLDAP.User> LINQtoLDAP.DemoDataContext::get_Users() IL_000d: ldtoken LINQtoLDAP.User IL_0012: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_0017: ldstr "usr" IL_001c: call class [System.Core]System.Linq.Expressions.ParameterExpression [System.Core]System.Linq.Expressions.Expression::Parameter(class [mscorlib]System.Type, string) IL_0021: stloc.2 IL_0022: ldloc.2 IL_0023: ldtoken method instance string LINQtoLDAP.User::get_Name() IL_0028: call class [mscorlib]System.Reflection.MethodBase [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype [mscorlib]System.RuntimeMethodHandle) IL_002d: castclass [mscorlib]System.Reflection.MethodInfo IL_0032: call class [System.Core]System.Linq.Expressions.MemberExpression [System.Core]System.Linq.Expressions.Expression::Property(class [System.Core]System.Linq.Expressions.Expression, class [mscorlib]System.Reflection.MethodInfo) IL_0037: ldtoken method instance bool [mscorlib]System.String::Contains(string) IL_003c: call class [mscorlib]System.Reflection.MethodBase [mscorlib]System.Reflection.MethodBase::GetMethodFromHandle(valuetype [mscorlib]System.RuntimeMethodHandle) IL_0041: castclass [mscorlib]System.Reflection.MethodInfo IL_0046: ldc.i4.1 IL_0047: newarr [System.Core]System.Linq.Expressions.Expression IL_004c: stloc.3 IL_004d: ldloc.3 IL_004e: ldc.i4.0 IL_004f: ldstr "Bart" IL_0054: ldtoken [mscorlib]System.String IL_0059: call class [mscorlib]System.Type [mscorlib]System.Type::GetTypeFromHandle(valuetype [mscorlib]System.RuntimeTypeHandle) IL_005e: call class [System.Core]System.Linq.Expressions.ConstantExpression [System.Core]System.Linq.Expressions.Expression::Constant(object, class [mscorlib]System.Type) IL_0063: stelem.ref IL_0064: ldloc.3 IL_0065: call class [System.Core]System.Linq.Expressions.MethodCallExpression [System.Core]System.Linq.Expressions.Expression::Call(class [System.Core]System.Linq.Expressions.Expression, class [mscorlib]System.Reflection.MethodInfo, class [System.Core]System.Linq.Expressions.Expression[]) IL_006a: ldc.i4.1 IL_006b: newarr [System.Core]System.Linq.Expressions.ParameterExpression IL_0070: stloc.s CS$0$0002 IL_0072: ldloc.s CS$0$0002 IL_0074: ldc.i4.0 IL_0075: ldloc.2 IL_0076: stelem.ref IL_0077: ldloc.s CS$0$0002 IL_0079: call class [System.Core]System.Linq.Expressions.Expression`1<!!0> [System.Core]System.Linq.Expressions.Expression::Lambda<class [System.Core]System.Linq.Func`2<class LINQtoLDAP.User,bool>>(class [System.Core]System.Linq.Expressions.Expression, class [System.Core]System.Linq.Expressions.ParameterExpression[]) IL_007e: call class [System.Core]System.Linq.IQueryable`1<!!0> [System.Core]System.Linq.Queryable::Where<class LINQtoLDAP.User>(class [System.Core]System.Linq.IQueryable`1<!!0>, class [System.Core]System.Linq.Expressions.Expression`1<class [System.Core]System.Linq.Func`2<!!0,bool>>) IL_0083: stloc.1 IL_0084: ret } // end of method Program::Main

Whoosh, pretty scary isn't it? No worries, again it's plain simple :-). This piece of code shows how lazy the C# compiler was (well, sort of) because it didn't translate the code of the query to IL at all; instead it dumped its expression tree in code (which, however, takes another big implementation effort of course). This means that the tree parsing will be up to the query provider implementation, for example to translate it to SQL (or in our case to LDAP). Let's take a quick look:

  • IL_0008 represents the from clause on line 5; it gets the users table back (from usr in ctx.Users).
  • IL_001c creates a ParameterExpression which takes a parameter from type LINQtoLDAP.User (line IL_000d gets the token for the type while IL_0012 converts it to a Type instance, this is standard output for a typeof(T) instruction) and a parameter name ("usr"). Essentially, this is the part of line 5 indicated as bold: from usr in ctx.Users. The expression is kept in slot 2 of the local variables on line IL_0021.
  • IL_0032 creates a MemberExpression that takes two parameters. The first parameter comes from lne IL_0022 where we stored the ParameterExpression on the stack; the second one comes from IL_0028 (indirectly after a cast on line IL_002d) where we get a MethodInfo (reflection) object for the get_Name property getter. Basically this is the translation for where usr.Name.Contains("Bart").
  • IL_0065 then creates a MethodCallExpression that takes three parameters. This is the translation of where usr.Name.Contains("Bart"). An overview of the parameters:
    • The first one refers to the MemberExpression created above.
    • The second one is a MethodInfo object that represent the method to be called; this is constructed in lines IL_0037 to IL_0041.
    • The last one is an array of parameters that needs to be passed to the method call at runtime. This array is constructed in line IL_0047 with a dimension of 1 (IL_0046). The first element of the array is supplied in lines IL_004e to IL_0063 where a ConstantExpression is created for the literal "Bart" in where usr.Name.Contains("Bart").
  • IL_0079 creates a lambda expression representing a Func<LINQtoLDAP.User, bool> predicate delegate, based on two parameters:
    • An expression representing the body of the lambda; which is the MethodCallExpression from above.
    • A parameter, which is the ParameterExpression from above.
  • IL_007e finally calls System.Linq.Queryable.Where which is a factory method to create yet another series of nodes in the expression tree, representing a Where method call. Here the IQueryable<T> instance is passed in and ultimately this will result in a call being made to the IQueryable's methods to parse the tree (see future posts).

At runtime, you can inspect the expression tree which is represented in variable res:

In here, I've only shown a few levels in the tree, up to the Where method call. The tree looks like this (simplified):

  • [System.Linq.Expressions.MethodCallExpression] = {Table(dbo.Users).Where(usr => usr.Name.Contains("Bart"))}
    • Method = {System.Linq.IQueryable`1[LINQtoLDAP.User] Where[User](System.Linq.IQueryable`1[LINQtoLDAP.User], System.Linq.Expressions.Expression`1[System.Linq.Func`2[LINQtoLDAP.User,System.Boolean]])}
    • Object = null
    • Arguments[0] = [System.Linq.Expressions.ConstantExpression] = {Table(dbo.Users)}
    • Arguments[1] = [System.Linq.Expressions.UnaryExpression] = {usr => usr.Name.Contains("Bart")}
      • Operand = [System.Linq.Expressions.LambdaExpression] = {usr => usr.Name.Contains("Bart")}
        • Body = [System.Linq.Expressions.MethodCallExpression] = {usr.Name.Contains("Bart")}
          • Method = {Boolean Contains(System.String)}
          • Object = [System.Linq.Expressions.MemberExpression] = {usr.Name}
          • Arguments[0] = [System.Linq.Expressions.ConstantExpression] = {"Bart"}

The picture below (click to enlarge) shows the expression tree (partially!) in the debugger:

 

Notice that the resulting tree altready contains some parsing results, such as Table(dbo.Users), which have been created by the query provider for LINQ-to-SQL at runtime by inspecting various attributes (such as TableAttribute applied on the User class). Ultimately, the provide is capable of translating the whole tree into a SQL statement:

SELECT [t0].[ID], [t0].[Name]
FROM [dbo].[Users] AS [t0]
WHERE [t0].[Name] LIKE @p0

which is finally sent to the database based on project configuration properties that were auto-generated (app.config):

<?xml version="1.0" encoding="utf-8" ?> <configuration> <configSections> </configSections> <connectionStrings> <add name="LINQtoLDAP.Properties.Settings.msdbConnectionString" connectionString="Data Source=.\SQLEXPRESS;Initial Catalog=msdb;Integrated Security=True" providerName="System.Data.SqlClient" /> </connectionStrings> </configuration>

To conclude this discussion, always think of IQueryable<T> implementations as tree parsers. From the compiler's point of view, IL code to create a tree is generated. From a provider's point of view, the CreateQuery method will be called, given that expression tree, for further processing, ultimately yielding a translation into a domain-specific query language like SQL or XPath/XQuery or LDAP or WQL or CAML or ... you name it. We'll focus on creating a query provider for LDAP in the next posts that will do similar things as LINQ-to-SQL. Time to refresh your tree traversal algorithms :-).

Read on ... The IQueryable tales - LINQ to LDAP - Part 2: Getting started with IQueryable<T>

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Here we are again for some cool LINQ stuff. In the past I've been blogging on C# 3.0 language innovation quite a lot, including the core features that enable LINQ in its cute and nice form:

These posts focused on how C# 3.0 provides new constructs on top of existing ones in order to make various things simpler. In case you didn't read these posts yet, do it right now :-).

As you probably know by now, LINQ consists of a few pieces. One is the integration of querying syntax in programming languages like C# 3.0 and VB 9.0. This is what the integrated portion stands for. At the other side, different APIs exist that provide us with a gateway to underlying data sources. Today five of these are delivered by Microsoft in the current CTPs:

  • LINQ-to-Objects - talks to in-memory objects (see my LINQSQO project; SQO stands for Standard Query Operators)
  • LINQ-to-SQL - talks to SQL Server databases
  • LINQ-to-XML - talks to hierarchical data represented in XML
  • LINQ-to-DataSets - talks to DataSet objects and underlying DataTables with their relationships
  • LINQ-to-Entities - talks to "entities", part of ADO.NET 3.0

There are a few differences between all of these however. Although the query syntax remains the same (it's integrated with the language after all and it has a "universal" character), the mechanisms that drive the functionality are different. For LINQ-to-Objects, the entire query expression is translated into IL at compile time by some mechanical procedure as outlined below:

List<Person> lstPersons = new List<Person>() { new Person { Name = "Bart", Age = 24 }, new Person { Name = "John", Age = 59 } }; var res = from person in lstPersons where person.Name == "Bart" select new { Name = person.Name.ToUpper() };

becomes

List<Person> lstPersons = new List<Person>() { new Person { Name = "Bart", Age = 24 }, new Person { Name = "John", Age = 59 } }; var res = lstPersons.Where(person => person.Name == "Bart").Select(person => new { Name = person.Name.ToUpper() });

where methods like Where and Select are extension methods to IEnumerable<T> that take a lambda expression as their parameter. The whole mechanism is driven by iterators behind the scenes.

However, LINQ query "providers" like LINQ-to-SQL and LINQ-to-XML work in a different way. Here we want to be able to translate the LINQ query to another domain specific query language (respectively SQL and XPath/XQuery) at runtime. In order to make this possible, the C# compiler doesn't generate IL code representing the query right away, but instead it spits out the code to create an in-memory representation of the query in the shape of an expression tree (see this link too). The compiler is told to do so when it encounters an IQueryable interface implementation used in a query expression.

In subsequent posts in this "The IQueryable tales - LINQ to LDAP" series, we'll talk about the creation of a query provider for LINQ that's capable of talking to Active Directory (and other LDAP data sources potentially) over LDAP. To set your mind, take a look at the concept of filtering expressions in LDAP on RFC 2254 and on TechNet (yes, an IT Pro resource on a developer's blog). As a little example to wet your appetite, take a look at a few queries below:

var users = new DirectoryObject<User>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); var groups = new DirectoryObject<Group>(new DirectoryEntry("LDAP://localhost"), SearchScope.Subtree); var res1 = from usr in users select usr; Console.WriteLine("QUERY 1\n======="); foreach (var w in res1) Console.WriteLine("{0}: {1} {2}", w.Name, w.Description, w.PasswordLastSet); Console.WriteLine(); var res2 = from usr in users where usr.Name == "A*" select usr; Console.WriteLine("QUERY 2\n======="); foreach (var w in res2) Console.WriteLine("{0}'s full name is {1}", w.Name, w.Dn); Console.WriteLine(); int n = 10; var res3 = from usr in users where usr.Name == GetQueryStartWith("A") && usr.LogonCount > n && usr.Description.Contains("Built-in") select new { usr.Name, usr.Description, usr.Groups, MaxLogonCount = n }; Console.WriteLine("QUERY 3\n======="); foreach (var w in res3) { Console.WriteLine("{0} has logged on {2} times or more and belongs to {1} groups:", w.Name, w.Groups.Length, w.MaxLogonCount); foreach (string group in w.Groups) Console.WriteLine("- {0}", group); } Console.WriteLine(); var res4 = from usr in users where (usr.Name.StartsWith("A") && usr.LogonCount > 2 * n) || usr.Name == "Guest" select new { usr.Name, usr.Description, usr.Dn, usr.PasswordLastSet, Stats = new { usr.PasswordLastSet, usr.LogonCount, TwiceLogonCount = usr.LogonCount * 2 } }; Console.WriteLine("QUERY 4\n======="); foreach (var w in res4) Console.WriteLine("{0} has been logged on {1} times; password last set on {2}", w.Name, w.Stats.TwiceLogonCount - w.Stats.LogonCount, w.PasswordLastSet); Console.WriteLine(); var res5 = from usr in users orderby usr.Name ascending //not supported in LDAP; alternative in-memory sort select usr; Console.WriteLine("QUERY 5\n======="); foreach (var w in res5) Console.WriteLine("{0}: {1}", w.Name, w.Description); Console.WriteLine(); var res6 = from grp in groups where grp.Name.EndsWith("ators") select new { grp.Name, MemberCount = grp.Members.Length }; Console.WriteLine("QUERY 6\n======="); foreach (var w in res6) Console.WriteLine("{0} has {1} members", w.Name, w.MemberCount); Console.WriteLine();

The result of the queries above looks like this on my test machine (including the corresponding LDAP queries that were created at runtime):

You'll be able to execute those against a live Active Directory domain controller or an ADAM instance pretty soon! This being said, you should be warned not to expect a feature complete query provider for LDAP but rather a hands-on demo implementation. In the meantime, get ready for subsequent posts in this series by installing the March 07 CTP of Orcas (if you haven't done yet).

Read on ... The IQueryable tales - LINQ to LDAP - Part 1: Key concepts

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Just to let you guys know ... It has been released; it being Enterprise Library 3.0. More information on Tom Hollander's blog.Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

 

Note: This post is part of the follow-up for the Developer & IT Pro Days 2007 session entitled "Internet Information Services (IIS) 7.0 - End-to-End Overview of Microsoft's New Web Application Server". Attendees will be able to download the slides and will get the video recording of the entire session as well. Using these posts, attendees should be capable of reproducing the entire demo part of the session. Furthermore, others who didn't attend the session will be able to get a better idea of what IIS 7.0 provides by following the demo steps outlined in these posts. In order to implement the demos, one should have IIS 7.0 installed on Windows Vista (or Windows "Longhorn" Server).

 

Introduction

In the previous part of this series, we focused on IIS 7.0's configuration system by extending our image handler sample with web.config-based configuration. Today we'll take it one step further by making the configuration of our handle available through IIS Manager by means of a custom module. As you'll see, creating custom modules for IIS Manager is pretty straightforward, thanks to the managed code support.

 

IIS 7.0 Manager

We've been faced with the IIS Manager of IIS 7.0 quite a lot already. Just press WIN+R, enter inetmgr, click OK and accept the UAC dialog to refresh your mind:

Our goal is to add an "icon" to the middle area of the screen, in order to make the settings for the image copyright handler accessible directly. Such an addition to IIS Manager is called a module, maybe a bit of a misnomer due to the conflicting name with the module-handler stuff we discussed before.

 

A custom module

Right, here we are with our image copyright handler and support for configuration through web.config (see the previous part of this series). Let's extend it with an IIS Manager module:

  1. In Visual Studio 2005 - still running as administrator - add a new C# class library project named PictureHandlerConfiguration to the solution by right-clicking the solution root node in Solution Explorer and choosing Add, New Project...
  2. Strong-name the assembly generated by the class library project by opening the Properties node of the PictureHandlerConfiguration project



    and going to the Signing tab. Mark the checkbox "Sign the assembly" and choose new from the "Choose a strong name key file:" dropdown box:



    In the dialog that appears, give the key file a name "key.snk" and disable the password protection for sake of the demo:


  3. Next, add a few references to the PictureHandlerConfiguration project by right-clicking it and choosing Add Reference... Go to the .NET tab and choose System.Windows.Forms. Click OK.



    Open up the Add Reference... dialog again and go to the Browse tab, navigate to %windir%\system32\inetsrv and select both Microsoft.Web.Administration.dll and Microsoft.Web.Management.dll. Click OK.


  4. Right-click the PictureHandlerConfiguration project and choose Add, User Control... Call it ImageCopyrightConfiguration.cs and press OK:

     

    This control will be displayed in the IIS Manager upon completion of this demo script. It will display the settings and allow settings to be changed.
  5. Design the control as displayed below:

     

    The controls are the following (see numbering above):

    0. chkEnabled
    1. lblMessage  2. txtMessage
    3. lblColor    4. txtColor (ReadOnly=true)    5. btnColor
    6. btnSave   7. lblStatus

  6. Add a ColorDialog control named colorDialog to the designer surface too:


  7. Double-click the btnColor button and add the following piece of code (pretty straightforward to understand):
    private void btnColor_Click(object sender, EventArgs e) { if (txtColor.Text != "") colorDialog.Color = Color.FromName(txtColor.Text); if (colorDialog.ShowDialog() == DialogResult.OK) txtColor.Text = colorDialog.Color.Name; }

  8. Select the Class1.cs file in the Solution Explorer and rename it to PictureHandlerSettings.cs. Open it and replace the Class1 with the code below (make sure to keep the namespace):
    class PictureHandlerSettings : ConfigurationSection { public bool Enabled { get { return (bool)base["enabled"]; } set { base["enabled"] = value; } } public string Message { get { return (string)base["message"]; } set { base["message"] = value; } } public System.Drawing.Color Color { get { return System.Drawing.Color.FromName((string)base["color"]); } set { base["color"] = value.Name; } } }


    You'll need to import the Microsoft.Web.Administration namespace to have the ConfigurationSection base class in scope.

     
  9. Go back to the ImageCopyrightConfiguration control's code view and add the following private members to the class definition:
    private ServerManager mgr; private string siteName; private string virtualPath; private PictureHandlerSettings settings;


    Again, you'll need to import the Microsoft.Web.Administration namespace, this time for the ServerManager type.
  10. Change the constructor of the control like this:
    public ImageCopyrightConfiguration(ServerManager mgr) { this.mgr = mgr; InitializeComponent(); }


    Notice we didn't set the siteName and virtualPath yet; the reason for this will become apparent further on.
  11. Next, we'll add a ReadSettings method that uses the ServerManager instance to query IIS for the current settings:
    private void ReadSettings() { Configuration config = mgr.GetWebConfiguration(siteName, virtualPath); settings = (PictureHandlerSettings)config.GetSection( "system.webServer/imageCopyright", typeof(PictureHandlerSettings)); }


    This code is completely the same as the one we used in the previous post to get the applicable settings inside our handler implementation.
  12. Now, we need to use the settings to populate the control's UI elements with the values. We'll do this in an Initialize method that calls our ReadSettings method from step 11:
    public void Initialize(string siteName, string virtualPath) { this.siteName = siteName; this.virtualPath = virtualPath; ReadSettings(); txtColor.Text = settings.Color.Name; txtMessage.Text = settings.Message; chkEnabled.Checked = settings.Enabled; }


    This is where the siteName and virtualPath values are set. The Initialize method will be called by the module page we'll talk about further on.
  13. Finally, for what the control definition is concerned, go back to the designer and double-click the btnSave button. Add code to the event handler like this:
    private void btnSave_Click(object sender, EventArgs e) { try { settings.Color = Color.FromName(txtColor.Text); settings.Message = txtMessage.Text; settings.Enabled = chkEnabled.Checked; mgr.CommitChanges(); lblStatus.Text = "Settings saved successfully!"; lblStatus.ForeColor = Color.DarkGreen; } catch (Exception ex) { lblStatus.Text = ex.Message; lblStatus.ForeColor = Color.DarkRed; } ReadSettings(); }


    In this code, the settings object is modified by assigning the values from the UI element to the various properties. Using the ServerManager instance, changes are committed by calling CommitChanges. Finally, if everything goes right, a status message is displayed in the lblStatus label. One thing that's rather important is to call ReadSettings again, to re-read the settings from configuration. This is required because you can only commit changes of a settings object once, so you have to obtain a new settings object for subsequent changes.
  14. Now it's time to wrap the control in a module page. It's okay to think of such a page as the right-hand side portion of IIS Manager where the real functionality is displayed when opening a module:



    Such a module page is a Windows Forms control, on which we'll put our own ImageCopyrightConfiguration control this time. This approach allows us to use the User Control designer in Visual Studio 2005. To create the module page, add a new class file to the PictureHandlerConfiguration project by right-clicking the project in the Solution Explorer and choosing Add, Class... giving it a name of ModuleStuff.cs. Replace the ModuleStuff class definition by the following (make sure the namespace is kept):
    class ImageCopyrightUIPage : ModulePage { private ServerManager mgr; private ImageCopyrightConfiguration c; public ImageCopyrightUIPage() { mgr = new ServerManager(); c = new ImageCopyrightConfiguration(mgr); Controls.Add(c); } // // Override OnActivated // }


    You'll need to import the Microsoft.Web.Management.Client.Win32 namespace:



    The constructor of the ImageCopyrightUIPage class creates an instance of ServerManager (for which you need to import the Microsoft.Web.Administrator namespace again). Furthermore, it creates an instance of our control created in steps 4-13 and adds it to the Controls collection (effectively putting the control in the left top corner of the module page's surface).
  15. Next, override OnActivated in the class definition from step 14 and add a bit of code:
    protected override void OnActivated(bool initialActivation) { base.OnActivated(initialActivation); if (initialActivation) c.Initialize(Connection.ConfigurationPath.SiteName, Connection.ConfigurationPath.ApplicationPath + Connection.ConfigurationPath.FolderPath); }


    This is where we do our initialization work for the control by calling Initialize. The reason for doing the initialization over here is that only from this point on, we have access to the Connection object, which is our context object to get information about the currently selected node in the IIS Manager's tree on the left. Basically, we query the connection for the site's name as well as the complete path to the currently selected node. Notice we only perform the initialization when the module page is activated for the first time ("initialActivation").
  16. Once we have our module page definition, it's time to wrap it inside a module. A module wraps the module page (which is the module's user interface) together with information to create a shortcut ("icon") in the IIS Manager's control panel. Add the following class definition to the ModuleStuff.cs file, inside the namespace definition:
    class ImageCopyrightUI : Module { // // Override Initialize // // // Override IsPageEnabled // }


    Again you need to import a namespace, this time the Microsoft.Web.Management.Client namespace.

     
  17. Now, we'll override both the Initialize and IsPageEnabled methods, as displayed below:
    class ImageCopyrightUI : Module { protected override void Initialize(IServiceProvider serviceProvider, Microsoft.Web.Management.Server.ModuleInfo moduleInfo) { base.Initialize(serviceProvider, moduleInfo); IControlPanel controlPanel = (IControlPanel)GetService(typeof(IControlPanel)); controlPanel.RegisterPage(new ModulePageInfo(this, typeof(ImageCopyrightUIPage), "Image Copyright", "Configuration for the image copyright demo.")); } protected override bool IsPageEnabled(ModulePageInfo pageInfo) { Connection conn = (Connection)GetService(typeof(Connection)); ConfigurationPathType pt = conn.ConfigurationPath.PathType; return pt == ConfigurationPathType.Site || pt == ConfigurationPathType.Application; } }


    The Initialize method obtains a reference to the control panel of the IIS Manager. In here, the icons are displayed that provide access to the underlying module's functionality, represented by the module page. By using the GetService method, a service is retrieved; this allows to write very flexible code. In this case, we retrieve the control panel service, but you could also hook up your own services and retrieve a bunch of other services. An example of a custom service would be a class that allows communication to some database to import settings. The ModulePageInfo object is used to represent the "shortcut" to the underlying module page (you could have more than one module page too, each represented by one instance of ModulePageInfo), containing the caption of the icon, the descrption of the page (which will be shown in a tooltip when you hover over the icon), as well as the type. Other overloads to the ModulePageInfo constructor exist, that also take images for the icon. In our case, we didn't specify an icon, so a default one will be chosen by IIS Manager as you'll see further on.

    The IsPageEnabled method is used by IIS Manager to query the module whether or not it should make some module page (passed in through the pageInfo parameter) available. In this case we have a one-on-one mapping between the module and a module page, so we don't bother about the pageInfo parameter. In this demo, we've chosen to make the configuration module page only available when a site or application node is selected (thus not on the server root level). Notice that the Connection object is retrieved through GetService too.

    As a side-remark, modules are capable of hosting tasks as well. These tasks are displayed in the right-hand side column of the IIS Manager. We won't elaborate on this for now.
  18. Right, we have a module page and a module. Last but not least, we have arrived at the uppermost layer in the hierarchy: the module provider. Again, add it to the ModuleStuff.cs file inside the namespace scope:
    class ImageCopyrightUIModuleProvider : ModuleProvider { public override Type ServiceType { get { return null; } } public override bool SupportsScope(ManagementScope scope) { return true; } public override ModuleDefinition GetModuleDefinition(IManagementContext context) { return new ModuleDefinition(Name, typeof(ImageCopyrightUI).AssemblyQualifiedName); } }


    Again, a namespace needs to be imported, this time Microsoft.Web.Management.Server:



    This is the most "dirty" one of the three classes since it's the least visual of the three. Basically it wraps a module by means of a ModuleDefinition object that points to the fully qualified type name of the module and the assembly implementing it. Just accept this piece of plubming; it's required for IIS Manager to load the module itself (amongst a custom service if you require one).
  19. Finally, compile the class library by pressing CTRL-SHIFT-B. Next, go to the Visual Studio 2005 Command Prompt (running as Administrator) and cd into the bin\Debug folder of the PictureHandlerConfiguration project. Execute gacutil -i PictureHandlerConfiguration.dll to put the assembly in the GAC to make it available for IIS Manager:


  20. In order to make the module available for IIS Manager we need to modify the XML-based configuration of the IIS Manager. To do so, open the %windir%\system32\inetsrv\config\administration.config file in Visual Studio 2005, running as Administrator. Locate the <configuration>/<moduleProviders> section and add the following:
    <!-- !!! CHANGE PKT !!! --> <add name="ImageCopyright" type="PictureHandlerConfiguration.ImageCopyrightUIModuleProvider, PictureHandlerConfiguration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5be231eaf2cd0956, processorArchitecture=MSIL" />


    Important! Change the PublicKeyToken value by the value of your assembly's public key token (it will be different from the one shown above since you've generated a key yourself). You can get the value either by using gacutil -l or sn -T as shown below:



    In my case, the configuration entry becomes:
    <add name="ImageCopyright" type="PictureHandlerConfiguration.ImageCopyrightUIModuleProvider, PictureHandlerConfiguration, Version=1.0.0.0, Culture=neutral, PublicKeyToken=e549d40da8d9d26f, processorArchitecture=MSIL" />


    Finally, locate the <configuration>/<location path=".">/<modules> section and add the module in there as follows:
    <add name="ImageCopyright" />

  21. Time to test! Make sure the configuration file from step 20 is saved properly and open up IIS Manager. Go to Web Sites\Default Web Site\devdays and observe the Image Copyright icon being available on the right:

     

    As you can see, the icon's caption as well as the description are visible. When you double-click the icon, the module page appears:

     

    Change the color to Green by clicking the ... button and finally hit Save:



    You'll see that the web.config file of devdays has changed like this:
    <imageCopyright enabled="true" message="Copyright (C) Developer &amp; IT Pro Days 2007" color="Green" />


    Refresh the waterfall.jpg file in the browser (http://localhost/devdays/waterfall.jpg?percent=30):

Congratulations, you've completed your first IIS 7.0 handler with its own configuration and management module! Download the code for this final episode over here.

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

 

Note: This post is part of the follow-up for the Developer & IT Pro Days 2007 session entitled "Internet Information Services (IIS) 7.0 - End-to-End Overview of Microsoft's New Web Application Server". Attendees will be able to download the slides and will get the video recording of the entire session as well. Using these posts, attendees should be capable of reproducing the entire demo part of the session. Furthermore, others who didn't attend the session will be able to get a better idea of what IIS 7.0 provides by following the demo steps outlined in these posts. In order to implement the demos, one should have IIS 7.0 installed on Windows Vista (or Windows "Longhorn" Server).

 

Introduction

In the previous part of this series, a simple HTTP handler was created to resize images dynamically and to put copyright messages on images dynamically. Of course, we don't want to hardcode all of the settings for these copyright messages, including the color and the message itself. Instead, it's a much more desirable to have these values come from some kind of configuration. In today's post, we'll focus on the configuration system in IIS 7.0 in more detail.

 

IIS 7.0 Configuration

As you probably know, IIS 6.0 and earlier stored their configuration settings in a centralized database called the metabase. Before the advent of IIS 6.0, the metabase was a binary file; as of IIS 6.0, it's also possible to use an XML-based metabase. As you can imagine, centralization of the configuration is quite easy from the web server developer's point of view, but for the end-users the story is quite different. In case (re)configuration of a deployed application is required, one has to make a call to the support department of the company or to the hosting company in order to apply the change to the app's or site's configuration, which is far from ideal. Even for simple settings like the order of the default pages (default.aspx, default.htm, ...) or custom error pages, one has to touch the metabase. Luckily, different (web-based) tools were created to overcome this issue, but there's no such thing as a unified approach to configuration.

In contrast to the IIS metabase story, there's the power of the web.config file in the wonderful world of ASP.NET, which makes it possible to xcopy-deploy settings together with the application's binaries and static content. Furthermore, this approach allows for easy overriding of settings in subapplications or for given "locations" by means of a hierarchy of web.config files, stored in subdirecties.

IIS 7.0 gets rid of the metabase (the metabase is DEAD) and replaces it by a hierarchical configuration mechanism based on web.config files, that also allows for delegation. This means that IT admins can control which sections of the configuration can be overriden by who. For example, you might allow people to register their own handlers, while you don't like them to hook up a module or touch the authentication configuration. The picture below illustrates the hierarchical model:

In the previous posts, we've already been faced with the web.config file repeatedly. Today, we'll extend the schema of the configuration with our own entries and we'll see how the IIS 7.0 APIs allow for easy querying of settings.

 

Extending and testing the schema

Let's get started by extending the schema with a custom definition. In order to do so, follow the steps outlined below. Continue from the result obtained in the previous post:

  1. Inside Visual Studio 2005, running as Administrator, right-click the Solution node in Solution Explorer and choose Add, New Item...:


  2. Choose XML file as the template and specify ImageCopyright.xml as the file name:


  3. In the XML file, put the following schema definition:
    <configSchema> <sectionSchema name="system.webServer/imageCopyright"> <attribute name="enabled" type="bool" defaultValue="false" /> <attribute name="message" type="string" defaultValue="Your Copyright Message" /> <attribute name="color" type="string" defaultValue="Yellow" /> </sectionSchema> </configSchema>


    This schema is hooked up in the system.webServer/imageCopyright section (see sectionSchema) and has three attributes specified, all with their type and the default value specified.
  4. Save the schema via File, Save ImageCopyright.xml As... and go to the %windir%\system32\inetsrv\config\schema folder. Save the file with the original file name:



    Notice the other files that reside in this folder, including the IIS schema, the WCF schema and the ASP.NET schema. Open these files to get a better idea of the flexibility the IIS 7.0 configuration schema definition capability offers, including validation settings, the use of enums and flags, element nesting, uniqueness checks, etc.
  5. Now go to File, Open, File (CTRL-O) and open the ApplicationHost.config file from %windir%\system32\inetsrv\config. This is the place where IIS 7.0 keeps references to the registered schemas. That is, putting a schema definition file in the config\schema folder isn't enough to make it work. Additionally, you need to register the schema with the server's configuration. To do this, locate the <configuration>/<configSections>/<sectionGroup name="system.webServer"> section and add a section element to it, as shown below:
    <sectionGroup name="system.webServer"> <section name="imageCopyright" overrideModeDefault="Allow" />


    Make sure that the imageCopyright entry isn't misspelled, casing matters!
  6. Now, open a command line running as administrator (right-click the shortcut to cmd.exe, choose Run as Administrator), go to the %windir%\system32\inetsrv folder and execute appcmd list config -section:system.webServer/imageCopyright. The result should look like this:



    This indicates that IIS 7.0 has found the entry in the schema together with the schema definition inside the schema directory. If the result doesn't look as bright as the one shown above, revisit steps 3 to 5 and check all spelling carefully.
  7. At this point in time, we can already apply settings to a web.config file by changing the web.config file either manually or via some tool like appcmd. As an example, execute the following commands:

    appcmd set config "Default Web Site" -section:system.webServer/imageCopyright -enabled:true -color:Blue
    appcmd set config "Default Web Site/devdays" -section:system.webServer/imageCopyright -color:Red -message:"Copyright (C) Developer & IT Pro Days 2007"


  8. Check whether the settings were applied successfully by taking a look at the %SystemDrive%\inetpub\wwwroot\web.config file as well as the %SystemDrive%\inetpub\wwwroot\devdays\web.config file:

     

The settings applied in step 7 use the concept of overriding. If you'd request an image on the Default Web Site root level, the handler would be enabled and the color would be set to Blue, overriding the default values of enabled=false and color=Yellow. However, the message would still be "Your Copyright Message" (see step 3). One level deeper in the hierarchy, inside the devdays app, the color setting has been overridden to Red and the message has been set to "Copyright (C) Developer & IT Pro Days 2007". We'll see the result in a minute when we've changed our code to incorporate the settings.

 

Consuming the configuration in code

The next step in our configuration journey is to use the settings in our handler effectively. Let's take a look, starting from the handler's implementation from the previous post:

  1. In the web site project, right-click and choose Add Reference...



    Go to the Browse tab and navigate to %windir%\system32\inetsrv. Select Microsoft.Web.Administration.dll and click OK:


  2. At the bottom of the PictureHandler.cs file, outside the class definition, add the following class:
    class PictureHandlerSettings : ConfigurationSection { public bool Enabled { get { return (bool)base["enabled"]; } set { base["enabled"] = value; } } public string Message { get { return (string)base["message"]; } set { base["message"] = value; } } public System.Drawing.Color Color { get { return System.Drawing.Color.FromName((string)base["color"]); } set { base["color"] = value.Name; } } }


    This is a strongly-typed configuration class definition which we'll use to get our settings back. Instead we could also get the settings without a class wrapper, but this approach pays off over time; for example, take a look at the added functioanlity in the Color property to convert a string representation of the color to a Color instance and vice versa. In order for this code to work correctly, you'll need to import the Microsoft.Web.Administration namespace:


  3. Next, locate the ProcessRequest method of the handler's implementation and add a few lines of code at the top of the method as shown below:
    public void ProcessRequest(HttpContext context) { ServerManager mgr = new ServerManager(); Configuration config = mgr.GetWebConfiguration( System.Web.Hosting.HostingEnvironment.SiteName, context.Request.Path); PictureHandlerSettings settings = (PictureHandlerSettings)config.GetSection( "system.webServer/imageCopyright", typeof(PictureHandlerSettings));


    This code connects to the IIS 7.0 machine by means of the ServerManager instance, then retrieves the configuration that applies to the current site (HostingEnvironment.SiteName) and the current request (Request.Path) and finally retrieves the settings section for system.webServer/imageCopyright using the strongly typed settings class from step 2. This is incredibly powerful indeed since it does all of the overriding stuff for you. For example, for the Default Web Site root, the settings would be combined from applicationHost.config and web.config since only the enabled attribute was overridden on the web site root's level. Similarly, settings form the Default Web Site root and the devdays application are combined when requesting something like http://localhost/devdays/waterfall.jpg.
  4. Finally, change the ProcessRequest method to consume the settings. The complete code listing for ProcessRequest looks like this (changes marked in comments):
    public void ProcessRequest(HttpContext context) { ServerManager mgr = new ServerManager(); Configuration config = mgr.GetWebConfiguration( System.Web.Hosting.HostingEnvironment.SiteName, context.Request.Path); PictureHandlerSettings settings = (PictureHandlerSettings)config.GetSection( "system.webServer/imageCopyright", typeof(PictureHandlerSettings)); string path = context.Request.PhysicalPath; Image img = Bitmap.FromFile(path); string p = context.Request.QueryString["percent"]; int percent; if (p != null && int.TryParse(p, out percent) && percent > 0 && percent < 100) { Image t = img; img = new Bitmap(img.GetThumbnailImage(img.Width * percent / 100, img.Height * percent / 100, null, IntPtr.Zero)); t.Dispose(); } //<NEW> if (settings.Enabled) { //string msg = "Copyright Microsoft (C) 2007"; string msg = settings.Message; using (Graphics g = Graphics.FromImage(img)) { Font f = new Font("Arial", 10, FontStyle.Bold); SizeF s = g.MeasureString(msg, f); PointF pos = new PointF(img.Width - s.Width - 5, img.Height - s.Height - 5); //g.DrawString(msg, f, Brushes.Yellow, pos); g.DrawString(msg, f, new SolidBrush(settings.Color), pos); } } //</NEW> img.Save(context.Response.OutputStream, ImageFormat.Jpeg); img.Dispose(); }


    Notice we didn't disable the zooming functionality if the enabled attribute is set to false.

  5. Compile the application by pressing CTRL-SHIFT-B and navigate to http://localhost/devdays/waterfall.jpg?percent=30 in the browser. Observe that the settings from step 7 (above) have taken effect:

     
  6. Feel free to play around with the configuration and the overriding functionality.

The code for this post can be downloaded over here. In the next post in this series, we'll put the crowne on our work by extending the IIS Manager with a module that allows easy configuration for our IT Pro friends. Stay tuned!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

More Posts « Previous page - Next page »