Sunday, February 03, 2008 3:16 PM bart

Method invocation cmdlets in Windows PowerShell

Introduction

It should be a common introduction slogan for Windows PowerShell in the meantime, but let me repeat it: Windows PowerShell closes the gap between developers and IT Pros. Why do I make this statement (again)? Well, giving IT Pro people full access to the joys of the .NET Framework and richness of objects is certainly a good thing. The other way around, having developers think about making their products manageable by means of providers and cmdlets is certainly a good thing as well since it makes software products stand out much better in production environments.

Nevertheless, there are a few places where the developer's mindset and the ITPro's clash a little. On such place might look a little philosophical at first: invocation of operations on objects. The latter noun, "objects", is well-known in both camps. It's about something (an entity that can represent virtually anything, in the management space often referred to as a management object e.g. representing a mailbox) one can query data from (properties) and invoke operations on (methods). The latter aspect is the one we'll cover in this post.

 

Prefix or postfix? No big deal uh...

Why dedicate a post to method invocation? Let me divide the world in two artificial camps: the Prefix Club and the Postfix Club. The Prefix Club likes to say upfront what they want to do and then think about the subject of the operation. The Postfix Club tends to think the other way around: victim first, operation second. In the Prefix Club we find the IT Pros (amongst others of course):

c:\temp> type bar.txt
c:\temp> taskkill /PID 1234
c:\temp> dsadd <user> ...

Over in the Postfix Club we find quite some developers (excluding some rare groups that speak LISP etc):

File.OpenRead("bar.txt").ReadAllLines()
Process.GetProcessById(1234).Kill()
directoryEntry.Children.Add(<user>)

See the core difference? The main reason postfix works for developers is likely the tool support (IntelliSense anyone?) and although PowerShell has similar constructs (maybe not as visual as IntelliSense) there's still a difference in discoverability. PowerShell has basically two ways of performing management operations, the more natural one being the use of cmdlets that are designed to be discoverable by their name (verb-noun convention). The other one is invoking methods on objects, more in a developer style with discoverability through get-member (gm). Cmdlets are prefix by nature (verb = what, noun = target) while method invocations are - as mentioned before - postfix.

Taking a bit more distance, the funny thing of the recent set of language enhancements is the intrinsic support in C# 3.0 and VB 9.0 to revert the operation invocation order by means of extension methods, but we won't go that far in this post (maybe next time). Just think of the transform an extension method performs on an object:

victim.Operation(...)
Extensions.Operation(victim, ...)

 

Our mission

What about reducing the burden put on developers to make their objects manageable in a more natural PowerShell way? Assume you have some rich object that has a bunch of methods and you simply want to expose these methods as prefix-style cmdlets. Let's take a look at one that already exists in PowerShell: Stop-Process. Essentially the implementation of Stop-Process is fairly simple since it's base functionality is to, euhm, stop a process using .NET Framework functionality. I guess the Process.Kill method rings a bell:

Stop-Process p --> p.Kill()

Of course, cmdlets often provide a way to add more flexibility, e.g. by allowing a process name to be passed in instead of a PID or a Process instance, possibly using wildcards, etc. But having a way to expose .NET APIs in a natural way as cmdlets (without blocking later functionality enrichment) sounds compelling. This is what this post is all about.

Note: Make sure you have some basic understanding of cmdlets and the art of writing these (together with PowerShell snap-ins). Check out my Easy Windows PowerShell cmdlet development and debugging post for more information.

 

Methods as cmdlets - the basics

Cmdlets in PowerShell are written as classes that derive from the Cmdlet base class in System.Automation, with one core method called ProcessRecord. The simplest cmdlet looks like this:

[Cmdlet("Get", "Greeting")]
class
GetGreetingCmdlet : Cmdlet
{
   public override void ProcessRecord()
   {
      WriteRecord("Hello World");
   }
}

Essentially one could think of the cmdlet in a reverse way: how would this thing look like when encapsulated in a PowerShell-inaware class? Likely something like this:

static class MyStuff
{
   static string GetGreeting()
   {
      return "Hello World";
   }
}

Say we have such a class, how could we go ahead an expose it in PowerShell in a simple and straightforward way? Please welcome the method invocation cmdlet. What we'd like to achieve (and we'll extend this step by step in this and future posts) is this:

[Cmdlet("get", "greeting")]
class GetGreetingCmdlet : MethodCallCmdlet
{
   // We're missing essential information in this cmdlet's metadata: what type to invoke on?
}

Of course we need to think about a lot more than just this scenario: static methods versus instance methods, parameterization with overloading, etc. No worries, we'll get there eventually...

 

Static versus instance

How could we model static versus instance method calls? Let's start with the latter case. A way to allow instance methods is by taking the argument from the PowerShell pipeline and invoking the method on it. Say we want to encapsulate the String.Trim method, we could do it like this:

[Cmdlet("Trim", "String")]
class TrimStringCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public string Input { get; set; }
}

Using it should be as straightforward as:

PS> "  Bart    " | trim-string
Bart

All the magic to invoke the right method would be encapsulated inside the MethodCallCmdlet from which we derive. How would it find the method? Either it needs a hint or it can work automagically by looking at the CmdletAttribute: try Trim() first, then try TrimString(), then fail. If there's a hint, that takes precedence like this:

[Cmdlet("Trim", "String"), InstanceMethod("Trim")]
class TrimStringCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public string Input { get; set; }
}

Static methods need a hint anyway (the type to invoke the static method on) and don't consume input from the pipeline (for now...):

[Cmdlet("get", "greeting"), StaticMethod(typeof(MyStuff))]
class GetGreetingCmdlet : MethodCallCmdlet
{
}

 

Why this exercise?

There are various reasons. First of all, it has value by itself: the ability to encapsulate a method in a cmdlet in a straightforward manner seems to have some useful applications. Secondly, it just happened I was writing quite some plumbing code for some project on dynamic method invocations that largely applies over here (although the version I'm presenting here is greatly simplified concerning overloads and generic parameters). Also, this post will show some interesting real applications of LINQ to Objects that escape from the typical sorts of applications. Last but not least, it's part of a larger thing I have in mind that may or may not appear on the surface of the globe over time.

 

A design overview

I already explained how we're going to support static methods versus instance methods. Another thing to think about is parameterization: methods take arguments and have possibly overloads. The latter requires some advanced infrastructure to determine the right overload which on its turn requires intrinsic knowledge of the call that's taking place, so we won't focus on this right now as it can get fairly complicated. A more important thing though is establishing some mapping for these method arguments. A good way to do this seems to be the use of positional PowerShell parameters. For example, an instance method call with one parameter could look like this:

[Cmdlet("Add", "Days")]
public class AddDaysCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public DateTime Input { get; set; }

    [Parameter(Mandatory = true, Position = 1)]
    public double NumberOfDays { get; set; }
}

The invocation of such a method call cmdlet would look like this:

PS> [DateTime]::Now | Add-Days -NumberOfDays 10
PS> [DateTime]::Now | Add-Days -N 10
PS> [DateTime]::Now | Add-Days 10

The distinction between Mandatory parameters and non-mandatory ones could be used for overloading and we'll provide some amount of plumbing for it although the finishing touch won't be there to distinguish between different overloads at runtime. It suffices to say that for such a thing you'd either need runtime information about the parameters that were provided to the cmdlet invocation or there'd need to be some über-Nullable concept that works on reference types as well (I'll leave it to the reader to figure out why this is a true statement :-)).

 

Custom attributes

Let's start by defining the custom attribute definitions we'll use in our implementation:

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)]
public sealed class InstanceMethodAttribute : TargetMethodAttribute
{
    public InstanceMethodAttribute() : base(null, null) { }
    public InstanceMethodAttribute(string methodName) : base(methodName, null) { }
}

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = false)]
public sealed class StaticMethodAttribute : TargetMethodAttribute
{
    public StaticMethodAttribute(Type type) : base(null, type) { }
    public StaticMethodAttribute(Type type, string methodName) : base(methodName, type) { }
}

public abstract class TargetMethodAttribute : Attribute
{
    protected TargetMethodAttribute(string methodName, Type staticType)
    {
        MethodName = methodName;
        StaticType = staticType;
    }

    public string MethodName { get; set; }
    public Type StaticType { get; set; }
}

These are pretty simply to understand. Although different designs are possible where there'd be just one custom attribute with some enum to denote the type, we'll stick with this design for now since it serves our purpose well.

 

Where all the magic happens: ProcessRecord

Small methods are key, so the following fragment might be a little disappointing for now, but this is where all the magic happens...

public class MethodCallCmdlet : Cmdlet
{
    protected override void ProcessRecord()
    {
        //
        // Implementing cmdlet's type.
        //
        Type cmdletType = GetType();

        //
        // Get custom attributes.
        //
        TargetMethodAttribute tma = GetTargetMethodAttribute(cmdletType);
        CmdletAttribute ca = GetCmdletAttribute(cmdletType);

        //
        // Static or not?
        //
        if (tma != null && tma.StaticType != null)
            ProcessRecordForStaticMethod(cmdletType, tma, ca);
        else
            ProcessRecordForInstanceMethod(cmdletType, tma, ca);
    }

    ...
}

As you know by now, cmdlets derive from Cmdlet (more advanced ones from PSCmdlet) and implement ProcessRecord. So that's what we do. Let's zoom in a little: first we'll load the metadata from the custom attributes, which is fairly trivial:

private static TargetMethodAttribute GetTargetMethodAttribute(Type cmdletType)
{
    //
    // Only one (subtype of) TargetMethodAttribute is allowed.
    //

    var tmas = cmdletType.GetCustomAttributes(typeof(TargetMethodAttribute), false).Cast<TargetMethodAttribute>();
    if (tmas.Count() > 1)
        throw new InvalidOperationException("Invalid number of TargetMethod attributes on " + cmdletType);

    return tmas.FirstOrDefault();
}

private static CmdletAttribute GetCmdletAttribute(Type cmdletType)
{
    //
    // Need to find the Cmdlet attribute.
    //
    var ca = cmdletType.GetCustomAttributes(typeof(CmdletAttribute), false).Cast<CmdletAttribute>().FirstOrDefault();
    if (ca == null)
        throw new InvalidOperationException("Missing Cmdlet attribute");

    return ca;
}

Notice the lightweight use of LINQ to cast the sequence of retrieved attributes and to extract the first one. For our TargetMethodAttribute we need to do a bit more checking because our class-hierarchy allows multiple applications of the subclass attributes, something would be avoided in a design with just one custom attribute (though such a design would require a few less-intuitive constructor overloads for the custom attribute).

The real stuff goes on in ProcessRecordFor*Method. These two methods are fairly simple as well:

private void ProcessRecordForStaticMethod(Type cmdletType, TargetMethodAttribute tma, CmdletAttribute ca)
{
    //
    // Do real processing.
    //

    ProcessRecordInternal(cmdletType, tma, ca, tma.StaticType, null, BindingFlags.Static);
}

private void ProcessRecordForInstanceMethod(Type cmdletType, TargetMethodAttribute tma, CmdletAttribute ca)
{
    //
    // Determine the lhs "this" parameter (the one bound to the pipeline).
    //
    object target = GetThisParameter(cmdletType);

    //
    // Do real processing.
    //
    ProcessRecordInternal(cmdletType, tma, ca, target.GetType(), target, BindingFlags.Instance);
}

Don't worry about the parameters yet, it will become clear soon. Parameters 4 to 6 are the ones that differ. The last one is fairly intuitive and passes on the type of method we're looking for. Parameters 4 and 5 denote the declaring type of the method (which is straightforward in the static case since it's specified in the StaticMethodAttribute) and the invocation target respectively.

 

The instance method case: who are we calling?

That's what the GetThisParameter method is for. It basically searches for the ParameterAttribute in the current class that is marked as ValueFromPipeline. There's a little bit of magic going on here:

private object GetThisParameter(Type cmdletType)
{
    //
    // Get the property with a ParameterAttribute
    //
    var pi = (from prop in cmdletType.GetProperties()
              let pa = (ParameterAttribute)prop.GetCustomAttributes(typeof(ParameterAttribute), false).SingleOrDefault()
              where pa != null && pa.ValueFromPipeline
              select prop).SingleOrDefault();

    if (pi == null)
        throw new InvalidOperationException("Couldn't find cmdlet parameter bound to the pipeline");

    //
    // Get the underlying object. In case it's a PSObject, unwrap it.
    //
    object target = pi.GetValue(this, null);
    PSObject psTarget = target as PSObject;
    if (psTarget != null)
        target = psTarget.BaseObject;

    return target;
}

Woohoo! A real LINQ expression :-). In the first line we get the one and only Parameter that's bound to the pipeline. Basically we retrieve the set of ParameterAttribute attributes on all the properties individually and check whether or not the Parameter has the ValueFromPipeline property set. SingleOrDefault returns null if there's no result or the result otherwise (and throws an exception if there's more than one instance - a case we don't check explicitly in here). The second part deserves some explanation as well. Since PowerShell encapsulates objects in a PSObject in certain cases we detect this special case since we want to call a method on the real object, not on some wrapper around it. So if we see a PSObject, we unwrap the object that's nested inside.

 

Where all the magic happens (seriously!): ProcessRecordInternal

This is the real core of the cmdlet:

private void ProcessRecordInternal(Type cmdletType, TargetMethodAttribute tma, CmdletAttribute ca, Type targetType, object target, BindingFlags flags)
{
    //
    // Get candidate overloads.
    //
    var overloads = GetMethodOverloads(tma, ca, targetType, flags);

    //
    // Get the cmdlet method parameters.
    //
    PropertyInfo[] reqParams;
    PropertyInfo[] optParams;
    GetMethodParameters(cmdletType, flags == BindingFlags.Instance, out reqParams, out optParams);

    //
    // Find the right overload. First longest match is our preference.
    // Notice we don't figure out which parameters were specified, so there's no real notion of runtime overloading.
    //
    MethodInfo match = (from overload in overloads
                        orderby overload.Method.GetParameters().Length descending
                        where overload.Matches(reqParams, optParams)
                        select overload.Method).FirstOrDefault();
    if (match == null)
        throw new InvalidOperationException("No suitable overload found.");

    //
    // Invoke the method and return the result.
    //

    WriteObject(Invoke(match, target, reqParams, optParams));
}

Here we do some lightweight (as mentioned before) form of overload resolution. It's needed anyhow to determine a good match but its behavior isn't really dynamic at runtime (meaning that omitting a certain parameter causes another method overload to be called dynamically). In fact, method call cmdlets could do some preprocessing of the cmdlet metadata and keep it aside for subsequent calls in order to optimize for performance, but we won't go that far in this post.

 

Finding overloads

Finding overloads happens through reflection once more and again we're going to use LINQ. We represent a method overload by an instance of MethodOverload (original isn't it?) that captures the MethodInfo as well as an array of types for the parameters. This is merely a convenience thing.

private static IEnumerable<MethodOverload> GetMethodOverloads(TargetMethodAttribute tma, CmdletAttribute ca, Type type, BindingFlags flags)
{
    flags |= BindingFlags.Public;

    //
    // Find suitable overloads on the specified type with the specified candidate names.
    //
    return from methodName in GetMethodNames(tma, ca)
           join mi in type.GetMethods(flags) on methodName equals mi.Name
           select new MethodOverload() {
               Method = mi,
               Parameters = (from p in mi.GetParameters() select p.ParameterType).ToArray()
           };
}

private static IEnumerable<string> GetMethodNames(TargetMethodAttribute tma, CmdletAttribute ca)
{
    //
    // Hint specified?
    //
    if (tma != null && tma.MethodName != null)
        yield return tma.MethodName;

    //
    // Intelligent fallback defaults.
    //
    yield return ca.VerbName + ca.NounName;
    yield return ca.VerbName;
}

The GetMethodNames finds an ordered list of best matches for method names (one could implement a more stricter policy of course) and a join with the real methods available on the specified type takes care of finding the suitable overloads which are projected as MethodOverload instances. The base definition of MethodOverload looks like this although we'll extend it in a minute:

class MethodOverload
{
    public MethodInfo Method { get; set; }
    public Type[] Parameters { get; set; }

    ...
}

 

Determining method parameters from the cmdlet metadata

This task is again one we can use LINQ for:

private static void GetMethodParameters(Type cmdletType, bool hasThisParam, out PropertyInfo[] mandatory, out PropertyInfo[] optional)
{
    var res = from p in
                  (from prop in cmdletType.GetProperties()
                   let pa = (ParameterAttribute)prop.GetCustomAttributes(typeof(ParameterAttribute), false).SingleOrDefault()
                   where pa != null
                   select new { Property = prop, Attribute = pa })
              where !p.Attribute.ValueFromPipeline && p.Attribute.Position > (hasThisParam ? 0 : -1)
              orderby p.Attribute.Position
              select p;

    mandatory = res.TakeWhile(p => p.Attribute.Mandatory).Select(p => p.Property).ToArray();
    optional = res.SkipWhile(p => p.Attribute.Mandatory).Select(p => p.Property).ToArray();
}

In the inner query we project all the cmdlet type's properties that have a ParameterAttribute onto an anonymous type that holds the PropertyInfo and the ParameterAttribute. In the outer query we filter out parameters from the pipeline (these are reserved as the "this" parameter for instance method calls as explained above) and the lifted "this" parameter on position 0 in case there's such a parameter (currently the hasThisParam method argument is only set when we handle instance method calls, but there's another case where this could be useful... tip: new Orcas language features). Furthermore we order by Position to get the ordered list of parameters that we'll feed into the underlying method based on the mapping scheme.

Finally, we distinguish between mandatory and optional parameters and piggyback on the characteristic that mandatory parameters should come before optional ones, at least when we consider positional parameters (something we should validate in a real production quality implementation, to make sure no "pure named" parameters are present). To retrieve both lists of parameter categories we can take advantage of the TakeWhile and SkipWhile LINQ extension methods.

 

The final logic before ...

Now all the metadata extracting machinery is in place we can focus on matching the right overload and calling the method. We saw the code before but let's repeat it:

    //
    // Find the right overload. First longest match is our preference.
    // Notice we don't figure out which parameters were specified, so there's no real notion of runtime overloading.
    //
    MethodInfo match = (from overload in overloads
                        orderby overload.Method.GetParameters().Length descending
                        where overload.Matches(reqParams, optParams)
                        select overload.Method).FirstOrDefault();
    if (match == null)
        throw new InvalidOperationException("No suitable overload found.");

    //
    // Invoke the method and return the result.
    //

    WriteObject(Invoke(match, target, reqParams, optParams));

The matching logic is straightforward once more as it relies for it's core functionality on a method called Matches defined on the MethodOverload class:

internal bool Matches(PropertyInfo[] reqParams, PropertyInfo[] optParams)
{
    //
    // Should have at least as much parameters as the number or required parameters.
    //
    if (reqParams.Length > Parameters.Length)
        return false;

    //
    // Check required parameters. All need to match.
    //
    int i;
    for (i = 0; i < reqParams.Length; i++)
    {
        if (!MatchParameter(reqParams[i], Parameters[i]))
            return false;
    }

    //
    // Check optional parameters. If we run out of parameters before matching all optional parameters, that's fine. However, a mismatch causes termination.
    //
    int j;
    for (j = 0; j < optParams.Length && i < Parameters.Length; j++, i++)
    {
        if (!MatchParameter(optParams[j], Parameters[i]))
            return false;
    }

    //
    // Only valid if no parameters left.
    //
    return i >= Parameters.Length;
}

private static bool MatchParameter(PropertyInfo p, Type parameterType)
{
    //
    // Assignable is okay.
    //

    return (p.PropertyType.IsAssignableFrom(parameterType));
}

This implementation takes in the sets of required and optional parameters and tries to match the required ones (which are - what's in a name - required) and as much of the optional ones as it can find. I decided to leave this machinery (which is essentially ported from some other dynamic language project I'm doing) in although such overload resolution isn't fully implemented in this sample. If you're interested in more details, feel free to drop me a mail.

Note: This is something that easily deserves lots of refinements (e.g. to allow parameters with different types to match convertible types - e.g. assigning an int to a long parameter) but let's stick with simplicity for the purpose of this post. Other refinements (that go definitely out of the scope of this post) could handle translation of PS ScriptBlocks into anonymous methods (lambda BLOCKED EXPRESSION which will become more or less possible with the new PSParser class in PowerShell 2.0.

 

... we finally invoke the method

Simplicity at last:

private object Invoke(MethodInfo match, object target, PropertyInfo[] reqParams, PropertyInfo[] optParams)
{
    //
    // Get the parameters.
    //

    object[] parameters = (from p in reqParams.Concat(optParams).Take(match.GetParameters().Length)
                           select p.GetValue(this, null)).ToArray();

    //
    // Invoke the method.
    //
    WriteVerbose(match.ToString() + " on " + match.DeclaringType.ToString());
    return match.Invoke(target, parameters);
}

The last line does the real stuff while the first line of code prepares the call by grabbing all the required Parameter property values again using a simple LINQ statement.

 

Action!

image

The cmdlets used in the sample are shown below:

[Cmdlet("Get", "Greeting"), StaticMethod(typeof(MyStuff))]
public class GetGreetingCmdlet : MethodCallCmdlet
{
}

[Cmdlet("Add", "Days")]
public class AddDaysCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public DateTime Input { get; set; }

    [Parameter(Mandatory = true, Position = 1)]
    public double NumberOfDays { get; set; }
}

[Cmdlet("Trim", "String")]
public class TrimStringCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public string Input { get; set; }
}

[Cmdlet("Replace", "String")]
public class ReplaceStringCmdlet : MethodCallCmdlet
{
    [Parameter(Mandatory = true, ValueFromPipeline = true)]
    public string Input { get; set; }

    [Parameter(Mandatory = true, Position = 1)]
    public string Old { get; set; }

    [Parameter(Mandatory = true, Position = 2)]
    public string New { get; set; }
}

The creation of these wrapper cmdlets is even so mechanical you could write a reflection-based tool for it (hint!). Update: I've uploaded the sources for download over here. Keep in mind this is sample code, so treat it as such (no guarantees are made).

Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Comments

# re: Method invocation cmdlets in Windows PowerShell

Tuesday, February 05, 2008 3:01 PM by Mike Street

Bart,

Is there somewhere that the complete code is posted, as it is no quite clear how all of this hangs together

Bedankt !

# re: Method invocation cmdlets in Windows PowerShell

Tuesday, February 05, 2008 9:00 PM by bart

Hi Mike,

Thanks for your feedback. Although this is part of a bigger project I'm doing in my free time, I extracted the relevant sources into a separate project and added a link to the post at the bottom. As usual, no guarantees are made expressed or implied.

Hope this helps,

-Bart

# LINQ through PowerShell

Saturday, June 07, 2008 8:44 PM by B# .NET Blog

In a reaction to my post on LINQ to MSI yesterday, Hal wrote this: I don&#39;t know enough about the

# BUGBUG: poor title &raquo; Blog Archive &raquo; Brilliant Powershell posts

Pingback from  BUGBUG: poor title  &raquo; Blog Archive   &raquo; Brilliant Powershell posts

# User links about "methodoverload" on iLinkShare

Tuesday, April 07, 2009 5:18 PM by User links about "methodoverload" on iLinkShare

Pingback from  User links about "methodoverload" on iLinkShare

# PowerShell method overload resolution bug? | Q Sites

Friday, July 19, 2013 11:20 PM by PowerShell method overload resolution bug? | Q Sites

Pingback from  PowerShell method overload resolution bug? | Q Sites