Windows PowerShell through IronRuby - Writing a custom PSHost

bart — Sun, 06 Jul 2008 08:00:00 GMT

Introduction

Lately I've been playing quite a bit with DLR technologies, including IronRuby. During some experiments I came to the conclusion that the Kernel.` method isn't implemented yet in the current version. This `backtick` method allows executing OS commands from inside a Ruby program. It's a bit like Process.Start, redirecting the standard output as a string to the Ruby program for further use (actually the Kernel.exec method is precisely implemented like this).

One thing I like about the NotImplementedException is the fact it points unambiguously to the source that's missing :-). Indeed, if you browse the IronRuby source, you'll find this:

[RubyMethod("`", RubyMethodAttributes.PrivateInstance)]
[RubyMethod("`", RubyMethodAttributes.PublicSingleton)]
public static MutableString ExecuteCommand(CodeContext/*!*/ context, object self, [NotNull]MutableString/*!*/ command) {
// TODO:
throw new NotImplementedException();
}

Actually there are a couple of things here that are worth to discuss besides the current void of the method body:

Strings in Ruby are mutable as opposed to strings in the CLR/BCL. Therefore they are wrapped in a MutableString class.
The weird /*!*/ notation indicates not-nullness, mirrored after the equivalent uncommented form in Spec# (e.g. MutableString! means a null-nullable string). To work with regular C#, the NotNullAttribute is used.
Methods like this one are not invoked by Ruby directly, instead the RubyMethodAttribute declaration carries the metadata that provides the entry-point to the method (as well as other metadata).

Kernel.`

I won't cover the differences between Kernel.`, Kernel.system and Kernel.exec; more information can be found here. The backtick one is our target for the scope of this post:

`cmd` => string

Returns the standard output of running cmd in a subshell. The built-in syntax %x{…} uses this method. Sets $? to the process status.
   `date`                   #=> "Wed Apr  9 08:56:30 CDT 2003\n"
   `ls testdir`.split[1]    #=> "main.rb"
   `echo oops && exit 99`   #=> "oops\n"
   $?.exitstatus            #=> 99

Actually I want to focus on (yet another) powerful feature in PowerShell, namely the ability to create custom hosts. What we want to achieve here is that the backtick syntax (or the equivalent %x syntax) runs the specified command as a PowerShell command (or pipeline of multiple cmdlets), emitting the string output as the `'s methods return value. Notice though this actually downgrades ones of the core principles of PowerShell concerning the use of objects through the pipeline rather than falling back to strings. One could easily think of a more powerful way to expose the results of a PowerShell invocation as PSObjects in Ruby but we'll keep that for later.

Important: This post outlines no more than the capability to hook up PowerShell in IronRuby through Kernel.`. Obviously no promises are made about the way Kernel.` will eventually be implemented in IronRuby as we move forward.

Building a custom PS host

Creating custom PS hosts isn't that hard, depending on how much functionality you want to take over. We'll stick with the basics of console I/O, actually just the O in this. What we want to get done is this:

Build up a runspace containing the command passed to Kernel.` (in addition to some more pipeline commands to produce the right output, see further).
Invoke the built-up pipeline.
Retrieve the string output from the host, concatenate it into one big string and return that one to the caller of Kernel.`.

Preparing for PowerShell programming

In order to extend PowerShell, you'll need to add a reference to System.Management.Automation.dll which can be found in the Reference Assemblies folder (click to enlarge):

Runspaces

Let's start at the very top by implementing a method called "InvokePS" that sets up the infrastructure to call PowerShell:

public static class RubyToPS
{
    public static string InvokePS(string command)
    {
        RubyPSHost host = new RubyPSHost();

        using (Runspace runspace = RunspaceFactory.CreateRunspace(host))
        {
            runspace.Open();

            using (Pipeline pipeline = runspace.CreatePipeline())
            {
                pipeline.Commands.AddScript(command);
                pipeline.Commands[0].MergeMyResults(PipelineResultTypes.Error, PipelineResultTypes.Output);                 pipeline.Commands.Add("out-default");

                pipeline.Invoke();
            }
        }

        return ((RubyPSHostUserInterface)host.UI).Output;
    }
}

The RubyPSHost class will be shown next, let's focus on the Runspace stuff for now. A runspace serves as the entry-point to the PowerShell engine and encapsulates all the state needed to execute pipelines. Once we've opened the runspace, a pipeline is created to which we add the passed-in command as a script. This allows more than just one cmdlet invocation to be executed (e.g. "gps | where { $_.WorkingSet64 -gt 50MB }"). To send output to the host we append Out-Default to the pipeline:

NAME
    Out-Default

SYNOPSIS
    Send the output to the default formatter and the default output cmdlet. Thi
    s cmdlet has no effect on the formatting or output. It is a placeholder tha
    t lets you write your own Out-Default function or cmdlet.

SYNTAX
    Out-Default [-inputObject <psobject>] [<CommonParameters>]

DETAILED DESCRIPTION
    The Out-Default cmdlet send the output to the default formatter and the def
    ault output cmdlet. This cmdlet has no effect on the formatting or output.
    It is a placeholder that lets you write your own Out-Default function or cm
    dlet.

The MergeMyResults call is used to ensure that error objects produced by the first command are merged into the output (otherwise you'll get an exception instead). Finally the output is retrieved from the host after invoking the pipeline. How this works will be covered in a minute.

To read more about PowerShell runspaces, check out my other posts on the topic.

Deriving from PSHost

Custom PowerShell hosts derive from the abstract PSHost base class. There's quite some stuff that can be done here but we'll stick with the absolute minimum functionality required to reach our goals:

internal class RubyPSHost : PSHost
{
    private Guid _hostId = Guid.NewGuid();
    private RubyPSHostUserInterface _ui = new RubyPSHostUserInterface();

    public override Guid InstanceId
    {
        get { return _hostId; }
    }

    public override string Name
    {
        get { return "RubyPSHost"; }
    }

    public override Version Version
    {
        get { return new Version(1, 0); }
    }

    public override PSHostUserInterface UI
    {
        get { return _ui; }
    }

    public override CultureInfo CurrentCulture
    {
        get { return Thread.CurrentThread.CurrentCulture; }
    }

    public override CultureInfo CurrentUICulture
    {
        get { return Thread.CurrentThread.CurrentUICulture; }
    }

    public override void EnterNestedPrompt()
    {
        throw new NotImplementedException();
    }

    public override void ExitNestedPrompt()
    {
        throw new NotImplementedException();
    }

    public override void NotifyBeginApplication()
    {
        return;
    }

    public override void NotifyEndApplication()
    {
        return;
    }

    public override void SetShouldExit(int exitCode)
    {
        return;
    }
}

More information about all of those methods and properties can be found on MSDN. The most important one to us the the UI property that points at our PSHostUserInterface implementation called RubyPSHostUserInterface.

Implementing PSHostUserInterface

Where the PSHost class provides basic information concerning the metadata of the host (name, version, id)m lifetime of the host (nested prompt, execit commands) and general settings (cultures), the PSHostUserInterface class deals with "dialog-oriented and line-oriented interaction between the cmdlet and the user, such as writing to, prompting for, and reading from user input" (from MSDN). The part we're interested in the writing to part. We won't deal with prompts or user interaction - if one wants to do this, the Kernel.` command is no longer non-interactive (a possible alternative way to implement this would be to spawn PowerShell.exe and just get the shell's output here - the default host would take care of all user interaction if required; the only problem is that prompts would appear in the Kernel.` output as well). Implementation of this class isn't that hard either:

internal class RubyPSHostUserInterface : PSHostUserInterface
{
    private StringBuilder _sb;

    public RubyPSHostUserInterface()
    {
        _sb = new StringBuilder();
    }

    public override void Write(ConsoleColor foregroundColor, ConsoleColor backgroundColor, string value)
    {
        _sb.Append(value);
    }

    public override void Write(string value)
    {
        _sb.Append(value);
    }

    public override void WriteDebugLine(string message)
    {
        _sb.AppendLine("DEBUG: " + message);
    }

    public override void WriteErrorLine(string value)
    {
        _sb.AppendLine("ERROR: " + value);
    }

    public override void WriteLine(string value)
    {
        _sb.AppendLine(value);
    }

    public override void WriteVerboseLine(string message)
    {
        _sb.AppendLine("VERBOSE: " + message);
    }

    public override void WriteWarningLine(string message)
    {
        _sb.AppendLine("WARNING: " + message);
    }

    public override void WriteProgress(long sourceId, ProgressRecord record)
    {
        return;
    }

    public string Output
    {
        get
        {
            return _sb.ToString();
        }
    }

    public override Dictionary<string, PSObject> Prompt(string caption, string message, System.Collections.ObjectModel.Collection<FieldDescription> descriptions)
    {
        throw new NotImplementedException();
    }

    public override int PromptForChoice(string caption, string message, System.Collections.ObjectModel.Collection<ChoiceDescription> choices, int defaultChoice)
    {
        throw new NotImplementedException();
    }

    public override PSCredential PromptForCredential(string caption, string message, string userName, string targetName, PSCredentialTypes allowedCredentialTypes, PSCredentialUIOptions options)
    {
        throw new NotImplementedException();
    }

    public override PSCredential PromptForCredential(string caption, string message, string userName, string targetName)
    {
        throw new NotImplementedException();
    }

    public override PSHostRawUserInterface RawUI
    {
        get { return null; }
    }

    public override string ReadLine()
    {
        throw new NotImplementedException();
    }

    public override System.Security.SecureString ReadLineAsSecureString()
    {
        throw new NotImplementedException();
    }
}

The core of our implementation lies in the fact that all Write* methods emit their data to a StringBuilder instance that aggregates all output sent to the host. This is the data that gets retrieved by our InvokePS method on the last line:

return ((RubyPSHostUserInterface)host.UI).Output;

Notice this isn't the absolute end of host-level extensibility in PowerShell. A PSHostUserInterface class can point at a PSHostRawUserInterface object that controls host window characteristics (such as the size, position and title of the window). Actually it would be interesting to implement this one as well in order to provide an accurate BufferSize that will be used by PowerShell to control the maximum length of individual lines before wrapping to the next line. The reason this would be a good idea is that screen-scraping Ruby programs should be able to ignore different wrapping behavior depending on the hosting command window (which would cause programs to behave differently depending where they run). Ideally there would be no wrapping at all (letting the DLR IronRuby command-line host dealing with wrapping when printing data to the screen). I'll leave this exercise to the reader.

Hooking it up

All of the above has been implemented in a separate strong-named Class Library which I'm just referencing in the IronRuby.Libraries project. This is actually very quick-and-dirty, making IronRuby directly dependent on our assembly and by extension Windows PowerShell. A way around this would be to load the assembly dynamically possibly based on an environment variable. There are lots of possibilities here which we consider just an implementation detail for now. The only thing left to do is to call our InvokePS method which requires some conversions between System.String and MutableString:

[RubyMethod("`", RubyMethodAttributes.PrivateInstance)]
[RubyMethod("`", RubyMethodAttributes.PublicSingleton)]
public static MutableString ExecuteCommand(CodeContext/*!*/ context, object self, [NotNull]MutableString/*!*/ command) {
return MutableString.Create(RubyToPS.InvokePS(command.ConvertToString()));
}

That's it! Here's the result:

Note: The \r\n insertions in the output for display by Ruby's console cause things to wrap a bit nasty given the default of 80 characters buffer width. I've adjusted to 83 characters to make this render correctly. With some smart "Raw UI host" one could eliminate some issues here - however the internal contents of the string is more important since the app will likely rely on that (otherwise you'd simply run a PowerShell interactive shell, wouldn't you?). Just as one sample, here's the output of the each_line iterator:

Does look an awful lot like PowerShell, doesn't it?

Cheers!

B# .NET Blog : IronRuby