Windows PowerShell

I'm a firm believer of the "innovation through integration" theme. As a fan of MSBuild and PowerShell I wondered what it would take to bring the two worlds closer together. This post outlines the result of a short but 'powerful' experiment. In order to read this post, I strongly recommend to check out my recent post named The custom MSBuild task cookbook to learn about writing and debugging custom MSBuild tasks.

 

Hosting PowerShell

In order to run PowerShell in a customized environment (as opposed to the default shell that comes with the technology) one needs to work with runspaces. Essentially a runspace allows to host the PowerShell engine and interact with it through pipelines. We'll only cover very basic communication in this post. If one wants to feed back data from PowerShell to the MSBuild output for instance, a PSHost implementation would be required but that goes far beyond the scope of this post.

I've posted about runspaces earlier in my A first introduction to Windows PowerShell runspaces post about one year ago. You might want to check out that post for more information on hosting PowerShell.

 

Introducing PSBuild

Far from original, I admit, but let's call our baby PSBuild. In order to implement it, create a new class library project (C#) and add references to the following assemblies:

image

Including the MSBuild assemblies has been covered in the The custom MSBuild task cookbook post; for more information on the System.Management.Automation assembly, see my Easy Windows PowerShell cmdlet development and debugging post (step 2).

 

Implementing the task

First on our to-do list is implementing the custom MSBuild task in the C# code file. It's barely 45 lines:

using System;
using System.Management.Automation;
using System.Management.Automation.Runspaces;
using System.Text;
using Microsoft.Build.Framework;
using Microsoft.Build.Utilities;

namespace PSBuild
{
    public class InvokeScript : Task
    {
        [Required]
        public ITaskItem Script { get; set; }

        [Required]
        public string Function { get; set; }
        public ITaskItem[] Parameters { get; set; }

        public override bool Execute()
        {
            RunspaceConfiguration runspaceConfig = RunspaceConfiguration.Create();

            using (Runspace runspace = RunspaceFactory.CreateRunspace(runspaceConfig))
            {
                runspace.Open();

                StringBuilder commandLine = new StringBuilder();
                commandLine.Append(Function + " ");

                foreach (ITaskItem parameter in Parameters)
                {
                    commandLine.AppendFormat("\"{0}\" ", parameter.ItemSpec);
                }

                using (RunspaceInvoke scriptInvoker = new RunspaceInvoke(runspace))
                {
                    scriptInvoker.Invoke(Script.ItemSpec);
                    scriptInvoker.Invoke(commandLine.ToString());
                }
            }

            return true;
        }
    }
}

Usual disclaimers apply - this code is far from ideal and is just meant to illustrate the concept. Talking about a concept... Let's discuss:

  • We require two parameters: Script containing a PowerShell script block and Function pointing to the function to be invoked. The Parameters for invocation are optional since a function may have no parameters obviously.
  • Notice the type of the Script and Parameters properties. By using ITaskItem we can integrate nicely with MSBuild as we'll see further.
  • The Execute method does all the work. Essentially we create a Runspace and invoke two commands: first is the Script definition itself, second is the invocation of the script based on the Function and Parameters values.
  • Error handling was omitted from the code above - a production quality implementation needs to catch errors and return false in case of an error. Also, some logging (Log property on Task) would be welcome, e.g. to print the command-line that's being invoked (tip: Log.LogCommandLine).

 

Testing the task

Check out my The custom MSBuild task cookbook post for instructions on testing custom MSBuild tasks. I'll just show a sample MSBuild file below that invokes a script:

image

The UsingTask imports our task library built in the previous step. To define the script, we simply define a MyScript tag under a PropertyGroup element. In here we define a function called "ProcessList" that takes in two arguments. I've spread it across two lines to show that local variables (and in extension to that - try it yourself :-) - more advanced scripting techniques) simply work. Finally, we invoke our InvokeScript task somewhere, in this case in the Debug target (again, see The custom MSBuild task cookbook for more info) but you could imagine it to be part of your core build definition. The InvokeScript task references the MyScript through the property reference syntax $(...) of MSBuild; the Function is a simple string and in Parameters we put a semi-colon separated list of parameters which will be assigned $args[0] ... $args[n] in the invoked PowerShell script.

One could imagine the parameterization of the InvokeScript task to be much more complete and flexible (e.g. one could drop the Function attribute and simply execute some script) but that's just a matter of implementation. Also a way to feed back results isn't too difficult (RunspaceInvoke::Invoke returns a collection of PSObjects).

Here's what it does:

image

Notice that one can use any MSBuild variable in the parameterization which gives us a tremendous amount of power. For example, one could write a script that pre-processes all files in @(Compile), and leverage all of the PowerShell and .NET Framework power to do so. I leave it to the reader to experiment with the possibilities.

 

Happy PSBuilding!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Before posting another post in the Windows PowerShell domain, I thought it was useful to write down the way to write a simple cmdlet in Visual Studio, with easy iterative debugging support. I've used this recipe myself quite a bit, including in on-stage presentations where you want to avoid messing up things at any cost :-).

 

Step 1 - Create a Class Library project

In Visual Studio, create a new Class Library project in the language of your choice (e.g. C#).

image

 

Step 2 - Import references

Next, we need to reference two assemblies. The first one is the System.Management.Automation assembly from Windows PowerShell. If you've installed the Windows SDK on your machine, you'll find it under %programfiles%\Reference Assemblies\Windows Powershell\v1.0:

image

Add a reference to it:

image

Next, add a reference to System.Configuration.Install because our cmdlet needs installer support in order to register its snap-in (see below):

image

Solution Explorer should look like this now:

image

 

Step 3 - Write the basics of the cmdlet

In the Class1.cs file of your project (feel free to rename obviously), write your first cmdlet by deriving from Cmdlet:

image

Use the smart tip to import System.Management.Automation:

image

Finally implement the basic cmdlet by overriding ProcessRecord:

image

And adding some code to it:

image

Finally don't forget to add the CmdletAttribute to it (in practice use VerbsCommon to get a list of common verbs to be used as the cmdlet verb instead of my "Say" verb below which isn't standard PS vocab):

image

 

Step 5 - Add a snap-in

Snap-ins are used to deploy a set of cmdlets, providers, etc as a management package. Creating one is easy by adding a class deriving from PSSnapIn, which is an abstract class with three properties:

image

Don't forget to attribute it with RunInstaller(true) which requires to import System.ComponentModel:

image

image

 

Step 6 - Build and register the snap-in

Build the solution, and open a Visual Studio Command Prompt running as Administrator. Go to the bin\Debug output folder of your project and invoke installutil -i on the compiled assembly as shown below:

image

 

Step 7 - Set up a debugging console for our snap-in

Now launch Windows PowerShell and invoke get-pssnapin -registered (don't forget -registered!) to see that our registration was successful:

image

In the screenshot above we've also added the snap-in using add-pssnapin MySnapIn and tested it by invoking our say-hello cmdlet. Next we export the console configuration to a psc1 (PowerShell Console v1) file:

image

This is an XML file containing the configuration of a PowerShell console with respect to imported snap-ins, as show below:

image

 

Step 8 - Hook up the debugger

Close Windows PowerShell and go back to our project. Open the Project Properties and go to the Debug tab. In here, specify the program to launch as Windows PowerShell and the arguments to be -PSConsoleFile <path to your debug.psc1 file>:

image

To test, set a breakpoint in our cmdlet and hit F5:

image

Invoke the cmdlet from the started Windows PowerShell instance:

image

The breakpoint is hit! Mission completed...Happy cmdlet debugging!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

You've probably already heard about this new feature in .NET 3.5: extension methods. If not, check out a few posts on this topic:

As a quick recap, extension methods allow you to "extend" an existing type with additional methods, without touching the type's implementation itself. In fact, it's nothing more than syntactical sugar on top of static methods. The code below shows such a set of extension methods written in C# 3.0:

using System;

namespace MyExtensions
{
    public static class Extensions
    {
        private static Random rand = new Random();

        public static string Reverse(this string s)
        {
            char[] c = s.ToCharArray();
            Array.Reverse(c);
            return new string(c);
        }

        public static string Permute(this string s, int n)
        {
            char[] c = s.ToCharArray();

            for (int i = 0; i < n; i++)
            {
                int a = rand.Next(s.Length);
                int b = rand.Next(s.Length);
                char t = c[a];
                c[a] = c[b];
                c[b] = t;
            }

            return new string(c);
        }

        public static int Power(this int i, int n)
        {
            int res = 1;
            for (int j = 0; j < n; j++)
                res *= i;
            return res;
        }
    }
}

These methods allow you to write the following, if the MyExtensions namespace is imported:

string name = "Bart";
string reverseName = name.Reverse(); //traB
string fuzzyName = name.Permute(1); //can produce different results where two letters are switched, e.g. Brat
int n = 2;
int kb = n.Power(10); //1024

So far so good. But what does all of this magic have to do with PowerShell? The answer: nothing (so far). As you know, Windows PowerShell 1.0 was developed in the .NET 2.0 timeframe, so it would be very strange if PowerShell understood extension methods. And indeed, it doesn't ... which made me think how we could make this available using some plumbing. Here's what I came up with.

Windows PowerShell has a feature called the Extended Type System. This allows types to be extended with additional properties, methods, etc in order to provide additional IT admin convenience. For example, take a look at the types.ps1xml file in the %windir%\system32\WindowsPowerShell\v1.0 folder on your system. In there you'll find things like:

<Type>
    <Name>System.Array</Name>
    <Members>
        <AliasProperty>
            <Name>Count</Name>
            <ReferencedMemberName>Length</ReferencedMemberName>
        </AliasProperty>
    </Members>
</Type>

This defines an "alias property" which adds a property called Count to each instance of System.Array, pointing to the Length property available on System.Array. So, you can write this:

> $a = "Bart", "John"
> $a.Count
2

In fact, if you use get-member on $a, you'll see the AliasProperty listed out there (click to enlarge):

image

In a similar way, one can make different types of extensions: Alias Properties, Code Properties, Note Properties, Script Properties, Code Methods, Script Methods. Take a closer look at types.ps1xml for additional samples. Back to our mission now. It seems ETS is an appropriate vehicle to make extension methods available using Script Methods. Basically, we'll provide a script for each extension method that takes the set of original parameters and rewrites these to become parameters of the static method. For example, if you write:

> $name = "Bart"
> $name.Reverse()

the last call should become:

> [MyExtensions.Extensions]::Reverse($name)

Similarly, a Power call on an int should be translated from:

> $n = 2
> $n.Power(10)

into:

> [MyExtensions.Extensions]::Power($n, 10)

Taking possible method overloads into account, we should end up with something like this:

<?xml version="1.0" encoding="utf-16"?>
<Types>
  <Type>
    <Name>System.String</Name>
    <Members>
      <ScriptMethod>
        <Name>Reverse</Name>
        <Script>
          switch ($args.Count) {
            0 { [MyExtensions.Extensions]::Reverse($this) }
            default { throw "No overload for Reverse takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
      <ScriptMethod>
        <Name>Permute</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Permute($this, $args[0]) }
            default { throw "No overload for Permute takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
  <Type>
    <Name>System.Int32</Name>
    <Members>
      <ScriptMethod>
        <Name>Power</Name>
        <Script>
          switch ($args.Count) {
            1 { [MyExtensions.Extensions]::Power($this, $args[0]) }
            default { throw "No overload for Power takes the specified number of parameters." }
          }
        </Script>
      </ScriptMethod>
    </Members>
  </Type>
</Types>

Once we have such a file, it can be "imported" in Windows PowerShell using the Update-TypeData cmdlet:

image

All we have to do is call this cmdlet as follows:

> Update-TypeData -prependPath MyExtensions.ps1xml

But it gets even better: this chunk of XML is something that's an ideal candidate for dynamic code XML generation. Guess what, let's use LINQ for this task and wrap the "extension method export" functionality in a custom cmdlet:

using System;
using System.IO;
using System.Linq;
using System.Management.Automation;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Text;
using System.Xml;
using System.Xml.Linq;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [Cmdlet("Import", "ExtensionMethods")]
    public class ImportExtensionMethods : PSCmdlet
    {
        [Parameter(Mandatory=true)]
        public Assembly Assembly { get; set; }

        [Alias("ns"), Parameter(Mandatory=false)]
        public string Namespace { get; set; }

        protected override void ProcessRecord()
        {
            if (Namespace == null)
                Namespace = "";

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace != null && t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

            StringBuilder sb = new StringBuilder();

            using (TextWriter tw = new StringWriter(sb))
            {
                using (XmlTextWriter xtw = new XmlTextWriter(tw))
                {
                    xtw.Indentation = INDENT;
                    xtw.Formatting = Formatting.Indented;
                    res.WriteTo(xtw);
                }
            }

            base.WriteObject(sb.ToString());
        }

        static int INDENT = 2;

        static string GetScriptFor(IGrouping<string, MethodInfo> m)
        {
            StringBuilder sb = new StringBuilder();
            sb.AppendFormat("\r\n{0}switch ($args.Count) {{\r\n", new String(' ', INDENT * 5));

            foreach (var e in m)
            {
                int n = e.GetParameters().Length - 1;
                sb.AppendFormat("{0}{1} {{ {2} }}\r\n", new String(' ', INDENT * 6), n, GetScriptFor(e.DeclaringType.FullName, e.Name, n));
            }

            sb.AppendFormat("{0}default {{ throw \"No overload for {1} takes the specified number of parameters.\" }}\r\n", new String(' ', INDENT * 6), m.Key);
            sb.AppendFormat("{0}}}\r\n{1}", new String(' ', INDENT * 5), new String(' ', INDENT * 4));

            return sb.ToString();
        }

        static string GetScriptFor(string type, string method, int n)
        {
            StringBuilder sb = new StringBuilder();
            sb.Append("$this");

            for (int i = 0; i < n; i++)
                sb.AppendFormat(", $args[{0}]", i);

            string args = sb.ToString();

            return String.Format("[{0}]::{1}({2})", type, method, args);
        }
    }
}

Quite a bit of code, but lots of plumbing to get a smooth output (notice the code can be improved in many places). The GetScriptFor methods are pretty simple to understand and generate the script for a given (group of) method (overloads) associated with a method name (the second GetScriptFor method is a helper to get the method calls themselves, using the $this and $args variables). For what the core functionality is concerned, take a look at the ProcessRecord method that contains a LINQ query that looks for all extension methods in the given assembly and namespace:

            var res =
                new XDocument(
                    new XElement("Types",
                        from t in Assembly.GetTypes()
                        where t.Namespace IsNot Nothing AndAlso t.Namespace.StartsWith(Namespace)
                        from m in t.GetMethods(BindingFlags.Public | BindingFlags.Static)
                        where m.GetCustomAttributes(typeof(ExtensionAttribute), false).Length == 1
                        group m by m.GetParameters()[0].ParameterType into g
                        where !g.Key.IsGenericType
                        select
                            new XElement("Type",
                                new XElement("Name", g.Key.FullName),
                                new XElement("Members",
                                    from m in g
                                    group m by m.Name into h
                                    select
                                        new XElement("ScriptMethod",
                                           new XElement("Name", h.Key),
                                            new XElement("Script", GetScriptFor(h))
                                    )
                                )
                            )
                    )
                );

In here, we're using the new System.Xml.Linq API to contruct an XML fragment on the fly (LINQ to XML style). A method is considered to be an extension method if it's public, static and marked with a System.Runtime.CompilerServices.ExtensionAttribute custom attribute. Furthermore, it's first parameter (the "this" parameter in C# 3.0) shouldn't be a generic type, since PowerShell doesn't have first-level support for generics at the moment (this restriction rules out the use of System.Core's extension methods unfortunately, at the moment). Using the grouping constructs, methods are grouped per type and per method name (to account for overloads). VB folks have an easier job when it comes down to generating XML fragments. The query above looks as follows in VB 9.0 (more info on VB 9.0 XML integration):

image

When writing a cmdlet, we need a snap-in as its distribution vehicle; below is a simple one:

using System.ComponentModel;
using System.Management.Automation;

namespace BdsSoft.PowerShell.ExtensionMethods
{
    [RunInstaller(true)]
    public class ExtensionMethodsSnapIn : PSSnapIn
    {
        public override string Description
        {
            get { return "This Windows PowerShell snap-in provides support for .NET Framework 3.5 Extension Methods."; }
        }

        public override string Name
        {
            get { return "BdsSoft.PowerShell.ExtensionMethods"; }
        }

        public override string Vendor
        {
            get { return "BdsSoft"; }
        }
    }
}

Strong-name the assembly, build it and run installutil -i against the generated dll file. Finally, create the following PowerShell script, ImportExtensionMethods.ps1:

if ($args.Count -lt 2)
{
   throw "Usage: .\ImportExtensionMethods.ps1 output assembly [namespace]"
}
else
{
   $emOutput = (join-path (split-path $profile) $args[0])

   Import-ExtensionMethods -assembly $args[1] -namespace $args[2] | Out-File $emOutput
   Update-TypeData -PrependPath $emOutput
}

This simplifies calling the Import-ExtensionMethods cmdlet and to update the ETS type data, so that the extensions become available. In the screenshot below, you can see the whole thing in use:

image

All you need to do is:

  1. Make sure the snap-in is loaded, using add-pssnapin (you can move this call to your PowerShell profile if you want to load it automatically).
  2. Call ImportExtensionMethods.ps1, passing in a name for the ETS ps1xml file that should be generated followed by the assembly that contains the extension methods (use Assembly.LoadFrom or Assembly.Load to get the assembly either by file name or by assembly name, i.e. for GAC'ed assemblies).

Of course, once you have the ps1xml files, you could simply adapt your PowerShell profile file in order to load the assemblies and call Update-TypeData to load the ps1xml file. In the next screenshot you can see the generated ps1xml file (click to enlarge):

image 

Have fun!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Dear blogging friends, I'm back from some blog silence due to other priorities during last weeks . Although it will take some time to get back up to speed in the blogosphere, I decided to leave you no longer hungry for some cool stuff and posted this simple warming-up sample of Windows PowerShell runspaces.

 

Runspaces in a few words

One of the coolest things about Windows PowerShell is the fact that the entire command line processor and pipeline infrastructure can be hosted by applications other than the PowerShell.exe executable itself. Basically, that's what the MMC 3.0 layering on top of Windows PowerShell is all about, which has been fully realized in the Exchange Server 2007 product. In this and upcoming posts I want to show you how you can leverage the power of all this magic in your own applications, allowing you to realize such a layering yourself: build cmdlets and host the PS engine!

 

A simple snap-in

In order to illustrate the use of a runspace, we need to create some snap-in first (or you could reuse an existing one). Please revisit my previous posts on Windows PowerShell for more information on the creation of cmdlets; in this post I'll assume you know what a cmdlet is and how you create a parameterized one. Below you can find the code for a snap-in containing two cmdlets which are rather self-explanatory:

using System; using System.Collections.Generic; using System.Text; using System.Management.Automation; using System.ComponentModel; namespace DemoSnapIn { [Cmdlet(VerbsCommon.Get, "Greeting")] public class SayHelloCmdlet : PSCmdlet { private string name; [Parameter(Mandatory = true, Position = 0)] public string Name { get { return name; } set { name = value; } } protected override void ProcessRecord() { this.WriteObject("Hello " + name); } } [Cmdlet(VerbsCommon.Get, "Permutations")] public class PermutationGeneratorCmdlet : PSCmdlet { private string _string; [Parameter(Mandatory = true, Position = 0)] public string String { get { return _string; } set { _string = value; } } protected override void ProcessRecord() { foreach (string s in GetPermutations(_string)) WriteObject(s); } static void Swap(StringBuilder sb, int i, int j) { char t = sb[i]; sb[i] = sb[j]; sb[j] = t; } static IEnumerable<string> GetPermutations(string s) { int len = s.Length; StringBuilder sb = new StringBuilder(len); sb.Append(s); return GetPermutations(sb, 0, len - 1); } static IEnumerable<string> GetPermutations(StringBuilder sb, int b, int e) { if (b == e) yield return sb.ToString(); else { for (int i = 0; i <= e - b; i++) { Swap(sb, b, b + i); foreach (string r in GetPermutations(sb, b + 1, e)) yield return r; Swap(sb, b + i, b); } } } } [RunInstaller(true)] public class MySnapIn : PSSnapIn { public override string Name { get { return "DemoSnapIn"; } } public override string Description { get { return "Sample snap-in"; } } public override string Vendor { get { return "Bart"; } } } }

Note: the permutation searcher is just a simple straightforward implementation to illustrate cmdlets that produce multiple results; please ignore the algorithm used which has a bit of a quick-n-dirty fashion :-)

Put the code above in a C# class library project, add a reference to System.Management.Automation (you might want to copy the System.Management.Automation.dll file from the GAC to some folder in order to reference it - recall the GAC can be found under %windir%\assembly\GAC_MSIL through the command-line); compile the whole thing (preferably with a strong name) and open up a VS2005 command-line (under Windows Vista launched using "Run as Administrator" in order to have the necessary privileges to write to the registry, a task executed by installutil.exe). In the command-prompt, navigate to the bin\Debug or bin\Release folder (according to the build type you chose) and execute installutil.exe -i DemoSnapIn.dll (assuming the assembly is named DemoSnapIn.dll of course):

Next, open Windows PowerShell and check whether the registration was successful by calling get-pssnapin -registered:

You should be able to call both the get-greetings and get-permutations cmdlets once you have added the snap-in to the shell:

 

Hosting a runspace

Time to call the cmdlets from inside another application. To illustrate this, create a new console application in C#, add a reference to System.Management.Automation (you might want to copy the System.Management.Automation.dll file from the GAC to some folder in order to reference it - recall the GAC can be found under %windir%\assembly\GAC_MSIL through the command-line). Next, add the following code:

using System; using System.Management.Automation.Runspaces; using System.Management.Automation; namespace DemoCmdHost { class Program { const string name = "Bart"; const string perm = "Abc"; static void Main(string[] args) { RunspaceConfiguration config = RunspaceConfiguration.Create(); PSSnapInException ex; config.AddPSSnapIn("DemoSnapIn", out ex); if (ex != null) throw ex; using (Runspace rs = RunspaceFactory.CreateRunspace(config)) { rs.Open(); Pipeline p = rs.CreatePipeline(); Command cmd = new Command("get-greeting"); cmd.Parameters.Add("Name", name); p.Commands.Add(cmd); Console.WriteLine(p.Invoke()[0].BaseObject as string); } using (Runspace rs = RunspaceFactory.CreateRunspace(config)) { rs.Open(); Pipeline p = rs.CreatePipeline(); Command cmd = new Command("get-permutations"); cmd.Parameters.Add("String", perm); p.Commands.Add(cmd); foreach (PSObject o in p.Invoke()) Console.WriteLine(o.BaseObject); } } } }

That's it! Here's the result you should get:

More will follow later. Take a look at the Windows SDK documentation for Windows PowerShell too.

 

Calling all Windows Vista users - Download Windows PowerShell now

Over here:

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Did you know Windows PowerShell has out-of-the-box support for COM objects? Many of you will know by now that Windows PowerShell is an object-oriented shell, so it supports objects like .NET classes:

PS C:\temp> $ts = new-object TimeSpan(1,0,0) PS C:\temp> $ts Days : 0 Hours : 1 Minutes : 0 Seconds : 0 Milliseconds : 0 Ticks : 36000000000 TotalDays : 0,0416666666666667 TotalHours : 1 TotalMinutes : 60 TotalSeconds : 3600 TotalMilliseconds : 3600000 PS C:\temp> [DateTime]::Now Thursday 30 November 2006 2:41:49 PS C:\temp> [DateTime]::Now + $ts Thursday 30 November 2006 3:41:53

However, interop with older technologies (in casu COM) will stay a need for the foreseeable future. So, Windows PowerShell can work with this kind of objects too. The classical example is using Internet Explorer in Windows PowerShell:

PS C:\temp> $ie = new-object -Com internetexplorer.application PS C:\temp> $ie | gm -MemberType Method TypeName: System.__ComObject#{d30c1661-cdaf-11d0-8a3e-00c04fc9e26e} Name MemberType Definition ---- ---------- ---------- ClientToWindow Method void ClientToWindow (int, int) ExecWB Method void ExecWB (OLECMDID, OLECMDEXECOPT, Variant, Var... GetProperty Method Variant GetProperty (string) GoBack Method void GoBack () GoForward Method void GoForward () GoHome Method void GoHome () GoSearch Method void GoSearch () Navigate Method void Navigate (string, Variant, Variant, Variant, ... Navigate2 Method void Navigate2 (Variant, Variant, Variant, Variant... PutProperty Method void PutProperty (string, Variant) QueryStatusWB Method OLECMDF QueryStatusWB (OLECMDID) Quit Method void Quit () Refresh Method void Refresh () Refresh2 Method void Refresh2 (Variant) ShowBrowserBar Method void ShowBrowserBar (Variant, Variant, Variant) Stop Method void Stop () PS C:\temp> $ie.Navigate2("http://blogs.bartdesmet.net/bart") PS C:\temp> $ie | gm V* TypeName: System.__ComObject#{d30c1661-cdaf-11d0-8a3e-00c04fc9e26e} Name MemberType Definition ---- ---------- ---------- Visible Property bool Visible () {get} {set} PS C:\temp> $ie.Visible = $true

In the sample above you can see all of the goodness of Windows PowerShell working together with COM, i.e. the ability to create objects, to reflect against the object to find out methods and properties, and to invoke operations. However, a much cooler sample of COM interop is to use the Microsoft Agent, which you might still know from the old Office days with the Office assistants. The cool thing about it is that it's exposed as a COM API, so you can use it everywhere you see fit. So, what about a PowerShell assistant that can speak (i.e. produce audio) to you? Here's the sample step-by-step.

Step 1 - Load and show Merlin

First we'll load and show our assistant: Merlin. To do this, we have to create an instance of the COM type Agent.Control.2 and load the character "Merlin" in it. Here's how:

PS C:\temp> $agent = new-object -com Agent.Control.2 PS C:\temp> $agent.Connected = 1 PS C:\temp> $agent.Characters.Load("Merlin") PS C:\temp> $merlin = $agent.Characters.Character("Merlin") PS C:\temp> $merlin.Show()

Needless to say you can discover this info too using get-member:

PS C:\temp> $agent | gm TypeName: System.__ComObject#{8563ff20-8ecc-11d1-b9b4-00c04fd97575} Name MemberType Definition ---- ---------- ---------- ShowDefaultCharacterProperties Method void ShowDefaultCharacterPropertie... AudioOutput Property IAgentCtlAudioObjectEx AudioOutput... Characters Property IAgentCtlCharacters Characters () ... CommandsWindow Property IAgentCtlCommandsWindow CommandsWi... Connected Property bool Connected () {get} {set} PropertySheet Property IAgentCtlPropertySheet PropertyShe... RaiseRequestErrors Property bool RaiseRequestErrors () {get} {... SpeechInput Property IAgentCtlSpeechInput SpeechInput (... Suspended Property bool Suspended () {get}

 

PS C:\temp> $merlin | gm TypeName: System.__ComObject#{de8ef600-2f82-11d1-acac-00c04fd97575} Name MemberType Definition ---- ---------- ---------- Activate Method bool Activate (Variant) GestureAt Method IAgentCtlRequest GestureAt (short, short) Get Method IAgentCtlRequest Get (string, string, Variant) Hide Method IAgentCtlRequest Hide (Variant) Interrupt Method IAgentCtlRequest Interrupt (IDispatch) Listen Method bool Listen (bool) MoveTo Method IAgentCtlRequest MoveTo (short, short, Variant) Play Method IAgentCtlRequest Play (string) Show Method IAgentCtlRequest Show (Variant) ShowPopupMenu Method bool ShowPopupMenu (short, short) Speak Method IAgentCtlRequest Speak (Variant, Variant) Stop Method void Stop (Variant) StopAll Method void StopAll (Variant) Think Method IAgentCtlRequest Think (string) Wait Method IAgentCtlRequest Wait (IDispatch) Active Property short Active () {get} AnimationNames Property IAgentCtlAnimationNames AnimationNames () {get} AutoPopupMenu Property bool AutoPopupMenu () {get} {set} Balloon Property IAgentCtlBalloonEx Balloon () {get} Commands Property IAgentCtlCommandsEx Commands () {get} Description Property string Description () {get} {set} ExtraData Property string ExtraData () {get} GUID Property string GUID () {get} HasOtherClients Property bool HasOtherClients () {get} Height Property short Height () {get} {set} HelpContextID Property int HelpContextID () {get} {set} HelpFile Property string HelpFile () {get} {set} HelpModeOn Property bool HelpModeOn () {get} {set} IdleOn Property bool IdleOn () {get} {set} LanguageID Property int LanguageID () {get} {set} Left Property short Left () {get} {set} MoveCause Property short MoveCause () {get} Name Property string Name () {get} {set} OriginalHeight Property short OriginalHeight () {get} OriginalWidth Property short OriginalWidth () {get} Pitch Property int Pitch () {get} SoundEffectsOn Property bool SoundEffectsOn () {get} {set} Speed Property int Speed () {get} SRModeID Property string SRModeID () {get} {set} SRStatus Property int SRStatus () {get} Top Property short Top () {get} {set} TTSModeID Property string TTSModeID () {get} {set} Version Property string Version () {get} VisibilityCause Property short VisibilityCause () {get} Visible Property bool Visible () {get} Width Property short Width () {get} {set}

 

Step 2 - Playing animations

One of the things characters can do is play some animation. To get a list of the animations, query $agent.AnimationNames. A sample to find all animations with an o in it is shown below:

PS C:\temp> $merlin.AnimationNames | where { $_ -match "o" } RestPose GestureDown Show Processing Acknowledge DontRecognize StopListening GetAttention GetAttentionReturn Congratulate_2 Announce Congratulate Confused MoveRight MoveLeft MoveUp MoveDown WriteContinued DoMagic1 DoMagic2 LookDown LookDownBlink LookDownReturn LookLeft LookLeftBlink LookLeftReturn LookRight LookRightBlink LookRightReturn LookUp LookUpBlink LookUpReturn ReadContinued GetAttentionContinued Process

To play an animation it's suffient to call $agent.Play(animationname), for example:

PS C:\temp> $merlin.Play("Congratulate")

with the following result:

Step 3 - Think and speak

Time to make Merlin do some work. One thing he can (pretend to) do is think using the Think method taking a string:

PS C:\temp> $merlin.Think("PowerShell is fun!")

Next we'll make Merlin speak. You can better turn sound on a low volume if colleagues are in the near neighborhood (or maybe you just want to track attention, your choice). Let's start with a simple sample:

PS C:\temp> $merlin.Speak("Welcome to Windows PowerShell COM support!")

 

Step 4 - Something more useful

So, Speak takes a string as an argument. Windows PowerShell can provide us with strings on lots of places. Time to bring these two together. For example, do you want a speaking directory listing?

PS C:\temp> dir *.dll | foreach { $merlin.Speak("Start of directory listing") } { $_.Name; $merlin.Speak($_.Name + " " + $_.Length +" bytes."); Start-Sleep 5; } { "End of directory listing" }

Or what about a service state indicator?

PS C:\temp> get-service [c-d]* | foreach { $merlin.Speak($_.Name + " is " + $_.Status + "."); Start-Sleep 5 }

Or printing a set of memory intensive processes?

PS C:\temp> get-process | sort WorkingSet -descending | select -first 5 | foreach { $merlin.Speak($_.ProcessName + " uses " + $_.WorkingSet + " bytes of memory."); Start-Sleep 5 }

And for real geeks, what about a C# program listing?

PS C:\temp> type downloadfilecmdlet.cs | foreach { $merlin.Speak($_); Start-Sleep 5 }

 

Step 5 - Suppress Merlin's output

In all the screenshots above you see some unwanted output. In order to suppress this, here's a little trick: append out-null, like this:

PS C:\temp> dir *.dll | foreach { $merlin.Speak("Start of directory listing") | out-null } { $_; $merlin.Speak($_.Name + " " + $_.Length +" bytes.") | out-null; Start-Sleep 5; } { "End of directory listing" } PS C:\temp> get-service | foreach { $_; $merlin.Speak($_.Name + " is " + $_.Status + ".") | out-null; Start-Sleep 5 } PS C:\temp> get-process | foreach { $_; $merlin.Speak($_.ProcessName + " uses " + $_.WorkingSet + " bytes of memory.") | out-null; Start-Sleep 5 } PS C:\temp> type downloadfilecmdlet.cs | foreach { $_; $merlin.Speak($_) | out-null; Start-Sleep 5 }

Using $_ you can print the objects themselves to retain typical output, as shown below (for the first line):

 

Step 6 - Say goodbye to Merlin

To hide the Merlin character, just ask him kindly to disappear:

PS C:\temp> $merlin.Hide()

Enjoy COM interop in Windows PowerShell!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

Today (11/29/06) I received a mail from one of my blog readers:

Hi Bart,

in the following Block you mention an powershell commandlet for creating an SHA hash:

http://community.bartdesmet.net/blogs/bart/archive/2006/06/23/4106.aspx

Could you please publish or mail me some code snipes? My own code wont work proper.

THX for sharing so much knowledge about .NET.

Regards,

Thomas

Apparently I promised so time in the past to upload a cmdlet for file hashing but it never made it to my blog. So here it is today.

 

A file hasher cmdlet

Let's create a cmdlet for file hashing, called get-hash. It should take two parameters: one with the algorithm desired (SHA1, MD5, SHA256, SHA384, SHA512) and one with the file. The latter one can either be passed from the command line (e.g. dir *.cs | get-hash sha1 should work fine) or using some aliases specifying the name of the file as a string. Taking all these requirements together, we end up with the following:

1 using System; 2 using System.ComponentModel; 3 using System.IO; 4 using System.Management.Automation; 5 using System.Security.Cryptography; 6 using System.Text; 7 8 [Cmdlet("get", "hash")] 9 public class HashCmdlet : PSCmdlet 10 { 11 private string algorithm; 12 13 [Parameter(Position = 0, Mandatory = true)] 14 public string Algorithm 15 { 16 get { return algorithm; } 17 set { algorithm = value; } 18 } 19 20 private string file; 21 22 [Alias("File", "Name")] 23 [Parameter(Position = 1, Mandatory = true, ValueFromPipelineByPropertyName = true)] 24 public string FullName 25 { 26 get { return file; } 27 set { file = value; } 28 } 29 30 protected override void ProcessRecord() 31 { 32 HashAlgorithm algo = HashAlgorithm.Create(algorithm); 33 if (algo != null) 34 { 35 StringBuilder sb = new StringBuilder(); 36 using (FileStream fs = new FileStream(file, FileMode.Open)) 37 foreach(byte b in algo.ComputeHash(fs)) 38 sb.Append(b.ToString("x2")); 39 WriteObject(sb.ToString()); 40 } 41 else 42 { 43 string s = String.Format("Algorithm {0} not found.", algorithm); 44 ErrorRecord err = new ErrorRecord(new ArgumentException(s), s, ErrorCategory.InvalidArgument, null); 45 WriteError(err); 46 } 47 } 48 } 49 50 [RunInstaller(true)] 51 public class HashSnapin : PSSnapIn 52 { 53 public override string Name { get { return "FileHasher"; } } 54 public override string Vendor { get { return "Bart"; } } 55 public override string Description { get { return "Computes file hashes."; } } 56 }

A few remarks:

  • The second parameter, FullName, can be taken from the pipeline (ValueFromPipelineByPropertyName set to true). The reason for the chosen name "FullName" is the fact that a System.IO.FileInfo object has such a property name, and we want that property name to match our property name.
  • Using aliases, the second parameter can be supplied using File or Name too.
  • Line 39 write the string representation of the hash to the pipeline using WriteObject. Alternatively, one might also output the byte array retrieved from ComputeHash (line 37) but that would be less user-friendly in my opinion.
  • In case an unknown algorithm is specified, we construct an ErrorRecord object that's passed on to the shell using WriteError.
  • Stuff on lines 50 to 56 creates a simple snap-in that's used to make the cmdlet available.

 

Compilation and installation instructions

Download the code and execute the following steps on a VS2005 command line:

  • Make sure the PATH environment variable has csc.exe and installutil.exe on it (set PATH=%PATH%;%windir%\Microsoft.NET\Framework\v2.0.50727)
  • Save the file as hashcmdlet.cs
  • Copy System.Management.Automation from the GAC (%windir%\assembly\GAC_MSIL\System.Management.Automation\<version_pktoken>\System.Management.Automation.dll) to the current folder with the hashcmdlet.cs code file
  • Execute csc /t:library /r:System.Management.Automation.dll hashcmdlet.cs
  • Install the snap-in using installutil -i hashcmdlet.dll

 

Demo

Below you can see a sample of our get-hash cmdlet:

1 PS C:\temp> add-pssnapin filehasher 2 3 PS C:\temp> type extfi.ps1xml 4 <Types> 5 <Type> 6 <Name>System.IO.FileInfo</Name> 7 <Members> 8 <ScriptProperty> 9 <Name>MD5</Name> 10 <GetScriptBlock> 11 get-hash md5 $this 12 </GetScriptBlock> 13 </ScriptProperty> 14 <ScriptProperty> 15 <Name>SHA1</Name> 16 <GetScriptBlock> 17 get-hash sha1 $this 18 </GetScriptBlock> 19 </ScriptProperty> 20 </Members> 21 </Type> 22 </Types> 23 24 PS C:\temp> update-typedata extfi.ps1xml 25 26 PS C:\temp> dir *.cs | gm -type P* 27 28 TypeName: System.IO.FileInfo 29 30 Name MemberType Definition 31 ---- ---------- ---------- 32 PSChildName NoteProperty System.String PSChildName=bar.cs 33 PSDrive NoteProperty System.Management.Automation.PSDriveInfo PS... 34 PSIsContainer NoteProperty System.Boolean PSIsContainer=False 35 PSParentPath NoteProperty System.String PSParentPath=Microsoft.PowerS... 36 PSPath NoteProperty System.String PSPath=Microsoft.PowerShell.C... 37 PSProvider NoteProperty System.Management.Automation.ProviderInfo P... 38 Attributes Property System.IO.FileAttributes Attributes {get;set;} 39 CreationTime Property System.DateTime CreationTime {get;set;} 40 CreationTimeUtc Property System.DateTime CreationTimeUtc {get;set;} 41 Directory Property System.IO.DirectoryInfo Directory {get;} 42 DirectoryName Property System.String DirectoryName {get;} 43 Exists Property System.Boolean Exists {get;} 44 Extension Property System.String Extension {get;} 45 FullName Property System.String FullName {get;} 46 IsReadOnly Property System.Boolean IsReadOnly {get;set;} 47 LastAccessTime Property System.DateTime LastAccessTime {get;set;} 48 LastAccessTimeUtc Property System.DateTime LastAccessTimeUtc {get;set;} 49 LastWriteTime Property System.DateTime LastWriteTime {get;set;} 50 LastWriteTimeUtc Property System.DateTime LastWriteTimeUtc {get;set;} 51 Length Property System.Int64 Length {get;} 52 Name Property System.String Name {get;} 53 MD5 ScriptProperty System.Object MD5 {get=get-hash md5 $this;} 54 Mode ScriptProperty System.Object Mode {get=$catr = "";... 55 SHA1 ScriptProperty System.Object SHA1 {get=get-hash sha1 $this;} 56 57 PS C:\temp> dir *.cs | format-table Name,MD5,SHA1 58 59 Name MD5 SHA1 60 ---- --- ---- 61 bar.cs d541e9719077844ba1fa136... 8662e86f3302578a59da5e... 62 downloadfilecmdlet.cs 0c74a0c905f3b1cd6e22d52... ab3c4dcee4f9e3c48daded... 63 hashcmdlet.cs 41b01139d6168df3f3cec13... dd478c60f77b19b64fa0d7... 64 test.cs 477405d2be4a8f327d39a01... c632fe67a71baa0f333675... 65 66 PS C:\temp> dir *.cs | format-list Name,MD5,SHA1 67 68 Name : bar.cs 69 MD5 : d541e9719077844ba1fa13626f5122cb 70 SHA1 : 8662e86f3302578a59da5e9c936b69ab0d4ff9aa 71 72 Name : downloadfilecmdlet.cs 73 MD5 : 0c74a0c905f3b1cd6e22d52831b92b31 74 SHA1 : ab3c4dcee4f9e3c48daded97f01ee01e8c572a2a 75 76 Name : hashcmdlet.cs 77 MD5 : 41b01139d6168df3f3cec13b9663e633 78 SHA1 : dd478c60f77b19b64fa0d7c62944ec1b948419c9 79 80 Name : test.cs 81 MD5 : 477405d2be4a8f327d39a015db255fdf 82 SHA1 : c632fe67a71baa0f333675f5cdc16fc547772c33 83 84 PS C:\temp> get-hash MD5 bar.cs 85 d541e9719077844ba1fa13626f5122cb 86 87 PS C:\temp> get-hash SHA1 bar.cs 88 8662e86f3302578a59da5e9c936b69ab0d4ff9aa 89 90 PS C:\temp> get-hash SHA256 bar.cs 91 a1e6764cf77d02804e909427aff62ade6b9894924a69284f3d83fd0d2904548b 92 93 PS C:\temp> get-hash SHA384 bar.cs 94 875bde9e789f88e76aa9fe18f82adc8a8beb920cdf1d50692a0b9473ecc296a750e888844a184d7 95 6e610d434a3bec3a5 96 97 PS C:\temp> get-hash SHA512 bar.cs 98 f88b40bd4618dcb99e17af14c7f2368b00ea55a9b7d6d71e73d519e197ca3d6c3847fd46834fb1e 99 c4acb9c45729441ac76611de2f7f86b032b59e1a3b7384a3a 100 101 PS C:\temp> get-hash bla bar.cs 102 get-hash : Algorithm bla not found. 103 At line:1 char:9 104 + get-hash <<<< bla bar.cs

This sample is available in the download as well. Let's explain it in a bit more detail:

  • One line 1, the snap-in is loaded (which should be installed using installutil -i hashcmdlet.dll in the previous paragraph) using add-pssnapin.
  • Next, we leverage the power of the ETS (Extended Type System) to add two script properties MD5 and SHA1 to the FileInfo object. To do this, you should write an xml file which contains the line 4 to 22 and save it as a .ps1xml file. This file is loaded using update-typedata in line 24. This file is available in the download as well.
  • Now notice that FileInfo is extended with the MD5 and SHA1 ScriptProperty members using a get-member invocation on the output for get-childitem *.cs (assuming you have .cs files in your c:\temp folder). Lines 53 and 55 contain the newly added properties.
  • To use these script properties, observe the commands invoked in lines 57 and 66. These use format-table and format-list to visualize the additional properties.
  • Of course you can invoke get-hash directly too, as shown on lines 84, 87, 90, 93, 97 and 101. All supported algorithms are illustrated.

One drawback of our cmdlet is that it can't report progress when a hash operation takes a bit of time, especially for larger files. So, use it with care and rely on an explicit call to get-hash when you need to calculate a hash.

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Only a couple of days ago I posted about creating a file downloader cmdlet in Windows PowerShell which contained the following little sentence:

One could make this method more complex in order to provide a seconds remaining estimate based on the download speed observed.

One of my readers (dotnetjunkie) was so kind to leave a piece of feedback:

I think you should add transfer speed and estimated remaining time, that would make it even more useful and cooler ! ;)

Well, I couldn't agree more. So I revisited my piece of code I wrote some weeks ago (12th of November to be precise) and added download speed tracking and a seconds remaining indicator.

There are undoubtly different ways to implement such an estimate. My approach is to measure the number of bytes transferred during intervals of approximately 5 seconds (kind of "instant download speed") and to derive the estimated time remaining from this. I'll discuss the code changes in a few steps.

Step 1 - Add a few private members

We need 4 additional members to produce statistics:

  • The first one is a Stopwatch to measure elapsed time. We'll poll this counter every time the DownloadProgressChanged event is fired. In case it's above 5 seconds, we update the statistics and restart the counter.
  • Secondly, we'll cache an indicator that keeps the current transfer speed as a string of the format n [bytes|KB|MB|GB|...]/sec. This value will be updated every 5 seconds.
  • Next, the seconds remaining are kept. It's updated every 5 seconds and during these intervals it just counts down every one second.
  • Finally, a transferred bytes indicator is used for calculation of the bytes transferred the last 5 seconds.

Here's the piece of code:

/// <summary> /// Stopwatch used to measure download speed. /// </summary> private Stopwatch sw = new Stopwatch(); /// <summary> /// Bytes per second indicator (bytes/sec, KB/sec, MB/sec, ...). /// </summary> private string bps = null; /// <summary> /// Seconds remaining indicator. /// </summary> private int secondsRemaining = -1; /// <summary> /// Number of bytes already transferred. /// </summary> private long transferred = 0;

Step 2 - Let the count begin

In the ProcessRecord method we start our Stopwatch; just that:

// // Check validity for download. Will throw an exception in case of transport protocol errors. // using (clnt.OpenRead(_url)) { } // // Start download speed stopwatch. // sw.Start(); // // Download the file asynchronously. Reporting will happen through events on background threads. // clnt.DownloadFileAsync(_url, _file);

Step 3 - Calculate stats and report progress

Time for the real stuff. On to the DownloadProgressChanged event handler. When we observe that the Stopwatch has an elapsed time of 5 or more seconds, we'll stop it, update stats and restart it. The code is shown below:

1 /// <summary> 2 /// Reports download progress. 3 /// </summary> 4 private void webClient_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e) 5 { 6 // 7 // Update statistics every 5 seconds (approx). 8 // 9 if (sw.Elapsed >= TimeSpan.FromSeconds(5)) 10 { 11 sw.Stop(); 12 13 // 14 // Calculcate transfer speed. 15 // 16 long bytes = e.BytesReceived - transferred; 17 double bps = bytes * 1000.0 / sw.Elapsed.TotalMilliseconds; 18 this.bps = BpsToString(bps); 19 20 // 21 // Estimated seconds remaining based on the current transfer speed. 22 // 23 secondsRemaining = (int)((e.TotalBytesToReceive - e.BytesReceived) / bps); 24 25 // 26 // Restart stopwatch for next 5 seconds. 27 // 28 transferred = e.BytesReceived; 29 sw.Reset(); 30 sw.Start(); 31 } 32 33 // 34 // Construct a ProgressRecord with download state information but no completion time estimate (SecondsRemaining < 0). 35 // 36 ProgressRecord pr = new ProgressRecord(0, String.Format("Downloading {0}", _url.ToString(), _file), String.Format("{0} of {1} bytes transferred{2}.", e.BytesReceived, e.TotalBytesToReceive, this.bps != null ? String.Format(" (@ {0})", this.bps) : "")); 37 pr.CurrentOperation = String.Format("Destination file: {0}", _file); 38 pr.SecondsRemaining = secondsRemaining - (int)sw.Elapsed.Seconds; 39 pr.PercentComplete = e.ProgressPercentage; 40 41 // 42 // Report availability of a ProgressRecord item. Will cause the while-loop's body in ProgressRecord to execute. 43 // 44 lock (pr_sync) 45 { 46 this.pr = pr; 47 prog.Set(); 48 } 49 }

So, what's going on here. Basically we want to provide a seconds remaining estimate on line 38 and a download speed estimate on line 36. This should be pretty self-explanatory. The real work happens in lines 11 to 30 where the number of bytes transferred in the last 5 seconds are obtained and divided by the expired milliseconds during the last 5 seconds (which should be around 5000 obviously). The rest is maths, except for the BpsToString call as shown below.

Step 4 - A download speed indicator

BpsToString is the method to convert the bytes per second rate to a friendly string representation:

/// <summary> /// Constructs a download speed indicator string. /// </summary> /// <param name="bps">Bytes per second transfer rate.</param> /// <returns>String represenation of the transfer rate in bytes/sec, KB/sec, MB/sec, etc.</returns> private string BpsToString(double bps) { string[] m = new string[] { "bytes", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB" }; //dreaming of YB/sec int i = 0; while (bps >= 0.9 * 1024) { bps /= 1024; i++; } return String.Format("{0:0.00} {1}/sec", bps, m[i]); }

I think the code fragment above is pretty optimistic for what transfer speeds is concerned, but with the expected life time of PowerShell in mind this should be no luxury :-).

Step 5 - The result & code download

This is the result (needless to say the figures are indicative only, it are estimates after all):

And here's the code download link.

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

A few days ago, I posted this little quiz on batch scripting mysteries. The batch script given was:

@echo off for %%f in (a,b,c) do ( echo 1 %%f 2 set x=x1 %%f x2 echo %x% )

Here's the output:

C:\temp>test
1 a 2
ECHO is off.
1 b 2
ECHO is off.
1 c 2
ECHO is off.

C:\temp>test
1 a 2
x1 c x2
1 b 2
x1 c x2
1 c 2
x1 c x2

Not what you might have expected :-). I did some research but HELP SET didn't give the complete answer, so a workaround seems a good solution:

@echo off for %%f in (a,b,c) do ( echo 1 %%f 2 call :print %%f ) goto eof :print set x=x1 %1 x2 echo %x% :eof

And of course Windows PowerShell helps us out too:

PS C:\Users\Bart> ('a','b','c') | foreach { echo "1 $_ 2"; $x = "x1 $_ x2"; echo $x } 1 a 2 x1 a x2 1 b 2 x1 b x2 1 c 2 x1 c x2

Yet another reason to choose Windows PowerShell? (Notice the variable expansion in strings too.)

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Introduction

In my previous post on file downloads in PowerShell I mentioned how to download a file in Windows PowerShell using System.Net.WebClient. No big deal if you know the Base Class Libraries. However, there was a drawback: the lacking download status reporting. In order to fix this, I've created a simple sample cmdlet that uses reporting (WriteProgress) while performing a download. Enter download-file.

The code

The basic idea is pretty simple:

  • Inside a cmdlet we'll call some WebClient method to perform a file download from a specified URL (through some parameter) and to a local file location (which can be an optional parameter, since we can derive the file name from the source URL as well)
  • Instead of blocking on some DownloadFile call, we want to do things asynchronously (i.e. DownloadFileAsync) in order to be able to report progress during the download.
  • The events DownloadProgressChanged and DownloadFileCompleted of the System.Net.WebClient will help us to accomplish the progress reporting.

We'll call our cmdlet download-file, so the declaration will be like this:

[Cmdlet("download", "file")] public class DownloadFileCmdlet : PSCmdlet { }

Parameterization of a cmdlet is piece of cake as explained in a bunch of previous posts already, and will look like this:

/// <summary> /// Url of the file that has to be downloaded. /// </summary> [Parameter(Mandatory = true, Position = 0)] public Uri Url { get { return _url; } set { _url = value; } } private string _file; /// <summary> /// Target file name (optional). /// </summary> [Parameter(Position = 1)] public string To { get { return _file; } set { _file = value; } }

Next, we'll have to define the ProcessRecord overload of the cmdlet to do the processing. This is where things get tricky because of the following reasons:

  • Starting an asynchronous download using DownloadFileAsync doesn't report HTTP error codes directly. Since we want to catch these possible errors, we'll have to do some trick.
  • Events like DownloadProcessChanged and DownloadFileCompleted are raised on a background thread. From such a thread it's not valid to call the PSCmdlet's WriteProgress (or any other Write*) method.
  • We have to wait for the background download to complete before ProcessRecord is exited. In other words, DownloadFileCompleted has to run before we can exit ProcessRecord.

The solution to all of the problems above is some nice piece of thread synchronization. Basically the "main thread" (the one where ProcessRecord was called on) has to create the WebClient, hook in event handlers and start the asynchronous download job. Once that's done, it has to wait for any of two events to occur: either a ProgressRecord instance is available or DownloadFileCompleted has executed. In the first case, we can perform a WriteProgress to report progress on the right thread. In the second case, we can exit the ProcessRecord method because of download completion.

Here's the complete code:

1 using System; 2 using System.ComponentModel; 3 using System.IO; 4 using System.Management.Automation; 5 using System.Net; 6 using System.Threading; 7 8 [Cmdlet("download", "file")] 9 public class DownloadFileCmdlet : PSCmdlet 10 { 11 /// <summary> 12 /// Wait handle to report download completion. 13 /// </summary> 14 private ManualResetEvent exit = new ManualResetEvent(false); 15 16 /// <summary> 17 /// Wait handle to report availability of a ProgressRecord item in pr. 18 /// </summary> 19 private AutoResetEvent prog = new AutoResetEvent(false); 20 21 /// <summary> 22 /// Array of the wait handles above (set in ProcessRecord) to perform WaitAny. 23 /// </summary> 24 private WaitHandle[] evts; 25 26 /// <summary> 27 /// ProgressRecord indicating the current download status. 28 /// </summary> 29 private ProgressRecord pr; 30 31 /// <summary> 32 /// Synchronization object for pr. 33 /// </summary> 34 private object pr_sync = new object(); 35 36 private Uri _url; 37 38 /// <summary> 39 /// Url of the file that has to be downloaded. 40 /// </summary> 41 [Parameter(Mandatory = true, Position = 0)] 42 public Uri Url 43 { 44 get { return _url; } 45 set { _url = value; } 46 } 47 48 private string _file; 49 50 /// <summary> 51 /// Target file name (optional). 52 /// </summary> 53 [Parameter(Position = 1)] 54 public string To 55 { 56 get { return _file; } 57 set { _file = value; } 58 } 59 60 /// <summary> 61 /// Entry-point for the cmdlet processing. 62 /// </summary> 63 protected override void ProcessRecord() 64 { 65 // 66 // Construct wait handles array for WaitHandle.WaitAny calls. 67 // 68 evts = new WaitHandle[] { exit, prog }; 69 70 // 71 // If no target file name was specified, derive it from the url's file name portion. 72 // 73 if (_file == null) 74 { 75 string[] fs = _url.LocalPath.Split('/'); 76 if (fs.Length > 0) 77 _file = fs[fs.Length - 1]; 78 } 79 80 // 81 // Construct web client object and hook in event handlers to report progress and completion. 82 // 83 WebClient clnt = new WebClient(); 84 clnt.DownloadProgressChanged += new DownloadProgressChangedEventHandler(webClient_DownloadProgressChanged); 85 clnt.DownloadFileCompleted += new AsyncCompletedEventHandler(webClient_DownloadFileCompleted); 86 87 try 88 { 89 // 90 // Check validity for download. Will throw an exception in case of transport protocol errors. 91 // 92 using (clnt.OpenRead(_url)) { } 93 94 // 95 // Download the file asynchronously. Reporting will happen through events on background threads. 96 // 97 clnt.DownloadFileAsync(_url, _file); 98 99 // 100 // Wait for any of the events (exit, prog) to occur. 101 // In case of index 0 (= exit), stop processing. 102 // In case of index 1 (= prog), report progress. 103 // 104 while (WaitHandle.WaitAny(evts) != 0) //0 is exit event 105 { 106 lock (pr_sync) 107 { 108 WriteProgress(pr); 109 } 110 } 111 112 // 113 // Write file info object for the target file. Can be used for further processing on the pipeline. 114 // 115 WriteObject(new FileInfo(_file)); 116 } 117 catch (WebException ex) 118 { 119 // 120 // Report an error. Could be more specific for what the ErrorCategory is concerned, by mapping HTTP error codes. 121 // 122 WriteError(new ErrorRecord(ex, ex.Status.ToString(), ErrorCategory.NotSpecified, clnt)); 123 } 124 } 125 126 /// <summary> 127 /// Reports download progress. 128 /// </summary> 129 private void webClient_DownloadProgressChanged(object sender, DownloadProgressChangedEventArgs e) 130 { 131 // 132 // Construct a ProgressRecord with download state information but no completion time estimate (SecondsRemaining < 0). 133 // 134 ProgressRecord pr = new ProgressRecord(0, String.Format("Downloading {0}", _url.ToString(), _file), String.Format("{0} of {1} bytes transferred.", e.BytesReceived, e.TotalBytesToReceive)); 135 pr.CurrentOperation = String.Format("Destination file: {0}", _file); 136 pr.SecondsRemaining = -1; 137 pr.PercentComplete = e.ProgressPercentage; 138 139 // 140 // Report availability of a ProgressRecord item. Will cause the while-loop's body in ProgressRecord to execute. 141 // 142 lock (pr_sync) 143 { 144 this.pr = pr; 145 prog.Set(); 146 } 147 } 148 149 /// <summary> 150 /// Reports download completion. 151 /// </summary> 152 private void webClient_DownloadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e) 153 { 154 // 155 // Signal the exit state. Will cause the while-loop in ProcessRecord to terminate. 156 // 157 exit.Set(); 158 } 159 } 160 161 [RunInstaller(true)] 162 public class DownloadFileSnapIn : PSSnapIn 163 { 164 public override string Name { get { return "DownloadFile"; } } 165 public override string Vendor { get { return "Bart De Smet"; } } 166 public override string Description { get { return "Allows file download."; } } 167 }

Starting at the bottom of the cmdlet definition we can see the two event handlers, webClient_DownloadProgressChanged and webClient_DownloadFileCompleted. The first one creates a ProgressRecord (lines 134-137) which is the way for a cmdlet to communicate status to the PowerShell host application. The parameters and properties are self-explanatory. One could make this method more complex in order to provide a seconds remaining estimate based on the download speed observed. In this basic sample we're happy with some status messages and a percentage. In order to report progress, some thread synchronization stuff is needed. Remember the WriteProgress method can only be called on the ProcessRecord's thread. So, we copy (line 144) the constructed ProgressRecord to the pr member of the class (line 29) which is synchronized by pr_sync (line 34, used in line 142). Finally the availability of the record is signaled using the wait handle prog in line 145. Notice it's an AutoResetEvent (line 19), which means that it gets reset automatically (to false) once the consuming thread (ProcessRecord) has sucked it (in WaitAny in our case, line 104). The webClient_DownloadFileCompleted event handler is straightforward an just signals the exit "download completion" state on line 157 that will be caught on line 104's WaitAny.

Real work happens in ProcessRecord of course. First (line 68) an array of wait handles gets constructed to be used in the WaitAny call (line 104). WaitAny means to wait till either one of these handles has been set. For example: if a progress record is available, prog will be set (by webClient_DownloadProgressChanges on line 145) and WaitAny will return 1 because prog is on index 1 in the evts handles array. In a similar way, WaitAny will return 0 if the 0'th element of evts, i.e. exit, has been set (by webClient_DownloadFileCompleted on line 157). Next, the case of no specified target file is taken care of in lines 73 to 78, taking the source file name as the target name. Now (lines 83-85) the WebClient instance is created and the event handlers are hooked in. Finally, the download progress can be started. In order to cause an exception - for example in case of a 404 error code - before download begins, we call OpenRead on line 92. The asynchronous download job is started on line 97.

The summum of our code is in lines 104-110, where ProgressRecord instances are consumed every time the prog wait handle is set. These are reported to the PowerShell host by means of a WriteProgress call (line 108), taking care of the required locking. Finally, in line 115, a FileInfo object for the target file is written on the pipeline which might be useful for further processing. WebExceptions are caught on line 117 and reported via WriteError on line 122.

The snap-in for the cmdlet goes without further explanation, see lines 161 to 167.

 

Compilation

Compiling this goes as follows:

  • Make sure the PATH environment variable has csc.exe and installutil.exe on it (set PATH=%PATH%;%windir%\Microsoft.NET\Framework\v2.0.50727)
  • Save the file as downloadfilecmdlet.cs
  • Copy System.Management.Automation from the GAC (%windir%\assembly\GAC_MSIL\System.Management.Automation\<version_pktoken>\System.Management.Automation.dll) to the current folder with the downloadfilecmdlet.cs code file
  • Execute csc /t:library /r:System.Management.Automation.dll downloadfilecmdlet.cs
  • Install the snap-in using installutil -i downloadfilecmdlet.dll

 

Demo

See the pictures below for the cmdlet in action (H):

Download the code over here.

Have fun!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Time for a little PowerShell tip for my IT Pro friends. I know it's just BCL (Base Class Library) stuff, but nevertheless you might find it useful: what about downloading a file from an HTTP server during script execution? Here it is:

PS C:\temp> $clnt = new-object System.Net.WebClient
PS C:\temp> $clnt | gm d*


   TypeName: System.Net.WebClient

Name                MemberType Definition
----                ---------- ----------
Dispose             Method     System.Void Dispose()
DownloadData        Method     System.Byte[] DownloadData(String address), S...
DownloadDataAsync   Method     System.Void DownloadDataAsync(Uri address), S...
DownloadFile        Method     System.Void DownloadFile(String address, Stri...
DownloadFileAsync   Method     System.Void DownloadFileAsync(Uri address, St...
DownloadString      Method     System.String DownloadString(String address),...
DownloadStringAsync Method     System.Void DownloadStringAsync(Uri address),...


PS C:\temp> $url = "http://www.bartdesmet.net/download/ps.txt"
PS C:\temp> $file = "c:\temp\ps.txt"
PS C:\temp> $clnt.DownloadFile($url,$file)
PS C:\temp> type $file
Welcome to Windows PowerShell 1.0! 

Drawback to this approach: no download reporting (as with write-progress) while downloading a large file. Solution: coming up later. Enjoy!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

More Posts « Previous page - Next page »