Tuesday, March 25, 2008 11:09 PM bart

Windows PowerShell 2.0 Feature Focus - Background Jobs

Two weeks ago I did a little tour through Europe spreading the word on a couple of our technologies including Windows PowerShell 2.0. In this blog series I'll dive into a few features of Windows PowerShell 2.0. Keep in mind though it's still very early and things might change towards RTW - all samples presented in this series are based on the CTP which is available over here.

 

Introduction

After a couple of language-related features, we move into infrastructure in this post. Enter the world of remoting in Windows PowerShell 2.0, at least partially. So what's up? The Universal Code Execution Model, to which the Remoting features belong, is one of the core enhancements of Windows PowerShell 2.0 allowing script to be run either locally or remotely in different modes:

  • 1-to-many "fan-out" - execute script on a (large) number of machines (e.g. web farms)
  • Many-to-1 "fan-in" - delegating administration of a server by hosting PowerShell inside the service
  • 1-on-1 "interactive" - managing a server remotely much like "secure telnet"

In addition to this, other features are part of the Universal Code Execution Model:

  • Restricted Runspaces - the idea of having a runspace that's restricted in what it can do (e.g. concerning the operations and the language used)
  • Mobile Object Model - the plumbing that makes it possible to have objects travel around the network (i.e. serialization and deserialization infrastructure)
  • Eventing - adding the concept of events to PowerShell manageability, allowing actions to be taken when certain events occur
  • Background jobs - running commands (cmdlets, scripts) in the background asynchronously

For the remote execution (remote meaning cross runspace boundaries), Windows PowerShell uses the WS-MGMT protocol which is enabled by the WINRM service:

image

The nice thing about using web services is its firewall friendliness. However, in order to enable PowerShell to work with it, one needs to run a script first: $pshome\Configure-WSMAN.ps1. It opens the required ports, checks the service is installed and executes a set of winrm configuration commands that enable endpoints.

 

Background jobs

We'll stick with background jobs for now. There are 6 cmdlets to manage background jobs, known by the PSJob noun in PowerShell 2.0 speak:

image

What's better to start with than Start-PSJob? Here's the syntax:

PS C:\temp> start-psjob -?

NAME
    Start-PSJob

SYNOPSIS
    Creates and starts a Windows PowerShell background job (PsJob) on a local or remote computer.

SYNTAX
    Start-PSJob [-Command] <String> [[-ComputerName] <String[>] [-Credential <PSCredential>] [-Port <Int32>] [-UseSSL] [-ShellName <String>] [-ThrottleLimit <Int32>] [-InputObject <PSObject>] [-Name <String>] [<CommonParameters>]

    Start-PSJob [-Command] <String> [[-Runspace] <RemoteRunspaceInfo[>] [-ThrottleLimit <Int32>] [-InputObject <PSObject>] [-Name <String>] [<CommonParameters>]

Notice the synopsis: on a local or remote computer. This is where remoting enters the picture, with the concept of a remote runspace. We won't go there though, let's stick with local execution and start a command:

start-psjob "start-sleep 30"

This will show the following:

image

Normally, "start-sleep 30" would block the interactive console for 30 seconds (feel free to try). However, now we have sent off the command to the background, in a session with Id 1. The way this works roughly is by having runspaces and communication channels between them to send commands and receive data. The fact data is available is indicated by the HasMoreData property on the job. Without going in too much details, running commands remotely follows the same idea and results can be streamed back from the server to the client so that you can retrieve results piece-by-piece.

Back to our sample now. Of course we can stop a background job by using stop-psjob:

image

What the sample above shows as well is waiting for a PSJob to complete when you need to get the results at that particular point in time, e.g. after having done some more foreground work. Notice the wait-psjob above is blocked for the remainder of the 30 seconds while the background job is completing. Ultimately it returns like this:

image

 

Where's my data, dude?

(Added a comma to disambiguate with some other Microsoft product:-)). Having background jobs is one thing, but getting results back is another thing. For Parallel Extensions folks, it like drawing the line between a Task and a Future<T>. So dude, where's my cup of T? The answer lies in the difference between wait-psjob and receive-psjob. While wait-psjob simply waits for a job to finish, receive-psjob receives data from it. What's really happening is that the foreground runspace talks to the background session to get data back which travels across boundaries (whether or not that boundary is on the local machine or across the network), cf. the Mobile Object Model.

image

An interesting thing to look at is the get-member output for these objects, more specifically the NoteProperty members on it:

image

This is where you see the objects are really deserialized across boundaries, which is one of the tasks of the Mobile Object Model. For example, the PSIPHostEntry shows the origin of the object which is particularly useful when working with remoting. In this context notice that a background job can spawn other background jobs by itself, meaning objects might be aggregated from various sources before they travel your way.

Another thing to realize is that data is streaming in. Assume you're asking for a bunch of results to come in from a remote server. These results are typically emitted by the pipeline object-by-object (unless there's a cmdlet that returns an array of objects or so, which can look to the pipeline - depending on how the result is returned - as one big object) so it makes sense to get the current results, wait for new ones to be produced and get subsequent results. Essentially the pseudo-algorithm is:

while ($job.HasMoreData)
{
    receive-psjob $job
    # do some other stuff
}

Here's a concrete sample:

image

The first time I called receive-psjob only the "get-process | select -f 5" pipeline would have yielded results, so I receive that data while the HasMoreData flag is still set to true. About 30 seconds later, I call receive-psjob $bar again. By then the results of "get-service | select -f 5" have come in too, and HasMoreData indicates there's nothing more to come (the State indicates the background job has completed).

Enjoy your dream-PSJob!

Del.icio.us | Digg It | Technorati | Blinklist | Furl | reddit | DotNetKicks

Filed under:

Comments

# re: Windows PowerShell 2.0 Feature Focus - Background Jobs

Wednesday, March 26, 2008 12:27 AM by Joannes Vermorel

Those are really excellent features. I was looking for a 'nohup' equivalent ever since the release of PSH V1.

# PowerShell 2.0 &#8211; Background Jobs &laquo; Kevin Pelgrims

Tuesday, April 27, 2010 10:19 AM by PowerShell 2.0 – Background Jobs « Kevin Pelgrims

Pingback from  PowerShell 2.0 &#8211; Background Jobs &laquo; Kevin Pelgrims

# Powershell equivalent of bash ampersand (&amp;) for forking/running background processes - Programmers Goodies

Pingback from  Powershell equivalent of bash ampersand (&amp;) for forking/running background processes - Programmers Goodies