Vista Tutorial - FEATURE REQUEST: control over whether child processes block

In unix shells, child processes always block unless you tell them not to.

Eg,

$ xcalc # this blocks and doesn't return until xcalc is done
$ xcalc & # this does not block
[1]+ xcalc # it's running in the background and here's the job number

In PoSH, there seems to be no way to control this. In general, it seems that
programs block if they are console programs, but not if they are GUI
programs. However, sometimes one would want a GUI program to block, or a
console program to not block.

I realize it would complicate your "output pipeline", and your single $error
array.

But I would be perfectly content if you could not "background" a process
unless the end of the pipe was something other than the console. I would
also be content if backgrounded processes had no access to $error.

But I don't like having to start up a new instance of PoSH (which takes 9
seconds on my 3.06ghz machine -- another gripe I have), navigate to the same
directory, run the command I want, and exit, because the command would block
my main PoSH window.

I'd much rather be able to do:

build >outfile & # run my build process in the background, write the output
to 'outfile'

(I know you're using & for something else already, but you get the idea.)
Then when the build is done, I'll get a notification.

Even more importantly, I'd like to be able to force PoSH to wait for a GUI
processes to finish.

For instance:

$temp = createTempFile()
cat "some initial content" >$temp
openEditor $temp # let the user edit it
# the user is done, so something with $temp
rm $temp

I know I could probably simulate some of these things by bypassing PoSH and
spawning new processes myself, but this is the kind of thing that should be
handled in the shell.

> new instance of PoSH (which takes 9
> seconds on my 3.06ghz machine -- another gripe I have)

As for this particular performance issue, please, see thread "PS is not
NGen-ed on installation. Why?" or just run once the following code:

cd $pshome
ls *.dll | % {&
"$env:SystemRoot\Microsoft.NET\Framework\v2.0.50727\ngen.exe" install
$_.fullname}

--
Thanks,
Roman

A cmdlet that allows highly granular control over process creation would be
extremely useful. What I do right now is use a function wrapping up
System.Diagnostics.Process that handles it for me. Note that there is a
problem with this for some applications; in many cases, specific processes
actually start child processes and exit, so it may sometimes be necessary to
trace parentage for all new processes as well.

I suggest filing this as a feature request on Connect. For now, here's my
workaround function:

function Start-Process
{
Param(
[string]$Filename,
[string]$ArgumentString = [System.String]::Empty,
[switch]$Wait,
$si = New-Object System.Diagnostics.ProcessStartInfo
$si.Filename = $Filename;
if($ArgumentString){$si.Arguments = $ArgumentString};
#$si.Filename, $si.Arguments;
$ps = [System.Diagnostics.Process]::Start($si);
$ps.WaitForExit();
$ps;
}

Note that the returned process is still usable for details, and if you do
have processes descended from the one you invoked, you can check existing
processes to see if they are descended from that process id.


"Adam Milazzo" <adammila@microsoft.com> wrote in message
news:Oyc5baavGHA.1512@TK2MSFTNGP04.phx.gbl...
> In unix shells, child processes always block unless you tell them not to.
>
> Eg,
>
> $ xcalc # this blocks and doesn't return until xcalc is done
> $ xcalc & # this does not block
> [1]+ xcalc # it's running in the background and here's the job number
>
> In PoSH, there seems to be no way to control this. In general, it seems
> that programs block if they are console programs, but not if they are GUI
> programs. However, sometimes one would want a GUI program to block, or a
> console program to not block.
>
> I realize it would complicate your "output pipeline", and your single
> $error array.
>
> But I would be perfectly content if you could not "background" a process
> unless the end of the pipe was something other than the console. I would
> also be content if backgrounded processes had no access to $error.
>
> But I don't like having to start up a new instance of PoSH (which takes 9
> seconds on my 3.06ghz machine -- another gripe I have), navigate to the
> same directory, run the command I want, and exit, because the command
> would block my main PoSH window.
>
> I'd much rather be able to do:
>
> build >outfile & # run my build process in the background, write the
> output to 'outfile'
>
> (I know you're using & for something else already, but you get the idea.)
> Then when the build is done, I'll get a notification.
>
> Even more importantly, I'd like to be able to force PoSH to wait for a GUI
> processes to finish.
>
> For instance:
>
> $temp = createTempFile()
> cat "some initial content" >$temp
> openEditor $temp # let the user edit it
> # the user is done, so something with $temp
> rm $temp
>
> I know I could probably simulate some of these things by bypassing PoSH
> and spawning new processes myself, but this is the kind of thing that
> should be handled in the shell.
>

"Alex K. Angelopoulos [MVP]" <aka@online.mvps.org> wrote in message
news:OXykFkhvGHA.2120@TK2MSFTNGP03.phx.gbl...

> I suggest filing this as a feature request on Connect. For now, here's my
> workaround function:

Oops. Minor error in the function. I've fixed it and also added a timeout:

function Start-Process
{
Param(
[string]$Filename,
[string]$ArgumentString = [System.String]::Empty,
[switch]$Wait,
[int]$Timeout = 0)
$si = New-Object System.Diagnostics.ProcessStartInfo
$si.Filename = $Filename;
if($ArgumentString){$si.Arguments = $ArgumentString};
$ps = [System.Diagnostics.Process]::Start($si);
if($Wait){
if($Timeout -gt 0){$ps.WaitForExit($Timeout*1000)}
else{$ps.WaitForExit();}
}
$ps;
}

Alex K. Angelopoulos [MVP] wrote:
> A cmdlet that allows highly granular control over process creation would be
> extremely useful. What I do right now is use a function wrapping up
> System.Diagnostics.Process that handles it for me. Note that there is a
> problem with this for some applications; in many cases, specific processes
> actually start child processes and exit, so it may sometimes be necessary to
> trace parentage for all new processes as well.
>
> I suggest filing this as a feature request on Connect. For now, here's my
> workaround function:
> [snip]
> Note that the returned process is still usable for details, and if you do
> have processes descended from the one you invoked, you can check existing
> processes to see if they are descended from that process id.
Cool. Yeah, workarounds are possible, and you could even extend this
function to add some kind of job control, but the problems with it are:

1) No support for PS pipelining and CmdLets. It only works for Win32
programs, and they can't participate in PS pipelines. So you can't
easily do:

% longRunningTask | my-filter >output & # run in background
[1]+ longRunningTask (running)

2) Only works if you know the path to the executable name. So it won't
work if the user has defined a PS function to open, say, his favorite
editor:

% editor file.txt # invoke user's favorite editor
% StartProcess editor # don't know the exact path...

It works fine when 'editor' is a program, but not if the user defines it
with an alias or a function.

3) You have to construct the arguments string yourself, requiring
escaping of arguments, etc. Not an impossible task, but more complicated
than you might think to get perfectly right. :-) It'd be nice to have
shell support here...

The third could be fixed with a support function to do escaping, the
second can be worked around by using variables instead of functions
(like they did in CMD.exe -- StartProcess $EDITOR), but the first really
needs shell support.

That said, creating processes manually does help a little bit.

Some comments inline. This gets messy because of how Windows processes
generally work, unfortunately. Your points about what we really need are
spot on, but you're being imprecise about a couple of things.

Your points towards the end do get into some important low-level issues that
_do_ need to be addressed. Some of them are on the shopping list, but
couldn't be done in the timeframe for version 1.

"Adam Milazzo" <adamm@san.rr.com> wrote in message
news:OhV8uzjvGHA.3912@TK2MSFTNGP03.phx.gbl...
> Alex K. Angelopoulos [MVP] wrote:

>> Note that the returned process is still usable for details, and if you do
>> have processes descended from the one you invoked, you can check existing
>> processes to see if they are descended from that process id.

> Cool. Yeah, workarounds are possible, and you could even extend this
> function to add some kind of job control, but the problems with it are:
>
> 1) No support for PS pipelining and CmdLets.

I think I see what you're after, but as you phrased it this isn't true. You
can get exit status back if you are waiting for termination, and if you are
running the process in a separate thread, you can check it from PowerShell.

Is the following a decent paraphrase of what you're talking about?
' you cannot start a process asynchronously, capturing its output, and bring
its output buffer to the foreground when you want from a PowerShell prompt.'

Not only is that very true, but the same problem exists for PowerShell
cmdlets/functions/scripts in the initial release: you can't asynchronously
execute a same-console PowerShell command. This is a significant issue that
needs to be handled. There's a long-term plan involving runspaces which
should also help with some of .NET's own un-scriptish versioning issues.

> ... It only works for Win32 programs, and they can't participate in PS
> pipelines. So you can't easily do:

You mean GUI subsystem programs, I think - console-based programs are
generally Win32 as well.

> % longRunningTask | my-filter >output & # run in background
> [1]+ longRunningTask (running)

right. And even though you can do async execution if you're willing to
forego the output access, you're toast if you're trying to work via a telnet
shell since the process would detach from the window.



> 2) Only works if you know the path to the executable name. So it won't
> work if the user has defined a PS function to open, say, his favorite
> editor:
>
> % editor file.txt # invoke user's favorite editor
> % StartProcess editor # don't know the exact path...
>
> It works fine when 'editor' is a program, but not if the user defines it
> with an alias or a function.

Unfortunately true, because this isn't a direct invocation. You can indeed
use functions, variables, and aliases here, but they would all need to
evaluate to appropriate elements, which kind of reduces the ease of use,
since you can't mix the commandline arguments into this - process start here
doesn't allow passing a complete command-line. :|

> 3) You have to construct the arguments string yourself, requiring escaping
> of arguments, etc. Not an impossible task, but more complicated than you
> might think to get perfectly right. :-) It'd be nice to have shell
> support here...

And this problem will still exist with a background invocation operator,
unfortunately - although the applications that will have problems are
limited. Since PS parses every command line and then re-assembles it for
external executables, an unescaped bare argument in this form:
-<x>:<value>
is reassembled as:
-<x>: <value>
This shouldn't be a problem for most POSIX apps I think, but ones that use
homegrown parsers can be in trouble.

> The third could be fixed with a support function to do escaping, the
> second can be worked around by using variables instead of functions (like
> they did in CMD.exe -- StartProcess $EDITOR), but the first really needs
> shell support.
>
> That said, creating processes manually does help a little bit.

>> Cool. Yeah, workarounds are possible, and you could even extend this
>> function to add some kind of job control, but the problems with it are:
>>
>> 1) No support for PS pipelining and CmdLets.
>
> I think I see what you're after, but as you phrased it this isn't true. You
> can get exit status back if you are waiting for termination, and if you are
> running the process in a separate thread, you can check it from PowerShell.
>
> Is the following a decent paraphrase of what you're talking about?
> ' you cannot start a process asynchronously, capturing its output, and bring
> its output buffer to the foreground when you want from a PowerShell prompt.'
More like "You cannot run an arbitrary PS command (ie, a strongly-typed
pipeline containing objects and not just text) asynchronously."

I'm not too concerned about whether I can bring its output buffer to the
foreground. As long as I can redirect the output somewhere that I can
get at it later, I'm content. :-)

> Not only is that very true, but the same problem exists for PowerShell
> cmdlets/functions/scripts in the initial release: you can't asynchronously
> execute a same-console PowerShell command. This is a significant issue that
> needs to be handled. There's a long-term plan involving runspaces which
> should also help with some of .NET's own un-scriptish versioning issues.
Yeah, I would want to be able to run any arbitrary PowerShell command
asynchronously.

I think the easiest implementation may involve running each command in a
thread, and the thread always waiting for everything in that command to
complete. If it's in the foreground, it's running in the main thread. If
it's running in the background, it's running in a new thread that's also
blocking on that command. When the thread terminates, it posts a
notification that can be displayed by the main thread. Then job control
would be implemented as manipulation of these threads... suspending,
killing, etc.

>> ... It only works for Win32 programs, and they can't participate in PS
>> pipelines. So you can't easily do:
>
> You mean GUI subsystem programs, I think - console-based programs are
> generally Win32 as well.
Yes, I mean GUI programs.

>> % longRunningTask | my-filter >output & # run in background
>> [1]+ longRunningTask (running)
>
> right. And even though you can do async execution if you're willing to
> forego the output access, you're toast if you're trying to work via a telnet
> shell since the process would detach from the window.
Hmm, I don't know the details about how telnet shells are implemented.
Is the shell not running on the server?

>> 2) Only works if you know the path to the executable name. So it won't
>> work if the user has defined a PS function to open, say, his favorite
>> editor:
>>
>> % editor file.txt # invoke user's favorite editor
>> % StartProcess editor # don't know the exact path...
>>
>> It works fine when 'editor' is a program, but not if the user defines it
>> with an alias or a function.
>
> Unfortunately true, because this isn't a direct invocation. You can indeed
> use functions, variables, and aliases here, but they would all need to
> evaluate to appropriate elements, which kind of reduces the ease of use,
> since you can't mix the commandline arguments into this - process start here
> doesn't allow passing a complete command-line. :|
>
>> 3) You have to construct the arguments string yourself, requiring escaping
>> of arguments, etc. Not an impossible task, but more complicated than you
>> might think to get perfectly right. :-) It'd be nice to have shell
>> support here...
>
> And this problem will still exist with a background invocation operator,
> unfortunately - although the applications that will have problems are
> limited. Since PS parses every command line and then re-assembles it for
> external executables, an unescaped bare argument in this form:
> -<x>:<value>
> is reassembled as:
> -<x>: <value>
> This shouldn't be a problem for most POSIX apps I think, but ones that use
> homegrown parsers can be in trouble.
Well, if it's a problem in any case, then it's a kind of separate issue.
I'm not too concerned about this one...

"Adam Milazzo" <adamm@san.rr.com> wrote in message
news:%231KuylpvGHA.4512@TK2MSFTNGP05.phx.gbl...

> I'm not too concerned about whether I can bring its output buffer to the
> foreground. As long as I can redirect the output somewhere that I can get
> at it later, I'm content. :-)

OK. This is coming - I don't know when, but runspaces are definitely on the
road map, and redirection in some form _will_ work. Theoretically, this is
already possible, although in a rather ugly fashion: you would have to
serialize output using Export-Csv or Export-CliXml, then invoke a PowerShell
instance that imports the data, runs a command, then exports it when done,
and your main instance can check the process periodically to see if it has
exited yet. This is of course an ugly approach, suitable only as a stopgap.

(snipping bits I don't know about)

>> right. And even though you can do async execution if you're willing to
>> forego the output access, you're toast if you're trying to work via a
>> telnet shell since the process would detach from the window.
> Hmm, I don't know the details about how telnet shells are implemented. Is
> the shell not running on the server?

It is. What I mean is that unlike an on-console session, you wouldn't be
able to "see" a process that has detached - it would be running on the
server, but in a separate hidden window not accessible from the client and
which the telnet session on the server has no means to connect to. so you
won't see anything going on in that window.