Vista - Wrap command shell in System.Diagnostics.Process

Does anyone know why using System.Diagnostics.Process to "wrap" a console
application does not always transmit the I/O, depending on what processes
you're trying to "consume"? PowerShell, for example, does not seem to
process any I/O through the Process object.

I know that in the case of PowerShell there are better ways to "wrap" the
console by directly interfacing with the assemblies of
System.Management.Automation or some similarly named namespace, but I'm
trying to use generic command line wrappers for multiple types of processes
that use the console I/O, and PowerShell was a handy example of why this
won't work.

ProcessStartInfo psi = new ProcessStartInfo(
@"C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe");
psi.Arguments = "-NoLogo";
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
//psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
bool started = process.Start();
if (started)
{
process.StandardInput.WriteLine("2+2");
process.StandardInput.Flush();
string ret = process.StandardOutput.ReadLine(); // <-- stalls here
System.Console.WriteLine("PowerShell says \"2+2=" + ret + "\".");
}


Another one I was trying to "wrap" was the original implementation of Dave
Raggett's HTML Tidy. The stuff below sometimes stalls on ReadToEnd(). It
seemed to always stall until I added "process.StandardInput.Close();" after
"process.StandardInput.Flush();" but it still stalls on ReadToEnd() half the
time.

...
_TidyProcessStartInfo = new ProcessStartInfo(
Directory.GetParent(System.Reflection.Assembly.GetExecutingAssembly().Location)
+ "\\tidy.exe");
_TidyProcessStartInfo.UseShellExecute = false;
_TidyProcessStartInfo.CreateNoWindow = true;
_TidyProcessStartInfo.RedirectStandardInput = true;
_TidyProcessStartInfo.RedirectStandardOutput = true;
_TidyProcessStartInfo.RedirectStandardError = true;
.. . .
Process process = new Process();
process.StartInfo = _TidyProcessStartInfo;
process.ErrorDataReceived += new
DataReceivedEventHandler(Exe_ErrorDataReceived);
bool started = process.Start();
if (started)
{
//process.StandardInput.AutoFlush = true;
process.StandardInput.WriteLine(input);
process.StandardInput.Flush();
process.StandardInput.Close();
ret = process.StandardOutput.ReadToEnd();
}



Thanks,
Jon

On Apr 5, 12:20 pm, "Jon Davis" <j...@REMOVE.ME.PLEASE.jondavis.net>
wrote:
> Does anyone know why using System.Diagnostics.Process to "wrap" a console
> application does not always transmit the I/O, depending on what processes
> you're trying to "consume"? PowerShell, for example, does not seem to
> process any I/O through the Process object.
>
> I know that in the case of PowerShell there are better ways to "wrap" the
> console by directly interfacing with the assemblies of
> System.Management.Automation or some similarly named namespace, but I'm
> trying to use generic command line wrappers for multiple types of processes
> that use the console I/O, and PowerShell was a handy example of why this
> won't work.
>
> ProcessStartInfo psi = new ProcessStartInfo(
> @"C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe");
> psi.Arguments = "-NoLogo";
> psi.UseShellExecute = false;
> psi.CreateNoWindow = true;
> psi.RedirectStandardInput = true;
> psi.RedirectStandardOutput = true;
> //psi.RedirectStandardError = true;
> Process process = new Process();
> process.StartInfo = psi;
> bool started = process.Start();
> if (started)
> {
> process.StandardInput.WriteLine("2+2");
> process.StandardInput.Flush();
> string ret = process.StandardOutput.ReadLine(); // <-- stalls here
> System.Console.WriteLine("PowerShell says \"2+2=" + ret + "\".");
>
> }
>
> Another one I was trying to "wrap" was the original implementation of Dave
> Raggett's HTML Tidy. The stuff below sometimes stalls on ReadToEnd(). It
> seemed to always stall until I added "process.StandardInput.Close();" after
> "process.StandardInput.Flush();" but it still stalls on ReadToEnd() half the
> time.
>
> ..
> _TidyProcessStartInfo = new ProcessStartInfo(
> Directory.GetParent(System.Reflection.Assembly.GetExecutingAssembly().Location)
> + "\\tidy.exe");
> _TidyProcessStartInfo.UseShellExecute = false;
> _TidyProcessStartInfo.CreateNoWindow = true;
> _TidyProcessStartInfo.RedirectStandardInput = true;
> _TidyProcessStartInfo.RedirectStandardOutput = true;
> _TidyProcessStartInfo.RedirectStandardError = true;
> . . .
> Process process = new Process();
> process.StartInfo = _TidyProcessStartInfo;
> process.ErrorDataReceived += new
> DataReceivedEventHandler(Exe_ErrorDataReceived);
> bool started = process.Start();
> if (started)
> {
> //process.StandardInput.AutoFlush = true;
> process.StandardInput.WriteLine(input);
> process.StandardInput.Flush();
> process.StandardInput.Close();
> ret = process.StandardOutput.ReadToEnd();
>
> }
>
> Thanks,
> Jon

You are probably hitting a deadlock issue when redirecting out. See
the docs on the RedirectStandardOutput property for more information:

http://msdn2.microsoft.com/en-us/lib...ardoutput.aspx

Chris

"Chris Dunaway" <dunawayc@gmail.com> wrote in message
news:1175799852.010209.282490@n59g2000hsh.googlegroups.com...
> You are probably hitting a deadlock issue when redirecting out. See
> the docs on the RedirectStandardOutput property for more information:
>
> http://msdn2.microsoft.com/en-us/lib...ardoutput.aspx
>
> Chris

Clearly it is deadlocking, the problem is I don't know how to apply any
workaround to the code I provided in the OP given the samples provided in
the referenced link, which although appears detailed at first glance is
really too brief to be useful.

Perhaps someone has the time and patience to make my two samples work?

Jon

On Thu, 05 Apr 2007 13:24:44 -0700, Jon Davis
<jon@REMOVE.ME.PLEASE.jondavis.net> wrote:

> Clearly it is deadlocking, the problem is I don't know how to apply any
> workaround to the code I provided in the OP given the samples provided in
> the referenced link, which although appears detailed at first glance is
> really too brief to be useful.
>
> Perhaps someone has the time and patience to make my two samples work?

Most of us probably don't have the associated programs you're running
installed. I know I don't.

That said, reading the documentation it occurs to me that you may be
running into a problem with your processing of the StandardError stream.
That is, I don't see anything in your code that would read from that
stream. According to the documentation, if your own application does not
keep up with reading data from the streams, the child process may block
once the stream's buffer is full. That would prevent your child process
from continuing, while your parent process sits there waiting to read more
from the other stream.

If you are going to redirect both StandardError and StandardOutput, it
seems to me that the only robust way to do that is to ensure that both
streams are being read from simultaneously. You can do that by providing
for asychronous reading of at least one of them (ie BeginOutputReadLine
and/or BeginErrorReadLine). That way, you can ensure that you will always
be reading from both and not preventing the child process from
continuing. Alternatively, use two different threads to read from the two
streams.

Both of these techniques are documented near the end of the "Remarks"
section for the RedirectStandardOutput property in MSDN (the link you say
is "too brief to be useful").

Pete

StandardError was commented out, to avoid that exact scenario.

Jon

"Peter Duniho" <NpOeStPeAdM@nnowslpianmk.com> wrote in message
newsp.tqb4yrsb8jd0ej@petes-computer.local...
> On Thu, 05 Apr 2007 13:24:44 -0700, Jon Davis
> <jon@REMOVE.ME.PLEASE.jondavis.net> wrote:
>
>> Clearly it is deadlocking, the problem is I don't know how to apply any
>> workaround to the code I provided in the OP given the samples provided in
>> the referenced link, which although appears detailed at first glance is
>> really too brief to be useful.
>>
>> Perhaps someone has the time and patience to make my two samples work?
>
> Most of us probably don't have the associated programs you're running
> installed. I know I don't.
>
> That said, reading the documentation it occurs to me that you may be
> running into a problem with your processing of the StandardError stream.
> That is, I don't see anything in your code that would read from that
> stream. According to the documentation, if your own application does not
> keep up with reading data from the streams, the child process may block
> once the stream's buffer is full. That would prevent your child process
> from continuing, while your parent process sits there waiting to read more
> from the other stream.
>
> If you are going to redirect both StandardError and StandardOutput, it
> seems to me that the only robust way to do that is to ensure that both
> streams are being read from simultaneously. You can do that by providing
> for asychronous reading of at least one of them (ie BeginOutputReadLine
> and/or BeginErrorReadLine). That way, you can ensure that you will always
> be reading from both and not preventing the child process from
> continuing. Alternatively, use two different threads to read from the two
> streams.
>
> Both of these techniques are documented near the end of the "Remarks"
> section for the RedirectStandardOutput property in MSDN (the link you say
> is "too brief to be useful").
>
> Pete

In addition to my previous post...

Obviously if you do not actually need to read the StandardError stream,
then perhaps the simplest solution is to not redirect that stream in the
first place.

On Thu, 05 Apr 2007 17:10:33 -0700, Jon Davis
<jon@REMOVE.ME.PLEASE.jondavis.net> wrote:

> StandardError was commented out, to avoid that exact scenario.

Not in the second example you gave.

In any case, you don't have a sample that anyone else can use to try to
reproduce your problem, so it's not possible for anyone to look directly
at what's going on. In your first example, you use ReadLine which will
block until there's a line to be read. Maybe that line never comes. In
your second example, you use ReadToEnd which cannot complete if the child
process gets blocked itself. Since RedirectStandardError isn't commented
out, it's entirely possible that's your deadlock there.

Absent a minimal-but-complete sample of code that reliably reproduces the
problem, there may not be any better advice you can get. The best method
IMHO would be to run both processes under a debugger and just look to see
what they are waiting on when things appear to hang. However, I still
can't figure out how to get VS2005's debugger to allow me to debug threads
the way previous versions did, so I'm a bit hesitant to suggest that,
since it might not be as useful advice as it initially seems.

Pete

"Peter Duniho" <NpOeStPeAdM@nnowslpianmk.com> wrote in message
newsp.tqb8fs0u8jd0ej@petes-computer.local...
> In any case, you don't have a sample that anyone else can use to try to
> reproduce your problem, so it's not possible for anyone to look directly
> at what's going on.

Lots of people have PowerShell, it's a Windows Update piece.

Just swap out PowerShell for CMD.exe:

using System;
using System.Collections.Generic;
using System.Diagnostics;
public class MyClass
{
public static void Main()
{
try {
ProcessStartInfo psi = new ProcessStartInfo(
@"C:\Windows\system32\cmd.exe");
psi.UseShellExecute = false;
psi.CreateNoWindow = true;
psi.RedirectStandardInput = true;
psi.RedirectStandardOutput = true;
//psi.RedirectStandardError = true;
Process process = new Process();
process.StartInfo = psi;
bool started = process.Start();
if (started)
{
process.StandardOutput.ReadLine(); // "Microsoft Windows"
process.StandardOutput.ReadLine(); // "Copyright Microsoft"
process.StandardOutput.ReadLine(); // [blank line following logo]
process.StandardOutput.ReadLine(); // [command entry (echo)]
process.StandardInput.WriteLine("echo Blah");
process.StandardInput.Flush();
string ret = process.StandardOutput.ReadLine(); // <-- stalls here
System.Console.WriteLine("CMD.exe says " + ret + "\".");
}
} catch (Exception e) {
}
RL();
}

}

On Fri, 06 Apr 2007 10:11:42 -0700, Jon Davis
<jon@REMOVE.ME.PLEASE.jondavis.net> wrote:

> Lots of people have PowerShell, it's a Windows Update piece.
>
> Just swap out PowerShell for CMD.exe:

I don't know what you mean by "Windows Update piece". I have Windows
Update, use it all the time, but don't have PowerShell installed.

In any case, thank you for the sample code. On my computer, it runs just
fine as long as I remove the extra ReadLine you've got (the one commented
"command entry (echo)").

For what it's worth, all I did was break in the debugger when the program
got blocked, and saw that it was waiting at that line which showed that
you never got to the point of writing to the input stream, meaning you
were reading too many "discardable" lines up front.

Of course, one of the things this exercise illustrates is the fragility of
the approach you're using. Even if it works fine now (as it does on my
computer...can't say whether that's true on yours), you're tied to a
precise sequence of lines of input and output. A trivial bug in your own
code causes the whole thing to just stop, and of course if there are any
changes to the external console application that can really mess things up
too (either by making your application just stop in the wrong place, or
perhaps worse by causing your application to send the wrong commands to
the process).

Granted, this is all just sample code so who knows whether this is really
how you're doing things. But if it is, beware. Many pitfalls lie ahead.

Pete

"Peter Duniho" <NpOeStPeAdM@nnowslpianmk.com> wrote in message
newsp.tqdkfvmb8jd0ej@petes-computer.local...
> On Fri, 06 Apr 2007 10:11:42 -0700, Jon Davis
> <jon@REMOVE.ME.PLEASE.jondavis.net> wrote:
>
>> Lots of people have PowerShell, it's a Windows Update piece.
>>
>> Just swap out PowerShell for CMD.exe:
>
> I don't know what you mean by "Windows Update piece". I have Windows
> Update, use it all the time, but don't have PowerShell installed.
>
> In any case, thank you for the sample code. On my computer, it runs just
> fine as long as I remove the extra ReadLine you've got (the one commented
> "command entry (echo)").
>
> For what it's worth, all I did was break in the debugger when the program
> got blocked, and saw that it was waiting at that line which showed that
> you never got to the point of writing to the input stream, meaning you
> were reading too many "discardable" lines up front.

I added them one by one as I read the ReadLine() which returned:

- two lines of the logo
- an empty line
- the echo of my "echo" command, BEFORE it gets executed.

Are you sure that what you commented out is not the command itself being
echoed rather than the execution of the ECHO command? Perhaps we have
different versions.

> Of course, one of the things this exercise illustrates is the fragility of
> the approach you're using. Even if it works fine now (as it does on my
> computer...can't say whether that's true on yours), you're tied to a
> precise sequence of lines of input and output.

Understood, but while generally such an interface would be useless as a
generic console app solution I do have some precise sequences in mind, or as
in the case of Tidy and some other EXEs I'm dealing with a constant of
either input one command and get back one line of output (ReadLine()), input
one command and get back several lines of results (ReadToEnd()), or input
multiple lines as a single input and read back the output. I am not dealing
with processes that have a dialogue of more than one input or more than one
string of output.

> ... beware. Many pitfalls lie ahead.

Totally. I would still like to build out my process wrappers described above
and I'm still not fully understanding what I'm missing.

Jon