Redirecting stdout from a process invoked from a plugin

[Ton]

In my "MVPRecoder" app, I do something similar....

One other thing I have done is start seperate "stdout" and "stderr" reader-threads to check / evaluate what's written to stdout/stderr...

Here's the (C#) code:

Code:

            Process proc = new Process();
            proc.EnableRaisingEvents = false;
            proc.StartInfo.FileName = recode_path;
            proc.StartInfo.Arguments = command_line;
            proc.StartInfo.UseShellExecute = false;
            proc.StartInfo.RedirectStandardOutput = true;
            proc.StartInfo.RedirectStandardError = true;
            
            proc.Start();

            StdReader cOut = new StdReader();
            cOut.theReader = proc.StandardOutput;
            Thread tOut = new Thread(new ThreadStart(cOut.StdOutReader));

            StdReader cErr = new StdReader();
            cErr.theReader = proc.StandardError;
            Thread tErr = new Thread(new ThreadStart(cErr.StdErrReader));

            tOut.Start();
            tErr.Start();
            proc.WaitForExit();
            tOut.Abort();
            tErr.Abort();

And here's the code for the stdout/stderr reader class:

Code:

    class StdReader
    {
        public StreamReader theReader;
        public string data;
        
        public void StdErrReader() {
            
            try {
                while (true) {
                    data = theReader.ReadLine();
                    // Ignore all standard errors...
                    if (!data.StartsWith("WARNING"))
                        if (!data.StartsWith("ERROR: SCR:"))
                            if (!data.StartsWith("1 duplicate frame(s)"))
                                if (data.IndexOf("buffer underflow") == 0)
                                    Console.WriteLine(data);
                }
            } catch {}
        }
        public void StdOutReader() {
            
            try {
                while (true) {
                    data = theReader.ReadLine();
                    if (data.StartsWith("Pos:")) {
                        Console.Write("\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b");
                        Console.Write(data.Substring(4, 28).TrimStart());
                     }
                }
            } catch {}
        }    
    }

the "\b\b"'s are interpreted by the console on WinX to delete previous character on the console..... That way you can make a nice progress-indicator (from the command-line anyway).

[alibert]

Hi,

Any ideas? Sympathy, even?

since .NET 2, there two new events in the Process class: ErrorDataReceived and OutputDataReceived. Instead of doing a "busy wait" on the process' output, you can simply add an eventhandler and you'll get the output deleivered to the handler method.

In my DboxRecorder plugin, I'm doing something like this:

Code:

// Start the process
ggrabProcess = new Process();
ggrabProcess.StartInfo = ggrabStartInfo;
ggrabProcess.ErrorDataReceived += new DataReceivedEventHandler(GgrabErrorDataReceived);
ggrabProcess.Start();

// Start the asynchronous read of the standard error stream
ggrabProcess.BeginErrorReadLine();

And:

Code:

private void GgrabErrorDataReceived(object sender, DataReceivedEventArgs e)
{
    if (IRecordingSource.RecordingStatus.RECORDING_IN_PROGRESS == RecordingStatus &&
        null != e.Data && true == e.Data.Contains("timeout"))

[...]

-alibert

[Ton]

Cool,

I'll implement that when I update MVPRecode to 2.0.....

Thanks for sharing that...

[acheyne]

ubu - Sorry for not actually trying to compile and run that - I'm on a machine without a C# development environment. I was simply looking at some C/C++ code I have that does what you want. I had assumed that C# is built on top of the lower level SDKs and would work properly.

My other suggestion would be to start your process up through a separate thread if your only problem is having to wait for the app to exit.

[NiteShdw]

Hi,


since .NET 2, there two new events in the Process class: ErrorDataReceived and OutputDataReceived. Instead of doing a "busy wait" on the process' output, you can simply add an eventhandler and you'll get the output deleivered to the handler method.

My first attemp was to use that same function "BeginReadLine()" with EventHandlers. I came across the problem that the Event was only fired at the beginning and end of running comskip. After the program was finished, I basically just ended up with the line that said "Commercials found".

I also had this problem with mpgtx. What I think is happening is that rather than doing a WriteLine (or equivilent), the console apps are do a write and then writing backspaces and rewriting the line in order to show a progress indicator. Apparently, the BeginReadLine() function doesn't fire the event until an actual line break is written to the buffer.

So, is it possible to just read the underlying stream as it's being written? And use my own buffer to check for certains strings (like "95%")?

[Ton]

Well, if you look at my post earlier, you see that I started a seperate thread....

in that thread my code currently uses "ReadLine" to read from the buffer (thus waiting for the line-feed to start reading), but you could replace that with another read-function (that reads a max number of characters), and that should allow you to catch the characters as they're being written...

Only thing I'm not sure of (and I lack the time to do a quick "looksee" for you), is what happens (underwater) with the backspaces....

[ubu]

My other suggestion would be to start your process up through a separate thread if your only problem is having to wait for the app to exit.

As I said in my earlier response:

In the event, I was able to produce an identical result by creating a separate thread (using a BackgroundWorker object) and then using that to run the code that invokes the application and writes its output to a file.

Thanks to everybody for their suggestions. The posts from Ton and alibert really point up the fact that .Net (usually) offers a number of different ways to solve the same problem. Unfortunately this creates a new problem: deciding which is the best one to use .

Could you post the code that you are using now to redirect standard output while waiting for the application to finish?

I'll post the code for my version of the solution later today. Gotta run now.

[ubu]

Could you post the code that you are using now to redirect standard output while waiting for the application to finish?

So here's the approach I took (somewhat simplified for demonstration purposes) using a BackgroundWorker to run my processes asynchronously. It seems to work very well. I'm actually starting 1 to n processes within the background thread and, when they have all completed, reading the output file(s) they created. The stdout of each is captured to a separate log file for later perusal.

The actual method to run the process (which I have in a separate class - UbuStreamImporter):

Code:

public void Download_From_DynSource(String dynSourceApp)
        {
            String logFile = dynSourceApp.Replace(".exe", ".Playlist.log");
            ProcessStartInfo startInfo = new ProcessStartInfo(dynSourceApp, "");
            startInfo.UseShellExecute = false;
            startInfo.RedirectStandardOutput = true;
            startInfo.CreateNoWindow = true;
            Process myProcess = Process.Start(startInfo);
            String logLines = myProcess.StandardOutput.ReadToEnd();
            myProcess.WaitForExit();
            File.WriteAllText(logFile, logLines);
         }

In the plugin class, I add: BackgroundWorker backgroundWorker = new BackgroundWorker();
and then, in the plugin constructor:

Code:

backgroundWorker.DoWork += new DoWorkEventHandler(backgroundWorker_DoWork);
            backgroundWorker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(backgroundWorker_RunWorkerCompleted);
            Initiate_DynSource_Downloads();

Then:

Code:

private void Initiate_DynSource_Downloads()
        {
            ArrayList myDynSourceList = new ArrayList();
            foreach (DataRow myDynSource in xmlSchema11.Dynamic_Sources.Rows)
            {
                String myDynSourceApp = myDynSource["DynSource_App"].ToString();
                myDynSourceList.Add(myDynSourceApp);
            }
            backgroundWorker.RunWorkerAsync(myDynSourceList);
        }

The RunWorkerAsync call triggers this method:

Code:

private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
        {
            ArrayList myDynSourceList = new ArrayList();
            myDynSourceList = (ArrayList)e.Argument;
            UbuStreamImporter myImporter = new UbuStreamImporter();
            foreach (String myDynSourceApp in myDynSourceList)
                myImporter.Download_From_DynSource(myDynSourceApp);
        }

I need to know when the processes have all finished since I want to read the file(s) the apps have created. When the backgroundWorker thread completes, this method gets called:

Code:

private void backgroundWorker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
        {
             foreach (DataRow myDynSource in xmlSchema11.Dynamic_Sources.Rows)
                Import_From_DynSource(myDynSource);
             refreshStationList();
        }

I didn't need to use it but one neat feature is you can use BackgroundWorker.ReportProgress to return the percentage of completion and a "state object" back to the calling thread (so you can check on it's progress from time to time). One caveat (obvious, I suppose): code residing in the background thread can't refer to variables (or anything else) in the parent thread.

As I said in my previous post, I'm not sure if this is a better or worse approach than those Ton and alibert are using. Just different. alibert's solution is startlingly elegant (and certainly the simplest - it passes the "Ockham's Razor" test). If you needed to know when the process had completed (as I do), you could just check Process.HasExited from time to time, I guess.

[alibert]

Hi,

If you needed to know when the process had completed (as I do), you could just check Process.HasExited from time to time, I guess.

there's an event for this purpose, too:

http://msdn2.microsoft.com/en-us/library...xited.aspx

-alibert

[ww4397]

Cool,

I'll implement that when I update MVPRecode to 2.0.....

Thanks for sharing that...

Where can I find MVPRecode? I don't see it on the Wiki.