Capturing console output with Delphi 2010/XE (revised)

Following my previous post (Capturing console output with Delphi 2010/XE) and with all the great comments received on it, I have decided to publish the new solution provided by Lübbe Onken which solves the hanging issue when capturing the output for different kind of commands like ping, netstat, etc. The problem occurs on the last ReadFile which this solution will fix.
Here I'm summarizing Lübbe Onken comments on this issue:
"The current implementation assumes that if the external process is not finished yet, there must be something available to be read from the read pipe. This is not necessarily the case. If we use PeekNamedPipe to check for available data before entering the internal repeat loop, everything is fine.
I also put the CloseHandle into try ... finally and moved Application.ProcessMessages behind the internal read loop, because IMHO the screen update is better handled after processing the callback than before. But this is just cosmetic."

Here you can find the source code:


usage:

I want to say thanks to everyone who spend time looking at this issue and for making the community work and grow.
Jordi

Comments

  1. Very interesting, thank you.
    Now a related question: how to interrupt long tasks that are being captured from console?
    i.e. say I want to interrupt traceroute www.google.com after the first few hops?

    ReplyDelete
    Replies
    1. Hi Fabio,

      Just add a boolean variable on the repeat to exit it:
      until (dRunning <> WAIT_TIMEOUT) or breakLoop;

      This will help you break the loop.

      Jordi

      Delete
  2. Finally I found working example. Thank you!

    ReplyDelete
  3. Thank you! Been looking for exactly this example. Tried a number of different console capture examples, some hanging as mentioned. None worked for use with the FireBird console app, GFIX. Works beautifully. For anyone else wanting to apply it to GFIX, just remember to disconnect from the database before running GFIX.
    Chuck Belanger

    ReplyDelete
    Replies
    1. You're welcome Charles. Great that it is useful!!
      Jordi

      Delete
  4. that's really interesting, with a smart way!
    I just receive this error:
    "ERROR: The target system must be running a 32 bit OS."
    (running XE on WIN7 x64)
    ...is there a way to solve it... or is it impossible?

    ReplyDelete
    Replies
    1. Hi Filippo,

      I can't get this error and I'm running it under Windows 7 64 bits as well.
      What are you trying to execute in the command line?

      Jordi

      Delete
  5. hola, supongamos que tengo que enviar un ctrl+c para cancelar la accion de algun programa, como lo podria hacer?
    saludos

    ReplyDelete
    Replies
    1. Hola,

      Si miras en los comenarios defino un breakLoop. Puedes marcarlo como true si pulsas las teclas cntrl+c y tener asi mas control en tu aplicación.
      Jordi

      Delete
  6. Thanks Jordi,
    it's definitively an interesting way!
    Does it functions under delphi 2007?

    ReplyDelete
    Replies
    1. Hi,

      It should work, but you will have to test it.
      The section that uses generics and anonymous methods won't work though.
      Jordi

      Delete
  7. hey, im usin it for ffmpeg and after ~10 minutes it crashes. Any idea?

    ReplyDelete
    Replies
    1. Hi, Could it be the ffmpeg is doing something else behind the scene? I have used this approach for batch files running multiple applications lasting for 40 min and it works like a charm. Never had any issue.
      Do you get any sort of error?

      Post it here and we'll have a look.

      Jordi

      Delete
  8. Hello,
    What a shame I so not know how to use it. Should I trat it as a component or prepere component somehow or maybe I shoulf just add it to my code in unit1(form 1)? Sorry for that, I am starting, but It would solve all my problems.
    Anja

    ReplyDelete
    Replies
    1. Hi Anja,

      Just put it in a form and run it.

      Jordi

      Delete
    2. Thank you for help, but one more question. Does it communicate with console or just capture the output. I mean do I need to prepare any connection with the console? Why in usage there is 'java -version'?
      Anja

      Delete
    3. Hi Anja,

      The methode creates a pipe with the console, so it captures everything is being dumped. You don't have to prepare anything, the method is sorting all those things for you. I'm using java -version because this command displays values in the console. Just type in your cmd -> java -version and you should get something similar to:

      java version "1.7.0_05"
      Java(TM) SE Runtime Environment (build 1.7.0_05-b05)
      Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)

      etc.

      I hope this sorts your issues.

      Jordi

      Delete
  9. I have already tried something like this. It is only a trying, cause I'd like to delete DosCommand in my original project and reprece it with this solution. Unfortunetelly nothing works. I do not know why, I am sure that I did something wrong. Could you take a look, please? It is a new project to text it(and also does not work):
    Source code:

    unit Unit1;

    interface

    uses
    Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
    Dialogs, StdCtrls;

    type
    TForm1 = class(TForm)
    Memo1: TMemo;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
    procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);

    private
    { Private declarations }
    public
    { Public declarations }
    end;

    var
    Form1: TForm1;

    implementation

    {$R *.dfm}

    //Anonymous procedure approach by Lars Fosdal
    type
    TArg = reference to procedure(const Arg: T);

    procedure TForm1.CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
    const
    CReadBuffer = 2400;
    var
    saSecurity: TSecurityAttributes;
    hRead: THandle;
    hWrite: THandle;
    suiStartup: TStartupInfo;
    piProcess: TProcessInformation;
    pBuffer: array [0 .. CReadBuffer] of AnsiChar;
    dBuffer: array [0 .. CReadBuffer] of AnsiChar;
    dRead: DWORD;
    dRunning: DWORD;
    dAvailable: DWORD;
    begin
    saSecurity.nLength := SizeOf(TSecurityAttributes);
    saSecurity.bInheritHandle := true;
    saSecurity.lpSecurityDescriptor := nil;
    if CreatePipe(hRead, hWrite, @saSecurity, 0) then
    try
    FillChar(suiStartup, SizeOf(TStartupInfo), #0);
    suiStartup.cb := SizeOf(TStartupInfo);
    suiStartup.hStdInput := hRead;
    suiStartup.hStdOutput := hWrite;
    suiStartup.hStdError := hWrite;
    suiStartup.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;
    suiStartup.wShowWindow := SW_HIDE;
    if CreateProcess(nil, PChar(ACommand + ' ' + AParameters), @saSecurity, @saSecurity, true, NORMAL_PRIORITY_CLASS, nil, nil, suiStartup,
    piProcess) then
    try
    repeat
    dRunning := WaitForSingleObject(piProcess.hProcess, 100);
    PeekNamedPipe(hRead, nil, 0, nil, @dAvailable, nil);
    if (dAvailable > 0) then
    repeat
    dRead := 0;
    ReadFile(hRead, pBuffer[0], CReadBuffer, dRead, nil);
    pBuffer[dRead] := #0;
    OemToCharA(pBuffer, dBuffer);
    CallBack(dBuffer);
    until (dRead < CReadBuffer);
    Application.ProcessMessages;
    until (dRunning <> WAIT_TIMEOUT);
    finally
    CloseHandle(piProcess.hProcess);
    CloseHandle(piProcess.hThread);
    end;
    finally
    CloseHandle(hRead);
    CloseHandle(hWrite);
    end;
    end;


    procedure TForm1.Button1Click(Sender: TObject);
    begin
    CaptureConsoleOutput('java -version', TEdit1.Text,
    procedure(const Line: PAnsiChar)
    begin
    Memo1.Lines.Add(String(Line));
    end
    );
    end;

    end.


    ReplyDelete
    Replies
    1. Hi Anja,

      Everything looks OK. What version of delphi are you using? Check with other comments. Instead of typing java -version, try typing any other command you want to display in the memo.

      Regards,
      Jordi

      Delete
    2. Hello,
      first of all thank you for helping me. I am using delphi 2010. To be clear Embarcadero RAD Studio 2010. Previously I used Delphi 2007 and it is an environment I know much better. The problem as far as I can see is with two lines:
      unit Unit1;

      interface

      uses
      (...);

      type
      (...);
      procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg); //here is a problem cause I have not declared TArg<>

      When I try to declare it before the procedure itself I get information that ":" expected "<" found.
      Im getting confused. If you have any idea I will be grateful..
      Anja

      Delete
    3. Well,
      Thank you very much for help. I started program, there was only stupid mistake I repeat type formula 2 times. I will inform you about results of using it. Thank you very much.
      Anja

      Delete
    4. Hi Anja,

      Glad that you are sorting out your issues. Please let us know your results.

      Jordi

      Delete
  10. Yes... This worked 1000%...
    Thank very much...

    ReplyDelete
  11. Yes... This is worked very well...
    Thank you so much...

    ReplyDelete
  12. Wow this is great, working on my TOR console capture on Delphi XE2, i have search in many time, many of those hang in form, but i found this usefull here
    thank 4 all,...

    ReplyDelete
  13. It does not work for me, maybe someone has an advice for this.
    I have Windows7 64 and DelphiXE5

    First I tried java -version which makes the CreateProcess return false.

    Then I tried putting 'CMD /c '+command in the CreateProcess, which makes CreateProcess return true, but the message is that it cannot find java.

    I am trying this for quite some time and I run out of possible solutions.

    ReplyDelete
  14. I found what went wrong.

    I was trying to start 64bit Java from a 32 bit application. This does not work.

    ReplyDelete
  15. Excelente Jordi, buen trabajo.
    - Acabo de implementar para crear backups con rar.exe
    - El aplicativo es disparado por medio de un method con DataSnap y es perfecto sin problemas.

    Gracias.

    Startkill
    Lima-Perú

    ReplyDelete
  16. Hi, I am having trouble getting this to work. I am running DelphiXE. Any help you can offer most appreciated.

    I have a form, with a memo, an edit box, and a button. I get these error messages:

    [DCC Error] Unit1.pas(15): E2003 Undeclared identifier: 'TArg'
    [DCC Error] Unit1.pas(34): E2037 Declaration of 'CaptureConsoleOutput' differs from previous declaration
    [DCC Error] Unit1.pas(15): E2065 Unsatisfied forward or external declaration: 'TForm1.CaptureConsoleOutput'
    [DCC Fatal Error] Project1.dpr(5): F2063 Could not compile used unit 'Unit1.pas'

    Here is the full code:

    unit RunCmdLine;

    interface

    uses
    Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
    Dialogs, StdCtrls;

    type
    TForm1 = class(TForm)
    Memo1: TMemo;
    Edit1: TEdit;
    Button1: TButton;
    procedure Button1Click(Sender: TObject);
    procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);

    private
    { Private declarations }
    public
    { Public declarations }
    end;

    var
    Form1: TForm1;

    implementation

    {$R *.dfm}

    //Anonymous procedure approach by Lars Fosdal
    type
    TArg = reference to procedure(const Arg: T);

    procedure TForm1.CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
    const
    CReadBuffer = 2400;
    var
    saSecurity: TSecurityAttributes;
    hRead: THandle;
    hWrite: THandle;
    suiStartup: TStartupInfo;
    piProcess: TProcessInformation;
    pBuffer: array [0 .. CReadBuffer] of AnsiChar;
    dBuffer: array [0 .. CReadBuffer] of AnsiChar;
    dRead: DWORD;
    dRunning: DWORD;
    dAvailable: DWORD;
    begin
    saSecurity.nLength := SizeOf(TSecurityAttributes);
    saSecurity.bInheritHandle := true;
    saSecurity.lpSecurityDescriptor := nil;
    if CreatePipe(hRead, hWrite, @saSecurity, 0) then
    try
    FillChar(suiStartup, SizeOf(TStartupInfo), #0);
    suiStartup.cb := SizeOf(TStartupInfo);
    suiStartup.hStdInput := hRead;
    suiStartup.hStdOutput := hWrite;
    suiStartup.hStdError := hWrite;
    suiStartup.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;
    suiStartup.wShowWindow := SW_HIDE;
    if CreateProcess(nil, PChar(ACommand + ' ' + AParameters), @saSecurity, @saSecurity, true, NORMAL_PRIORITY_CLASS, nil, nil, suiStartup,
    piProcess) then
    try
    repeat
    dRunning := WaitForSingleObject(piProcess.hProcess, 100);
    PeekNamedPipe(hRead, nil, 0, nil, @dAvailable, nil);
    if (dAvailable > 0) then
    repeat
    dRead := 0;
    ReadFile(hRead, pBuffer[0], CReadBuffer, dRead, nil);
    pBuffer[dRead] := #0;
    OemToCharA(pBuffer, dBuffer);
    CallBack(dBuffer);
    until (dRead < CReadBuffer);
    Application.ProcessMessages;
    until (dRunning <> WAIT_TIMEOUT);
    finally
    CloseHandle(piProcess.hProcess);
    CloseHandle(piProcess.hThread);
    end;
    finally
    CloseHandle(hRead);
    CloseHandle(hWrite);
    end;
    end;

    procedure TForm1.Button1Click(Sender: TObject);
    begin
    CaptureConsoleOutput('java -version', Edit1.Text,
    procedure(const Line: PAnsiChar)
    begin
    Memo1.Lines.Add(String(Line));
    end
    );
    end;

    end.


    Thanks,
    Aaron

    ReplyDelete
    Replies
    1. Hi Aaron,

      You need to move the declaration of TArg on the first apparition of type.

      Regards,
      Jordi

      Delete
    2. HI Jordi,

      I've stuck with this problem. you told that
      " You need to move the declaration of TArg on the first apparition of type "

      So, where would i declare of it.

      Thx very much.
      Jordi

      Delete
  17. Hi Jordi,

    This procedure is working if a issue a 'cmd /c dir /s c:\*'

    But i don't have success using it to display a 7za.exe (7-zip command line) output. The pipe doesn't not return line by line progress.


    I mind it is related to 7za.exe but can't find what I can do.

    Any ideas?

    Thanks.

    João

    ReplyDelete
    Replies
    1. Hi João,

      That's because 7za.exe must display the internal results differently and not in the pipe.

      Regards,
      Jordi

      Delete
    2. Hi Jordi,

      Could you please try following command:

      7z.exe x -y "C:\Linux.iso" -o"E:\"

      7z.exe is changing the last line of the console output like %1, %2, %3 and so.

      But the %1, %2 lines does not come to the pipe.

      Is there any way to capture the %1, %2 output?

      I am looking for a solution for days.

      Thank you.

      Delete
    3. Hi Jordi,

      Could you please try the following command with Windows Dos Command Prompt and with the Delphi CaptureConsoleOutput procedure:

      7z.exe x -y "C:\Linux.iso" -o"E:\"

      7z.exe is changing the last line of the output with %1, %2, %3, ... and so. It is not adding new lines. It seems changing the last line.

      We can't capture the %1, %2, %3 output with pipe (CaptureConsoleOutput procedure).

      Is there any way to capture %1, %2 output?

      I am looing for a solution for days. How can I capture it?

      Thank you.

      Delete
  18. A better solution is to close your handle to the write end of the pipe (hWrite) after calling CreateProcess. That way, ReadFile() will exit with ERROR_BROKEN_PIPE when the child process exits, and you don't have to poll.

    ReplyDelete
  19. This a best solution!
    Can you help me about words with accent or special character return? Because in my return like a SVN command the files with word accented return String is broken.

    ReplyDelete
  20. Hi Jordi

    I have a console application that uses Windows' SetConsoleCursorPosition to output text at some particular position. For example it writes on the console: 'time step is 0.04 sec'. The user then sees the 0.04 updating each time step, on the same line in the console window. Otherwise the window would have thousands of lines.

    Now when I use the CaptureConsoleOutput function the memo grows to thousands of lines, because the call back function adds lines to the memo.

    Do you think it is possible to change this so that the output is similar to the original console ouput, i.e. that it updates certain positions in the memo instead of adding lines?

    regards,
    Jan

    ReplyDelete
    Replies
    1. Hi Jan,

      You'll have to play with it. Probably you could buffer it previously and then render the results later on.

      Regards,
      Jordi

      Delete
  21. This works in Delphi XE7, but NOT IN REAL-TIME. The output is written to the memo only when the console program terminates. So how can I make it work in real-time?

    ReplyDelete
    Replies
    1. Hi Peter,

      The pipe is opened and awaits the finalisation of the process. That's why WaitForSingleObject is used for so then the console output can be captured. To do what you are expecting, you would have to look for an alternative way.

      Regards,
      Jordi

      Delete
  22. Hi,

    many thanks for this wonderful help. Is it possible that you write a version that does the output only after executing not while executing? All examples I found do not work, maybe you can help?

    ReplyDelete
  23. Thank Jordi. This is the best and perfect for me exactly. I am always using it when need to capture the console output in real time.

    One most important question for me, how can i ABORT or CANCEL a process which take a long process?

    An ilustration, i put a TMemo, TEdit and TButton.
    i'am running FFMPEG command line to convert a big size video file into other format. for example: ffmpeg -i video.mp4 -vn -ab 256 audio.mp3

    This process could sometime need more than 30 minutes. SO i want to abort it before it's finished without closing application. How to do this?

    while in native CMD window i just can do it with a simple task by sending CTRL+C keystroke to the console. Please help.

    Btw, i am using it in a simple thread execute like this:

    TConvert = class(TThread)
    protected
    procedure Execute; Override
    end;

    No matter it's running in thread or not, i just need to be able to Cancel it when needed.
    Again, please help. thank you

    ReplyDelete
    Replies
    1. Hi There,

      Probably you'll have to break the look. I think that if you look in the comments you might find the solution. I do remember this:

      "Just add a boolean variable on the repeat to exit it:
      until (dRunning <> WAIT_TIMEOUT) or breakLoop;" So have a look as to see if this makes sense to you.

      I know it was a bit tricky to stop the pipe but I think it's doable.

      Cheers,
      Jordi

      Delete
    2. Thank you Jordi. I'll try to dig it more

      Delete
    3. This comment has been removed by the author.

      Delete
  24. Hi! I have some issues with this. It breaks some of the sentences, as well as it being printed in "chunks" rather than one line at the time.
    I have checked manually directly from the cmd, and there it does not have the same print.

    I have tried to change the CReadBuffer, and it seems to change the behaviour, but still not able to make it proper.

    Can you give some advice?

    ReplyDelete
    Replies
    1. I have been working on this for days now... no one?
      It is important that my delphi app shows exactly the same as the cmd window. I've made sure there is nothing in my code making it to break the line(sentance). It's clear that it breaks when the buffer is filled. This process only prints after certain buffer is reached, so you dont need to wait for the whole process is finished. Thats good if you only need it printing and for people to read and understand it. However, if your program are using the output for errorhandling etc, it will not work well.

      Delete
    2. Late to the party but try changing
      Memo1.Lines.Add(String(Line));
      to
      Memo1.SelText := String(Line);

      Delete
  25. Hi, Jordi.
    I need to use that Procedure on my Delphi 7, is that possible ?
    please help. Thank You in Advance

    ReplyDelete

Post a Comment

Popular Posts