Problem with AllocConsole and child processes Windows 7

I have a GUI window that allocates a console using AllocConsole and then executes a child console process which should write to the console window I allocated. When I execute the program from the Start/Run menu or from a command prompt window,it works as expected. If I execute the program via the Windows Explorer or a desktop shortcut, the console window appears but nothing is written to it. I am using Windows 7 64bit. This problem did not occur under Windows XP.

I am setting the bInheritHandles to TRUE, setting STARTF_USESTDHANDLES flag in the startupifno dwFlags, and am setting hStdOutput to GetStdHandle (STD_OUTPUT_HANDLE).

It would be good to see your code that replicates the problem, compilable and runnable.

Quote:

---

Originally Posted by Igor Vartanov

It would be good to see your code that replicates the problem, compilable and runnable.

---

Hope this is enough for you.

int execute_command(char *command)
{
HANDLE outp;
COORD coord;
STARTUPINFO si;
PROCESS_INFORMATION pi;
DWORD exitcode;
BOOL ret;

if(AllocConsole())
{
outp = GetStdHandle(STD_OUTPUT_HANDLE);
coord.X = 80;
coord.Y = 500;
SetConsoleScreenBufferSize(outp,coord);
SetConsoleTitle("Text Output");
}

GetStartupInfo(&si);
memset(&pi,0,sizeof(PROCESS_INFORMATION));
si.hStdInput = GetStdHandle (STD_INPUT_HANDLE);
si.hStdOutput = GetStdHandle (STD_OUTPUT_HANDLE);
si.hStdError = GetStdHandle (STD_ERROR_HANDLE);
si.dwFlags |= STARTF_USESTDHANDLES;

ret = CreateProcess(NULL,command,NULL,NULL,TRUE,0,NULL,NULL,&si,&pi);
if (ret != TRUE)
return(-1);

exitcode = STILL_ACTIVE;
if (pi.hProcess)
{
MSG msg;
ret = WAIT_TIMEOUT;
while (ret == WAIT_TIMEOUT || exitcode == STILL_ACTIVE)
{
ret = WaitForSingleObject(pi.hProcess,10);
if (!GetExitCodeProcess(pi.hProcess,&exitcode))
MessageBox(NULL, "GetExitCodeProcess Failure", NULL, MB_OK);
if (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
DispatchMessage(&msg);
}
}
if (pi.hProcess)
CloseHandle(pi.hProcess);
if (pi.hThread)
CloseHandle(pi.hThread);

return(exitcode);
}

This worked fine in my tests, Win XP and 7 64-bit:

Code:

---

int execute_command(char *command)
{
        HANDLE outp;
        COORD coord;
        STARTUPINFO si = {sizeof(si)};
        PROCESS_INFORMATION pi = {0};
        DWORD exitcode;
        BOOL ret;

        if(AllocConsole())
        {
                outp = GetStdHandle(STD_OUTPUT_HANDLE);
                coord.X = 80;
                coord.Y = 500;
                SetConsoleScreenBufferSize(outp,coord);
                SetConsoleTitle("Text Output");
        }

        ret = CreateProcess(NULL,command,NULL,NULL,TRUE,0,NULL,NULL,&si,&pi);
        if (ret != TRUE)
                return(-1);

        exitcode = STILL_ACTIVE;
        if (pi.hProcess)
        {
                MSG msg;
                ret = WAIT_TIMEOUT;
                while (ret == WAIT_TIMEOUT || exitcode == STILL_ACTIVE)
                {
                        ret = WaitForSingleObject(pi.hProcess,10);
                        if (!GetExitCodeProcess(pi.hProcess,&exitcode))
                                MessageBox(NULL, "GetExitCodeProcess Failure", NULL, MB_OK);
                        if (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
                                DispatchMessage(&msg);
                }
        }
        if (pi.hProcess)
                CloseHandle(pi.hProcess);
        if (pi.hThread)
                CloseHandle(pi.hThread);

        return(exitcode);
}

---

Did your child process write to stdout and the output was shown in the console window?
Did you try to run your program by double-clicking on the executable file in the windows explorer or running it from a desktop shortcut that has as its target the executable file?

Quote:

---

Originally Posted by Rich C

Did your child process write to stdout and the output was shown in the console window?
Did you try to run your program by double-clicking on the executable file in the windows explorer or running it from a desktop shortcut that has as its target the executable file?

---

Rich, why don't you just try my code out yourself? :)

Thanks Igor. Your code worked. When I first saw the code, I thought you were just repeating my code.

The difference is in STARTUPINFO. You essentially zeroed it out (except for the size); I used GetStartupInfo and then set some fields in it. Looking at my code, do you know what fields I set incorrectly?

Thanks again for the help.

I'm pretty sure that there's something with proper handles duplication/security/inheritance, as your code gave me an 'Invalid handle' message been launched from explorer. (Sorry, I don't remember the details anymore, as I delved into this too many years ago. When I have some time for this, I'll take a more intent look.)

When you provide no explicit startup info, Windows itself does that for you, and of course it does that proper way.

What's interesting to me is that this worked in Windows XP and prior versions of Windows.

Thanks again,
Rich

Quote:

---

Originally Posted by Rich C

What's interesting to me is that this worked in Windows XP and prior versions of Windows.

---

There are a lot of code examples that work with older OSes but will not work or work erratically as soon as you go to a later operating system. Most of this due to programmer fault, and the other OSes just happen to not exercise the bad code. Other times it is documented that the OS handles things differently, but the programmer didn't do their research thoroughly.

Regards,

Paul McKenzie

I have a feeling its the misuse of the GetStartupInfo function when spawing child processes.

Rich

Quote:

---

Originally Posted by Rich C

I have a feeling its the misuse of the GetStartupInfo function when spawing child processes.

Rich

---

If you read the documentation for the STARTUPINFO struct, you see the first member has to have the number of bytes set (cb). Your example didn't do this (next time, use code tags when posting code), but Igor's does.

All the code I've ever written that uses STARTUPINFO has set the initial member to the number of bytes that the struct takes up. This goes back to Windows NT (maybe before that). So in reality, you were getting away with this mistake until now.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Paul McKenzie

If you read the documentation for the STARTUPINFO struct, you see the first member has to have the number of bytes set (cb). Your example didn't do this (next time, use code tags when posting code), but Igor's does.

Regards,

Paul McKenzie

---

I assumed when I called GetStartupInfo to fill in the STARTUPINFO structure, the first member of the STARTUPINFO structure would have been set

Rich

Quote:

---

Originally Posted by Rich C

I assumed when I called GetStartupInfo to fill in the STARTUPINFO structure, the first member of the STARTUPINFO structure would have been set

Rich

---

No. The Windows API probably checks this number and depending on what it is, does certain logic.

Also, how could Windows set this to a correct number, when all you're doing is passing a pointer? You can't get the correct sizeof() a type by using a pointer. That's why your code has to set it, and then the API call picks up on this number and does whatever it does.

For example, the Windows structure changes the number of members in a future version of the SDK. The Windows API function hopefully keeps track of this change, and looks at the cb member to figure out what to do with this version of that struct (is it an old version, new version, etc.). Another possible scenario is that the API call knows what the cb number should be, and checks if your version is the right one by looking at the cb member, who knows.

All I am certain of is that any SDK struct that has a member where the number of bytes is set, you need to set it to ensure that the function doesn't behave erratically. That's been my experience, even if the API docs don't say so explicitly.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Paul McKenzie

No. The Windows API probably checks this number and depending on what it is, does certain logic.

Also, how could Windows set this, when all you're doing is passing a pointer? You can't get the correct sizeof() a type by using a pointer. That's why your code has to set it, and then the API call picks up on this number and does whatever it does.

Regards,

Paul McKenzie

---

Thanks Paul. Good catch.

Rich

Quote:

---

Originally Posted by Rich C

Thanks Paul. Good catch.

Rich

---

It's one of those things that has happened to me, albeit with a different struct (I think it was the OPENFILENAME struct). I always inspect the struct that Windows API wants to use, just to see if a "size of struct" member is available. I set it, even if in reality, it wouldn't / shouldn't make a difference.

Regards,

Paul McKenzie

Paul,

I did the following in the function and it did not work. Note: I set si.cb:
if(AllocConsole())
{
outp = GetStdHandle(STD_OUTPUT_HANDLE);
coord.X = 80;
coord.Y = 500;
SetConsoleScreenBufferSize(outp,coord);
SetConsoleTitle("Text Output");
}

si.cb = sizeof(si);
GetStartupInfo(&si);

ret = CreateProcess(NULL,command,NULL,NULL,FALSE,0,NULL,NULL,&si,&pi);
if (ret != TRUE)
return(-1);

Rich

Quote:

---

Originally Posted by Rich C

Paul,

I did the following in the function and it did not work.

---

Define what you mean by "did not work".

Second, why are you not calling GetLastError() if CreateProcess fails?

http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx

Quote:

---

Return value
If the function succeeds, the return value is nonzero.

If the function fails, the return value is zero. To get extended error information, call GetLastError.

---

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Paul McKenzie

Define what you mean by "did not work".

Second, why are you not calling GetLastError() if CreateProcess fails?

http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx

Regards,

Paul McKenzie

---

Sorry, I was assuming you had looked at my original post. This was simply taking Igor's code and using GetStartupInfo. The problem is that when executing the code from the windows explorer, the console window appears but nothing is written to it.
Here is the complete function. You must supply a driver program to execute a child process that writes to standard output.

int execute_command(char *command)
{
HANDLE outp;
COORD coord;
STARTUPINFO si = {sizeof(si)};
PROCESS_INFORMATION pi = {0};
DWORD exitcode;
BOOL ret;

if(AllocConsole())
{
outp = GetStdHandle(STD_OUTPUT_HANDLE);
coord.X = 80;
coord.Y = 500;
SetConsoleScreenBufferSize(outp,coord);
SetConsoleTitle("Text Output");
}
si.cb = sizeof(si);
GetStartupInfo(&si);

ret = CreateProcess(NULL,command,NULL,NULL,TRUE,0,NULL,NULL,&si,&pi);
if (ret != TRUE)
return(-1);

exitcode = STILL_ACTIVE;
if (pi.hProcess)
{
MSG msg;
ret = WAIT_TIMEOUT;
while (ret == WAIT_TIMEOUT || exitcode == STILL_ACTIVE)
{
ret = WaitForSingleObject(pi.hProcess,10);
if (!GetExitCodeProcess(pi.hProcess,&exitcode))
MessageBox(NULL, "GetExitCodeProcess Failure", NULL, MB_OK);
if (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
DispatchMessage(&msg);
}
}
if (pi.hProcess)
CloseHandle(pi.hProcess);
if (pi.hThread)
CloseHandle(pi.hThread);

return(exitcode);
}

Is the problem with CreateProcess? Is the console being created OK but CreateProcess is failing? When you use exeute_command in your program, do you test the return value for -1 and display an error message box or something as you don't show how you are calling execute_command? What commands are you passing to execute_command? What is the value of GetLastError() if the CreateProcess function fails?

For every command you want executed, you are creating and destroying a new process. This takes time. Why not just create a new process once with a console window and have it execute cmd to get a command prompt window?

Quote:

---

Originally Posted by Rich C

Code:

---

    if(AllocConsole())
        {
                outp = GetStdHandle(STD_OUTPUT_HANDLE);
                coord.X = 80;
                coord.Y = 500;
                SetConsoleScreenBufferSize(outp,coord);
                SetConsoleTitle("Text Output");
        }

    si.cb = sizeof(si);
    GetStartupInfo(&si);

    ret = CreateProcess(NULL,command,NULL,NULL,FALSE,0,NULL,NULL,&si,&pi);
    if (ret != TRUE)
        return(-1);

---

---

Please, use Code tags while posting code snippets!
And never compare BOOL values returned by Windows APIs with TRUE! TRUE is #defined as 1 while a lot of APIs may return any other non-zero value as a succesful result.
So the correct comparison should be something like

Code:

---

    if (!ret)

---

or

Code:

---

    if (ret == 0)

---

And as others mentioned you have to call GetLastError to obtain the reason of the failure!

Quote:

---

Originally Posted by Rich C

Sorry, I was assuming you had looked at my original post. This was simply taking Igor's code and using GetStartupInfo.

---

First, use code tags when posting code:

[code]
Your code goes here
[/code]

That is how you use code tags.

Second, you need to help us out more. All of those API functions you're assuming work OK may not be OK, as they return values. Those values could indicate failure, but your original function (and the one you took from Igor's example) doesn't check for these errors. You should rewrite the code so that you are checking all return codes, and if the code is failure, then call GetLastError() (if the documentation for the function states this). This way, you and others helping you have a better picture of what is happening. Not only that, this is the way you're supposed to write any software that could be run on any computer, whether it's yours or a customer's machine.

The bottom line is that you should never assume that Windows functions just "work". I remember having to rewrite a bunch of functions to check for the return code from a bunch of GDI functions -- without the check for the return code, the functions were writing totally black pages when printing, causing the customer to waste vast amounts of ink because of a failed function (that was never checked for failure)!

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Rich C

I assumed when I called GetStartupInfo to fill in the STARTUPINFO structure, the first member of the STARTUPINFO structure would have been set

---

In addition to what Igor and Paul stated, I would like to suggest NOT to pass uninitialized data (garbage) to any Win API function that might expect NULL or valid pointer for some fields.
It costs you next to nothing to set everything to 0 before filling in relevant data.