Problem with AllocConsole and child processes Windows 7

I have a GUI window that allocates a console using AllocConsole and then executes a child console process which should write to the console window I allocated. When I execute the program from the Start/Run menu or from a command prompt window,it works as expected. If I execute the program via the Windows Explorer or a desktop shortcut, the console window appears but nothing is written to it. I am using Windows 7 64bit. This problem did not occur under Windows XP.

I am setting the bInheritHandles to TRUE, setting STARTF_USESTDHANDLES flag in the startupifno dwFlags, and am setting hStdOutput to GetStdHandle (STD_OUTPUT_HANDLE).

It would be good to see your code that replicates the problem, compilable and runnable.

Quote:

---

Originally Posted by Igor Vartanov

It would be good to see your code that replicates the problem, compilable and runnable.

---

Hope this is enough for you.

int execute_command(char *command)
{
HANDLE outp;
COORD coord;
STARTUPINFO si;
PROCESS_INFORMATION pi;
DWORD exitcode;
BOOL ret;

if(AllocConsole())
{
outp = GetStdHandle(STD_OUTPUT_HANDLE);
coord.X = 80;
coord.Y = 500;
SetConsoleScreenBufferSize(outp,coord);
SetConsoleTitle("Text Output");
}

GetStartupInfo(&si);
memset(&pi,0,sizeof(PROCESS_INFORMATION));
si.hStdInput = GetStdHandle (STD_INPUT_HANDLE);
si.hStdOutput = GetStdHandle (STD_OUTPUT_HANDLE);
si.hStdError = GetStdHandle (STD_ERROR_HANDLE);
si.dwFlags |= STARTF_USESTDHANDLES;

ret = CreateProcess(NULL,command,NULL,NULL,TRUE,0,NULL,NULL,&si,&pi);
if (ret != TRUE)
return(-1);

exitcode = STILL_ACTIVE;
if (pi.hProcess)
{
MSG msg;
ret = WAIT_TIMEOUT;
while (ret == WAIT_TIMEOUT || exitcode == STILL_ACTIVE)
{
ret = WaitForSingleObject(pi.hProcess,10);
if (!GetExitCodeProcess(pi.hProcess,&exitcode))
MessageBox(NULL, "GetExitCodeProcess Failure", NULL, MB_OK);
if (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
DispatchMessage(&msg);
}
}
if (pi.hProcess)
CloseHandle(pi.hProcess);
if (pi.hThread)
CloseHandle(pi.hThread);

return(exitcode);
}

This worked fine in my tests, Win XP and 7 64-bit:

Code:

---

int execute_command(char *command)
{
        HANDLE outp;
        COORD coord;
        STARTUPINFO si = {sizeof(si)};
        PROCESS_INFORMATION pi = {0};
        DWORD exitcode;
        BOOL ret;

        if(AllocConsole())
        {
                outp = GetStdHandle(STD_OUTPUT_HANDLE);
                coord.X = 80;
                coord.Y = 500;
                SetConsoleScreenBufferSize(outp,coord);
                SetConsoleTitle("Text Output");
        }

        ret = CreateProcess(NULL,command,NULL,NULL,TRUE,0,NULL,NULL,&si,&pi);
        if (ret != TRUE)
                return(-1);

        exitcode = STILL_ACTIVE;
        if (pi.hProcess)
        {
                MSG msg;
                ret = WAIT_TIMEOUT;
                while (ret == WAIT_TIMEOUT || exitcode == STILL_ACTIVE)
                {
                        ret = WaitForSingleObject(pi.hProcess,10);
                        if (!GetExitCodeProcess(pi.hProcess,&exitcode))
                                MessageBox(NULL, "GetExitCodeProcess Failure", NULL, MB_OK);
                        if (PeekMessage(&msg,NULL,0,0,PM_REMOVE))
                                DispatchMessage(&msg);
                }
        }
        if (pi.hProcess)
                CloseHandle(pi.hProcess);
        if (pi.hThread)
                CloseHandle(pi.hThread);

        return(exitcode);
}

---

Did your child process write to stdout and the output was shown in the console window?
Did you try to run your program by double-clicking on the executable file in the windows explorer or running it from a desktop shortcut that has as its target the executable file?

Quote:

---

Originally Posted by Rich C

Did your child process write to stdout and the output was shown in the console window?
Did you try to run your program by double-clicking on the executable file in the windows explorer or running it from a desktop shortcut that has as its target the executable file?

---

Rich, why don't you just try my code out yourself? :)

Thanks Igor. Your code worked. When I first saw the code, I thought you were just repeating my code.

The difference is in STARTUPINFO. You essentially zeroed it out (except for the size); I used GetStartupInfo and then set some fields in it. Looking at my code, do you know what fields I set incorrectly?

Thanks again for the help.

I'm pretty sure that there's something with proper handles duplication/security/inheritance, as your code gave me an 'Invalid handle' message been launched from explorer. (Sorry, I don't remember the details anymore, as I delved into this too many years ago. When I have some time for this, I'll take a more intent look.)

When you provide no explicit startup info, Windows itself does that for you, and of course it does that proper way.

What's interesting to me is that this worked in Windows XP and prior versions of Windows.

Thanks again,
Rich

Quote:

---

Originally Posted by Rich C

What's interesting to me is that this worked in Windows XP and prior versions of Windows.

---

There are a lot of code examples that work with older OSes but will not work or work erratically as soon as you go to a later operating system. Most of this due to programmer fault, and the other OSes just happen to not exercise the bad code. Other times it is documented that the OS handles things differently, but the programmer didn't do their research thoroughly.

Regards,

Paul McKenzie

I have a feeling its the misuse of the GetStartupInfo function when spawing child processes.

Rich

Quote:

---

Originally Posted by Rich C

I have a feeling its the misuse of the GetStartupInfo function when spawing child processes.

Rich

---

If you read the documentation for the STARTUPINFO struct, you see the first member has to have the number of bytes set (cb). Your example didn't do this (next time, use code tags when posting code), but Igor's does.

All the code I've ever written that uses STARTUPINFO has set the initial member to the number of bytes that the struct takes up. This goes back to Windows NT (maybe before that). So in reality, you were getting away with this mistake until now.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Paul McKenzie

If you read the documentation for the STARTUPINFO struct, you see the first member has to have the number of bytes set (cb). Your example didn't do this (next time, use code tags when posting code), but Igor's does.

Regards,

Paul McKenzie

---

I assumed when I called GetStartupInfo to fill in the STARTUPINFO structure, the first member of the STARTUPINFO structure would have been set

Rich

Quote:

---

Originally Posted by Rich C

I assumed when I called GetStartupInfo to fill in the STARTUPINFO structure, the first member of the STARTUPINFO structure would have been set

Rich

---

No. The Windows API probably checks this number and depending on what it is, does certain logic.

Also, how could Windows set this to a correct number, when all you're doing is passing a pointer? You can't get the correct sizeof() a type by using a pointer. That's why your code has to set it, and then the API call picks up on this number and does whatever it does.

For example, the Windows structure changes the number of members in a future version of the SDK. The Windows API function hopefully keeps track of this change, and looks at the cb member to figure out what to do with this version of that struct (is it an old version, new version, etc.). Another possible scenario is that the API call knows what the cb number should be, and checks if your version is the right one by looking at the cb member, who knows.

All I am certain of is that any SDK struct that has a member where the number of bytes is set, you need to set it to ensure that the function doesn't behave erratically. That's been my experience, even if the API docs don't say so explicitly.

Regards,

Paul McKenzie

Quote:

---

Originally Posted by Paul McKenzie

No. The Windows API probably checks this number and depending on what it is, does certain logic.

Also, how could Windows set this, when all you're doing is passing a pointer? You can't get the correct sizeof() a type by using a pointer. That's why your code has to set it, and then the API call picks up on this number and does whatever it does.

Regards,

Paul McKenzie

---

Thanks Paul. Good catch.

Rich