[RESOLVED] CreateThread only creating 27 threads max!!!

[AndrewCapon]

Hi Everyone,

We are porting a multi-threaded unix server application over to a win32 console app. The code over 1000000 lines long and produces a debug exe of about 20MB and a release exe of about 8MB.

We have a problem in that we can only create around 27 threads, this is not enough! Both the release and debug builds act the same!

As a test we used the following code called from the first line of main.

void AndysTestCode::CountThreads(void)
{
DWORD dwThreadId, dwThrdParam = 1;
HANDLE hThread;
char szMsg[80];
unsigned long uCount = 0;
unsigned long uFailed = 0;

for(unsigned int u=0; u< 50000; u++)
{
printf("Creating windows thread %u -", u);
hThread = CreateThread(
NULL, // default security attributes
128, // use default stack size
runWindowsThread, // thread function
&dwThrdParam, // argument to thread function
0, // use default creation flags
&dwThreadId); // returns the thread identifier

if (hThread == NULL)
{
printf("failed.\n" );
syserror();
}
else
printf("success.\n" );
}

LastError() when failed gives 8.

As you can see we are useing a tiny 128 stack for the test. If we change this to 1MB we get the same problem so to me it doesnt look like a stackspace issue.

If we set up a new console project the same code gives us > 2000 threads. We have set this project up to include the same libs as our main code which is basically the Oracle 9 runtime libs.

Has anyone else seen anything like this?

Any help would be really apreciated.

Cheers

Andy

[NMTop40]

Try __beginthread or __beginthreadex instead of CreateThread() ? I don't know if that would help at all.

50,000 far too many threads though. I doubt you'll ever get Windows to create that many. I think the most allowed is 64 (well that's the maximum in WaitForMultipleObjects).

[stober]

I agree with NMTop40 -- Windows will grind to a halt if you attempt to create anything close to that many threads. And its my experience that *nix will too. So I don't know how your server ran on that *nix computer with that many threads.

using _beginthread() instead of CreateThread() will not help because _beginthread() eventually calls CreateThread().

Here is what MSDN has to say about it.

The number of threads a process can create is limited by the available virtual memory. By default, every thread has one megabyte of stack space. Therefore, you can create at most 2028 threads. If you reduce the default stack size, you can create more threads. However, your application will have better performance if you create one thread per processor and build queues of requests for which the application maintains the context information. A thread would process all requests in a queue before processing requests in the next queue.

[AndrewCapon]

Hi there,

Thanks for you answers.

We are not really trying to create 50000 threads it is just some test code!

We do need more than 27 threads though!

__BeginThreadEx has been tried exactly the same result.


Let me state out problem again.

In our ported app when we run the test code above we only get 27 threads. If we set up another project with just the test code we get 2028 threads.

Now this limit of 27 is nothing to do with available virtual memory, it looks like something else.

If we run the test code (in a clean project with just that code) with a stack of 1MB we see the memory usage for the process go up till there is no more virtual memory available and then we cannot create any more threads, I am happy with this.

If we run the test code in our main project (the one we are porting) we only get 27 threads even though we have loads of virtual memory left. I am not happy with this!

I cannot believe it is something to do with the process size is it?

Cheers

Andy

[Mick]

Post the code for runWindowsThread(...). You are stepping on memory would be my logical guess. There is nothing that would prevent you from creating > 27 threads (sans resources) in the code you posted.

Memory overwrites or subtle bugs generally expose themselves when you move between compliers/OS's...

[Hobson]

Apache server creates 250 worker threads by default and can create much more when configuration file is edited. And all this is done on normal desktop machine running WinXP..

Hob

[AndrewCapon]

DWORD WINAPI runWindowsThread(LPVOID lpParam )
{
while(true)
Sleep(100000);
return 0;
}

Basically as a test to keep the thread alive just sleep.

[wildfrog]

I think the most allowed is 64 (well that's the maximum in WaitForMultipleObjects).

There is a 64k limit of handles a process can own (ie. the sum of threads, mutexes, semaphores etc cannot exceed 64k).

Although WaitForMultipleObject(...) has a maximum of 64, there are ways to owercome that. One way is to have several threads running WaitForMutipleObject(...) and then wait on those threads.

- petter

[AndrewCapon]

Please help, going mad.........

[cilu]

Use GetProcessHandleCount() to check how many handles your process owns before starting to create those threads.

[SuperKoko]

Check the stack reserved size of your project (it is defined in the STACKSIZE statement of the .DEF file).

The reserved size of the .exe file is used, not only for the main thread, but also for all threads created with CreateThread. The stack size specified as CreateThread parameter is the commited stack size.
You should set STACKSIZE to 0x100000 to get 1MB reserved stack.

Verify also that there are no linker command line parameter overriding this reserved stack size.

I hope that it is the solution!

[AndrewCapon]

Check the stack reserved size of your project (it is defined in the STACKSIZE statement of the .DEF file).

The reserved size of the .exe file is used, not only for the main thread, but also for all threads created with CreateThread. The stack size specified as CreateThread parameter is the commited stack size.
You should set STACKSIZE to 0x100000 to get 1MB reserved stack.

Verify also that there are no linker command line parameter overriding this reserved stack size.

I hope that it is the solution!

You absolute star this was the problem, we had a very large stack size defined here. I Changed it, did a full link and the problem has gone away.

Your name is now golden in our development group and we will build a statue of you to worship.

Once again thankyou and everyone else who has tried to help.

Cheers

Andy