memory allocation failure

[AlonHes]

Hi all,
I'm writing a project which need a large amount of memory allocation.
The code I use to allocate memory is:

void* pMem = new char[1000*1024*1024]

The memory allocation fail and the return value of new is NULL.
The maximum size of memory that I managed to allocate is 700MB.
The computer I working on has 2GB memory.
I also wrote a small program to check the problem and the same allocation succeeded.

Do you have any idea,suggestion why this is happening?
Thanks for your help

Alon

[Mr. Tomaszek]

I also wrote a small program to check the problem and the same allocation succeeded.

So, does allocation succeded or not?

[AlonHes]

It failed in the project but succeeded in a small program
( the main difference between them is that the program is a simple win32 console application and the project is a com server)

Alon

[Mr. Tomaszek]

You can cath exception that is thrown when new operator fails. For details see http://www.codeguru.com/forum/showth...5&pagenumber=1

[sephiroth2m]

Regardless of how many physical memory u have, ur process has about 2GB (on NT/2K/XP) of virtual address space. A memory request first finds if there's any contiguous block of the requested size there, if not, it'll fail. Virtual address spaces of GUI apps are ussually fragmentary by things such as system DLLs. While virtual address spaces of console apps are "cleaner" 'cos there're very few DLLs. If u want to allocate large memory, I think u should use "Large Memory Support" here, or try to reserve (or alloc) it as soon as possible (just before the COM Initializing, for ex.).

[NoHero]

And what about GlobalAlloc/GlobalLock ... or malloc/free ... Can they manage your big memory allocation???

ng

[Ness]

VirtualAlloc is perfect for large memory usage.

[AlonHes]

No They aren't. I tried all the other allocation function like malloc and globalAlloc and they all failed

[AlonHes]

Do you have an example how to use virtualAlloc?

[sephiroth2m]

Hi AlonHes, memory allocated functions are the same, VirtualAlloc just gives you some more low level options.
This code below outlines the layout of a virtual address space (0 - 0x7FFFFFFF), It also tells you the maximum size of a free block in the virtual address space, if you could not allocate that size, there'd be some other problem.
If you use it for both of your project and your console app, you'll find their address space layouts are different, that's why you could allocate larger of memory in console app.

#include <fstream>

void VirtualLayout()
{	std:: ofstream of;
	unsigned int uiQueriedAddress= 0, i= 0, nMaxBlock= 0, iMaxBlock= 0;
	MEMORY_BASIC_INFORMATION mbi;

	of.open("C:\\virtwalk.txt");
	of<<" -= Virtual address space's layout =-\n\n";

	while( uiQueriedAddress<0x7FFFFFFF )
	{
		of<<"Block "<<++i<<".__________________________________"<<std::endl;
		OutputMemQuery(of, (const void * far)uiQueriedAddress, mbi);
		uiQueriedAddress+= mbi.RegionSize;

		if(mbi.RegionSize > nMaxBlock)
		{	nMaxBlock= mbi.RegionSize;
			iMaxBlock= i;
		}
	}

	of<<"______________________________________"<<std::endl;
	of<<"Total block: "<<i<<std::endl;
	of<<"Maximum size of a free block: "<<nMaxBlock/1024<<"KB, ";
	of<<"at block: "<<iMaxBlock<<std::endl;
	of.close();
}

unsigned int OutputMemQuery(std:: ostream &os, const void * far uiQueriedAddress,
	MEMORY_BASIC_INFORMATION &mbi)
{	char szModuleName[256];
	
	VirtualQuery(uiQueriedAddress, &mbi, sizeof(MEMORY_BASIC_INFORMATION));
	if( (unsigned int)uiQueriedAddress>= 0x7FFFFFFF )
		return 0;
	
	os<<"Queried Address: \t"<<(unsigned int)uiQueriedAddress/1024<<"K"<<std::endl;
	os<<"BaseAddress: \t"<<(unsigned int)mbi.BaseAddress/1024<<"K"<<std::endl;
	os<<"AllocationBase: \t"<<(unsigned int)mbi.AllocationBase/1024<<"K"<<std::endl;
	os<<"RegionSize: \t"<<mbi.RegionSize/1024<<"K"<<std::endl;
	os<<"AllocationProtect: \t";
	switch(mbi.AllocationProtect)
	{	case PAGE_EXECUTE:
			os<<"PAGE_EXECUTE"<<std::endl;
			break;
		case PAGE_EXECUTE_READ:
			os<<"PAGE_EXECUTE_READ"<<std::endl;
			break;
		case PAGE_EXECUTE_READWRITE:
			os<<"PAGE_EXECUTE_READWRITE"<<std::endl;
			break;
		case PAGE_EXECUTE_WRITECOPY:
			os<<"PAGE_EXECUTE_WRITECOPY"<<std::endl;
			break;
		case PAGE_NOACCESS:
			os<<"PAGE_NOACCESS"<<std::endl;
			break;
		case PAGE_READONLY:
			os<<"PAGE_READONLY"<<std::endl;
			break;
		case PAGE_READWRITE:
			os<<"PAGE_READWRITE"<<std::endl;
			break;
		case PAGE_WRITECOPY:
			os<<"PAGE_WRITECOPY"<<std::endl;
			break;
		default:
			os<<"unknown"<<std::endl;
			break;
	}
	os<<"State: \t\t";
	switch(mbi.State)
	{	case MEM_COMMIT:
			os<<"MEM_COMMIT"<<std::endl;
			break;
		case MEM_FREE:
			os<<"MEM_FREE"<<std::endl;
			break;
		case MEM_RESERVE:
			os<<"MEM_RESERVE"<<std::endl;
			break;
		default:
			os<<"unknown"<<std::endl;
			break;
	}
	os<<"Protect: \t\t";
	switch(mbi.Protect)
	{	case PAGE_EXECUTE:
			os<<"PAGE_EXECUTE"<<std::endl;
			break;
		case PAGE_EXECUTE_READ:
			os<<"PAGE_EXECUTE_READ"<<std::endl;
			break;
		case PAGE_EXECUTE_READWRITE:
			os<<"PAGE_EXECUTE_READWRITE"<<std::endl;
			break;
		case PAGE_EXECUTE_WRITECOPY:
			os<<"PAGE_EXECUTE_WRITECOPY"<<std::endl;
			break;
		case PAGE_NOACCESS:
			os<<"PAGE_NOACCESS"<<std::endl;
			break;
		case PAGE_READONLY:
			os<<"PAGE_READONLY"<<std::endl;
			break;
		case PAGE_READWRITE:
			os<<"PAGE_READWRITE"<<std::endl;
			break;
		case PAGE_WRITECOPY:
			os<<"PAGE_WRITECOPY"<<std::endl;
			break;
		default:
			os<<"unknown"<<std::endl;
			break;
	}
	os<<"Type: \t\t";
	switch(mbi.Type)
	{	case MEM_IMAGE:
			GetModuleFileName((HMODULE)mbi.AllocationBase,szModuleName,256);
			os<<"MEM_IMAGE: "<<szModuleName<<std::endl;
			break;
		case MEM_MAPPED:
			os<<"MEM_MAPPED"<<std::endl;
			break;
		case MEM_PRIVATE:
			os<<"MEM_PRIVATE"<<std::endl;
			break;
		default:
			os<<"unknown"<<std::endl;
			break;
	}
	os<<std::endl;

	return (unsigned int)mbi.AllocationBase;
}

[pengch]

memory allocated must be continuation.
If you alloc memory at a small APP, it only used 0x00400000, 0x77000000 and so on. So that you maybe alloc memory the continuation a big memory.

But when you alloc memory in another big APP, it maybe used
0x00400000, 0x10200000(MSVCRTD.DLL), 0x5F000000(MFC42D.dll),
0x77000000(ntdll.dll). now, you only can alloc the continuation memory at 0x10300000 to 0x50000000. If you have a lot of DLL, the max continuation memory maybe becomed very small.