Best way to allocate large memory

[AlanCCC]

In my Visual C++ app, I know the total objects(CMyObject) to be allocated is 16728064 and each object is 64 byte, so the total memory to be allocated is 1GB. The memory will be allocated in the beginning, used in the whole lifetime of the app, and release in the end.

In such a case, what is the best way to allocate the memory?

Current I try to allocate the memory at the beginning, as follows:

CMyObject *p = new CMyObject[16728064];

// Perform tasks.

delete [] p;

But the allocation will fail for most of the time. Now I want to do as follows:

CMyObject *p[10];

p[0] = new CMyObject[1672806];

p[1] = new CMyObject[1672806];

…

// Perform tasks

Delete [] p[0];

….

This seems to work for some time.

Therefore, should I split the allocation into pieces as small as possible? Or are there any good solutions for such a situation?

Thanks

[razzle]

What is it that doesn't work?

The CMyObject pointer array is 16728064 times 4 byte which is like 70 MByte. It shouldn't be a problem to allocate that.

Is it when you start allocating the individual CMyObjects you run out of memory eventually? 1 GByte is quite a lot.

Since the data structure lives throughout the whole of the program and the size is known beforehand you can as well allocate it in global static memory (outside any function), like

CMyObject objects[16728064];

[OReubens]

Note that if you allocate something on the heap using new and it stays for the whole of the program you don't need to delete it at the end. The systems software will remove it when the program terminates.

Sorry, but that's horrible advice. (even just on it's own).
cleanup what you create. period. There is no good excuse for anything else.


It could even be disastrous depending on what the object does because while the memory may be cleaned up, the destructor code won't be called. This could leave all kinds of resources not being reclaimed.

[razzle]

Sorry, but that's horrible advice. (even just on it's own).
cleanup what you create. period. There is no good excuse for anything else.

Do you clean up before smart pointers do their job?

Probably not and in the same way there's no particular reason to clean up before the system does its job when a program terminates.

But you must know the rules of course and maybe one cannot expect that from most programmers. So since this is no big deal for me I clean up my post.

[OReubens]

Do you clean up before smart pointers do their job?

Probably not and in the same way there's no particular reason to clean up before the system does its job when a program terminates.

But you must know the rules of course and maybe one cannot expect that from most programmers. So since this is no big deal for me I clean up my post.

smart pointers and other forms of RAII are proper cleanup. The destructor is called and memory is properly cleaned up by the application.

Your 'advice' was to not bother at all and let the OS take care of cleanup. That's the bad advice, because while the OS will indeed properly clean up heap memory. It won't clean up ALL kinds of allocated memory and won't clean up all kinds of other resources. Even some resources managed by the OS aren't cleaned up. There's still a handfull of global resources that are associated with the windowstation or the session rather than the process. ANd there are resources that aren't OS managed (like database connections).

The advice should Always be "clean up yourself", be it explicitely or through RAII... both are acceptable. There are unfortunately a few cases where cleanup is out of your control (badly written 3rd party libs) and in that case it's good to know what will and won't be take care of by the OS when your app terminates, but that still doesn't make it 'right'.

[razzle]

smart pointers and other forms of RAII are proper cleanup.

And in my view the system cleanup after a program terminates is also proper. But okay, the system cleanup is a special one off situation and I realize it may be controversial and considered not politically correct to utilize it to your advantage. That's why I also removed this advice.

I never advice anyone not to bother. For example smart pointers won't cleanup properly if they're used circularly. You always need to bother.

[OReubens]

A typical application on Win32 has at best 2Gb at it's disposal.
In practice you have considerably less than that because part of that 2Gb of memory is being used for various other stuff that gets loaded into your executables process space (OS dll's, drivers, hooks, your exe's code, the stack, the environment, etc...

To make matters worse, all of the above isn't neatly packed into a single contiguous block of memory but instead is spread out all over the 2Gb, so your 2Gb of memory is fragmented.

When you want to allocate 1Gb as a single contiguous array, then that can only happen if there is a contiguous block of memory available in the 2Gb of process memory. Chances for that happening aren't high, so it'll fail.

the moral: On Win32 you shouldn't try to allocate more than a few hundred Mb (100 or 200Mb should be ok most of the time) of memory in a single chunk.

---

Now if you allocate a pointer array. 16m x 4bytes, you get 70Mb plus some allocation overhead (fine so far)
but when you then allocate 16m small objects, each of those objects will have a "small amount" of allocation overhead because windows stores extra stuff of it's own alongside each bit of memory you request. Iirc this is 32bytes or so.
so allocating the 16mil 64bytes object will end up using 1.5Gb of memory, and then you're entering the realm where you are exhausting your total process memory.

That and the fact that 16mil allocates will take some noticable amount of time.

Again not good.

The moral: don't allocate millions of (very) tiny objects.

---


So the solution:
Either compile for Win64, which will remove the problem that you can't allocate 1Gb in a contiguous chunk.
Or stay with Win32, and develop your own object pooling (combine several hundred/Thousand objects into a single allocation) to reduce the overhead per object, and keep track of the 'pages' yourself. This could be fairly easy or fairly complex depending on if you need to reallocate or have an ability to add/remove objects.

in the simplest way possible:

const int objectpagesize = 1024; // or whatever reasonable number you want

class CMyObjectPage
{
public:
   CMyObjectPage() { pItems = new CMyObject[objectpagesize ]; } 
   ~CMyObjectPage() { delete [] pItems; }
   CMyObject* pItems; // or whatever 'reasonable' number you want
};

CMyObjectPage aPages[16*1024];

// to access item at index: 
aPages[index / objectpagesize].pItems[index % objectpagesize ].DoSomething();

you can add operators to not need the .pItems in the access.
you can wrap a class around the array of pages so you can hide the internals from user code
i kept this as simple as possible.


So yes, you seemed to be on the right track...

Therefore, should I split the allocation into pieces as small as possible? Or are there any good solutions for such a situation?

no, quite the countrary, split the allocation into pieces as LARGE as you can afford.
fewer allocations means fewer low level heap management for the OS, and fewer overhead.
but the larger allocations may fail because there isn't a contiguous chunk of memory left available.
Pages of a few MB n size each would be my personal advice. But your mileage may vary on that.

[OReubens]

1) It's bad advice to suggest delete isn't needed. especially to novice users which might take that as a carte blanche to apply it liberally.
2) as I said, yes, windows will clean up the memory.
but other cleanup the class destructor does will not be called, and that may cause all kinds of undesired effects. yes, windows will clean up a lot of other resources such as window handles, brushes, dc's.... too, but not everything.

[razzle]

1)
It's bad advice to suggest delete isn't needed.

Yes that's your view and I've acknowledged that.

I've also explained why I don't agree but why I still did removed my advice.

So it's back to general information: The system will reclaim any heap memory you've "forgotten" to delete. There will be no system wide leaks because of that.

[OReubens]

The system will reclaim any heap memory you've "forgotten" to delete. There will be no system wide leaks because of that.

And that is my precise point...
you may leak resources other than memory, because the OS does not clean up EVERYTHING when your exe terminates. delete does more than just clean up memory, it calls destructors too.

[razzle]

And that is my precise point...
you may leak resources other than memory, because the OS does not clean up EVERYTHING when your exe terminates. delete does more than just clean up memory, it calls destructors too.

Program termination is a complex situation.

I've removed my advice.

I think your idea that RAII will handle it perfectly well is naive.

If you want to discuss this further start a new thread.

[OReubens]

the principle of RAII is that your destructors get called, so yes "it will handle it perfectly".

but there a couple special cases like the circular reference you mentioned where some "smart objects" won't work on their own without a nudge.
If you do get yourself into that case, I would call that a bug in the program and not the fact that RAII failed. (should not have used strong pointers)
RAII is a tool like so many others, it's not the holy grail that will solve ALL your problems

[razzle]

If you do get yourself into that case, I would call that a bug in the program and not the fact that RAII failed.

"If you get yourself into that case"

Well you see, it's not that simple.

In my view you can use the program termination process to your favour.

You think one shouldn't.

That's what we disagree about.

I've withdrawn my initial statement several times now.

I've agreed that as a first approximation you shouldn't rely on program termination.

Can we please leave this now.