Memory not released when list is cleared?

[LoomisP]

I have a problem where my memory usage does not decrease when I clear a list. Here is a simple example:

int main()
{
    list<int> il;
    int k;

    for ( k=0; k != 50000000; k++)
          til.push_back(k)

    til.clear();

    cin >> k

    return 1;
}

The cin >> k statment is just to keep the program from ending so that I can see the memory usage at that point. I can see the memory usage increasing during the for loop, but it does not decresae at the til.clear() statement. The program is using the same amount of memory when it gets to cin >> k whether I clear til or not.

I have also tried putting a til.resize(0) after til.clear(). Am I missing something or is my compiler broken?

Thanks,
Loomis

[jfaust]

clear() does not free memory. The capacity of the list is unchanges. To really free memory, you need to swap.

#include <list>
#include <iostream>

using namespace std;

int main()
{
	list<int> il;
	int k;

	for ( k=0; k != 50000000; k++)
		il.push_back(k);

	//il.clear();
	il.swap(list<int>());

	cin >> k;

	return 1;
}

By the way, it would be considerate if you would post code that actually compiles.

Jeff

[LoomisP]

Sorry for omitting the headers, I thought it would be better to be concise, but I can see the point if someone wishes to copy/paste/compile.

But your code does not compile either. My compiler does not let me call swap in that way. The argument has to be a reference, but I don't think the constructor can return a reference.

This also seems like going around the barn the long way. If I clear the list, and resize it to zero, why is all that memory still allocated to it? What purpose does that have?

Thanks,
Loomis

[dcjr84]

When you .clear() a list, it does not deallocate the memory used by the list. It simply empties the "contents" of each list element. Even if you .resize() it to zero, the memory is not released.

I could be wrong on this, but I think the memory you allocate to a list is not deallocated until it goes out of scope, and the list object is destroyed.

The reason I think this is because of the concept of temporal location. When the compiler dynamically requests a certain block of memory for the list, it has to invoke the OS to do so. This is a costly operation, so dynamic memory requests should be minimized.

So, consider if you had a list of 10,000 elements. The compiler requests from the OS all of that memory. Then, for some unknown reason, you clear() the entire list. If all that memory was released back to OS, and then you decided right after you cleared it you wanted the list to have 10,000 elements again, that would be a HUGE waste of time. It would have to go through this whole process again of acquiring the memory space.

I think the compiler is smarter than that, and knows that even though you want to clear the "contents" of the list, it is highly likely you will need to use that allocated memory, or at least a portion of it, sometime again in the near future. So, it keeps it around just in case. At least, thats my theory

Try putting the list in a block and see what happens....

#include <list>
#include <iostream>

using namespace std;

int main()
{
    //Just a special block to test scope of the list
    {
       list<int> til;
       int k;

       for ( int k=0; k != 500000; k++)
       {
             cout << k << endl;
             til.push_back(k);
       }
    }//I am betting the memory will be deallocated
      //when this block is exited, because the list will be destroyed.

    int a;
    cout << "Enter a key: ";
    cin >> a;

    return 0;
}

[LoomisP]

I tried your code and the memory was not deallocated when the list went out of scope. My understanding is that you are right, all the memory allocated in the scope should be deallocated, but at least for me it is not the case.

Did you check try running this code yourself and see wht happens? Maybe it is just my compiler that is a problem.

[dcjr84]

Yeah, I just compiled and ran the code. It appears to me that the memory is indeed released when the list object goes out of scope. From what I know about objects, when they go out of scope their destructor is called and any memory allocated during construction or the objects lifetime is released. So this would make sense to me.

In the Windows Task Manager, under Processes, as my program runs I can see it steadily increasing in memory usage as push_backs are performed, up to about 12,000 K. Then as soon as the block is exited, the push_backs are done, and it is hanging at the cin >> statement, the memory for my program drops back to 1,016 K, where it stays until I end the program.

[screetch]

it depends on many parameters.

Many implementations of the STL keep any memory they allocate in a pool for a later reuse, and some implementations may even never free it : that's the case with STLport, unless you ask it explicitely.

If you test your code with MSVC have a look at _CrtSetAllocHook which will allow you to specify a hook that will be called each time an alloc is done or memory is freed. Compile it in debug. See if the memory is freed or not at the end of the block. Also test the same code with STLport : i bet the results are not the same

[ZuK]

STL containers usually let you specify allocaters that take care of actually allocating and deallocationg the memory used for the objects.
Look at the documentation for your STL implementation, there might already be an allocation strategy available that does what you need. If not you can write your own allocators.
Kurt

[NMTop40]

This line is illegal C++:

il.swap(list<int>());

bceause swap takes a non-const reference and you cannot bind a temporary to it.

But you can make the call the other way round:

list<int>().swap( il );

because although you cannot bind it to a non-const reference, you may call a non-const method from one on the same line.

[Paul McKenzie]

I have a problem where my memory usage does not decrease when I clear a list. Here is a simple example:

There are two things you should be aware of.

The first thing is whether "delete" or free() is called when a container is cleared. As others pointed out, a std::list will not call delete/free for the memory it has allocated when you call clear(). The swap mechanism is used to force the list deallocate the memory.

But even if delete or free() were called, there is another aspect, and that is whether the heap manager actually frees the memory back to the operating system. There are heap managers that do not automatically call the operating system to deallocate the memory. The reason is that the heap manager could be coded so that it holds onto the allocated memory, in case you wish to allocate the memory once again in the program. In this case, all the heap manager has to do is manipulate a few pointers when asked to allocate the memory, and the job is done, probably much faster than allocating memory from the OS all over again.

If you are using Task Manager or similar program to determine whether memory is actually being freed, remember that you do not have control over what memory is actually released (or obtained) from the OS by using the default "new", and "delete" operators. Yes, Task Manager and similar programs that track OS memory can be a good indicator, but by no means should it be used to certify that your program is actually using new and delete (or malloc and free) correctly, or whether the container classes are coded correctly.

The only exception to this is if you (or the heap manager) strictly calls the OS memory functions to obtain and release memory. The heap manager would then have to be coded to obtain the memory from the OS each and every time new or malloc() is called, and call the OS function to release the memory each and every time delete or free() is called. Very few, if any good heap managers are written this way.

Regards,

Paul McKenzie

[jfaust]

This line is illegal C++:

il.swap(list<int>());

bceause swap takes a non-const reference and you cannot bind a temporary to it.

I think I believe you, but I'm confused because:
1. VS2005 compiles it. Although I've found areas where it does incorrectly compile things that should fail, I've not found anything as blatant as this.
2. The compiled code does the right thing.

C++ Coding Standards does recommend your approach (Item 82).

Jeff

[screetch]

under Visual 2005 this code issues a Warning (4239) //nonstandard extension used : A non-const reference may only be bound to an lvalue

use a warning lvl of 4 to see it. As you can see it's an extension.

[jfaust]

under Visual 2005 this code issues a Warning (4239) //nonstandard extension used : A non-const reference may only be bound to an lvalue

Grrrr. Bit again. You should have to explicitly turn on extensions via a compiler switch.

Jeff

[screetch]

there is a switch to turn them off but then <windows.h> won't parse because there are unnamed structures which is an extension.

in our project we use visual 2005 to get an executable but test our core libraries under mingw to detect otehr warning or stupid extensions.