|
-
July 21st, 2008, 12:56 AM
#9
Re: Memory losse and thread... advice?
@madric: I think I have now a complete image of what your crash situation was; I understood that you already removed the memory leak and that you are simply worried if the crash was not created by something else than these leaks, which have not yet filled the entire RAM you have on your machine...
- well, first of all: your computer may have more than 2gb RAM (I have a 4gb, set-up such as the OS would not use more than 1gb, so... most light apps I have running on my PC take less than 1gb... thus, always having 2gb available for 1 resource desperate application at a time), but... on a 32 bit architecture, the address space of 1 process is no bigger than 2gb.
- second: as you've mentioned, task manager is not a proper tool to monitor what happens to your process; better use Process Monitor, Process Explorer or PsTools Suite; still, lets say your application wasn't using more than 1.4 GB of virtual memory... still, your heap manager might refuse to allocate more virtual pages... from my personal experience, using the heap manager which comes with msvcrt.dll you can safely assume that you can allocate about half of your entire available RAM (as long as that is not 8GB or something )... so, lets say it has a limit of 1gb... with or without memory leaks, your code should handle the situation in which you can't allocate more memory. Remember, your process needs a lot of memory for everything else: loaded modules, stack for each threads, and so on... If your application design is made in such way that you limit the number of concurent threads and the maximum limit of allocated memory by each thread, then there's less concern about what can happen if suddenly your heap manager refuses to commit more virtual pages for you. Now that you've removed your memory leaks, shouldn't be more concerned... since your application would work just fine in 99% of regular production environments. It might fail a stress test... but would work perfect in normal, acceptable environment.
A few years ago, I've been working on a rendering engine that needed a lot of graphics card and RAM resources, as well as CPU time. Was designed as MT, but with fixed number of threads. Because of the things described above, I had to start reusing tiles of imagery allocated as soon as the half total RAM or 1gb limit was reached (which ever was smaller), or else, allocation of new tiles would fail soon after that limit. I've tested several HW environments, including dual-processor machines. Another surprise was offered by the call to calloc() which had to be changed into using malloc() and memset(), because for some reason calloc() was failling in situations when separate threads running on different processors were allocating zeroed memory. Today, I would replace memset() with ZeroMemory() API.
Let me remind you that you should keep in mind memory fragmentation when having a high number of re-allocations of bigger contiguous memory blocks. You should try reusing those already allocated buffers (since I've noticed your allocations are of same size), instead of deallocating them and re-allocating same size buffers again and again. Would be faster and safer!
With that said, I think the problem you had was generated by the memory leaks, but, if you have any more problems, or you don't wanna have more problems , you should be looking into what I have mentioned above.
Best of luck and happy programming,
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|