-
December 13th, 2010, 10:48 PM
#1
How to avoid OutOfMemoryException?
I am working on a game that needs to work with large data sets, basically it will process 1000 of them and store it's raw data into memory temporarily because it's fast and will not affect game play. Another thread will be called, then process each data set individually, storing them to the SQL database. These can be anywhere from a few bytes, to a few MB each.
So basically, there is lot of allocation/deallocation going on, and this is all happening in seconds, in a continuous loop. This is to save any changed data, as it gets changed (items, mobs etc).
Often, I will run into the dreaded OOME even though there is tons of ram available. According to my research, windows apps have a limit of about 1.5GB to 2GB. The app can reach near that at times. Is there not a way to circumvent this limit? I'm sure there are much larger apps out there that need to allocate much larger chunks of data, and need to hold much more data into memory, such as 3D CAD apps, games, etc... so what gives? How do they do it?
Another thing I was reading is that memory can become fragmented, and when allocating, if there is not a single chunk of space big enough, it will also crash, instead of allocating different parts, or doing some kind of defrag operation. So how would I go about dealing with that part? Is there maybe a defrag function of some sort I can call?
Thanks in advance for the help!
-
December 14th, 2010, 01:48 AM
#2
Re: How to avoid OutOfMemoryException?
When running on a 32bit setup, .NET have a limit of around 1.5 GB. (Address space properly).
Run on a 64bit setup if you need more.
-
December 14th, 2010, 12:08 PM
#3
Re: How to avoid OutOfMemoryException?
Will I really get more then 1.5GB in 64-bit though? I was reading somewhere that it's still a limitation. (on a per app basis).
-
December 15th, 2010, 01:23 AM
#4
Re: How to avoid OutOfMemoryException?
Yes - .NET can address a larger space in x64 and therefore you can use more memory.
Now, whether it allocates more physical memory or just uses swapping or what not, I've not looked in to, though.
But I ran some tests at one point because I encountered this problem while doing some XSLT via .NET and it crashed consistently with OutOfMemory at around 1.4-1.5 GB on a 32 bit setup.
However there might also be an issue with your code containing memory leaks and what not which might be fixed and helps, but well - as said - x32 bit crashes when using that much.
Last edited by Alsvha; December 15th, 2010 at 01:26 AM.
-
December 15th, 2010, 06:51 AM
#5
Re: How to avoid OutOfMemoryException?
I was reading somewhere that it's still a limitation. (on a per app basis).
There is still a limit, but I doubt you could afford the buy all that ram or even find a way of actually hooking it all up into one computer. It's a very very very large number
What you should probably do is run your app in a heap profiler and see if you're retaining objects in memory which you aren't expecting to retain.
www.monotorrent.com For all your .NET bittorrent needs
NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.
-
December 15th, 2010, 02:39 PM
#6
Re: How to avoid OutOfMemoryException?
Are you closing the SqlConnection and SqlCommand objects as outlined in this thread from last year?
http://www.codeguru.com/forum/showth...=483366&page=2
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|