dcsimg
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 21

Thread: Why can't I Dispose of a HashSet?

  1. #1
    Join Date
    Jul 2007
    Posts
    609

    Why can't I Dispose of a HashSet?

    I've been trying to troubleshoot why one of my apps randomly gets these huge slow downs durring a repetitious intense process that normally is fast (but needs to happen often). As a troubleshooting step I decided I will run Dispose() on every temp var as if I understand correctly, should prevent the garbage collector from having to run, and I think that is what is slowing down my app. Basically by not calling dispose on temp vars that keep getting declared and going out of scope, I am creating a memory leak, then the garbage collector comes to clean it, and that process might get called at a bad time (well, with this app, any time is a bad time, as it's not multithreaded and is a very time sensitive app). Am I right on this theory?

    Anyway, I noticed that MSDN says HashSet has a Dispose function but when I call it, it won't compile. I get some weird long explanation of an error but what I get from is basically "invalid funtion". This is the error I get:

    Code:
    Microsoft (R) Visual C# 2010 Compiler version 4.0.30319.1
    Copyright (C) Microsoft Corporation. All rights reserved.
    
    RunuoAoV\DataStorage\SqlStorageSystem.cs(738,15): error CS1061:
            'System.Collections.Generic.HashSet<Server.DataStorage.SqlSaveInfo>'
            does not contain a definition for 'Dispose' and no extension method
            'Dispose' accepting a first argument of type
            'System.Collections.Generic.HashSet<Server.DataStorage.SqlSaveInfo>'
            could be found (are you missing a using directive or an assembly
            reference?)
    According to MSDN, this should work:

    http://msdn.microsoft.com/en-us/library/bb339801.aspx

    Any reason why this is not working, and am I on the right track by calling Dispose on all temp vars to prevent garbage collection?
    http://www.uovalor.com :: Free UO Server

  2. #2
    Join Date
    May 2007
    Posts
    1,546

    Re: Why can't I Dispose of a HashSet?

    As a troubleshooting step I decided I will run Dispose() on every temp var as if I understand correctly, should prevent the garbage collector from having to run,
    No. Typically Dispose is used to ensure that limited system resources (like file handles or sockets) can be released back to the system as soon as you're done with them. The garbage collector will still have to run to collect the corresponding managed objects. The only way to prevent the GC from running is to allocate less objects. So your theory is wrong

    Use a profiler to figure out where it's hanging, you should be able to find something in there.
    www.monotorrent.com For all your .NET bittorrent needs

    NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.

  3. #3
    Join Date
    Jan 2010
    Posts
    1,133

    Re: Why can't I Dispose of a HashSet?

    Quote Originally Posted by Red Squirrel View Post
    Anyway, I noticed that MSDN says HashSet has a Dispose function but when I call it, it won't compile.
    No, you went to the wrong page - It says that HashSet<T>.Enumerator structure has a Dispose(). That is a different type.
    (HashSet<T> documentation is here.)

    I don't think your problem is related to the GC. Any multithreading in your app?

  4. #4
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    Quote Originally Posted by Mutant_Fruit View Post
    No. Typically Dispose is used to ensure that limited system resources (like file handles or sockets) can be released back to the system as soon as you're done with them. The garbage collector will still have to run to collect the corresponding managed objects. The only way to prevent the GC from running is to allocate less objects. So your theory is wrong

    Use a profiler to figure out where it's hanging, you should be able to find something in there.
    But what if I need to allocate lot of objects? It's just the nature of the program, there's no other way. It's just a very large scale app. So is C# not a good idea for such apps? Should I maybe just convert it to C++?

    There's multithreading but I took out that part for troubleshooting and it still does it. The slow down does not happen in the thread. Tried a profiler but it was useless. The slow down happens maybe 1 in 10000 if that, so the average time of each function will still be good. It always happen at a random part of my program so it's not even a specific thing that is slow.

    Guess I'll continue on. I was trying to find ways to redesign the whole process then thought maybe my problem is that I'm not disposing of objects but guess that would not do anything.
    http://www.uovalor.com :: Free UO Server

  5. #5
    Arjay's Avatar
    Arjay is offline Moderator / EX MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    13,248

    Re: Why can't I Dispose of a HashSet?

    What sort of app is it? What type of objects are you storing? Classes? Structs?

    Have you looked at the handle and thread count and memory usage?

  6. #6
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    It's all custom types, there are a couple thousand different types but they all derrive from 4 master types. There are other separate types that are used to hold information and each object has a declaration of that object. Overall, there is LOT of data. The app uses maybe 500MB or so of ram and this will grow over time as the game world becomes more populated.

    The app is a game server so basically every change is recorded into a queue and at set intervals 2000 of them are processed. This process takes about 50ms as to not impede with game play, but every now and then it will take a couple seconds causing a huge lag spike. Yesterday I even clocked one at like 5 minutes. It basically just serializes the object values into a long string. It never craps out at the same spot. This happens maybe once or twice a day so I'm even thinking of just living with it. Right now the live server uses a different save system where every 15 minutes there is a message that comes on to say that it's saving and then it takes a good 30 seconds to process everything at once. My system only processes the objects that changed and does it in small batches so overall there is way less lost game time, but those unexpected lag spikes can be quite nasty.

    I did not get anything out of the profiler given this is too infrequent. I used perfmon and it coincides with garbage cleanup, so I figured that was the cause. I tried changing a setting (forget the name) to increase/decrase garbage cleanup aggressiveness but that did not really do anything. Come to think of it, even before I implemented this system we would sometimes get random lag spikes that were unexplainable. I bet it was the same thing happening. It's just that this new process I introduced is probably using like half of the cpu time until the world is caught up so it's more likely that it happens within my catch stop watch that I added to debug this issue.

    I think I will just give up and live with it. I may even get lucky and it won't do it on the live server or something.
    http://www.uovalor.com :: Free UO Server

  7. #7
    Arjay's Avatar
    Arjay is offline Moderator / EX MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    13,248

    Re: Why can't I Dispose of a HashSet?

    What type of objects are you storing? Classes? Structs?

  8. #8
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    Mostly classes but there are some structs and enums as well that are part of those classes. They contain a mix of ints, strings, collections etc. If I had to guess how big they are I'd say 500 bytes to 10KB as far as ram usage goes per object but it really depends on the object and what it's for. A basic decorative object in game will be small compared to something like a house.

    I don't want to speak too soon but I think I may have figured out the issue. I gave the VM 4 gigs of ram and so far it has not done the slowdown thing. I think it was in fact a memory issue. But I'm speaking too soon I think. I will let it go for a couple days. Then I'll try to bring the ram down to 2GB as that's as far as I can go in the prod environment (ram is very expensive when it comes to rented servers). I'm hoping that's all it is because if it means paying 50 bucks extra a month so I can up the ram, then so be it, I'll pay it. My red hair is turning gray.
    http://www.uovalor.com :: Free UO Server

  9. #9
    Join Date
    May 2007
    Posts
    1,546

    Re: Why can't I Dispose of a HashSet?

    "It basically just serializes the object values into a long string. "
    If you are turning a list of objects into one single incredibly long string is this going to cause *huge* issues. The problem is that every time you concatenate two strings you create an entirely new string object which is the combination of the previous two and the other two are left for the GC to collect. If your strings get to be a few MB in size, this can be *serious* amounts of garbage and GC time. A simple heap profiler would catch this kind of issue.

    How/where are you storing this string anyway?
    www.monotorrent.com For all your .NET bittorrent needs

    NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.

  10. #10
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    It's a string builder during the serialize process and is converted into a string at the end. The string is then put into a SQL query, so it has to be done in one shot. while debugging I removed the serialize process and it just does the freeze thing somewhere else instead.

    My test of increasing the ram did not work either, but it did help. It did it 6 times overnight, some as high as 4 seconds.
    http://www.uovalor.com :: Free UO Server

  11. #11
    Join Date
    Jan 2010
    Posts
    1,133

    Re: Why can't I Dispose of a HashSet?

    Quote Originally Posted by Red Squirrel View Post
    It's all custom types, there are a couple thousand different types but they all derrive from 4 master types.
    Just one thing that bugs me: thousands of types? You sure you meant types? If so, design-wise, it seems too granular to me? Even for a game? Why so many types?

  12. #12
    Join Date
    May 2007
    Posts
    1,546

    Re: Why can't I Dispose of a HashSet?

    Have you tried running your app through a standard heap profiler to check if you're hitting issues similar to what I described earlier? Appending in a stringbuilder can also potentially hit similar issues (though nowhere near as bad) depending on how large the strinbuilder is getting. Getting garbage collection stats from a heap profiler should help significantly in further diagnosing this. The basic and free CLR profiler should do perfectly for now: http://msdn.microsoft.com/en-us/libr...howto13_topic2

    Rewriting your app in C or C++ is highly unlikely to help as this sounds like an algorithmic failure. You need to track down how/why this is happening before even considering a language rewrite.
    www.monotorrent.com For all your .NET bittorrent needs

    NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.

  13. #13
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    I'll try the heap profiler and see how that goes. I have not posted code as this is a super huge app and I don't expect anyone to actually sift through all that, but I'll try out general tips. I'll start by the profiler and go from there.

    And yes I did mean types. Basically every type of item or mobile in the game is represented by an actual type programatically. For example a dragon is one type, a drake is another type. A dragon might have a special ability such as being able to spit fire balls, so this code is coded within that type. some types have extra variables so they'll have those in their respectful serialize/deserialize function. A dragon will derrive from monster and monster derrives from mobile, for example. Don't know if that's the proper way to do things, but that's how it was done by the original devs.

    One problem I see with the way they did lot of these things is throughout the app there is lot of typecasting. I've read that having to do typecasting a lot is bad practice.
    http://www.uovalor.com :: Free UO Server

  14. #14
    Join Date
    Jan 2010
    Posts
    1,133

    Re: Why can't I Dispose of a HashSet?

    Quote Originally Posted by Red Squirrel View Post
    [...]but that's how it was done by the original devs.
    Is it an open-source project?

    Quote Originally Posted by Red Squirrel View Post
    And yes I did mean types. Basically every type of item or mobile in the game is represented by an actual type programatically. For example a dragon is one type, a drake is another type. A dragon might have a special ability such as being able to spit fire balls, so this code is coded within that type. some types have extra variables so they'll have those in their respectful serialize/deserialize function. A dragon will derrive from monster and monster derrives from mobile, for example.Don't know if that's the proper way to do things [...]
    Yeah, they designed it badly, abusing inheritance and creating a myriad of types, that is probably both hard to use and maintain.
    Now, redesigning that system would probably take forever, but even without that in prospect, I'd just like to make a few points.
    When designing a system, one should prefer composition over inheritance. Design patterns provide a set of commonly used solutions for certain types of engineering problems; they beautifully demonstrate when it's appropriate to pick inheritance, and when to pick composition.

    My point is, when you're creating an inheritance hierarchy, you have to decide where to stop (and it should be before you end up with thousands of types). So, instead of having
    mobile/monster/dragon
    and
    mobile/monster/drake
    (or is it
    mobile/monster/dragon/drake?)
    , you could stop at monster, and make the data define what it actually is in terms of presentation, and a specialized contained object(s) define what it is in terms of behavior.
    You just have to be creative about the public interface - it has to be both general enough and it has to be able to provide the required specialized functionality.

    For example, a good pattern to use here would be Strategy - it models (abstracts out) behavior so that it can be dynamically replaced. In practice, it means that your monster object would contain a Strategy object instance, but the actual type of that instance would be derived from Strategy, providing specialized behavior. The monster would then use the interface of Strategy to let it do the work for it.

    Code:
    // Strategy pattern mockup:
    // FireAttack, SoundAttack and ClawAttack all derive from Attack
    // which is the type of both properties.
    // Attack is an abstract Strategy, while the derived classes are ConcreteStrategy-es
    Monster dragon = new Monster("Red Dragon", dragonModelFilePath);
    dragon.PrimaryAttack = new FireAttack(damage1);
    dragon.SecondaryAttack = new ClawAttack(damage2);
    
    Monster drake = new Monster("Gray Drake", drakeModelFilePath);
    drake.PrimaryAttack = new ClawAttack(damage1);
    drake.SecondaryAttack = new SoundAttack(damage2);
    
    // The beauty of it is, it can be made to work with other types
    // (maybe a better naming is in order, but hey...)
    AutomatedWeapon tripBomb = new AutomatedWeapon("Trip Bomb", bombModelPath);
    tripBomb.Attack = new FireAttack(bombDamage);
    The point is, instead of having thousands of types, you can have the same richness by having far less types, but allowing for thousands of combinations.

    Quote Originally Posted by Red Squirrel View Post
    One problem I see with the way they did lot of these things is throughout the app there is lot of typecasting. I've read that having to do typecasting a lot is bad practice.
    Yeah, but occasionally it's OK, and sometimes even necessary. Sometimes it's faster and cleaner to do a typecast than to go through a series of messy indirections just to avoid it.
    Last edited by TheGreatCthulhu; May 16th, 2011 at 05:04 PM.

  15. #15
    Join Date
    Jul 2007
    Posts
    609

    Re: Why can't I Dispose of a HashSet?

    That actually mirrors some of my thoughts as well, glad I'm not the only one that thinks that. If I had designed this I would have rethinked lot of things and that's one of them. The classes should have parameters to define them further, and worse part is, THEY DO! But they still use separate classes to fill them.

    Now there is one advantage to the way they did it and it is used a lot in the game and it's that you can do stuff like:

    if(creature.GetType() == Typeof(Dragon)) But I suppose this could easily be done a different way.

    If I was to recode this in C++ (I'd choose that lang mostly so it could run on Linux) I would probably fix a lot of this stuff but before I attempt this I will probably want to read up more on better programming practices and advanced stuff so I could start on the right foot. I have coded a couple C++ and php apps from scratch in my time and while they work fine I'm sure I could do better.
    http://www.uovalor.com :: Free UO Server

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)