What is the major drawback of C# .NET?

[Parameswar]

Dear Friends,
I am working in .NEt tech. Like Asp.Net,VB.NET.
I want to know that What is the major drawback of C# .NET.

[TheCPUWizard]

It is difficult for people who want to shoot themselves in the foot to do so...

Seriously, about the only "real" issue is that you are definately tied to a Microsoft Platform. On the other hand, you can get the best of functionallity, easily for that platform.

[Shuja Ali]

Dear Friends,
I am working in .NEt tech. Like Asp.Net,VB.NET.
I want to know that What is the major drawback of C# .NET.
That is a vague question. Why are you looking for drawbacks?

[petes1234]

Dear Friends,
I am working in .NEt tech. Like Asp.Net,VB.NET.
I want to know that What is the major drawback of C# .NET.

Because it is made by Microsoft, and they are evil, I tell you, evil!!

Check out the latest Microsoft attempts to squash the open source community through specious lawsuits:
Microsoft patent claims complicated by GPLv3 (http://news.yahoo.com/s/infoworld/20070516/tc_infoworld/88592;_ylt=ArAG.ZnDgDQlcVR1gOI_hHSor7oF)
Sun Responds to Microsoft's Patent Claims (http://news.yahoo.com/s/nf/20070516/bs_nf/52325;_ylt=Ap76p_3JJ6Ym2XCfzA77lI6or7oF)

Except for that, and that it is mainly a tool for constructing windows apps, it's a pretty decent development environment and programming language. That Anders (http://www.ddj.com/184404602) is one smart dude.

[MadHatter]

Because it is made by Microsoft, and they are evil, I tell you, evil!! yep. and that oath you have to swear in blood before the the dark lord and his angles before they allow you to download the framework redistributable off msdn is a bit excessive IMO :p ;)

[Shuja Ali]

Because it is made by Microsoft, and they are evil, I tell you, evil!!Where did you get this one from? Can you give me specific reasons why they are Evil. Just because some one is using their patented technology and they are going against them doesn't mean that they are Evil.

There are so many other OpenSource projects that Microsoft is helping and encouraging. Have you checked those Open Source frameworks?

Beauty lies in the eyes of the beholder. If you have a perception that MS is evil, even when they don't do anything bad, you will still see them as Evil.

[MadHatter]

I believe pete was using humor there :D

[TheCPUWizard]

Originally Posted by petes1234
Because it is made by Microsoft, and they are evil, I tell you, evil!!

That may be, but who is to make a value judgement. ;)

Seriously, It is generally recognized that without the "Triad" of the late 1970's early 1980's [IBM, Intel, Microsoft], the explosion of computers into daily life for a vast majority of the population would not have occured at nearly the rate it did (if it occured at all).

[Shuja Ali]

I didn't know that. You see English is not my first language. :)

[petes1234]

I didn't know that. You see English is not my first language. :)
You could have fooled me. You are very adept at it.

OK, I'll admit that perhaps MS is not evil, but I do feel that they are obsessed with power, control and domination of the software market, that they have used their near-monopoly status to inhibit competition and intimidate the little guy (pretty much anyone else when compared w/ them). They have been dinged for this by the EU, and yet this behavior persists.

[TheCPUWizard]

You could have fooled me. You are very adept at it.

OK, I'll admit that perhaps MS is not evil, but I do feel that they are obsessed with power, control and domination of the software market, that they have used their near-monopoly status to inhibit competition and intimidate the little guy (pretty much anyone else when compared w/ them). They have been dinged for this by the EU, and yet this behavior persists.

And this is different than other large companies in so many different fields??? [neglecting those cases where intrusive goverment regulations are imposed]

btw: This should probably be split off to a non-technical part of CG....

[petes1234]

And this is different than other large companies in so many different fields??? [neglecting those cases where intrusive goverment regulations are imposed]
You make a good point, and this is certainly not a simple case of black and white, good and evil, but it's also a matter of scale. What MS does has a huge effect on the software industry as a whole, much greater than that of any other software company, bar none. When such dominance and intimidation has occurred in other industries in the past, governments are often forced into regulation (see for instance AT&T, Standard Oil, and the steel industry).

btw: This should probably be split off to a non-technical part of CG....I agree. I've gotten WAY off topic. Sorry!

/Pete

[cjard]

Dear Friends,
I am working in .NEt tech. Like Asp.Net,VB.NET.
I want to know that What is the major drawback of C# .NET.

Erm. My keyboard doesnt have a # key?!!

That's like.. a pretty bad thing uh? If I want to write C#, I have to write C Sharp all the time. Its very tiring

[cjard]

No wait.. I just realised there is a way bigger problem with C#.net... it doesnt exist?! Every time I try to go there my web browser says "invalid character in URL: #"

Can someone with a good web browser please save http://c#.net onto a floppy disk and post it to me?

[darwen]

I want to know that What is the major drawback of C# .NET.


Compared to what ? C++ ? Forth ? Lisp ? My washing machine ? :D

Also for what purpose ? Databasing/business applications ? Graphics applications ? Games ? Viruses ? Web sites/applications ? To make me a cup of tea at the end of the day ?

Most languages have their advantages and their disadvantages. To find these out convincingly you have to know which language you're comparing to which and for what purpose you intend to use the language for. It's also advisable to learn something of the languages for yourself.

Darwen.

P.S. The difference between C# and my washing machine is that I understand C#.

[thegrinch]

No, I think the only mistake those people at MS are doing is to try to make non-programmers able to program. They try to make everything so easy, that in the end, for me, everything is getting more and more unusual.
The best example are the good old destructors and all that "manual garbage collection" the programmer did himself. Believe me, that days, my applications needed less memory and had no leaks! Today, my app grows up to 500 (!!!) MB.

I do not think that each and every person has to be able to write programs. The technology that is necessary for this (->.net) handicaps professional programmers more than it helps the other people.

hope you could understand me.

kind regards

thegrinch

[TheCPUWizard]

The best example are the good old destructors and all that "manual garbage collection" the programmer did himself. Believe me, that days, my applications needed less memory and had no leaks! Today, my app grows up to 500 (!!!) MB.


1) it is extremely doubtful that all of your programs never had a memory leak even in the event of all possible exceptions.

2) Who care about the memory footprint. If your computer has 1GB and nothing else is running, then the program SHOULD make use of the available memory. [Which is why GC does not invest any effort until there is memory pressure, there simply is no point].

(As I have said many many times) I have been a professional developer for over 30 years, and C# has provided the most robust and effective environment I have worked in (for general business type applications).

[Zaccheus]

Can someone with a good web browser please save http://c#.net onto a floppy disk and post it to me?

I've found a mirror site: http://www.seesharp.com/








Hey, look: 'THIS DOMAIN IS FOR SALE'

[TheCPUWizard]

I've found a mirror site: http://www.seesharp.com/


Zaccheus << "SMACK!"; :D :D :D

[thegrinch]

2) Who care about the memory footprint. If your computer has 1GB and nothing else is running, then the program SHOULD make use of the available memory. [Which is why GC does not invest any effort until there is memory pressure, there simply is no point].

But the concept of managing the memory (in software) is slowing down everything. There is always a small "delay". This was in the good old WinAPI and MFC times not the case, you always got a direct immediate reaction.

I am extensively doing numerical computations, and I can tell you about very bad performance of .net apps. This also must be the case, if each and every memory access is "checked for validity".

Regards

thegrinch

[_uj]

No, I think the only mistake those people at MS are doing is to try to make non-programmers able to program. They try to make everything so easy, that in the end, for me, everything is getting more and more unusual.
The best example are the good old destructors and all that "manual garbage collection" the programmer did himself. Believe me, that days, my applications needed less memory and had no leaks! Today, my app grows up to 500 (!!!) MB.

I think this is an oversimplification. Okay modern OO languages like C# and Java are simpler than C++ in the sense that the syntax is cleaner and that the number of options have been reduced, but these languages are harder in many other ways. The emphasis in C# and Java is much more on OO techniques and the efficient use of powerful data structures. So although inexperienced programmers maybe make less traditional "shot in the foot" errors, they tend to make mistakes at a higher level like bad OO designs and use of inappropriate data structures.

So in my view the introduction of C# and Java hasn't made programming any easier, rather harder.

[TheCPUWizard]

But the concept of managing the memory (in software) is slowing down everything. There is always a small "delay". This was in the good old WinAPI and MFC times not the case, you always got a direct immediate reaction.


Actually in many circumstances the exact oposite is true. Every time you call delete in a native application there is a delay while the heap status is updated (marking the block as free, checking for adjecent blocks, collapsing free blocks, etc). With garbage collection, this does not happen!

The actual overall time is moved to the point where you are going to attempt an allocation. Unlike the claims by many, this timing is extremely predictable, if you are aware of the state of the entire system.



I am extensively doing numerical computations, and I can tell you about very bad performance of .net apps. This also must be the case, if each and every memory access is "checked for validity".


I also do some very intensive numerical applications [processing 256 channels of audio with 256 dynamic "parameters" at a frame rate of over 30fps] Many of these are highh/low/band/gap/comb filters, or other dsp algorithms.

I have absolutely no problems, but I have been very careful about allocations and effects on the GEN0, GEN1, GEB2, and LO heaps. Have you considered all of these things ars you were designing and implementing the code? Can you go through all (or at least a significant portion) of your object instances and references and determine the lokelyhood of which generation will collect them? Can you go through 100% of your objects and determine which are on the LOH? Have you used perfmon (or another tool) to measure the frequency/duration of your collections?

I do not mean this personally, it is jut that the vast majority of people writing managed code, and attempting near-realtime or true realtime applications have not taken these items into consideration. The single biggest issue (which takes many forms) is holding on to references to information for longer than necessary. Consider:

"conventional code" (pseudo)

SomeType *info;
info = new SomeType();
// do some things with info
....
// do some things that do not use info
...
// do some things with info (that to not require state of the info instance from above
...
delete info;


This is typically a bad implementation in managed code.

"conventional code" (pseudo)

SomeType *info;
info = new SomeType();
// do some things with info
delete info;
....
// do some things that do not use info
...
info = new SomeType();
// do some things with info (that to not require state of the info instance from above
...
delete info;


If one cannot explain why the second is (typically) much better than the first, then they do not understand the environment.


Regards

:wave: :wave:

thegrinch[/QUOTE]

[_uj]

If one cannot explain why the second is (typically) much better than the first, then they do not understand the environment.


I can explain why both are bad. :)

In the first case you may introduce a delay which translates into an unresponsiveness in the GUI. In the second case this is unlikely but it can still happen because you don't control when the GC decides to do kick in and do something. The whole issues should instead be resolved using threading so that extensive work is offloaded from the GUI thread.

[TheCPUWizard]

I can explain why both are bad. :)

In the first case you may introduce a delay which translates into an unresponsiveness in the GUI. In the second case this is unlikely but it can still happen because you don't control when the GC decides to do kick in and do something. The whole issues should instead be resolved using threading so that extensive work is offloaded from the GUI thread.

_uj,

Yes "work" should be done in a background thread if there is a UI. But that actually has nothing to do with the post. Assume a somple condition where a person is going to run a simple program from the command line, that will output results at the end. Assume program is simple and single threaded.

There is still a very significant difference between the two code samples I provided. If it is not immediately apparent what it is then [IMHO] it is unlikely that the reader will be capable of developing high performance managed applications.....

[_uj]

_uj,

Yes "work" should be done in a background thread if there is a UI. But that actually has nothing to do with the post. Assume a somple condition where a person is going to run a simple program from the command line, that will output results at the end. Assume program is simple and single threaded.

There is still a very significant difference between the two code samples I provided. If it is not immediately apparent what it is then [IMHO] it is unlikely that the reader will be capable of developing high performance managed applications.....

Well, you quoted thegrinch and I got the impression it was about GUI responsiveness.

Anyway I don't see the significant difference between the two code examples. Maybe you would care to explain.

[TheCPUWizard]

Maybe you would care to explain.

Well since you ask, and no one seems to be jumping on providing the answer.... :D

In the first case the info instance of SomeType is kept for the entire routine.

If // do some things that do not use info performs any allocations that would trigger a GC, then the instance will not be collected, and will be premoted from Gen0 to Gen1. This means that it will never be collected no matter how many Gen0 (the cheapest) are performed even after the routine exits.

Even worse is if the intermediate code does enough allocations that a Gen1 collection is performed, at this point, the object will persist until the next Gen2 (the most expensive) collection is performed.

So simply holding on to a reference for a few lines of code (especially if those lines either allocate, or call routines that allocate, or exist in a multithreaded environment where other threads might allocate) can have a potentially severe impact on system performance.

The second example addresses this. The reference is help on to for the absolute minimum time that it is needed. This allows the GC to run much more effectively.

---------
Test this on your (every reader of this thread who writes managed code). Use perfmon.exe (if you don't know what that is, go learn no, it is one of the most valuable system tools. Set it up to measure the following:

1) # of GEN0 collections
2) # of GEN1 collections
3) # of GEN2 collections
4) %Time spend in GC.
5) GEN0 Heap Size
6) GEN1 Heap Size
7) GEN2 Heap Size
8) Large Object Heap Size

If your progam is written well you should see one of the following:

1) Your program does all of it's allocation somewhere at startup. Therefore there will be very few if any executions of the CG. This condition is both rare and typically not found.

or

2) There are Gen0 collections happening on a fairly regular basis, little or no memory is being premoted to the Gen2. Gen1 collections tend to coincide with the frequency and occurance of "top level operations", and the Large Object Heap is small (ideally) or at least FLAT. If these conditions are met, ten it is typical to find less than 2-3% of the time spend in GC (average).

Unfortunately, unless the developer was very aware of conditions like I posted earlier (as well as making sure that objects which implement IDisposable are ONLY created and used from within "using blocks", and no references are held except for the one controlled by the using clause), it is quite likely that you will see creep into the Gen2 heap with either the total foot print of your program growing, or a large amount of time spend in the GC.

-------------------------------

The bottom line is that while the GC does inherently minimize managed memory leaks, it does NOT remove the responsibility of effective management from the programmer. This often involves a paradigm shift (mindset change) that is as (or more) significant than the shift that occured when switching from "structured programming" in "C" (or Fortran, Cobol, Pascal), to an "Object Oriented Programming" approach in C++, or.....

The biggest difference is that there was significant awareness that there was a shift from SP to OOP. But many (most?) developers are painfully ignorant of the many things that have to be done properly..

[Are you 100% confident the for EVERY "event += delegate" there is a cooresponding un-registration if you object is not being actively used while the object containing the event is still in use??? Even under all exception conditions?????]

[_uj]

Well since you ask, and no one seems to be jumping on providing the answer....


You posted pseudo-code similar to C++/CLI but what's the equivalent in C#?


So simply holding on to a reference for a few lines of code (especially if those lines either allocate, or call routines that allocate, or exist in a multithreaded environment where other threads might allocate) can have a potentially severe impact on system performance.

The second example addresses this. The reference is help on to for the absolute minimum time that it is needed. This allows the GC to run much more effectively.


In general I don't believe in "helping" the GC so I think your reasoning is flawed apart from maybe in some very special cases.

It really boils down to how likely is your scenario and how severe is the impact? Statistically how often will an object which is used strictly locally to a method get "accidentally" promoted to a higher GC generation? My guess is that this is extremely rare and nothing to bother about. If I'm right then your optimization actually slows things down because of the extra object creations it requires.

A much better rule is to avoid finalizers because objects sporting them will always be promoted to the highest GC generation (and thus always be GC expensive).

As a sidenote there's an optimizing technique called escape analysis. It means that the compiler automatically detects local-to-method objects. It can then allocate such objects on the local stack, or on the heap but treat them outside the generation system and discard them together with the stack when the method is left. Maybe the .NET compilers have escape analysis? In any case it would render your optimization unnecessary.

As another sidenote, C++/CLI allows you to declare reference variables in methods to have stack semantics. If you do, in principle you have made the escape analysis for the compiler and such objects could be handled very efficiently totally avoiding the GC cycle.

[TheCPUWizard]

You posted pseudo-code similar to C++/CLI but what's the equivalent in C#?

Just setting the reference to null is sufficient if there is no IDisposable or finalizer...


In general I don't believe in "helping" the GC so I think your reasoning is flawed apart from maybe in some very special cases.

Actually I have made a significant part of my living for the past 4 years in this exact arena. If you do not write code that is designed to work well in a garbage collector environment, and do not know the basic strategy of the GC, then you WILL write POORLY PERFORMING code.


It really boils down to how likely is your scenario and how severe is the impact? Statistically how often will an object which is used strictly locally to a method get "accidentally" promoted to a higher GC generation? My guess is that this is extremely rare and nothing to bother about.

You have to TEST each occurance. Put a WMI sentinal at the beginning and end of a code block. Use perfmon. If objects get promoted while the sentinel is active, then this IS occuring.


If I'm right then your optimization actually slows things down because of the extra object creations it requires.

The allocation of a new object is effectively

pointer =gen0ptr;
gen0ptr += sizeof(newObject);
return pointer;

On most machines this executes in well under 1uS (one millionth of a second), rarely significat.


A much better rule is to avoid finalizers because objects sporting them will always be promoted to the highest GC generation (and thus always be GC expensive).

Basically true, but nowhere have I mentioned finalizers, except indirectly when I mentioned IDIsposable. Owhich have finalizers should implement IDisposable, and IDisposable object should always be properly guarded to make sure Dispose is called, which in durn should cal SuppresssFinalizer(), so for performance Finalizers will NEVER execute.


As a sidenote there's an optimizing technique called escape analysis. It means that the compiler automatically detects local-to-method objects. It can then allocate such objects on the local stack, or on the heap but treat them outside the generation system and discard them together with the stack when the method is left. Maybe the .NET compilers have escape analysis? In any case it would render your optimization unnecessary.
[quote]
Once the last use of an object is detected within a scope, the compiler is free to (and in many cases will) allow the object to be collected. The usage of the instance at the bottom of my original sample however would proclude this from happening (and is why the sample has all three parts rather than just the first two...

[quote]
As another sidenote, C++/CLI allows you to declare reference variables in methods to have stack semantics. If you do, in principle you have made the escape analysis for the compiler and such objects could be handled very efficiently totally avoiding the GC cycle.
True, but there are issues with stack semantics also. For example, it does NOT adddress the issue I have outlined...

[_uj]

Actually I have made a significant part of my living for the past 4 years in this exact arena. If you do not write code that is designed to work well in a garbage collector environment, and do not know the basic strategy of the GC, then you WILL write POORLY PERFORMING code.

And the best way to do that is to follow the golden rule of GC. Allocate new objects when you need them and then just stop using them when you don't need them anymore.

The GC comes with a certain overhead so in general one should avoid extensive object creations but that's basically a design issue. See for example the Flyweight pattern. One should also avoid so called unvoluntary object retention, that is to unwittingly hold on to objects that are no longer needed.

Trying to second-guess and "help" the GC is not good. In fact that's often the cause of bad GC performance. Modern GC's are extremely efficient. They know more about your program than you ever will. Just leave object management to the GC and you get the best results.

If the GC turns out to be the problem then it's highly likely that the application, or parts of it, shouldn't use one in the first place, or use a deterministic GC instead. This is typical of the "optimization" you're suggesting. You're just tuning the symptoms, not solving the problem.


Basically true, but nowhere have I mentioned finalizers,

I mentioned finalizers as something to generally avoid because it's known to slow down recycling of objects for sure.


True, but there are issues with stack semantics also. For example, it does NOT adddress the issue I have outlined...

It certainly does in the context of the escape analysis I was talking about. This allows a compiler to render your "optimization" completely unnecessary. The object you're trying to make sure will stay in the lowest GC generation won't even enter the GC cycle.

[TheCPUWizard]

And the best way to do that is to follow the golden rule of GC. Allocate new objects when you need them and then just stop using them when you don't need them anymore.

100% true, but it is amazing how many people hold on to them much longer than necessary (e.g. using a member variable with the life of your object rather than local variables within the method

The GC comes with a certain overhead so in general one should avoid extensive object creations but that's basically a design issue. See for example the Flyweight pattern.

Here we differ. as I mentioned before the "overhead" of object creation is (approximately)

pointer =gen0ptr;
gen0ptr += sizeof(newObject);
return pointer;

When the GC runs, it starts with all rooted references and walks the object tree. (Assuming a Gen0 or Gen1 collection), any objects still in existance get COPIED to the next generation, and all of the pointers that reference it updated [this is overhead!], the heap pointer then gets set back to the base:

gen0ptr = gen0base;

So if during the timespan between collections you generated a few million objects that you no longer had references to, then ther would be the (again sub microsecond) allocation cost mentioned above), but there would be absolutely 0 overhead during the garbage collection phase.

In fact, the term "Garbage Collection" is really a misnomer, and there have been many articles written about this. What actually happens is that all valid objects are collected individually, and all of the garbage is evaporated with a single machine language instruction (or two).




One should also avoid so called unvoluntary object retention, that is to unwittingly hold on to objects that are no longer needed.

Again we agree, see my first response in this reply!


Trying to second-guess and "help" the GC is not good. In fact that's often the cause of bad GC performance. Modern GC's are extremely efficient. They know more about your program than you ever will. Just leave object management to the GC and you get the best results.

If you are talking about having GC.Collect(...) in your code, you are 100% right. This should never be done.


If the GC turns out to be the problem then it's highly likely that the application, or parts of it, shouldn't use one in the first place, or use a deterministic GC instead. This is typical of the "optimization" you're suggesting. You're just tuning the symptoms, not solving the problem.

Again, just based on my exprience, improper design that does not account for the behaviour of the GC is the root cause. But as the lyric goes "It's just my job 5 days a week"...


I mentioned finalizers as something to generally avoid because it's known to slow down recycling of objects for sure.

YUP. In fact some of my classes the I want ro really make sure properly get disposed have a finalizer that throws and abort exception (in development builds). If Dispose is properly invoked, then the finalizer never runs, otherwise I get to find and fix the problem.

(emphasis added for context).
It (stack allocation) certainly does in the context of the escape analysis I was talking about. This allows a compiler to render your "optimization" completely unnecessary. The object you're trying to make sure will stay in the lowest GC generation won't even enter the GC cycle.
You are correct that the object itself will have the duration of the stack frame and will not enter into GC (but again remember objects with no references impose 0 cost at GC time), however it CAN make things worse...

class MyBadClass
{
public MyBadClass()
{
for (int i =1; i<2000; ++i)
m_MyList.Add(new char[7500]);
}
private
ArrayList m_MyList = new ArrayList;
}

If you instanciate this on the stack, there is no way to release the object (and thus the memory in use by the internal array) until the stack frame exits. If a GC occurs at any point after the object is created on the stack, then the 2,000 objects in the array list will all get promoted to GEN1 (occuring a cose).

If, the instance of MyBadClass had been created on the heap and used for a duration shorter than the duration of the stack frame, this situation could have been avoided.

}

[_uj]

100% true

I think you're wrong when you claim that the GC has to be managed by the programmer to be efficient. On the contrary I think the GC is most efficient when the programmer just relies on it and uses sound programming techniques.

But, as a general advice, don't finalize. This is guaranteed to be inefficient.

Also note that what the GC does is memory management. Resource management is your responsibility.

[TheCPUWizard]

I think you're wrong when you claim that the GC has to be managed by the programmer to be efficient. On the contrary I think the GC is most efficient when the programmer just relies on it and uses sound programming techniques.

But, as a general advice, don't finalize. This is guaranteed to be inefficient.

Also note that what the GC does is memory management. Resource management is your responsibility.


My point is that "sound programming techniques" uin a GC'ed environment are radically different at fundamental levels than " sound programming techniques" in a programmatic memory management system.

[Mutant_Fruit]

Let me just wade in here as a guy who has done *extensive* optimisations regarding memory and cpu usage in C#.

As a general rule of thumb in managed languages, allocating on the heap is bad. Every object you allocate on the heap is memory that you cannot reclaim until a garbage collection occurs.

So you say "But what if i null out my references as soon as i'm done with them". That doesn't matter. You've still allocated the object and you've still created it on the heap.


SomeType *info;
info = new SomeType();
// do some things with info
delete info;
....
// do some things that do not use info
...
info = new SomeType();
// do some things with info (that to not require state of the info instance from above
...
delete info;

If you can't tell me why this code is *terrible* in most use cases, then you can't really talk about optimisation in a serious manner.

Let me talk about the general case. Suppose you do what this code suggests. You dump your reference as soon as you finish creating the object and then recreate a new one when you need it. Now, imagine that this bit of code is being called 100,000 times a second. Instead of only allocating 100,000 objects, you are now allocating 200,000 objects. You're now forcing your application to garbage collect twice as frequently.

Ok, so gen 0 collections are cheap, but they still take time.

Take this example as something i have hard tangible documented proof of:
Suppose you are writing a high performing socket based application. So, following the advice above, what you'd do is you'd instantiate a byte[] for each message you want to send and then null it out as soon as you're done with it.

Thats fine, the byte[] will only live for a very short time, a few ms at the most. Suppose the average message is 16kB in size and you have 20 clients communicating at 80kB/sec each, that's 100 byte arrays being allocated each second. Thats 1600kB a second. Thats a lot of memory.

So, you decide to optimise things properly. You implement a buffer pool so instead of nulling out the references, you instead put the byte[] in a pool where you can retrieve it the next time you need one. As a result, you have 100 buffers allocated which are either in use or in the pool. You have no allocations going on every second.

What effect does this have on memory usage? Amazingly enough, you'll find a reduction of over 10% in memory usage. Why? Because you are not allocating useless crap over and over again.


EDIT:

class MyBadClass
{
public MyBadClass()
{
for (int i =1; i<2000; ++i)
m_MyList.Add(new char[7500]);
}
private
ArrayList m_MyList = new ArrayList;
}

There's nothing inherently wrong with that class. As far as i can see, it's perfect. That bears a striking resemblance to my BufferManager class actually. The big difference is that i store about 1/7th of that, a mere 2 megabytes of byte[] i'm afraid. Amazingly enough, it improves my performance too.

The problem is not that you are storing char[], it's that you may be *needlessly* storing char[]. However that's not a fault of C# and the GC (or any other managed language), thats a fault of a stupid programmer. If that code was in C you'd have the exact same problem.

EDIT: Accidentally wrote Gen1 collections are cheap instead of Gen0

[TheCPUWizard]

Mutant_Fruit,

Not doubting your experience in this area [if you updated you profile to allow PM's some of this could be covered in more detail privately, then broght back to the list, but you have not chosen to do this...] but having worked almost exclusively with managed code for 5+ years, having been a MSFT employee (with direct access to the code base and the developers), I think I can "hold my own".


Ok, so gen 1 collections are cheap, but they still take time.

Yes this is true, but I am talking about GEN0 collections. Objects which do not have any rooted references at time of GEN0 have NO incremental cost [meaning the time is exactly the same to clean up 1 object as it is to clean up 2,000,000,000!]


As far as i can see, it's perfect. That bears a striking resemblance to my BufferManager class actually. The big difference is that i store about 1/7th of that, a mere 2 megabytes of byte[] i'm afraid. Amazingly enough, it improves my performance too.

Based on your statement, it is NOTHING like your class. Your class will use the LOH where fragmentation can (and often does) become a big procblem. It is the number 1 reason for having to periodically re-cycle service (COM++, IIS, etc) programs. My sizes were explicitly chosen to avoid the LOH, but I guess your "experience" did not make that immediately obvious.


Thats 1600kB a second. Thats a lot of memory.

I dont think that is much at all. If that was the only thing going on, and your program otherwised used 1/2 of the default process space for IIS, then this equates to a GC every 4.266 MINUTES. Assuming you have everything else stable (ie no GEN0 objects being promoted), you are about a 0.0390625% load being placed on the application by the GC....


You're now forcing your application to garbage collect twice as frequently.

Frequent GEN0 collections with little or no information being promoted (after application stabilization) has never been a performance problem in any of the .Net applications I have worked on. Many of them involving LOB applications in the financial, insurance, medical verticals....

[Mutant_Fruit]

Yes this is true, but I am talking about GEN0 collections. Aye, my mistake. I meant to write Gen0. Those are cheap. If you follow the Allocate -> Null it -> Allocate it again pattern you are going to increase the rate at which Gen0's happen. Then, as a direct result of this you increase the likelihood of a Gen1/2 as you are inducing more frequent GC's which could easily make the GC decide that it's time to promote objects (but that depends on how your other code i running).


Based on your statement, it is NOTHING like your class. Your class will use the LOH where fragmentation can (and often does) become a big procblem......My sizes were explicitly chosen to avoid the LOH, but I guess your "experience" did not make that immediately obvious.

If i remember correctly, 16kB is less than 85kB, which means my class doesn't use the LOH, exactly the same way as yours doesn't. In fact, i can also prove it with profiling graphs if you so wish.


Assuming you have everything else stable (ie no GEN0 objects being promoted), you are about a 0.0390625% load being placed on the application by the GC....

That's not the point i was making. The point i was making is that (in certain circumstances) when you remove ongoing allocations, you reduce the working set of your application. In those same circumstances if you used the Allocate -> Null -> Allocate again pattern, you'd end up increasing your working set.


Frequent GEN0 collections with little or no information being promoted (after application stabilization) has never been a performance problem

Thats the phrase word right there. That can be a hard thing to guarantee, especially as the application codebase gets larger, especially when it comes to GUI based apps. I fully agree though Gen 0 is cheap, provided you are't bumping objects to Gen1/2. The mid-life crisis situation is a perfect example of when you are allocating too many temporary objects and are artificially promoting objects which should've died in Gen0.

Anyway, the point i'm making is this: Don't try to optimise your allocations by nulling them out unless you have a *very* good reason to do so. The only good reason is that you expect your method to take a considerable amount of time to run and you only need your really large object for the first 3 lines of code.

The only way to truly find out if you need to optimise would be if you profiled your application extensively and noticed that you were allocating X amount of ClassA and you want to reduce that, or that Object Y kept being promoted to Gen1 where it would then die.

EDIT: The Gen0 size is about 2megabytes. Therefore if i were to allocate 1600kB a second from just byte[], i'd be inducing a Gen0 collection every second (nearly). This significantly increases the chances of objects being promoted to Gen 1 as compared to the case where i do not allocate 1600kB a second for buffers.

[_uj]

My point is that "sound programming techniques" uin a GC'ed environment are radically different at fundamental levels than " sound programming techniques" in a programmatic memory management system.

What I feel you have suggested so far is that one should second-guess the GC. One should base one's code on what one think gives the most favourable GC treatment.

Is this what you propose?

[TheCPUWizard]

What I feel you have suggested so far is that one should second-guess the GC. One should base one's code on what one think gives the most favourable GC treatment.

Is this what you propose?

NOt "second guessing" but rather a through understanding of what CG does and does not do. So many times I get asked "I just "released" a bunch of memory, but it still shows my program as very large. People who make statements like this are mis-understanding what the GC is and does.

Also one needs to look carefully at object lifetime. The "mid-life crisis" has already been mentioned in this thread. One should weigh the difference between create/use/destroy and create-use-keep-resuse(...)-destroy design patterns. If state is not being maintained between the uses, then what are the initialization costs?

If you are doing a transactional (in the most general sense) do you group your allocations for objects with longer scopes close to the beginning of the transaction processing phase? If you are allocating a number of objects of radically different sizes/complexity do you order them properly?

Since this is such a big topic, I am actually putting together a complete arcticle on it (will be a few weeks though). In there I have a stress test, that continually allocates 50Kb objects and holds no references to them. I end up with about 900 GEN0 collections per second, but it still is only taking 5-7% of the CPU. This indicates that the GEN0 collection is taking somewhere on the order of 100 microseconds. Granted this is an extreme case (ideal in some respects, worst case in others), but it forms the basis for some very interesting experiments with more complex scenarios.

[mariocatch]

Sorry, off topic. But if you dont mind sharing, where are you employed cpu wizard? You know what you're talking about and was just curious if anyone was taking advantage of that :)

[Mutant_Fruit]

Since this is such a big topic, I am actually putting together a complete arcticle on it (will be a few weeks though).
This is indeed a *huge* topic in which there are *no* hard and fast rules. My advice of reusing byte[] as opposed to constantly reallocating is only useful in certain circumstances, such as programs making heavy use of sockets (more than a dozen send/receives a second).

Implementing the same logic in a program which barely uses sockets could result in the byte[] needlessly being retained in memory when it really should be GCed.

Your pattern of
Allocate
Null out
Reallocate
Null out again
Reallocate again

may work in certain circumstances (i have no examples though) but in the general case it may just serve to promote objects to a higher generation needlessly.

The only hard and fast rule i can think of is this: Don't second guess the garbage collector until you have hard evidence that a certain section of code is responsible for a significant proportion of your allocations. That means profiling your code.

Reading a "rule" from some guy on the internet who says keeping byte[] in a BufferManager class is a great idea and then going ahead and doing that is just stupid, unless you are suffering the exact same problem that guy had.

EDIT: If you were just allocating 50kB segments constantly, you should lodge at 100% cpu usage. It's a bit weird that you're stuck at less than 10%. What OS/.NET version are you on?

[TheCPUWizard]

Sorry, off topic. But if you dont mind sharing, where are you employed cpu wizard? You know what you're talking about and was just curious if anyone was taking advantage of that :)

Mario, this is what private messages are for, please go back and re-read the FAQ's so you know how to enable them.

The answer to your question is in my signature....

[TheCPUWizard]

This is indeed a *huge* topic in which there are *no* hard and fast rules. My advice of reusing byte[] as opposed to constantly reallocating is only useful in certain circumstances, such as programs making heavy use of sockets (more than a dozen send/receives a second).

Implementing the same logic in a program which barely uses sockets could result in the byte[] needlessly being retained in memory when it really should be GCed.

Your pattern of
Allocate
Null out
Reallocate
Null out again
Reallocate again

may work in certain circumstances (i have no examples though) but in the general case it may just serve to promote objects to a higher generation needlessly.

The only hard and fast rule i can think of is this: Don't second guess the garbage collector until you have hard evidence that a certain section of code is responsible for a significant proportion of your allocations. That means profiling your code.

Reading a "rule" from some guy on the internet who says keeping byte[] in a BufferManager class is a great idea and then going ahead and doing that is just stupid, unless you are suffering the exact same problem that guy had.

EDIT: If you were just allocating 50kB segments constantly, you should lodge at 100% cpu usage. It's a bit weird that you're stuck at less than 10%. What OS/.NET version are you on?

Mutant_Fruit,

I completely agree with you in principal, Especially profiling code. When ever I am running and code during the development phase, I have at least Perfmon running on a second monitor showing me what is happening. I am consistantly amazed at the number of people writing code who have no idea how to use perfom, or even what it is. For serious metrics, I prefer the "Ants profiler".

Regarding the "constant allocation", yes the process is pegged at nearly 100% of CPU (measurement aliasing and other items cause it to sometimes only register in the very high 90's). What I was refering to was the GC taking 5-7% of the time [I will go back and edit to make clearer], and this will 900 GC/s per second.

[TheCPUWizard]

Emphasis added

Implementing the same logic in a program which barely uses sockets could result in the byte[] needlessly being retained in memory when it really should be GCed.

Your pattern of
Allocate
Null out
Reallocate
Null out again
Reallocate again

may work in certain circumstances (i have no examples though) but in the general case it may just serve to promote objects to a higher generation needlessly.


I do not see how this is possible unless there are other objects being allocated that have a longer lifecycle and thus have rooted references at the point where the allocations in question are.

The basis of the GC is that "Newer Objects have shorter lifespans". This (oversimplified) constructs like the following defeat that basis (AGAIN OVERSIMPLIFIED)


ClassA instanceA = new ClassA;
...
ClassB instanceB = new ClassB;
...
instanceA = null; // or instanceA goes out of scope
... // KeyPoint
instanceB = null; // or instanceB goes out of scope

In this case instanceB is a "newer object", but it has a longer scope than instanceA. Therefor if a GC occurs during KeyPoint, then instanceB will be promoted to GEN1.

I minor reorganization may be possible where this situation can be avoided.

In my previous response I mentioned perfmon. For my applications I have created 4 custom counters:

TransactionIdle
TransactionActivate
TransactionExecute
TransactionDeactivate


These counters are appropriately maintained by my operational classes. It give a good set of performance (and scalability) metreis not related to GC, but also provides insight into the applications use of memory.

In a single-threaded (or at least single active Transaction) scenario, only GEN0 collections should occur during the Idle, Execute, and Deactivate phases. If (significant) GEN1 collections (or worse and GEN2 collections) occur during these phases, I can often track it back to a correctable mid-life crisis.

If GEN2 collections occur once the program is past the startup phase (granted some will in larger applications, but even then the count should be extremely low compared to GEN1 (IMHO at least 1-2 orders of magnitude), then I look again for root causes.

So my approach is definately profile/measurement based when it comes to addressing GC issues. The flip side, is that having spend significant time programming (conseratively 12,000 hours) in a .Net environment, I am aware of patterns that tend to lead to problems, and patterns that tend to work well, as well as the use cases (especially with my overall architectural style).

But as you said, it is impossible to crreate a set of hard and fast general rules that apply to all circumstances...

[Mutant_Fruit]

ClassA instanceA = new ClassA;
...
ClassB instanceB = new ClassB;
...
instanceA = null; // or instanceA goes out of scope
... // KeyPoint
instanceB = null; // or instanceB goes out of scope


What about all the other objects you have? Suppose 1/4 of your allocations are from the Allocate/Null/Reallocate pattern. That means you are going to induce Gen0 collections 25% faster as you have 25% more allocations. This means that you will bump up objects to Gen1 which would normally have fallen out of scope by the time a Gen0 would have been performed. Gen1's are more expensive than Gen0, so you'd actually reduce performance.

Secondly, 95% of objects in your typical application are small, and methods complete quickly so you'll drop all method level references very quickly whether or not you null them out.

The final thing i have to say is that GC's only occur when you allocate a new object and don't have space or you call GC.Collect(). As a result, you're more likely to have a GC on the call to Classb obj = new ClassB(); then at the point where you null out the reference.

Different applications have wildly different allocation patterns, which need different techniques for maximizing performance. But in 95% of applications i'd say you won't need specialised logic to solve GC issues. Those 5% of cases would primarily be server applications dealing with large amounts of traffic.

[TheCPUWizard]

Taking the points out of order....
Different applications have wildly different allocation patterns, which need different techniques for maximizing performance. But in 95% of applications i'd say you won't need specialised logic to solve GC issues. Those 5% of cases would primarily be server applications dealing with large amounts of traffic.

I agree with this. And expecially in the case of applications where you want continous uptime on the scale of weeks/months/years (implying very careful attention to the LOH). About 80% of the applications I am involved with (the kind that warrant the services of an outside consultant) fall into this category. I rarely see codebases of under 100,000 lines

FYI: the project I am involved in right now has 2500+ classes in 175+ assemblies totalling over 4 million lines of code). The highest throughput application was an online betting system which had upwards of 2 million hits per minute at peak (immediately before the start of a large interntional sporting event). Granted that application was on a huge farm....

If you are only going to be writing small applications, then yes you can avoid this issue 99% (giving it a higer number than even you ;) ] of the time. However if you are involved in all types of projects, then the habits and design patterns that can make or break a large application tend to filter down to the smaller ones. Although the benefits may not be larg or even significant, there is no downside.


What about all the other objects you have? Suppose 1/4 of your allocations are from the Allocate/Null/Reallocate pattern. That means you are going to induce Gen0 collections 25% faster as you have 25% more allocations. This means that you will bump up objects to Gen1 which would normally have fallen out of scope by the time a Gen0 would have been performed. Gen1's are more expensive than Gen0, so you'd actually reduce performance.

If you are following the patterns I mentioned earlier (Last alloc=shorted life) then there will be one set of objects which gt promoted from GEN0 to GEN1, wether this happens on the first, second, or thousandth new allocation really does not matter.

Additionally the promotion of objects to GEN1 causes three potential types of impact (in increasing order of severity):

The cost of moving the object itself.
The cost of updating references to the moved object
The probability that this will cause a GEN1 collection.


I have never seen the first point cause a measurable impact. The second point typically occurs when developers design a class which will typically be used in a collection, and put references to the collection into the object itself...


Secondly, 95% of objects in your typical application are small, and methods complete quickly so you'll drop all method level references very quickly whether or not you null them out.

I dont know if I agree with this assesment. For most operations there is at least one method which has a duration as long as the overall transaction. What matters (in my mind) the most, is how many methods are invoked from within the bethod your are analyzing. Their size and complexity directly relates to the probability that a GC will occur while your method is action.


The final thing i have to say is that GC's only occur when you allocate a new object and don't have space or you call GC.Collect(). As a result, you're more likely to have a GC on the call to Classb obj = new ClassB(); then at the point where you null out the reference.

Agreed. (With the cavaet that GC.Collect() should never be called except in some very rare and specialized circumstances. My point was that if additional code was called which did significant allocations at the point I labeled, then instanceB would be promoted. If there was no need for continuance of state for instanceB, then releasing the reference prior to calling the code would prevent collection in the event that the called code did an allocation which triggered a GEN0 GC.