Using custom object arrays

[twinbeeuk]

I use the latest .NET framework. I've hunted high and low for a full solution to this. Coming from a C/C++ background, the below should print out "50" twice, but unfortunately, the second 'Dog' one crashes.

One 'solution' seems to involve 'further instantiation' by using the code: d[10] = new dog(); just before d[10].n=50, but then sounds painful because then if I want access to all 1000 elements, I'll need to use a For loop to instantiate each one individually. Also that wouldn't explain why int gets away with it, but dog doesn't.

Any thoughts? The error encountered is: "NullReferenceException was unhandled".

(On a vaguely related note, can I allocate structs on the heap?)

using System;
class dog { public int n; }

class Program
{
	static void CreateInt() {
		int[] d = new int[1000];
		d[10] = 50; Console.WriteLine(d[10]);
	}
	static void CreateDog() {
		dog[] d = new dog[1000];
		d[10].n = 50;
		Console.WriteLine(d[10].n);
	}
	static void Main(string[] args)
	{
		CreateInt();
		CreateDog();
		Console.ReadLine();
	}
}

[BigEd781]

1. Don't use arrays like that, use a collection class (i.e., List<T>)
2. You declare an array with enough storage capacity for 1000 dog objects, but that doesn't mean they are automatically initialized. The default value for a reference type is null, so of course you get a NullReferenceException when you try to call a method on it. ValueTypes cannot be null, so their default value is different (in your case, the deffault value for an int is 0).

[TheGreatCthulhu]

If we exclude pointers, which are only allowed in a so-called "unsafe" context, C# types are divided into value types (custom structs, then types like int, float, double, char, bool...), and reference types (like classes), whose objects are accessed via references.
Unlike C++ where variables are "initialized" with whatever junk was in memory, here all types have predefined default values. For ints its 0, for reference types it is null.
See: Types (C# Reference), Default Values Table (C# Reference)

AFAIK, you don't have control over where a struct would be allocated, this is left for the environment to decide (and it may chose the heap for a particular struct). This my seem a bit strange for someone with a C/C++ background, but believe me, the environment and the garbage collector do a great job.

EDIT: Correction - about the stack/heap stuff: I misremembered this article - it points out that it's only an implementation detail. It's a good read, and there's also part two.

EDIT 2: Or I didn't!

Of course, the simplistic statement I described is not even true. As the MSDN documentation correctly notes, value types are allocated on the stack sometimes.

Refer to the article (1) for more details.

[twinbeeuk]

Okay thanks both. Before I go any further, I'll say that I need optimal speed for array creation/access/writing and potentially gigabytes of elements in arrays.

"List<T>" sounds interesting - thanks I'll try that (however is there a performance hit?).

Instantiating the object elements using a For loop as below is not only painfully slow to create, but also is 4x greater in memory usage than expected (4 bytes x 50000000 = 200 megabytes, but C# uses 800 megs):

static void CreateDog() {
	dog[] d = new dog[50000000];
	for (int i = 0; i < 50000000; i++)  {
		d[i] = new dog();
	}
	....
}

Any way around this when using classes?

Is the 'unsafe' keyword an option here (back to pointers), or maybe I will need to use structs rather than classes?

And before anyone says: "just use C/C++", the problem is I need C#'s reflection feature. I have to say though, the compilation speed of C# is a breath of fresh air compared to C/C++

[BigEd781]

Based upon what information are you assuming that a 'dog' object would require only 4 bytes of memory? The fact that it only contains a single int field does not mean it will require only 4 bytes of memory.

Second, how are you profiling memory usage? Are you accounting for the memory needed for other aspects of your program?

Third, do you expect your program to run on a 32 bit machine at all? Don't assume that you can just allocate an array that requires GB's of memory. There is a limit on how much memory you can allocate in your program and it's around 2GB.

You seem to be a bit confused on how to properly estimate/test the performance requirements of a program. Also, since this is obviously meaningless code, we have no clue as to what you are actually trying to accomplish. I can almost guarantee that there are better ways to solve your particular problem, but you seem to have already made up your mind about the solution and now are just trying to get it to work by whatever means necessary.

[twinbeeuk]

The fact that it only contains a single int field does not mean it will require only 4 bytes of memory.

In C/C++ classes you'll find it does. I understand C# is probably using the extra memory for class overhead, and that's totally fine by me. But understand I need lean and mean

Second, how are you profiling memory usage? Are you accounting for the memory needed for other aspects of your program?

The task manager only. And I'm familiar with its quirks, but suffice to say I know the kind of patterns that are produced, have run it multiple times, and I'm sure others will agree that apx. 4x the memory is being used in this case than what would be expected from what is (essentially) an int array. I'm sure you will find the same if you put it to the test.

Are you accounting for the memory needed for other aspects of your program?

The program is "as above" - it's a test program. No more than that at the moment.

Third, do you expect your program to run on a 32 bit machine at all? Don't assume that you can just allocate an array that requires GB's of memory. There is a limit on how much memory you can allocate in your program and it's around 2GB.

Yes, I'm well aware of that limit on 32 bit platforms. I only said 'potentially' gigabytes.

I can almost guarantee that there are better ways to solve your particular problem,

I haven't discounted the List<T> idea. I just want to get as basic as I possibly can before I further explore the language. And let me know if there's a performance hit for List<T> if you know of one.

[BigEd781]

Well, the first faulty assumption regarding memory is that creating a dog is essentially the same as creating an int. It's not. First off you are creating a reference (pointer) as well as the object, so even if the reference is the native pointer size your are already at double your initial estimate. I know how C++ acts, but this is C#, a completely different language. Comparing the two is erroneous and akin to comparing apples and elephants. The syntax is really the only meaningful similarity between the two languages.

As far as using List<T> over an array, they perform very well. However, if you really want to know, just profile it. Make sure that you account for the JIT compilation process on the first execution of your code though.

I ran a test and got the same numbers you did. All I can say is that you can't assume a class with a single int is going to be 4 bytes. That is not documented or specified anywhere in the language spec, and if there were even an extra byte required for meta information you have now doubled the amount of memory needed to create 50,000,000 of your objects. It doesn't take much per object overhead for memory consumption to balloon when you are talking about that many objects.

My advice; if you really need that level of performance (in space) and predictability, then C# isn't the language you should be using for this project. Use a language that gives you fine grain and deterministic control over resources.

Also, when I said "I can almost guarantee that there are better ways to solve your particular problem", I didn't mean what collection class to use, I meant a different overall strategy/architecture. What high level problem are you trying to solve?

EDIT: I have to thank you for giving me something interesting to respond to though

[bixel]

I'm just a newb, but maybe some form of bit-wise logic programming can be done BEFORE determining if dog[] d = new dog[50000000] OR dog[] d = new dog[5]

if so - you can check for space, alert the user, etc...

[TheGreatCthulhu]

Article: How much memory does my .NET application use?

P.S. You said you're interested in reflection - this uses metadata, and metadata consumes some memory too. The important thing to remember about C# is that all (or the vast majority of) the general coding principles and best practices still apply. For a C++ dev, it may be hard to trust the GC, but generally speaking you should leave all the memory management/cleanup to it. You're also going to love the compiler error messages because most of the time they are much more meaningful and on-the-spot than their C++ counterparts.
And what BigEd781 said: it would really help if you were more specific about your requirements.

[BigEd781]

I'm just a newb, but maybe some form of bit-wise logic programming can be done BEFORE determining if dog[] d = new dog[50000000] OR dog[] d = new dog[5]

if so - you can check for space, alert the user, etc...

Well, there's really no point in testing the performance of a five element array. You know that's not going to be a problem, you need to test for your worst case scenario.

[twinbeeuk]

It wasn't an assumption - just a comparison. I've already said how I realize that C# seems to appear to create class overhead, and I'm fine with that. Heck, languages like PHP used around 16x the space for primitive types if I recall correctly (from when I checked years ago, maybe it's improved since).

I'm sure they have VERY good reason to create that overhead. Please don't think I'm saying "wow, how can they waste all that memory, the C# dev team must be terrible coders".

What I *am* saying is that I am looking for a solution, and your List<T> might fit the bill. However, so might using structs (which *do* use 4 bytes per struct instance for my dog example), or even unsafe, so I can go back to using pointers.

My advice; if you really need that level of performance (in space) and predictability, then C# isn't the language you should be using for this project. Use a language that gives you fine grain and deterministic control over resources.

From my research on the web, I'm fairly sure C# can cope just fine with big workloads, including heavy graphics-based apps which is what I am personally interested in.

Use a language that gives you fine grain and deterministic control over resources.

I need reflection and preferably .NET's GUI framework. I've explored countless alternatives, so don't worry about that.

Also, when I said "I can almost guarantee that there are better ways to solve your particular problem", I didn't mean what collection class to use, I meant a different overall strategy/architecture.

These things we're talking about form the fundamentals of C#. If the fundamentals don't work well (and I'm sure they will once I figure it out), then the rest won't.

EDIT: I have to thank you for giving me something interesting to respond to though

Glad to hear I'm hoping to create a realtime raytracer fyi. GPU accelerated too.

[Mutant_Fruit]

The minimum size of a "class" type is about 16 bytes for a class with no parameters. It's an estimate because i'm too lazy to look up the docs. It will vary between 32bit and 64bit systems. If you want bare metal memory usage use a struct. Consider this declaration

List<Custom> list = new List<Custom> (500);

with this class:

public class Custom { int Value; }

This will use about 20 bytes per instantiated element. If this was a struct:

public struct Custom { int Value; }

This would use exactly 4 bytes per instance, so roughly (4 * 500) bytes for the entire list. MSDN should have further reading on the difference between a class and a struct if you're interested.

[TheGreatCthulhu]

I'm hoping to create a realtime raytracer fyi. GPU accelerated too.

Well, I hope you realize that you need to have a good grasp of all the important C# language concepts if you want to create something like that. (And extrapolating your knowledge of C++ and general programming to C# cant hurt, where appropriate of course.)
Realtime means high performance, and they say that most of the performance comes from having a good design, and only then from optimizations, so keep that in mind because if the design is poor you can optimize all you like, and all for nothing.
An important thing for a C++ programmer to note about C# is that structs and classes aren't (nearly) one and the same thing, as they are in C++, but are semantically different, which has important consequences, including your design decisions - so learning about C#'s type system is a good way to start (more about C# structs here, in addition to the links above).


And don't get so hung up on List<T>: when BigEd781 suggested to use the List<T> generic class, it was just as a general coding practice, not as a better alternative for arrays, performance-wise (which it certainly isn't). It's just a collection class, in some ways similar to, er... I dont know - STL vector in C++.

[twinbeeuk]

Thanks both. Yes, I'll probably work on something simpler like an image processor first, just to get the grasp of C# more.

For the record, the extra bits that classes take up are a fixed amount, therefore (as implied by above), for large classes, it won't make much difference memory wise. However speed-wise, they're relatively slow to create compared to structs as noted previously.

[BigEd781]

Yeah, creating 50,000,000 wasn't very fast in my test either, but that can be dealt with. Can you create all of these objects up front, i.e., a one time cost? Do you really need to create 50,000,000 objects at once, or can you create them on an as-needed basis?