Managing many objects in memory

[markyoung1984]

I am creating an application that goes through about 400 objects one by one calling functions in a loop. This executes every 10 minutes. When the program is initially started the object properties are collected from a database and then there is no further interaction with the database, until the program terminates and data from the objects is written to the database.

At the moment I am holding these 400 objects in an array in memory and calling and modifying each one at whatever stage in the loop I am at.

I was wondering if there was a more efficient way to do this. Should I keep reading and writing results to the database instead of keeping them in memory or is there another way?

[Mutant_Fruit]

that's fine,. There's no need to to optimise something like that unless you *need* it to be faster and each individual object takes a long time to process or each individual object is massive (>500kB).

[dannystommen]

until the program terminates and data from the objects is written to the database.


Not really an answer to your question, but I doubt this is a 'safe' way of handling your data. What happens when the computer/server crashes, then the data is not written to the database, and you possibly lost changes made to the data.

[monalin]

My suggestion is that you save every modification to the database when it is made. Don't wait till the application shuts down to save them, like danny said, you will loose data at some point otherwise.

Whether or not to store them in memory is a different story. That depends on how long it takes to load the required data to perform the modifications you're making. Make sure that each time you run it you need to modify all 400 objects, if you're only modifying 40 of the 400 then you could probably optimize it there.

[vcdebugger]

Better to write to the database once every 100 objects are processed instead of waiting till the application ends to be 'safer' as told earlier by others.

[monalin]

Better to write to the database once every 100 objects are processed instead of waiting till the application ends to be 'safer' as told earlier by others.

Why wait every 100? Nothing is stopping it from crashing at item 50, you're no better off doing it every 100 objects than you are updating at item 400.