Quote Originally Posted by TSYS
Hmm, I see what you mean. Could the producer keep a copy of each record it puts in the pipe, deleting it when the consumer indicates it has been received and processed? That way, if the pipe collapses (because the consumer has died), the producer can re-stuff the pipe once it's been re-established.
good idea! but where should i put the record copies? if they're in non-persistant memory, they're lost if producer AND consumer die at once... if i put them into a file, i would have to remove them from the file, which is almost the same operations as when each record uses it's own file.

Quote Originally Posted by TSYS
Another possibility to ensure data stability is to borrow a trick from the aircraft industry: run multiple copies of each thread and have a "sanity check" mechanism to determine which producer is correct and which consumer is correct. This gets complicated, but you can extend it to have several producer / consumer threads running on several computers....
i think i will first check if there are enough system resources to do the one-file-per-record thing because this sounds like "work"... i don't need "rocket science" in the moment

@krzemo: i was thinking about using a database as buffer between producer and consumer, but that's too complicated and it's also too unsafe for me because i'm not sure how the DB reacts on low system resources.
i must be absolutely sure that my producer can immediately put the records somewhere where they can not get lost.


@