Kaikyro
September 15th, 2011, 06:08 AM
Hey guys, I'm looking for some high level advice,
I'm heading up development on an existing specialist in-memory database. One of the things we are looking into for improving concurrent performance is making the user requests execute on separate threads, the biggest problem here is (the typical database problem) that they all will only ever access the same set of data objects.
The core objects form a tree format, all containers used do not sort the objects and store pointers to the child objects which in turn store pointers to their children objects - This prevents any kind of object re-ordering in memory which some stl containers do (according to my limited understanding).
I understand the multithreading basics but almost all samples and tutorials use threads to execute isolated tasks which from my understanding will only be partially relevant in that I will create a worker class but I need to focus on the most efficient way to share these complex data structures to readers and writer threads.
How I'm presently planning on approaching this is by adding a semaphore counter on each tree leaf in the data structure so I can lock a tree branch from that point onwards (through subsequent branches) for a write operation (building the branches and data value at the end). But I still need to learn exactly how this would work.
Can anyone provide some high level advice for where I'm at presently in the design stage? I'm looking for a nudge in the right direction and as the whole application has already been developed without forethought for multithreading it will be quite a hard task as I understand it.
Edit: I should add that we use Visual Studios 2008 with the Intel C++ compiler and will be looking at using Intel Threading Building Blocks for their thread-safe containers.
I'm heading up development on an existing specialist in-memory database. One of the things we are looking into for improving concurrent performance is making the user requests execute on separate threads, the biggest problem here is (the typical database problem) that they all will only ever access the same set of data objects.
The core objects form a tree format, all containers used do not sort the objects and store pointers to the child objects which in turn store pointers to their children objects - This prevents any kind of object re-ordering in memory which some stl containers do (according to my limited understanding).
I understand the multithreading basics but almost all samples and tutorials use threads to execute isolated tasks which from my understanding will only be partially relevant in that I will create a worker class but I need to focus on the most efficient way to share these complex data structures to readers and writer threads.
How I'm presently planning on approaching this is by adding a semaphore counter on each tree leaf in the data structure so I can lock a tree branch from that point onwards (through subsequent branches) for a write operation (building the branches and data value at the end). But I still need to learn exactly how this would work.
Can anyone provide some high level advice for where I'm at presently in the design stage? I'm looking for a nudge in the right direction and as the whole application has already been developed without forethought for multithreading it will be quite a hard task as I understand it.
Edit: I should add that we use Visual Studios 2008 with the Intel C++ compiler and will be looking at using Intel Threading Building Blocks for their thread-safe containers.