Multiple-readers, single-writer synchronization lock between processes
There's a well-known algorithm that employs readers/writer lock synchronization between threads of a single process on a Windows platform using pure WinAPIs/C++: http://www.glennslayden.com/code/win...er-writer-lock
In my case I need to do this between several processes, i.e. the writer is in one process, and readers are in other processes. Any idea how to do that?
PS. I need this for an already developed project, so I can't use anything other than C++/MFC or pure WinAPIs. In other words I can't use Boost or the like extensions.
Re: Multiple-readers, single-writer synchronization lock between processes
Originally Posted by Igor Vartanov
Oh, actually named semaphore could be used for counting readers. And named event to indicate writing in progress.
Thanks, Igor. Replacing the critical section with global named mutex should work. The issue is in the counter of readers. I actually thought to use named semaphores for it too, but couldn't come up with a working code. The issue is that semaphores count down and this algorithm needs to count up & down.
So can I ask you to demonstrate what you meant in code?
Re: Multiple-readers, single-writer synchronization lock between processes
The idea actually was for reader to inverse the logic behind semaphore object and use ReleaseSemaphore(by 1) to indicate read operation start and WaitForSingleObject to indicate read completion. The semaphore should be created with maximum allowed count but initial count is zero. Of course, reader should catch the mutex and immediately release it after incrementing semaphore. Symmetrically, writer should not grab the mutex until semaphore becomes non-signalled.
One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
Re: Multiple-readers, single-writer synchronization lock between processes
Originally Posted by Igor Vartanov
The idea actually was for reader to inverse the logic behind semaphore object and use ReleaseSemaphore(by 1) to indicate read operation start and WaitForSingleObject to indicate read completion. The semaphore should be created with maximum allowed count but initial count is zero. Of course, reader should catch the mutex and immediately release it after incrementing semaphore. Symmetrically, writer should not grab the mutex until semaphore becomes non-signalled.
One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
But how would you know that there're no readers reading the data?
Re: Multiple-readers, single-writer synchronization lock between processes
Thanks, Igor.
Code:
DWORD waitRes = WaitForSingleObject(m_hSemRead, 0); // decrement semaphore counter
BOOL res = WAIT_TIMEOUT == waitRes; // here not sure about the exact retcode :)
Yes, I'm not sure about this one either. That's what I meant in my previous post that a semaphore object was never intended as just a counter. Although I do appreciate your "bold" approach to it.
One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
Yes, that becomes an issue in case I put your code into several processes. For instance, if one reader crashes and doesn't decrement the semaphore, it will indefinitely deadlock the writer.
Also this, I guess is just nitpicking:
Code:
// loop until last reader gone
waitRes = WaitForSingleObject(m_hSemRead, 0);
if (WAIT_OBJECT_0 == waitRes)
Sleep(10);
else if (WAIT_TIMEOUT == waitRes)
return TRUE;
else
return FALSE;
Re: Multiple-readers, single-writer synchronization lock between processes
Actually, the initial code is too dirty. Apparently, 2 AM isn't the best hour for writing clean code.
Attached please find the reviewed and debugged variant. (.sln file is not a normal solution but just a debugging session)
The change: writer should release semaphore too, like reader does, otherwise wait distorts the semaphore counter. BTW, this way it peeks into the semaphore counter as well.
Code:
BOOL EnterWrite()
{
PutLog(CString(TEXT("+ ENTER + ")) + TEXT(__FUNCTION__));
BOOL res = ResetEvent(m_hEvtNoRead); // prevents new reads
_ASSERT(res);
DWORD waitRes = 0;
do
{
// loop until last reader gone
LONG count = 0;
res = ReleaseSemaphore(m_hSemRead, 1, &count);
_ASSERT(res);
waitRes = WaitForSingleObject(m_hSemRead, 0);
if (count)
Sleep(10);
else if (WAIT_OBJECT_0 == waitRes)
{
res = TRUE;
break;
}
else
{
res = FALSE;
break;
}
} while (TRUE);
PutLog(CString(TEXT("- RETURN - ")) + CString(__FUNCTION__) + TEXT(", res = ") + ToString(res));
return res;
}
BOOL LeaveWrite()
{
PutLog(CString(TEXT("+ ENTER + ")) + TEXT(__FUNCTION__));
BOOL res = SetEvent(m_hEvtNoRead); // release waiting readers
_ASSERT(res);
PutLog(CString(TEXT("- RETURN - ")) + CString(__FUNCTION__) + TEXT(", res = ") + ToString(res));
return res;
}
Last edited by Igor Vartanov; June 18th, 2012 at 08:56 AM.
Re: Multiple-readers, single-writer synchronization lock between processes
Thanks again. I truly appreciate your unconventional approach to using semaphores.
I reworked your code to use two separate processes (for readers and a writer.) Sorry, I can't write concise solutions like you do, so I opted for a default console app template instead. To test it run the writer first (only one instance is allowed) and then start as many readers as you want. I also removed your log file output and instead redirected it to the screen to achieve faster output (to test synchronization mechanism) and also to give a tester visual clue that threads are running.
I had to remove your CRT I/O file functions in favor of lower-level APIs. Maybe I don't know much about fopen() but it seems to have its own built-in synchronization, which mixed with the results of the synchronization mechanism in this example.
Then, first off, let me correct a bug. When you create m_hEvtNoRead event, it must be set to signaled state. Otherwise the first reader will deadlock without the first writer call.
Then a couple of issues that I ran into (or maybe I should name them as questions):
1. The sleep(10) part of the EnterWrite() method seems to be a bottleneck:
Code:
waitRes = WaitForSingleObject(m_hSemRead, 0);
if (count)
Sleep(10);
but if I rewrite it into:
Code:
waitRes = WaitForSingleObject(m_hSemRead, 10);
if (count)
/*Sleep(10)*/;
it really speeds things up, but unfortunately I get occasional "read fail". Why do you think would that be?
PS. While typing this reply, I let the "writer" run on the background with Sleep(10) line... And it also gave me "read fail" after about 5-10 minutes of continuous running... There still must be an issue somewhere in the this lock...
After a quick overview it seems that the part of the code that changes the semaphore count must be enclosed in its own critical section. Don't you think?
2. The second issue is more challenging to fix. As I brought up before, if one of the readers crashes inside its critical section, the writer mechanism will also deadlock. To illustrate, search for the following line in the attached solution, and uncomment it:
Code:
#define INCLUDE_READER_CRASH //Uncommect this line to introduce an artifical "Reader" crash
This will make the reader process crash after 200 reads. When that happens watch the writer ... it will deadlock.
Re: Multiple-readers, single-writer synchronization lock between processes
I think I fixed the issue #1 by enclosing the semaphore in a critical section. Attached is an updated project. There's still the issue #2 remaining though ...
Re: Multiple-readers, single-writer synchronization lock between processes
unconventional approach to using semaphores
Well, the idea was got from MSDN article after reading about start with zero counter. So it's not that unconventional.
let me correct a bug. When you create m_hEvtNoRead event, it must be set to signaled state. Otherwise the first reader will deadlock without the first writer call.
It might be a feature, as long as without first write it might be senseless to read the void. You know this better, as I'm totally unaware of your ultimate design.
The sleep(10) part of the EnterWrite() method seems to be a bottleneck
I said you before that all my experience with synchronization was mostly about dealing with processes where hundreds of milliseconds never matter. Typically for me in business logic keeping precedence matters much more than super timings.
And in case when few milliseconds really is a bottleneck, I'd say that either you need some other synchronization solution, or, which is more probable, your synchronized multi-process reader-writer approach itself is a bottleneck.
but unfortunately I get occasional "read fail". Why do you think would that be?
Like any other pure architectural thing it needs to be analyzed for flaws (code review, sequence diagrams, etc.) and properly debugged (by offline logs or interactively). Seems like I just was able to invest an idea, but have no time to purify it, sorry. Maybe other guys here will help.
* The Best Reasons to Target Windows 8
Learn some of the best reasons why you should seriously consider bringing your Android mobile development expertise to bear on the Windows 8 platform.