Multiple-readers, single-writer synchronization lock between processes
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 28

Thread: Multiple-readers, single-writer synchronization lock between processes

  1. #1
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Multiple-readers, single-writer synchronization lock between processes

    There's a well-known algorithm that employs readers/writer lock synchronization between threads of a single process on a Windows platform using pure WinAPIs/C++:
    http://www.glennslayden.com/code/win...er-writer-lock

    In my case I need to do this between several processes, i.e. the writer is in one process, and readers are in other processes. Any idea how to do that?

    PS. I need this for an already developed project, so I can't use anything other than C++/MFC or pure WinAPIs. In other words I can't use Boost or the like extensions.

  2. #2
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    Named mutex of a name in global namespace (single reader/writer access). Or a dedicated read/write broker.
    Last edited by Igor Vartanov; June 16th, 2012 at 06:31 AM.
    Best regards,
    Igor

  3. #3
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    Oh, actually named semaphore could be used for counting readers. And named event to indicate writing in progress.
    Best regards,
    Igor

  4. #4
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Quote Originally Posted by Igor Vartanov View Post
    Oh, actually named semaphore could be used for counting readers. And named event to indicate writing in progress.
    Thanks, Igor. Replacing the critical section with global named mutex should work. The issue is in the counter of readers. I actually thought to use named semaphores for it too, but couldn't come up with a working code. The issue is that semaphores count down and this algorithm needs to count up & down.

    So can I ask you to demonstrate what you meant in code?
    Last edited by ahmd; June 16th, 2012 at 04:30 PM.

  5. #5
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    The idea actually was for reader to inverse the logic behind semaphore object and use ReleaseSemaphore(by 1) to indicate read operation start and WaitForSingleObject to indicate read completion. The semaphore should be created with maximum allowed count but initial count is zero. Of course, reader should catch the mutex and immediately release it after incrementing semaphore. Symmetrically, writer should not grab the mutex until semaphore becomes non-signalled.

    One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
    Best regards,
    Igor

  6. #6
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Quote Originally Posted by Igor Vartanov View Post
    The idea actually was for reader to inverse the logic behind semaphore object and use ReleaseSemaphore(by 1) to indicate read operation start and WaitForSingleObject to indicate read completion. The semaphore should be created with maximum allowed count but initial count is zero. Of course, reader should catch the mutex and immediately release it after incrementing semaphore. Symmetrically, writer should not grab the mutex until semaphore becomes non-signalled.

    One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
    But how would you know that there're no readers reading the data?

  7. #7
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    No readers is indicated by semaphore non-signalling state.
    Last edited by Igor Vartanov; June 17th, 2012 at 04:57 PM.
    Best regards,
    Igor

  8. #8
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Let's put it into code:
    Code:
    //For simplicity I'm writing it for a single process
    class MultiReaderSingleWriter
    {
    private:
    	CRITICAL_SECTION m_csWrite;
    	CRITICAL_SECTION m_csReaderCount;
    	//long m_cReaders;
    	HANDLE m_semCounter;
    	HANDLE m_hevReadersCleared;
    
    public:
    	MultiReaderSingleWriter()
    	{
    		m_semCounter = CreateSemaphore(NULL, 0, 1000, NULL);
    		InitializeCriticalSection(&m_csWrite);
    		InitializeCriticalSection(&m_csReaderCount);
    		m_hevReadersCleared = CreateEvent(NULL,TRUE,TRUE,NULL);
    	}
    
    	~MultiReaderSingleWriter()
    	{
    		WaitForSingleObject(m_hevReadersCleared,INFINITE);
    		CloseHandle(m_semCounter);
    		CloseHandle(m_hevReadersCleared);
    		DeleteCriticalSection(&m_csWrite);
    		DeleteCriticalSection(&m_csReaderCount);
    	}
    
    
    	void EnterReader(void)
    	{
    		EnterCriticalSection(&m_csWrite);
    		EnterCriticalSection(&m_csReaderCount);
    		//if (++m_cReaders == 1)
    		//	ResetEvent(m_hevReadersCleared);
    
    		ReleaseSemaphore(m_semCounter);   //Can we call it here with the count being 0?
    		//So how do you check the semaphore count here?
    		
    		LeaveCriticalSection(&m_csReaderCount);
    		LeaveCriticalSection(&m_csWrite);
    	}
    
    	void LeaveReader(void)
    	{
    		EnterCriticalSection(&m_csReaderCount);
    		//if (--m_cReaders == 0)
    		//	SetEvent(m_hevReadersCleared);
    
    		WaitForSingleObject(m_semCounter, 0);
    		//Same problem here??
    
    		LeaveCriticalSection(&m_csReaderCount);
    	}
    
    	void EnterWriter(void)
    	{
    		EnterCriticalSection(&m_csWrite);
    		WaitForSingleObject(m_hevReadersCleared,INFINITE);
    	}
    
    	void LeaveWriter(void)
    	{
    		LeaveCriticalSection(&m_csWrite);
    	}
    };
    It doesn't seem to work. I don't know that semaphores can be used as counters? I might be wrong. If so, correct me.

  9. #9
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    The code below is just an idea that needs to be proved, or disproved, or corrected. And no, I never did anything like this before.
    Code:
    #include <windows.h>
    #include <crtdbg.h>
    
    class MultiReaderSingleWriter
    {
    	HANDLE m_hSemRead;  // counts active readers
    	HANDLE m_hEvtNoRead;  // prevents read operation
    
    public:
    	MultiReaderSingleWriter()
    	{
    		m_hSemRead = CreateSemaphore(NULL, 0, 1000, NULL);
    		m_hEvtNoRead = CreateEvent(NULL, TRUE, FALSE, NULL);
    	}
    
    	~MultiReaderSingleWriter()
    	{
    		CloseHandle(m_hSemRead);
    		CloseHandle(m_hEvtNoRead);
    	}
    
    	BOOL EnterRead()
    	{
    		DWORD waitRes = WaitForSingleObject(m_hEvtNoRead, INFINITE); // blocks until write completion
    		_ASSERT(WAIT_OBJECT_0 == waitRes);	
    		LONG count = 0;
    		BOOL res = ReleaseSemaphore(m_hSemRead, 1, &count); // increments semaphore counter
    		// here you can log count to see internal semaphore counter before you incremented it
    		_ASSERT(res);
    		return res;
    	}
    
    	BOOL LeaveRead()
    	{
    		DWORD waitRes = WaitForSingleObject(m_hSemRead, 0); // decrement semaphore counter
    		BOOL res = WAIT_TIMEOUT == waitRes;  // here not sure about the exact retcode :)
    		_ASSERT(res);
    		return res;
    	}
    
    	BOOL EnterWrite()
    	{
    		BOOL res = SetEvent(m_hEvtNoRead); // prevents new reads
    		_ASSERT(res);
    		DWORD waitRes = 0;
    		do
    		{
    			// loop until last reader gone
    			waitRes = WaitForSingleObject(m_hSemRead, 0);
    			if (WAIT_OBJECT_0 == waitRes)
    				Sleep(10);
    			else if (WAIT_TIMEOUT == waitRes)
    				return TRUE;
    			else
    				return FALSE;
    		} while (TRUE);
    
    		return FALSE;
    	}
    
    	BOOL LeaveWrite()
    	{
    		BOOL res = ResetEvent(m_hEvtNoRead); // release waiting readers
    		_ASSERT(res);
    		return res;
    	}
    };
    Best regards,
    Igor

  10. #10
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Thanks, Igor.
    Code:
    DWORD waitRes = WaitForSingleObject(m_hSemRead, 0); // decrement semaphore counter
    BOOL res = WAIT_TIMEOUT == waitRes;  // here not sure about the exact retcode :)
    Yes, I'm not sure about this one either. That's what I meant in my previous post that a semaphore object was never intended as just a counter. Although I do appreciate your "bold" approach to it.

    One potential danger thing to be confirmed that died reader never blocks the whole mechanism.
    Yes, that becomes an issue in case I put your code into several processes. For instance, if one reader crashes and doesn't decrement the semaphore, it will indefinitely deadlock the writer.

    Also this, I guess is just nitpicking:
    Code:
    // loop until last reader gone
    waitRes = WaitForSingleObject(m_hSemRead, 0);
    if (WAIT_OBJECT_0 == waitRes)
    	Sleep(10);
    else if (WAIT_TIMEOUT == waitRes)
    	return TRUE;
    else
    	return FALSE;
    Wouldn't it be better to do:
    Code:
    waitRes = WaitForSingleObject(m_hSemRead, 10);
    and drop Sleep(10); to save those 10ms?

  11. #11
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    Actually, the initial code is too dirty. Apparently, 2 AM isn't the best hour for writing clean code.

    Attached please find the reviewed and debugged variant. (.sln file is not a normal solution but just a debugging session)

    The change: writer should release semaphore too, like reader does, otherwise wait distorts the semaphore counter. BTW, this way it peeks into the semaphore counter as well.
    Code:
        BOOL EnterWrite()
        {
            PutLog(CString(TEXT("+ ENTER + ")) + TEXT(__FUNCTION__));
            BOOL res = ResetEvent(m_hEvtNoRead); // prevents new reads
            _ASSERT(res);
    
            DWORD waitRes = 0;
            do
            {
                // loop until last reader gone
                LONG count = 0;
                res = ReleaseSemaphore(m_hSemRead, 1, &count);
                _ASSERT(res);
                waitRes = WaitForSingleObject(m_hSemRead, 0);
                if (count)
                    Sleep(10);
                else if (WAIT_OBJECT_0 == waitRes)
                {
                    res = TRUE;
                    break;
                }
                else
                {
                    res = FALSE;
                    break;
                }
            } while (TRUE);
    
            PutLog(CString(TEXT("- RETURN - ")) + CString(__FUNCTION__) + TEXT(", res = ") + ToString(res));
            return res;
        }
    
        BOOL LeaveWrite()
        {
            PutLog(CString(TEXT("+ ENTER + ")) + TEXT(__FUNCTION__));
            BOOL res = SetEvent(m_hEvtNoRead); // release waiting readers
            _ASSERT(res);
            PutLog(CString(TEXT("- RETURN - ")) + CString(__FUNCTION__) + TEXT(", res = ") + ToString(res));
            return res;
        }
    Attached Files Attached Files
    Last edited by Igor Vartanov; June 18th, 2012 at 09:56 AM.
    Best regards,
    Igor

  12. #12
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Thanks again. I truly appreciate your unconventional approach to using semaphores.

    I reworked your code to use two separate processes (for readers and a writer.) Sorry, I can't write concise solutions like you do, so I opted for a default console app template instead. To test it run the writer first (only one instance is allowed) and then start as many readers as you want. I also removed your log file output and instead redirected it to the screen to achieve faster output (to test synchronization mechanism) and also to give a tester visual clue that threads are running.

    I had to remove your CRT I/O file functions in favor of lower-level APIs. Maybe I don't know much about fopen() but it seems to have its own built-in synchronization, which mixed with the results of the synchronization mechanism in this example.

    Then, first off, let me correct a bug. When you create m_hEvtNoRead event, it must be set to signaled state. Otherwise the first reader will deadlock without the first writer call.

    Then a couple of issues that I ran into (or maybe I should name them as questions):

    1. The sleep(10) part of the EnterWrite() method seems to be a bottleneck:
    Code:
    waitRes = WaitForSingleObject(m_hSemRead, 0);
    if (count)
    	Sleep(10);
    but if I rewrite it into:
    Code:
    waitRes = WaitForSingleObject(m_hSemRead, 10);
    if (count)
    	/*Sleep(10)*/;
    it really speeds things up, but unfortunately I get occasional "read fail". Why do you think would that be?

    PS. While typing this reply, I let the "writer" run on the background with Sleep(10) line... And it also gave me "read fail" after about 5-10 minutes of continuous running... There still must be an issue somewhere in the this lock...

    After a quick overview it seems that the part of the code that changes the semaphore count must be enclosed in its own critical section. Don't you think?

    2. The second issue is more challenging to fix. As I brought up before, if one of the readers crashes inside its critical section, the writer mechanism will also deadlock. To illustrate, search for the following line in the attached solution, and uncomment it:
    Code:
    #define INCLUDE_READER_CRASH	//Uncommect this line to introduce an artifical "Reader" crash
    This will make the reader process crash after 200 reads. When that happens watch the writer ... it will deadlock.
    Attached Files Attached Files

  13. #13
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    I think I fixed the issue #1 by enclosing the semaphore in a critical section. Attached is an updated project. There's still the issue #2 remaining though ...
    Attached Files Attached Files

  14. #14
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,071

    Re: Multiple-readers, single-writer synchronization lock between processes

    unconventional approach to using semaphores
    Well, the idea was got from MSDN article after reading about start with zero counter. So it's not that unconventional.

    let me correct a bug. When you create m_hEvtNoRead event, it must be set to signaled state. Otherwise the first reader will deadlock without the first writer call.
    It might be a feature, as long as without first write it might be senseless to read the void. You know this better, as I'm totally unaware of your ultimate design.

    The sleep(10) part of the EnterWrite() method seems to be a bottleneck
    I said you before that all my experience with synchronization was mostly about dealing with processes where hundreds of milliseconds never matter. Typically for me in business logic keeping precedence matters much more than super timings.

    And in case when few milliseconds really is a bottleneck, I'd say that either you need some other synchronization solution, or, which is more probable, your synchronized multi-process reader-writer approach itself is a bottleneck.

    but unfortunately I get occasional "read fail". Why do you think would that be?
    Like any other pure architectural thing it needs to be analyzed for flaws (code review, sequence diagrams, etc.) and properly debugged (by offline logs or interactively). Seems like I just was able to invest an idea, but have no time to purify it, sorry. Maybe other guys here will help.
    Best regards,
    Igor

  15. #15
    Join Date
    Feb 2009
    Location
    Portland, OR
    Posts
    1,488

    Re: Multiple-readers, single-writer synchronization lock between processes

    Yes. Thank you. I appreciate it.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center