Timeout with RPC calls?

[Marc G]

Hi,

I have implemented some RPC mechanism in one of my applications. It has a server and client both running on the same machine. I'm using local RPC (ncalrpc). Now, when I call a method via RPC, i want to have some kind of way to timeout the call in case the server is not responding or is slow at responding. There is a function called RpcBindingSetOption which can be used to specify a timeout, but it only works with XP and not with local RPC.
I was thinking of putting the RPC method call in its own thread and simply terminate the thread from another thread when timeout elapses, but using TerminateThread is dangerous according to MSDN.

Does anyone has any nice solution to my problem?

[genossa]

Hello Mark,

I'm not very familiar with RPC Message Queuing, so I can't suggest a solution based on RPC specifics, but in order to avoid TerminateThread ()calls and at the same time to keep your algorithm you may try to use Fibers. Fibers are execution units that can be executed in the single thread context, i.e "thread in thread". Not sure if that can be a fine solution in your case, just giving an advice.

Hope this helps,
Hennadii.

[Marc G]

Thanks for the reply.

Those fibers do seem interesting, I've never used them yet, but I might in future. However, I don't think they are suited for my problem. Let me try to explain it a bit better:
If my RPC client calls an RPC method, my client waits until that RPC method call finishes. If my server is not responding or not responding fast enough, I want the client to somehow abort the RPC call. That's why I was thinking about putting that RPC method call inside its own thread. That way I could kill the thread from another thread when the call takes too long. BUT, TerminateThread is not safe AND I don't know what the consequences would be if i would simply terminate a thread that is currently executing/waiting on a RPC method call? So, I don't like this method, but I don't know any other methods...
Anyone knows how this can be done?

[Bond]

Can you think of any other way to determine if the server is available other than calling your RPC method? Maybe you can run a series of shorter tests to ensure that the server is available and responding before sending it RPC calls.

As for TerminateThread, the function exists for a reason. Sure, it's not the best way to go. I don't believe it calls destructors for any objects created locally to the thread, it won't ensure that any resources are automatically closed, etc., but I think you can SAFELY use it if you carefully code your thread.

However, like you mentioned, if you're waiting on a syncronous function to return, what's going to happen if you kill the thread it's processing in? When you make that function call, the return address is pushed onto the stack. And when you terminate the thread, you're terminating that address.

I don't know much about using RPC but you'd think there would be a SendMessageTimeout() equivalent.

[Arjay]

Hi,

I have implemented some RPC mechanism in one of my applications. It has a server and client both running on the same machine. I'm using local RPC (ncalrpc). Now, when I call a method via RPC, i want to have some kind of way to timeout the call in case the server is not responding or is slow at responding. There is a function called RpcBindingSetOption which can be used to specify a timeout, but it only works with XP and not with local RPC.
I was thinking of putting the RPC method call in its own thread and simply terminate the thread from another thread when timeout elapses, but using TerminateThread is dangerous according to MSDN.

Does anyone has any nice solution to my problem?

Can you move to Asynchronous RPC?

Arjay

[Marc G]

Well, let me explain my situation.

My 'server' application is my Wallpaper Cycler which manages all your wallpapers (tens of thousands are supported without slowing down ). Now I have written a screen saver that renders a slideshow of your wallpapers. The problem now is that Wallpaper Cycler is running as a user application on the main desktop. When you enable password protection for the screensaver in Windows, Windows will execute the screensaver on another desktop. Because of this the screensaver cannot easily communicate with the Wallpaper Cycler running on another virtual desktop. All this is of course on NT/2K/XP systems, not on 9x.

My first thought was to use network sockets and it worked. Unfortunately firewalls might interfere with this and users have to grant permission to the screensaver to use sockets.

So, I throwed that away and thought about local RPC. It works flawlessly, except for that little timeout problem.

The exact problem is that when the screensaver is running, Wallpaper Cycler might show a messagebox on some rare occasions when it encounters an error. When this messagebox is being shown, Wallpaper Cycler will wait until you press the OK button, BUT this messagebox is being shown on another desktop than the screensaver and thus the user won't see the messagebox and can't click OK. The screensaver blocks on the RPC call until the OK button is pressed. So this is a deadlock. Only solution is to press ctrl+alt+del which seems to kill the screensaver and switches back to the main desktop.

Of course, another way to solve this is to prevent any messageboxes from showing when the render function from Wallpaper Cycler is being called from the screensaver, but I don't like this solution.

Arjay, thanks for mentioning Asynchronous RPC. I'm gonna check it out and post my results.

[kirants]

Just a point, don't you think a server kind of exe should not be showing messageboxes ?

[Marc G]

Just a point, don't you think a server kind of exe should not be showing messageboxes ?

Yes, but it isn't a true server. It is a standard user application for interactive use, not simply running silently. The screensaver is a kind of 'addon' that interacts with this user application to get images. So, it's not a true server.

I also looked into asynchronous RPC, and I might be able to use that, but it only works for Windows 2K and XP. But then again, who is still using some older version of windows... so, i'm gonna try it out.

[Arjay]

Another approach is to use a service with a singleton com server (of course this will only work on non-9x machines). Each client, the screensavers on each desktop, will connect to the server and obtain the WallPaper Cycler data (for the whole machine). IMO, this might be simpler to implement than straight RPC, especially if you build the COM server [service] with ATL.

Arjay

[Marc G]

Well, I have tried the asynchronous RPC method and it works perfectly

I've never really used COM before, so I think that it won't be that easy to implement and it appears it doesn't help me out as your suggestion also only works on non-9x systems.

What I'm gonna do is load the asynchronous RPC functions at runtime using GetProcAddress and when the asynchronous RPC functions aren't available (on 9x and NT4) I'm just gonna fall back to synchronous RPC or perhaps using the TerminateThread method I described above. It won't hurt that much because the TerminateThread will only be called in case of a timeout which should happen rarely.

[Arjay]

Well, I have tried the asynchronous RPC method and it works perfectly

I've never really used COM before, so I think that it won't be that easy to implement and it appears it doesn't help me out as your suggestion also only works on non-9x systems.

What I'm gonna do is load the asynchronous RPC functions at runtime using GetProcAddress and when the asynchronous RPC functions aren't available (on 9x and NT4) I'm just gonna fall back to synchronous RPC or perhaps using the TerminateThread method I described above. It won't hurt that much because the TerminateThread will only be called in case of a timeout which should happen rarely.

Glad to hear you have a working solution.

Just to clarify, COM works on 9x (even DCOM works on 9x). What doesn't work is the service portion of my suggestion.

You can still create a COM exe server, but you can't make the server run as a service on 9x (services only run on NT/XP).

Confusing? You're dang right it is. Look at it this way.

You need a wall paper cycler that handles the wallpaper data for the whole machine, right? Okay, so let's put the wallpaper cycler functionality into a COM server.

Using COM you have three options:
1) COM Server via a dll (called in-proc server)
2) COM Server via a standalone exe (called an out of proc server)
3) COM Server via a NT service (only for non-9x)

The dll com server won't work in this situation because each process (or client in a different desktop) would load a different version. Sure you could share your data between the instances with a memory mapped file or standard file, but you run the risk of corrupting the shared data if one of the processes crashes.

The Out of proc exe server is a good bet because any clients that connect to it all get data served from the same exe (so it makes synchronizing the underlying file or mmf wallpaper data simpler). The drawback is on NT-based systems, each user will invoke a separate exe which kind of defeats the purpose.

The COM server hosted by an NT service, allows only one instance of the exe server to be present for the whole machine. A service starts up prior to any user logging in and is available (with the proper credentials) to all users. The cool thing about this (and COM in general) is that any of the client need only to make a simple call to connect to the COM server and the clients don't need to know any location information or other setup info that you may have to do for RPC.

The call (excluding error handling) for the clients would look something like:

 
IWallpaperCyclerPtr spWPCycler( __uuidof( WallpaperCycler ) );
spWPCycler->DoWorkOrGetInfo();

The nice thing about using COM is that you don't need to manage all the asynchronous calls, the heartbeat, the timeouts, etc - the COM subsytem manages it for you.

I have to believe that your client code using the async RPC is more complicated.

Anyway, I see that I failed in my quest to make you a COM convert.

No matter, the important thing is that you have a working solution.

Arjay

[Marc G]

No matter, the important thing is that you have a working solution.

Unfortunately, my current solution appears broken
I thought to dynamically load the async RPC functions at runtime and when they are not available fall back to some other method, BUT the resulting exe (even with dynamic loading of those async functions) still does not run on Windows 9x, because it appears the exe is still linked to some async functions that I can't load dynamically. So, I get an "missing export" error on 9x

Actually, after thinking about it, I don't really need a COM service like you mentioned, but a COM exe would suffice. There does not need to run exactly 1 Wallpaper Cycler on the machine. Each user has there own instance running of it, so a standard COM exe would be ok i think.

I'll gonna do some reading on COM programming. Would it be easy/difficult to add support for a COM server to an existing program without too much trouble?

EDIT: The current project is using MFC btw.

[Marc G]

I just read some documentation on COM and how to allow some kind of timeout on COM method calls and it appears that to support cancellation of COM calls, you need at least Windows 2000:

Windows 2000 is the first version of COM to support asynchronous method calls, which permit clients to make nonblocking calls to COM objects and objects to process incoming calls without blocking the calling threads.

So, this doesn't help me any further, as my current Async RPC implementation is already working on Windows 2000/XP...

I have to believe that your client code using the async RPC is more complicated.

Actually, it isn't that complex
The server part is very simple:

// Initialize RPC server
RPC_STATUS status;
status = RpcServerUseProtseqEpA(reinterpret_cast<unsigned char*>("ncalrpc"), RPC_C_LISTEN_MAX_CALLS_DEFAULT,
  reinterpret_cast<unsigned char*>("NuonSoftWPCRPCInterface"), NULL); 
status = RpcServerRegisterIf(WallpaperCycler_v1_0_s_ifspec, NULL, NULL); 
status = RpcServerListen(1, RPC_C_LISTEN_MAX_CALLS_DEFAULT, TRUE);

(error checking left out for clarity)

The client is also pretty simple:

// Setup RPC client
RPC_STATUS status;
status = RpcStringBindingComposeW(NULL, reinterpret_cast<unsigned short*>("ncalrpc"), NULL,
  reinterpret_cast<unsigned short*>("NuonSoftWPCRPCInterface"), NULL, &pszStringBinding); 
status = RpcBindingFromStringBinding(pszStringBinding, &hWallpaperCyclerBinding1);

(error checking left out for clarity)

An synchronous call from the client:

RPC_ASYNC_STATE Async;
RPC_STATUS status;
status = RpcAsyncInitializeHandle(&Async, sizeof(RPC_ASYNC_STATE));
if (status) { // error handling }

Async.UserInfo = NULL;
Async.NotificationType = RpcNotificationTypeEvent;
Async.u.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
if (Async.u.hEvent == 0) { // error handling }

// The actual asynhronous RPC call
GetImage(&Async, &iSize, &pImage, 0);

// Timeout GetImage call after 10 seconds.
DWORD dwWaitResult = WaitForSingleObject(Async.u.hEvent, 10000);
if (dwWaitResult == WAIT_TIMEOUT || dwWaitResult == WAIT_FAILED)
{
  RpcAsyncCancelCall(&Async, TRUE);
  CloseHandle(Async.u.hEvent);
  return;
}
CloseHandle(Async.u.hEvent);
int nReply = 0;
// Get the results of the asynchronous RPC call
RpcAsyncCompleteCall(&Async, &nReply);

Also, in the GetImage server implementation you need to have a call to RpcAsyncCompleteCall.

Of course, some cleanup as to be done when the client/server is closed.

[Arjay]

I just read some documentation on COM and how to allow some kind of timeout on COM method calls and it appears that to support cancellation of COM calls, you need at least Windows 2000:

So, this doesn't help me any further, as my current Async RPC implementation is already working on Windows 2000/XP...

Actually, it isn't that complex
The server part is very simple:

// Initialize RPC server
RPC_STATUS status;
status = RpcServerUseProtseqEpA(reinterpret_cast<unsigned char*>("ncalrpc"), RPC_C_LISTEN_MAX_CALLS_DEFAULT,
reinterpret_cast<unsigned char*>("NuonSoftWPCRPCInterface"), NULL); 
status = RpcServerRegisterIf(WallpaperCycler_v1_0_s_ifspec, NULL, NULL); 
status = RpcServerListen(1, RPC_C_LISTEN_MAX_CALLS_DEFAULT, TRUE);

(error checking left out for clarity)

The client is also pretty simple:

// Setup RPC client
RPC_STATUS status;
status = RpcStringBindingComposeW(NULL, reinterpret_cast<unsigned short*>("ncalrpc"), NULL,
reinterpret_cast<unsigned short*>("NuonSoftWPCRPCInterface"), NULL, &pszStringBinding); 
status = RpcBindingFromStringBinding(pszStringBinding, &hWallpaperCyclerBinding1);

(error checking left out for clarity)

An synchronous call from the client:

RPC_ASYNC_STATE Async;
RPC_STATUS status;
status = RpcAsyncInitializeHandle(&Async, sizeof(RPC_ASYNC_STATE));
if (status) { // error handling }
 
Async.UserInfo = NULL;
Async.NotificationType = RpcNotificationTypeEvent;
Async.u.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
if (Async.u.hEvent == 0) { // error handling }
 
// The actual asynhronous RPC call
GetImage(&Async, &iSize, &pImage, 0);
 
// Timeout GetImage call after 10 seconds.
DWORD dwWaitResult = WaitForSingleObject(Async.u.hEvent, 10000);
if (dwWaitResult == WAIT_TIMEOUT || dwWaitResult == WAIT_FAILED)
{
RpcAsyncCancelCall(&Async, TRUE);
CloseHandle(Async.u.hEvent);
return;
}
CloseHandle(Async.u.hEvent);
int nReply = 0;
// Get the results of the asynchronous RPC call
RpcAsyncCompleteCall(&Async, &nReply);

Also, in the GetImage server implementation you need to have a call to RpcAsyncCompleteCall.

Of course, some cleanup as to be done when the client/server is closed.

Compare the client code above to the COM code below - COM still wins

 
IWallpaperCyclerPtr spWPCycler( __uuidof( WallpaperCycler ) );
spWPCycler->DoWorkOrGetInfo();

Seriously though. If you need help structuring the COM code, let me know.

Arjay

[kirants]

Unfortunately, my current solution appears broken
I thought to dynamically load the async RPC functions at runtime and when they are not available fall back to some other method, BUT the resulting exe (even with dynamic loading of those async functions) still does not run on Windows 9x, because it appears the exe is still linked to some async functions that I can't load dynamically. So, I get an "missing export" error on 9x

I am not sure how you are loading the async calls. Is it using LoadLib/GetProcAddress.

Anyways, if you are making sure in code that the async RPC calls are protected by OS version check, you can avoid using dynamic loading, but use straight calls to the APIs. However, in your linker settings you have to specify this particular dll to be delay-loaded. Search for DELAYLOAD