CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 14 of 14
  1. #1
    Join Date
    Sep 2009
    Posts
    9

    Cool Sharing DLL functions between processes

    Hi gurus,

    The real-time application I'm developing consists of the main program ("master") which starts child processes ("tasks"). The master process creates shared memory block which is used by the task processes to exchange data and to interact with each other. There's a type library (implemented as DLLs) which contains class member functions of the objects that reside in shared memory.

    The question is how to load the DLL into the shared memory block (created by CreateFileMapping(..., PAGE_EXECUTABLE,...)?
    Loading DLLs with LoadLibrary() places the executable into the virtual address space of master process, so the loaded modules are not accessible from other processes.

    Loading DLLs separately in virtual address space of every process is not suitable because the pointers to the DLL functions are to be stored in shared objects (pointers to Virtual Function Table)

    There's some quite low-level way to solve the problem: loading the DLL with LoadLibrary() call, locating the module in local memory with its handle and then copying the module contents into the shared memory block. But in this case a new problem arises. After moving the executable to some other place in memory the absolute (not relative to the value of stack register) addresses which are referenced by the executable should be adjusted by the value of memory shift.

    Is there any way to solve the problem above? There may be some undocumented calls to the kernel.dll functions, are there? What were the reasons of Microsoft to specify the shared memory tag PAGE_EXECUTABLE ? How to place any executable into this space (except as with memcpy())?

    Sorry for big amount of questions.

    Julian Drachevsky aka Hobbiton

  2. #2
    Join Date
    Feb 2005
    Posts
    2,160

    Re: Sharing DLL functions between processes

    Instead of separate processes trying to access some shared resource, why not use worker threads in the master process? Then they have access to any dll namespace (or any resource) of the master process.

  3. #3
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Sharing DLL functions between processes

    The real-time application I'm developing consists of the main program ("master") which starts child processes ("tasks"). The master process creates shared memory block which is used by the task processes to exchange data and to interact with each other. There's a type library (implemented as DLLs) which contains class member functions of the objects that reside in shared memory.

    The question is how to load the DLL into the shared memory block
    No it's not. The question is how to re-design the application totally. And I'll try to explain why.

    Shared memory cannot contain objects, it can contain only PODs with no (smart) pointers. It's just because any pointer can point to a memory which resides only in its original process address space, and loses its meaning in context of any other process.

    On the same reason an object cannot be placed to shared memory, just because member functions use this pointer transparently, that would result in a total mess while being referenced.

    Besides, I'd like to here some explanation regarding "type library (implemented as DLLs)". I feel suspicious about that. Either it's not a type library (in its common meaning), or (if it is) it cannot be implemented as DLL.

    And the last thing I'd like to mention. There is one very good way to share objects among processes. The technology is well known, and it's called COM. The process of passing is called marshaling.
    Best regards,
    Igor

  4. #4
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by hoxsiew View Post
    Instead of separate processes trying to access some shared resource, why not use worker threads in the master process? Then they have access to any dll namespace (or any resource) of the master process.
    The master process does not execute any work except initializing shared memory and returning pointers to the objects on request of child processes. Then the tasks will process shared objects.
    Every task is responsible for specific operation on objects. Besides the tasks can be executed on different CPUs.

  5. #5
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Igor Vartanov View Post
    No it's not. The question is how to re-design the application totally. And I'll try to explain why.

    Shared memory cannot contain objects, it can contain only PODs with no (smart) pointers. It's just because any pointer can point to a memory which resides only in its original process address space, and loses its meaning in context of any other process.

    On the same reason an object cannot be placed to shared memory, just because member functions use this pointer transparently, that would result in a total mess while being referenced.

    Besides, I'd like to here some explanation regarding "type library (implemented as DLLs)". I feel suspicious about that. Either it's not a type library (in its common meaning), or (if it is) it cannot be implemented as DLL.

    And the last thing I'd like to mention. There is one very good way to share objects among processes. The technology is well known, and it's called COM. The process of passing is called marshaling.
    Igor,

    Thanks for detailed reply. It's pleasant to meet compatriot at this forum.

    All your remarks above are undoubtedly true, but only in theory. Practically there're no objections to use shared memory to store pointers. Moreover I've already developed the program that implements the approach I've described earlier. There's the only problem: shared objects that have been initialized by the process, can be used only by this process (in the scope of virtual function table) .
    Loaded DLL is actually the type library. The DLL contains implementation of class member functions which are defined in appropriate header file with the specifiers virtual and __declspec(dllexport)
    COM technology is well known. Again, it's true. As well as COM technology is very expensive. And it is not suitable for our applications (for some reasons that are not concerned to programming)
    Last edited by Hobbiton; September 11th, 2009 at 11:47 PM.

  6. #6
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Hobbiton View Post
    The master process does not execute any work except initializing shared memory and returning pointers to the objects on request of child processes. Then the tasks will process shared objects.
    Every task is responsible for specific operation on objects. Besides the tasks can be executed on different CPUs.
    Yes, you explain how it is done now. But not why the tasks cannot be implemented with threads. A bit more details would be good.

    You can't go with COM (though I do not understand why, "COM technology is very expensive" sounds not very convincing, sorry, but there are "some reasoms", I see). Fine. Why don't you follow hoxsiew's advice? Or even go with oldy-goldy client-server architecture? Okay, never mind, this is all yours.
    Practically there're no objections to use shared memory to store pointers.
    Practically it may be true in your particular case only. But generally there are lots of objections (for example, access synchronization across processes, object lifecycle control, memory allocation control, etc.). You have to guarantee that shared memory block becomes located by the same address in every process. And after that, the last thing you have to solve is loading DLL by the same addresses in all processes. Just make your dll be based on fixed addresses. Or implement your own DLL loader.

    PS. I wonder... Is this only me who considers this sharing objects practically suicidal? Or implementing hand-made marshaling totally impractical?

    PPS. I'd like to warn you about using well-known terms. "Type library" is known as binary form of IDL, so it never could be a dll. In case you voluntarily use this term having in mind some different meaning, you need to note that explicitly to be understood correctly.

    PPPS. I had no intention to offend. This is my last "negative" message in this thread.
    Last edited by Igor Vartanov; September 12th, 2009 at 01:14 AM.
    Best regards,
    Igor

  7. #7
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Igor Vartanov View Post
    But not why the tasks cannot be implemented with threads. A bit more details would be good.
    Yea, implementation of the tasks with threads rather than processes will be quite radical solution of the problem mentioned. But currently I may not change architecture of the application been already developed. Using COM architecture is not suitable mostly for the same reason.
    Practically it may be true in your particular case only. But generally there are lots of objections (for example, access synchronization across processes, object lifecycle control, memory allocation control, etc.).
    Again, you're right. My case is very particular because
    Master process
    guarantees that shared memory block becomes located by the same address in every process.
    Master is the only process responsible for creation and initialization of objects. Creation objects by the tasks is prohibited. The master process also remotely starts computational threads in each process providing synchronization and absence of dead locks. My case is actually very specific.
    And after that, the last thing you have to solve is loading DLL by the same addresses in all processes. Just make your dll be based on fixed addresses. Or implement your own DLL loader.
    You'll be laughing but right now I'm trying to understand how is LoadLibrary() works together with CreateFileMapping(). There must be the way to load DLL to the specific memory location rather then into arbitrary address generated by kernel.dll.
    Strange thing is observed: first loaded module is always placed at address 0x10000000, next one to the address 0x00360000, others are loaded sequentially to addresses multiplied to 0x00010000 (obviously this is memory granularity value). If the process calls CreateMemoryMapping(...,PAGE_EXECUTE_READWRITE,...) before loading DLLs then kernel skips appropriate shared memory block and go on loading modules to the memory location immediately after this block. I'm trying to find out how the kernel determines which memory block is free and what of operation it performs before loading the module.

    PS. I wonder... Is this only me who considers this sharing objects practically suicidal? Or implementing hand-made marshaling totally impractical?
    No. You are not alone. There're so many different people on this planet

    PPS. I'd like to warn you about using well-known terms. "Type library" is known as binary form of IDL, so it never could be a dll. In case you voluntarily use this term having in mind some different meaning, you need to note that explicitly to be understood correctly.
    Agreed. Being applied to COM objects the term "type library" has quite different meaning .

    PPPS. I had no intention to offend. This is my last "negative" message in this thread.
    You're welcome. Constructive criticism never offends me.

    I will repeat the question. How to load DLL to the specific memory location or how to fix up absolute addresses of the executable been copied?

  8. #8
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Sharing DLL functions between processes

    I hoped you could find those linker options yourself...
    /BASE
    /FIXED

    Now you need to find a safe address range for your dlls. FYI, all MS key dlls have fixed base addresses, and they cannot be relocated, so you should take care about eliminating address conflicts with your dlls. Good luck to you.

    PS. BTW, you'd better forget about your idea to shove the dlls into your shared memory. It's useless and misguiding, just believe me. It's enough to have them fix-based in every process, there's no need in any other "sharing."
    Attached Files Attached Files
    Last edited by Igor Vartanov; September 12th, 2009 at 04:15 PM.
    Best regards,
    Igor

  9. #9
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Igor Vartanov View Post
    I hoped you could find those linker options yourself...
    /BASE
    /FIXED

    Now you need to find a safe address range for your dlls. FYI, all MS key dlls have fixed base addresses, and they cannot be relocated, so you should take care about eliminating address conflicts with your dlls. Good luck to you.

    PS. BTW, you'd better forget about your idea to shove the dlls into your shared memory. It's useless and misguiding, just believe me. It's enough to have them fix-based in every process, there's no need in any other "sharing."
    That was my first idea when I started to solve "VFT for shared objects" problem a week ago. Just to load DLL at same address in each process address space. The additional problem arises when accepting the approach above. If somebody else is developing the additional DLL plug-in he doesn't know base addresses of DLLs been already loaded. In this case there may be address conflict.

    Nevertheless I don't see any contradiction between C++ programming ideology and "shove the dlls into your shared memory". The objects that have been created and initialized outside the tasks, reside in common shared memory. The class member functions intended to work on object content, also reside in shared memory (i.e. the same place where the objects reside)

    Anyway, thanks for support and constructive discussion
    Last edited by Hobbiton; September 12th, 2009 at 10:25 PM.

  10. #10
    Join Date
    Nov 2000
    Location
    Voronezh, Russia
    Posts
    6,620

    Re: Sharing DLL functions between processes

    If somebody else is developing the additional DLL plug-in he doesn't know base addresses of DLLs been already loaded. In this case there may be address conflict.
    Okay, from now on, you need your custom dll loader. There was some article on www.rsdn.ru about custom loader implementation. Maybe it will help.

    The objects that have been created and initialized outside the tasks, reside in common shared memory.
    The only objects with no pointers inside fit this idealistic approach. Otherwise they shoul take care about placing any referenced memory (pointed indirectly by members of any level of indirection) into the shared memory as well. Now you need custom allocator and heap manager.

    The class member functions intended to work on object content, also reside in shared memory (i.e. the same place where the objects reside)
    Yeah (and this results in need of custom loader). What size of shared memory is ultimately required for this? Your objects, memory heap, plugin code pages you never know beforehand how many are there... The need in orchestrating all this shaggy herd... This is the price of violating the only single simple rule: never share pure C++ objects. And you tell me the COM "is very expencive"?

    Besides, plugins normally use interfaces but not objects. Just for being able to be written in different languages (what makes real success for a system). And this (different languages, of course) makes the point of allocators and heap management way too complex.
    Last edited by Igor Vartanov; September 14th, 2009 at 03:29 AM.
    Best regards,
    Igor

  11. #11
    Join Date
    Feb 2002
    Posts
    4,640

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Hobbiton View Post
    The master process does not execute any work except initializing shared memory and returning pointers to the objects on request of child processes. Then the tasks will process shared objects.
    Every task is responsible for specific operation on objects. Besides the tasks can be executed on different CPUs.
    Multiple "machines" (read hardware), or multiple "CPUs" in the same machine? If it's the latter, then you can certainly use threads. Threads will execute on different CPU's in the same machine.

    Viggy

  12. #12
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Igor Vartanov View Post
    Okay, from now on, you need your custom dll loader. There was some article on www.rsdn.ru about custom loader implementation. Maybe it will help.
    Thanks a lot for reference.
    The only objects with no pointers inside fit this idealistic approach. Otherwise they shoul take care about placing any referenced memory (pointed indirectly by members of any level of indirection) into the shared memory as well. Now you need custom allocator and heap manager.
    That's right. The only pointers allowed in shared memory objects are the pointers which reference to the shared objects. This condition is fulfilled.
    Yeah (and this results in need of custom loader). What size of shared memory is ultimately required for this? Your objects, memory heap, plugin code pages you never know beforehand how many are there... The need in orchestrating all this shaggy herd... This is the price of violating the only single simple rule: never share pure C++ objects. And you tell me the COM "is very expensive"?
    Yeah, plug-in code functions should never use data except object fields and function parameters as well as the function should never return references or pointers to module global variables. That's OK.
    Besides, plug-ins normally use interfaces but not objects. Just for being able to be written in different languages (what makes real success for a system). And this (different languages, of course) makes the point of allocators and heap management way too complex.
    The only language used is unmanaged C++.
    So there's no need in COM approach.
    Now I think that the approach you mentioned earlier, i.e. to load DLL simultaneously in every process started at the same virtual address, is the easiest to implement.

  13. #13
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by MrViggy View Post
    Multiple "machines" (read hardware), or multiple "CPUs" in the same machine? If it's the latter, then you can certainly use threads. Threads will execute on different CPU's in the same machine.

    Viggy
    I meant multiple "CPUs" in the same machine. Anyway I may not use threads. Separate processes (each having its own virtual address space) are only the choice.

  14. #14
    Join Date
    Sep 2009
    Posts
    9

    Re: Sharing DLL functions between processes

    Quote Originally Posted by Igor Vartanov View Post
    Okay, from now on, you need your custom dll loader. There was some article on www.rsdn.ru about custom loader implementation. Maybe it will help.
    It's the exceptionally interesting article. The attached code is to be translated into C++, but the actual problem is to understand how does it work.
    The main disadvantage of the submitted approach is unavailability of using debugger for DLL code, because modules are loaded without registration in system.
    Last edited by Hobbiton; September 15th, 2009 at 05:35 PM.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured