CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 18

Thread: Passing STL arguments through shared libaries

  1. #1
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Passing STL arguments through shared libaries

    We have a small group on our program redesigning an API that is to be used by 3rd party developers to create a shared library that our system will use. The current API interface is extremely old, very cumbersome to use, and very C-like. The goal is to create a more C++-like interface, make it much easier to use, and add some additional functionality. The library delivered by the 3rd party developers adhering to the new API will be a shared object (.so file) used by our system on a Redhat Linux platform.

    An individual on our team recently pointed out that the API should not use any STL type arguments, pointing out that ďit would be in violation of the API (Application Binary I/F) spec that compilers must followĒ. He further argued that the only way this would work would be if the library and executables are compiled and linked exactly the same so all template implementations are the same across the linked modules. Since we do not have control over what compiler versions will be used, or what compiler flags will be used by the 3rd party developers, it cannot be guaranteed that the library will work.

    Iíve since done some light reading on the internet and have not come to a definitive conclusion as to whether these statements are accurate. It seems that there is much discrepancy (or confusion) as to whether or not the API is designed in a Windows environment (which uses DLLís) or a Linux environment (which uses SOís).

    So, is it true that defining a shared object API that passes STL arguments to and from its interface is not a smart design and may cause problems? Or, is this just a problem for MS and how their DLLís are laid out?

    Any insight would be greatly appreciated. Thanks.

  2. #2
    Join Date
    Nov 2003
    Posts
    1,902

    Re: Passing STL arguments through shared libaries

    It is accurate - for all platforms.
    You also should not pass CRT objects either - like FILE since the .so may be using a completely different CRT - you don't know.
    For the same reasons, you don't want malloc() memory in one modules and free() it in another.

    >> ... and very C-like
    You don't even know what compiler was used to create the .so - so only a standard ABI should be used when crossing module boundaries.

    gg

  3. #3
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Re: Passing STL arguments through shared libaries

    Thanks for the quick response. So, we're talking about basic POD types? If I understand you correctly, the only safe API design would be to use only POD types for all input and output to a shared object.

  4. #4
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,189

    Re: Passing STL arguments through shared libaries

    There is, of course, nothing to stop you implementing your 'shared library' as header files in the same way that the c++ standard library is implemented and include them in the programs that need to use them. As the header files will be compiled along with your code then the API interface can be as desired.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2019 (16.7.0)

  5. #5
    Join Date
    Nov 2003
    Posts
    1,902

    Re: Passing STL arguments through shared libaries

    Yeah, POD's.
    You could even expose your own structures, as long as the user's dev environment builds them with the same padding and alignment as yours.

    gg

  6. #6
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by Codeplug View Post
    Yeah, POD's.
    You could even expose your own structures, as long as the user's dev environment builds them with the same padding and alignment as yours.

    gg
    Great! Thanks for your help.

  7. #7
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by 2kaud View Post
    There is, of course, nothing to stop you implementing your 'shared library' as header files in the same way that the c++ standard library is implemented and include them in the programs that need to use them. As the header files will be compiled along with your code then the API interface can be as desired.
    Ok, so what you're saying is to define a standard set of header files providing the necessary API. Then, provide these headers to the 3rd party developers for usage in their implementation of the library. These developers will then provide a binary shared object (.so file). Finally, regardless of what platform, comp‎iler version, or compiler flags they used, we will then be able to link our code with the shared library using our own platform, compiler version, and compiler flags as long as we use the same header files where necessary.

    If this works, that's great, however, what about runtime dynamic linking vice compile time static linking? Is there a way
    to solve that problem?

  8. #8
    Join Date
    Nov 2003
    Posts
    1,902

    Re: Passing STL arguments through shared libaries

    >> As the header files will be compiled along with your code then the API interface can be as desired.
    My interpretation is your API will still be C-like, but you could still provide classes that wrap that API, which could use std:: facilities etc. Clients would use their own C++ compiler and libraries to compile the code - but nothing fancy crosses the API boundary.

    GDI+ does this on Windows for example. (C++ code wrapping a C api that is)

    gg

  9. #9
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,189

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by sszd View Post
    Ok, so what you're saying is to define a standard set of header files providing the necessary API. Then, provide these headers to the 3rd party developers for usage in their implementation of the library. These developers will then provide a binary shared object (.so file). Finally, regardless of what platform, comp‎iler version, or compiler flags they used, we will then be able to link our code with the shared library using our own platform, compiler version, and compiler flags as long as we use the same header files where necessary.

    If this works, that's great, however, what about runtime dynamic linking vice compile time static linking? Is there a way
    to solve that problem?
    What I'm suggesting for consideration is that you could implement the whole of the required shared library in the header files - just like the c++ Standard Library. No creation/linking of .so files etc. All the code for the library is in the header files. That's the way the c++ standard library is implemented. If you do
    Code:
    #include <string>
    then all the required code for the string class is included with the program code and gets compiled along with the program code. No .so (or .dll) file, no linking, no issues with using non-POD etc.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2019 (16.7.0)

  10. #10
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by 2kaud View Post
    What I'm suggesting for consideration is that you could implement the whole of the required shared library in the header files - just like the c++ Standard Library. No creation/linking of .so files etc. All the code for the library is in the header files. That's the way the c++ standard library is implemented.
    Thanks for the clarification. Unfortunately that would mean the library providers would have to expose possibly proprietary code. Some may be willing to do so, but others I know are not.

  11. #11
    Join Date
    Feb 2005
    Location
    Denver
    Posts
    353

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by Codeplug View Post
    My interpretation is your API will still be C-like, but you could still provide classes that wrap that API, which could use std:: facilities etc. Clients would use their own C++ compiler and libraries to compile the code - but nothing fancy crosses the API boundary.
    This is something we are certainly considering and will probably end up with.

  12. #12
    Join Date
    Jun 2015
    Posts
    208

    Re: Passing STL arguments through shared libaries

    There's a module system in pipeline. I think it's planned for C++ 17,

    http://www.open-std.org/jtc1/sc22/wg...2015/n4465.pdf

  13. #13
    Join Date
    Oct 2008
    Posts
    1,456

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by sszd View Post
    An individual on our team recently pointed out that the API should not use any STL type arguments, [...] as to whether these statements are accurate.
    strictly speaking, IMO it's not accurate, because there's nothing wrong in <using> STL type arguments in the API, what's wrong is to <pass> them through module boundaries ( the former does not imply the latter ).

    Indeed, there's also a third option along with what codeplug already suggested ( 1) use a C API, 2) use a C++ wrapper over a C API ) that is, you can write/use an existing interface solely devoted to the marshalling of the c++ types ( the server/client never directly "sees" the objects on the other side, that are continuously transferred by using some intermidiate, binary portable representation ).

    albeit technically equivalent to codeplug suggestion ( you always have a C api under the hood ), the advantage is that you don't need to write and mantain a C version of <your> library ( whether user exposed or not ) as you only focus on the marshalling mechanism. Moreover, you can use existing quality implementations ( from boost asio to google protocol buffers etc... ) and you'll end up with a more general solution ( it would work across network boundaries as well with almost no effort ).

    the disadvantages are the dipendency on such third party libraries, possible performance issues if overused, etc... so it's neither the holy grail nor the easier of solutions, of course.

    Quote Originally Posted by tiliavirga
    There's a module system in pipeline
    AFAIK modules are not designed for ABI portability ( the OP wants a DLL/so )

  14. #14
    Join Date
    Jun 2015
    Posts
    208

    Re: Passing STL arguments through shared libaries

    Quote Originally Posted by superbonzo View Post
    AFAIK modules are not designed for ABI portability ( the OP wants a DLL/so )
    That's right. But modules still is a portability (componentization for the purpose of sharing) option that most likely will be in the standard pretty soon.

    And there are ongoing discussions and even a proposal for a true ABI for C++,

    https://isocpp.org/blog/2014/05/n4028
    Last edited by tiliavirga; October 7th, 2015 at 01:01 AM.

  15. #15
    Join Date
    Oct 2008
    Posts
    1,456

    Re: Passing STL arguments through shared libaries

    that's interesting, but it still wouldn't solve the OP problem, that is ODR failures accross binary boundaries. For that we'd need something like standardized binary c++ modules ...

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)