-
October 5th, 2015, 12:38 PM
#1
Passing STL arguments through shared libaries
We have a small group on our program redesigning an API that is to be used by 3rd party developers to create a shared library that our system will use. The current API interface is extremely old, very cumbersome to use, and very C-like. The goal is to create a more C++-like interface, make it much easier to use, and add some additional functionality. The library delivered by the 3rd party developers adhering to the new API will be a shared object (.so file) used by our system on a Redhat Linux platform.
An individual on our team recently pointed out that the API should not use any STL type arguments, pointing out that “it would be in violation of the API (Application Binary I/F) spec that compilers must follow”. He further argued that the only way this would work would be if the library and executables are compiled and linked exactly the same so all template implementations are the same across the linked modules. Since we do not have control over what compiler versions will be used, or what compiler flags will be used by the 3rd party developers, it cannot be guaranteed that the library will work.
I’ve since done some light reading on the internet and have not come to a definitive conclusion as to whether these statements are accurate. It seems that there is much discrepancy (or confusion) as to whether or not the API is designed in a Windows environment (which uses DLL’s) or a Linux environment (which uses SO’s).
So, is it true that defining a shared object API that passes STL arguments to and from its interface is not a smart design and may cause problems? Or, is this just a problem for MS and how their DLL’s are laid out?
Any insight would be greatly appreciated. Thanks.
-
October 5th, 2015, 01:55 PM
#2
Re: Passing STL arguments through shared libaries
It is accurate - for all platforms.
You also should not pass CRT objects either - like FILE since the .so may be using a completely different CRT - you don't know.
For the same reasons, you don't want malloc() memory in one modules and free() it in another.
>> ... and very C-like
You don't even know what compiler was used to create the .so - so only a standard ABI should be used when crossing module boundaries.
gg
-
October 5th, 2015, 02:37 PM
#3
Re: Passing STL arguments through shared libaries
Thanks for the quick response. So, we're talking about basic POD types? If I understand you correctly, the only safe API design would be to use only POD types for all input and output to a shared object.
-
October 5th, 2015, 02:46 PM
#4
Re: Passing STL arguments through shared libaries
There is, of course, nothing to stop you implementing your 'shared library' as header files in the same way that the c++ standard library is implemented and include them in the programs that need to use them. As the header files will be compiled along with your code then the API interface can be as desired.
All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!
C++23 Compiler: Microsoft VS2022 (17.6.5)
-
October 5th, 2015, 03:21 PM
#5
Re: Passing STL arguments through shared libaries
Yeah, POD's.
You could even expose your own structures, as long as the user's dev environment builds them with the same padding and alignment as yours.
gg
-
October 5th, 2015, 03:42 PM
#6
Re: Passing STL arguments through shared libaries
Originally Posted by Codeplug
Yeah, POD's.
You could even expose your own structures, as long as the user's dev environment builds them with the same padding and alignment as yours.
gg
Great! Thanks for your help.
-
October 5th, 2015, 03:44 PM
#7
Re: Passing STL arguments through shared libaries
Originally Posted by 2kaud
There is, of course, nothing to stop you implementing your 'shared library' as header files in the same way that the c++ standard library is implemented and include them in the programs that need to use them. As the header files will be compiled along with your code then the API interface can be as desired.
Ok, so what you're saying is to define a standard set of header files providing the necessary API. Then, provide these headers to the 3rd party developers for usage in their implementation of the library. These developers will then provide a binary shared object (.so file). Finally, regardless of what platform, compiler version, or compiler flags they used, we will then be able to link our code with the shared library using our own platform, compiler version, and compiler flags as long as we use the same header files where necessary.
If this works, that's great, however, what about runtime dynamic linking vice compile time static linking? Is there a way
to solve that problem?
-
October 5th, 2015, 03:56 PM
#8
Re: Passing STL arguments through shared libaries
>> As the header files will be compiled along with your code then the API interface can be as desired.
My interpretation is your API will still be C-like, but you could still provide classes that wrap that API, which could use std:: facilities etc. Clients would use their own C++ compiler and libraries to compile the code - but nothing fancy crosses the API boundary.
GDI+ does this on Windows for example. (C++ code wrapping a C api that is)
gg
-
October 5th, 2015, 04:13 PM
#9
Re: Passing STL arguments through shared libaries
Originally Posted by sszd
Ok, so what you're saying is to define a standard set of header files providing the necessary API. Then, provide these headers to the 3rd party developers for usage in their implementation of the library. These developers will then provide a binary shared object (.so file). Finally, regardless of what platform, compiler version, or compiler flags they used, we will then be able to link our code with the shared library using our own platform, compiler version, and compiler flags as long as we use the same header files where necessary.
If this works, that's great, however, what about runtime dynamic linking vice compile time static linking? Is there a way
to solve that problem?
What I'm suggesting for consideration is that you could implement the whole of the required shared library in the header files - just like the c++ Standard Library. No creation/linking of .so files etc. All the code for the library is in the header files. That's the way the c++ standard library is implemented. If you do
then all the required code for the string class is included with the program code and gets compiled along with the program code. No .so (or .dll) file, no linking, no issues with using non-POD etc.
All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!
C++23 Compiler: Microsoft VS2022 (17.6.5)
-
October 5th, 2015, 04:50 PM
#10
Re: Passing STL arguments through shared libaries
Originally Posted by 2kaud
What I'm suggesting for consideration is that you could implement the whole of the required shared library in the header files - just like the c++ Standard Library. No creation/linking of .so files etc. All the code for the library is in the header files. That's the way the c++ standard library is implemented.
Thanks for the clarification. Unfortunately that would mean the library providers would have to expose possibly proprietary code. Some may be willing to do so, but others I know are not.
-
October 5th, 2015, 04:53 PM
#11
Re: Passing STL arguments through shared libaries
Originally Posted by Codeplug
My interpretation is your API will still be C-like, but you could still provide classes that wrap that API, which could use std:: facilities etc. Clients would use their own C++ compiler and libraries to compile the code - but nothing fancy crosses the API boundary.
This is something we are certainly considering and will probably end up with.
-
October 5th, 2015, 08:54 PM
#12
Re: Passing STL arguments through shared libaries
There's a module system in pipeline. I think it's planned for C++ 17,
http://www.open-std.org/jtc1/sc22/wg...2015/n4465.pdf
-
October 6th, 2015, 03:14 AM
#13
Re: Passing STL arguments through shared libaries
Originally Posted by sszd
An individual on our team recently pointed out that the API should not use any STL type arguments, [...] as to whether these statements are accurate.
strictly speaking, IMO it's not accurate, because there's nothing wrong in <using> STL type arguments in the API, what's wrong is to <pass> them through module boundaries ( the former does not imply the latter ).
Indeed, there's also a third option along with what codeplug already suggested ( 1) use a C API, 2) use a C++ wrapper over a C API ) that is, you can write/use an existing interface solely devoted to the marshalling of the c++ types ( the server/client never directly "sees" the objects on the other side, that are continuously transferred by using some intermidiate, binary portable representation ).
albeit technically equivalent to codeplug suggestion ( you always have a C api under the hood ), the advantage is that you don't need to write and mantain a C version of <your> library ( whether user exposed or not ) as you only focus on the marshalling mechanism. Moreover, you can use existing quality implementations ( from boost asio to google protocol buffers etc... ) and you'll end up with a more general solution ( it would work across network boundaries as well with almost no effort ).
the disadvantages are the dipendency on such third party libraries, possible performance issues if overused, etc... so it's neither the holy grail nor the easier of solutions, of course.
Originally Posted by tiliavirga
There's a module system in pipeline
AFAIK modules are not designed for ABI portability ( the OP wants a DLL/so )
-
October 7th, 2015, 12:56 AM
#14
Re: Passing STL arguments through shared libaries
Originally Posted by superbonzo
AFAIK modules are not designed for ABI portability ( the OP wants a DLL/so )
That's right. But modules still is a portability (componentization for the purpose of sharing) option that most likely will be in the standard pretty soon.
And there are ongoing discussions and even a proposal for a true ABI for C++,
https://isocpp.org/blog/2014/05/n4028
Last edited by tiliavirga; October 7th, 2015 at 01:01 AM.
-
October 7th, 2015, 05:11 AM
#15
Re: Passing STL arguments through shared libaries
that's interesting, but it still wouldn't solve the OP problem, that is ODR failures accross binary boundaries. For that we'd need something like standardized binary c++ modules ...
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|