C struct alignment across compilers and operating systems
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 14 of 14

Thread: C struct alignment across compilers and operating systems

  1. #1
    Join Date
    Oct 2013
    Posts
    5

    C struct alignment across compilers and operating systems

    I am working on extending an application that features a (rather old) plugin system. The plugin system is designed to work on x86 only, but support any and all compilers on windows, linux and also OSX.

    My goals are now to

    1) extend the plugin framework in a backwards compatible way
    2) port it elegantly to x64


    So far so good.

    The plugin subsystem essentially exports a bunch of SDK function like this:


    Code:
    extern "C" CDECL int somefunc(int a, int b);

    which can be used by the plugin, and expect the plugin to export some function in return.
    When a struct is supposed to be send to the plugin, the syntax of the system goes like this:


    Code:
    extern "C" CDECL int RequestStructField(int field, int structhandle);
    so the struct isnt handed over via a pointer, but every field has to be requested individually!
    From what i can tell from the comments, the developer was afraid there might be problems exchanging structs between different compilers, alignment and all.


    I would like to tear down this limitation, and pass pointers to structs to plugin directly. Or better: allow the plugin to give me a pointer to an empty struct which i subsequently fill.
    However, i have no control over the compiler or language used to write the plugins. Quiet some plugins are written in Delphi, some even in MASM ... both would have to support this way of passing structs.

    the question is: does that work?

    I have looked at other plugin systems, but they either use entirely different ways (XPCOM) or are limited to a single platform or compiler.

    Some of them have a note stating that the plugin must be compiled with "byte-aligned" structs.

    Information is spare, though.

    Any thoughts on this?




    2)

    x86 and x64 plugins are two different stories, x64 version of the app does NOT have load x86 plugins.
    I dont expect there to be too man problems (right?). But we'll see, i guess.

  2. #2
    Join Date
    Apr 1999
    Posts
    27,426

    Re: C struct alignment across compilers and operating systems

    From what i can tell from the comments, the developer was afraid there might be problems exchanging structs between different compilers, alignment and all.
    The developer was absolutely right.
    Some of them have a note stating that the plugin must be compiled with "byte-aligned" structs.
    Unless the entire struct is byte-aligned, and it is documented that the struct must be byte aligned, there is no guarantee what the receiver is getting in terms of alignment.
    the struct isnt handed over via a pointer, but every field has to be requested individually!
    So the question is why the user must know about individual fields? This is where the developer may have been short-sighted, and that is allow the user to pick out fields like this at such a low level. Instead, some soft of "setter/getter" functions could have been implemented so as to shield the user from knowing the internals.
    I would like to tear down this limitation, and pass pointers to structs to plugin directly.
    The way you "tear down the limitation" is to come up with some sort of API that works with your structs internally.

    What many systems do is create a handle (which is really a pointer to the struct), and pass that to the user by having the user call some sort of "Create" function. Then any subsequent changes to the struct is done by passing the handle to other API functions that uses the handle that was created. The user doesn't know or care what the internals of the struct is -- all they know is they have this handle, and any changes needed must call an API function with the provided handle.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; October 5th, 2013 at 12:55 PM.

  3. #3
    Join Date
    Oct 2013
    Posts
    5

    Re: C struct alignment across compilers and operating systems

    Instead, some soft of "setter/getter" functions could have been implemented so as to shield the user from knowing the internals.
    What do you mean? It turns out, they did pretty much waht you suggested (i think): the first argument of the function is a predefined ID (#define SOME_ID 1234), the handle paramter is actually a pointer to the struct.


    Is there any cross-compiler way to byte-align a struct?

  4. #4
    Join Date
    Apr 1999
    Posts
    27,426

    Re: C struct alignment across compilers and operating systems

    Quote Originally Posted by eduardschreder View Post
    Is there any cross-compiler way to byte-align a struct?
    No. You can only control alignment through some sort of #pragma or other compiler-dependent setting.

    Regards,

    Paul McKenzie

  5. #5
    Join Date
    Apr 2000
    Location
    Belgium (Europe)
    Posts
    3,845

    Re: C struct alignment across compilers and operating systems

    Also: there is no guarantee about __cdecl being available in x64
    the VS compiler entirely ignores calling convention and Always enforces it's only supported calling convention.

    Other compiler may not follow the same system, so if you write for VS, any compiler that doesn't do it the VS way won't be usable, or other way around, if you develop this in a compiler that does it differently then plugins developed in VS won't work.

    you can't call 32bit code from a 64bit app (and vice versa). In fact you can't even load a dll with a different "bitness".

    passing pointers around is opening a potentially dangerous door for your app stability and how easy it is to "hack" into your app.

  6. #6
    Join Date
    Oct 2013
    Posts
    5

    Re: C struct alignment across compilers and operating systems

    @Paul

    Alright, thanks.

    What about arrays such as int x[50].

    Are they "aligned" properly?


    @OReubens

    Good point, no __cdecl on x64. crap.

    Well, for x64 we should have standardized __fastcall, right? so it shouldnt matter. (?)

  7. #7
    Join Date
    Apr 2000
    Location
    Belgium (Europe)
    Posts
    3,845

    Re: C struct alignment across compilers and operating systems

    there is no "official and only standard" for x86 as far as I know.

    There is the way VS does it, which is the same way the Windows API expects it. So it's reasonable to assume that any other compiler that can target windows at least has support for that method. But that other compiler could do things differently "by default" and only do it the VS way for API calls. The other compiler would probably have a declaration to select "the Windows/VS way".

    an int x[50] will be int-aligned, unless you code specifically for it not to be... If you need guarantees on the receiving end, you may need to add checks for proper alignment.
    ANd since this'll be a pointer, it opens doors to potential abuse.

    It's "hard" to get pointer/buffer usage correct in your own code, when it's for code that interfaces with your own... it's nigh impossible to cover all the security issues. The guy that initially did this, was right in being suspicious about pointers.

  8. #8
    Join Date
    Oct 2013
    Posts
    5

    Re: C struct alignment across compilers and operating systems

    So it's reasonable to assume that any other compiler that can target windows at least has support for that method. But that other compiler could do things differently "by default" and only do it the VS way for API calls. The other compiler would probably have a declaration to select "the Windows/VS way".
    But then how does that "other" compiler link to a .dll file compiled by VCC?



    Yes, the array-alignmen seems to be guaranteed. That's good.

    I know we're going way off-road here, but is there, in theory and practice, any difference between

    int x[4];
    ,
    struct y
    {
    int x[4];
    };
    and
    struct
    {
    int a,b,c,d;
    };
    ?


    It seems a little silly.




    The guy that initially did this, was right in being suspicious about pointers.
    Security is not that much of an issue...but yeah, i see your point.

    The problem is, that i'd like to pass over several very large data structures, for which it's just too much work
    to implement a GetAttributXyz(HANDLE struct, int field); for each field.


    Thanks for your time!

  9. #9
    Join Date
    Apr 2000
    Location
    Belgium (Europe)
    Posts
    3,845

    Re: C struct alignment across compilers and operating systems

    Quote Originally Posted by eduardschreder View Post
    But then how does that "other" compiler link to a .dll file compiled by VCC?
    "it doesn't" or if it does, it either also does things the VC way, or it has declspec's of it's own to make it happen that way.

    One of the compilers I'm using has a __cwin64 declspec for all windows API interfacing (which you could also use to interface with VCC code). But it's native calling convention is different.



    Yes, the array-alignmen seems to be guaranteed. That's good.
    It depends what you mean by that, but. If you allow the compiler to use it's normal alignment then "usually".
    again, there are compilers that have different views on this.



    I know we're going way off-road here, but is there, in theory and practice, any difference between
    In theory: they're different types.
    In theory: this isn't guaranteed in and off itself from a pure language pov. (for 3 vs the other 2)
    In practice: with the right compiler settings, all 3 will have the same memory layout. 3 could have a different layout.

    also note that the alignment for the struct with an int array might be different than an int array on it's own.

    It seems a little silly.
    It isn't. It's a manifestation of the flexibility and power of C++.
    And just because you CAN do things in C++ doesn't mean you SHOULD do them. There's rights and not-so-rights


    The problem is, that i'd like to pass over several very large data structures, for which it's just too much work
    to implement a GetAttributXyz(HANDLE struct, int field); for each field.
    it's just a bunch of copy/paste imo. That may end up being a lot easier than trying/testing structures and dealing with all the nasties that arrise from it.

    There are many ways you could do this.
    the problem with returning structures... Who's going to clean them up? You can pass a structure by reference/pointer, but that has issues too.

    You could use the COM approach with VARIANT's and return safe arrays.
    You could return a single string that needs to be parsed on the client end. (you could return XML for example).

    Regardless, if you want to return something other than a simple type, you're going to have to deal with a certain amount of practical issues (such as who's taking care of allocation/freeing), and potentially with security issues.

    And don't dismiss the security issue. A friend of mine got sued (and lost) because someone abused the scripting system he had added in his 'simple" application to hack into a secured server) (the scripts allowed just about anything, and anyone could create scripts, even a guest user, the problem being the scripts were executed in a service running at full elevation).

  10. #10
    Join Date
    Apr 1999
    Posts
    27,426

    Re: C struct alignment across compilers and operating systems

    Quote Originally Posted by eduardschreder View Post
    it's just too much work to implement a GetAttributXyz(HANDLE struct, int field); for each field.
    So write a code generator. An input file has the field names, and the code generator generates each Get/Set function depending on the field name.

    Regards,

    Paul McKenzie

  11. #11
    Join Date
    Oct 2013
    Posts
    5

    Re: C struct alignment across compilers and operating systems

    Alright.

    So write a code generator.
    I am less concerned about myself, than about the users, who end up having to call all those functions.

    I took a look at two large applications who make available C pugin APIs (pidgin and vlc player), and they DO pass around pointers to structs! However, due to the sheer mass of code involved, i was unable to identify HOW exactly it is all implemented. Though i didnt see any alignment hacks.



    Anyways, i'll let all this sink in for a while.



    Thanks for your time patience here.

  12. #12
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,329

    Re: C struct alignment across compilers and operating systems

    Just because something can be done and has been done, doesn't mean it should be done. Just because something seems to work doesn't mean it actually works properly all the time for everybody.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  13. #13
    Join Date
    Apr 1999
    Posts
    27,426

    Re: C struct alignment across compilers and operating systems

    Quote Originally Posted by eduardschreder View Post
    I took a look at two large applications who make available C pugin APIs (pidgin and vlc player), and they DO pass around pointers to structs!
    This doesn't take away from what was stated previously. If the structs are byte-aligned, then there is no issue.

    The Windows API has functions where you need to declare a struct, fill it with info, and pass a pointer to it to the API functions (look at GetOpenFileName and the OPENFILENAME struct). The trick is that the API headers ensure that the structs are byte-aligned, otherwise the API function will have erratic behaviour (probably crash) when the program is run.

    On the development side, when you #include the header for the API struct, you get byte-alignment for that struct in a seamless fashion. You wouldn't even know that the structs are byte-aligned until you look deep into the header files. So more than likely, the applications you brought up have turned on byte-alignment for the struct using #ifdef, custom header files depending on compiler used, etc.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; October 24th, 2013 at 09:51 AM.

  14. #14
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,329

    Re: C struct alignment across compilers and operating systems

    Under MSVC, one of the properties is to specify struct member alignment. 1, 2, 4, 8, 16 bytes or default (4 bytes on my system).
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center