Help:: Cross Platform Struct Member Alignment

[DaDB1]

I am building a windows gui app to manage a database. The database is created and runs on a unix system, and I have tools written in pure C that I use to manipulate the database from the command line, but I wanted a gui app to make things more user friendy. The tools I have written when compiled either by VC6 as a C program, gcc on linux, or cygwin all work fine. Transfering the database manipulation code to a VC6 project and compiling as part of a C++ gui app results in the structures not being able to be read due to improper bit field alignment within the structures.

In short, the database can be read/written via a C compiled source, but not via C++ compiled source.

When determining structure size - printf("d", sizeof(struct my_struct));
In a C program it prints as 3524 bytes. I assume that most C compilers align the bit fields in a 4 bit structure.
In a C++ program it prints the same structure as 3508 bytes, and that's assuming that the compiler uses an 8 bit structure alignment.

I have tried the /ZP(n) compiler switch, as well as the #pragma pack(...) directives. The min /ZP1 results in a struct size of 3501 bytes, while the /ZP4, /ZP8, and max /ZP16 all result in a struct size of 3508 bytes. I have also tried the /J compiler switch to set char alignment, and the UNALIGNED keyword when declaring structure pointers, all of these methods failed to align the structure bit fields to a readable state by a C++ program.

I assumed that since the original program and database is completely portable C programming, that I could also port it to C++. Is there anything that I am missing, or is this just impossible to do?

Thanks...

[cup]

Small correction: It is byte structure alignment: not bit structure alignment.

1) When making up the structure are all the shorts/longs together and are all the chars together or are they mixed? Different implementations will pack struct{ char x; char z; long y; } differently.

2) Are you using any ints? What happens when you change them to short/long?

Whenever I've had this sort of problem it is normally the packing and the ints.

[stober]

With a little testing, and without seeing any of your structures, your problem could be the way bit fields are declared. Using VC6 compiler these two structs have different sizes even with the same packing: The only difference is that in str1 the bit fields were declared as unsigned long, while in str2 they are unsigned short. The size of the structures are the same whether they are compiled as a C (in a *.c file) or C++ (in a C++ file). Personally, I always use byte-align (pack 1) to get the smallest possible size for structures that are transferred between applications, such as database client/server programs.


struct str1
{
char name[23];
unsigned long b1:1;
unsigned long b2:1;
int x;
char last[21];
};

struct str2
{
char name[23];
unsigned short b1:1;
unsigned short b2:1;
int x;
char last[21];
};

[AnthonyMai]

The problem is probably the sizeof(char) is different in C and C++. In C it is 4 bytes, in C++ it is 1. That's why your C++ structure looks smaller even if you use 8 bytes align versus 4 bytes align.

It would help if you can post how your structure is defined.

[Yves M]

sizeof(char) is different in C and C++. In C it is 4 bytes, in C++ it is 1

What are you on about ? sizeof(char) = 1 in both C and C++.

You have just failed your interview for C programmer :p

[DaDB1]

What I meant by bit field alignment is under 'Storage and Alignment of Structures' in the MSDN:
"Adjacent bit fields are packed into the same 1-, 2-, or 4-byte allocation unit if the integral types are the same size and if the next bit field fits into the current allocation unit without crossing the boundary imposed by the common alignment requirements of the bit fields. "

What I am trying to compemnsate for is the fact that:
C compilers: Packs structures on 4-byte boundaries
C++ compilers: Packs structures on 8-byte boundaries (default)
and even when changing the C++ compiler to use 4-byte boundaries, it fails to properly align the structure bit fields.


There are two separate databases I'd like to be able to read with a C++ program. One is the player entries database, the other is the player 'team' (order) database. They both have the same structure member alignment problem, so I won't go into details about the order database. The one displayed here is the player entries database,

This database is existing and cannot be remade. It can only be modified locally to be read by a C++ program, and switching of variables is not an option as I intend to write to the file and it needs to remain readable by the original C program afterwards. Possibly adding filler(buffer) bytes as needed to align the structure might be possible, but determining where to add them is a problem.

The structure in the existing player database consists of two parts. First a header at the beginning which contains the number of entries that will follow, then each individual entry follows. This is a C program, so some types were defined:

typedef unsigned short word;
typedef int boolean;

Then some relevant enums:

enum
{
TAG_FILE_NAME_LENGTH = 8,
MAXIMUM_LOGIN_LENGTH = 15,
MAXIMUM_PASSWORD_LENGTH = 15,
MAXIMUM_PLAYER_NAME_LENGTH = 31,
MAXIMUM_DESCRIPTION_LENGTH = 431,
MAXIMUM_NUMBER_OF_GAME_TYPES = 16,
MAXIMUM_BUDDIES = 16,
MAXIMUM_ORDER_MEMBERS = 16,
OLD_MAXIMUM_ORDER_MEMBERS = 32,
MAXIMUM_PACKED_PLAYER_DATA_LENGTH = 128,
NUMBER_OF_TRACKED_OPPONENTS = 10
};

enum
{
_overall,
_pure,
_third_party,
MAXIMUM_NUMBER_OF_PURE_MAPS = 128
};


/*----------Header structure-----------*/

struct net_user_db_header
{
unsigned long player_count; // number of entries
unsigned long unused[40];
};


/*----------Entry structure------------*/

struct net_user_db_entry
{
unsigned long signature; // beginning of each entry
struct net_player_datum player;
};


/*-------------------------------------*/



The entry structure contains a structure:

/*----------Parts of the Entry structure------------*/

struct net_player_datum
{
unsigned long player_id;
char login[MAXIMUM_LOGIN_LENGTH + 1];
char password[MAXIMUM_PASSWORD_LENGTH + 1];
boolean administrator_flag;
unsigned int special_flags;
boolean player_is_banned_flag;
boolean unused_flags[5];
long last_login_ip_address;
long last_login_time;
long last_game_time;
long last_ranked_game_time;
long room_id;
struct buddy_entry buddies[MAXIMUM_BUDDIES];
short order_index;
short icon_index;
short icon_collection_name[TAG_FILE_NAME_LENGTH + 1];
char name[MAXIMUM_PLAYER_NAME_LENGTH + 1];
char team_name[MAXIMUM_PLAYER_NAME_LENGTH + 1];
struct rgb_color primary_color, secondary_color;
char description[MAXIMUM_DESCRIPTION_LENGTH + 1];
long banned_time;
long ban_duration;
long times_banned;
short country_code;
short style;
struct net_player_score_datum unranked_score; struct net_player_score_datum ranked_score; struct net_player_score_datum ranked_scores_by_game_type[MAXIMUM_NUMBER_OF_GAME_TYPES];
struct net_player_score_datum pure_ranked_score;
struct net_player_score_datum pure_ranked_scores_by_game_type[MAXIMUM_NUMBER_OF_GAME_TYPES];
struct net_player_score_datum third_party_ranked_score;
struct net_player_score_datum third_party_ranked_scores_by_game_type[MAXIMUM_NUMBER_OF_GAME_TYPES];
unsigned short games_by_map[MAXIMUM_NUMBER_OF_PURE_MAPS];


};


The net_player_datum structure itself contains other structures:

/*----------Parts of the net_player_datum structure------------*/


struct buddy_entry
{
unsigned long player_id;
char active;
char c[3];
};


struct rgb_color
{
word red, green, blue;
word flags;
};


struct net_player_score_datum

short games_played;
short wins, losses, ties;
long damage_inflicted, damage_received;
short disconnects;
word pad;
short points;
short rank;
short highest_points;
short highest_rank;
unsigned long numerical_rank;
char unused[16];
};

/*----------EOF------------*/

[AnthonyMai]

What are you on about ? sizeof(char) = 1 in both C and C++.

You have just failed your interview for C programmer


sizeof(char) may not be 1 in some old code and old compiler. At some time point they decided that sizeof() of any data type would be the minimum storage required to store the data, so on some system sizeof(char) could be 4. That may have changed.

See this thread:

[URL=http://www.codeguru.com/forum/showthread.php?&threadid=181092&highlight=sizeofchar[/URL]

[AnthonyMai]

See this link:

www.codeguru.com/forum/showthread.php?&threadid=181092&highlight=sizeofchar (http://www.codeguru.com/forum/showthread.php?&threadid=181092&highlight=sizeofchar)

[Philip Nicoletti]

sizeof(char) = 1 in C++ (always)

in C, I thought sizeof(char) = sizeof(int)

[stober]

DaDb1: A couple things struck me as I was looking at your structures. This may, or may not, be relevalt, but are you sure the size of a boolean and word are the same ? As I recall from my old C programming, boolean was defined as either char or int, depending on the compiler, and the sizeof(word) was either 16 or 32 bytes, again compiler dependent.

[stober]

Originally posted by Philip Nicoletti
sizeof(char) = 1 in C++ (always)

in C, I thought sizeof(char) = sizeof(int)

Not on the C compilers I used (Lattice C, Microsoft, and Borland). I first learned C in 1980 and it has always been 1 byte on those compilers.

Maybe what you have in mind is the number of bytes pushed onto the stack during function calls -- that is different. In that case, 16-bit code pushes 2 bytes and 32-bit programs push 4 bytes because of the increased speed.

[DaDB1]

Originally posted by stober
DaDb1: A couple things struck me as I was looking at your structures. This may, or may not, be relevalt, but are you sure the size of a boolean and word are the same ? As I recall from my old C programming, boolean was defined as either char or int, depending on the compiler, and the sizeof(word) was either 16 or 32 bytes, again compiler dependent.

Thank you Stober, you hit the nail on the head. I've had problems with structure member alignment in the past, and recieved similar errors then, so I assumed that it was the same problem this time.
As far as I know, C doesn't define the boolean or word data types. In my code, boolean is defined as type int. In C++ boolean is defined as unsigned char and BOOL is defined as type int.


By making this change I was able to get this thing flying through thousands of entries:

#if defined(__cplusplus)
#define boolean BOOL
#else
typedef int boolean;
#endif // #if defined(__cplusplus)


Thank you very much for the help with this, I might not have thought to look at the typedef's as I thought boolean was defined as an int in C++ in the first place.

Thanks again,
Alan

[JamesSchumacher]

But what type of packing does the structure used in the database have?

If you can rewrite both apps - the standard way of controlling structure packing is via the <pshpackX.h> header file where X is defined to be the packing alignment. Of course, include <poppack.h> to set back to default alignment.

#include <pshpack1.h>

struct WHATEVER {
// ....
};

#include <poppack.h>

[DaDB1]

Originally posted by JamesSchumacher
But what type of packing does the structure used in the database have?

If you can rewrite both apps - the standard way of controlling structure packing is via the <pshpackX.h> header file where X is defined to be the packing alignment. Of course, include <poppack.h> to set back to default alignment.

#include <pshpack1.h>

struct WHATEVER {
// ....
};

#include <poppack.h>

I don't have the option of re-writing the old app (it's 10's of thousands of lines of code), nor do I have the option of re-creating the database it uses. I just wanted to access the database through C++, which now with the typedef change that Stober picked up on and some minor structure packing changes I am now able to do.

Thank you all for your help. :)