parse a string to bytearray to struct
I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size, correct?
I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct. I also noticed the size of the byte array written is different than the structure. Why is that?
If there is better approach, please suggest one.
Code:
#include "stdafx.h"
#include <stdio.h>
#include <string>
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
void GetByteArray(const std::string& str, unsigned char* bytes)
{
// get chars 0-3 from str to get string for year
std::string strYear = str.substr(0,4);
short sYear = (short)atoi(strYear.c_str());
// get chars 5-6 from str to get string for month
std::string strMonth = str.substr(5,2);
short sMonth = (short)atoi(strMonth.c_str());
// get chars 8-9 from str to get string for day
std::string strDay = str.substr(8,2);
short sDay = (short)atoi(strDay.c_str());
// get chars 11-17 from str to get string for Val1
std::string strVal1 = str.substr(11,7);
float fVal1 = (float)atof(strVal1.c_str());
// get chars 19-25 from str to get string for Val2
std::string strVal2 = str.substr(19,7);
float fVal2 = (float)atof(strVal2.c_str());
size_t i = 0;
// cast strYear to short and append to byte array
memcpy(&bytes[i], &sYear, sizeof(short));
i += sizeof(short);
// cast strMonth to short and append to byte array
memcpy(&bytes[i], &sMonth, sizeof(short));
i += sizeof(short);
// cast strDay to short and append to byte array
memcpy(&bytes[i], &sDay, sizeof(short));
i += sizeof(short);
// cast strAsk to float and append to byte array
memcpy(&bytes[i], &fVal1, sizeof(float));
i += sizeof(float);
// cast strBid to float and append to byte array
memcpy(&bytes[i], &fVal2, sizeof(float));
i += sizeof(float);
}
int main(int argc, char* argv[])
{
// string to parse
std::string strData = "2011.06.30,1.40562,1.40559";
unsigned char MyBytes[sizeof(TD)];
GetByteArray(strData, MyBytes);
// i = 14 from GetByteArray
int i = sizeof(TD); // i = 16 from structure size
// error here! my floats in td are undefined
TD* td = (TD*)MyBytes;
return 0;
}
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
cbpetro
I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct.
What made you think that there isn't any padding bytes between those struct members?
In other words, adding sizeof(type) to your "i" variable isn't guaranteed to get you to the specific member. Bottom line is that unless the struct has no padding bytes in between those members (some compilers have a #pragma or some other means to pack the struct), your code isn't going to work.
Regards,
Paul McKenzie
Re: parse a string to bytearray to struct
ok thanks. This works for my microsoft 32-bit compiler.
Code:
#pragma pack(push) // push current alignment to stack
#pragma pack(1) // set alignment to 1 byte boundary
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
#pragma pack(pop) // restore original alignment from stack
Re: parse a string to bytearray to struct
Quote:
I also noticed the size of the byte array written is different than the structure
Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member? You should never determine the size of the structure this way. Always use sizeof() to get the size of any type.
Also, your code did make the byte array the same size as the structure, even in your original non-working code. You used sizeof(TD) to determine the number of bytes in the array.
Regards,
Paul McKenzie
Re: parse a string to bytearray to struct
Quote:
Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member?
No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.
Code:
int j = sizeof(TD); // j = 14 from structure size with pragma
int k = sizeof(TD); // k = 16 from structure size without pragma
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
cbpetro
No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.
No, you didn't understand.
In your original code and post, you stated that the sizeof the byte array was different than the size of the structure. That is false.
The size of the byte array was always the same size as the structure. Would you like proof? Take away the pragma, and do this:
Code:
#include <assert.h>
unsigned char MyBytes[sizeof(TD)];
//...
assert( sizeof(MyBytes) == sizeof(TD) );
You will see that assertion will always be true.
Regards,
Paul McKenzie
Re: parse a string to bytearray to struct
I see, you are correct. But I don't understand why at the end of my "GetByteArray" function the sum of the bytearray is 14. Is summing up the size of each variable for placement in the byte array correct?
Code:
void GetByteArray(const std::string& str, unsigned char* bytes)
{
// get chars 0-3 from str to get string for year
std::string strYear = str.substr(0,4);
short sYear = (short)atoi(strYear.c_str());
// get chars 5-6 from str to get string for month
....
...
..
// cast strAsk to float and append to byte array
memcpy(&bytes[i], &fVal1, sizeof(float));
i += sizeof(float);
// cast strBid to float and append to byte array
memcpy(&bytes[i], &fVal2, sizeof(float));
i += sizeof(float);
// i = 14 ?
}
If I add another variable to the byte array it will be at location 14 when the byte array size is 16. This does not make sense to me.
Re: parse a string to bytearray to struct
Read Paul's original response:
Quote:
What made you think that there isn't any padding bytes between those struct members?
The sizeof a struct isn't necessarily the sum of the sizeof it's parts.
Viggy
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
cbpetro
ok thanks. This works for my microsoft 32-bit compiler.
Code:
#pragma pack(push) // push current alignment to stack
#pragma pack(1) // set alignment to 1 byte boundary
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
#pragma pack(pop) // restore original alignment from stack
GCC way
Code:
#pragma pack(push, 1)
struct TD {
...
};
#pragma(pop)
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
MrViggy
Read Paul's original response:
The sizeof a struct isn't necessarily the sum of the sizeof it's parts.
Viggy
But we've determined that the size of struct = size of byte array. Therefore the bytearray is padded as well, correct?
Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.
Re: parse a string to bytearray to struct
No. An array is a continuous chunk of memory, with each item in the array having the same size. It's not a grouping of different things. A struct is a grouping of (possibly differently sized) things.
Viggy
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
cbpetro
Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.
You have to keep your struct packed on the byte boundary to not introduce any gaps in-between the members.
The issue you need to consider is if you or someone else changes the order of the members of your struct. Then you have to change the reading code to match the order of the items. Another thing to consider is if you add a member in the middle of the struct.
Maybe you should consider "auto-generating" the struct and the code to read it, as right now, manually making changes could lead to hard-to-find bugs. Write a program that creates a header of the struct, and the C++ code to process it.
Regards,
Paul McKenzie
Re: parse a string to bytearray to struct
Quote:
Originally Posted by
cbpetro
I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size...
If you really want the smaller size, note that both month and day fit in single byte.
Coincidentally, since short and two bytes take exactly 4 bytes, you wouldn’t have a packing issue…:)
Re: parse a string to bytearray to struct
As mentioned, it is usually not possible to assume a struct will align correctly on any given byte array. That, or the amount of assumptions/restrictions you need to do are usually prohibitive.
What you are trying to do is not unlike serialization. The easiest method (and usually recommend method I believe), is to just take your attributes 1 by 1, and write them to the array. To unserialize, read your blocks and write them to your attributes 1 by 1.
It might sound like a hassle, but it is just writing 2 functions, and performance usually isn't so mind-bogglingly tight you can't afford it. Besides, you already wrote 1 of the two functions, why not just write the other?
Code:
#include <string> //string
#include <iostream> //cout
#include <iterator> //ostream_iterator
#include <algorithm> //string
#include <iomanip> //hex setw
const size_t ARRAY_SIZE = sizeof(short)*3 + sizeof(float)*2;
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
void WriteByteArray(const TD& td, unsigned char* bytes)
{
size_t i = 0;
// cast strYear to short and append to byte array
(short&) bytes[i] = td.sYear;
i += sizeof(short);
// cast strMonth to short and append to byte array
(short&) bytes[i] = td.sMonth;
i += sizeof(short);
// cast strDay to short and append to byte array
(short&) bytes[i] = td.sDay;
i += sizeof(short);
// cast strAsk to float and append to byte array
(float&) bytes[i] = td.fVal1;
i += sizeof(float);
// cast strBid to float and append to byte array
(float&) bytes[i] = td.fVal2;
//i += sizeof(float);
}
void ReadByteArray(unsigned char const * bytes, TD& td)
{
size_t i = 0;
// cast strYear to short and append to byte array
td.sYear = (short&) bytes[i];
i += sizeof(short);
// cast strMonth to short and append to byte array
td.sMonth = (short&) bytes[i];
i += sizeof(short);
// cast strDay to short and append to byte array
td.sDay = (short&) bytes[i];
i += sizeof(short);
// cast strAsk to float and append to byte array
td.fVal1 = (float&) bytes[i];
i += sizeof(float);
// cast strBid to float and append to byte array
td.fVal2 = (float&) bytes[i];
//i += sizeof(float);
}
int main(int argc, char* argv[])
{
TD td = {2011, 7, 15, 1.5, 1e10};
unsigned char MyBytes[ARRAY_SIZE];
std::cout << "Array size: " << ARRAY_SIZE << std::endl;
std::cout << std::hex << std::setw(3);
WriteByteArray(td, MyBytes);
std::cout << "encoded data:" << std::endl;
std::copy(MyBytes, MyBytes+ARRAY_SIZE, std::ostream_iterator<int>(std::cout, ", "));
std::cout << std::endl;
unsigned char MyBytes2[ARRAY_SIZE] =
{
0x9e, 0x07, // 1950
0x0c, 0x00, // 12
0x1e, 0x00, // 30
0xf9, 0x02, 0x15, 0x50, //1e10
0x00, 0x00, 0xc0, 0x3f //1.5
};
ReadByteArray(MyBytes2, td);
std::cout << std::dec << std::setw(0);
std::cout << "decoded data:" << std::endl;
std::cout << td.sYear << " " << td.sMonth << " " << td.sDay << " " << td.fVal1 << " " << td.fVal2 << std::endl;
return 0;
}
Note that neither endianness nor the fact that "short" is not the same on all machines, were taken into account here. If endianness should be taken into account, then you have use this kind of per-object approach anyways.
