-
July 14th, 2011, 01:57 PM
#1
parse a string to bytearray to struct
I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size, correct?
I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct. I also noticed the size of the byte array written is different than the structure. Why is that?
If there is better approach, please suggest one.
Code:
#include "stdafx.h"
#include <stdio.h>
#include <string>
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
void GetByteArray(const std::string& str, unsigned char* bytes)
{
// get chars 0-3 from str to get string for year
std::string strYear = str.substr(0,4);
short sYear = (short)atoi(strYear.c_str());
// get chars 5-6 from str to get string for month
std::string strMonth = str.substr(5,2);
short sMonth = (short)atoi(strMonth.c_str());
// get chars 8-9 from str to get string for day
std::string strDay = str.substr(8,2);
short sDay = (short)atoi(strDay.c_str());
// get chars 11-17 from str to get string for Val1
std::string strVal1 = str.substr(11,7);
float fVal1 = (float)atof(strVal1.c_str());
// get chars 19-25 from str to get string for Val2
std::string strVal2 = str.substr(19,7);
float fVal2 = (float)atof(strVal2.c_str());
size_t i = 0;
// cast strYear to short and append to byte array
memcpy(&bytes[i], &sYear, sizeof(short));
i += sizeof(short);
// cast strMonth to short and append to byte array
memcpy(&bytes[i], &sMonth, sizeof(short));
i += sizeof(short);
// cast strDay to short and append to byte array
memcpy(&bytes[i], &sDay, sizeof(short));
i += sizeof(short);
// cast strAsk to float and append to byte array
memcpy(&bytes[i], &fVal1, sizeof(float));
i += sizeof(float);
// cast strBid to float and append to byte array
memcpy(&bytes[i], &fVal2, sizeof(float));
i += sizeof(float);
}
int main(int argc, char* argv[])
{
// string to parse
std::string strData = "2011.06.30,1.40562,1.40559";
unsigned char MyBytes[sizeof(TD)];
GetByteArray(strData, MyBytes);
// i = 14 from GetByteArray
int i = sizeof(TD); // i = 16 from structure size
// error here! my floats in td are undefined
TD* td = (TD*)MyBytes;
return 0;
}
-
July 14th, 2011, 02:14 PM
#2
Re: parse a string to bytearray to struct
Originally Posted by cbpetro
I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct.
What made you think that there isn't any padding bytes between those struct members?
In other words, adding sizeof(type) to your "i" variable isn't guaranteed to get you to the specific member. Bottom line is that unless the struct has no padding bytes in between those members (some compilers have a #pragma or some other means to pack the struct), your code isn't going to work.
Regards,
Paul McKenzie
Last edited by Paul McKenzie; July 14th, 2011 at 02:16 PM.
-
July 14th, 2011, 02:44 PM
#3
Re: parse a string to bytearray to struct
ok thanks. This works for my microsoft 32-bit compiler.
Code:
#pragma pack(push) // push current alignment to stack
#pragma pack(1) // set alignment to 1 byte boundary
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
#pragma pack(pop) // restore original alignment from stack
-
July 14th, 2011, 02:52 PM
#4
Re: parse a string to bytearray to struct
I also noticed the size of the byte array written is different than the structure
Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member? You should never determine the size of the structure this way. Always use sizeof() to get the size of any type.
Also, your code did make the byte array the same size as the structure, even in your original non-working code. You used sizeof(TD) to determine the number of bytes in the array.
Regards,
Paul McKenzie
-
July 14th, 2011, 03:06 PM
#5
Re: parse a string to bytearray to struct
Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member?
No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.
Code:
int j = sizeof(TD); // j = 14 from structure size with pragma
int k = sizeof(TD); // k = 16 from structure size without pragma
-
July 14th, 2011, 03:09 PM
#6
Re: parse a string to bytearray to struct
Originally Posted by cbpetro
No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.
No, you didn't understand.
In your original code and post, you stated that the sizeof the byte array was different than the size of the structure. That is false.
The size of the byte array was always the same size as the structure. Would you like proof? Take away the pragma, and do this:
Code:
#include <assert.h>
unsigned char MyBytes[sizeof(TD)];
//...
assert( sizeof(MyBytes) == sizeof(TD) );
You will see that assertion will always be true.
Regards,
Paul McKenzie
-
July 14th, 2011, 03:28 PM
#7
Re: parse a string to bytearray to struct
I see, you are correct. But I don't understand why at the end of my "GetByteArray" function the sum of the bytearray is 14. Is summing up the size of each variable for placement in the byte array correct?
Code:
void GetByteArray(const std::string& str, unsigned char* bytes)
{
// get chars 0-3 from str to get string for year
std::string strYear = str.substr(0,4);
short sYear = (short)atoi(strYear.c_str());
// get chars 5-6 from str to get string for month
....
...
..
// cast strAsk to float and append to byte array
memcpy(&bytes[i], &fVal1, sizeof(float));
i += sizeof(float);
// cast strBid to float and append to byte array
memcpy(&bytes[i], &fVal2, sizeof(float));
i += sizeof(float);
// i = 14 ?
}
If I add another variable to the byte array it will be at location 14 when the byte array size is 16. This does not make sense to me.
-
July 14th, 2011, 03:31 PM
#8
Re: parse a string to bytearray to struct
Read Paul's original response:
What made you think that there isn't any padding bytes between those struct members?
The sizeof a struct isn't necessarily the sum of the sizeof it's parts.
Viggy
-
July 14th, 2011, 03:36 PM
#9
Re: parse a string to bytearray to struct
Originally Posted by cbpetro
ok thanks. This works for my microsoft 32-bit compiler.
Code:
#pragma pack(push) // push current alignment to stack
#pragma pack(1) // set alignment to 1 byte boundary
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
#pragma pack(pop) // restore original alignment from stack
GCC way
Code:
#pragma pack(push, 1)
struct TD {
...
};
#pragma(pop)
-
July 14th, 2011, 03:48 PM
#10
Re: parse a string to bytearray to struct
Originally Posted by MrViggy
Read Paul's original response:
The sizeof a struct isn't necessarily the sum of the sizeof it's parts.
Viggy
But we've determined that the size of struct = size of byte array. Therefore the bytearray is padded as well, correct?
Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.
-
July 14th, 2011, 03:53 PM
#11
Re: parse a string to bytearray to struct
No. An array is a continuous chunk of memory, with each item in the array having the same size. It's not a grouping of different things. A struct is a grouping of (possibly differently sized) things.
Viggy
-
July 14th, 2011, 03:59 PM
#12
Re: parse a string to bytearray to struct
Originally Posted by cbpetro
Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.
You have to keep your struct packed on the byte boundary to not introduce any gaps in-between the members.
The issue you need to consider is if you or someone else changes the order of the members of your struct. Then you have to change the reading code to match the order of the items. Another thing to consider is if you add a member in the middle of the struct.
Maybe you should consider "auto-generating" the struct and the code to read it, as right now, manually making changes could lead to hard-to-find bugs. Write a program that creates a header of the struct, and the C++ code to process it.
Regards,
Paul McKenzie
-
July 14th, 2011, 04:37 PM
#13
Re: parse a string to bytearray to struct
Originally Posted by cbpetro
I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size...
If you really want the smaller size, note that both month and day fit in single byte.
Coincidentally, since short and two bytes take exactly 4 bytes, you wouldn’t have a packing issue…
Vlad - MS MVP [2007 - 2012] - www.FeinSoftware.com
Convenience and productivity tools for Microsoft Visual Studio:
FeinWindows - replacement windows manager for Visual Studio, and more...
-
July 15th, 2011, 03:09 AM
#14
Re: parse a string to bytearray to struct
As mentioned, it is usually not possible to assume a struct will align correctly on any given byte array. That, or the amount of assumptions/restrictions you need to do are usually prohibitive.
What you are trying to do is not unlike serialization. The easiest method (and usually recommend method I believe), is to just take your attributes 1 by 1, and write them to the array. To unserialize, read your blocks and write them to your attributes 1 by 1.
It might sound like a hassle, but it is just writing 2 functions, and performance usually isn't so mind-bogglingly tight you can't afford it. Besides, you already wrote 1 of the two functions, why not just write the other?
Code:
#include <string> //string
#include <iostream> //cout
#include <iterator> //ostream_iterator
#include <algorithm> //string
#include <iomanip> //hex setw
const size_t ARRAY_SIZE = sizeof(short)*3 + sizeof(float)*2;
struct TD
{
short sYear;
short sMonth;
short sDay;
float fVal1;
float fVal2;
};
void WriteByteArray(const TD& td, unsigned char* bytes)
{
size_t i = 0;
// cast strYear to short and append to byte array
(short&) bytes[i] = td.sYear;
i += sizeof(short);
// cast strMonth to short and append to byte array
(short&) bytes[i] = td.sMonth;
i += sizeof(short);
// cast strDay to short and append to byte array
(short&) bytes[i] = td.sDay;
i += sizeof(short);
// cast strAsk to float and append to byte array
(float&) bytes[i] = td.fVal1;
i += sizeof(float);
// cast strBid to float and append to byte array
(float&) bytes[i] = td.fVal2;
//i += sizeof(float);
}
void ReadByteArray(unsigned char const * bytes, TD& td)
{
size_t i = 0;
// cast strYear to short and append to byte array
td.sYear = (short&) bytes[i];
i += sizeof(short);
// cast strMonth to short and append to byte array
td.sMonth = (short&) bytes[i];
i += sizeof(short);
// cast strDay to short and append to byte array
td.sDay = (short&) bytes[i];
i += sizeof(short);
// cast strAsk to float and append to byte array
td.fVal1 = (float&) bytes[i];
i += sizeof(float);
// cast strBid to float and append to byte array
td.fVal2 = (float&) bytes[i];
//i += sizeof(float);
}
int main(int argc, char* argv[])
{
TD td = {2011, 7, 15, 1.5, 1e10};
unsigned char MyBytes[ARRAY_SIZE];
std::cout << "Array size: " << ARRAY_SIZE << std::endl;
std::cout << std::hex << std::setw(3);
WriteByteArray(td, MyBytes);
std::cout << "encoded data:" << std::endl;
std::copy(MyBytes, MyBytes+ARRAY_SIZE, std::ostream_iterator<int>(std::cout, ", "));
std::cout << std::endl;
unsigned char MyBytes2[ARRAY_SIZE] =
{
0x9e, 0x07, // 1950
0x0c, 0x00, // 12
0x1e, 0x00, // 30
0xf9, 0x02, 0x15, 0x50, //1e10
0x00, 0x00, 0xc0, 0x3f //1.5
};
ReadByteArray(MyBytes2, td);
std::cout << std::dec << std::setw(0);
std::cout << "decoded data:" << std::endl;
std::cout << td.sYear << " " << td.sMonth << " " << td.sDay << " " << td.fVal1 << " " << td.fVal2 << std::endl;
return 0;
}
Note that neither endianness nor the fact that "short" is not the same on all machines, were taken into account here. If endianness should be taken into account, then you have use this kind of per-object approach anyways.
Last edited by monarch_dodra; July 15th, 2011 at 08:51 AM.
Reason: typo
Is your question related to IO?
Read this C++ FAQ article at parashift by Marshall Cline. In particular points 1-6.
It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|