parse a string to bytearray to struct
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 14 of 14

Thread: parse a string to bytearray to struct

  1. #1
    Join Date
    Aug 2000
    Location
    Virginia, US
    Posts
    158

    parse a string to bytearray to struct

    I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size, correct?

    I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct. I also noticed the size of the byte array written is different than the structure. Why is that?


    If there is better approach, please suggest one.

    Code:
    #include "stdafx.h"
    #include <stdio.h>
    #include <string>
    
    struct TD
    {
    	short sYear;
    	short sMonth;
    	short sDay;
    
            float fVal1;
    	float fVal2;
    };
    
    void GetByteArray(const std::string& str, unsigned char* bytes)
    {
    	// get chars 0-3 from str to get string for year 
    	std::string strYear = str.substr(0,4);
    	short sYear = (short)atoi(strYear.c_str());
    	// get chars 5-6 from str to get string for month 
    	std::string strMonth = str.substr(5,2);
    	short sMonth = (short)atoi(strMonth.c_str());
    	// get chars 8-9 from str to get string for day 
    	std::string strDay = str.substr(8,2);
    	short sDay = (short)atoi(strDay.c_str());
    	// get chars 11-17 from str to get string for Val1 
    	std::string strVal1 = str.substr(11,7);
    	float fVal1 = (float)atof(strVal1.c_str());
    	// get chars 19-25 from str to get string for Val2 
    	std::string strVal2 = str.substr(19,7);
    	float fVal2 = (float)atof(strVal2.c_str());
    
    	size_t i = 0;
    	// cast strYear to short and append to byte array
    	memcpy(&bytes[i], &sYear, sizeof(short));
    	i += sizeof(short);
    
    	// cast strMonth to short and append to byte array
    	memcpy(&bytes[i], &sMonth, sizeof(short));
    	i += sizeof(short);
    
    	// cast strDay to short and append to byte array
    	memcpy(&bytes[i], &sDay, sizeof(short));
    	i += sizeof(short);
    
    	// cast strAsk to float and append to byte array
    	memcpy(&bytes[i], &fVal1, sizeof(float));
    	i += sizeof(float);
    
    	// cast strBid to float and append to byte array
    	memcpy(&bytes[i], &fVal2, sizeof(float));
    	i += sizeof(float);
    }
    
    int main(int argc, char* argv[])
    {
    	// string to parse
    	std::string strData = "2011.06.30,1.40562,1.40559";
    
    	unsigned char MyBytes[sizeof(TD)];
    	GetByteArray(strData, MyBytes);
    
    	// i = 14 from GetByteArray
    	int i = sizeof(TD); // i = 16 from structure size
    
    	// error here! my floats in td are undefined
    	TD* td = (TD*)MyBytes;
    
    	return 0;
    }

  2. #2
    Join Date
    Apr 1999
    Posts
    27,427

    Re: parse a string to bytearray to struct

    Quote Originally Posted by cbpetro View Post
    I wrote a function "GetByteArray" to parse the string into the byte array. The problem I have is the byte array does not cast back the floats into my TD struct.
    What made you think that there isn't any padding bytes between those struct members?

    In other words, adding sizeof(type) to your "i" variable isn't guaranteed to get you to the specific member. Bottom line is that unless the struct has no padding bytes in between those members (some compilers have a #pragma or some other means to pack the struct), your code isn't going to work.

    Regards,

    Paul McKenzie
    Last edited by Paul McKenzie; July 14th, 2011 at 02:16 PM.

  3. #3
    Join Date
    Aug 2000
    Location
    Virginia, US
    Posts
    158

    Re: parse a string to bytearray to struct

    ok thanks. This works for my microsoft 32-bit compiler.

    Code:
    #pragma pack(push)  // push current alignment to stack
    #pragma pack(1)     // set alignment to 1 byte boundary
    struct TD
    {
    	short sYear;
    	short sMonth;
    	short sDay;
    
            float fVal1;
    	float fVal2;
    };
    #pragma pack(pop)   // restore original alignment from stack

  4. #4
    Join Date
    Apr 1999
    Posts
    27,427

    Re: parse a string to bytearray to struct

    I also noticed the size of the byte array written is different than the structure
    Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member? You should never determine the size of the structure this way. Always use sizeof() to get the size of any type.

    Also, your code did make the byte array the same size as the structure, even in your original non-working code. You used sizeof(TD) to determine the number of bytes in the array.

    Regards,

    Paul McKenzie

  5. #5
    Join Date
    Aug 2000
    Location
    Virginia, US
    Posts
    158

    Re: parse a string to bytearray to struct

    Going back, how did you "notice" this? Were you (erroneously) adding up the number of bytes of each member?
    No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.

    Code:
    	int j = sizeof(TD); // j = 14 from structure size with pragma
    	int k = sizeof(TD); // k = 16 from structure size without pragma

  6. #6
    Join Date
    Apr 1999
    Posts
    27,427

    Re: parse a string to bytearray to struct

    Quote Originally Posted by cbpetro View Post
    No, I don't believe so. The size of my TD structure is different with and without the pragma directives. Adding up the number of bytes always is 14.
    No, you didn't understand.

    In your original code and post, you stated that the sizeof the byte array was different than the size of the structure. That is false.

    The size of the byte array was always the same size as the structure. Would you like proof? Take away the pragma, and do this:
    Code:
    #include <assert.h>
    
    unsigned char MyBytes[sizeof(TD)];
    //...
    assert( sizeof(MyBytes) == sizeof(TD) );
    You will see that assertion will always be true.

    Regards,

    Paul McKenzie

  7. #7
    Join Date
    Aug 2000
    Location
    Virginia, US
    Posts
    158

    Re: parse a string to bytearray to struct

    I see, you are correct. But I don't understand why at the end of my "GetByteArray" function the sum of the bytearray is 14. Is summing up the size of each variable for placement in the byte array correct?

    Code:
    void GetByteArray(const std::string& str, unsigned char* bytes)
    {
    	// get chars 0-3 from str to get string for year 
    	std::string strYear = str.substr(0,4);
    	short sYear = (short)atoi(strYear.c_str());
    	// get chars 5-6 from str to get string for month 
    
            ....
            ...
            ..
    
    	// cast strAsk to float and append to byte array
    	memcpy(&bytes[i], &fVal1, sizeof(float));
    	i += sizeof(float);
    
    	// cast strBid to float and append to byte array
    	memcpy(&bytes[i], &fVal2, sizeof(float));
    	i += sizeof(float);
    
            // i = 14 ?
    }
    If I add another variable to the byte array it will be at location 14 when the byte array size is 16. This does not make sense to me.

  8. #8
    Join Date
    Feb 2002
    Posts
    4,640

    Re: parse a string to bytearray to struct

    Read Paul's original response:
    What made you think that there isn't any padding bytes between those struct members?
    The sizeof a struct isn't necessarily the sum of the sizeof it's parts.

    Viggy

  9. #9
    Join Date
    Jan 2009
    Posts
    1,689

    Re: parse a string to bytearray to struct

    Quote Originally Posted by cbpetro View Post
    ok thanks. This works for my microsoft 32-bit compiler.

    Code:
    #pragma pack(push)  // push current alignment to stack
    #pragma pack(1)     // set alignment to 1 byte boundary
    struct TD
    {
    	short sYear;
    	short sMonth;
    	short sDay;
    
            float fVal1;
    	float fVal2;
    };
    #pragma pack(pop)   // restore original alignment from stack
    GCC way
    Code:
    #pragma pack(push, 1)
    struct TD {
    ...
    };
    #pragma(pop)

  10. #10
    Join Date
    Aug 2000
    Location
    Virginia, US
    Posts
    158

    Re: parse a string to bytearray to struct

    Quote Originally Posted by MrViggy View Post
    Read Paul's original response:

    The sizeof a struct isn't necessarily the sum of the sizeof it's parts.

    Viggy
    But we've determined that the size of struct = size of byte array. Therefore the bytearray is padded as well, correct?

    Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.

  11. #11
    Join Date
    Feb 2002
    Posts
    4,640

    Re: parse a string to bytearray to struct

    No. An array is a continuous chunk of memory, with each item in the array having the same size. It's not a grouping of different things. A struct is a grouping of (possibly differently sized) things.

    Viggy

  12. #12
    Join Date
    Apr 1999
    Posts
    27,427

    Re: parse a string to bytearray to struct

    Quote Originally Posted by cbpetro View Post
    Again, I ask do I have to consider the padding each time I add a new set of bytes to my byte array.
    You have to keep your struct packed on the byte boundary to not introduce any gaps in-between the members.

    The issue you need to consider is if you or someone else changes the order of the members of your struct. Then you have to change the reading code to match the order of the items. Another thing to consider is if you add a member in the middle of the struct.

    Maybe you should consider "auto-generating" the struct and the code to read it, as right now, manually making changes could lead to hard-to-find bugs. Write a program that creates a header of the struct, and the C++ code to process it.

    Regards,

    Paul McKenzie

  13. #13
    Join Date
    Aug 2000
    Location
    New York, NY, USA
    Posts
    5,526

    Re: parse a string to bytearray to struct

    Quote Originally Posted by cbpetro View Post
    I'm trying to parse a string into individual variables defined by a struct into a byte array. I want to pass the byte array to a com server object and not the string because it's smaller size...
    If you really want the smaller size, note that both month and day fit in single byte.
    Coincidentally, since short and two bytes take exactly 4 bytes, you wouldn’t have a packing issue…
    Vlad - MS MVP [2007 - 2012] - www.FeinSoftware.com
    Convenience and productivity tools for Microsoft Visual Studio:
    FeinViewer - an integrated GDI objects viewer for Visual C++ Debugger, and more...

  14. #14
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,290

    Re: parse a string to bytearray to struct

    As mentioned, it is usually not possible to assume a struct will align correctly on any given byte array. That, or the amount of assumptions/restrictions you need to do are usually prohibitive.

    What you are trying to do is not unlike serialization. The easiest method (and usually recommend method I believe), is to just take your attributes 1 by 1, and write them to the array. To unserialize, read your blocks and write them to your attributes 1 by 1.

    It might sound like a hassle, but it is just writing 2 functions, and performance usually isn't so mind-bogglingly tight you can't afford it. Besides, you already wrote 1 of the two functions, why not just write the other?

    Code:
    #include <string>  //string
    
    #include <iostream>  //cout
    #include <iterator>  //ostream_iterator
    #include <algorithm> //string
    #include <iomanip>   //hex setw
    
    const size_t ARRAY_SIZE = sizeof(short)*3 + sizeof(float)*2;
    
    struct TD
    {
      short sYear;
      short sMonth;
      short sDay;
    
      float fVal1;
      float fVal2;
    };
    
    void WriteByteArray(const TD& td, unsigned char* bytes)
    {
      size_t i = 0;
    
      // cast strYear to short and append to byte array
      (short&) bytes[i] = td.sYear;
      i += sizeof(short);
    
      // cast strMonth to short and append to byte array
      (short&) bytes[i] = td.sMonth;
      i += sizeof(short);
    
      // cast strDay to short and append to byte array
      (short&) bytes[i] = td.sDay;
      i += sizeof(short);
    
      // cast strAsk to float and append to byte array
      (float&) bytes[i] = td.fVal1;
      i += sizeof(float);
    
      // cast strBid to float and append to byte array
      (float&) bytes[i] = td.fVal2;
      //i += sizeof(float);
    }
    
    void ReadByteArray(unsigned char const * bytes, TD& td)
    {
      size_t i = 0;
    
      // cast strYear to short and append to byte array
      td.sYear  = (short&) bytes[i];
      i += sizeof(short);
    
      // cast strMonth to short and append to byte array
      td.sMonth = (short&) bytes[i];
      i += sizeof(short);
    
      // cast strDay to short and append to byte array
      td.sDay   = (short&) bytes[i];
      i += sizeof(short);
    
      // cast strAsk to float and append to byte array
      td.fVal1 = (float&) bytes[i];
      i += sizeof(float);
    
      // cast strBid to float and append to byte array
      td.fVal2 = (float&) bytes[i];
      //i += sizeof(float);
    }
    
    int main(int argc, char* argv[])
    {
      TD td = {2011, 7, 15, 1.5, 1e10};
      unsigned char MyBytes[ARRAY_SIZE];
    
      std::cout << "Array size: " << ARRAY_SIZE << std::endl;
    
      std::cout << std::hex << std::setw(3);
    
      WriteByteArray(td, MyBytes);
      std::cout << "encoded data:" << std::endl;
      std::copy(MyBytes, MyBytes+ARRAY_SIZE, std::ostream_iterator<int>(std::cout, ", "));
      std::cout << std::endl;
    
      unsigned char MyBytes2[ARRAY_SIZE] =
      {
        0x9e, 0x07, // 1950
        0x0c, 0x00, // 12
        0x1e, 0x00, // 30
        0xf9, 0x02, 0x15, 0x50, //1e10
        0x00, 0x00, 0xc0, 0x3f  //1.5
      };
    
      ReadByteArray(MyBytes2, td);
      std::cout << std::dec << std::setw(0);
      std::cout << "decoded data:" << std::endl;
      std::cout << td.sYear << " " << td.sMonth << " " << td.sDay << " " << td.fVal1 << " " << td.fVal2 << std::endl;
    
      return 0;
    }
    Note that neither endianness nor the fact that "short" is not the same on all machines, were taken into account here. If endianness should be taken into account, then you have use this kind of per-object approach anyways.
    Last edited by monarch_dodra; July 15th, 2011 at 08:51 AM. Reason: typo
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center