Variable struct size over socket.

[StrangeWill]

So I'm having a bit of an issue here, I don't know if it's more of a design issue or implementation issue....

So I have two machines that communicate to each other over sockets, C# is giving me some issues, but that can wait till later....

What I need to be able to do is send arrays of data back and forth between the two machines, basically in a struct like this:

struct someData
{
int id;
string command;
string *params;
};

Params must be a pointer so it can be dynamically allocated.

So when I cast someData into a byte array, I'm actually just sending a pointer, thats not what I need.

I could build my own routine to go into the pointer, break down the byte information and tack it on, but I can't think of a way to keep track of that kind of data, especially because I can be dealing with lots of it.

Are they any suggestions on how to transfer flexible data like this over a network? It's a hard tossup for me if this is a network question or a bad design choice question. I need to transfer this data, but maybe there is an alternate way to package it I'm not seeing.


I'm using Windows/C# and Linux/C++, this is the C++ element and I need to keep in mind that I'll have to deal with getting the data into a C# environment without too much struggle and vice versa. Also I'll eventually be adding byte implementation being as I'll be loading anything from textual data to binary data, so cramming everything together and separating them again with nulls is not going to work for me. :(


A second implementation I can try is kind of a handshake idea, again slipping back into the network design over data handling, I could do this weird handshake type thing for each piece of data, and fork every stream so that multiple streams can be run, but that seems extremely inefficient.


This data transfer method mainly comes from a project I did a few months ago at school, it was efficient but the data sized was fixed and didn't allow for anything over 256 bytes (which was fine for the project), and I want to fix it and use it for a work project. I haven't been writing C/C++ long so I may be missing a bit.

[Paul McKenzie]

So I'm having a bit of an issue here, I don't know if it's more of a design issue or implementation issue....The solution to your problem is the same solution you would have if you were saving the information to a file, and then in another program, attempt to read the data you saved from the file.

You brought up networks, but this problem is much more general than network issues. You have some structure, and you must have some sort of scheme to save and another scheme to read successfully what was saved. The act of saving and retrieving data is called serialization.
What I need to be able to do is send arrays of data back and forth between the two machines,The two "machines" is not relevant, as discussed above. It is a matter of saving and retrieving data from one source to another -- it could be a simple file save and read.
basically in a struct like this:

struct someData
{
int id;
string command;
string *params;
};

You cannot send/save a std::string as-is, as it is a non-POD type.
Params must be a pointer so it can be dynamically allocated.No it doesn't. It could be a std::vector<std::string>.

But regardless, somehow, you have to save information that describes the string and the number of strings in the second parameter, and then given this description, read the requisite number of strings.

Something like this:
43 9 Joe Smith 3 4 John 3 Joe 5 Simon
The first number is the id, the second number is the number of characters in the command (9), followed by the command. The 3 is the number of strings in params. The params are paired, with the first number the number of characters, and the second the string itself.

This is only one way, but in general this is no different than the same issue of saving and reading information to a file.

Regards,

Paul McKenzie

[MikeAThon]

Paul's observation that this is a more general concept is right on target. It's related to serialization. Start here: http://www.parashift.com/c++-faq-lite/serialization.html

Mike