List<byte> -> string -> List<byte>
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 28

Thread: List<byte> -> string -> List<byte>

  1. #1
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    List<byte> -> string -> List<byte>

    Does anyone know if there is a way to conver a List<byte> into a string, and then convert that string back into a List<byte> later?

    It doesn't look like List<byte> supports GetBytes. I was thinking about putting all the elements into a string that is comma delimited and then reading them back again. But I'm not sure of a fast way to read a comma delimited string. It seems like that operation would take too long.

    I need to transmit a small list with a socket. I want to use a string to I don't have to deal with a lot of the advance socket programming of sending objects. The comma delimited solution will work but I'm just wondering if there is a better/faster way.

    The list will not be very big at the maximum it will have 74 elements inside of it. None of the elements will be larger then 2 bytes so I figured if I converted 1-9 with a leading 0 then everything in that list would be 2 bytes and it would be a lot faster to read.

    If I did it that way I may not even need a comma delimited string. I would just read a string in 2 bytes at a time, each read will be an element for the new List<byte> on the other machine. Maybe by using string.Substring().

    But I always though that string operations like that were very slow. I need something that is going to be fast enough to keep up with my game.

    Thanks.
    Microsoft Framework Programmer
    http://kensino.com

  2. #2
    Join Date
    Oct 2007
    Posts
    3

    Re: List<byte> -> string -> List<byte>

    List<byte> lstBytes;
    ...
    ...

    lstBytes.ToArray(); // will give you a byte array

    The class below should help.. You can call the ToArray method of your list or you can modify the class below to accept a List<byte>.

    Code:
    public class BinUtil
        {
            private static readonly string[] HexDigits = 
    		{ 
                "00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "0A", "0B", "0C", "0D", "0E", "0F", 
                "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", "1A", "1B", "1C", "1D", "1E", "1F", 
                "20", "21", "22", "23", "24", "25", "26", "27", "28", "29", "2A", "2B", "2C", "2D", "2E", "2F", 
                "30", "31", "32", "33", "34", "35", "36", "37", "38", "39", "3A", "3B", "3C", "3D", "3E", "3F", 
                "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "4A", "4B", "4C", "4D", "4E", "4F", 
                "50", "51", "52", "53", "54", "55", "56", "57", "58", "59", "5A", "5B", "5C", "5D", "5E", "5F", 
                "60", "61", "62", "63", "64", "65", "66", "67", "68", "69", "6A", "6B", "6C", "6D", "6E", "6F", 
                "70", "71", "72", "73", "74", "75", "76", "77", "78", "79", "7A", "7B", "7C", "7D", "7E", "7F", 
                "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "8A", "8B", "8C", "8D", "8E", "8F", 
                "90", "91", "92", "93", "94", "95", "96", "97", "98", "99", "9A", "9B", "9C", "9D", "9E", "9F", 
                "A0", "A1", "A2", "A3", "A4", "A5", "A6", "A7", "A8", "A9", "AA", "AB", "AC", "AD", "AE", "AF", 
                "B0", "B1", "B2", "B3", "B4", "B5", "B6", "B7", "B8", "B9", "BA", "BB", "BC", "BD", "BE", "BF", 
                "C0", "C1", "C2", "C3", "C4", "C5", "C6", "C7", "C8", "C9", "CA", "CB", "CC", "CD", "CE", "CF", 
                "D0", "D1", "D2", "D3", "D4", "D5", "D6", "D7", "D8", "D9", "DA", "DB", "DC", "DD", "DE", "DF", 
                "E0", "E1", "E2", "E3", "E4", "E5", "E6", "E7", "E8", "E9", "EA", "EB", "EC", "ED", "EE", "EF", 
                "F0", "F1", "F2", "F3", "F4", "F5", "F6", "F7", "F8", "F9", "FA", "FB", "FC", "FD", "FE", "FF" 
            };
    
            /// <summary>
            /// Determines if specified string is a hexadecimal string.
            /// </summary>
            /// <param name="hexString">string to check</param>
            /// <returns>Boolean indicating whether or not the specified string is a hexadecimal string</returns>
            /// <remarks>"00FF0A12" is an example of a hexadecimal string</remarks>
            public static bool IsHexString(string hexString)
            {
                if (hexString == null || ((hexString.Length % 2) != 0))
                {
                    return false;
                }
    
                foreach (char c in hexString)
                {
                    if (!Uri.IsHexDigit(c))
                    {
                        return false;
                    }
                }
    
                return true;
            }
    
            /// <summary>
            /// Converts a byte array to a hexadecimal string representation.
            /// </summary>
            /// <param name="bytes">Array to be converted</param>
            /// <returns>Hexadecimal string</returns>
            public static string ToHexString(byte[] bytes)
            {
                StringBuilder retVal = new StringBuilder(bytes.Length * 2);
                for (int i = 0; i < bytes.Length; i++)
                {
                    retVal.Append(HexDigits[bytes[i]]);
                }
    
                return retVal.ToString();
            }
    
            /// <summary>
            /// Converts a byte array to a hexadecimal string representation with specified separator after every byte.
            /// </summary>
            /// <param name="bytes">Array to be converted</param>
            /// <param name="separator">Separator to add after each byte</param>
            /// <returns>Hexadecimal string</returns>
            public static string ToHexString(byte[] bytes, string separator)    
            {
                StringBuilder retVal = new StringBuilder((bytes.Length * 2) + (separator.Length * bytes.Length));
                for (int i = 0; i < bytes.Length; i++)
                {
                    retVal.Append(HexDigits[bytes[i]]);
                    retVal.Append(separator);
                }
    
                return retVal.ToString();
            }
    
            /// <summary>
            /// Converts a section of a byte array to a hexadecimal string representation.
            /// </summary>
            /// <param name="bytes">Array to be converted</param>
            /// <param name="offset">The location in bytes array from which to start converting data</param>
            /// <param name="length">The number of bytes to convert</param>
            /// <returns>Hexadecimal string</returns>
            public static string ToHexString(byte[] bytes, int offset, int length)
            {
                StringBuilder retVal = new StringBuilder(length * 2);
    
                for (int i = 0; i < length; i++)
                {
                    retVal.Append(HexDigits[bytes[offset + i]]);
                }
    
                return retVal.ToString();
            }
    
            /// <summary>
            /// Converts a section of a byte array to a hexadecimal string representation with specified separator after every byte.
            /// </summary>
            /// <param name="bytes">Array to be converted</param>
            /// <param name="offset">The location in bytes array from which to start converting data</param>
            /// <param name="length">The number of bytes to convert</param>
            /// <param name="separator"></param>
            /// <returns>Hexadecimal string</returns>
            public static string ToHexString(byte[] bytes, int offset, int length, string separator)
            {
                StringBuilder retVal = new StringBuilder((bytes.Length * 2) + (separator.Length * bytes.Length));
    
                for (int i = 0; i < length; i++)
                {
                    retVal.Append(HexDigits[bytes[offset + i]]);
                    retVal.Append(separator);
                }
    
                return retVal.ToString();
            }
    
            /// <summary>
            /// Convert a hexadecimal string to a byte array.
            /// </summary>
            /// <param name="hexString">Hexadecimal string to convert</param>
            /// <returns>byte array</returns>
            public static byte[] ToByteArray(string hexString)
            {
                byte[] retVal = new byte[0];
    
                if (!IsHexString(hexString))
                {
                    throw new ArgumentException("Error: Illegal hex string. " + hexString);
                }
    
                retVal = new byte[hexString.Length / 2];
    
                for (int i = 0; i < retVal.Length; i++)
                {
                    retVal[i] = Convert.ToByte(hexString.Substring(i*2, 2), 16);
                }
    
                return retVal;   
            }
        }
    Last edited by cjard; October 20th, 2007 at 10:20 AM.

  3. #3
    Join Date
    Sep 2004
    Posts
    1,361

    Re: List<byte> -> string -> List<byte>

    Or you can use Convert.ToBase64String (Byte[]) and the FromBase64String methods.

    If it is binary data, Id just say leave it that way if you can handle it. You can just make the first 2 bytes sent the length of the send. Then whomever is receiving the data just reads the first two bytes to see how much is left to read and then reads the rest of the data. No padding or conversions needed.

  4. #4
    Join Date
    May 2007
    Posts
    1,546

    Re: List<byte> -> string -> List<byte>

    BitConverter is *definately* the way to go as opposed to using the code above. However, sockets prefer byte[] when sending data, so why not just send the array itself?
    www.monotorrent.com For all your .NET bittorrent needs

    NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.

  5. #5
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    Quote Originally Posted by Mutant_Fruit
    BitConverter is *definately* the way to go as opposed to using the code above. However, sockets prefer byte[] when sending data, so why not just send the array itself?
    I don't know how to send the List<byte> array itself. I tried sending objects before and I posted a thread on this forum asking for help and no one seems to know how to do it correctly, so I gave up on seding arrays or other objects over the network.

    No one on this forum knows how to do it. So I have to use strings instead.

    http://www.codeguru.com/forum/showthread.php?t=436127
    Microsoft Framework Programmer
    http://kensino.com

  6. #6
    Arjay's Avatar
    Arjay is online now Moderator / MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    11,359

    Re: List<byte> -> string -> List<byte>

    Is the receiving app on the other side of the socket a .net or Java app?

  7. #7
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    Quote Originally Posted by Arjay
    Is the receiving app on the other side of the socket a .net or Java app?
    It's a .Net windows service.

    I have tried to modify my code again to send the size data along with the socket because it seems some Send methods are stepping on other Send methods and they are getting commands that are concat when they should be two seperate commands. I made some changes but now I just get error messages.

    Server:
    Code:
    		// TRANSMIT A MESSAGE TO ALL CONNECTED CLIENTS
    		public void TransmitToAllClients(string msgToTransmit, NetCommandType commandType) {
     
    			byte[] bteNetworkStringBytes = Encoding.ASCII.GetBytes(NetHelper.WriteNetworkString(commandType, msgToTransmit));
     
    			Socket workerSocket = null;
    			try {
    				for(int i = 0; i < m_workerSockets.Count; i++) {
    					workerSocket = (Socket)m_workerSockets[i];
    					if((workerSocket != null) && (workerSocket.Connected)) {
     
    #if PRNT_DBG_MSGS
    						BetaLogger.Logger.LogEvent(string.Format("TRANSMIT TO ALL CLIENTS: {0}", NetHelper.WriteNetworkString(commandType, msgToTransmit)));
    #endif
     
    						//// ==========================================================================
    						workerSocket.Send(bteNetworkStringBytes);
    						//// ==========================================================================
    					}
    				}
    			} catch(SocketException) {
    				// TODO: IMPLEMENTATION, IF WE ENCOUNTER AN EXCEPTION WE STILL NEED TO KEEP
    				// SENDING THE DATA TO ALL THE REMAINING SOCKETS.
    			}
    		}
    Client
    Code:
    		public void OnDataReceived(IAsyncResult asyn) {
     
    #if PRNT_DBG_MSGS
    			BetaLogger.Logger.LogEvent("CLIENT PORT MANAGER: ENTERED ON_DATA_RECEIVED METHOD");
    #endif
     
    			DataPacket dataPack = (DataPacket)asyn.AsyncState;
     
    			try {
     
    				// END THE SOCKET READ AND STORE THE NUMBER OF CHARACTERS READ
     
    				int charsRead = dataPack.Sock.EndReceive(asyn);
     
    				// CREATE AN ARRAY LARGE ENOUGH TO HOLD ALL THE DATA READ FROM THE SOCKET
    				char[] chars = new char[charsRead + 1];
     
    				// DECODE THE DATA, WRITE RESULTS INTO chars
    				Decoder deco = Encoding.ASCII.GetDecoder();
    				deco.GetChars(dataPack.DataBuffer, 0, charsRead, chars, 0);
     
    				// GET THE NETWORK STRING THAT WAS SENT OVER THE WIRE
    				string recString = new string(chars);
     
    				// DISINTEGRATE THE NETWORK STRING
    				NetCommandType cmdType;
    				string cmdData; // TODO :: I THINK THE OUT PARAMS ARE STEPPING ON THE TOES OF OTHER THREADS/BEING OVER WRITEN BY OTHER THREADS
     
    #if PRNT_DBG_MSGS
     
    				BetaLogger.Logger.LogEvent("CLIENT PORT MANAGER: ATTEMPTING TO READ NETWORK STRING");
    				BetaLogger.Logger.LogEvent(string.Format("COPY OF DATA RECEIVED: {0}", recString));
    #endif
     
    				// IF WE HAVE RECEIVED A VALID NETWORK STRING, FIRE IT.
    				if(NetHelper.ReadNetworkString(recString, out cmdType, out cmdData)) {
     
    #if PRNT_DBG_MSGS
    					BetaLogger.Logger.LogEvent("CLIENT PORT MANAGER: NETWORK STRING HAS BEEN READ - ATTEMPTING EXECUTION");
    #endif
     
    					ExecuteNetworkCommand(cmdType, cmdData);
     
    #if PRNT_DBG_MSGS
    					BetaLogger.Logger.LogEvent("CLIENT PORT MANAGER: ATTEMPTED EXECUTION OF NETWORK STRING HAS BEEN PASSED");
    #endif
    				}
     
    				// WAIT FOR MORE DATA ON THE SOCKET
    				WaitForData();
     
    			} catch(ObjectDisposedException) {
    				// TODO: SERVER CLOSED CONNECTED ?? 
    			} catch(SocketException) {
    				// TODO: IMPLEMENTATION
    			}
    		}
    I changed the red lines with the following but I just get errors. According to MSDN this is what that param is for.

    Server
    Code:
    workerSocket.Send(bteNetworkStringBytes, bteNetworkStringBytes.Length, SocketFlags.None);
    Client
    Code:
    dataPack.Sock.EndReceive(asyn);
    int charsRead = dataPack.Sock.Receive(dataPack.DataBuffer, dataPack.Sock.Available, SocketFlags.None);
    Guess I don't have something right again.
    Microsoft Framework Programmer
    http://kensino.com

  8. #8
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    This is actually two different commands to my client, but for some reason they randomly get concat, it doesn't happen very often, but it shouldn't happen at all.

    000401000507 (this should really be two commands)

    command 1: 000401 (which means they are connected to the server, and their client id = 1)

    command 2: 000507 (which means load message 07 from the resource file and display it)

    The first 4 bytes are always the command, and the rest is the command data. They are both converted to enums. Its a lot better then sending the actual data or a very long string.

    They are not concat when the leave the server, only when they arrive at the client, my BetaLogger will prove that. Do you think this is caused from the OUT params I am using in the OnDataReceived method? Are these not thread safe?

    -----
    edited
    -----

    I have confirm that this appears to be because the messages are being sent to the client too fast. This has to be what is happening. I have been testing for hours, and I cannot see any other reason for this to happen.

    So I thought about making a queue system to only send 1 message per second to each client but that will not work. I would need to do the same on the server side because the server will have an extrem amount of client connections. I can't really to this queue in reverse.

    It looks like my best option is to stick with the size option. I need to send the size with the string, I can send it as the first four bytes, this is not a problem for me. But I cannot figure out how to get it on the other side. I need to get the size from the first 4 bytes of the string, read that command and then all remaining bytes are pushed the next command.

    Its possible I know, but I cannot figure out how to do it, because on my OnReceived method it reads the data in, and doesn't really allow me a place to grab size data before reading anything. I can't get the size data until I've read the string.

    My string could be like this

    001200010009

    first 4: 12 bytes to read, 2nd 4: command, 3rd 4: command data

    My string may not always be that simple however, command data may not always be 4 bytes, the rest will always be four bytes. I will never need to send a string larger then 9999 bytes. I will never even get close to that. All my strings are actually stored in the client, I just send the ID of the string that the client needs to load.

    I will look into it in the mean time. If I can get every single message to be the exact same size coming over the network kind of like a HASH, then I will always know how many bytes to read. Problem is I cannot reverse a hash so I have no way to know what the original message was. I think I can try to get something working,

    problem is sending that array byte bytes, I still have no idea how to do that, and I cannot gurantee how many bytes will be in that array. That will toss my fixed command length right out the door.

    ------

    Well this is all I can come up with, bitConverter does not with with a List<byte> for some stupid reason. This works but it is very slow. I cannot use it.

    Code:
    byte[] bteList = newbyte[100];
    
    for(byte i = 0; i < 100; i++) {
    
    bteList[i] = i;
    
    }
    
    for(byte i = 0; i < bteList.Length; i++) {
    
    listBox1.Items.Add(bteList[i]);
    
    }
    
    string bytes = Convert.ToBase64String(bteList);
    
    textBox1.Text = bytes;
    
    byte[] bteList2 = Convert.FromBase64String(bytes);
    
    for(byte i = 0; i < bteList2.Length; i++) {
    
    listBox2.Items.Add(bteList2[i]);
    
    }
    

    Last edited by Kensino; October 20th, 2007 at 12:55 AM.
    Microsoft Framework Programmer
    http://kensino.com

  9. #9
    Arjay's Avatar
    Arjay is online now Moderator / MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    11,359

    Re: List<byte> -> string -> List<byte>

    Since it's a web service, why not connect to it using the class generated when adding a web service reference? It'll handle the serialization/deserialization for you.

  10. #10
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    i dont know what your talking about. what is a web service? i am building a windows game.

    regardless it seems impossible. even when i send a fixed string of 8 bytes its impossible to work correctly because they are async methods and it seems one thread always steps on another thread and this causing the issue. There is nothing I can do to prevent a thread from walking on another thread.

    I thought there was a LOCK command or something....?????? I don't know but I'm tired and I'm going to bed. Do you know of a lock command? Does that prevent a thread from using a method until its unlocked, and then that thread can use it? kind of like queuing the threads?

    thanks
    Microsoft Framework Programmer
    http://kensino.com

  11. #11
    Join Date
    May 2007
    Posts
    1,546

    Re: List<byte> -> string -> List<byte>

    BitConverter works on byte[]. This is typically what a socket will give you as it's received data. If not, just call myList.ToArray(), which will give you a byte[] and then use bitconverter on that.

    As for the messages appearing on each other, what you need to do is have a way of knowing the length of the data you are supposed to receive. The typical way to do this is to first send an int which represents the amount of data you're about to send, then send the data.

    On the receiving side you read out 4 bytes, convert it to an int (bitconverter can do this for you), then you read that many bytes from your socket/network stream. Then read 4 bytes again etc etc.

    Finally, you need to be careful that you send your messages sequentially. Its up to you to make sure that one of your threads in your client doesn't send a message while a second thread is in the middle of sending one. Either a) read previous posts in the forum or b) google thread safety and the 'lock' statement or 3) Read tutorials on the web which show you how to do this kind of basic problem
    www.monotorrent.com For all your .NET bittorrent needs

    NOTE: My code snippets are just snippets. They demonstrate an idea which can be adapted by you to solve your problem. They are not 100% complete and fully functional solutions equipped with error handling.

  12. #12
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    Thanks Mutant_Fruit. I am not going to give up even as I've been stuck on this for a week. I know it has to be possible. I think you are right. I think I know what happening now because I have a few methods in my server one that sends to all clients and one that sends to a single client. It appears that these are stepping on each other. If I can move them into one single method, then control it with the lock statement I think it will be less likely for them to do that, then I will try to send the size before my transmission.

    I am going to get this working no matter how long it takes. I'll be working on it today. I'll post back when I find a solution, or if I run into any more problems. I think I kinda understand what is going on now.

    Thanks.
    Microsoft Framework Programmer
    http://kensino.com

  13. #13
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    Its not working. I modified my code to receive a string with the first 4 bytes are size data.

    Code:
    		public void WaitForData() {
    			try {
    
    				DataPacket dataPack = new DataPacket();
    				dataPack.Sock = m_ClientSocket;
    
    				int charsRead = m_ClientSocket.Receive(dataPack.DataBuffer, 4, SocketFlags.None);
    				char[] chars = new char[charsRead + 1];
    				Decoder deco = Encoding.ASCII.GetDecoder();
    				deco.GetChars(dataPack.DataBuffer, 0, charsRead, chars, 0);
    				string sizeBytes = new string(chars);
    				int sizeOfNetworkString;
    
    				if(int.TryParse(sizeBytes, out sizeOfNetworkString)) {
    					if(m_SocketCallback == null)
    						m_SocketCallback = new AsyncCallback(OnDataReceived);
    
    					m_asyncResult = m_ClientSocket.BeginReceive(dataPack.DataBuffer, (charsRead + 1) , sizeOfNetworkString, SocketFlags.None, m_SocketCallback, dataPack);
    				}
    
    			} catch(SocketException) {
    				// TODO: IMPLEMENTATION
    			}
    		}
    Now I get an error as soon as I enter the OnDataReceived method

    The IAsyncResult object was not returned from the corresponding asynchronous method on this class.
    Parameter name: asyncResult
    Cannot someone just please show me how to do it, so I understand and I can move on. I have been trying for a long time. I posted my original thread almost a week ago. I don't see why its so hard to do.

    I'm begnining to think .Net's network support is not very good.
    Microsoft Framework Programmer
    http://kensino.com

  14. #14
    Arjay's Avatar
    Arjay is online now Moderator / MS MVP Power Poster
    Join Date
    Aug 2004
    Posts
    11,359

    Re: List<byte> -> string -> List<byte>

    Quote Originally Posted by Kensino
    i dont know what your talking about. what is a web service? i am building a windows game.

    regardless it seems impossible. even when i send a fixed string of 8 bytes its impossible to work correctly because they are async methods and it seems one thread always steps on another thread and this causing the issue. There is nothing I can do to prevent a thread from walking on another thread.

    I thought there was a LOCK command or something....?????? I don't know but I'm tired and I'm going to bed. Do you know of a lock command? Does that prevent a thread from using a method until its unlocked, and then that thread can use it? kind of like queuing the threads?

    thanks
    Sorry, I misread '.net windows service' as '.net web service'.
    There are lock commands in .net. If you need multiple readers, check out the ReaderWriterLock class. If you would like a RAII approach to locking, check out the IDisposable and Using block thread.

  15. #15
    Join Date
    Oct 2007
    Location
    Alaska
    Posts
    62

    Re: List<byte> -> string -> List<byte>

    Well I'll try it again. Seems like this is taking forever and I'm not getting anywhere. I cannot understand why there are no good articles/tutorials on this. I tell you once I get this working I will be doing a in depth article on the subject for my website. All the help I have found on the Internet on this topic has been pretty much useless. I found a lot of articles but not many show a complete example, and none of them show you how to send objects or show you how to send size bytes. It sounds simple but its not.
    Microsoft Framework Programmer
    http://kensino.com

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center