[RESOLVED] 8 bit strings Encoding issue

This is a unit test to ensure 8 bit data (00-FF) can be placed in an string moved to a byte array and vice versa and nothing is lost in this conversion.

it seems that base64String does not seem to be able to hold the correct 8 bit data and as a result content of dest and fileblock1 wont match.


What improper encoding am i doing wrong?

Code:

---

            string filename;
            FileInfo ff;
            FileStream fss;
            BinaryReader br;

            filename = textBox1.Text;
            ff = new FileInfo(filename);
            byte[] FileBlock1 = new byte[ff.Length];
            fss = new FileStream(filename, FileMode.Open);
            br = new BinaryReader(fss);
            br.Read(FileBlock1, 0, (int)ff.Length);
            fss.Close();

// file block shows the correct data in the file

string base64String = null;
int position = -1;

System.Text.UTF8Encoding encoding = new System.Text.UTF8Encoding();
base64String = System.Convert.ToBase64String(FileBlock1, 0, FileBlock1.Length);
byte[] dest = new byte[FileBlock1.Length];
Array.Copy(encoding.GetBytes(base64String), dest, FileBlock1.Length);
                if (ByteArrayCompare(dest, FileBlock1, ref position))
                {
                    textBox2.Text += "matched\r\n";
                }
                else
                {
                    textBox2.Text += "DID not matched .position" + position.ToString() + "\r\n"; ;
                }

---

get rid of your UTF8 encoding. use Convert.FromBase64String to get your original bytes back

Code:

---

byte[] orig = new byte[256];
for ( int i = 0; i < 256; ++i ) {
    orig[i] = (byte)i;
}
string b64 = Convert.ToBase64String( orig );
byte[] sec = Convert.FromBase64String( b64 );
Debug.Assert( BytesEqual( orig, sec ), "Bytes are not equal" );

//...

private static bool BytesEqual( byte[] orig, byte[] sec ) {
    bool res = true;
    if ( orig == null || sec == null || orig.Length != sec.Length ) {
        res = false;
    }
    if ( res ) {
        for ( int i = 0; i < orig.Length; ++i ) {
            if ( orig[i] != sec[i] ) {
                res = false;
                break;
            }
        }
    }
    return res;
}

---

apologies
i did not scroll down enough to see the code ;)

Thank You

You can't convert an arbitrary file to an ASCII, UTF8, UTF16 or UTF32 string. You *must* use a base64 string. If you take a random executable and convert it to any kind of string which is not a base64 string, you'll end up with a different byte sequence when you convert the string back to an array of bytes. This is because there are certain sequences which are invalid and will be silently discarded by the byte -> string converter.

To show this error more clearly, do something like this:

Code:

---

byte[] bytes = File.ReadAllBytes ("myfile.exe");

// This will create an encoder which throws an exception if the bytes aren't convertable
UTF8Encoder enc = new UTF8Encoder (true, true);

// This will more than likely throw an exception to let you know the bytes
// could not be converted to a string as you're trying to convert a byte
// sequence which is *not* an actual string
string s = enc.GetString (bytes);

---