problem regarding binary data processing

[aryan1]

Hi All,

I want to do some processing on some binary data extracted from an "istream" object using std::getline as follows:

string line;
FileStream dataFile;
... // form dataFile properly here
std::getline( dataFile.in, line );//read a line

In the above code snippet, dataFile is a binary stream.

Additionally, I want to carry out some further processing on "line" string to see if its FIRST two bytes are CRLF (carriage return, line feed).

For example, given the following binary data as the value of "line" string, I should be able to identify that the first two bytes are NULL characters instead of x0D x0A, and to extract the remaining data, that is, x31 x39 x30 x35 x0D, and store it in the same/another string:

00 00 31 39 30 35 0D

With "line" has the above binary data as its value, if I try to print its value using syslog() function, I simply get nothing ("") even though only the first two characters are NULL, not all the characters. On the other hand, I can observe that its length is 7 using line.length() member function.

By the way, I am using Ubuntu 9.10.

Any help is appreciated.

Thanks.

[Richard.J]

the string object handles chars, including 0x0.
If you pass the content of the string to syslog() using c_str(), then you get a pointer to an array whose first 2 bytes are NULL. This is interpreted as an empty C-string.

[Paul McKenzie]

the string object handles chars, including 0x0.

That's not true. A std::string can handle any character -- you have to call the correct functions to do binary processing.

The std::string knows its length via member variable(s), not by a terminating NULL, which is one reason why it can be used for binary data.

Regards,

Paul McKenzie

[Paul McKenzie]

Hi All,

I want to do some processing on some binary data extracted from an "istream" object using std::getline as follows:

If you're going to use std::string, you have to call the correct constuctors, functions, etc. to handle binary data.

1) You construct strings using the constructor that takes a pointer and an argument.

2) You concatenate a binary character sequence onto a std::string by using string::append(), not += or +. Those latter operators only work for binary data if the right-hand side is also a std::string. If it's a memory block, char array, etc. that contains a NULL character, then += and + will not work correctly.

These are just some of the things to watch out for when handling binary data and strings.

Regards,

Paul McKenzie

[Lindley]

Additionally, I want to carry out some further processing on "line" string to see if its FIRST two bytes are CRLF (carriage return, line feed).

That's going to be problematic for a few reasons.

First, the getline() function discards its delimiters (LF by default), so if you use getline() to read, then you will *never* end up with a LF in the result. The istream::get function will not discard delimiters (but doesn't read them either, it leaves them in the stream), but does not support std::strings directly; you could use a std::stringbuf.

Second, the CRLF pattern is unique to Windows files. Text files created on another system may lack the CR (or, in very rare cases, may lack the LF). The primary reason for preferring text mode over binary mode on Windows is so that you can disregard this discrepancy; CRLF is automatically reduced to LF when it is encountered.

[aryan1]

The istream::get function will not discard delimiters (but doesn't read them either, it leaves them in the stream), but does not support std::strings directly; you could use a std::stringbuf

What about using istream::read function ?

Is it capable of reading CRLF as is without discarding any of the delimiters as shown below ?

char delim[2];
dataFile.in.read(delim, 2);
//check delim array to see if it contains CRLF bytes

[Lindley]

Absolutely, read() is unformatted IO.

However, be careful----if you're reading 2 bytes at a time, CR may be the *second* byte, in which case LF would not show up until the next read.

I could make a better suggestion if I knew more about what you were actually doing. If CRLFs are really all you're looking for, there might be a better approach.

[dsp8]

Why not try fgetc?

[Lindley]

fgetc() is a stdio function. Usable, but generally not mixed with C++ streams.

The iostream equivalent is the version of istream::get() taking a char parameter.