Re: Read binary file with line delimeter
Thanks one more time 2kaud.
I've tested, compiles without errors, but is not printing the content of the blocks, I only receive this.
Code:
V[0]=vEJXXYW1_D1211308071344S dE‰4Uÿw
This is the interpreted content before the first delimiter "ff77". I'm not sure what is happens. Maybe is not reading literally the bytes and because of that is not finding the FF77.
When I see in an hex editor, the first 64 bytes are the following:
Code:
76454a58585957315f44313231313330
38303731333434065320644501893455
ff7700000153206445018934550f8147
4549232fffff0015000a4800015a0002
Re: Read binary file with line delimeter
Can you zip the first few thousand bytes of the file and post so I can take a look at it. I tried it on a test text file here I created that was over 1000 bytes and it showed the expected output.
1 Attachment(s)
Re: Read binary file with line delimeter
The attached file has 5 delimiters "FF77", so, 5 blocks.
I want to have stored in block variable the bytes literally, without any convertion to ascii.
The first 1024 bytes are below, showing the content of the first block and partially the content of 2nd block.
Code:
76454a58585957315f4431323131333038303731333434065320644501893455
ff7700000153206445018934550f81474549232fffff0015000a4800015a0002
4200016000013300013600013700015b00017e00016900006a00007900009300
012200002100010900010a000126000102000104000105000106000110000108
00012b00002c00012d00012e00015500015600072a00002f0000300000310000
ff7900800932c90688888000a000800935c90600008000000080093cc9068888
80008000800943c9068888800080000582003706010000010065000000020000
0200180000000300000300170000000400000400010000000500000500150000
000a00ffff006500000007802ec918059181475269531fffffff009181475269
531fffff000103ca030808fecb0a00000000000000000000cc0101811bc90b00
9181475269567fffffffca06000000000000cb0103cc0101ff77000002532064
45018934551f81474554768fffff0015000a4800015a00024200016000013300
013600013700015b00016600016500017700017800017e00016900006a000079
00009300012200002100010900010a0001260001020001040001050001060001
1000010800012b00002c00012d00012e00015500015600072a00002f00003000
00310000ff7900800932c90688888000a000800935c90600008000000080093c
PS:I've put txt extension to be able to upload the file.
Thanks for the help.
Re: Read binary file with line delimeter
The delimeter is NOT "FF77". The delimeter is 0xFF77 - which is a different ball game! Also the data contains 0x00 which is usually used to indicate the end of a string. Basically the functions work with ASCII text not binary data. That's why they work fine with the test data but not with your actual binary file.
I'll have a look at the functions over the next few days.
Re: Read binary file with line delimeter
Try this.
Code:
#include <iostream>
#include <fstream>
#include <vector>
#include <iomanip>
using namespace std;
typedef unsigned char BYTE;
typedef unsigned short int WORD;
typedef vector<BYTE> bVec;
#ifndef LOBYTE
#define LOBYTE(w) ((BYTE)((WORD)(w) & 0xff))
#endif
#ifndef HIBYTE
#define HIBYTE(w) ((BYTE)((WORD)(w) >> 8))
#endif
class FileFields
{
private:
ifstream ifs;
bool opened;
public:
FileFields() : opened(false) {}
~FileFields() {
if (opened)
ifs.close();
}
bool open(const char* name);
bool getField(bVec& field, WORD delim = 0xFF77);
};
bool FileFields::open(const char* name) {
ifs.open(name, ios::binary);
return (opened = ifs.is_open());
}
bool FileFields::getField(bVec& field, WORD delim)
{
char by;
bool cont = true;
field.clear();
if (!opened || !ifs.good())
return false;
for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
if (by == HIBYTE(delim))
if (ifs.peek() == LOBYTE(delim))
cont = false;
if (cont)
field.push_back(by);
}
return true;
}
void display(const bVec bv)
{
for (int i = 0; i < bv.size(); i++)
cout << setw(2) << setfill('0') << hex << (int)bv[i];
cout << endl << endl;
}
int main()
{
FileFields ff;
bVec bv;
if (!ff.open("binary.txt")) {
cout << "Cannot open file!" << endl;
return 1;
}
while (ff.getField(bv))
display(bv);
return 0;
}
Re: Read binary file with line delimeter
Hello 2kaud again,
Thanks for this help really. I'm testing and the code prints content of the binary correctly.
Now, I'm trying to print similarly each block, like block[0], block[1] etc like before, but I tried in dispaly function,
but I see now that function handles one byte at a time.
How can I print each block separately?
Thanks for your help, I'm trying to understand the way how you do it.
Re: Read binary file with line delimeter
Display prints one block separated by line feeds. In the main program, the vector bv holds one block (data between 2 delimeters) from the file. So you can use the contents of the vector bv to process one block at a time. The class function getField returns a vector containing the contents of the next block each time it is called.
If you print to print a block heading before each block, try
Code:
int main()
{
FileFields ff;
bVec bv;
int blk = 0;
if (!ff.open("binary.txt")) {
cout << "Cannot open file!" << endl;
return 1;
}
while (ff.getField(bv)) {
cout << "block " << blk++ << endl;
display(bv);
}
return 0;
}
Re: Read binary file with line delimeter
Hello 2kaud,
I'm not sure what happens, but trying in that way I receive "block 0" and in the next file the content of complete binary file. So, it seems like all content of the file is contained in block 0.
Code:
block 0
76454a58585957315f4431323131333038303731333434065320644501893455ff7700000......
I don't see clearly yet how to print something like this:
Code:
block 0
76454a58585957315f4431323131333038303731333434065320644501893455
block 1
000001...
block 2
000002...
block 3
000003...
.
.
block N
00000N...
Thanks again for the help.
Re: Read binary file with line delimeter
From the partial data file you attached in post #33, the output I get from this program is
Code:
block 0
76454a58585957315f4431323131333038303731333434065320644501893455
block 1
00000153206445018934550f81474549232fffff0015000a4800015a0002420001
00013700015b00017e00016900006a00007900009300012200002100010900010a
010400010500010600011000010800012b00002c00012d00012e00015500015600
300000310000ff7900800932c90688888000a000800935c9060000800000008009
8000800943c9068888800080000582003706010000010065000000020000020018
00170000000400000400010000000500000500150000000a00ffff006500000007
475269531fffffff009181475269531fffff000103ca030808fecb0a0000000000
01811bc90b009181475269567fffffffca06000000000000cb0103cc0101
block 2
00000253206445018934551f81474554768fffff0015000a4800015a0002420001
00013700015b00016600016500017700017800017e00016900006a000079000093
010900010a00012600010200010400010500010600011000010800012b00002c00
5500015600072a00002f0000300000310000ff7900800932c90688888000a00080
00000080093cc906888880008000800943c90688888000800005900f0102000000
ffff00910f01020000013a81475269559fffff009310010c0000009f8147526905
010e000000eb81475269596fffff00970f01010006f69981475269563fffff0094
0100ffff0000010195060003790001ea0582003706010000010065000000020000
00000300170000000400000400010000000500000500150000000a00ffff006500
009181475269539fffffff009181475269539fffff000103ca030808fecb0a0000
00cc0101811bc90b009181475269567fffffffca06000000000000cb0103cc0101
What compiler are you using? I use MSVC.
Try replacing
Code:
if (by == HIBYTE(delim))
if (ifs.peek() == LOBYTE(delim))
with
Code:
if ((BYTE)by == 0xff)
if ((BYTE)ifs.peek() == 0x77)
Is your type char default signed or unsigned? I have default unsigned. If yours is signed, that may be the problem.
UPDATE That is probably the problem. I changed my default to signed and got the same issue you have. Putting (BYTE) in the 2 if statements makes it work if default char is signed.
Re: Read binary file with line delimeter
Hello 2kaud,
Thanks!!!
I'm using MinGW and Dev C++(TDM GCC) compilers, both with the same issue, but you're rigth, replacing the 2 if with (BYTE) makes it work! why?
In main function I had to add "\n" in order to print in different row as below, I'm not sure if you needed to do that to get the output you posted.
Code:
cout << "\nblock " << blk++ << endl;
Code:
block 0
76454a58585957315f4431323131333038303731333434065320644501893455
block 1
00000153206445018934550f81474549232fffff0015000a4800015a00024200
016000013300013600013700015b00017e00016900006a000079000093000122
00002100010900010a0001260001020001040001050001060001100001080001
2b00002c00012d00012e00015500015600072a00002f0000300000310000ff79
00800932c90688888000a000800935c90600008000000080093cc90688888000
8000800943c90688888000800005820037060100000100650000000200000200
180000000300000300170000000400000400010000000500000500150000000a
00ffff006500000007802ec918059181475269531fffffff009181475269531f
ffff000103ca030808fecb0a00000000000000000000cc0101811bc90b009181
475269567fffffffca06000000000000cb0103cc0101
block 2
00000253206445018934551f81474554768fffff0015000a4800015a00024200
016000013300013600013700015b00016600016500017700017800017e000169
00006a00007900009300012200002100010900010a0001260001020001040001
0500010600011000010800012b00002c00012d00012e00015500015600072a00
002f0000300000310000ff7900800932c90688888000a000800935c906000080
00000080093cc906888880008000800943c90688888000800005900f01020000
00308147526905ffffff00910f01020000013a81475269559fffff009310010c
0000009f8147526905ffffff0101960f010e000000eb81475269596fffff0097
0f01010006f69981475269563fffff00940e0001000001000100ffff00000101
95060003790001ea058200370601000001006500000002000002001800000003
00000300170000000400000400010000000500000500150000000a00ffff0065
00000007802ec918009181475269539fffffff009181475269539fffff000103
ca030808fecb0a00000000000000000000cc0101811bc90b009181475269567f
ffffffca06000000000000cb0103cc0101
block 3
Question:
Does the code open the complete file in memory or only read block by block?
Thanks in advance.
Re: Read binary file with line delimeter
Quote:
In main function I had to add "\n" in order to print in different row as below, I'm not sure if you need to do that to get the out you posted.
In function display, have you got the final cout << endl << endl; statement? I get blank lines between my blocks.
Quote:
replacing the 2 if with (BYTE) makes it work! why?
Because if char is typed signed, it can never equal the value 0xff as any value over 127 (0x7f) is treated as negative, so a signed value compared to 0xff is always false! Using the (BYTE) cast, treats it as an unsigned number which can then be compared successfully to 0xff.
Re: Read binary file with line delimeter
Thanks 2kaud for explanation. I undertand better.
Now I see that each block could be processed within the while loop!
Question:
Does the code open the complete file in memory or only read block by block?
Now, I only need to add the function of process that will parse each block within the while loop rigth?
I'd like to parse each block using regex.
Thanks for all help!
Re: Read binary file with line delimeter
I would've taken a more generic approach, utilizing more function parameters instead of hardcoding everything in there...
Code:
void get_fields(const std::string& s, const std::string& find_str, std::vector<std::string>& vec)
{
if (s.find(find_str, 0) == std::string::npos) { vec.push_back(s); return; }
size_t index[2] = { 0, 0 };
const size_t len = find_str.length();
do
{
index[1] = s.find(find_str, index[0]);
if (!(x = s.substr(index[0], index[1] - index[0])).empty()) vec.push_back(x);
index[0] = index[1] + len;
} while (index[1] != std::string::npos);
}
int main(void)
{
const std::string find_str("FF77");
const std::string s("Test1FF77Test2FF77Test3FF77Some textFF77other textFF772");
std::vector<std::string> v;
get_fields(s, find_str, v);
for (unsigned i = 0; i < v.size(); i++)
std::cout << "v[" << i << "]=" << v[i] << std::endl;
return 0;
}
*edit: Wow, didn't even notice all of the other replies on the last page. I'll need to do more reading.
Re: Read binary file with line delimeter
Quote:
Originally Posted by
AceInfinity
I would've taken a more generic approach, utilizing more function parameters instead of hardcoding everything in there...
Code:
void get_fields(const std::string& s, const std::string& find_str, std::vector<std::string>& vec)
{
if (s.find(find_str, 0) == std::string::npos) { vec.push_back(s); return; }
size_t index[2] = { 0, 0 };
const size_t len = find_str.length();
do
{
index[1] = s.find(find_str, index[0]);
if (!(x = s.substr(index[0], index[1] - index[0])).empty()) vec.push_back(x);
index[0] = index[1] + len;
} while (index[1] != std::string::npos);
}
int main(void)
{
const std::string find_str("FF77");
const std::string s("Test1FF77Test2FF77Test3FF77Some textFF77other textFF772");
std::vector<std::string> v;
get_fields(s, find_str, v);
for (unsigned i = 0; i < v.size(); i++)
std::cout << "v[" << i << "]=" << v[i] << std::endl;
return 0;
}
*edit: Wow, didn't even notice all of the other replies on the last page. I'll need to do more reading.
The delimeter as stated in my post #34 is 0xff77 and NOT "FF77". On previous pages we've already covered using string find which is not what is required when working with binary files.
Re: Read binary file with line delimeter
Quote:
Originally Posted by
2kaud
The delimeter as stated in my post #34 is 0xff77 and NOT "FF77". On previous pages we've already covered using string find which is not what is required when working with binary files.
No need for the criticism here. I don't see how hard it could be for somebody to change the function parameters around to be honest, although, my post served as an example... Perhaps you have neglected to read my edit? ;)