Read binary file with line delimeter - Page 3
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 3 of 11 FirstFirst 123456 ... LastLast
Results 31 to 45 of 156

Thread: Read binary file with line delimeter

  1. #31
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Thanks one more time 2kaud.

    I've tested, compiles without errors, but is not printing the content of the blocks, I only receive this.

    Code:
    V[0]=vEJXXYW1_D1211308071344S dE‰4U˙w
    This is the interpreted content before the first delimiter "ff77". I'm not sure what is happens. Maybe is not reading literally the bytes and because of that is not finding the FF77.

    When I see in an hex editor, the first 64 bytes are the following:
    Code:
    76454a58585957315f44313231313330
    38303731333434065320644501893455
    ff7700000153206445018934550f8147
    4549232fffff0015000a4800015a0002
    Last edited by Philidor; October 12th, 2013 at 05:50 PM.

  2. #32
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    Can you zip the first few thousand bytes of the file and post so I can take a look at it. I tried it on a test text file here I created that was over 1000 bytes and it showed the expected output.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  3. #33
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    The attached file has 5 delimiters "FF77", so, 5 blocks.

    I want to have stored in block variable the bytes literally, without any convertion to ascii.

    The first 1024 bytes are below, showing the content of the first block and partially the content of 2nd block.
    Code:
    76454a58585957315f4431323131333038303731333434065320644501893455
    ff7700000153206445018934550f81474549232fffff0015000a4800015a0002
    4200016000013300013600013700015b00017e00016900006a00007900009300
    012200002100010900010a000126000102000104000105000106000110000108
    00012b00002c00012d00012e00015500015600072a00002f0000300000310000
    ff7900800932c90688888000a000800935c90600008000000080093cc9068888
    80008000800943c9068888800080000582003706010000010065000000020000
    0200180000000300000300170000000400000400010000000500000500150000
    000a00ffff006500000007802ec918059181475269531fffffff009181475269
    531fffff000103ca030808fecb0a00000000000000000000cc0101811bc90b00
    9181475269567fffffffca06000000000000cb0103cc0101ff77000002532064
    45018934551f81474554768fffff0015000a4800015a00024200016000013300
    013600013700015b00016600016500017700017800017e00016900006a000079
    00009300012200002100010900010a0001260001020001040001050001060001
    1000010800012b00002c00012d00012e00015500015600072a00002f00003000
    00310000ff7900800932c90688888000a000800935c90600008000000080093c
    PS:I've put txt extension to be able to upload the file.

    Thanks for the help.
    Attached Files Attached Files
    Last edited by Philidor; October 12th, 2013 at 06:09 PM.

  4. #34
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    The delimeter is NOT "FF77". The delimeter is 0xFF77 - which is a different ball game! Also the data contains 0x00 which is usually used to indicate the end of a string. Basically the functions work with ASCII text not binary data. That's why they work fine with the test data but not with your actual binary file.

    I'll have a look at the functions over the next few days.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  5. #35
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    Try this.

    Code:
    #include <iostream>
    #include <fstream>
    #include <vector>
    #include <iomanip>
    using namespace std;
    
    typedef unsigned char BYTE;
    typedef unsigned short int WORD;
    typedef vector<BYTE> bVec;
    
    #ifndef LOBYTE
    	#define LOBYTE(w)	((BYTE)((WORD)(w) & 0xff))
    #endif
    
    #ifndef HIBYTE
    	#define HIBYTE(w)	((BYTE)((WORD)(w) >> 8))
    #endif
    
    class FileFields
    {
    private:
    	ifstream	ifs;
    	bool		opened;
    
    public:
    	FileFields() : opened(false) {}
    
    	~FileFields() {
    		if (opened)
    			ifs.close();
    	}
    
    	bool open(const char* name);
    
    	bool getField(bVec& field, WORD delim = 0xFF77);
    };
    
    bool FileFields::open(const char* name) {
    	ifs.open(name, ios::binary);
    	return (opened = ifs.is_open());
    }
    
    bool FileFields::getField(bVec& field, WORD delim)
    {
    char	by;
    
    bool	cont = true;
    
    	field.clear();
    
    	if (!opened || !ifs.good())
    		return false;
    
    	for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
    		if (by == HIBYTE(delim))
    			if (ifs.peek() == LOBYTE(delim))
    				cont = false;
    
    		if (cont) 
    			field.push_back(by);
    	}
    
    	return true;
    }
    
    void display(const bVec bv)
    {
    	for (int i = 0; i < bv.size(); i++)
    		cout << setw(2) << setfill('0') << hex << (int)bv[i];
    	
    	cout << endl << endl;
    }
    
    
    int main()
    {
    FileFields	ff;
    
    bVec	bv;
    
    	if (!ff.open("binary.txt")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    	while (ff.getField(bv))
    		display(bv);
    
    	return 0;
    }
    Last edited by 2kaud; October 13th, 2013 at 03:28 PM.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  6. #36
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud again,

    Thanks for this help really. I'm testing and the code prints content of the binary correctly.

    Now, I'm trying to print similarly each block, like block[0], block[1] etc like before, but I tried in dispaly function,
    but I see now that function handles one byte at a time.

    How can I print each block separately?

    Thanks for your help, I'm trying to understand the way how you do it.

  7. #37
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    Display prints one block separated by line feeds. In the main program, the vector bv holds one block (data between 2 delimeters) from the file. So you can use the contents of the vector bv to process one block at a time. The class function getField returns a vector containing the contents of the next block each time it is called.

    If you print to print a block heading before each block, try

    Code:
    int main()
    {
    FileFields	ff;
    
    bVec	bv;
    
    int blk = 0;
    
    	if (!ff.open("binary.txt")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    	while (ff.getField(bv)) {
    		cout << "block " << blk++ << endl;
    		display(bv);
    	}
    
    	return 0;
    }
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  8. #38
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud,

    I'm not sure what happens, but trying in that way I receive "block 0" and in the next file the content of complete binary file. So, it seems like all content of the file is contained in block 0.
    Code:
    block 0
    76454a58585957315f4431323131333038303731333434065320644501893455ff7700000......
    I don't see clearly yet how to print something like this:
    Code:
    block 0
    76454a58585957315f4431323131333038303731333434065320644501893455
    block 1
    000001...
    block 2
    000002...
    block 3
    000003...
    .
    .
    block N
    00000N...
    Thanks again for the help.

  9. #39
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    From the partial data file you attached in post #33, the output I get from this program is

    Code:
    block 0
    76454a58585957315f4431323131333038303731333434065320644501893455
    
    block 1
    00000153206445018934550f81474549232fffff0015000a4800015a0002420001
    00013700015b00017e00016900006a00007900009300012200002100010900010a
    010400010500010600011000010800012b00002c00012d00012e00015500015600
    300000310000ff7900800932c90688888000a000800935c9060000800000008009
    8000800943c9068888800080000582003706010000010065000000020000020018
    00170000000400000400010000000500000500150000000a00ffff006500000007
    475269531fffffff009181475269531fffff000103ca030808fecb0a0000000000
    01811bc90b009181475269567fffffffca06000000000000cb0103cc0101
    
    block 2
    00000253206445018934551f81474554768fffff0015000a4800015a0002420001
    00013700015b00016600016500017700017800017e00016900006a000079000093
    010900010a00012600010200010400010500010600011000010800012b00002c00
    5500015600072a00002f0000300000310000ff7900800932c90688888000a00080
    00000080093cc906888880008000800943c90688888000800005900f0102000000
    ffff00910f01020000013a81475269559fffff009310010c0000009f8147526905
    010e000000eb81475269596fffff00970f01010006f69981475269563fffff0094
    0100ffff0000010195060003790001ea0582003706010000010065000000020000
    00000300170000000400000400010000000500000500150000000a00ffff006500
    009181475269539fffffff009181475269539fffff000103ca030808fecb0a0000
    00cc0101811bc90b009181475269567fffffffca06000000000000cb0103cc0101
    What compiler are you using? I use MSVC.

    Try replacing

    Code:
    	if (by == HIBYTE(delim))
    			if (ifs.peek() == LOBYTE(delim))
    with

    Code:
    	if ((BYTE)by == 0xff)
    		if ((BYTE)ifs.peek() == 0x77)
    Is your type char default signed or unsigned? I have default unsigned. If yours is signed, that may be the problem.

    UPDATE That is probably the problem. I changed my default to signed and got the same issue you have. Putting (BYTE) in the 2 if statements makes it work if default char is signed.
    Last edited by 2kaud; October 13th, 2013 at 06:15 PM.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  10. #40
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud,

    Thanks!!!

    I'm using MinGW and Dev C++(TDM GCC) compilers, both with the same issue, but you're rigth, replacing the 2 if with (BYTE) makes it work! why?

    In main function I had to add "\n" in order to print in different row as below, I'm not sure if you needed to do that to get the output you posted.

    Code:
    cout << "\nblock " << blk++ << endl;
    Code:
    block 0
    76454a58585957315f4431323131333038303731333434065320644501893455
    block 1
    00000153206445018934550f81474549232fffff0015000a4800015a00024200
    016000013300013600013700015b00017e00016900006a000079000093000122
    00002100010900010a0001260001020001040001050001060001100001080001
    2b00002c00012d00012e00015500015600072a00002f0000300000310000ff79
    00800932c90688888000a000800935c90600008000000080093cc90688888000
    8000800943c90688888000800005820037060100000100650000000200000200
    180000000300000300170000000400000400010000000500000500150000000a
    00ffff006500000007802ec918059181475269531fffffff009181475269531f
    ffff000103ca030808fecb0a00000000000000000000cc0101811bc90b009181
    475269567fffffffca06000000000000cb0103cc0101
    block 2
    00000253206445018934551f81474554768fffff0015000a4800015a00024200
    016000013300013600013700015b00016600016500017700017800017e000169
    00006a00007900009300012200002100010900010a0001260001020001040001
    0500010600011000010800012b00002c00012d00012e00015500015600072a00
    002f0000300000310000ff7900800932c90688888000a000800935c906000080
    00000080093cc906888880008000800943c90688888000800005900f01020000
    00308147526905ffffff00910f01020000013a81475269559fffff009310010c
    0000009f8147526905ffffff0101960f010e000000eb81475269596fffff0097
    0f01010006f69981475269563fffff00940e0001000001000100ffff00000101
    95060003790001ea058200370601000001006500000002000002001800000003
    00000300170000000400000400010000000500000500150000000a00ffff0065
    00000007802ec918009181475269539fffffff009181475269539fffff000103
    ca030808fecb0a00000000000000000000cc0101811bc90b009181475269567f
    ffffffca06000000000000cb0103cc0101
    block 3
    Question:
    Does the code open the complete file in memory or only read block by block?

    Thanks in advance.
    Last edited by Philidor; October 13th, 2013 at 06:48 PM. Reason: Question:

  11. #41
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    In main function I had to add "\n" in order to print in different row as below, I'm not sure if you need to do that to get the out you posted.
    In function display, have you got the final cout << endl << endl; statement? I get blank lines between my blocks.

    replacing the 2 if with (BYTE) makes it work! why?
    Because if char is typed signed, it can never equal the value 0xff as any value over 127 (0x7f) is treated as negative, so a signed value compared to 0xff is always false! Using the (BYTE) cast, treats it as an unsigned number which can then be compared successfully to 0xff.
    Last edited by 2kaud; October 13th, 2013 at 07:01 PM.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  12. #42
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Thanks 2kaud for explanation. I undertand better.

    Now I see that each block could be processed within the while loop!

    Question:
    Does the code open the complete file in memory or only read block by block?

    Now, I only need to add the function of process that will parse each block within the while loop rigth?

    I'd like to parse each block using regex.

    Thanks for all help!

  13. #43
    Join Date
    Feb 2013
    Location
    Canada
    Posts
    51

    Re: Read binary file with line delimeter

    I would've taken a more generic approach, utilizing more function parameters instead of hardcoding everything in there...

    Code:
    void get_fields(const std::string& s, const std::string& find_str, std::vector<std::string>& vec)
    {
    	if (s.find(find_str, 0) == std::string::npos) { vec.push_back(s); return; }
    	size_t index[2] = { 0, 0 };
    	const size_t len = find_str.length();
    	do
    	{
    		index[1] = s.find(find_str, index[0]);
    		if (!(x = s.substr(index[0], index[1] - index[0])).empty()) vec.push_back(x);
    		index[0] = index[1] + len;
    	} while (index[1] != std::string::npos);
    }
    
    int main(void)
    {
    	const std::string find_str("FF77");
    	const std::string s("Test1FF77Test2FF77Test3FF77Some textFF77other textFF772");
    	std::vector<std::string> v;
    	get_fields(s, find_str, v);
    	for (unsigned i = 0; i < v.size(); i++)
    		std::cout << "v[" << i << "]=" << v[i] << std::endl;
    	return 0;
    }
    *edit: Wow, didn't even notice all of the other replies on the last page. I'll need to do more reading.
    Last edited by AceInfinity; October 14th, 2013 at 01:20 AM.

    Microsoft MVP .NET Programming (2012 - Present)
    ®Crestron DMC-T Certified Automation Programmer

  14. #44
    Join Date
    Dec 2012
    Location
    England
    Posts
    2,701

    Re: Read binary file with line delimeter

    Quote Originally Posted by AceInfinity View Post
    I would've taken a more generic approach, utilizing more function parameters instead of hardcoding everything in there...

    Code:
    void get_fields(const std::string& s, const std::string& find_str, std::vector<std::string>& vec)
    {
    	if (s.find(find_str, 0) == std::string::npos) { vec.push_back(s); return; }
    	size_t index[2] = { 0, 0 };
    	const size_t len = find_str.length();
    	do
    	{
    		index[1] = s.find(find_str, index[0]);
    		if (!(x = s.substr(index[0], index[1] - index[0])).empty()) vec.push_back(x);
    		index[0] = index[1] + len;
    	} while (index[1] != std::string::npos);
    }
    
    int main(void)
    {
    	const std::string find_str("FF77");
    	const std::string s("Test1FF77Test2FF77Test3FF77Some textFF77other textFF772");
    	std::vector<std::string> v;
    	get_fields(s, find_str, v);
    	for (unsigned i = 0; i < v.size(); i++)
    		std::cout << "v[" << i << "]=" << v[i] << std::endl;
    	return 0;
    }
    *edit: Wow, didn't even notice all of the other replies on the last page. I'll need to do more reading.
    The delimeter as stated in my post #34 is 0xff77 and NOT "FF77". On previous pages we've already covered using string find which is not what is required when working with binary files.
    All advice is offered in good faith only. You are ultimately responsible for effects of your programs and the integrity of the machines they run on.

  15. #45
    Join Date
    Feb 2013
    Location
    Canada
    Posts
    51

    Re: Read binary file with line delimeter

    Quote Originally Posted by 2kaud View Post
    The delimeter as stated in my post #34 is 0xff77 and NOT "FF77". On previous pages we've already covered using string find which is not what is required when working with binary files.
    No need for the criticism here. I don't see how hard it could be for somebody to change the function parameters around to be honest, although, my post served as an example... Perhaps you have neglected to read my edit?

    Microsoft MVP .NET Programming (2012 - Present)
    ®Crestron DMC-T Certified Automation Programmer

Page 3 of 11 FirstFirst 123456 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center