CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 7 of 11 FirstFirst ... 45678910 ... LastLast
Results 91 to 105 of 156
  1. #91
    Join Date
    Apr 1999
    Posts
    27,449

    Re: Read binary file with line delimeter

    Quote Originally Posted by Philidor View Post
    Hello Paul,

    But create 64 bit program once compiled, will run on a 32 bit machine?
    No. A 64-bit program won't run on a 32-bit system. If it's some sort of utility that you're going to run on your own computer, and you have a 64-bit OS, then maybe the 64-bit option is one to consider.

    Regards,

    Paul McKenzie

  2. #92
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Quote Originally Posted by 2kaud View Post
    Yes - but are you still getting the error reported in post #84 or does it now run to completion correctly?
    Hello 2kaud,

    No, i don't receive error, only the code it seems to process only 8534 blocks, but I don't know if it is because it's finding a "78" when processes the block 8534 and think that is the end of file. Because of that I put as comment the "else" statement to test that and doing that processed much more blocks but stopped suddenly as I said.

    I'll put as comment the else statement in your last code and I'll test again to answer you what happens.

  3. #93
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    [see my Private Message re the data]
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  4. #94
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    OK. The problem is that the data file has a corrupted block 65330. So getField was returning a block just containing "00". This string has a size of 2. The substr is requesting a block starting at position 6 which in this case is beyond the end of the string so substr throws an out_of_range exception which is not handled by the program hence abnormal termination!

    I've altered the getField routine to try to deal with this problem but you'll need to look at this block of data to see what is the problem.

    The new code is

    Code:
    #include <iostream>
    #include <iomanip>
    #include <fstream>
    #include <sstream>
    using namespace std;
    
    typedef unsigned char BYTE;
    typedef unsigned short int WORD;
    
    const char hconv[16] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
    
    #ifndef LOBYTE
    	#define LOBYTE(w)	((BYTE)((WORD)(w) & 0xff))
    #endif
    
    #ifndef HIBYTE
    	#define HIBYTE(w)	((BYTE)((WORD)(w) >> 8))
    #endif
    
    class FileFields
    {
    private:
    	ifstream	ifs;
    	bool		opened;
    
    public:
    	FileFields() : opened(false) {}
    
    	~FileFields() {
    		if (opened)
    			ifs.close();
    	}
    
    	bool open(const char* name);
    
    	bool getField(string& field, WORD delim = 0xFF77);
    };
    
    bool FileFields::open(const char* name) {
    	ifs.open(name, ios::binary);
    	return (opened = ifs.is_open());
    }
    
    bool FileFields::getField(string& field, WORD delim)
    {
    char	by;
    
    bool	cont;
    
    BYTE	next;
    
    	if (!opened || !ifs.good())
    		return false;
    
    	do {
    		cont = true;
    		field = "";
    
    		for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
    			if ((BYTE)by == HIBYTE(delim))
    				if ((next = (BYTE)ifs.peek()) == LOBYTE(delim))
    					cont = false;
    				else
    					if (next == 0x78) {
    						cont = false;
    						ifs.setstate(ios::failbit);
    					}
    
    			if (cont) {
    				field += hconv[(BYTE)by >> 4];
    				field += hconv[(BYTE)by & 0xf];
    			}
    		}
    	} while (field.size() <= 2 && ifs.good());
    
    	return true;
    }
    
    int main()
    {
    FileFields	ff;
    
    	if (!ff.open("d:\\philidor\\bin2g")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    string block;
    
    	for (unsigned int blk = 0; ff.getField(block); blk++) {
    		if (blk == 0) continue;
    
    		if (blk > 65328) {
    			cout << "block " << blk << endl;
    			istringstream iss(block.substr(0, 6));
    			unsigned long int x = 0;
    
    			iss >> hex >> x;
    			cout << x << "," << block.size() << ",";
    
    			if (block.size() >= 7)
    				cout << block.substr(6, 16);
    			else
    				cout << block;
    
    			cout << endl;
    		}
    	}
    
    	return 0;
    }
    For test purposes, I've only output the data for block numbers greater than 65328. You'll need to change the contents of the for loop in main() to undertake the required processing for each block. Also note that for my testing purpose, I've changed the name of the file being opened.

    The 'good' news is that the problem wasn't size related so the 'fast' code using c++ streams is fine.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  5. #95
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud,

    Thank you. I've been doing some tests and it processed all file of 2GB in 4 min! (without regex function).

    But I've found an issue, in block 65399 (65399 = 00FF77 in hex = block delimiter), the first 3 bytes are missing, because it seems is being confused with delimiter.

    Code:
    block 65397 = 00FF75 in hex
    00FF75-532064019857384F50440429467FFFFF0015000A48000..
    
    block 65398 = 00FF76 in hex
    00FF76-532064019857385F50440429468FFFFF0015000A48000..
    
    block 65399 = 00FF77 in hex, but is missing in block 65399 
    532064019857386F50440429469FFFFF0015000A4800015A001..
    
    block 65400 = 00FF78 in hex
    00FF78-532064019857387F50440429470FFFFF0015000A48000..
    
    block 65401 = 00FF79 in hex
    00FF79-532064019857388F50440429471FFFFF0015000A48000..

  6. #96
    Join Date
    Apr 1999
    Posts
    27,449

    Re: Read binary file with line delimeter

    Quote Originally Posted by Philidor View Post
    Hello 2kaud,

    Thank you. I've been doing some tests and it processed all file of 2GB in 4 min! (without regex function).

    But I've found an issue, in block 65399 (65399 = 00FF77 in hex = block delimiter), the first 3 bytes are missing, because it seems is being confused with delimiter.
    Is it the original file that has this data, or is it the program not getting the bytes correctly?

    Open the original file and see if you can find the data. If the data is indeed 00FF77 and it is not a delimiter, then the original file has inconsistencies. You can't do anything about that until the file is changed, or the set of rules of how to interpret the file are changed.

    Regards,

    Paul McKenzie

  7. #97
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello Paul,

    Yes, is within the file. This is the small piece of file in text format where this occured.
    The file doesn't have inconsistencies, only happens that the 3 bytes after the delimiter FF77 are correlative values that say the block number, so increment one by one.

    The code works well, because 00ff77 is just another string that contains the delimiter sequence, so I'll try to see how to avoid the code removes the 00ff77 correlative.

    Code:
    005500005600072a00002f0000300000310000ff3400800932c9060000a00000
    00800935c9060000a000000080093cc9060000a0000000ff7700ff7653206401
    9857385f50440429468fffff0015000a4800015a001442000133000136000137
    00017e00016900017900009300012200002100012600011000012b00002c0000
    2d00002e00005500005600072a00002f0000300000310000ff3400800932c906
    0000a0000000800935c9060000a000000080093cc9060000a000000007802ec9
    18059181495269539fffffff009181495269539fffff000103ca030808fecb0a
    00000000000000000000cc0101ff7700ff77532064019857386f50440429469f
    ffff0015000a4800015a00144200013300013600013700017e00016900017900
    009300012200002100012600011000012b00002c00002d00002e000055000056
    00072a00002f0000300000310000ff3400800932c9060000a0000000800935c9
    060000a000000080093cc9060000a0000000ff7700ff78532064019857387f50
    440429470fffff0015000a4800015a00144200013300013600013700017e0001
    6900017900009300012200002100012600011000012b00002c00002d00002e00
    005500005600072a00002f0000300000310000ff3400800932c9060000a00000
    00800935c9060000a000000080093cc9060000a0000000

  8. #98
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    True! The easiest way for this is to deal with it in main() thus keeping getField() simple as returning the data between 2 delimeters.

    A possible revised program is

    Code:
    #include <iostream>
    #include <iomanip>
    #include <fstream>
    #include <sstream>
    using namespace std;
    
    typedef unsigned char BYTE;
    typedef unsigned short int WORD;
    
    const char hconv[16] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
    
    #ifndef LOBYTE
    	#define LOBYTE(w)	((BYTE)((WORD)(w) & 0xff))
    #endif
    
    #ifndef HIBYTE
    	#define HIBYTE(w)	((BYTE)((WORD)(w) >> 8))
    #endif
    
    class FileFields
    {
    private:
    	ifstream	ifs;
    	bool		opened;
    
    public:
    	FileFields() : opened(false) {}
    
    	~FileFields() {
    		if (opened)
    			ifs.close();
    	}
    
    	bool open(const char* name);
    
    	bool getField(string& field, WORD delim = 0xFF77);
    
    };
    
    bool FileFields::open(const char* name) {
    	ifs.open(name, ios::binary);
    	return (opened = ifs.is_open());
    }
    
    bool FileFields::getField(string& field, WORD delim)
    {
    char	by;
    
    bool	cont = true;
    
    BYTE	next;
    
    	field = "";
    
    	if (!opened || !ifs.good())
    		return false;
    
    	for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
    		if ((BYTE)by == HIBYTE(delim))
    			if ((next = (BYTE)ifs.peek()) == LOBYTE(delim))
    				cont = false;
    			/*else
    				if (next == 0x78) {
    					cont = false;
    					ifs.setstate(ios::failbit);
    				}*/
    
    		if (cont) {
    			field += hconv[(BYTE)by >> 4];
    			field += hconv[(BYTE)by & 0xf];
    		}
    	}
    
    	return true;
    }
    
    int main()
    {
    FileFields	ff;
    
    	if (!ff.open("d:\\philidor\\bin2g")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    string	block;
    
    	for (unsigned int blk = 0; ff.getField(block); blk++) {
    		if (blk == 0) continue;
    
    		if (block.size() <= 2) {
    			string block1;
    
    			ff.getField(block1);
    			block += "ff77" + block1;
    		}
    
    		istringstream iss(block.substr(0, 6));
    		unsigned long int x = 0;
    
    		iss >> hex >> x;
    		if (x != blk) {
    			cout << "block " << blk << endl << x << "," << block.size() << ",";
    			if (block.size() >= 7) {
    				cout << block.substr(0, 6) << "," << block.substr(6,16);
    			} else {
    				cout << block;
    			}
    			cout << endl;
    		}
    		//cout << x << "," << block.substr(6,16) << endl;
    	}
    
    	return 0;
    }
    Note that in some cases the block number obtained from the file does not match the expected block number and this version of main() will highlight these blocks.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  9. #99
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    I note your PM comment re the increased run time and have changed the code to try to reduce this. The revised code is

    Code:
    #include <iostream>
    #include <iomanip>
    #include <fstream>
    #include <sstream>
    #include <ctime>
    using namespace std;
    
    typedef unsigned char BYTE;
    typedef unsigned short int WORD;
    typedef unsigned long int DWORD;
    
    const char hconv[16] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
    
    #ifndef LOBYTE
    	#define LOBYTE(w)	((BYTE)((WORD)(w) & 0xff))
    #endif
    
    #ifndef HIBYTE
    	#define HIBYTE(w)	((BYTE)((WORD)(w) >> 8))
    #endif
    
    const WORD SEPAR = 0xFF77;
    
    class FileFields
    {
    private:
    	ifstream	ifs;
    	bool		opened;
    
    public:
    	FileFields() : opened(false) {}
    
    	~FileFields() {
    		if (opened)
    			ifs.close();
    	}
    
    	bool open(const char* name);
    
    	bool getBlock(string& field, DWORD& block, WORD delim = SEPAR);
    	bool getField(string& field, WORD delim = SEPAR);
    
    };
    
    bool FileFields::open(const char* name) {
    	ifs.open(name, ios::binary);
    	return (opened = ifs.is_open());
    }
    
    bool FileFields::getBlock(string& field, DWORD& number, WORD delim)
    {
    BYTE	num[3];
    
            number = 0;
    
    	if (!opened || !ifs.good())
    		return false;
    
    	ifs.read((char*)num, 3);
    	number = (num[0] << 16) + (num[1] << 8) + num[2];
    
    	return getField(field);
    }
    
    bool FileFields::getField(string& field, WORD delim)
    {
    char	by;
    
    bool	cont = true;
    
    	field = "";
    
    	if (!opened || !ifs.good())
    		return false;
    
    	for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
    		if ((BYTE)by == HIBYTE(delim))
    			if ((BYTE)ifs.peek() == LOBYTE(delim))
    				cont = false;
    
    		if (cont) {
    			field += hconv[(BYTE)by >> 4];
    			field += hconv[(BYTE)by & 0xf];
    		}
    	}
    
    	return true;
    }
    
    int main()
    {
    FileFields	ff;
    
    	if (!ff.open("d:\\philidor\\bin2g")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    string	header;
    
    	ff.getField(header);
    
    string	block;
    
    DWORD number;
    
    time_t timest = time(NULL);
    
    	for (DWORD blk = 1; ff.getBlock(block, number); blk++) {
    
    		//if (number != blk) 
    		//cout << "block " << blk << endl << number << "," << block.size() << "," << block.substr(0, 16) << endl;
    	}
    
    	cout << "Time taken: " << time(NULL) - timest;
    	return 0;
    }
    The changes are
    - Obtain the first header field before the for loop in main so not checking blk in every loop (so blk now starts at 1 in the for loop).

    - Added new class function getBlock() which returns the block of data as a hex string following the block number and also returns the block number as a separate parameter of getBlock. Note that the string returned by getBlock does not now start with the 6 character block number.

    - Added simple timer to display time taken to process file.

    These changes should make processing quicker and also deal with having a block number of 0xff77 without having to have specific overhead code to deal with it.

    Let us know how you get on.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  10. #100
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud,

    Thank you!

    I've tested and this time the block number 65399 (00FF77 in hex) is printing completely too.

    The complete file, printing block number, size, blk and first 32 characters of each block took 216 seconds (3.6 min)!

    I think it'll work in a 32 machine too, I'll continue doing testing but it seems is very fast code and ouputs all blocks from a 2GB file.

    What I'm not sure if the code actually loads all file in memory or does it read the input file sequentially?

    Thanks really for all the help 2kaud.

    Best regards

  11. #101
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    What I'm not sure if the code actually loads all file in memory or does it read the input file sequentially?
    The program reads the input file sequentially (using get/read). How the underlying class/runtime does this is down to the class/runtime.

    I think it'll work in a 32 machine too, I'll continue doing testing but it seems is very fast code and ouputs all blocks from a 2GB file.
    It works for me on my 32 bit computer.

    If you can explain what your outString() function is trying to accomplish (without using regex notation!) I'll see if the functanality can be easily done in c++ without using regex classes which should make that function quicker.
    Last edited by 2kaud; October 26th, 2013 at 02:55 PM.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  12. #102
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    You might get slightly faster results if you pre-set the capacity of the block string in main(). Try

    Code:
    string	block;
    	block.reserve(7000);
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  13. #103
    Join Date
    Oct 2013
    Posts
    63

    Re: Read binary file with line delimeter

    Hello 2kaud,

    Thank you for the answer and yes, it seems to work a little faster with pre-set block size.

    Now I'll try to explain what are the conditions to get the other substrings from each block. For this purpose please see the sample block below with the subtrings of interest highligthed with red, blue or green:

    Code:
    532064450189342f81474557521fffff0015000a4800015a0001420001600001
    3300013600013700015b00016600016500017700016900006a00007900009300
    012200002100010900012600010800012b00002c00012d00012e000155000156
    00072a00002f0000300000310000ff7900800932c90600000000a000800935c9
    0600000000000080093cc906000000008000800943c90600000000800005910f
    01020000000d8147526905ffffff009310010c0000000d8147526905ffffff01
    01960f010c0000000d81475269565fffff00940e01020102010001ffffff0201
    0201950600000000000005820037060100000100650000000200000200180000
    000300000300170000000400000400010000000500000500150000000a00ffff
    006500000007802ec918009181475269555fffffff009181475269555fffff00
    0103ca03001cfecb0a00000000000000000000cc0101811bc90b009181475269
    557fffffffca06000000000000cb010bcc0101

    For the first 2 strings:
    They always appear in each block and are fixed length (8 bytes = 16 characters) and with substring (0,16) and (16,16) could be extracted.
    The issue I used regex here is because normally both substrings have one or more of letter "f" and I want to print them without f's. So, intead of print 532064450189342f and 81474557521fffff I want to print only 532064450189342 and 81474557521.

    For the next sub strings:
    The other substrings don't appear in all blocks. They appear only when FF79 is present within each block. So, the substrings will be inside the sub-block that begin with FF790080+4 characters + c906 (ff7900800932c906 in this example) and ends before 9506. In this example I show the sub-block underlined.

    But the substrings I want from sub-block actually begin after "05" is present and each substring begins with 9X (where X=0,1,2,3,6,7).
    The number of substrings could be variable, sometime could be 2 subtringss after "05", sometimes 4 or more.

    If at least one of the previous substrings exist, the substring in green that begins with 940E will be present and always will be at the end.

    Then, for this block, the preliminar output (combining first 2 strings and the other subtrings if they exist) will be:

    Code:
    Block #|532064450189342|81474557521|910f01020000000d8147526905ffffff00|9310010c0000000d8147526905ffffff0101|960f010c0000000d81475269565fffff00|940e01020102010001ffffff02010201
    The byte after 91, 93, 94 or 96 etc, is the lenght of the substring. For example for the first substring 910f01020000000d8147526905ffffff00, you can see that 91 is followed by 0F(15 in decimal). Then after 910F there are 15 bytes (30 characters).

    I say this is the preliminar output, since if the substrings could be get, then each substring needs a subroutine to separate it in pieces printing some pieces as they are and other converting from hex to decimal. This would be at the end, the first step is get the preliminary output mentioned above.

    I hope is not to complicated without regex. I saw it too much complicated without regex due that substrings don't appear in a fixed position of the block and not always appear.

    Thanks in advance again for the time and help.

  14. #104
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    For the first 2 strings, does the 'f' always appear at the end of the string, or can 'f' appear anywhere in the string? Is it just the 'f' at the end you don't want or 'f' that appears anywhere within this 16 bytes?

    PS I think the 'f' always only appears at the end. Can you confirm.

    PPS In the preliminar output does it start with a '|' and the various parts separated by '|'?
    Last edited by 2kaud; October 27th, 2013 at 08:01 AM.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

  15. #105
    2kaud's Avatar
    2kaud is online now Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    7,822

    Re: Read binary file with line delimeter

    I've incorrporated the first part (first 2 strings) into the getBlock() function which now returns the required data as the string firstpart. Is this what you are after?

    Code:
    #include <iostream>
    #include <iomanip>
    #include <fstream>
    #include <sstream>
    #include <ctime>
    using namespace std;
    
    typedef unsigned char BYTE;
    typedef unsigned short int WORD;
    typedef unsigned long int DWORD;
    
    const char hconv[16] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F'};
    
    #ifndef LOBYTE
    	#define LOBYTE(w)	((BYTE)((WORD)(w) & 0xff))
    #endif
    
    #ifndef HIBYTE
    	#define HIBYTE(w)	((BYTE)((WORD)(w) >> 8))
    #endif
    
    const WORD SEPAR = 0xFF77;
    
    class FileFields
    {
    private:
    	ifstream	ifs;
    	bool		opened;
    
    public:
    	FileFields() : opened(false) {}
    
    	~FileFields() {
    		if (opened)
    			ifs.close();
    	}
    
    	bool open(const char* name);
    
    	bool getBlock(string& field, DWORD& number, string& firstpart, WORD delim = SEPAR);
    	bool getField(string& field, WORD delim = SEPAR);
    
    };
    
    bool FileFields::open(const char* name) {
    	ifs.open(name, ios::binary);
    	return (opened = ifs.is_open());
    }
    
    bool FileFields::getBlock(string& field, DWORD& number, string& firstpart, WORD delim)
    {
    BYTE	num[3],
    	first[16],
    	by,
    	ub,
    	lb;
    
    	number = 0;
    	firstpart = "|";
    
    	if (!opened || !ifs.good())
    		return false;
    
    	ifs.read((char*)num, 3);
    	number = (num[0] << 16) + (num[1] << 8) + num[2];
    
    	if (!ifs.good())
    		return false;
    
    	ifs.read((char*)first, 16);
    
    	for (int p = 1; p <= 2; p++) {
    		const int last = p * 8;
    		for (int i = (p - 1) * 8; i < last; i++)
    			if ((ub = ((by = first[i]) >> 4)) < 0xf) {
    				firstpart += hconv[ub];
    				if ((lb = (by & 0x0f)) < 0xf)
    					firstpart += hconv[lb];
    				else
    					break;
    			} else
    				break;
    
    		firstpart += '|';
    	}
    
    	return getField(field);
    }
    
    bool FileFields::getField(string& field, WORD delim)
    {
    char	by;
    
    bool	cont = true;
    
    	field = "";
    
    	if (!opened || !ifs.good())
    		return false;
    
    	for (ifs.get(by); cont && ifs.gcount(); ifs.get(by)) {
    		if ((BYTE)by == HIBYTE(delim))
    			if ((BYTE)ifs.peek() == LOBYTE(delim))
    				cont = false;
    
    		if (cont) {
    			field += hconv[(BYTE)by >> 4];
    			field += hconv[(BYTE)by & 0xf];
    		}
    	}
    
    	return true;
    }
    
    int main()
    {
    FileFields	ff;
    
    	if (!ff.open("d:\\philidor\\bin2g")) {
    		cout << "Cannot open file!" << endl;
    		return 1;
    	}
    
    string	header;
    	ff.getField(header);
    
    string	block;
    	block.reserve(7000);
    
    string firstpart;
    	firstpart.reserve(40);
    
    DWORD	number;
    
    time_t timest = time(NULL);
    
    	for (DWORD blk = 1; ff.getBlock(block, number, firstpart); blk++) {
    		if (blk < 20 )
    			cout << number << "," << firstpart << "," << block.substr(0, 10) << endl;
    		else 
    			break;
    	
    		//if (number != blk) 
    		//cout << "block " << blk << endl << number << "," << block.size() << "," << block.substr(0, 16) << endl;
    	}
    
    	cout << "Time taken: " << time(NULL) - timest << endl;
    	return 0;
    }
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++23 Compiler: Microsoft VS2022 (17.6.5)

Page 7 of 11 FirstFirst ... 45678910 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured