Iterators- Help..

[Kohinoor24]

Hi,

I have a problem in hand now.Any help is appreciated.
I have 2 files.I have to read data from 1 file & write to another file.Iam doing that right now as follows:

Opening the "file to read" in a stream.Reading it to a buffer & then writing the buffer to the "output stream".(In Binary mode)
The code is like as follows:

// opening the file to read
ifstream inputFile("File to read",ios::in|ios::binary);
if(inputFile.is_open())
{
istream::pos_type beginpos = inputFile.tellg();
inputFile.seekg(0,ios::end);
istream::pos_type endpos = inputFile.tellg();
inputFile.seekg(0, ios::beg);
int numBytes = endpos - beginpos;
char *buffer= new char[numBytes+1];
inputFile.read(buffer,numBytes);

//File to write
if(!OutPutFile.eof())
m_ResourceGroup.seekp(0,ios::end);
OutPutFile.write(buffer,numBytes);
}

Now this way of writing is causing performance problems.I would like to use the istream & ostream iterators instead of the "read" & "write" calls.Iam no good in iterators & I havent used it in file operations.

Can any one show me how to do it with respect to my code above...

Thanks...

[Yves M]

Hum, what sort of performance problem do you get ?

[Kohinoor24]

When I check my whole application with a "Performance Analyser",This function takes a lot of percentage time compared to the other function calls.So I want to make this call faster.Lots of file will be read using this Function,Thats why...

[Yves M]

Hum, well, the basic problem is that you are reading and writing to the harddisk, which is inherently slow. I don't think iterators would help speed things up. You might want to eliminate a few of the seek operations and try using c-style fread and fwrite.

[KevinHall]

Kohinoor, you only seek 3 times. If you are copying large files, then the hard disk is most likely your bottleneck -- as Yves pointed out. If you are copying many, many small files in this manner, then the seeking (which also happens to be related to the hard disk) is probably your bottleneck. I'd be very surprised if your performance had anything to do with C/C++.

- Kevin

[Kohinoor24]

mmm..,Okay.I will remove some of the seek operations as u said.This "iteartor" is an idea from my group leader.So he wants that to be done & tested for performance.He is a kind of adamant person.So I have to do it.
Could u please tell me with respect to my code how,Iterators can be used..

[galathaea]

Do iterators here mean file pointers? Or something more quixotic?

One way to get better performance, if the files are realatively small compared to memory, would be to read the input file into memory and build the output file in memory. Then, iterators for an stl vector or mere pointer array accessing can be used, and the output can be saved when done. As well, if this is on a Windows system, file mapping may be better still since it will conserve physical memory usage (though your virtual space will still be the same) and map in only those pages you access. Similar functionality can be found on other OS's.

I hope I have helped!

[Edit: This only works if this is to be done over and over, which is what I assumed from the performance issue... Sorry I forgot to state on first posting]

[KevinHall]

I don't believe std::fstream, std::ifstream, or std::ofstream have iterators. Anyway, iterators are nice for traversing complex structures such as linked-lists, trees, etc.... But where a simple while loop or a for loop with simple math will do, iterators have no performance gain (and may possibly have a performance hit).

Personally, I don't see how using iterators (you'd have to create your own here -- yuck) would help.

- Kevin

I hope your boss can understand your dilemma.

[Kohinoor24]

Thanks for ur suggestion.
By iterators,I meant istream_iterator & ostream_iteartor.

[KevinHall]

That's supposed to read std:: ofstream (without the space)above. CodeGuru automatically replaces the combination of ':' and 'o' with a yawn...

How does one get around that without inserting a space?

Also, how does have a graphic displayed by their names in the postings?

[KevinHall]

Well, you learn something new every day. :) The book I learned the STL from only discusses istream_iterator and ostream_iterator for one paragraph -- and that paragraph shows up 16 chapters before istream and ostream. Some book! :mad:

Thanks for enlightening me.

- Kevin

P.S. I still don't see how the iterators are going to improve performance...

[Kohinoor24]

I just read now that istreambuf_ iterator & ostreambuf_iterator does the job.Iam working on binary data.I dont know wheather that will work or not...

[Yves M]

Originally posted by KevinHall
That's supposed to read std:: ofstream (without the space)above. CodeGuru automatically replaces the combination of ':' and 'o' with a yawn...

How does one get around that without inserting a space?

Also, how does have a graphic displayed by their names in the postings?

When you post, there is a check-box at the bottom
Disable Smilies in This Post
To choose an avatar, go to user cp, Edit Options, Change Avatar.

For Kohinoor, I think that using stream iterators will only make your performance issue worse, if it does change anything. It might also not change a thing.

[Philip Nicoletti]

Just a note: I don't know if you left code
out of your original post or not, but you
do a "new []" and no "delete []".

[galathaea]

Wow. Something more quixotic that I never heard of. Great lesson here!

[Federal102]

I think this is what you may be trying to get at...



int main()
{
ifstream infile("test.txt");
ofstream outfile("output.txt");
ostream_iterator<DString> outIter(outfile, "\n");
istream_iterator<DString> inIter(infile);
istream_iterator<DString> endIter;
copy(inIter, endIter, outIter);

return 0;
}



This will copy the contents of test.txt to output.txt.

Please note the use of "DString" - my own derived string class.

This code does not work correctly with the string class provided with VC++6.0 sp5.

Hope this gives you the general idea.

[Kohinoor24]

Iam working with files on Binary mode.Will these iterators work for Binary mode as well apart from Text mode.

[Federal102]

I have only worked with these iterators on text files, but they should work on binary.

Replace DString with char and add ios::binary and see what happens.