Reading a file using istream_iterator

[Richard.J]

I have read the latest post about istream_iterator here and this leads to another question.
I have a file which contains char sequences ("strings"), separated by \0.
I'd like to be able to read the file using istream_iterators. I have been able to accomplish this when the delimiter in the file is \n rather than \0.
Could someone please help me out?

// using istringstream instead of ifstream for this sample code
std::istringstream iss(
//  "text10\0text20\0text30\0text40\0text50\0text60\0" // this does not work
  "text10\ntext20\ntext30\ntext40\ntext50\ntext60\n" // this works
);

int main()
{
    std::istream_iterator<std::string> it_first(iss);
    std::istream_iterator<std::string> it_last;
    std::ostream_iterator<std::string> it_out(std::cout, " ");
    std::copy(it_first, it_last, it_out);
    std::cout << std::endl;
}

[superbonzo]

well, there are two reasons why your code won't work:

first, the istream_iterator<T> invokes T's operator>> and std::string operator>> uses std::isspace ( with the stream locale ) as a separator test, that in turn always return false for the null character. Hence, your input stringstream should extract a single string with multiple null-characters inside.

actually, the line "std::istringstream iss( "text10\0text20\0text30\0text40\0text50\0text60\0" );" invokes the converting std::string constructor which simply copy the first null terminated string in the stringstream; so, in the end a single "text10" string will be extracted.

Anyway, I suppose your real code uses a file stream instead, doesn't it ?

if yes, I'd use the std::getline( some_istream, some_string_buffer, 0 ) function in a loop instead ...

[Richard.J]

Well yes, at the moment the sample code always only extracts the first string "text10".

I'd like to use a real file, but even with my sample code and using std::getline on it I did not get the desired result.
I'll make a new test program that works on a file.

[monarch_dodra]

Hey, that kind of looks like my code!

Anywhoo, AFAIK, and according to this, istream_iterator, is just an iterator adaptor, that uses operator>> internally, which means that if you can't write this anyways:

#include <iostream>
#include <sstream>
#include <iterator>

// using istringstream instead of ifstream for this sample code
std::istringstream iss(
  "text10\0text20\0text30\0text40\0text50\0text60\0" // this does not work
//  "text10\ntext20\ntext30\ntext40\ntext50\ntext60\n" // this works
);

int main()
{
  std::string word;
  while (iss >> word)
  {
    std::cout << word << " ";
  }
}

Then istream_iterator, by design, can't work.
Unless I'm mistaken, isn't '\0' universally recognized a stream terminator? I may be mistaken (I AM!), but I believe that unless you clear that '\0' after each read, than no solution can work. Even this fails:

[EDIT]: See super bonzo's reply as to why this doesn't actually work.

#include <iostream>
#include <sstream>
#include <iterator>

// using istringstream instead of ifstream for this sample code
std::istringstream iss(
  "text10\0text20\0text30\0text40\0text50\0text60\0" // this does not work
//  "text10\ntext20\ntext30\ntext40\ntext50\ntext60\n" // this works
);

int main()
{
  std::string word;
  while (getline(iss, word, '\0'))
  {
    std::cout << word << " ";
  }
}

[superbonzo]

<EDIT: uhm, the following is in reply to post#3, of course>

yes, and I already explained why getline does not work with your code, it's because of the way you're initializing the stringstream. Consider the following:

std::istringstream iss( "text10\0text20\0text30\0text40\0text50\0text60\0" );

int main()
{
	std::string buffer;

	while( std::getline( iss, buffer, char() ) )
	{
		std::cout << buffer << std::endl;
	}
}

this won't work, outputting "text10". But this:

char iss_buf[] = "text10\0text20\0text30\0text40\0text50\0text60\0";
std::istringstream iss( std::string( iss_buf, sizeof(iss_buf) ) );

int main()
{
	std::string buffer;

	while( std::getline( iss, buffer, char() ) )
	{
		std::cout << buffer << std::endl;
	}
}

works as expected.

[SkyNetTo]

Quick test on a real file.

#include <iostream>
#include <fstream>
#include <iterator>
#include <string>

class line {
	std::string data;
public:
	friend std::istream &operator>>(std::istream &is, line &l) {
		std::getline(is, l.data,'\0');
		return is;
	}
	operator std::string() const { return data; }    
};



int main()
{
	std::ifstream in("in.txt");
	std::istream_iterator<line> it_first(in);
	std::istream_iterator<line> it_last;
	std::ostream_iterator<std::string> it_out(std::cout, "\n");
	std::copy(it_first, it_last, it_out);
	in.close();
	std::cout << std::endl;
}

Seems to work.

[superbonzo]

Even this fails:

that fails for the reason explained in post #2

[Richard.J]

Thanks superbonzo, I think I understand. I'll give it a try in my real prog.

monarch_dodra, the sample code is indeed a copy of yours. The other thread with istream_operator motivated my to give the istream_iterator a try and I started with your sample, modifying it, testing around and get to no solution.