Re: Mixing istream and FILE*
You may find this article useful. It shows how to convert between fstream and FILE* handles, going through C file descriptors as an intermediate step. It might only work on Windows, though, as the article also mentions native Windows file handles.
Re: Mixing istream and FILE*
Yeah, fd() doesn't seem to be a standard fstream method as far as I can tell.
The other option I found----fpos::get_fpos_t(), which would give me something to pass to fsetpos()----also appears to be Windows-only.
Re: Mixing istream and FILE*
Since your C++ objects can write to streams, you can do that, and then write the stream internals to the FILE:
Code:
template <typename T>
void writeToFileViaStream(FILE* file, const T& object, bool binaryMode = false)
{
std::ios_base::openmode openMode = std::stringstream::in |
std::stringstream::out |
(binaryMode ? std::binary : 0;
std::stringstream ioss(openMode)
ioss << object;
//This part is copied verbatim from cplusplus.com
//there might be better ways to do this, but I just wanted a quick proof of concept.
//Watch out for memory leaks!
{
// get length of file:
ioss.seekg (0, ios::end);
length = is.tellg();
ioss.seekg (0, ios::beg);
// allocate memory:
char buffer = new char [length];
// read data as a block:
ioss.read(buffer,length);
ioss.close();
fwrite(buffer, 1, length, file);
delete[] buffer;
}
}
This has the advantage of being pretty straight forward interface wise. It is just like fwrite, but can be used on C++ objects that only know operator>>.
Heck, given the signature, I'm sure we could just call this function "fwrite" and it would work. I would not recommend doing that though :/
The disadvantages include 2 extra writes: 1 to the stream, and another to a temporary buffer. I'm sure we can avoid the second copy, but I'm not concentrating on performance here.
IMO, the BIG disadvantage of this method is that it wraps C++ functionality into a C interface. I would really have liked to do the opposite, but you just can't do much with a FILE, apart from writing to a file... You could try toying with the FILE buffer, but that is so dangerous I shudder at the very thought.
(This of course also works for fread)
Is this a satisfactory solution to your problem?
Re: Mixing istream and FILE*
I'm sure that will work great for writing....but then, for writing, I could simply write the C++ half, close the file, fopen() it again in append mode, and proceed. Writing isn't hard.
The difficulty comes when I try to read it again.
For now, I'm making the assumption that tellg() returns the file position in bytes, although I haven't been able to find any documentation which confirms that. I can pass that in fseek() easily enough; I know *that* operates in bytes. And the only downside of fseek() is that it takes a long argument, which is occasionally not big enough (operating on a 4GB+ file on a Win64 system....long remains a 32-bit type). Probably not going to be an issue in my case though.
Re: Mixing istream and FILE*
Well I was supposing you wanted something a bit more clever than opening/closing the file every time by hand. Although I guess you could do that inside the my "WriteViaStreamMethod", a side effect is that it would probably change your file handle, which the caller might not expect.
I thought my method could also be used for reading, but I just now realized that there is no way to know in advance how many bytes will have to be read into the buffer, making my method not-usable.
I just don't see any "elegant" way around this problem. I guess there is no way around this problem but opening closing the file every time.
Quote:
tellg() returns the file position in bytes, although I haven't been able to find any documentation which confirms that.
tellg returns an iterator, so you don't even have the guarantee it returns an amount of bytes either! Just an offset from the begining. If you get the tellg from the beginning of the file, and the current tellg, then std::distance(tellg_first, tellg_current) should give you the amount of elements read so far by your stream. Since an element is pretty much guaranteed to be a char, I think it is safe* to assume that 1 element = 1 byte.
*Just be careful of binary non-binary misalignment. If your file is in text mode, then a single element like "\n" could represent 2 bytes. However, ftell should have the same behavior too, so as long as you open both the stream and file in the same mode, you should be safe.
That, and when opening your file, you can just do
Code:
long numbytes = myFile->ftell();
...
Close File, open stream
...
streampos current = myFstream.tellg(); //gets an iterator to "begin"
std::advance(current, numbytes);
myFstream.seekg(current);
There, keep us posted on your findings.
Re: Mixing istream and FILE*
Another option would be to read the first object from the ifstream, then read the remainder of the file into a string, then use tmpfile() to generate a temporary file. fwrite() the string into the temporary file, seek back to the beginning, and proceed to read it.
I'm not sure if the temporary file would be in memory or on disk, though. If it's in memory then the inefficiency essentially amounts to a couple of extra copies. If it's on disk, this is probably not ideal.
Re: Mixing istream and FILE*
Quote:
Originally Posted by
Lindley
At first I thought someone might have cleverly devised an istream-derived class which simply wraps a FILE*. Maybe it would be slower than an ifstream, but it would do what I need. However, after a bit of searching I've found no such thing, surprisingly.
I'm pretty sure gcc's implementation of fstream does this. Perhaps you could write your own based on that?
Re: Mixing istream and FILE*
Quote:
Originally Posted by
Lindley
Another option would be to read the first object from the ifstream, then read the remainder of the file into a string, then use tmpfile() to generate a temporary file. fwrite() the string into the temporary file, seek back to the beginning, and proceed to read it.
I'm not sure if the temporary file would be in memory or on disk, though. If it's in memory then the inefficiency essentially amounts to a couple of extra copies. If it's on disk, this is probably not ideal.
I'm pretty sure tmpfile opens a file on disc. You just have the guarantee (kind of) that the file name will be unique and super complicated.
Also, If your file is something like 2GB, and you have interleaving C-C++ objects, then welcome to slow ville.
Here's a crazy idea. Start by reading the entire file into a stream, and then keep both open in parallel. Everytime you read from one, just advance by the same amount in the other. You'd have bit of space overhead, but you wouldn't be opening and closing your file every object read, nor copying all it's content to a temporary file every time.
This wouldn't work quite as well with write, but as already discussed, there are other alternatives for writing, and you rarelly need to mix read an write operations on the same file.
Re: Mixing istream and FILE*
maybe you'll find this article useful...
Re: Mixing istream and FILE*
Quote:
Originally Posted by
superbonzo
maybe you'll find
this article useful...
Awesome link.
Code:
int main()
{
std::cout << "read then write" << std::endl;
{
FILE* fileOut = fopen("hello.txt", "w");
syncbuf buf(fileOut);
std::ostream out(&buf);
fputs("hello", fileOut);
out << " the ";
fputs("world", fileOut);
fclose(fileOut);
}
{
FILE* fileIn = fopen("hello.txt", "r");
syncbuf buf(fileIn);
std::istream in(&buf);
std::string hello;
char the[256];
std::string world;
in >> hello;
fscanf(fileIn, "%s", the);
in >> world;
std::cout << hello << " " << the << " " << world << std::endl;
std::cout << "hello:" << "\"" << hello << "\"" << std::endl;
std::cout << "the: " << "\"" << the << "\"" << std::endl;
std::cout << "world:" << "\"" << world << "\"" << std::endl;
fclose(fileIn);
}
std::cout << std::endl;
std::cout << "read and write" << std::endl;
{
FILE* file = fopen("hello2.txt", "rw");
syncbuf buf(file);
std::iostream io(&buf);
fputs("hello", file);
io << " the ";
fputs("world", file);
std::string hello;
char the[256];
std::string world;
io >> hello;
fscanf(file, "%s", the);
io >> world;
std::cout << hello << " " << the << " " << world << std::endl;
std::cout << "hello:" << "\"" << hello << "\"" << std::endl;
std::cout << "the: " << "\"" << the << "\"" << std::endl;
std::cout << "world:" << "\"" << world << "\"" << std::endl;
fclose(file);
}
}
Code:
read then write
hello the world
hello:"hello"
the: "the"
world:"world"
read and write
hello
hello:""
the: "hello"
world:""
Writing to a file works. Reading to a file works. However, reading from a rw file seems to have problems (the write is correct though).
To be honest, I have close to no experience with FILE manipulation, and I hate manipulating stream internals with tellg or seekg etc, so I don't fully understand what is going on (I get the gist of it, but the actual code is a bit obscure).
Is there something missing from the streambuf (the author says it is incomplete I believe), or am I doing something wrong?
Re: Mixing istream and FILE*
Cool, I'll definitely keep that link handy. I'm thinking at this point, though, it might be easier to just write operators for streams. After all, C structs are all public anyway, so it's not like there would be access problems....
Re: Mixing istream and FILE*
Quote:
Originally Posted by
Lindley
Cool, I'll definitely keep that link handy. I'm thinking at this point, though, it might be easier to just write operators for streams. After all, C structs are all public anyway, so it's not like there would be access problems....
Bah, it isn't that complicated, you should at least give it a try. If you open your file in either read or write (not both, don't know why), it should work fine.
Even if in the end your objects are only structs, you'll still end up writing a bunch stream operators, as opposed to a single syncfilestream class.
PS: Just in case you didn't notice, the code on that link's page 2 works. It doesn't require more than a copy paste.
Re: Mixing istream and FILE*
Oh, and I'm sure I'll grab it---could be useful. However, a virtual function call for every single character strikes me as serious overkill nonetheless.
Re: Mixing istream and FILE*
Incidentally, does the read/write mode work if you make a sync() call between them? I know normally you're supposed to either use fseek() (which is not supported by that implementation, that's the incomplete part) or fflush() between reads and writes.
Re: Mixing istream and FILE*
Quote:
Originally Posted by
Lindley
Incidentally, does the read/write mode work if you make a sync() call between them? I know normally you're supposed to either use fseek() (which is not supported by that implementation, that's the incomplete part) or fflush() between reads and writes.
I have no idea. To be honest, this problem is a bit over my knowledge, and everything I tried here was on a "let's see what happens if I write this" basis. According to cplusplus, sync does nothing by default. In this case, I think there might be a use for it in mixed read/write, but I would not know how to use it. Further more, the provided implementation is not complete. It does basic read/write , but it lacks all seek and get functions. They are probably not too hard to implement though.
Quote:
Originally Posted by
Lindley
Oh, and I'm sure I'll grab it---could be useful. However, a virtual function call for every single character strikes me as serious overkill nonetheless.
Well technically, when using a normal fstream or a stringstream, you are calling streambuf's and stringbuf's functions via virtual call too, so syncstream's overhead should be any more than standard streams. Unless they have special optimizations.
...
Sill, I say interface comes before implementation. syncstream seems like the correct solution to me. If after trying syncstream, you notice you have an io bottleneck, then I guess it could be time to look for another solution.
Re: Mixing istream and FILE*
Quote:
Originally Posted by
monarch_dodra
Well technically, when using a normal fstream or a stringstream, you are calling streambuf's and stringbuf's functions via virtual call too, so syncstream's overhead should be any more than standard streams. Unless they have special optimizations.
streambuf and stringbuf maintain a buffer, so the virtual functions like overflow() only need to be called whenever the buffer is full. If the buffer size is, say, 512 bytes, this means overflow() only has to be called after every 512th character as opposed to every character. That can make a big difference.
Of course, if this is the only problem, nothing stops you from implementing a buffer for syncbuf too.
