Mixing istream and FILE*
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 1 of 2 12 LastLast
Results 1 to 15 of 17

Thread: Mixing istream and FILE*

  1. #1
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Mixing istream and FILE*

    I'm working with some of my own code which defines operator>>, and some C objects which support read() functions taking a FILE*. I would like to store both types of object in the same file, one after the other.

    At first I thought someone might have cleverly devised an istream-derived class which simply wraps a FILE*. Maybe it would be slower than an ifstream, but it would do what I need. However, after a bit of searching I've found no such thing, surprisingly.

    So my next thought was, I'll just open an ifstream, read the first object, tellg(), close it, open the FILE*, and fseek() to the same location.

    However, I'm not certain whether tellg() is guaranteed to return a result in bytes which would translate directly to the value needed by fseek(). Does anyone know what the story is there?

    Also, any better ideas for how I could manage this?

  2. #2
    Join Date
    Apr 2004
    Location
    Canada
    Posts
    1,342

    Re: Mixing istream and FILE*

    You may find this article useful. It shows how to convert between fstream and FILE* handles, going through C file descriptors as an intermediate step. It might only work on Windows, though, as the article also mentions native Windows file handles.
    Last edited by HighCommander4; May 6th, 2010 at 05:56 PM.
    Old Unix programmers never die, they just mv to /dev/null

  3. #3
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    Yeah, fd() doesn't seem to be a standard fstream method as far as I can tell.

    The other option I found----fpos::get_fpos_t(), which would give me something to pass to fsetpos()----also appears to be Windows-only.

  4. #4
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,292

    Re: Mixing istream and FILE*

    Since your C++ objects can write to streams, you can do that, and then write the stream internals to the FILE:

    Code:
    template <typename T>
    void writeToFileViaStream(FILE* file, const T& object, bool binaryMode = false)
    {
        std::ios_base::openmode openMode = std::stringstream::in |
                                           std::stringstream::out |
                                           (binaryMode ? std::binary : 0;
            
        std::stringstream ioss(openMode)
        ioss << object;
    
        //This part is copied verbatim from cplusplus.com
        //there might be better ways to do this, but I just wanted a quick proof of concept.
        //Watch out for memory leaks!
        {
            // get length of file:
            ioss.seekg (0, ios::end);
            length = is.tellg();
            ioss.seekg (0, ios::beg);
    
            // allocate memory:
            char buffer = new char [length];
    
            // read data as a block:
            ioss.read(buffer,length);
            ioss.close();
    
            fwrite(buffer, 1, length, file);
    
            delete[] buffer;
        }
    }
    This has the advantage of being pretty straight forward interface wise. It is just like fwrite, but can be used on C++ objects that only know operator>>.

    Heck, given the signature, I'm sure we could just call this function "fwrite" and it would work. I would not recommend doing that though :/

    The disadvantages include 2 extra writes: 1 to the stream, and another to a temporary buffer. I'm sure we can avoid the second copy, but I'm not concentrating on performance here.

    IMO, the BIG disadvantage of this method is that it wraps C++ functionality into a C interface. I would really have liked to do the opposite, but you just can't do much with a FILE, apart from writing to a file... You could try toying with the FILE buffer, but that is so dangerous I shudder at the very thought.

    (This of course also works for fread)

    Is this a satisfactory solution to your problem?
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  5. #5
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    I'm sure that will work great for writing....but then, for writing, I could simply write the C++ half, close the file, fopen() it again in append mode, and proceed. Writing isn't hard.

    The difficulty comes when I try to read it again.

    For now, I'm making the assumption that tellg() returns the file position in bytes, although I haven't been able to find any documentation which confirms that. I can pass that in fseek() easily enough; I know *that* operates in bytes. And the only downside of fseek() is that it takes a long argument, which is occasionally not big enough (operating on a 4GB+ file on a Win64 system....long remains a 32-bit type). Probably not going to be an issue in my case though.

  6. #6
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,292

    Re: Mixing istream and FILE*

    Well I was supposing you wanted something a bit more clever than opening/closing the file every time by hand. Although I guess you could do that inside the my "WriteViaStreamMethod", a side effect is that it would probably change your file handle, which the caller might not expect.

    I thought my method could also be used for reading, but I just now realized that there is no way to know in advance how many bytes will have to be read into the buffer, making my method not-usable.

    I just don't see any "elegant" way around this problem. I guess there is no way around this problem but opening closing the file every time.

    tellg() returns the file position in bytes, although I haven't been able to find any documentation which confirms that.
    tellg returns an iterator, so you don't even have the guarantee it returns an amount of bytes either! Just an offset from the begining. If you get the tellg from the beginning of the file, and the current tellg, then std::distance(tellg_first, tellg_current) should give you the amount of elements read so far by your stream. Since an element is pretty much guaranteed to be a char, I think it is safe* to assume that 1 element = 1 byte.

    *Just be careful of binary non-binary misalignment. If your file is in text mode, then a single element like "\n" could represent 2 bytes. However, ftell should have the same behavior too, so as long as you open both the stream and file in the same mode, you should be safe.

    That, and when opening your file, you can just do

    Code:
    long numbytes = myFile->ftell();
    
    ...
    Close File, open stream
    ...
    streampos current = myFstream.tellg(); //gets an iterator to "begin"
    std::advance(current, numbytes);
    myFstream.seekg(current);
    There, keep us posted on your findings.
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  7. #7
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    Another option would be to read the first object from the ifstream, then read the remainder of the file into a string, then use tmpfile() to generate a temporary file. fwrite() the string into the temporary file, seek back to the beginning, and proceed to read it.

    I'm not sure if the temporary file would be in memory or on disk, though. If it's in memory then the inefficiency essentially amounts to a couple of extra copies. If it's on disk, this is probably not ideal.

  8. #8
    Join Date
    Apr 2004
    Location
    Canada
    Posts
    1,342

    Re: Mixing istream and FILE*

    Quote Originally Posted by Lindley View Post
    At first I thought someone might have cleverly devised an istream-derived class which simply wraps a FILE*. Maybe it would be slower than an ifstream, but it would do what I need. However, after a bit of searching I've found no such thing, surprisingly.
    I'm pretty sure gcc's implementation of fstream does this. Perhaps you could write your own based on that?
    Old Unix programmers never die, they just mv to /dev/null

  9. #9
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,292

    Re: Mixing istream and FILE*

    Quote Originally Posted by Lindley View Post
    Another option would be to read the first object from the ifstream, then read the remainder of the file into a string, then use tmpfile() to generate a temporary file. fwrite() the string into the temporary file, seek back to the beginning, and proceed to read it.

    I'm not sure if the temporary file would be in memory or on disk, though. If it's in memory then the inefficiency essentially amounts to a couple of extra copies. If it's on disk, this is probably not ideal.
    I'm pretty sure tmpfile opens a file on disc. You just have the guarantee (kind of) that the file name will be unique and super complicated.

    Also, If your file is something like 2GB, and you have interleaving C-C++ objects, then welcome to slow ville.

    Here's a crazy idea. Start by reading the entire file into a stream, and then keep both open in parallel. Everytime you read from one, just advance by the same amount in the other. You'd have bit of space overhead, but you wouldn't be opening and closing your file every object read, nor copying all it's content to a temporary file every time.

    This wouldn't work quite as well with write, but as already discussed, there are other alternatives for writing, and you rarelly need to mix read an write operations on the same file.
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  10. #10
    Join Date
    Oct 2008
    Posts
    1,131

    Re: Mixing istream and FILE*

    maybe you'll find this article useful...

  11. #11
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,292

    Re: Mixing istream and FILE*

    Quote Originally Posted by superbonzo View Post
    maybe you'll find this article useful...
    Awesome link.

    Code:
    int main()
    {
        std::cout << "read then write" << std::endl;
        {
            FILE* fileOut = fopen("hello.txt", "w");
            syncbuf buf(fileOut);
            std::ostream out(&buf);
            fputs("hello", fileOut);
            out << " the ";
            fputs("world", fileOut);
            fclose(fileOut);
        }
    
        {
            FILE* fileIn = fopen("hello.txt", "r");
            syncbuf buf(fileIn);
            std::istream in(&buf);
    
            std::string hello;
            char the[256];
            std::string world;
            in >> hello;
            fscanf(fileIn, "&#37;s", the);
            in >> world;
    
            std::cout << hello << " " << the << " " << world << std::endl;
            std::cout << "hello:" << "\"" << hello << "\"" << std::endl;
            std::cout << "the:  " << "\"" << the   << "\"" << std::endl;
            std::cout << "world:" << "\"" << world << "\"" << std::endl;
    
            fclose(fileIn);
        }
    
        std::cout << std::endl;
    
        std::cout << "read and write" << std::endl;
        {
            FILE* file = fopen("hello2.txt", "rw");
            syncbuf buf(file);
            std::iostream io(&buf);
            fputs("hello", file);
            io << " the ";
            fputs("world", file);
    
            std::string hello;
            char the[256];
            std::string world;
            io >> hello;
            fscanf(file, "%s", the);
            io >> world;
    
            std::cout << hello << " " << the << " " << world << std::endl;
            std::cout << "hello:" << "\"" << hello << "\"" << std::endl;
            std::cout << "the:  " << "\"" << the   << "\"" << std::endl;
            std::cout << "world:" << "\"" << world << "\"" << std::endl;
            fclose(file);
        }
    }

    Code:
    read then write
    hello the world
    hello:"hello"
    the:  "the"
    world:"world"
    
    read and write
     hello
    hello:""
    the:  "hello"
    world:""
    Writing to a file works. Reading to a file works. However, reading from a rw file seems to have problems (the write is correct though).

    To be honest, I have close to no experience with FILE manipulation, and I hate manipulating stream internals with tellg or seekg etc, so I don't fully understand what is going on (I get the gist of it, but the actual code is a bit obscure).

    Is there something missing from the streambuf (the author says it is incomplete I believe), or am I doing something wrong?
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  12. #12
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    Cool, I'll definitely keep that link handy. I'm thinking at this point, though, it might be easier to just write operators for streams. After all, C structs are all public anyway, so it's not like there would be access problems....

  13. #13
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,292

    Re: Mixing istream and FILE*

    Quote Originally Posted by Lindley View Post
    Cool, I'll definitely keep that link handy. I'm thinking at this point, though, it might be easier to just write operators for streams. After all, C structs are all public anyway, so it's not like there would be access problems....
    Bah, it isn't that complicated, you should at least give it a try. If you open your file in either read or write (not both, don't know why), it should work fine.

    Even if in the end your objects are only structs, you'll still end up writing a bunch stream operators, as opposed to a single syncfilestream class.

    PS: Just in case you didn't notice, the code on that link's page 2 works. It doesn't require more than a copy paste.
    Is your question related to IO?
    Read this C++ FAQ LITE article at parashift by Marshall Cline. In particular points 1-6.
    It will explain how to correctly deal with IO, how to validate input, and why you shouldn't count on "while(!in.eof())". And it always makes for excellent reading.

  14. #14
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    Oh, and I'm sure I'll grab it---could be useful. However, a virtual function call for every single character strikes me as serious overkill nonetheless.

  15. #15
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Mixing istream and FILE*

    Incidentally, does the read/write mode work if you make a sync() call between them? I know normally you're supposed to either use fseek() (which is not supported by that implementation, that's the incomplete part) or fflush() between reads and writes.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center