dcsimg
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 5 of 5
  1. #1
    Join Date
    Nov 2010
    Posts
    138

    Using ifstream, string stream, and extraction operator >>

    Let's say I have a file that is entirely strings but it's binary due to some nulls ('\0') in between the strings. So technically, it's not an ascii file (32 - 128), it's got some 0's in it. So using, cat, it would be cat -A thefile.txt to see that there are nulls in it. cat thefile.txt would just show the pure ascii text.

    I can create an ifstream object and read it in line by line using either of the following:

    #include (the proper include files, sorry I've omitted them here)

    using namespace std;
    string buffer;
    ifstream if;

    getline(if, buffer, '\n'); or getline(if, buffer);

    or:

    if >> buffer;

    The above 3 different methods will ALL read an entire line from the file. I can easily
    prove this by cout << "buffer = " << buffer; and it will display the line from the file,
    including any nulls or whatever.

    But now suppose, I want to use extraction (as in I want to tokenize the lines into parsed strings, right into string variables).

    My problem is that stringstream does NOT behave in this manner unless you have a true mixture of binary globs of data. As soon as you stringstream extract into a string variable, it reverts right back to the behavior of the getline and grabs
    the entire line.

    Is there anyway I can use extraction and have it understand that I want ONE string
    at a time? For example, by setting a delimiter?

    I know I can set a delimiter in the getline function. It's just that I want to do it with the extraction operator (ie. modify stringstream and/or ifstream's extraction).

    Or is there yet another class that I'm missing? I do not want to use boost in this project. And I do not want only Windows-specific classes - I want something generic.
    If I really do have to "roll my own", then fine, I have to do that.

    In my case, if I DO have to roll a new extraction or overload the behavior, it will need to be done for all types, not just strings - I would guess.

    Thanks.
    Last edited by WhatNow46; February 21st, 2018 at 10:59 AM.

  2. #2
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    6,275

    Re: Using ifstream, string stream, and extraction operator >>

    What compiler are you using?

    Code:
    if >> buffer;
    Extracts from the stream ignoring any initial white space chars until a white space char is found. Thus if the file contains

    qwe rty

    then the above would only extract qwe to buffer.

    If you want to change what are the white space chars used by >> then AFAIK this can't be done.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.7.4)

  3. #3
    Join Date
    Nov 2010
    Posts
    138

    Re: Using ifstream, string stream, and extraction operator >>

    I'm using GNU, currently without c11 extensions. I was hoping that stringstream was different than ifstream as
    far as tokenizing. I want a tokenizer type of stream operator.

    The text in the file looks more like this:

    The cat^@dog^@ which if just outputting with cat would look like this:

    The catdog

    The ^@ is how text-file dump utilities display nulls ('\0').

    So it is a text file with nulls in-between the strings and it can have ALL the spaces it wants in each string.

    I would think I can change the behavior that it simply reads in an entire line. I just wondered if it's been done already.
    If not, I need to write my own extractor. I might just need to create a class that has that behavior. I will start working on one later today if noone informs me of an otherwise solution.
    Last edited by WhatNow46; February 21st, 2018 at 11:29 AM.

  4. #4
    2kaud's Avatar
    2kaud is offline Super Moderator Power Poster
    Join Date
    Dec 2012
    Location
    England
    Posts
    6,275

    Re: Using ifstream, string stream, and extraction operator >>

    Basically you want to extract each element separated by a null? Not using >>. It treats \0 as being part of a string and not a white space. So in the above example if>> buffer would set buffer to cat\0dog\0. If you want to extract each element as delimited by \0 then you'll need to write your own. There may well be one already written somewhere. Have you tried searching the internet?

    PS a CSV parser may be what you're after. I've just tried cat\0dog\0 with my basic csv parser with delimiter set to \0 and it extracted each element fine.
    All advice is offered in good faith only. All my code is tested (unless stated explicitly otherwise) with the latest version of Microsoft Visual Studio (using the supported features of the latest standard) and is offered as examples only - not as production quality. I cannot offer advice regarding any other c/c++ compiler/IDE or incompatibilities with VS. You are ultimately responsible for the effects of your programs and the integrity of the machines they run on. Anything I post, code snippets, advice, etc is licensed as Public Domain https://creativecommons.org/publicdomain/zero/1.0/ and can be used without reference or acknowledgement. Also note that I only provide advice and guidance via the forums - and not via private messages!

    C++17 Compiler: Microsoft VS2017 (15.7.4)

  5. #5
    Join Date
    Nov 2010
    Posts
    138

    Re: Using ifstream, string stream, and extraction operator >>

    Thanks for the info - fixed me right up. I just found a snippet of C++ code that does the job nicely. Now I can just
    hand a buffer of '\0' delimited strings, and it fills up a vector with each string in the line. I had wanted to use an operator to do this but I guess I need to somehow start with this for now. Otherwise, I'm too worrisome over such a detail as whether I call extraction or make a function call. At least I have the information read in now.

    The below code was found at this URL: http://www.zedwood.com/article/cpp-csv-parser

    Declarations:
    std::vector<std::string> csv_read_row(std::istream &in, char delimiter);
    std::vector<std::string> csv_read_row(std::string &in, char delimiter);

    Implementations:
    std::vector<std::string> csv_read_row(std::string &line, char delimiter)
    {
    std::stringstream ss(line);
    return csv_read_row(ss, delimiter);
    }

    std::vector<std::string> csv_read_row(std::istream &in, char delimiter)
    {
    std::stringstream ss;
    bool inquotes = false;
    std::vector<std::string> row;//relying on RVO
    while(in.good())
    {
    char c = in.get();
    if (!inquotes && c=='"') //beginquotechar
    {
    inquotes=true;
    }
    else if (inquotes && c=='"') //quotechar
    {
    if ( in.peek() == '"')//2 consecutive quotes resolve to 1
    {
    ss << (char)in.get();
    }
    else //endquotechar
    {
    inquotes=false;
    }
    }
    else if (!inquotes && c==delimiter) //end of field
    {
    row.push_back( ss.str() );
    ss.str("");
    }
    else if (!inquotes && (c=='\r' || c=='\n') )
    {
    if(in.peek()=='\n') { in.get(); }
    row.push_back( ss.str() );
    return row;
    }
    else
    {
    ss << c;
    }
    }
    }

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width




On-Demand Webinars (sponsored)


×
We have made updates to our Privacy Policy to reflect the implementation of the General Data Protection Regulation.