Build an intelligent CSV parser?
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 3 of 3

Thread: Build an intelligent CSV parser?

  1. #1
    Join Date
    Apr 2010
    Posts
    19

    Build an intelligent CSV parser?

    Hi everyone,

    I'm a little new to C++, so I hope I'm posing a relatively simplier problem here... Essentially, I am writing a program which will input a CSV file with multiple lines of data, 30 data values per line. I want to extract each individual data value and store it in a smaller string for analysis later. The format of the file is:

    ==============================================
    Value01,Value02,Value03, ... Value29,Value30
    Value01,Value02,Value03, ... Value29,Value30
    Value01,Value02,Value03, ... Value29,Value30
    ...etc...
    ==============================================

    I want to extract and store Value01 in String01, Value02 in String02, and so on.

    I've figured out the mechanics of how to do this statically, i.e. manually create String01, String02, String03, etc. Each line in the CSV has 30 values, but tomorrow it may swell to 40 or 50 or 100 values per line. I'd like to do this "extract every value into a dedicated string" thing a little more intelligently. It would be great to do something like this:

    ==============================================
    void CVS_Parser(int NumOfValues)
    {
    // Each row in MyFile has NumOfValues data values in it.
    for(int i=0; i<NumOfValues; i++)
    {
    // Create string Value_${i}
    }

    // open MyFile, a CSV
    while(not at CSF.eof)
    {
    // extract one line from the CSV file
    for( int j=0; j<NumOfValues; j++ )
    {
    // Extract the j_th value from the current line
    // Store that value in String_${j}
    }
    // Do stuff with all these data values
    // Clear all strings for the next loop
    }
    }
    ==============================================

    Here's the actual code I already have: (written to collect only four values, but you get the idea)

    ==============================================
    {
    int size=5000;
    char * MasterString, * x, * y;
    char * String01, * String02, * String03, * String04;
    MasterString = (char*) malloc (sizeof(char) * 5000);
    String01 = (char*) malloc (sizeof(char) * 30);
    String02 = (char*) malloc (sizeof(char) * 30); // Some way to do this smarter???
    String03 = (char*) malloc (sizeof(char) * 30);
    String04 = (char*) malloc (sizeof(char) * 30);

    fstream f("MyFile", ios::in | ios::binary);

    while (!f.eof())
    {
    // Clear all strings...
    memset(MasterString, 0, 5000);
    memset(String01, 0, 30);
    memset(String02, 0, 30); // Again - is there a smarter way???
    memset(String03, 0, 30);
    memset(String04, 0, 30);

    // Extract one line from the CSV file...
    f.getline(MasterString, size);

    // Reset pointers x and y to the beginning of MasterString:
    x = y = MasterString;

    // Extract first value
    y = strcasestr(MasterString, ",");
    String01 = strncpy(String01, MasterString, y-MasterString);
    cout<<String01<<" -- ";

    // Extract second value
    y = y+1; x = y;
    y = strcasestr(x, ",");
    String02 = strncpy(String02, x, y-x);
    cout<<String02<<" -- ";

    // Extract third value
    y = y+1; x = y;
    y = strcasestr(x, ","); // Code for extracting Value02 is exactly the same
    String03 = strncpy(String03, x, y-x); // as code for extracting Value03
    cout<<String03<<" -- ";

    // Extract LAST value
    y = y+1; x = y;
    y = strrchr( y, '\0' );
    String04 = strncpy(String04, x, y-x);
    cout<<String04<<"\n";

    } // end of "while (!f.eof())" loop...!

    cout<<"Done!\n";
    }

    ==============================================

  2. #2
    Join Date
    Jun 2009
    Location
    France
    Posts
    2,297

    Re: Build an intelligent CSV parser?

    Oh god, my eyes!

    I'm just kidding. But honestly, that program is 99% C and 1% C++. Forget everything you know about arrays, malloc, memset etc.

    You need to learn about std::string. With this little baby, no need to manage memory, to manage size etc.

    For all your input ouput needs, I'm sure this article should give you great insight on how to do it the smart way.

  3. #3
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,888

    Re: Build an intelligent CSV parser?

    My quick solution:
    Code:
    void readCSV(const string &filename, vector< vector<string> > &filecontents)
    {
        ifstream in(filename.c_str());
        string line;
        vector<string> valrow;
    
        filecontents.clear();
    
        while (getline(in, line))
        {
            istringstream linestream(line);
            string value;
            valrow.clear();
            while (getline(linestream, value, ','))
                valrow.push_back(value);
            file_contents.push_back(valrow);
        }
    }
    I'll leave it as an exercise to figure out precisely what this is doing and how. I'm also assuming that you don't need to worry about quotation marks or values containing commas; if you do, things get a bit more complicated.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center