string manipulation

[dellthinker]

Hi all, I'm trying to find a function that allows a string to be broken up in pieces. What I mean by that is this ...

char buffer[521]="I want this string broken down into words";

I suppose my question is this, is there a function that differentiates spacing with std::string or does it even matter at all? Thanx.

[Lindley]

One approach would be to construct an istringstream with the data, and then extract each whitespace-separated string from it sequentially.

[VictorN]

There is also MFC AfxExtractSubString function

[laitinen]

..or you could try strtok or strtok_s

[Amleto]

or boost has some extensive string manipulation algorithms.

[Lindley]

..or you could try strtok

strtok is probably the most efficient option if you are working with a C-style string in an editable buffer, and you don't need the original data anymore once the tokenization is complete. Also, there's the same cautions and caveats you always have when dealing with C-style strings.

I tend to think that the efficiency win is mitigated by the safety concerns, but each case must be considered on its merits.

[dellthinker]

I think strtok() will be something I'm willing to try. Thanx for the advice.

[Paul McKenzie]

I think strtok() will be something I'm willing to try. Thanx for the advice.

You also need to know that strtok() uses a static buffer.

This means that you are limited in the ways you use this function. You have to strok "one string at a time". When you are fully done with one string, then you can safely strtok another string.

In other words, this would fail miserably:

char str1[100];
char str2[100];

strtok( str1, ....);
for (int i ... )
{
   for (int j ... )
        strtok( str2 ... )
   //...
   strtok(NULL, ... );  // this is for the str1 string
}

You can't have strtok going on with two strings intermingled like this.

Regards,

Paul McKenzie

[krmed]

You also need to know that strtok() uses a static buffer.

This means that you are limited in the ways you use this function. You have to strok "one string at a time".

Very true, although if the OP has the need to split multiple strings, strtok_s() can handle that. (VS2005 and later).

[Marc G]

If your compiler already supports C++0x Regex library, you can use that.
Something as follows:

#include <iostream>
#include <string>
#include <regex>

using namespace std;

int main(int argc, char** argv)
{
    regex reg("\\s*[,;\\s]+\\s*");
    while (1)
    {
        cout << "Enter a string to split (q=quit): ";
        string str;
        if (!getline(cin, str) || str == "q")
            break;
        const sregex_token_iterator end;
        for (sregex_token_iterator iter(str.begin(), str.end(), reg, -1);
            iter != end; ++iter)
        {
            cout << "\"" << *iter << "\"" << endl;
        }
    }
    return 0;
}

[dellthinker]

I have found a working solution...

#include <iostream>
#include <string>
#include <sstream>
using namespace std;

int main ()
{
  string str,str2,str3,str4;
  size_t pos;
  cout << "Enter 3 words separated by a space character " << endl;
  getline(cin,str);

  str2 = str.substr(0,8);
  str3 = str.substr(9,8);
  pos = str.find_last_of(" ");
  str4 = str.substr(pos);
  
  cout << str2 << endl;
  cout << str3 << endl;
  
  stringstream stream(str4);
  while( getline(stream, str4, ' ') ){cout << str4;}
  return 0;
}

Just in case I forget I can always refer to this post. And for any other people with the same problem. This will take a string with 3 sub strings and save them to 3 separate strings provided you type it like this "Orange Apple Watermelon", and given that the first 2 sub strings are 8 characters in length. Maybe one of the more advanced dev's here can come up with an even better solution.

[Amleto]

err, that's hardly tokenising on space is it? it just splits the input up at hard-coded positions.

#include <iostream>
#include <string>
#include <sstream>
using namespace std;

int main ()
{
  string str;
  cout << "Enter 3 words separated by a space character " << endl;
  getline(cin,str);

  stringstream stream(str);
  
  while( getline(stream, str2, ' ') )
  {
    cout << str2 << endl;
  }
  return 0;
}

and only slightly better is:

#include <iostream>
#include <string>
#include <sstream>

#include <vector>
#include <algorithm>


using namespace std;

vector<string> tokenize(string const & str)
{
  vector<string> out;

  string temp;

  stringstream stream(str);
  while( getline(stream, temp, ' ') )
  {
    out.push_back(temp);
  }

  return out;
}

void print(string const & s)
{
  cout << s << endl;
}

int main ()
{
  string str;
  cout << "Enter 3 words separated by a space character " << endl;
  getline(cin,str);

  vector<string> toks = tokenize(str);
  
  std::for_each(toks.begin(), toks.end(), print);


  return 0;
}