Compilation Error

[kambiz_j]

I'm trying to implement a string tokenizer using the strtok() function but I keep get the following error;

error C2440: 'initializing' : cannot convert from 'const char *' to 'char *'

when trying to compile the following code;


string myString = "string to be tokenized";
char *sp;

sp = strtok(myString.c_str(), " ");
while (sp)
{
cout << sp << endl;
sp = strtok(NULL, "");
}



Also I need to keep the input as a string object as that is how it's used throughout my application. Any ideas on how to resolve this.

Cheers

Kam

[Andreas Masur]

'strtok' expects a 'char *' for its first argument not a 'const char *'. There it gives you the compiling error. You could get it to work by simply remove the 'constness'...

string myString = "string to be tokenized";
char *sp;

sp = strtok(const_cast<char *>(myString.c_str()), " ");
while (sp)
{
cout << sp << endl;
sp = strtok(NULL, "");
}



BUT this way is dangerous. If you cast away the constness of a STL string you can modify it directly through the pointer. This will most-likely cause undefined behaviour since the way the STL string controls its content can differ from a normal character array.

Therefore you should use a standard character array temporarily...

string myString = "string to be tokenized";
char szTempArray[myString.length() + 1] = "";
char *sp;

// Copy string
strcpy(szTempArray, myString.c_str());

sp = strtok(szTempArray, " ");
while (sp)
{
cout << sp << endl;
sp = strtok(NULL, "");
}




Ciao, Andreas

"Software is like sex, it's better when it's free." - Linus Torvalds

[Graham]

Don't use strtok. Do it the C++ way:

string myString = "string to be tokenised";
string::size_type pos = 0;
while (pos != string::npos)
{
string::size_type loc = myString.find_first_of(" ", pos);
cout << myString.substr(pos, loc - pos);
pos = loc;
}



That's got a couple of errors in it (not checking for npos in the loop, not skipping over a sequence of separators) - but the basic idea's there: use facilities provided by STL to do the job - don't rely on unsafe C functions. In fact, it ought to be possible to write a functor that duplicates strtok in a totally type-and-const-safe manner without relying on statics within a function.

He who breaks a thing to find out what it is, has left the path of wisdom - Gandalf

[cpitis]

You should NEVER do that! The strtok modifies the buffer it works on! If you work with std::string, use the STL algorithms, like find_first, etc.

[NMTop40]

except of course you can't do this:

char szTempArray[myString.length() + 1] = "";



but would have to use new, thus:

char * szTempArray = new char[myString.length() + 1 ];



If you don't want to bother with having to delete it later you can use:

auto_ptr<char> szTempArray( new char[myString.length() + 1 ] );




If you want to avoid using new (although it is provided with C++ for a reason) you can use std::vector<char>, and use &v[0] to work. As with an array allocated with new, you need to make the size of the array one more than the size of the string.

Having said that, you can probably std::string as well but here you MUST append a NULL character at the end after assigning it, because std::string itself is not guaranteed to do that for you. You can however guarantee that std::string is an array of characters (vector and string are always both arrays).

Thus (if you are going to do it):

string myString = "string to be tokenised";
myString.push_back('\0'); // important
// then
char * sp = &myString[0]; // should work, I think



However, when you have finished tokenising, ensure you use newly created objects. The pointers that you found while tokenising will probably become invalidated, so create new strings for each token as found.


The best things come to those who rate

[cpitis]

You cannot use auto_ptr class with new[], because it deallocates the memory using delete, not delete[]. On most standard C++ compliant compilers this may cause memory leaks.

But, a class like auto_ptr can easily be designed for object array allocations.