CString -vs- std:string ?

[Axter]

Even in your results, the std::string is about 40% slower the CString.
That's a big difference to me.

And using reserve is not a valid argument for this test. You don't always know the size that you're going to need for your buffer in advance.

[Paul McKenzie]

I made a mistake and thought you were concatenating. All you really tested was the access time to get to a certain element. Therefore you tested one particular trait of CString against std::string.

Most applications (as my previous post suggested) attempt to concatenate, search, substring, etc. a string. Finding the 100th element in a string is not something that would be done repeatedly in a loop.

Regards,

Paul McKenzie

[Axter]

I made a mistake and thought you were concatenating. All you really tested was the access time to get to a certain element. Therefore you tested one particular trait of CString against std::string.

Most applications (as my previous post suggested) attempt to concatenate, search, substring, etc. a string. Finding the 100th element in a string is not something that would be done repeatedly in a loop.

Regards,

Paul McKenzie

That was just one test.
I have peform other test comparing std::string peformance with CString, and on most test CString out peformance std::string.

There are specific senerios in which std::string is more efficient, but on normal average usage, CString is far more efficient.

I like to point out, that I'm only making this comparison with the std::string that comes with VC++.
There are probably third party std::string classes that do a much better job, and probably out peforms CString. But the VC++ std::string class is not one of them.

[Paul McKenzie]

That was just one test.
I have peform other test comparing std::string peformance with CString, and on most test CString out peformance std::string.

There are specific senerios in which std::string is more efficient, but on normal average usage, CString is far more efficient.

I like to point out, that I'm only making this comparison with the std::string that comes with VC++.
There are probably third party std::string classes that do a much better job, and probably out peforms CString. But the VC++ std::string class is not one of them.

Possibly. I heard that PJ Plauger improved on the std::string in VC 7.0, but I'm not sure.

Anyway, I changed your code to do a concatenation and a substr:

#include <string>
#include <afxtempl.h>

int dummy_var = 0;

void function1(const char*Data)
{
    CString strData = Data;
    for ( int i = 0; i < 1000; ++i)
      strData += strData.Mid(0, 500);

}

void function2(const char*Data)
{
    std::string strData = Data;
    for ( int i = 0; i < 1000; ++i)
      strData += strData.substr(0, 500);
}

I did 10 iterations of function1 and function2. The results:

Time duration is as follow:
LenTime1 = 1390
LenTime2 = 1438
LenTime3 = 94
LenTime4 = 203
LenTime5 = 234

Adding reserve:

void function2(const char*Data)
{
    std::string strData = Data;
    strData.reserve(1000000);
    for ( int i = 0; i < 1000; ++i)
      strData += strData.substr(0, 500);
}

The times are now this:

Time duration is as follow:
LenTime1 = 1360
LenTime2 = 15
LenTime3 = 94
LenTime4 = 172
LenTime5 = 234

To your point about reserve():

The reserve() is one of those functions that are usually not coded on the first cut of a program. It's when you find out how the string class is used and when bottlenecks occur, then the change to call reserve() is made. So how much memory to reserve() is not based on any hard numbers up front when you start coding, but on sheer experience when your program goes through the rigors.

Regards,

Paul McKenzie

[Axter]

Possibly. I heard that PJ Plauger improved on the std::string in VC 7.0, but I'm not sure.

I peform test on VC++ 7.1 compiler, and the CString class out peforms the std::string class on that version also.

Anyway, I changed your code to do a concatenation and a substr:

Your results still showed CString out performing std::string.
On my own test, it shows CString has an even better improvement.
I also question whether your test is valid since you removed code I put in there that force the compiler not to optimize away the code.

Adding reserve:
Time duration is as follow:
LenTime1 = 1360
LenTime2 = 15
LenTime3 = 94
LenTime4 = 172
LenTime5 = 234

Good try , but no cigar.
This is not a valid test, since you added the reserve code to the std::string function, but did not do the same logic for the CString function.
If I added the same logic to the CString function, you'll see CString still out peforms std::string.

Test the following valid code test:

int dummy_var = 0;
const int SizeData = 512;

void function1(const char*Data)
{
	CString strData = Data;
	strData.GetBuffer (1000000);
	strData.ReleaseBuffer();
	for ( int i = 0; i < 1000; ++i)
      strData += strData.Mid(0, 500);
	dummy_var += strData[99];
}

void function2(const char*Data)
{
	std::string strData = Data;
	strData.reserve(1000000);
	for ( int i = 0; i < 1000; ++i)
      strData += strData.substr(0, 500);
	dummy_var += strData[99];
}


int _tmain(int argc, TCHAR* argv[], TCHAR* envp[])
{
	int nRetCode = 0;
	
	const QtyTimesTest = 999;
	DWORD StartTime, LenTime1=0, LenTime2=0;
	char DummyData[SizeData+1];
	memset(DummyData, 'x', SizeData);
	DummyData[SizeData] = 0;
	
	{
		StartTime = GetTickCount();
		for (int i = 0;i < QtyTimesTest;++i)
		{
			function1(DummyData);
		}
		LenTime1 += (GetTickCount() - StartTime);
	}
	
	{
		StartTime = GetTickCount();
		for (int i = 0;i < QtyTimesTest;++i)
		{
			function2(DummyData);
		}
		LenTime2 += (GetTickCount() - StartTime);
	}
	
	{
		StartTime = GetTickCount();
		for (int i = 0;i < QtyTimesTest;++i)
		{
			function2(DummyData);
		}
		LenTime2 += (GetTickCount() - StartTime);
	}
	
	{
		StartTime = GetTickCount();
		for (int i = 0;i < QtyTimesTest;++i)
		{
			function1(DummyData);
		}
		LenTime1 += (GetTickCount() - StartTime);
	}
	
#ifdef _DEBUG
	printf("You're not running release version!!!\nTest is only valid in release version.\n");
#else
	printf("Time duration is as follow:\n\
		   CString      = %li\n\
		   std::string = %li\n",
		   LenTime1, LenTime2);
#endif
	if (!dummy_var) printf("dummy %i", dummy_var);
	system("pause");
	
	return nRetCode;
}

The above test code shows the following results:

Time duration is as follow:
CString = 9140
std::string = 39266


This shows the CString out performming std::string by a ratio of 4 to 1.
The CSting class is 400% faster even when using the reserve method.

I also ran the above code on VC++ 7.1, and I got the following results:

Time duration is as follow:
CString = 11032
std::string = 18828


This show that on 7.1 the std::string class has improve peformance compare to VC++ 6.0, but CString still does about 70% better.

[Paul McKenzie]

Your results still showed CString out performing std::string.

By how much? Not 4 to 1 or 10 to 1 as your test shows. They are virtually identical when I run the same tests. Such vast differences in tests should be explained before proclaiming which is faster or slower.

Regards,

Paul McKenzie

[Axter]

By how much? Not 4 to 1 or 10 to 1 as your test shows. They are virtually identical when I run the same tests. Such vast differences in tests should be explained before proclaiming which is faster or slower.

Regards,

Paul McKenzie

I don't put too much wait on your test, since you've removed some of the code that insures the compiler does not optimize away some of the code.
That might be why you're getting different results.

Did you run the last test code I posted? (without removing validation code)

It shows a ratio of 4 to 1. CString is 400% faster on that test.


OK....
I just ran the same exact executable on a Windows 2000 machine an it gave a result similer to what you posted.

Time duration is as follow:
CString = 11843
std::string = 14016


On my Windows XP machine, the CString class seems to be much much faster then the std::string class.
So I guess the result will very from OS to OS, and from one computer to another.

In either case, CString is still doing much better.
So according to above results, if you code using CString instead of std::string your code peformance will be any where from 1000% to 30% faster then std::string.

[Paul McKenzie]

I changed the order of what function is performed first. I did the std::string test before the CString test, and the timing is as follows:

Time duration is as follow:
CString = 6860
std::string = 6809

I did not change any code -- all I did was run your latest example verbatim, and for the latest test above, changed the order of which function was done first (and also changed the output so that the timings don't get mixed up).

And if I did remove code, what would that matter? Tests should hold up no matter what code I tried, so long as both are similar. Also, all of the tests that I've run, except for the code where I added the reserve() for the std::string, is the code you posted as-is.

Regards,

Paul McKenzie

[Axter]

I changed the order of what function is performed first. I did the std::string test before the CString test, and the timing is as follows:

I find that hard to believe. NOT calling you a liar. I just don't think your code is getting compiled right, or something is wrong with your computer/OS.
What OS are you testing on, and does it have the latest updates?
Do you have the latest SDK for your compiler?

Can you email me your executable? (codeguru@axter.com)

I changed the code around so that std::string comes first, and I still get similar results:

Time duration is as follow:
CString = 6780
std::string = 22391

I did not change any code -- all I did was run your latest example verbatim, and for the latest test above, changed the order of which function was done first (and also changed the output so that the timings don't get mixed up).

According to what you posted before, you did modify the code. Unless what you posted is not the same as what you tested.

And if I did remove code, what would that matter? Tests should hold up no matter what code I tried, so long as both are similar.
Paul McKenzie

It does make a difference. The compiler can complete optimize away code that it determines it's not being used.
If you call a function, and nothing gets returns or saved from that function, the compiler can optimize away the function call.

Even if the code is similar, the compiler can optimize away one function, and not the other, or optimize away both functions, in which case the results would be numbers that are similar. (Like the results you posted)

[Paul McKenzie]

You're not being fair to std::string with your ANSI vs UNICODE example. Add this to your project:

#ifdef _UNICODE
typedef tstring std::wstring;
#else
typedef tstring std::string;
#endif

and we're on a more equal footing, since that's what happens with CString, anyway. Then the only difference is between the two is that you have to call the c_str() function on the tstring.

Another advantage is that you can use std::string and std::wstring simultaneously in the same program. I don't think there is a CWideString class that you could use in a non-Unicode program.

Also, std::string is actually the template basic_string<char>, therefore you can change the allocator (if necessary), and basically do anything wrt character streams.

As far as the implicit conversion, my experience is that it causes more problems than it's supposed to solve. Try to figure out why you are getting weird conversions going on "behind your back" at run-time or resolve those "ambiguous function overload" compiler errors. This is for implicit conversions in general, and not really specific to CString.

Regards,

Paul McKenzie

[Paul McKenzie]

I am now working on a Windows 2K machine, with Service Pack 6 installed for VC++ 6.0. Pentium 3, 1.2GHZ processor.

I'm just reporting what I'm getting when running your program. I e-mailed you the executable.

A difference of 4 or 10 to 1 is just too large of a difference. If there was this much difference, it could be easily explained in the code of both CString and std::string (i.e. std::string does this, while CString does that). Nothing strikes me in the code that I've seen in both CString and std::string that would have this difference. I've also never heard of others touting such differences in speed in CString vs. std::string. Minimal differences, yes, but not on the order of 4 or 10 fold.

As to the order of the functions -- maybe the difference is caching, who knows. But I have seen situations where timings were different, depending on which tests are done first.

Regards,

Paul McKenzie

[Axter]

I just ran the executable you emailed, me and I'm getting similar results to what you posted.

Either something is wrong with my compiler or your compiler.
Or your code is not the same as mine.
Can you post or email EXACTLY what you compile (Copy and Paste).

I compiled my code as a Win32 Console application with MFC support.
Is that how you compiled yours?

[Paul McKenzie]

I compiled as a Win32 console program, using MFC in a shared DLL.

The way I set up my project is to have a Win32 console program, with no MFC support, and then just change the project settings to use the MFC shared libraries. Of course, you have to know what to include to get everything working.

Attached is the code that I used. Again, I would be very surprised (even shocked) if a commercial implementation of std::string (any implementation) were 4 to 10 times slower than CString, which is why I had to verify your claims.

Also, I was perusing the internet, and found some discussions on using GetBuffer()/ReleaseBuffer() to do the same as reserve(). According to what I've read, GetBuffer()/ReleaseBuffer() isn't documented to do the same as reserve(), and you're relying on undefined/undocumented behavior.

Regards,

Paul McKenzie

[Paul McKenzie]

I ran the executables you sent me, and here are my results:

For VC 6.0

Time duration is as follow:
CString = 13379
std::string = 13981

For VC 7.1:
Time duration is as follow:
CString = 12638
std::string = 12769

Regards,

Paul McKenzie

[JohnW@Wessex]

Ran the original program on my machine; Athlon 1.4GHz
VC 6.0, Release code, code generation for Pentium.

Time duration is as follow:
LenTime1 = 240 <<<< CString
LenTime2 = 160 <<<< std::string
LenTime3 = 150
LenTime4 = 181
LenTime5 = 220
Press any key to continue . . .