When dealing with big text files and large directories of files that you need to search / parse, I was wondering if anyone knows how much faster C++ is than Perl approximately? Is it worth it to develop in C++ over Perl in these cases?
It would depend on the target machine to some degree, but in modern hardware (this side of a single core 1Ghz chip), a C++ version would probably leave the CPU so idle, depending on what you mean by parsing, that the bottleneck is likely the read speed and random access time of the file system.
That said, if the parsing is demanding then that portion of a C++ version would be more efficient and possibly faster. Without more detail as to what you're parsing and doing with the results, I can't be more specific.
If my post was interesting or helpful, perhaps you would consider clicking the 'rate this post' to let me know (middle icon of the group in the upper right of the post).
However, my experience is based on data processing, where the files were read and written using an API (which was written in C). And as a matter of fact, in this scenario the opposite of what JVene posted was true: The lighter the processing the more significant the difference was (due to the fact that the penalty for reading a file via a C API became bigger when the actual processing was lighter). I don't think, the above will hold true though for processing where the file is read/written using the language's native functions.
More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity. --W.A.Wulf
Premature optimization is the root of all evil --Donald E. Knuth