Why my application is so LAZY? Please help me out!!!!

Hello everybody :wave: ,
my application is a dialog based.It takes a "Text file" as INPUT and after parsing the input file it gives a "Text file" as OUTPUT.
But to generate the "text file" (I mean the OUTPUT file) it taking around
3.30 hours!!! :eek:
I dont know why it is taking that much time. :ehh:

And moreover this application is written by some other programmer, so now I'm scaring to debug the application for finding the reason for the delay in generating the output file.

Anybody come across with this type of problem?
Any clues?
Thanks.

Without seeing your code, we have no idea :D

Quote:

---

Orginally posted by szz_zoli
Without seeing your code, we have no idea :D

---

I'm sorry zoli, due to some reasons I cant share the code.

Any wild guesses?
Thanks....

Can you tell us, what is your program supposed to do?
How big is the input/output file?
What algorithm(s) were used?

How big is you input text file?

How BIIIIIIIIG is your input file?

Quote:

---

Orginally posted by szz_zoli

Can you tell us, what is your program supposed to do?
How big is the input/output file?
What algorithm(s) were used?

---

Input is of size around some 30 KB and output file will be around 4 MB .
No special algorithms were used.

Actually the input file contains 2 digits to 3 digits numbers, using these values my application performs some multiplications, substractions, divisions etc.
Nothing more than that.

I'm sure.

Thanks...

I think the operations take so long time. Maybe you have WHILE or FOR structures which takes a long time to be completed.

Can you write the steps of the operations? Whitout the code is hard to understand why. :ehh:

To read from a file and write into another file shouldn't take more than 5 seconds(depends of PC).

Quote:

---

Originally Posted by shinde

Input is of size around some 30 KB and output file will be around 4 MB .
No special algorithms were used.

Actually the input file contains 2 digits to 3 digits numbers, using these values my application performs some multiplications, substractions, divisions etc.
Nothing more than that.

I'm sure.

Thanks...

---

Hey...

Are u doing a lot of string manipulations??? i mean a lot????
using CString and assigining, appending, ect in a loop of some sort????

Quote:

---

Orginally posted by bujcat
Can you write the steps of the operations?

---

Ok bujcat, I will try my best. Please give me some time.

Thanks...

Since u are playing with numbers, check out with just one entry in input file and see the performance.

First suggestion: profile your code. See if there are any easy to identify bottlenecks

Second suggestion: Give us something to work with. If you can't share code, can you share the psuedo code? Flow chart? We can only guess what might be wrong if you can't share what's happening.

Third suggestion: Tell us what your program does. What exactly are you trying to process? Where is the data coming from? Where are you writing the output to? How much data are you analyzing? What do you have to do to the text?

Fourth suggestion: Check for common string manipulation performance killers:

* unnecessary copying of string data. See if you can return references to strings instead

* inserting / removal of text in the midlle of a very large string. If this operation occurs often, it's usually quicker to read data as a vector of smaller strings -- for example a text file can often be read as a vector of text lines.

Fifth suggestion: Hire a contractor. If you don't want to debug it yourself, there are a lot of people out there willing to do it if they get paid.

Quote:

---

Originally Posted by Pink Panther
Since u are playing with numbers, check out with just one entry in input file and see the performance.

---

Gooood Idea!
Ok, I will try now.

bujcat, dont mind, I'm delaying your request for some time.
I will try as Pink Panther mentioned.

Meanwhile please give your valuable suggestions...
Thanks.

Profiling will help.

Use Function Profiling to find:

1. Number of times a Function is executed.
2. Time spent in each Function.

This FAQ is relevant too.

Quote:

---

Orginally posted by Kevin Hall
First suggestion: profile your code. See if there are any easy to identify bottlenecks

---

Ok.

Quote:

---

Orginally posted by Kevin Hall
Second suggestion: Give us something to work with. If you can't share code, can you share the psuedo code? Flow chart? We can only guess what might be wrong if you can't share what's happening.

---

Hummmmm...

Quote:

---

Orginally posted by Kevin Hall
Third suggestion: Tell us what your program does. What exactly are you trying to process? Where is the data coming from? Where are you writing the output to? How much data are you analyzing? What do you have to do to the text?

---

Please Kindly read my previous replies.Thank you.

Quote:

---

Orginally posted by Kevin Hall
Fourth suggestion: Check for common string manipulation performance killers:

* unnecessary copying of string data. See if you can return references to strings instead

* inserting / removal of text in the midlle of a very large string. If this operation occurs often, it's usually quicker to read data as a vector of smaller strings -- for example a text file can often be read as a vector of text lines.

---

Yes, you are right, I will look at it.

Quote:

---

Orginally posted by Kevin Hall
Fifth suggestion: Hire a contractor. If you don't want to debug it yourself, there are a lot of people out there willing to do it if they get paid.

---

Hahaha, hey friend, actually it is written by some other programmer, otherwise I would have tried to find the problem.

I can debug, I believe it takes long time.
for example, this application calls a funtions for checking spaces in the input file for more than 1 million (100000) times!!!
And lots of while and for loops are there.

Now I'm trying here for any luck (I mean any help from you gurus...)
If no luck then I have to sit and debug...
That's really very scaring me......

Thanks...

Quote:

---

Orginally posted by Siddhartha
Profiling will help.

Use Function Profiling to find:

1. Number of times a Function is executed.
2. Time spent in each Function.

This FAQ is relevant too.

---

Hai Siddhartha,
Ok, I will do profiling and reply you soon....
Thank You.

Hi,

my suggestion is, try to implement logging for your application with time.

From that you can find out where it is taking long time.


Regards
MJValan

Quote:

---

Orginally posted by mjvalan
my suggestion is, try to implement logging for your application with time.
From that you can find out where it is taking long time.

---

Hai mjvalan, logging for application with time? how to do that? please can you explain me? I'm little confused.

Thanks.

You may ve seen log files in your system, which has information about execution of some applications in your machine.

In your program write some extra lines to create a file and write some messages with the system time.

Ex

main()

{
CreateFile("c:\xyz.log",...);
GetsystemeTime() // I am not writing exact funtions.

writeFile(Time+" Entering into main");
....
....
....
GetsystemTime()
WriteFile(Time+"calling fun blabla");
blabla();
GetsystemTime()
WriteFile(Time+" fun blabla completed ");


....
...
GetsystemTime();
writeFile(Time+"Exiting function main");



}


==============================

Accroding to your requiremnet you can increase log statements in your program.

Now by veryfing the log file you can find out that, howmuch time function blabla has taken.

If you feel blabla is taking more time , then add some log statements in function blabla. So that you can find out the exact statement where it is taking more time.


(Most of the programs will take more time in for/ while loops)

Thanks a lot mjvalan for giving the hint.
I will also try this way.

Thanks to all.....

BTW, FYI, in the FAQ link mentioned before, Andreas has posted a Timer Class. You may use that too.

Quote:

---

Originally Posted by shinde

Ok.


I can debug, I believe it takes long time.
for example, this application calls a funtions for checking spaces in the input file for more than 1 million (100000) times!!!
And lots of while and for loops are there.

Now I'm trying here for any luck (I mean any help from you gurus...)
If no luck then I have to sit and debug...
That's really very scaring me......

Thanks...

---

Sounds like you have your answer.

Quote:

---

Orginally Posted by Siddhartha
BTW, FYI, in the FAQ link mentioned before, Andreas has posted a Timer Class. You may use that too.

---

Ok, sure, thank you Siddhartha.

Quote:

---

Orginally Posted by GCDEF
Sounds like you have your answer.

---

Hai GCDEF, sorry, I did not get you!!!
What do you mean? please explain....

I'm leaving for today. Tommarrow I hope, after following all your suggestions, I may(will) come up with the good result, and reply you all about the status.

But GCDEF, please reply me.Thanks.

Thanks to all.

Quote:

---

Originally Posted by shinde

Ok, sure, thank you Siddhartha.

Hai GCDEF, sorry, I did not get you!!!
What do you mean? please explain....

I'm leaving for today. Tommarrow I hope, after following all your suggestions, I may(will) come up with the good result, and reply you all about the status.

But GCDEF, please reply me.Thanks.

Thanks to all.

---

Look at what I quoted. You said it's calling the same function to check for spaces over a million times. Why?

I have to respond, please take this kindly:

Quote:

---

Originally Posted by shinde

I'm sorry zoli, due to some reasons I cant share the code.

Any wild guesses?
Thanks....

---

We are not fortune tellers or psychic. No one can determine what makes your program slow without seeing what you're doing.

Quote:

---

And moreover this application is written by some other programmer, so now I'm scaring to debug the application for finding the reason for the delay in generating the output file

---

But isn't that what programmers are supposed to do at some point in their programming lives, and that is to debug other people's code? If you're scared to debug it, how in the world are you going to fix it?

Regards,

Paul McKenzie

I think you maybe using MFC's CArray class with structures having static members? This can dramatically slow down your application. This because CArray is have this problem. If you are using this class, I strongly recommend eleminating this using other options.

He also may be calling malloc() and free() a million times.

The problem is that the poster has yet to reveal *any* code to us, so guessing what is wrong won't be productive.

Regards,

Paul McKenzie

I have parsed a text file of about 25 mb and stored the data into database table.
Its taking just 37 minutes to do that on a PIV,256 MB Machine.
VC++ is really too fast in parsing files ur extraction and calculation routine has got the problem.

I am sure, try using interger datatypes in places of floats where not required.

Regards
PS

:blush: :blush:
I'm extreamly sorry for giving you trouble. It's my mistake, I did not checked the code properly.
I found the problem.

sleep() function is called in my application :blush: :blush: !!!

If possible forgive me.

Thanks to all......

With Regards........

Quote:

---

Originally Posted by shinde

:blush: :blush:
I'm extreamly sorry for giving you trouble. It's my mistake, I did not checked the code properly.
I found the problem.

sleep() function is called in my application :blush: :blush: !!!

If possible forgive me.

Thanks to all......

With Regards........

---

Ha ha ha

It looks like ur code is not the only thing sleepin ;)

Happens to everyone sometimes

Hey one more thing...
Be careful when u do stuff like....

Quote:

---

this application calls a funtions for checking spaces in the input file for more than 1 million (100000) times!!!
And lots of while and for loops are there.

---

Searchin inside a string, putting it in a loop, and using CStrings will KILL ur application on performance. Eliminate the option of using CStrings completely on such occassions. Use char, wchar_t on such occassions.

Quote:

---

Originally Posted by shinde

Please Kindly read my previous replies.Thank you.

---

Yeah, you're right. I just replied to the first question. I didn't read all the way down. My mistake.

Quote:

---

Originally Posted by shinde

If no luck then I have to sit and debug...
That's really very scaring me......

---

I know you've solved your problem now, but let me be honest with you for a second. You need to learn to debug -- and to not be afraid of it. It's a part of any (successful) programmer's life! It's never fun to do under pressure, but that comes with the territory. Anyway, I strongly urge you to either buy a book about debugging and/or scour the web for such information. Also, look around for books/web content on defensive programming skills: static assertions (compile-time asserts), smart asserts, tracing, discussions about coupling, cohesion, and testability, etc.... Another book I'd recommend reading is "The Pragmatic Programmer" -- it talks about defensive programming, but not in the ways I mentioneed above -- it alone is not a complete guide to defensive programming. Definitely read the book, but do look for other defensive programming books too. These things will make you a much more valuable programmer to any buisness -- your own or to some company.

Quote:

---

Originally Posted by shinde

sleep() function is called in my application!!!

---

LOL! Things such as this happen.

Quote:

---

Originally Posted by shinde

If possible forgive me.

Thanks to all......

---

Don't worry about it. No problem! :)

Quote:

---

Originally Posted by Pink Panther

Searchin inside a string, putting it in a loop, and using CStrings will KILL ur application on performance. Eliminate the option of using CStrings completely on such occassions. Use char, wchar_t on such occassions.

---

I strongly disagree. Actually, due to CString's reference counting mechanism, I've seen code which was actually faster with CString as compared to the use of char arrays (where string memory was constantly reallocated and copied). Also, any string operation which needs to check the length of the string is faster with CString as when searching the NULL terminator in a char array.

Quote:

---

Originally Posted by gstercken

I strongly disagree. Actually, due to CString's reference counting mechanism, I've seen code which was actually faster with CString as compared to the use of char arrays (where string memory was constantly reallocated and copied). Also, any string operation which needs to check the length of the string is faster with CString as when searching the NULL terminator in a char array.

---

Yup. :thumb::thumb::thumb:

I have seen code that modifies strings byte by byte, and character by character.

Such code is difficult to understand, and tough to maintain.

The usage of String classes like CString, or std:string not only makes the code a lot shorter (and reduces time to delivery), but, also makes it easy to explain, understand and maintain functionality.

When used correctly, these classes are as fast, or faster.