Best way to debug a "Runtime Error - abnormal termination" that takes HOURS to crash?

[pbrama]

I have an application that all and all works well. I made some simple code changes (moved some lines out of a loop basically) and now the program has decided to crash. Obviously it has "something" to do with moving those lines and because there are not alot, it "should" be relatively easy to figure out what is happening... it is not.

The main problem is that the program runs typically for 6-12 hours doing some background stuff BEFORE it crashes with a: RUNTIME ERROR - ABNORMAL PROGRAM TERMINATION. I try to make some code changes to correct this, and have to wait, ANOTHER 6-12 hours to see if it is fixed. After over a week of playing this game, it is getting very very old and now I am down to the fact that it has to be something ELSE caused by these lines being moved... a variable overflow, a loop issue, a memory issue... something.

Because this doesn't happen in the IDE where I can watch all these variables and such and even see where it crashes.... I am now at a loss for the best way to debug this OUTSIDE of the IDE. Fortunately I have never run into this where the app crashes, on it's own, hours later so this is totally new to me.

Any all suggestions, tips, recommendations would be greatly appreciated.

Thanks....

[Paul McKenzie]

I have an application that all and all works well. I made some simple code changes (moved some lines out of a loop basically) and now the program has decided to crash.

That is a tell-tale sign that the application was not "well" to begin with. You were just lucky that the application didn't crash on your before. All you did when you added, moved, or removed lines was to move the bug to another part of the code, and this time, that bug became finally exposed.

The main problem is that the program runs typically for 6-12 hours doing some background stuff BEFORE it crashes with a: RUNTIME ERROR - ABNORMAL PROGRAM TERMINATION. I try to make some code changes to correct this,

You should never make coding changes until you know what the issue is. All you're doing when you make coding changes at random is again, move the bug somewhere else, and that somewhere else may now mask the bug so you cannot duplicate the problem again. Leave the code alone until you know what causes the problem.

Does your program do any logging? A log of what is being done can sometimes reveal where the problem lies. Also, if you started the program from outside the IDE, and then from the IDE attach to the running process, does the application crash? Another option is to spend the time to write a crash dump module so that when a crash occurs, you have information on what has happened.

Last, are you running a release build or a debug build outside the IDE?

The bottom line is that when your application crashes, you should have been prepared to get as much information as possible, regardless of how long it takes to crash. An app that just says "runtime error abnormal termination" and you're just stuck with no further information indicates that the app is lacking proper diagnostics, whether it is logging, crash dump generation, etc.

Regards,

Paul McKenzie

[pbrama]

Creating a log file would be my next step. Was hoping there might be some "unknown to me" tool or compile switch that might give me some debug information I might be able to use to narrow down where this is crashing.... again, takes HOURS to fail and ALOT happens in those hours.

Can't get program to run exe then run from the IDE ('attach' as you mentioned)... actually not sure how to do that... never had too, so....

Thanks

Pete

[Igor Vartanov]

I would say that runtime error happens due to memory corruption most of the times, which in its turn may happen due to various reasons. Buffer overflows, wild pointers, calling convention violation, etc. Logging could help find the reason as well as just misguide you. So, first thing to start with should be catching crash dump and try to see what "runtime error" really means, and what happens to your thread stacks.

[Paul McKenzie]

Can't get program to run exe then run from the IDE ('attach' as you mentioned)... actually not sure how to do that... never had too, so....

1) Load your project in Visual Studio
2) Go to the Debugging options in your project. Choose "Attach".
3) Start your application from outside the IDE (pretend you don't have Visual Studio installed).
4) Then in Visual Studio, start debugging. You will be attached to the running process.

OR

1) Load your project in Visual Studio
2) Start your application from outside the IDE
3) Go to the "Debug" menu and choose "Attach to process".
4) A list of running processes are presented to you and you attach to your program.

The latter is how you debug presently running processes, even if you didn't start the process yourself. If your program crashes, you will be presented in the Visual Studio debugger with the current state of the program that crashed. From there, you gather as much information as possible (call stack, values of variables, etc.).

Last, unless your application is multithreaded, in this day and age of C++, memory corruption bugs should be rare or basically eliminated if the program uses the proper constructs. Things such as using container classes instead of using raw new[]/delete[] calls, using string classes instead of C-style string manipulations, smart pointers instead of raw pointers, etc.

Also, a thing to look for is

1) the lack of checking of return codes from functions (if the function fails, does the code check for the failure?).

2) Improper C++ coding constructs that just happen to compile "OK" (things such as using malloc() to create non-POD C++ objects, using memset(), ZeroMemory() or similar C-style functions to wipe out non-POD C++ objects, applying C-style casts to "shut up" compiler errors, etc.)

Regards,

Paul McKenzie