Quote Originally Posted by JohnW@Wessex View Post
Scary! Thankfully bugs and timing glitches in our code only result in a mail item landing in the wrong box. A small chance of a paper cut maybe.
No kiddiny, after the turbo blew (it shattered the housing - which sould not have happened), they were pulling shrapnel out of the walls for weeks..

Back (at least closer) to the topic.

The issue of which tools to use to test code is a serious one, but the selection usually depends on the persons needs and goals. I have found it very difficult to qualify one as "better" than the others across the possibilities of situations. Fortunately many of them are free, and a few hours is typically all it takes to make a personal evaluation.

This does not really address the question of "HOW" do you test your code. WhiteBox, BlackBox, GreyBox are completely different approaches. There are also domain bounded test methods (done test numbers like 10^20 for single item prices in a shopping cart application) and full perumation test methods (which can often run for a very very long time).

Since much of my early experience was in demanding applications (Military, Space, etc) where a defect was simply too cost prohibitive to consider, comprehensive (whatever that really means) testing was the norm.

When people who do not come from a high testing background look at serious test methodologies, they are often concerned with the cost ($$$ and time is money) involved. Many reject it out of hand without even making in depth studies.

OVer the past 25 years (since the founding of my company) we have often evaluated places where we could "scale back" on some of our testing from the (sometime extreme) methodologies that served us well in the past. Because we are alread invested in the methodologies and have large suites of test application, the "ramp up" costs are mainly ancient history, and we have to actually consider the costs of creating "lighter" test platforms. Our repeated experience [YMMV] has been that there is NO payoff to decreasing the level of testing.

The conclusion (from my point of view) is that a driving factor is if you are in this for the "long haul" or is software development a transient or secondary activity. If the former, then any investment in testing will payoff; if the latter then the risks of releasing buggy code may be more tolerable, and indepth testing may not be a financial win.