-
April 26th, 2013, 02:16 PM
#16
Re: Calculation Problems
OK, after reading that article (thank you Paul) and many others, I added modified every calculation I was using that involved a decimal value (.18 for tax, .07 for surcharge and .015-.035 fro discount) to the following tResults.
Code:
double tResult = 0;
tResult = totalFCost * TIP_TAX_RATE; // .18
return floor((tResult + 0.005) * 100) / 100;
-
April 29th, 2013, 08:19 AM
#17
Re: Calculation Problems
The above may work for your case specifically...
but it is not guaranteed to give you the answer you expect for every possible input (such is the nature of working with floating point).
0.005 cannot be represented in a double so this value is going to end up being an approximation.
tResult equally is not accurate but an approximation.
the end result might be just a notch too high or a notch too low causing floor() to return the wrong value.
The end divide by 100 again introduces additional inaccuracies.
There are methods so you can correct the inaccuracies on each step but this isn't easy stuff, there's reasons why the recommendation is to simply avoid it altogether.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|