Calculation Problems - Page 2
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Page 2 of 2 FirstFirst 12
Results 16 to 17 of 17

Thread: Calculation Problems

  1. #16
    Join Date
    Apr 2013
    Posts
    12

    Re: Calculation Problems

    OK, after reading that article (thank you Paul) and many others, I added modified every calculation I was using that involved a decimal value (.18 for tax, .07 for surcharge and .015-.035 fro discount) to the following tResults.

    Code:
    	double tResult = 0;
    	tResult = totalFCost * TIP_TAX_RATE; // .18
    	return floor((tResult + 0.005) * 100) / 100;

  2. #17
    Join Date
    Apr 2000
    Location
    Belgium (Europe)
    Posts
    3,763

    Re: Calculation Problems

    The above may work for your case specifically...
    but it is not guaranteed to give you the answer you expect for every possible input (such is the nature of working with floating point).

    0.005 cannot be represented in a double so this value is going to end up being an approximation.
    tResult equally is not accurate but an approximation.
    the end result might be just a notch too high or a notch too low causing floor() to return the wrong value.
    The end divide by 100 again introduces additional inaccuracies.


    There are methods so you can correct the inaccuracies on each step but this isn't easy stuff, there's reasons why the recommendation is to simply avoid it altogether.

Page 2 of 2 FirstFirst 12

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Azure Activities Information Page

Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center