MFC floating point precision problem
CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 4 of 4

Thread: MFC floating point precision problem

  1. #1
    Join Date
    Mar 2009
    Posts
    2

    MFC floating point precision problem

    Hi all,
    I use VS 2005 and my MFC application is a simple calculator. When I have
    double x = 0.3 + 0.4, x is 0.699...8. Also when I cast the string "0.3" to double I get 0.299...8. Can you tell me where the problem is? Is it a compiler issue? Any help will be usefull.
    Thanks in advance and regards,
    Peter.

  2. #2
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,893

    Re: MFC floating point precision problem

    First off, floats aren't exact: There's no way around it, that's just how it is.

    Second, I hope you meant convert "0.3" to double, because casting a string to double would be very, very wrong.

  3. #3
    Join Date
    Mar 2009
    Posts
    2

    Re: MFC floating point precision problem

    Yes, I meant convert. I use _tstof to convert it to double. So, If I want in my calculator 0.3 + 0.3 to be 0.6, I shouldn't use double or float? If I need precise floating point how to this - string manipulations?

  4. #4
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Fairfax, VA
    Posts
    10,893

    Re: MFC floating point precision problem

    You could use an infinite precision library. Probably a waste of effort, though----just use IO formatting to round off any precision issues on the display.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  


Windows Mobile Development Center


Click Here to Expand Forum to Full Width

This is a CodeGuru survey question.


Featured


HTML5 Development Center