CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 12 of 12

Threaded View

  1. #1
    Join Date
    Sep 2008
    Posts
    3

    Angry Testing primality of long doubles

    I've been tearing my hair out over this for awhile, and I have a feeling I'm going to kick myself when someone shows me what I'm doing wrong. I've written a very basic boolian function to test input for primality (full function in the attached header). It works perfectly for all integer values I can throw at it, but when I start using floating-point numbers (which I've tried to design it to handle), I start running into weird problems.

    For example, the number 9.9999999978e10, which is clearly divisible by 2, should be caught here:
    Code:
      if (static_cast<int>( fmodl(n, 2) ) == 0)             // tests for divisibility by 2
       {return false;}
       else
    The variable n in this case being a long double, the input to the prime() function.

    As far as I know, typecasting the result of the modulus will drop anything after the decimal point, (and indeed, a simple test program running static_cast<int>(fmodl(9.9999999978e10,2)) will return 0), eliminating the weirdness that I've heard comes with comparisons with floating-point numbers.

    Oddly, prime() is always returning true with non-integer numbers, and if the problem is with this comparison or somewhere else in the function, for the life of me I can't figure out why.

    Any insight is much appreciated.

    ~Jap
    Attached Files Attached Files

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured