|
-
May 30th, 2002, 03:35 AM
#1
fpu problem
To my mind, the problem is that the fpu works in binary
In the most cases, there are problems of accuracy with float. It occurs when the number can't be represented by a sum of 1/(2^n)
Some float numbers can be exactly represented such as 0.5 (1/2^2) or 0.3125 (1/2^3 + 1/2^5) for instance.
That is not the case with 0.1, and the number will be between two possibles representations
Unfortunately, the fpu have never been able to choose the closer value : they always whoose the upper value. That's why you have a 1 at the end of your 0.1000000000000000000000001
It is shocking because with integer numbers, the problem does not exists, but everybody knows that the computer are bad calculators.
Hope this help
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|