|
-
November 29th, 2021, 03:48 AM
#5
Re: Float point number simple arithmetic.
In addition to my post #4.
Not all decimal numbers have a finite digital representation. This includes 0.1, which requires an infinite number of bits. So when you assign 0.1 to a single-precision FP, there is a truncation error. When you print the FP, you see 0.1000000015, where the 15 is the error. The 15, however, is not part of the significant-digit range and should not be considered. (If you instead assign 0.125, there is no truncation because it has a finite digital representation that fits within the significant-digit range.)
Now say you assign 0.1000000015 to a double-precision FP. This is a different story because now the 15 is part of the significant-digit range of the FP. If you square this FP, a 3 appears at a certain position determined by the place of the 15. If you instead square 0.1015, the 3 shows up again but in another position. This happens both with single and double precision. It is because 0.1015 is within the significant-digit range of both, as is the 3 in the squared result.
Last edited by wolle; December 1st, 2021 at 01:24 AM.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|