|
-
October 22nd, 2009, 08:00 AM
#1
loss of precision while using CString Format function
Hi,
iam facing issues with loss of precision when using CString's Format function, heres a code snippet.
Code:
double func(double dvar)
{
return dvar*0.00277777778;
}
//code
CString tempStr;
double var = func(10); //var is 0.0277777778
tempStr.Format("%G",var); //tempStr is 0.0277778
How do i prevent this loss of precision?
thanks in advance
-
October 22nd, 2009, 08:14 AM
#2
Re: loss of precision while using CString Format function
Looks like the format string is identical to the one taken by sprintf()----so read up on it:
http://www.cplusplus.com/reference/c...stdio/sprintf/
Pay particular attention to the precision argument.
You should also be aware that 18 digits is enough to fully reconstruct the binary value of any double-precision number. Anything less is likely to represent a slightly different number when converted from a string back to a double, although the distinction may be dwarfed by normal floating point error.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|