CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com
Results 1 to 2 of 2
  1. #1
    Join Date
    May 2009
    Posts
    23

    loss of precision while using CString Format function

    Hi,

    iam facing issues with loss of precision when using CString's Format function, heres a code snippet.
    Code:
    double func(double dvar)
    {
    	return dvar*0.00277777778;
    }
    //code
    	CString tempStr;
    	double var = func(10);	//var is 0.0277777778
    	tempStr.Format("%G",var);	//tempStr is 0.0277778
    How do i prevent this loss of precision?

    thanks in advance

  2. #2
    Lindley is offline Elite Member Power Poster
    Join Date
    Oct 2007
    Location
    Seattle, WA
    Posts
    10,895

    Re: loss of precision while using CString Format function

    Looks like the format string is identical to the one taken by sprintf()----so read up on it:
    http://www.cplusplus.com/reference/c...stdio/sprintf/

    Pay particular attention to the precision argument.

    You should also be aware that 18 digits is enough to fully reconstruct the binary value of any double-precision number. Anything less is likely to represent a slightly different number when converted from a string back to a double, although the distinction may be dwarfed by normal floating point error.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  





Click Here to Expand Forum to Full Width

Featured