-
May 22nd, 2009, 08:32 AM
#16
Re: INT Function Rounding Problems
Yes, and not yet finished.
@David, I'm not sure if this explins what happens here.
Remember the core question is
Why is Int(tv * 15 + 0.5) correctly 32 and
why is Int(tv * Val("15") + 0.5) wrongly 31
Val("15") can only evaluate to 15, but I think it depends what datatype this 15 is gonna be.
If tv is a double (not a currency) Val("15") as a multiplier evaluates to a double, too, and the result is correct.
I suspect when tv is a currency, Val("15") becomes a currency, too, as well as 0.5, and then and only then we get the error.
Only I can't explain then, why the result is ok when you give 15 literally...
It's really a little disquieting.
-
May 22nd, 2009, 10:01 AM
#17
Re: INT Function Rounding Problems
The Val function returns a double.
If this is the case then I would assume that the tv variable (if it is a currency) would have to be converted to a double as double variables have a much larger range.
Converting the result of the val function to a currency may result in overflow errors whereas converting the tv to a double would only affect the accuracy of the number.
WoF: The question is also why if tv is a currency variable Int(tv * Val("15") + 0.5) returns 31 but (tv * Val("15") + 0.5) returns 32. Basically int(32)=31!
-
May 22nd, 2009, 04:08 PM
#18
Re: INT Function Rounding Problems
Right, right you are.
And now after this one gave me the willies for days now, I'll add to this some more.
Code:
dim tv as Single
tv = 2.1
debug.print tv * Val("15") 'prints 31.4999985694885
debug.print tv * 15 'prints 31.5
Maybe there is the reason. We get conversion to a single where it is not 31.5 but 31.49...
Then the rounding of the Int() function gets it and cuts off to 31.
Then Adding 0.5 makes 31.5 and the final rounding of the Int() function drops bakc to 31.
Well I'm not sure, but it is something like this.
-
May 22nd, 2009, 07:18 PM
#19
Re: INT Function Rounding Problems
Didn't you read my post?
To make them equal, I added cInt()
Code:
Private Sub Form_Load()
Dim tv As Single
tv = 2.1
Debug.Print tv * CInt(Val("15"))
Debug.Print tv * 15
End Sub
-
May 24th, 2009, 05:12 PM
#20
Re: INT Function Rounding Problems
David, I read your post full well.
Now your introduction of CInt() produces a new aspect of weirdness.
Try this
Code:
Dim tv As Single
tv = 2.1
'Debug.Print tv * CInt(Val("15"))
'Debug.Print tv * 15
Debug.Print Int(tv * CInt(Val("15") + 0.5))
Debug.Print Int(tv * CInt(15) + 0.5)
If you remove the Int function in fornt of the expressions you'll see some binary rounding error for sure.
-
May 24th, 2009, 11:00 PM
#21
Re: INT Function Rounding Problems
Adding cInt() does it again. What answer do you want?
Code:
Private Sub Form_Load()
Dim tv As Single
tv = 2.1
Debug.Print tv * CInt(Val("15")) ' 31.5
Debug.Print tv * 15 ' 31.5
Debug.Print Int(tv * CInt(Val("15") + 0.5)) ' 33
Debug.Print Int(tv * CInt(15) + 0.5) ' 31
Debug.Print CInt(tv * CInt(15) + 0.5) ' 32
Debug.Print (tv * CInt(Val("15") + 0.5)) ' 33.6
Debug.Print (tv * CInt(15) + 0.5) ' 31.9999985694885
End Sub
-
May 25th, 2009, 05:12 PM
#22
Re: INT Function Rounding Problems
Yes, I admitted already that CInt does it. There were already two more solutions how to avoid the problem, but there was no real explanation, why that darn error happens.
-
May 25th, 2009, 05:55 PM
#23
Re: INT Function Rounding Problems
I posted the latest code, along with the link to my group. They didin't understand what the question is? Then they pointed to the link I gave before explaining Banker's Rounding. Int() does not, but Cint() does use it.
-
May 26th, 2009, 06:22 AM
#24
Re: INT Function Rounding Problems
Ok, I will not longer insist in this, since the reason of this might be the bankers or no bankers rounding.
So in memory of this code:
Code:
dim tv as Single
tv = 2.1
debug.print tv * Val("15") 'prints 31.4999985694885
debug.print tv * 15 'prints 31.5
I will issue the warning never to use the Val() function within an expression where rounding takes place.
Because the full integral value of 15 is 'rounded' down to 14.9999999 (something like that)
What I wanted to point out is, the Val() function should NOT do any (un)rounding at all, bankers or no.
-
May 26th, 2009, 06:15 PM
#25
Re: INT Function Rounding Problems
I think Val() evaluates as a Long, which is the rounding problem.
-
May 27th, 2009, 01:04 AM
#26
Re: INT Function Rounding Problems
?typename(val("1"))
Double
-
May 27th, 2009, 01:10 AM
#27
Re: INT Function Rounding Problems
I ran some tests. Val("15") returns 15. The problem only seems to occur here when tv is defined as a single and the val() statement is used without being converted to a specific type. Vb is guessing at what type of var should be returned and apparently in this case it is incorrect.
If I use a double or currency it returns correctly, if I place the val() inside a csng() or cint() it returns correct. If I place it inside a clng() or Ccur() or Cdbl() it shows the issue.
It would seem that one should assign the result of the Val() statement into the proper variable type before doing math operations on the data returned by this function.
-
May 27th, 2009, 08:43 AM
#28
Re: INT Function Rounding Problems
Yes, that's what I concluded, too:
Do not use Val() inside expressions with different datatypes.
-
May 27th, 2009, 08:56 AM
#29
Re: INT Function Rounding Problems
Val function is Evil.
Try
? val("3,512.12")
A good reason not to use it.
-
May 27th, 2009, 05:53 PM
#30
Re: INT Function Rounding Problems
Yes, well it is known that the val function stops at the next nonnumerical character, which - in this case - is the comma. That's not very good, but it's a well known fact.
What I did not know is, that it will fail at evaluating an integer to the point, when used within an expression of other numeric data types.
Since other type conversion functions like CCur(), CInt(), CDbl() take string type arguments as well, your advice not to use Val() is not wrong, though.
Last edited by WoF; May 27th, 2009 at 05:57 PM.
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|