# INT Function Rounding Problems

Show 50 post(s) from this thread on one page
Page 2 of 3 First 123 Last
• May 22nd, 2009, 08:32 AM
WoF
Re: INT Function Rounding Problems
Yes, and not yet finished.
@David, I'm not sure if this explins what happens here.
Remember the core question is
Why is Int(tv * 15 + 0.5) correctly 32 and
why is Int(tv * Val("15") + 0.5) wrongly 31
Val("15") can only evaluate to 15, but I think it depends what datatype this 15 is gonna be.
If tv is a double (not a currency) Val("15") as a multiplier evaluates to a double, too, and the result is correct.
I suspect when tv is a currency, Val("15") becomes a currency, too, as well as 0.5, and then and only then we get the error.
Only I can't explain then, why the result is ok when you give 15 literally...
It's really a little disquieting.
• May 22nd, 2009, 10:01 AM
Killa69
Re: INT Function Rounding Problems
The Val function returns a double.
If this is the case then I would assume that the tv variable (if it is a currency) would have to be converted to a double as double variables have a much larger range.
Converting the result of the val function to a currency may result in overflow errors whereas converting the tv to a double would only affect the accuracy of the number.

WoF: The question is also why if tv is a currency variable Int(tv * Val("15") + 0.5) returns 31 but (tv * Val("15") + 0.5) returns 32. Basically int(32)=31!
• May 22nd, 2009, 04:08 PM
WoF
Re: INT Function Rounding Problems
Right, right you are.
And now after this one gave me the willies for days now, I'll add to this some more.
Code:

```dim tv as Single tv = 2.1 debug.print tv * Val("15") 'prints 31.4999985694885 debug.print tv * 15          'prints 31.5```
Maybe there is the reason. We get conversion to a single where it is not 31.5 but 31.49...
Then the rounding of the Int() function gets it and cuts off to 31.
Then Adding 0.5 makes 31.5 and the final rounding of the Int() function drops bakc to 31.

Well I'm not sure, but it is something like this.
• May 22nd, 2009, 07:18 PM
dglienna
Re: INT Function Rounding Problems

To make them equal, I added cInt()
Code:

```Private Sub Form_Load()     Dim tv As Single     tv = 2.1     Debug.Print tv * CInt(Val("15"))     Debug.Print tv * 15 End Sub```
• May 24th, 2009, 05:12 PM
WoF
Re: INT Function Rounding Problems
Now your introduction of CInt() produces a new aspect of weirdness. :)
Try this
Code:

```    Dim tv As Single     tv = 2.1     'Debug.Print tv * CInt(Val("15"))     'Debug.Print tv * 15     Debug.Print Int(tv * CInt(Val("15") + 0.5))     Debug.Print Int(tv * CInt(15) + 0.5)```
If you remove the Int function in fornt of the expressions you'll see some binary rounding error for sure.
• May 24th, 2009, 11:00 PM
dglienna
Re: INT Function Rounding Problems

Code:

```Private Sub Form_Load()     Dim tv As Single     tv = 2.1     Debug.Print tv * CInt(Val("15")) ' 31.5     Debug.Print tv * 15 ' 31.5     Debug.Print Int(tv * CInt(Val("15") + 0.5)) ' 33     Debug.Print Int(tv * CInt(15) + 0.5) ' 31     Debug.Print CInt(tv * CInt(15) + 0.5) ' 32     Debug.Print (tv * CInt(Val("15") + 0.5)) ' 33.6     Debug.Print (tv * CInt(15) + 0.5) ' 31.9999985694885 End Sub```
• May 25th, 2009, 05:12 PM
WoF
Re: INT Function Rounding Problems
Yes, I admitted already that CInt does it. There were already two more solutions how to avoid the problem, but there was no real explanation, why that darn error happens.
• May 25th, 2009, 05:55 PM
dglienna
Re: INT Function Rounding Problems
I posted the latest code, along with the link to my group. They didin't understand what the question is? Then they pointed to the link I gave before explaining Banker's Rounding. Int() does not, but Cint() does use it.
• May 26th, 2009, 06:22 AM
WoF
Re: INT Function Rounding Problems
Ok, I will not longer insist in this, since the reason of this might be the bankers or no bankers rounding.

So in memory of this code:
Code:

```dim tv as Single tv = 2.1 debug.print tv * Val("15") 'prints 31.4999985694885 debug.print tv * 15          'prints 31.5```
I will issue the warning never to use the Val() function within an expression where rounding takes place.
Because the full integral value of 15 is 'rounded' down to 14.9999999 (something like that)
What I wanted to point out is, the Val() function should NOT do any (un)rounding at all, bankers or no.
• May 26th, 2009, 06:15 PM
dglienna
Re: INT Function Rounding Problems
I think Val() evaluates as a Long, which is the rounding problem.
• May 27th, 2009, 01:04 AM
vb5prgrmr
Re: INT Function Rounding Problems
?typename(val("1"))
Double
• May 27th, 2009, 01:10 AM
DataMiser
Re: INT Function Rounding Problems
I ran some tests. Val("15") returns 15. The problem only seems to occur here when tv is defined as a single and the val() statement is used without being converted to a specific type. Vb is guessing at what type of var should be returned and apparently in this case it is incorrect.

If I use a double or currency it returns correctly, if I place the val() inside a csng() or cint() it returns correct. If I place it inside a clng() or Ccur() or Cdbl() it shows the issue.

It would seem that one should assign the result of the Val() statement into the proper variable type before doing math operations on the data returned by this function.
• May 27th, 2009, 08:43 AM
WoF
Re: INT Function Rounding Problems
Yes, that's what I concluded, too:
Do not use Val() inside expressions with different datatypes.
• May 27th, 2009, 08:56 AM
d.paulson
Re: INT Function Rounding Problems
Val function is Evil.:D

Try
? val("3,512.12")

A good reason not to use it.
• May 27th, 2009, 05:53 PM
WoF
Re: INT Function Rounding Problems
Yes, well it is known that the val function stops at the next nonnumerical character, which - in this case - is the comma. That's not very good, but it's a well known fact.
What I did not know is, that it will fail at evaluating an integer to the point, when used within an expression of other numeric data types.
Since other type conversion functions like CCur(), CInt(), CDbl() take string type arguments as well, your advice not to use Val() is not wrong, though. :)
Show 50 post(s) from this thread on one page
Page 2 of 3 First 123 Last