CodeGuru Home VC++ / MFC / C++ .NET / C# Visual Basic VB Forums Developer.com

# Thread: INT Function Rounding Problems

1. ## Re: INT Function Rounding Problems

Yes, and not yet finished.
@David, I'm not sure if this explins what happens here.
Remember the core question is
Why is Int(tv * 15 + 0.5) correctly 32 and
why is Int(tv * Val("15") + 0.5) wrongly 31
Val("15") can only evaluate to 15, but I think it depends what datatype this 15 is gonna be.
If tv is a double (not a currency) Val("15") as a multiplier evaluates to a double, too, and the result is correct.
I suspect when tv is a currency, Val("15") becomes a currency, too, as well as 0.5, and then and only then we get the error.
Only I can't explain then, why the result is ok when you give 15 literally...
It's really a little disquieting.

2. ## Re: INT Function Rounding Problems

The Val function returns a double.
If this is the case then I would assume that the tv variable (if it is a currency) would have to be converted to a double as double variables have a much larger range.
Converting the result of the val function to a currency may result in overflow errors whereas converting the tv to a double would only affect the accuracy of the number.

WoF: The question is also why if tv is a currency variable Int(tv * Val("15") + 0.5) returns 31 but (tv * Val("15") + 0.5) returns 32. Basically int(32)=31!

3. ## Re: INT Function Rounding Problems

Right, right you are.
And now after this one gave me the willies for days now, I'll add to this some more.
Code:
```dim tv as Single
tv = 2.1
debug.print tv * Val("15") 'prints 31.4999985694885
debug.print tv * 15           'prints 31.5```
Maybe there is the reason. We get conversion to a single where it is not 31.5 but 31.49...
Then the rounding of the Int() function gets it and cuts off to 31.
Then Adding 0.5 makes 31.5 and the final rounding of the Int() function drops bakc to 31.

Well I'm not sure, but it is something like this.

4. ## Re: INT Function Rounding Problems

Didn't you read my post?

To make them equal, I added cInt()
Code:
```Private Sub Form_Load()
Dim tv As Single
tv = 2.1
Debug.Print tv * CInt(Val("15"))
Debug.Print tv * 15
End Sub```

5. ## Re: INT Function Rounding Problems

David, I read your post full well.
Now your introduction of CInt() produces a new aspect of weirdness.
Try this
Code:
```    Dim tv As Single
tv = 2.1
'Debug.Print tv * CInt(Val("15"))
'Debug.Print tv * 15
Debug.Print Int(tv * CInt(Val("15") + 0.5))
Debug.Print Int(tv * CInt(15) + 0.5)```
If you remove the Int function in fornt of the expressions you'll see some binary rounding error for sure.

6. ## Re: INT Function Rounding Problems

Adding cInt() does it again. What answer do you want?

Code:
```Private Sub Form_Load()
Dim tv As Single
tv = 2.1
Debug.Print tv * CInt(Val("15")) ' 31.5
Debug.Print tv * 15 ' 31.5
Debug.Print Int(tv * CInt(Val("15") + 0.5)) ' 33
Debug.Print Int(tv * CInt(15) + 0.5) ' 31
Debug.Print CInt(tv * CInt(15) + 0.5) ' 32
Debug.Print (tv * CInt(Val("15") + 0.5)) ' 33.6
Debug.Print (tv * CInt(15) + 0.5) ' 31.9999985694885
End Sub```

7. ## Re: INT Function Rounding Problems

Yes, I admitted already that CInt does it. There were already two more solutions how to avoid the problem, but there was no real explanation, why that darn error happens.

8. ## Re: INT Function Rounding Problems

I posted the latest code, along with the link to my group. They didin't understand what the question is? Then they pointed to the link I gave before explaining Banker's Rounding. Int() does not, but Cint() does use it.

9. ## Re: INT Function Rounding Problems

Ok, I will not longer insist in this, since the reason of this might be the bankers or no bankers rounding.

So in memory of this code:
Code:
```dim tv as Single
tv = 2.1
debug.print tv * Val("15") 'prints 31.4999985694885
debug.print tv * 15           'prints 31.5```
I will issue the warning never to use the Val() function within an expression where rounding takes place.
Because the full integral value of 15 is 'rounded' down to 14.9999999 (something like that)
What I wanted to point out is, the Val() function should NOT do any (un)rounding at all, bankers or no.

10. ## Re: INT Function Rounding Problems

I think Val() evaluates as a Long, which is the rounding problem.

11. Member
Join Date
Apr 2009
Posts
394

## Re: INT Function Rounding Problems

?typename(val("1"))
Double

12. Super Moderator Power Poster
Join Date
Jul 2008
Location
WV
Posts
5,265

## Re: INT Function Rounding Problems

I ran some tests. Val("15") returns 15. The problem only seems to occur here when tv is defined as a single and the val() statement is used without being converted to a specific type. Vb is guessing at what type of var should be returned and apparently in this case it is incorrect.

If I use a double or currency it returns correctly, if I place the val() inside a csng() or cint() it returns correct. If I place it inside a clng() or Ccur() or Cdbl() it shows the issue.

It would seem that one should assign the result of the Val() statement into the proper variable type before doing math operations on the data returned by this function.

13. ## Re: INT Function Rounding Problems

Yes, that's what I concluded, too:
Do not use Val() inside expressions with different datatypes.

14. Member +
Join Date
Jan 2000
Location
Posts
595

## Re: INT Function Rounding Problems

Val function is Evil.

Try
? val("3,512.12")

A good reason not to use it.

15. ## Re: INT Function Rounding Problems

Yes, well it is known that the val function stops at the next nonnumerical character, which - in this case - is the comma. That's not very good, but it's a well known fact.
What I did not know is, that it will fail at evaluating an integer to the point, when used within an expression of other numeric data types.
Since other type conversion functions like CCur(), CInt(), CDbl() take string type arguments as well, your advice not to use Val() is not wrong, though.
Last edited by WoF; May 27th, 2009 at 05:57 PM.

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•

Click Here to Expand Forum to Full Width