Printable View
I'm currently using a string as single value
say:
"
Dim test As String
test = 0
"
and with a command button:
"
test = test + 0.1
"
it seems to work fine on my computer and on some others
when i used single, sometimes it turned the value test to 0.5000001
but if i declare test as a string it all works out fine
can anyone explain to me how this is possible ?
antlion. I have moved your post to a new thread, as you have originally replied to an eight year old thread
It is not usual to compute numbers with string variables, although it works as soon as you have initialized the variable with temp = 0.
If you'd have left temp empty, temp = temp + 0.1 will produce a type mismatch error.
When you work with the data type Single, due to the method how a computer is processing fractional data in binary digits, some rounding error occurs.
This error does not appear when the data type Double is used.
When you use strings, VBs automatic type conversion takes place. Before a string can be used in a numeric expression it will be converted to a numeric data type and the numeric result will be converted bnack to a string before it is stored again in the string variable.
As it seems VB converts both expressions (temp and 0.1) into Doubles before computing, so as the rounding error of the Singles does not appear.
It is called a floating point error. It is inherent in the way the processor handles the numeric data. This also applies to the Double data type. However, the Currency data type will give you a few decimal places, but without any floating point error.
Did you tried something like this?Quote:
Originally Posted by antlion
I didn't run the code, but it shoud work with explicit type conversion.Quote:
Dim test As String
test = CStr(0)
and with a command button:"
test = CStr( CSng(test) + 0.1 )