|
-
April 18th, 2001, 07:02 PM
#1
Subtracting variables
Quick question.
when I try to subtract two variables I sometimes end up with results that look like this 2.3E+121223 instead of 23.
-
April 19th, 2001, 07:32 AM
#2
Re: Subtracting variables
what is your question? Is it scientific format that you don't like?
Try to format the result
Format(YourResult,"0.00")
Iouri Boutchkine
[email protected]
-
April 19th, 2001, 08:40 AM
#3
Re: Subtracting variables
I think your problem is the data type of the variables you are using.
If you store them in a floating point type, such as double, then you can strike some of these problems to do with precision of numbers.
If your numbers are integers, just use an integer data type (or long if greater than 32000-odd)
-
April 19th, 2001, 10:08 AM
#4
Re: Subtracting variables
Thanks, but i am dealing with fractional numbers, and i must use a Single variable type. Is there any way to avoid these results? what causes them any way
-
April 19th, 2001, 10:14 AM
#5
Re: Subtracting variables
My problem is that the result should be 23, and not some other long unusable number.
-
April 19th, 2001, 10:26 AM
#6
Re: Subtracting variables
I think it would help if you would provide some sample of the code you are using and/or the numbers you are subtracting that produce this result.
-K
-
April 19th, 2001, 10:29 AM
#7
Re: Subtracting variables
The problem is that floating point data types like single and double cannot always accurately represent fractional values.
That's because fractional values are continuous, whereas they have to be represented as an exact value in the data structure (as a series of 1s and 0s).
If you use a Currency Data type which exactly represents data, that should solve your problem, as long as you are happy to have a precision of 4 decimal places in your data.
i.e. 1.2345 - 1.1234 should work fine
-
April 19th, 2001, 10:57 AM
#8
Re: Subtracting variables
If your variables are fractional numbers but you want the answer to be a whole number (Integer)
try using the CInt Function:
Result = CInt(Var1 - Var2)
this will return a whole number but it will be rounded. You could do this several different ways.
you could declare your result variable as type Variant and use the CDec funtion to return
the result as a decimal and then extract everything left of the decimal point:
Dim Result as Variant
Dim Pos as Integer
'get Decimal Value Of Equation Result
Result = CDec(Var1 - Var2)
'Extract The Whole Number
Pos = InStr(1, Str$(Result), Chr(46))
Result = Val(mid$(Str$(Result), 1, Pos - 1))
Kris
Software Engineer
Phoenix,AZ
Kris
Software Engineer
Phoenix, AZ USA
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|