| |
|
|
 Erasmus.Herold | Hallöchen.
I have the following Rundungsproblem. has someone a idea?
temp! = 64.975
print "DEMO 1: ", temp!
temp! = ROUND(temp!,2)
print "DEMO 2: ", temp!
Output the screen: DEMO 1: 64.975000 DEMO 2: 64.970000
I expect but: DEMO 2: 64.980000
thanks for eure ideas. |
|
|
| |
|
|
|
 p.specht
 | the called Inkomensurabilität: The number 64.975 is binary only slightly To small representable. remedy would here only, with 1+1e-15 To multiplizieren, so The latest the 16 signifikanten to put a "Schubs" sustain. then is ever another number on this unfavourable place on the Zahlengeraden...
one sees what there very happens, if one itself any 17 Nachkommastellen Show can. greeting
|
|
|
| XProfan 11Computer: Gerät, daß es in Mikrosekunden erlaubt, 50.000 Fehler zu machen, zB 'daß' statt 'das'... | 11/13/18 ▲ |
|
|
|
|
 Erasmus.Herold | gavel!
thanks - so try I my Happiness. |
|
|
| |
|
|