Digital Workshop

Am I doing something really silly here. Please tell me I am, as I keep get a very odd answer.

I keep getting 5.55112e-017 instead of 0

Page variable called TestVarMe set to the number 1 to being.

Code:



Debug after a few run throughs:
TestVarMe 0.8
TestVarMe 0.6
TestVarMe 0.4
TestVarMe 0.2
TestVarMe 5.55112e-017
TestVarMe -0.2
TestVarMe -0.4
TestVarMe -0.6
TestVarMe -0.8
TestVarMe -1
TestVarMe -1.2
TestVarMe -1.4
TestVarMe -1.6
TestVarMe -1.8
TestVarMe -2
TestVarMe -2.2


If I change the subtraction to 0.25, it works, but 0.2 just throws up 5.55112e-017

Thanks, LMC

That is probably just a ZERO with a very small amount of positive spin.

If you started with a value of 0.800, that should not happen. But if you've done some earlier calculations, it might be some residual small value.

See attached sample, started from scratch, with no other code. Still the same out come.

Hi LMC,

Not uncommon when programming in Opusscript, C++ etc. Computers cannot store exact values of floating point numbers. Google "machine epsilon" to understand the issues involved, or check out the page below for a basic explanation:

http://www.akiti.ca/MachEps.html

Just apply a correction when the number approaches zero.
Change your script to::

function SubtractPtTwo()
{
TestVarMe = TestVarMe - SubtractVar;
if(Math.abs(TestVarMe)<0.000001)
{
TestVarMe=0;
}
Debug.trace("TestVarMe " + TestVarMe + "\n")
}

Paul

Paul,

(okay, this is more of a joke ) I am 'non-plussed'.

If anyone has Opus 6, can you try the same code to see if it works? I hope I don't have to change old publications to check for this.

Happens in Opus 4,5,6 and 7.

Mack

That's good to know, so code will have the same outcome across the versions.