Issues with (fix)

Discussion in 'AutoCAD' started by stevej, Jul 21, 2003.

  1. stevej

    stevej Guest

    Hello all,

    Two questions really, I'm trying to test whether a real ends with .0 (eg 200.0)
    I've tried using (fix) then subtracting the real and integer to see if it's 0.0. This seems to work generally but now and again (fix) is giving me a different figure (sometimes something along the lines of 3.54451e-006, sometimes 199) so I was wondering if this is normal behaviour. If it is then, is there another way to check for the above?

    Thanks, Steve
     
    stevej, Jul 21, 2003
    #1
  2. stevej

    David Bethel Guest

    It has to do with the way 'puters deal with ans stores REAL numbers.

    I'd use something like;

    (equal (rem 200.0 1) 0 1e-14)

    Your never going to 100% sure with large values.

    -David
     
    David Bethel, Jul 21, 2003
    #2
  3. It's a rounding problem with some REALS that cannot be represented exactly
    by binary.

    One way around it:

    (defun IsWholeNumber (x)
    (= 0.0 (atof (rtos (- x (fix x)) 2 8))))


    --
    R. Robert Bell, MCSE
    www.AcadX.com


    | Hello all,
    | Two questions really, I'm trying to test whether a real ends with .0 (eg
    200.0)
    | I've tried using (fix) then subtracting the real and integer to see if
    it's 0.0. This seems to work generally but now and again (fix) is giving me
    a different figure (sometimes something along the lines of 3.54451e-006,
    sometimes 199) so I was wondering if this is normal behaviour. If it is
    then, is there another way to check for the above?
    |
    | Thanks, Steve
    |
     
    R. Robert Bell, Jul 21, 2003
    #3
  4. Yup, that's better... no string conversion.

    --
    R. Robert Bell, MCSE
    www.AcadX.com


    | It has to do with the way 'puters deal with ans stores REAL numbers.
    |
    | I'd use something like;
    |
    | (equal (rem 200.0 1) 0 1e-14)
    |
    | Your never going to 100% sure with large values.
    |
    | -David
    |
    | stevej wrote:
    | > Hello all,
    | >
    | > Two questions really, I'm trying to test whether a real ends with .0 (eg
    200.0)
    | > I've tried using (fix) then subtracting the real and integer to see if
    it's 0.0.
    | > This seems to work generally but now and again (fix) is giving me a
    different
    | > figure (sometimes something along the lines of 3.54451e-006, sometimes
    199) so I
    | > was wondering if this is normal behaviour. If it is then, is there
    another way
    | > to check for the above?
    | >
    | > Thanks, Steve
    | >
    |
     
    R. Robert Bell, Jul 21, 2003
    #4
  5. stevej

    David Bethel Guest

    Robert,

    On second thought

    (rem 1.99999999999999999999999 1) wouldn't be a good thing. -David
     
    David Bethel, Jul 21, 2003
    #5
Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.