Python Forum

Hello,

I was using the IDLE today and typed in 33*25.4 and it evaluated to 838.1999999999999

I also tried the Python web shell and it gave me the same answer?

Does anyone know why this would happen? Have I found a bug of some kind, or is it just an operator error?

Thanks!

That is the correct value. The binary value closest to 838.2. Computers work with binary numbers, not decimals. Most decimal numbers do not have a binary equivalent.

Read through the python tutorial on this:
https://docs.python.org/3/tutorial/floatingpoint.html

Thank you both for your help with this. I will keep this in mind with my programs and find ways to adjust when necessary. I suppose there isn't a way to set up a program to globally round all floats to 8 is there? I didn't see anything like that in the tutorial. They just said to round your final numbers to what you want.

Thanks again!

Let me guess. You're converting inch to mm.

The slowest solution is to work with Decimals.

from decimal import Decimal



def inch2mm(inch, precision=None):
    value = float(Decimal(str(inch)) * Decimal("25.4"))
    # optional rounding if required
    if precision is not None:
        value = round(value, precision)
    return value

Another trick is to do the calculation with integers instead of floats.

print(33 * 25.4)
print(33 * 254 / 10)

Yes, I am converting from inch to mm and mm to inch every day. I'm an engineer :)

I was also thinking about calculating it as an integer instead of a float, but ended up with this (see below), which I really wasn't expecting either. This base 2 stuff really blows my mind!

>>> format(33*(254/10),'.17g')
'838.19999999999993'
>>> format((33*254)/10,'.17g')
'838.20000000000005'

I now understand why I sometimes see these very small errors in SolidWorks, which is always rounded to 8 decimal places.

Thanks for your help!