Python Forum
33*25.4 - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: 33*25.4 (/thread-30581.html)

33*25.4 - schascheck - Oct-27-2020

Hello,

I was using the IDLE today and typed in 33*25.4 and it evaluated to 838.1999999999999

I also tried the Python web shell and it gave me the same answer?

Does anyone know why this would happen? Have I found a bug of some kind, or is it just an operator error?

Thanks!

RE: 33*25.4 - deanhystad - Oct-27-2020

That is the correct value. The binary value closest to 838.2. Computers work with binary numbers, not decimals. Most decimal numbers do not have a binary equivalent.

RE: 33*25.4 - bowlofred - Oct-27-2020

Read through the python tutorial on this:
https://docs.python.org/3/tutorial/floatingpoint.html

RE: 33*25.4 - schascheck - Oct-27-2020

Thank you both for your help with this. I will keep this in mind with my programs and find ways to adjust when necessary. I suppose there isn't a way to set up a program to globally round all floats to 8 is there? I didn't see anything like that in the tutorial. They just said to round your final numbers to what you want.

Thanks again!

RE: 33*25.4 - DeaD_EyE - Oct-27-2020

Let me guess. You're converting inch to mm.

The slowest solution is to work with Decimals.

from decimal import Decimal



def inch2mm(inch, precision=None):
    value = float(Decimal(str(inch)) * Decimal("25.4"))
    # optional rounding if required
    if precision is not None:
        value = round(value, precision)
    return value
Another trick is to do the calculation with integers instead of floats.
print(33 * 25.4)
print(33 * 254 / 10)


RE: 33*25.4 - schascheck - Oct-27-2020

Yes, I am converting from inch to mm and mm to inch every day. I'm an engineer :)

I was also thinking about calculating it as an integer instead of a float, but ended up with this (see below), which I really wasn't expecting either. This base 2 stuff really blows my mind!

>>> format(33*(254/10),'.17g')
'838.19999999999993'
>>> format((33*254)/10,'.17g')
'838.20000000000005'

I now understand why I sometimes see these very small errors in SolidWorks, which is always rounded to 8 decimal places.

Thanks for your help!