##### 33*25.4
 33*25.4 schascheck Programmer named Tim Posts: 7 Threads: 2 Joined: Oct 2020 Reputation: Oct-27-2020, 12:45 AM Hello, I was using the IDLE today and typed in 33*25.4 and it evaluated to 838.1999999999999 I also tried the Python web shell and it gave me the same answer? Does anyone know why this would happen? Have I found a bug of some kind, or is it just an operator error? Thanks! Reply deanhystad Da Bishop Posts: 1,959 Threads: 11 Joined: Feb 2020 Reputation: Oct-27-2020, 01:09 AM (This post was last modified: Oct-27-2020, 01:09 AM by deanhystad.) That is the correct value. The binary value closest to 838.2. Computers work with binary numbers, not decimals. Most decimal numbers do not have a binary equivalent. Reply bowlofred Da Bishop Posts: 1,119 Threads: 3 Joined: Mar 2020 Reputation: Oct-27-2020, 01:56 AM Read through the python tutorial on this: https://docs.python.org/3/tutorial/floatingpoint.html Reply schascheck Programmer named Tim Posts: 7 Threads: 2 Joined: Oct 2020 Reputation: Oct-27-2020, 03:38 AM Thank you both for your help with this. I will keep this in mind with my programs and find ways to adjust when necessary. I suppose there isn't a way to set up a program to globally round all floats to 8 is there? I didn't see anything like that in the tutorial. They just said to round your final numbers to what you want. Thanks again! Reply DeaD_EyE Da Bishop Posts: 1,613 Threads: 6 Joined: May 2017 Reputation: Oct-27-2020, 08:47 AM Let me guess. You're converting `inch` to `mm`. The slowest solution is to work with Decimals. ```from decimal import Decimal def inch2mm(inch, precision=None): value = float(Decimal(str(inch)) * Decimal("25.4")) # optional rounding if required if precision is not None: value = round(value, precision) return value```Another trick is to do the calculation with integers instead of floats. ```print(33 * 25.4) print(33 * 254 / 10)``` My code examples are always for Python >=3.6.0 Almost dead, but too lazy to die: https://sourceserver.info All humans together. We don't need politicians! Reply schascheck Programmer named Tim Posts: 7 Threads: 2 Joined: Oct 2020 Reputation: Oct-27-2020, 02:46 PM Yes, I am converting from inch to mm and mm to inch every day. I'm an engineer :) I was also thinking about calculating it as an integer instead of a float, but ended up with this (see below), which I really wasn't expecting either. This base 2 stuff really blows my mind! >>> format(33*(254/10),'.17g') '838.19999999999993' >>> format((33*254)/10,'.17g') '838.20000000000005' I now understand why I sometimes see these very small errors in SolidWorks, which is always rounded to 8 decimal places. Thanks for your help! Reply

Forum Jump:

### User Panel Messages

##### Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020