Python Forum
precision in formatting float
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
precision in formatting float
#1
Quote:The precision is a decimal number indicating how many digits should be displayed after the decimal point for a floating point value formatted with 'f' and 'F', or before and after the decimal point for a floating point value formatted with 'g' or 'G'.

the library reference for 3.7.2 says the above, specifically "after the decimal point". yet, what i find resulting from format() is that the precision value includes the decimal point itself. by setting the precision to 6, i get 5 decimal digits after the decimal point.

has anyone else observed this? i don't know whether this is a bug or a documentation error.

i am using 3.6.9 and reading docs for 3.7.2 because PDF was messed up for 3.6. i wonder if any of this has changed in later versions.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
(Apr-30-2022, 09:43 PM)Skaperen Wrote: yet, what i find resulting from format() is that the precision value includes the decimal point itself. by setting the precision to 6, i get 5 decimal digits after the decimal point.
>>> n = 1.1122334455
>>> print(f"The numbers after decimal place {n:.6f}")
The numbers after decimal place 1.112233
>>> print(f"The numbers after decimal place {n:.12f}")
The numbers after decimal place 1.112233445500
>>> 
>>> print("The numbers after decimal place {:.6f}".format(n))
The numbers after decimal place 1.112233
>>> print("The numbers after decimal place {:.12f}".format(n))
The numbers after decimal place 1.112233445500
So work as expected,make no difference if use Python 3.6 or as i use here newest Python 3.10.4.
The rule is:
f'{value:{width}.{precision}}'
width specifies the number of characters total to display.
Reply
#3
i get the same results you do with your examples. these are what i expect from reading the docs. i need to go find what i did to get the apparently non-compliant results.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  quest for better float formatting Skaperen 0 1,620 Mar-02-2022, 08:34 PM
Last Post: Skaperen
  functio to do fixed formatting of int/float Skaperen 0 2,395 Dec-26-2018, 09:55 PM
Last Post: Skaperen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020