Python Forum
Trouble with decimal precision
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Trouble with decimal precision
#1
Hi everyone, I have a problem with precision in decimal, I want got 10 numbers after coma, but with context 10 I got 15, why? Thanks.
with localcontext() as ctx:
... ctx.prec = 10
... print(Decimal('1.0000000000') * Decimal('0.0000042372'))
...
0.000004237200000
>>> with localcontext() as ctx:
... ctx.prec = 5
... print(Decimal('1.0000000000') * Decimal('0.0000042372'))
Reply
#2
I think it's working as intended. "precision" is ignoring the leading zeros, instead of being how many decimal places to include. Check this out:
>>> import decimal as dec
>>> dec.getcontext().prec = 1
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004
>>> dec.getcontext().prec = 5
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.0000042372
>>> dec.getcontext().prec = 10
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004237200000
>>> with dec.localcontext() as ctx:
...   ctx.prec = 1
...   print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
...
0.000004
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004237200000
Reply
#3
I think what you're looking for isn't the context, but rather the quantize method: https://docs.python.org/3.7/library/deci...l.quantize

Quote:
>>> Decimal('1.41421356').quantize(Decimal('1.000'))
Decimal('1.414')

At the bottom of the docs, in the FAQ, is this snippet:
Quote:
>>> TWOPLACES = Decimal(10) ** -2       # same as Decimal('0.01')
>>> # Round to two places
>>> Decimal('3.214').quantize(TWOPLACES)
Decimal('3.21')
Reply
#4
You use setcontext, getcontext, and precision, see the example under the "Precision" heading at https://pymotw.com/3/decimal/
Reply
#5
(Feb-14-2019, 08:14 PM)nilamo Wrote: I think it's working as intended. "precision" is ignoring the leading zeros, instead of being how many decimal places to include. Check this out:
>>> import decimal as dec
>>> dec.getcontext().prec = 1
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004
>>> dec.getcontext().prec = 5
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.0000042372
>>> dec.getcontext().prec = 10
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004237200000
>>> with dec.localcontext() as ctx:
...   ctx.prec = 1
...   print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
...
0.000004
>>> print(dec.Decimal('1.00000000000000') * dec.Decimal('0.0000042372'))
0.000004237200000

Thanks! I didn't think's about this.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Gmpy2 Newbie Working on Precision charlesrkiss 5 563 Jan-23-2024, 04:23 PM
Last Post: charlesrkiss
  Precision with Decimal Type charlesrkiss 9 738 Jan-18-2024, 06:30 PM
Last Post: charlesrkiss
  Precision conversion garynewport 2 953 Oct-19-2022, 10:47 AM
Last Post: garynewport
Photo Filtering data and precision during calculations Scientifix 0 1,787 Mar-30-2021, 01:00 PM
Last Post: Scientifix
  High-Precision Board Voltage Reading from Python into PD Liam484 1 2,087 Mar-29-2021, 02:57 PM
Last Post: Marbelous
  Decimal (Global precision setting) Valentina 3 3,642 Aug-23-2019, 11:53 AM
Last Post: Valentina
  python decimal scale precision massimo_m 5 5,432 Aug-22-2019, 11:47 AM
Last Post: perfringo
  testing for Decimal w/o importing decimal every time Skaperen 7 4,465 May-06-2019, 10:23 PM
Last Post: Skaperen
  precision error chris64 1 2,242 Nov-29-2017, 09:40 PM
Last Post: micseydel

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020