Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Precision with Decimal Type
#1
I'm having some problems with getting the desired precision, and something like this also happens.

>>> G=Decimal(0.999999999999999999999900000000000000)
>>> print(G)
1

What am I doing wrong?

See the text and image below for extra stuff, when I was working with theta. Theta was generated in the same python script and session:

import math
import decimal
from decimal import getcontext


getcontext().prec = 36

theta = decimal.Decimal(1/2 * math.asin(2 * (F/N)))

theta: 1.00000000000000010105568267120302867849435113356548754381947219371795654296875E-11

C = decimal.Decimal(math.cos(theta))

C: 1


Thank you for reading this!

   
Reply


Messages In This Thread
Precision with Decimal Type - by charlesrkiss - Jan-16-2024, 06:03 PM
RE: Precision with Decimal Type - by rob101 - Jan-16-2024, 08:21 PM
RE: Precision with Decimal Type - by charlesrkiss - Jan-16-2024, 11:49 PM
RE: Precision with Decimal Type - by charlesrkiss - Jan-17-2024, 03:44 AM
RE: Precision with Decimal Type - by perfringo - Jan-17-2024, 07:57 AM
RE: Precision with Decimal Type - by charlesrkiss - Jan-17-2024, 10:45 PM
RE: Precision with Decimal Type - by casevh - Jan-18-2024, 05:55 AM
RE: Precision with Decimal Type - by charlesrkiss - Jan-18-2024, 02:37 AM
RE: Precision with Decimal Type - by charlesrkiss - Jan-18-2024, 05:21 PM
RE: Precision with Decimal Type - by charlesrkiss - Jan-18-2024, 06:30 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Gmpy2 Newbie Working on Precision charlesrkiss 5 679 Jan-23-2024, 04:23 PM
Last Post: charlesrkiss
  Precision conversion garynewport 2 1,038 Oct-19-2022, 10:47 AM
Last Post: garynewport
Photo Filtering data and precision during calculations Scientifix 0 1,833 Mar-30-2021, 01:00 PM
Last Post: Scientifix
  High-Precision Board Voltage Reading from Python into PD Liam484 1 2,140 Mar-29-2021, 02:57 PM
Last Post: Marbelous
  Type hinting - return type based on parameter micseydel 2 2,577 Jan-14-2020, 01:20 AM
Last Post: micseydel
  Decimal (Global precision setting) Valentina 3 3,735 Aug-23-2019, 11:53 AM
Last Post: Valentina
  python decimal scale precision massimo_m 5 5,597 Aug-22-2019, 11:47 AM
Last Post: perfringo
  testing for Decimal w/o importing decimal every time Skaperen 7 4,604 May-06-2019, 10:23 PM
Last Post: Skaperen
  Using Decimal Type instead of Float Bytales 1 2,980 Mar-02-2019, 06:28 PM
Last Post: Bytales
  Trouble with decimal precision Cassie 4 3,238 Feb-14-2019, 08:23 PM
Last Post: Cassie

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020