Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Precision conversion
#3
The problem is that the standard output for Fortran is to use the D, except where the exponent is 3 or more digits long; then the D is dropped.

So, in Fortran output 4.99722-235 is equivalent to 4.99722D-235 or 4.99722e-235; and not an equation.

Aside from that (and this only affects the first two lines and only in two results, but these cannot be ignored), your solution is far nicer than mine. Smile
Reply


Messages In This Thread
Precision conversion - by garynewport - Oct-12-2022, 11:49 AM
RE: Precision conversion - by deanhystad - Oct-12-2022, 04:00 PM
RE: Precision conversion - by garynewport - Oct-19-2022, 10:47 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Gmpy2 Newbie Working on Precision charlesrkiss 5 606 Jan-23-2024, 04:23 PM
Last Post: charlesrkiss
  Precision with Decimal Type charlesrkiss 9 819 Jan-18-2024, 06:30 PM
Last Post: charlesrkiss
Photo Filtering data and precision during calculations Scientifix 0 1,802 Mar-30-2021, 01:00 PM
Last Post: Scientifix
  High-Precision Board Voltage Reading from Python into PD Liam484 1 2,111 Mar-29-2021, 02:57 PM
Last Post: Marbelous
  Decimal (Global precision setting) Valentina 3 3,684 Aug-23-2019, 11:53 AM
Last Post: Valentina
  python decimal scale precision massimo_m 5 5,504 Aug-22-2019, 11:47 AM
Last Post: perfringo
  Trouble with decimal precision Cassie 4 3,179 Feb-14-2019, 08:23 PM
Last Post: Cassie
  precision error chris64 1 2,262 Nov-29-2017, 09:40 PM
Last Post: micseydel

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020