issue with converting a scientific notation to standard notation

[thomaswfirth]

I am reading the value from a DMM and the returned number is in Scientific notation. When I try to convert that number to get a decimal number, I get an error.

If I run the same script but cut and paste the reported DMM number into "number = 9.985660E-01" instead it works fine.
Are there some hidden characters I'm not seeing.? Is there a better way to convert to a standard decimal number from the DMM number?

Using number from DMM:
1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 # number = 9.985660E-01
22 print("%.7f"%number) # Does not work using number in from DMM
23 rm.close()

PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65078' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.925

9.349951E-01 #Value from DMM used in line 22

Traceback (most recent call last):
File "c:\Users\thoma\Desktop\Python\xxxx.py", line 22, in <module>
print("%.7f"%number) # Does not work using number in from DMM
~~~~~~^~~~~~~
TypeError: must be real number, not str
PS C:\Users\thoma\Desktop\Python>


1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 number = 9.985660E-01
22 print("%.7f"%number) # uses number in line 21 and works fine
23rm.close()

PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65120' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.930

9.346759E-01

0.9985660 # Converted number from line 21
PS C:\Users\thoma\Desktop\Python>

deanhystad write Jun-01-2023, 06:15 PM:
Please post all code, output and errors (it it's entirety) between their respective tags. Refer to BBCode help topic on how to post. Use the "Preview Post" button to make sure the code is presented as you expect before hitting the "Post Reply/Thread" button.

[deanhystad]

In your code, val and number are strings, not a number. To convert numeric strings to numbers use int(str) or float(str).

val = float(ps.query('IOUT?'))
number = float(dmm.query("MEAS?"))

[thomaswfirth]

Wow, something so simple that solved the issue, thanks. I have a lot to learn but it is keeping my mind working.

[buran]

@thomaswfirth, please, use BBCode tags and also DO NOT include line numbers with your code

[rajeshgk]

The error you're encountering is due to the fact that the value returned from the DMM is a string, and you're trying to format it as a float using the %.7f format specifier. To resolve this issue, you need to convert the string to a float before formatting it.

Here's an updated version of your code that converts the DMM value to a decimal number and formats it as a float:

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

import pyvisa
import time

rm = pyvisa.ResourceManager()
ps = rm.open_resource('ASRL8::INSTR')
ps.baud_rate = 115200
dmm = rm.open_resource('ASRL10::INSTR')
dmm.baud_rate = 115200

ps.write('VSET:1.0?')
ps.write('OUT:1')
ps.write('IOUT:1?')
time.sleep(3)
val = ps.query('IOUT?')
print("PS Current Reading:", val)

dmm.write('CONF:VOLTage:DC')
time.sleep(4)
number = float(dmm.query("MEAS?")) # Convert the DMM value to a float
print(number)
print("%.7f" % number) # Format the float value with 7 decimal places

rm.close()
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

By converting the DMM value to a float using float(dmm.query("MEAS?")), you can then format it as a float using the %.7f format specifier without encountering the TypeError.

Make sure that the DMM value you're receiving is a valid numerical value in scientific notation. If the DMM value contains any non-numeric characters or special characters, it may cause issues when converting it to a float.

buran write Jun-06-2023, 06:20 PM:
Please, use proper tags when post code, traceback, output, etc.
See BBcode help for more info.