Python Forum

I am reading the value from a DMM and the returned number is in Scientific notation. When I try to convert that number to get a decimal number, I get an error.

If I run the same script but cut and paste the reported DMM number into "number = 9.985660E-01" instead it works fine.
Are there some hidden characters I'm not seeing.? Is there a better way to convert to a standard decimal number from the DMM number?

Using number from DMM:
1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 # number = 9.985660E-01
22 print("%.7f"%number) # Does not work using number in from DMM
23 rm.close()

PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65078' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.925

9.349951E-01 #Value from DMM used in line 22

Traceback (most recent call last):
File "c:\Users\thoma\Desktop\Python\xxxx.py", line 22, in <module>
print("%.7f"%number) # Does not work using number in from DMM
~~~~~~^~~~~~~
TypeError: must be real number, not str
PS C:\Users\thoma\Desktop\Python>


1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 number = 9.985660E-01
22 print("%.7f"%number) # uses number in line 21 and works fine
23rm.close()

PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65120' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.930

9.346759E-01

0.9985660 # Converted number from line 21
PS C:\Users\thoma\Desktop\Python>

In your code, val and number are strings, not a number. To convert numeric strings to numbers use int(str) or float(str).

val = float(ps.query('IOUT?'))
number = float(dmm.query("MEAS?"))

Wow, something so simple that solved the issue, thanks. I have a lot to learn but it is keeping my mind working.

@thomaswfirth, please, use BBCode tags and also DO NOT include line numbers with your code

The error you're encountering is due to the fact that the value returned from the DMM is a string, and you're trying to format it as a float using the %.7f format specifier. To resolve this issue, you need to convert the string to a float before formatting it.

Here's an updated version of your code that converts the DMM value to a decimal number and formats it as a float:

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

import pyvisa
import time

rm = pyvisa.ResourceManager()
ps = rm.open_resource('ASRL8::INSTR')
ps.baud_rate = 115200
dmm = rm.open_resource('ASRL10::INSTR')
dmm.baud_rate = 115200

ps.write('VSET:1.0?')
ps.write('OUT:1')
ps.write('IOUT:1?')
time.sleep(3)
val = ps.query('IOUT?')
print("PS Current Reading:", val)

dmm.write('CONF:VOLTage:DC')
time.sleep(4)
number = float(dmm.query("MEAS?")) # Convert the DMM value to a float
print(number)
print("%.7f" % number) # Format the float value with 7 decimal places

rm.close()
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

By converting the DMM value to a float using float(dmm.query("MEAS?")), you can then format it as a float using the %.7f format specifier without encountering the TypeError.

Make sure that the DMM value you're receiving is a valid numerical value in scientific notation. If the DMM value contains any non-numeric characters or special characters, it may cause issues when converting it to a float.