+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: issue with converting a scientific notation to standard notation (/thread-40092.html)
issue with converting a scientific notation to standard notation - thomaswfirth - Jun-01-2023
I am reading the value from a DMM and the returned number is in Scientific notation. When I try to convert that number to get a decimal number, I get an error.
If I run the same script but cut and paste the reported DMM number into "number = 9.985660E-01" instead it works fine.
Are there some hidden characters I'm not seeing.? Is there a better way to convert to a standard decimal number from the DMM number?
Using number from DMM:
1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 # number = 9.985660E-01
22 print("%.7f"%number) # Does not work using number in from DMM
23 rm.close()
PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65078' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.925
9.349951E-01 #Value from DMM used in line 22
Traceback (most recent call last):
File "c:\Users\thoma\Desktop\Python\xxxx.py", line 22, in <module>
print("%.7f"%number) # Does not work using number in from DMM
~~~~~~^~~~~~~
TypeError: must be real number, not str
PS C:\Users\thoma\Desktop\Python>
1 import pyvisa
2 import time
3 rm = pyvisa.ResourceManager()
4 # print(rm.list_resources())
5 ps = rm.open_resource('ASRL8::INSTR')
6 ps.baud_rate = 115200
7 dmm = rm.open_resource('ASRL10::INSTR')
8 dmm.baud_rate = 115200
9
10 ps.write('VSET:1.0?')
11 ps.write('OUT:1')
12 ps.write('IOUT:1?')
13 time.sleep(3)
14 val = ps.query('IOUT?')
15 print("PS Current Reading:",val)
16
17 dmm.write('CONF:VOLTage:DC')
18 time.sleep(4)
19 number = (dmm.query("MEAS?"))
20 print(number)
21 number = 9.985660E-01
22 print("%.7f"%number) # uses number in line 21 and works fine
23rm.close()
PS C:\Users\thoma\Desktop\Python> c:; cd 'c:\Users\thoma\Desktop\Python'; & 'C:\Users\thoma\AppData\Local\Programs\Python\Python311\python.exe' 'c:\Users\thoma\.vscode\extensions\ms-python.python-2023.8.0\pythonFiles\lib\python\debugpy\adapter/../..\debugpy\launcher' '65120' '--' 'c:\Users\thoma\Desktop\Python\xxxx.py'
PS Current Reading: 00.930
9.346759E-01
0.9985660 # Converted number from line 21
PS C:\Users\thoma\Desktop\Python>
RE: issue with converting a scientific notation to standard notation - deanhystad - Jun-01-2023
In your code, val and number are strings, not a number. To convert numeric strings to numbers use int(str) or float(str).
val = float(ps.query('IOUT?'))
number = float(dmm.query("MEAS?"))
RE: issue with converting a scientific notation to standard notation - thomaswfirth - Jun-01-2023
Wow, something so simple that solved the issue, thanks. I have a lot to learn but it is keeping my mind working.
RE: issue with converting a scientific notation to standard notation - buran - Jun-01-2023
@thomaswfirth, please, use BBCode tags and also DO NOT include line numbers with your code
RE: issue with converting a scientific notation to standard notation - rajeshgk - Jun-06-2023
The error you're encountering is due to the fact that the value returned from the DMM is a string, and you're trying to format it as a float using the %.7f format specifier. To resolve this issue, you need to convert the string to a float before formatting it.
Here's an updated version of your code that converts the DMM value to a decimal number and formats it as a float:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
import pyvisa
import time
rm = pyvisa.ResourceManager()
ps = rm.open_resource('ASRL8::INSTR')
ps.baud_rate = 115200
dmm = rm.open_resource('ASRL10::INSTR')
dmm.baud_rate = 115200
ps.write('VSET:1.0?')
ps.write('OUT:1')
ps.write('IOUT:1?')
time.sleep(3)
val = ps.query('IOUT?')
print("PS Current Reading:", val)
dmm.write('CONF:VOLTage:DC')
time.sleep(4)
number = float(dmm.query("MEAS?")) # Convert the DMM value to a float
print(number)
print("%.7f" % number) # Format the float value with 7 decimal places
rm.close()
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
By converting the DMM value to a float using float(dmm.query("MEAS?")), you can then format it as a float using the %.7f format specifier without encountering the TypeError.
Make sure that the DMM value you're receiving is a valid numerical value in scientific notation. If the DMM value contains any non-numeric characters or special characters, it may cause issues when converting it to a float.