Python Forum
save in CSV and plotting in real time
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
save in CSV and plotting in real time
#1
import serial                                         # import Serial Library
import numpy as np                                    # Import numpy
import adafruit_max31856
import matplotlib.pyplot as plt                       #import matplotlib library
from drawnow import *
import datetime
import math
import csv


Temp1=[]
Temp2=[]
plt.ion()
cnt=0

def makeFig():  
    plt.title('My Super Beautiful Live Streaming Sensor Data :-)')      #Plot the title
    plt.grid(True)#Turn the grid on
    plt.xlabel('Time')                                #set xlabels
    plt.ylabel('Temperature')                         #Set ylabels
    plt.plot(Temp1, 'ro-',label='Temp1')              #plot the temperature
    plt.plot(Temp2, 'b^-',label='Temp2')
    plt.legend(loc='upper right')                     #plot the legend
    plt.show()

## user-defined params

serialPort = 'COM3'                                                  ## the serial device
path = 'C:\\Users\\Alphinity\\Desktop\\python3\\'                    ## the output file path
outputFile = "signalSerial.csv"                                      ## the output file name


ser = serial.Serial('COM3', 115200)
outputFile = path + outputFile


f = open(outputFile,'w')             
f.write(outputFile)                  
f.close()


print ("Writing the serial stream into file: " + outputFile)
print (" [to see the stream: tail -f '+outputFile+' ]")
print (" [to exit: ctrl+c (the elegant way :) ]")

while True:                               # While loop that loops forever
    line = ser.readline()
    string = line
    Temp1.append(string[0])                      #Build our tempF array by appending temp readings
    Temp2.append(string[1])
    drawnow(makeFig)
    plt.pause(.000001)
    cnt=cnt+1
    if(cnt>50):                            #If you have 50 or more points, delete the first one from the array
        Temp1.pop(0)                       #This allows us to just see the last 50 data points
        Temp2.pop(0)
    if (string != ''):
      f = open(outputFile, 'a')
      f.write(str(string))
      f.write("\n")
      f.close()
      print(string.decode())               ## ctrl+c to stop the code

valueRead.decode().strip()
---------------------------------------
ERROR

When I run and is created the file signalSerial.csv I get not only the temperatures but is like that:
Error:
b'26.72 , 26.92\r\n' b'26.84 , 27.02\r\n' b'26.85 , 27.01\r\n' b'26.73 , 26.91\r\n' b'26.77 , 26.95\r\n' b'26.77 , 27.03\r\n' b'26.76 , 27.01\r\n' b'26.73 , 26.98\r\n'
Error:
The second erro is that in the graph both the temperatures appear higher than 60 degrees ,and is impossible.
Thank you so much for the help
Reply
#2
import serial                                          
import numpy as np                                    
import adafruit_max31856
import matplotlib.pyplot as plt                        
from drawnow import *
import datetime
import math
import csv
 
 
Temp1=[]
Temp2=[]
plt.ion()
cnt=0
 
def makeFig():  
    plt.title('My Super Beautiful Live Streaming Sensor Data :-)')      
    plt.grid(True)#Turn the grid on
    plt.plot()
    plt.xlabel('Time')                                
    plt.ylabel('Temperature')                          
    plt.plot(Temp1, 'ro-',label='Temp1')              
    plt.plot(Temp2, 'b^-',label='Temp2')
    plt.legend(loc='upper right')                      
    plt.show()
 
 
 
## user-defined params
 
serialPort = 'COM4'                                                  
path = 'C:\\Users\\Alphinity\\Desktop\\python3\\'              
outputFile = "signalSerial.csv"                    
 
 
ser = serial.Serial('COM4', 115200)
outputFile = path + outputFile
 
 
f = open(outputFile,'w')            
f.write(outputFile)                  
f.close()                          
 
print ("Writing the serial stream into file: " + outputFile)
print (" [to see the stream: tail -f '+outputFile+' ]")
print (" [to exit: ctrl+c (the elegant way :) ]")
 
 
while True:                                
    line = ser.readline()
    string = line
    Temp1.append(string[0])                      
    Temp2.append(string[1])
    drawnow(makeFig)
    plt.pause(.000001)
    cnt=cnt+1
    if(cnt>50):                            
        Temp1.pop(0)                        
        Temp2.pop(0)
    if (string != ''):
      f = open(outputFile, 'a')
      f.write(str(string))
      f.write("\n")
      f.close()
      print(string.decode())                
 
valueRead.decode().strip()
Error:
When I go to open the file .csv i get this reading, but in the serial are shown perfectly, how I can fix this problem ? b'22.47 , 21.91\r\n' b'22.50 , 21.94\r\n' b'22.49 , 21.98\r\n' b'22.52 , 22.09\r\n' b'22.62 , 22.03\r\n' b'22.58 , 22.02\r\n' b'22.52 , 21.97\r\n' b'22.45 , 21.99\r\n' b'22.52 , 21.89\r\n' b'22.59 , 21.97\r\n' b'22.61 , 21.95\r\n' b'22.59 , 21.85\r\n' b'22.58 , 21.96\r\n' b'22.63 , 21.87\r\n' b'22.69 , 21.99\r\n' b'22.55 , 21.92\r\n' b'22.59 , 21.94\r\n' b'22.55 , 21.89\r\n' b'22.58 , 21.92\r\n'
Thank you so much for the help
Reply
#3
Please, don't start new threads continuously.
I thought you've already got answer in the first thread you post
you need to decode the response from the sensor. Eventually parse the resulting string and save to file. What exactly is not clear? you decode the response at the end, just for the print.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#4
Really sorry, it's just because I don't know where I have to decode exactly, I tried every where but I just receive only error message, for this I decide to write here, but sorry I thought that i could write in the morning.
Ok I will not do another post
Reply
#5
Here is untested code with some changes to make it better
import serial                                          
import numpy as np                                    
import adafruit_max31856
import matplotlib.pyplot as plt                        
from drawnow import drawnow
import datetime
import math
import csv
import os
from collections import deque
from decimal import Decimal

# instead of list, use collections.deque
# deque has optional maxlen argument that set max number of elements
# if append new element and len exceed maxlen elemnt from the other side is removed
temp1 = deque(maxlen=50) # make deque with length 50
temp2 = deque(maxlen=50) # make deque with length 50
plt.ion()

def make_fig():  
    plt.title('My Super Beautiful Live Streaming Sensor Data :-)')      
    plt.grid(True)#Turn the grid on
    plt.plot()
    plt.xlabel('Time')                                
    plt.ylabel('Temperature')                          
    plt.plot(temp1, 'ro-',label='Temp1')              
    plt.plot(temp2, 'b^-',label='Temp2')
    plt.legend(loc='upper right')                      
    plt.show()


if __name__ == '__main__':
    ## user-defined params
    serialPort = 'COM4'                                                  
    out_path = r'C:\Users\Alphinity\Desktop\python3\'              
    file_name = "signalSerial.csv"                    
    ser = serial.Serial('COM4', 115200)
    out_file = os.path.join(out_path, file_name)

    print (f"Writing the serial stream into file: {out_file}")
    print (f" [to see the stream: tail -f {out_file} ]")
    print (" [to exit: ctrl+c (the elegant way :) ]")
    while True:                                
        line = ser.readline().decode().strip()
        if line:
            line = line.split(',') # get list of 2 strings
            line = [Decimal(temp.strip()) for temp in line] # make line list of two Decimal objects
            t1, t2 = line # unpack line into two variables t1 and t2
            temp1.append(t1)                      
            temp2.append(t2)
            drawnow(make_fig)
            plt.pause(.000001)
            with open(out_fiile, 'a') as f:
                wrtr = csv.writer(f)
                wrtr.writerow(line)

            # as alternative to last 3 lines
            with open(out_fiile, 'a') as f:
                f.write(f'{t1},{t2}\n')
I didn't change your drawing of the plot and definitely program structure could be better
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#6
I changed the things as you wrote, really thank you so much for the answer, unfortunately I tried until now, but I'm getting always this error:

Error:
Traceback (most recent call last): File "C:\Users\Alphinity\Desktop\python3\passato.py", line 59, in <module> line = [Decimal(temp.strip()) for temp in line] File "C:\Users\Alphinity\Desktop\python3\passato.py", line 59, in <listcomp> line = [Decimal(temp.strip()) for temp in line] decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
Reply
#7
You need to learn how the debug small programs (there is nice link in my signature)
Obviously you get some data that it fails to convert to Decimal number. Do you get any data written in the csv file or error happens from beginning?
Let's add a print function to see what line looks like before the error
add
print(repr(line))
once before each of lines 46 and 47
and let me know what line looks like just before the error
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#8
Before the line 46 i get this error:
Error:
File "C:\Users\Alphinity\Desktop\python3\passato.py", line 49, in <listcomp> line = [Decimal(temp.strip()) for temp in line] decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
Before the line 47 I'm getting this:
Error:
Traceback (most recent call last): File "C:\Users\Alphinity\Desktop\python3\passato.py", line 49, in <module> line = [Decimal(temp.strip()) for temp in line] File "C:\Users\Alphinity\Desktop\python3\passato.py", line 49, in <listcomp> line = [Decimal(temp.strip()) for temp in line] decimal.InvalidOperation: [<class 'decimal.ConversionSyntax'>]
Reply
#9
of course you will get that error again - we did nothing to prevent or silence it. the question is what is printed before the error...i.e. what is line when cause error...
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  how to save to multiple locations during save cubangt 1 509 Oct-23-2023, 10:16 PM
Last Post: deanhystad
  Plotting by Time, error mansoorahs 1 707 May-16-2023, 09:46 AM
Last Post: Larz60+
  Non-blocking real-time plotting slow_rider 5 3,455 Jan-07-2023, 09:47 PM
Last Post: woooee
  Real time database satyanarayana 3 1,623 Feb-16-2022, 01:37 PM
Last Post: buran
  Real time data satyanarayana 3 20,255 Feb-16-2022, 07:46 AM
Last Post: satyanarayana
  Real time Detection and Display Gilush 0 1,764 Feb-05-2022, 08:28 PM
Last Post: Gilush
  time setup for realtime plotting of serial datas at high sampling rate alice93 6 3,648 Jan-07-2022, 05:41 PM
Last Post: deanhystad
  Real-Time output of server script on a client script. throwaway34 2 2,010 Oct-03-2021, 09:37 AM
Last Post: ibreeden
  Real Time Audio Processing with Python Sound-Device not working Slartybartfast 2 3,882 Mar-14-2021, 07:20 PM
Last Post: Slartybartfast
  Plotting A Time Series With Shaded Recession Bars adamszymanski 1 3,119 Jan-24-2021, 09:08 PM
Last Post: nealc

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020