Bottom Page

Thread Rating:
  • 3 Vote(s) - 3.33 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 matplotlib help with reading values from CSV file
#1
Hi,

I am trying to add a feature to a python script written by someone else who used to work at the same company as me. I am by no means a python expert but I have a general understanding.

The software basically looks at a serial input string, and logs this value to a CSV file, and also shows a live trend graph of the value. What I now need to do is add a button to the GUI which generates a graph of the contents of the CSV file.

I have managed to add the button, and I have entered the code to take the values from the CSV file, this works fine with a test file (two columns of numbers). The problem I have is, that the "real" value which will be logged into my CSV file is a value in metric tonnes, the way the original python script is written is that it stores the values in the CSV file as a number with a T unit at the end. So when i try and plot using matplotlib I get an error as it isn't a numeric value. For example my CSV file may look like this (where the first value is time):

16:10:24,1.00T
16:10:25,1.10T
16:10:26,1.15T
16:10:27,1.16T
16:10:28,1.14T

Can anybody tell me how I can generate a graph using matplotlib which will ignore the T unit notation in the CSV file?

This is my current code:

def graph(self):
x,y = np.loadtxt('test.csv', delimiter=',', unpack=True)
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()

Thanks
Quote
#2
You can post process y, which should be the metric tons.

def conv_to_float(values):
    return [float(v.rstrip('T')) for v in values]
Or just where you read the values of your csv. For each value you just rstrip 'T' or replace 'T' with an empty string. After this you convert it to a float or Decimal. Decimal is able to represent decimal Values. Float is representing binary values. For plotting the values, float is good enough. If you need precise calculations based on the values, you should better use Decimal.
My code examples are always for Python >=3.6.0
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Quote
#3
@ DeaD_EyE

I'm sorry but I don't know how to integrate that into my code. Would this be done outside of the 'def graph' I have?

thanks


I have actually changed my code slightly to:

def graph(self):
x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,])
y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,])
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()
Quote
#4
First you should read this and fix your indentation: https://python-forum.io/misc.php?action=help&hid=25
My code examples are always for Python >=3.6.0
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Quote
#5

def graph(self):
    x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,])
    y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,])
    plt.plot(x,y)
    plt.xlabel('Time')
    plt.ylabel('Load Value')
    plt.title('Logged Load\n')
    plt.show()

Quote
#6
Interesting. The behavior of np.loadtext is strange. I think it's used more for lists with numbers. It is possible to do the conversion with this function. But you must also define a datatype. I'm not so familiar with numpy.

I like the classic way with modules from the stdlib of Python.
Here a very compact example with csv and datetime:

import datetime
import csv
import matplotlib.pyplot as plt


def reader(csv_file, base_date, skiprows=2):
    with open(csv_file) as fd:
        reader = csv.reader(fd, delimiter=',')
        for _ in range(skiprows):
            next(reader)
        for time, tons in reader:
            time = time.split(':')
            hour, minute, second = map(int, time)
            dtime = base_date.replace(hour=hour, minute=minute, second=second, microsecond=0)
            tons = float(tons.rstrip('T'))
            yield dtime, tons

base_date = datetime.datetime.now()
generator = reader('data.csv', base_date)
x, y = zip(*generator) # transpose, like unpack in numpy.loadtext

plt.title('Time / Tons')
plt.xlabel('Time')
plt.ylabel('Tons')
plt.plot_date(x, y, fmt='-o', color='green')
plt.show()
I think this is not the best approach. Matplotlib has for example helper functions for dates.
If you read my other posts, you'll notice that i like generators.
My code examples are always for Python >=3.6.0
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Quote
#7
(Aug-21-2017, 08:26 AM)cps13 Wrote: 16:10:24,1.00T
16:10:25,1.10T
16:10:26,1.15T
16:10:27,1.16T
16:10:28,1.14T

def graph(self):
x,y = np.loadtxt('test.csv', delimiter=',', unpack=True)
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()

I have not tested in in this code, but I suggest a quickfix like this: (given that the T is always at the end of the string.

plt.plot(x,y) ----> plt.plot(x,float(y[:-1]))
[:-1] removes the last character of the string before float turns it into a decimal number.

Hope that works as a simple solution.

- Andre
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  reading tab file Mandiph 1 351 Sep-05-2019, 01:03 PM
Last Post: ThomasL
  Reading values into df.loc silkpantsman 0 246 Jul-26-2019, 08:34 AM
Last Post: silkpantsman
  Reading a .dat file in Python mohd_umair 4 11,479 Apr-24-2019, 12:07 PM
Last Post: mohd_umair
  Reading time steps from nc file ankurk017 1 780 Jul-16-2018, 07:06 PM
Last Post: woooee
  reading, modifying and writing json file metalray 2 6,977 Jun-06-2018, 03:09 PM
Last Post: metalray
  Selecting special values in txt file Scientifix 1 687 May-01-2018, 04:18 PM
Last Post: killerrex
  Reading json file as pandas data frame? Alberto 1 5,649 Feb-05-2018, 12:43 AM
Last Post: snippsat
  Need help reading in a binary file shaynehansen 2 1,670 Jun-22-2017, 10:04 AM
Last Post: zivoni
  How to form a dataframe reading separate dictionaries from .txt file? Doug 0 1,342 Jan-23-2017, 01:32 AM
Last Post: Doug

Forum Jump:


Users browsing this thread: 1 Guest(s)