Python Forum
matplotlib help with reading values from CSV file
Thread Rating:
  • 3 Vote(s) - 3.33 Average
  • 1
  • 2
  • 3
  • 4
  • 5
matplotlib help with reading values from CSV file
#1
Hi,

I am trying to add a feature to a python script written by someone else who used to work at the same company as me. I am by no means a python expert but I have a general understanding.

The software basically looks at a serial input string, and logs this value to a CSV file, and also shows a live trend graph of the value. What I now need to do is add a button to the GUI which generates a graph of the contents of the CSV file.

I have managed to add the button, and I have entered the code to take the values from the CSV file, this works fine with a test file (two columns of numbers). The problem I have is, that the "real" value which will be logged into my CSV file is a value in metric tonnes, the way the original python script is written is that it stores the values in the CSV file as a number with a T unit at the end. So when i try and plot using matplotlib I get an error as it isn't a numeric value. For example my CSV file may look like this (where the first value is time):

16:10:24,1.00T
16:10:25,1.10T
16:10:26,1.15T
16:10:27,1.16T
16:10:28,1.14T

Can anybody tell me how I can generate a graph using matplotlib which will ignore the T unit notation in the CSV file?

This is my current code:

def graph(self):
x,y = np.loadtxt('test.csv', delimiter=',', unpack=True)
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()

Thanks
Reply
#2
You can post process y, which should be the metric tons.

def conv_to_float(values):
    return [float(v.rstrip('T')) for v in values]
Or just where you read the values of your csv. For each value you just rstrip 'T' or replace 'T' with an empty string. After this you convert it to a float or Decimal. Decimal is able to represent decimal Values. Float is representing binary values. For plotting the values, float is good enough. If you need precise calculations based on the values, you should better use Decimal.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#3
@ DeaD_EyE

I'm sorry but I don't know how to integrate that into my code. Would this be done outside of the 'def graph' I have?

thanks


I have actually changed my code slightly to:

def graph(self):
x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,])
y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,])
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()
Reply
#4
First you should read this and fix your indentation: https://python-forum.io/misc.php?action=help&hid=25
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#5
def graph(self):
    x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,])
    y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,])
    plt.plot(x,y)
    plt.xlabel('Time')
    plt.ylabel('Load Value')
    plt.title('Logged Load\n')
    plt.show()
Reply
#6
Interesting. The behavior of np.loadtext is strange. I think it's used more for lists with numbers. It is possible to do the conversion with this function. But you must also define a datatype. I'm not so familiar with numpy.

I like the classic way with modules from the stdlib of Python.
Here a very compact example with csv and datetime:

import datetime
import csv
import matplotlib.pyplot as plt


def reader(csv_file, base_date, skiprows=2):
    with open(csv_file) as fd:
        reader = csv.reader(fd, delimiter=',')
        for _ in range(skiprows):
            next(reader)
        for time, tons in reader:
            time = time.split(':')
            hour, minute, second = map(int, time)
            dtime = base_date.replace(hour=hour, minute=minute, second=second, microsecond=0)
            tons = float(tons.rstrip('T'))
            yield dtime, tons

base_date = datetime.datetime.now()
generator = reader('data.csv', base_date)
x, y = zip(*generator) # transpose, like unpack in numpy.loadtext

plt.title('Time / Tons')
plt.xlabel('Time')
plt.ylabel('Tons')
plt.plot_date(x, y, fmt='-o', color='green')
plt.show()
I think this is not the best approach. Matplotlib has for example helper functions for dates.
If you read my other posts, you'll notice that i like generators.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#7
(Aug-21-2017, 08:26 AM)cps13 Wrote: 16:10:24,1.00T
16:10:25,1.10T
16:10:26,1.15T
16:10:27,1.16T
16:10:28,1.14T

def graph(self):
x,y = np.loadtxt('test.csv', delimiter=',', unpack=True)
plt.plot(x,y)
plt.xlabel('Time')
plt.ylabel('Load Value')
plt.title('Logged Load\n')
plt.show()

I have not tested in in this code, but I suggest a quickfix like this: (given that the T is always at the end of the string.

plt.plot(x,y) ----> plt.plot(x,float(y[:-1]))
[:-1] removes the last character of the string before float turns it into a decimal number.

Hope that works as a simple solution.

- Andre
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Reading large crapy text file in anaconda to profile data syamatunuguntla 0 811 Nov-18-2022, 06:15 PM
Last Post: syamatunuguntla
  Matplotlib scatter plot in loop with None values ivan_sc 1 2,236 Nov-04-2021, 11:25 PM
Last Post: jefsummers
  [solved] Save a matplotlib figure into hdf5 file paul18fr 1 2,472 Jun-08-2021, 05:58 PM
Last Post: paul18fr
  How to form a dataframe reading separate dictionaries from .txt file? Doug 1 4,198 Nov-09-2020, 09:24 AM
Last Post: PsyPy
  reading tab file Mandiph 1 2,152 Sep-05-2019, 01:03 PM
Last Post: ThomasL
  Reading values into df.loc silkpantsman 0 1,465 Jul-26-2019, 08:34 AM
Last Post: silkpantsman
  Reading a .dat file in Python mohd_umair 4 23,480 Apr-24-2019, 12:07 PM
Last Post: mohd_umair
  Reading time steps from nc file ankurk017 1 2,547 Jul-16-2018, 07:06 PM
Last Post: woooee
  reading, modifying and writing json file metalray 2 10,884 Jun-06-2018, 03:09 PM
Last Post: metalray
  Reading json file as pandas data frame? Alberto 1 8,309 Feb-05-2018, 12:43 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020