matplotlib help with reading values from CSV file - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: matplotlib help with reading values from CSV file (/thread-4494.html) |
matplotlib help with reading values from CSV file - cps13 - Aug-21-2017 Hi, I am trying to add a feature to a python script written by someone else who used to work at the same company as me. I am by no means a python expert but I have a general understanding. The software basically looks at a serial input string, and logs this value to a CSV file, and also shows a live trend graph of the value. What I now need to do is add a button to the GUI which generates a graph of the contents of the CSV file. I have managed to add the button, and I have entered the code to take the values from the CSV file, this works fine with a test file (two columns of numbers). The problem I have is, that the "real" value which will be logged into my CSV file is a value in metric tonnes, the way the original python script is written is that it stores the values in the CSV file as a number with a T unit at the end. So when i try and plot using matplotlib I get an error as it isn't a numeric value. For example my CSV file may look like this (where the first value is time): 16:10:24,1.00T 16:10:25,1.10T 16:10:26,1.15T 16:10:27,1.16T 16:10:28,1.14T Can anybody tell me how I can generate a graph using matplotlib which will ignore the T unit notation in the CSV file? This is my current code: def graph(self): x,y = np.loadtxt('test.csv', delimiter=',', unpack=True) plt.plot(x,y) plt.xlabel('Time') plt.ylabel('Load Value') plt.title('Logged Load\n') plt.show() Thanks RE: matplotlib help with reading values from CSV file - DeaD_EyE - Aug-21-2017 You can post process y, which should be the metric tons. def conv_to_float(values): return [float(v.rstrip('T')) for v in values]Or just where you read the values of your csv. For each value you just rstrip 'T' or replace 'T' with an empty string. After this you convert it to a float or Decimal. Decimal is able to represent decimal Values. Float is representing binary values. For plotting the values, float is good enough. If you need precise calculations based on the values, you should better use Decimal. RE: matplotlib help with reading values from CSV file - cps13 - Aug-21-2017 @ DeaD_EyE I'm sorry but I don't know how to integrate that into my code. Would this be done outside of the 'def graph' I have? thanks I have actually changed my code slightly to: def graph(self): x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,]) y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,]) plt.plot(x,y) plt.xlabel('Time') plt.ylabel('Load Value') plt.title('Logged Load\n') plt.show() RE: matplotlib help with reading values from CSV file - DeaD_EyE - Aug-21-2017 First you should read this and fix your indentation: https://python-forum.io/misc.php?action=help&hid=25 RE: matplotlib help with reading values from CSV file - cps13 - Aug-21-2017 def graph(self): x = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[0,]) y = np.loadtxt('log0.csv', delimiter=',', skiprows=2,usecols=[1,]) plt.plot(x,y) plt.xlabel('Time') plt.ylabel('Load Value') plt.title('Logged Load\n') plt.show() RE: matplotlib help with reading values from CSV file - DeaD_EyE - Aug-22-2017 Interesting. The behavior of np.loadtext is strange. I think it's used more for lists with numbers. It is possible to do the conversion with this function. But you must also define a datatype. I'm not so familiar with numpy. I like the classic way with modules from the stdlib of Python. Here a very compact example with csv and datetime: import datetime import csv import matplotlib.pyplot as plt def reader(csv_file, base_date, skiprows=2): with open(csv_file) as fd: reader = csv.reader(fd, delimiter=',') for _ in range(skiprows): next(reader) for time, tons in reader: time = time.split(':') hour, minute, second = map(int, time) dtime = base_date.replace(hour=hour, minute=minute, second=second, microsecond=0) tons = float(tons.rstrip('T')) yield dtime, tons base_date = datetime.datetime.now() generator = reader('data.csv', base_date) x, y = zip(*generator) # transpose, like unpack in numpy.loadtext plt.title('Time / Tons') plt.xlabel('Time') plt.ylabel('Tons') plt.plot_date(x, y, fmt='-o', color='green') plt.show()I think this is not the best approach. Matplotlib has for example helper functions for dates. If you read my other posts, you'll notice that i like generators. RE: matplotlib help with reading values from CSV file - andre - Aug-22-2017 (Aug-21-2017, 08:26 AM)cps13 Wrote: 16:10:24,1.00T I have not tested in in this code, but I suggest a quickfix like this: (given that the T is always at the end of the string. plt.plot(x,y) ----> plt.plot(x,float(y[:-1])) [:-1] removes the last character of the string before float turns it into a decimal number. Hope that works as a simple solution. - Andre |