Python Forum

Hello,

I have a CSV file that looks something like this:

strings
strings
data (float)
data (float)
data (float)
strings
strings
strings
data (float)
data (float)
...
etc

The file contains many rows and two columns and I would like to load e.g. all the columns in row 1 to 5 in a numpy array. Additionally, I would like to store e.g. row 6-144 in another array (all those rows are data) etc. Is there a way to do this? I tried things like np.loadtxt, csv.reder and pandas.read_csv but I must be missing something. I would prefer to do it without using a loop due to the time consumption when the files get very large.

If someone use MATLAB and know the python way to do this, here's a hint:
MATRIX = dlmread('FILE.csv',',',[5 0 144 1]);
This stores row 5 to 144 and colum 0 to 1 if a matlab code.

Thanks in advance!

Sounds like an assignment.
Give it your best shot and we will be glad to help when you get stuck.

Hello,

It is not an assignment. I'm doing my second years in the applied mechanics masters and we've been using MATLAB the first year. All the companies/industries in our city are using python so I'm learning python during the summers and that's why I try to convert my own MATLAB scripts to python to get a skeleton with hints which I can use later on in life. I'm getting a lot of data files for which I can't control the exact output when doing the simulations which are a tricky part.

I managed to get it to work by doing this:

wholelist = list(csv.reader(open('FILE.csv')))
set1 = np.array(wholelist[5:144],dtype=float)

But that is in two lines and I cant control it as much as I would like. Would be great if someone has another way of doing it.

Thanks!

you can read the data line by line. It's very difficult to attempt without a data file,

but something like this (Totally untested as I have no data file):

import csv

with open('File.csv') as fp:
    reader = csv.reader(fp)
    fields = None
    fields = [field for field in reader]
    # process fields here
    next(reader)

it's late in the evening for Larz and maybe he is tired so he suggest something that will raise exception

import csv
 
with open('File.csv') as fp:
    reader = csv.reader(fp)
    fields = None # no need of this
    fields = [field for field in reader] # you will read the whole file here
    # process fields here
    next(reader) # this will rise an exception because reader is exhausted - you are at the end of the file

better

import csv
 
with open('File.csv') as fp:
    reader = csv.reader(fp)
    for line in reader: # iterate over lines in the file
        # process line here

Buran is correct, it was about 4:00 A.M. and I haden't slept yet.
With his solution, line will be a list of a single csv row.

It seems to do the job well, thanks!