+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: csv reader Index out of range (/thread-7443.html)
csv reader Index out of range - mathisp64 - Jan-10-2018
I work with large CSV files that often times are too large to open in Excel. I'm trying to write a program that will extract a dozen or so columns from the 1460 columns in the file. I know the header names that I need, but I cannot get my parse program to work.
import csv
datafile = open("C:\\Users\\Administrator\\Desktop\\test.csv",'r')
reader = csv.reader(datafile,delimiter=',')
outfile= open("C:\\Users\\Administrator\\Desktop\\csv.csv",'w')
writer = csv.writer(outfile, delimiter=',')
header=[]
parameter=['Rec #','AirSpd_ADC-1','AirSpd_ADC-2','AirTemp-1','AirTemp-2']
i=0
for row in reader: # get index of header names for iterating
if i == 8:
for name in row:
if name in parameter:
header.append(row.index(name))
i+=1
datafile.seek(0) # reset to use csv reader again
for row in reader:
for col in row:
for indx in header:
outfile.write(row[indx])Error:>>IndexError: string index out of rangeRE: csv reader Index out of range - mpd - Jan-10-2018
You need to format your post with the code tags to make it readable: https://python-forum.io/misc.php?action=help&hid=25
Also, does your IndexError provide a stack trace that says which line is blowing up?
RE: csv reader Index out of range - wavic - Jan-10-2018
Without a proper code formatting, the least I can see is that you are using the reader iterable in two for loops. But after the first one, it is exhausted already.
RE: csv reader Index out of range - mathisp64 - Jan-11-2018
My apologies for the format issue guys. My first post. Mpd,the trace back is to the code: outfile.write(row[indx]). Wavic, you are correct the csv reader is exhausted in the first for reader loop, but after hours of searching online I found a thread with the .seek(0) method. Best I can tell it resets the file to the top so that the reader can be iterated over again. And it works.
After drawing a diagram of the code last night I am certain that there are too many "for" loops in the last for statement.
RE: csv reader Index out of range - buran - Jan-11-2018
you are over-complicating something simple as
import csv
infile = 'infile.csv' # file with large number of columns, dummy filednames field1, field2...field1460
outfile = 'outfile.csv' # you need just 3 out of 1460 columns field2, field10, filed25, in that order
fieldnames = ['field2', 'field10', 'field25']
with open(infile, 'r') as in_f, open(outfile, 'w', newline='') as out_f:
rdr = csv.DictReader(in_f)
wrtr = csv.DictWriter(out_f, fieldnames=fieldnames, extrasaction='ignore')
wrtr.writeheader()
for row in rdr:
wrtr.writerow(row)