CSV file into a python list is not completely imported - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: CSV file into a python list is not completely imported (/thread-9164.html) |
CSV file into a python list is not completely imported - socertis - Mar-24-2018 i want to read a csv file into a python list. sometimes, it produces an i have searched a lot and tried to use flush() or seek(0,1) functions. i am using the csv built-in module in python. also, i used pandas.read_csv. but always, the list in not complete sometimes. i am working on windows 10 machine, with python2.7 64-bit.here is my code: import csv from io import open j = 0 k=0 csv_filename = 'G:\work and courses\Medrar\heirarchies\hier1.csv' directories_lst = [] def fn(): global j global k global directories_lst try: csvFile = open(csv_filename,'r') directories_lst=[row for row in map(list, csv.reader(csvFile))[1:]] csvFile.close() for i in directories_lst: try: print i except: k = k+1 except: j = j+1 fn()when that function ran for about 1000 times, it produces like 300 exceptions, at the body of the for loop. everybody say that it is a problem with python io module implementation in windows. i need a workaround to solve that problem. that all the content of my .csv file. (it is just a sample) [inline]folder_1,folder_2,folder-3,folder_4
f1,f12,f121,f1211 f1,f13,f131,f1311 f1,f11,f111,f114 f1,f14,f141,f1414 f1,f15,f151,f1514 f2,f21,f212,f2124 f2,f22,f222,f2224 f2,f23,f232,f2324 f2,f24,f242,f2424 f3,f31,f313,f3134 f3,f32,f323,f3234[/inline] RE: CSV file into a python list is not completely imported - Gribouillis - Mar-25-2018 (Mar-24-2018, 05:36 PM)socertis Wrote: everybody say that it is a problem with python io module implementation in windowsWho is everybody? I don't believe that the bug is in the (heavily tested) standard library. There is no mention of unicode in your program. What is the csv file's encoding? are there unicode characters outside the ascii range? RE: CSV file into a python list is not completely imported - socertis - Mar-26-2018 everbody on the internet i mean (stackoverflow and similar websites). the file is ascii encoded. RE: CSV file into a python list is not completely imported - Gribouillis - Mar-26-2018 Have you tried with python 3? RE: CSV file into a python list is not completely imported - nilamo - May-04-2018 (Mar-24-2018, 05:36 PM)socertis Wrote: when that function ran for about 1000 times, it produces like 300 exceptions, at the body of the for loopI find that hard to believe, as the file has been fully read and closed by the time you're at the for loop. How are you running it 1000 times? |