Mar-14-2018, 06:12 PM
Whenever I try to make a list of >5,000 items in python idle, it freezes and becomes unresponsive. Is there a way to work with large python lists without using a lot of data or memory?
I was thinking to create a list you could have python print each element of your list one by one into a text file, and it would print each item in a new line. This would create a text file with 10,000+ lines with 4-5 words in each line. For accessing the list, you could have python pull one item at a time, and just iterate over every line of the text file.
My question is, is the above method of dealing with lists I described as doable? If so, can someone direct me to a module/tutorial involving that? If not, is there a better way to deal with large lists?
BTW I looked at numpy arrays, but I don't if I can use that as I will not know the size of the list until it is created. I only know that it will be in the ballpark of 10,000+ items.
I was thinking to create a list you could have python print each element of your list one by one into a text file, and it would print each item in a new line. This would create a text file with 10,000+ lines with 4-5 words in each line. For accessing the list, you could have python pull one item at a time, and just iterate over every line of the text file.
My question is, is the above method of dealing with lists I described as doable? If so, can someone direct me to a module/tutorial involving that? If not, is there a better way to deal with large lists?
BTW I looked at numpy arrays, but I don't if I can use that as I will not know the size of the list until it is created. I only know that it will be in the ballpark of 10,000+ items.