Python Forum
Python: Automated Script to Read Multiple Files in Respective Matrices
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python: Automated Script to Read Multiple Files in Respective Matrices
#1
Hello,

I am a beginner and trying to automate a simple problem. I am reading multiple files one-by-one and saving the data in respective matrices. I would like to make my code short by automating (e.g., by looping).

Would appreciate any suggestions, the files are delimited and named numerically (File_1, File_2, etc.). Here is what I am doing as of now:

    file1 = np.loadtxt('File_1.txt', delimiter=',')
    file2 = np.loadtxt('File_2.txt', delimiter=',')
    ...

    matrix1 = file1[:,:]
    matrix2 = file2[:,:]
    ...
Reply
#2
Instead of storing the data in regular variables (like matrix1), store them in a list or a dictionary.

You can create the filenames in a loop by range(), but it might be better to use os.listdir() or glob.glob() to get a list of all the files that exist that match what you want.

matrix_data = {}
for file_index in range(1,10):
    filename = f"File_{file_index}.txt"
    file = np.loadtxt(filename, delimiter=',')
    matrix_data[filename] = file[:,:]
# can refer to matrix_data["File_1.txt"] for the data...
Reply
#3
Hi Bowlofred:

Thanks, so if I am interested in accessing a particular element is there any clever way? I am doing this:

a = matrix_data["file2.txt"]
print(a[1,1])
Reply
#4
If you have several operations on the data, your way is fine. You can temporarily (or in a loop) assign it to a shorter name (like "a" in your example) and then do your work.

But if you just need the element, you can do it in one line instead of two with

print(matrix_data["file2.txt"][1,1])
Reply
#5
Thanks. Yes I am doing quite some operations and saving the data in separate variables is what I desire.

One last thing though. Is there a possibility to generate the variable name using loop (a1, a2, etc)? I want to do something like given below. Only change I need is to automatically let python change a1 to a2 in the second iteration and so on. Hence: file1->a1 ; file2->a2 ; ...

filenames = ['file1.txt', 'file2.txt']

for i in range(0,2):
    file = np.loadtxt(filenames[i], delimiter=',')
    a1 = file[:,:]
    print(a1)
Reply
#6
Why do you need to change the variable? Just leave it as "a" each time through the loop as you deal with each matrix in turn.

In general you don't want to create dynamic top-level variables. If you need a dynamic name, make it the key of a dictionary.
Reply
#7
I actually need to carry out operations on each element of a1 with each element of a2 after they are fully read. Would appreciate if you can elaborate more on how to assign keys in this particular case?
Reply
#8
If there's only two things, just assign them outside the loop. If there's more than two things, how do you decide which is a1 and which is a2?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Eliminate entering QR - Whatsapp web automated by selenium akanowhere 1 3,016 Jan-21-2024, 01:12 PM
Last Post: owalahtole
  python convert multiple files to multiple lists MCL169 6 1,431 Nov-25-2023, 05:31 AM
Last Post: Iqratech
  splitting file into multiple files by searching for string AlphaInc 2 812 Jul-01-2023, 10:35 PM
Last Post: Pedroski55
  Is there a *.bat DOS batch script to *.py Python Script converter? pstein 3 3,000 Jun-29-2023, 11:57 AM
Last Post: gologica
Question Need help for a python script to extract information from a list of files lephunghien 6 1,032 Jun-12-2023, 05:40 PM
Last Post: snippsat
  script to calculate data in csv-files ledgreve 0 1,054 May-19-2023, 07:24 AM
Last Post: ledgreve
  Merging multiple csv files with same X,Y,Z in each Auz_Pete 3 1,084 Feb-21-2023, 04:21 AM
Last Post: Auz_Pete
  unittest generates multiple files for each of my test case, how do I change to 1 file zsousa 0 918 Feb-15-2023, 05:34 PM
Last Post: zsousa
  How to read in mulitple files efficiently garynewport 3 844 Jan-27-2023, 10:44 AM
Last Post: DeaD_EyE
  Question about Creating an Automated Process in Python Supratik1234 0 712 Jan-13-2023, 08:29 PM
Last Post: Supratik1234

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020