Python Forum
.py pandas matplotlib .xlsx files
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
.py pandas matplotlib .xlsx files
#1
I am facing a problem with my Python script and I hope someone can help me with it.

I have a list of file names, such as f1, f2, f3, etc. These files are in the xlsx format, and they are located in different folders. "THE REAL PROBLEM IS THIS F FILES HAS REACHED MORE THEN 450 IN JUST 3 MONTHS TIME AND ITS JUST ADDING HOW TO SIMPLIFY IT THIS"

I want to divide these files into two segments, red and green, and I want to add the file names manually to these two lists.

Here is an example of how the lists would look like:

All red: [f1, f4, f5] All green: [f2, f3]

I keep adding files to these two lists based on the experiment results.

Then I have a little script that I run that finds the relative path of the directories where the files are located and looks for the files in all folders.

Here is the code:

f1 = '/exm1.xlsx'
f2 = '/exm2.xlsx'
f3 = '/sub1.xlsx'
f4 = '/sub2.xlsx'
f5 = '/eco1.xlsx'
f6 = '/eco2.xlsx'
# and so on...
I manually divide these files into two categories: "red" and "green". For example:

all_red_files = [f1, f4, f5]
all_green_files = [f2, f3]
I keep adding more files based on experiment results. For example:
red_files = [f113]
I want to simplify this process as the list of files and their names keep getting bigger (e.g., f1, f2, f3, ...).

I use the following script to find the files in different directories:

# Finds relative path
# Change up one directory
os.chdir("..")
# Find current path
d = os.getcwd()

# Gets all directories in the folder as a tuple
directories = [os.path.join(d, o) for o in os.listdir(d) if os.path.isdir(os.path.join(d, o))]

# Looks for the files in all folders
file_green = []
for directory in directories:
    for file in all_green_files:
        if os.path.exists(directory + file):
            f = directory + file
            file_green.append(f)
            print('file found:\n', f)

file_red = []
for directory in directories:
    for file in red_files:
        if os.path.exists(directory + file):
            f = directory + file
            file_red.append(f)
            print('file found:\n', f)
How can I simplify this process as my list of files and their names keep getting bigger?
buran write Mar-23-2023, 09:36 AM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.
Reply
#2
Cross-posted at StackOverflow

For start I can repeat what I commented there as well
Quote:Keep adding [hardcoded and seemingly random] file names manually into categories is recipe for disaster. Use standard name pattern to programmatically separate red/green categories (or separate the files at the time of creation in different folders). Consider using a database for the results (instead of files).
QubeStory likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  comparison of two xlsx files Jlyk 4 1,919 Sep-20-2021, 07:13 AM
Last Post: Jlyk
  Matplotlib: How do I convert Dates from Excel to use in Matplotlib JaneTan 1 3,163 Mar-11-2021, 10:52 AM
Last Post: buran
  Extracting information from .xlsx files hobbyist 0 1,578 Jan-06-2021, 07:20 PM
Last Post: hobbyist
  How to groupby Months showing average order value - Pandas & matplotlib Rwood90 0 1,837 Oct-20-2020, 12:53 PM
Last Post: Rwood90
  How can I speed up my openpyxl program reading Excel .xlsx files? deac33 0 3,363 May-04-2020, 08:02 PM
Last Post: deac33
  How to access all xlsx files in all subdirectories? Krszt 2 5,375 Mar-19-2019, 11:00 AM
Last Post: DeaD_EyE
  reading .xls files and saving as .xlsx jon0852 1 6,855 Oct-17-2017, 08:32 PM
Last Post: buran
  Looping .xlsx files in folder/subfolders copy pasting currentregion HarrisQ 4 5,503 Apr-17-2017, 06:35 AM
Last Post: HarrisQ

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020