Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Read all csv files, and store the last line from each folder
I want to read all csv files and read each file and capture last line form each file.
I use the below code, but I struck.

import os
from glob import glob
PATH = "D:\data\project\*\*\InputFiles"
EXT = "*.csv"
all_csv_files = [file
                 for path, subdir, files in os.walk(PATH)
                 for file in glob(os.path.join(path, EXT))]
with pathlib written here without testing, but I think it's right or close to it:
from pathlib import Path

p = Path("D:\data\project\*\*\InputFiles")
csv_files = [csvfile for csvfile in p.iterdir() if csvfile.is_file() and csvfile.suffix == '.csv']
for filename in csv_files:
I use below code. Here, the there will be 4 levels (subfolders) so I use

Exammple: Rootfolder/project/cataegory after this I do not know folder names, but there will be 4 more subfolders and in the last folder my file exist.

My file name is: .Dailycollection.log

If file exist after this : Rootfolder/project/cataegory, then I want to capture full path of the file.
p = Path("D:\Mekala_Backupdata\*\*\*\*\")
csv_files = [csvfile for csvfile in p.iterdir() if csvfile.is_file() and csvfile.suffix == '*.Dailycollection.log']
for filename in csv_files:
I can't run the following code as I don't use windows, but this should list all csv files in all directories
within and below a root directory of D:\data\project\*\*\InputFiles (you will have to supply actual values for '*')
from pathlib import Path

def walk_dir(starting_dir):
    flist = []
    for path in Path(starting_dir).iterdir():
        if path.is_file():
            if path.suffix == '.csv':
        elif path.is_dir():

    for file in flist:

if __name__ == '__main__':
    start_path = 'D:\data\project\*\*\InputFiles'
    import os
    srcdir = Path('.')
    savefile = srcdir / 'allcsvfiles.txt'
    walk_dir(start_path, savefile)
Actually the last layer (sub)directory does not know (not fixed). But the file will be saved into this last folder. Hence I want to use the wild card.
The file is available at :D:\Mekala_Backupdata\*\*\*\*\ --> here the file.

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  os.walk does not see files that are in the folder kiton 1 191 Jan-21-2020, 07:26 PM
Last Post: micseydel
  Downloading And Saving Zip Files To A Particular Path Folder eddywinch82 2 211 Jan-06-2020, 07:56 PM
Last Post: eddywinch82
  remove files from folder older than X days kerzol81 2 242 Jan-03-2020, 11:55 PM
Last Post: snippsat
  Read csv file, parse data, and store in a dictionary markellefultz20 4 363 Nov-26-2019, 03:33 PM
Last Post: DeaD_EyE
  How to read text file line by line SriRajesh 1 157 Nov-05-2019, 01:51 PM
Last Post: snippsat
  Loop through folder of Excel Files and extract single column fioranosnake 2 192 Oct-28-2019, 05:19 PM
Last Post: fioranosnake
  Replace string in many files in a folder metro17 8 412 Oct-16-2019, 06:46 PM
Last Post: ndc85430
  How do I read the HTML files in a directory and write the content into a CSV file? glittergirl 1 225 Sep-23-2019, 11:01 AM
Last Post: Larz60+
  Read each line, replace string and save into a new file igormonteiro 2 350 Sep-15-2019, 01:24 PM
Last Post: buran
  Can read files in command prompt but not IDE Exsul 2 226 Sep-13-2019, 01:40 AM
Last Post: jsira2003

Forum Jump:

Users browsing this thread: 1 Guest(s)