Python Forum

I am hoping someone could help me read in a master directory file which tells me what .csv to read through and save in memory and then move to next file to read until eof()

first .csv has names
CL
NG
GC

those names tell me where in my c:\Data to get the file called CL etc etc.

I have imported

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import scipy as sp

pd.read_csv('c:/QlData/Continuous/Daily/CL.csv')

so this i guess is reading in the file named CL but once i review the huge amount of data in CL I want to then use a for loop to go to next line in master directory and read in NG and so on.

I just cannot find anything online to help me do this besides static reading process like I have above?
Any help would be appreciated

(Sep-21-2020, 01:32 AM)CEC68 Wrote: [ -> ]besides static reading process like I have above?

Do you mean hard coded file names to be processed?
If I understood you correctly, you want to find all *.csv files in a specific folder/subfolders and load them one-by-one (and do something with data). The following code snippet could help you to do this:

import os
for root, dirs, files in os.walk("C:/Data"):
    for name in files:
        if name.endswith('csv'):
            data = pd.read_csv(os.path.join(root, name))
            # do something with data...

CEC68

scidam