I have several csv files with different kinds of data on them. Currently I am making a dataframe for each one and then merging the dataframes. I also have to do a process on the data (calculating total scores) and then I'm committing them to a database. It works but I find it hard to implement new kinds of csv files with different data on them.
I was wondering what other people do to make this process easy and scalable?
I was wondering what other people do to make this process easy and scalable?