Python Forum
Concatenate multiple csv files
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Concatenate multiple csv files
#1
Hello python gurus,

I am trying to concatenate multiple data frames but when I run the application it gives me this error:
KeyError: 'Level Date must be same as name (None)'
I am following this tutorial: Python for finance

The tutorial's way of joining the csv files generates a combined csv file with all the stocks but gives a new column for each stock (500). Ideally I just want one column called "Tickers" not 500. So I am trying melt instead of outer join but not getting it to work.


Full code is here: https://codeshare.io/amDv7o

def compile_data():
    with open("sp500tickers.pickle", "rb") as f:
        tickers = Cpickle.load(f)

    main_df = pd.DataFrame()

    for count, ticker in enumerate(tickers):
        df = pd.read_csv('stock_dfs/{}.csv'.format(ticker))
        df.reset_index('Date', inplace=True)

        df.rename(columns={'Adj Close': ticker}, inplace=True)
        df.drop(['Open', 'High', 'Low', 'Close', 'Volume'], 1, inplace=True)

        if main_df.empty:
            main_df = df
        else:
            main_df = main_df.melt(id_vars=['Date'], var_name='Ticker', value_name='Closed')

        if count % 10 == 0:
            print(count)
    print(main_df.head())

    main_df.to_csv('sp500_joined_closes.csv')

compile_data()
Reply
#2
always show full error traceback, verbatim, and include which imports are being used.
I don't use pandas (or rarely pickle) much, but this line looks suspect:
df.reset_index('Date', inplace=True)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Import multiple CSV files into pandas Krayna 0 1,719 May-20-2021, 04:56 PM
Last Post: Krayna
  Concatenate 3D arrays paul18fr 1 2,646 Apr-09-2021, 02:49 PM
Last Post: paul18fr
  Loading multiple JSON files to create a csv 0LI5A3A 0 2,106 Jun-28-2020, 10:35 PM
Last Post: 0LI5A3A
  Append Multiple CSV files Nidhesh 2 2,503 Jul-03-2019, 11:55 AM
Last Post: Nidhesh
  How to extract different data groups from multiple CSV files using python Rafiz 3 3,243 Jun-04-2019, 05:20 PM
Last Post: jefsummers
  concatenate mcgrim 1 2,236 Mar-22-2019, 01:31 PM
Last Post: buran
  Grab columns from multiple files, combine into one jon0852 0 2,019 Feb-12-2019, 02:53 AM
Last Post: jon0852
  read multiple .xlsx files and text files in a directory BNB 11 25,690 Jun-07-2017, 07:42 AM
Last Post: BNB
  Read CSV Files with multiple headers into Python DataFrame UGuntupalli 12 26,790 Jan-26-2017, 03:07 PM
Last Post: UGuntupalli

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020