Python Forum
Inserting data from python list into a pandas dataframe
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Inserting data from python list into a pandas dataframe
I have the following files in AAMC_K.txt, AAU.txt, ACU.txt, ACY.txt, and AE.txt in a folder called AMEX. I am trying to merge these text files into one dataframe. I have tried to do so with pd.merge() but I get an error that the merge function needs a right and left parameter and my data is in a python list. How can I merge the data in the data_list into one pandas dataframe.

import pandas as pd
import os

textfile_names = os.listdir("AMEX")
data_list = []

for i in range(len(textfile_names)):
   data = pd.read_csv("AMEX/"+textfile_names[i], index_col=None, header=0)

frame = pd.merge(data_list, on='<DTYYYYMMDD>', how='outer')
"AE.txt" <TICKER>,<PER>,<DTYYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT> AE,D,19970102,000000,12.6250,12.6250,11.7500,11.7500,144,0 AE,D,19970103,000000,11.8750,12.1250,11.8750,12.1250,25,0 AAU.txt <TICKER>,<PER>,<DTYYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT> AAU,D,20020513,000000,0.4220,0.4220,0.4220,0.4220,0,0 AAU,D,20020514,000000,0.4177,0.4177,0.4177,0.4177,0,0 ACU.txt <TICKER>,<PER>,<DTYYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT> ACU,D,19970102,000000,5.2500,5.3750,5.1250,5.1250,52,0 ACU,D,19970103,000000,5.1250,5.2500,5.0625,5.2500,12,0 ACY.txt <TICKER>,<PER>,<DTYYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT> ACY,D,19980116,000000,9.7500,9.7500,8.8125,8.8125,289,0 ACY,D,19980120,000000,8.7500,8.7500,8.1250,8.1250,151,0
I want the output to be filtered with the DTYYYYMMDD and put into one dataframe frame.
<TICKER>,<PER>,<DTYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT>,<TICKER>,<PER>,<DTYYYMMDD>,<TIME>,<OPEN>,<HIGH>,<LOW>,<CLOSE>,<VOL>,<OPENINT> ACU,D,19970102,000000,5.2500,5.3750,5.1250,5.1250,52,0,AE,D,19970102,000000,12.6250,12.6250,11.7500,11.7500,144,0 ACU,D,19970103,000000,5.1250,5.2500,5.0625,5.2500,12,0,AE,D,19970103,000000,11.8750,12.1250,11.8750,12.1250,25,0
** Moderator Note ** Use python, not icode for more than 1 line of code

Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas Data frame column condition check based on length of the value aditi06 1 129 Jul-28-2021, 11:08 AM
Last Post: jefsummers
  empty row in pandas dataframe rwahdan 3 324 Jun-22-2021, 07:57 PM
Last Post: snippsat
Question [Solved] How to refer to dataframe column name based on a list lorensa74 1 387 May-17-2021, 07:02 AM
Last Post: lorensa74
  [Pandas] Write data to Excel with dot decimals manonB 1 411 May-05-2021, 05:28 PM
Last Post: ibreeden
  pandas.to_datetime: Combine data from 2 columns ju21878436312 1 599 Feb-20-2021, 08:25 PM
Last Post: perfringo
Question Pandas - Creating additional column in dataframe from another column Azureaus 2 673 Jan-11-2021, 09:53 PM
Last Post: Azureaus
  Comparing results within a list and appending to pandas dataframe Aryagm 1 531 Dec-17-2020, 01:08 PM
Last Post: palladium
  How to search for specific string in Pandas dataframe Coding_Jam 1 736 Nov-02-2020, 09:35 AM
Last Post: PsyPy
  Inserting a variable as column name in sqlite3 python JellyCreeper6 3 1,062 Nov-01-2020, 01:44 PM
Last Post: ibreeden
  PANDAS: DataFrame | White Spaces & Special Character Removal traibr 1 1,158 Sep-10-2020, 07:02 PM
Last Post: eddywinch82

Forum Jump:

User Panel Messages

Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020