Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Need help with Pandas
#1
my question here
Hi,
I have a huge problem and i cant solve it on my own. I'm sure it is simple but
I cant get it.
You need these 2 files for the short script: www.file-upload.net/download-12647206/djia.csv.html...#
and
www.file-upload.net/download-12647207/PreisMoatStanley2013.dat.html...#

The problem is to get the index to Date. My index is always empty

import pandas as pd
import numpy as np
import datetime as dt
import matplotlib.pyplot as plt
import pandas_datareader.data as web
import datetime

paper = pd.read_csv('PreisMoatStanley2013.dat',
                    delimiter = ' ',
                    parse_dates=[0,1,100,101])



data = pd.DataFrame({'GoogleWE': paper['Google End Date'],
                    'debt': paper['debt'].astype(np.float64),
                    'DJIADate': paper['DJIA Date'],
                    'DJIAClose': paper['DJIA Closing Price']
                    .astype(np.float64)})
data.set_index('DJIADate')
print paper [:5]

import quandl
#api_key = open('quandl_key.txt','r').read()

djia = pd.read_csv("djia.csv", index_col=0)

print djia[:3]

djia_closes = djia['Close'].reset_index()
djia_closes.set_index('Date')
print djia_closes[:3]
#s152
data = pd.merge(data, djia_closes,
left_on='DJIADate', right_on='Date')
data.drop(['DJIADate'], inplace=True, axis=1)
data = data.set_index('Date') # Problem seems to be here

print data[:3]
Shell message :
Empty DataFrame
Columns: [DJIAClose, GoogleWE, debt, Close]
Index: []


How it should looks like:
Out[8]:
DJIAClose GoogleWE debt Close
Date
2004-01-12 10485.18 2004-01-10 0.21 10485.2
2004-01-20 10528.66 2004-01-17 0.21 10528.7
2004-01-26 10702.51 2004-01-24 0.21 10702.5


Maybe someone have any idea to help me.
Thanks
Reply
#2
not even one? Smash
Reply
#3
Hello

Sorry for being late. My stupid ISP has blocked the http://www.file-upload.net link and I was not able to download it quicker. Had to use proxy etc to reach there

Going line by line execution found that the below merge is returning blank dataframe.

data = pd.merge(data, djia_closes,left_on='DJIADate', right_on='Date')
Try this

import pandas as pd
import numpy as np
import datetime as dt
import matplotlib.pyplot as plt
#import pandas_datareader.data as web
import datetime
 
paper = pd.read_csv('PreisMoatStanley2013.dat',
                    delimiter = ' ',
                    parse_dates=[0,1,100,101])
 
 
 
data = pd.DataFrame({'GoogleWE': paper['Google End Date'],
                    'debt': paper['debt'].astype(np.float64),
                    'DJIADate': paper['DJIA Date'],
                    'DJIAClose': paper['DJIA Closing Price']
                    .astype(np.float64)})
newdata = data.set_index('DJIADate')
djia = pd.read_csv("djia.csv", index_col=0)
djia_closes = djia['Close'].reset_index()
newdjia_closes = djia_closes.set_index('Date')
findata = pd.merge(newdata, newdjia_closes, left_index=True, right_index=True)

#findata.drop(['DJIADate'], inplace=True, axis=1)
#data = findata.set_index('Date') # Problem seems to be here
 
print (findata[:3])
output

Quote: DJIAClose GoogleWE debt Close
2004-01-12 10485.18 2004-01-10 0.21 10485.2
2004-01-20 10528.66 2004-01-17 0.21 10528.7
2004-01-26 10702.51 2004-01-24 0.21 10702.5

It was fun troubleshooting this - thank You Dance
Reply
#4
@ radioactive9What
What the hell you made it. I'm stacking on this for 3 days and i totaly couldnt explain it to myself.
I post the question on 3 different sites and you the first who have a solution Clap
Thanks for your help!!!
You have any suggestions to learn pandas?
Im using this book (Mastering Pandas for finance) but sometimes there are mistakes in it.
Reply
#5
Hello

Well I am 3 months old in python programming.
My approach may not be liked by many. I avoid reading books - But I just try to write whatever I can lay hands on. Solve small problems.

If you like reading and learning : The below book is considered as best in terms of Pandas. He is considered as father of Pandas
Python for Data Analysis: Data Wrangling with ...
Book by Wes McKinney
Reply
#6
Hi radioactiv9,
like you I'm new to Python. I started around april this year by reading several books and the documentation. But sometimes its realy annoying to see old book examples they arnt working anymore. But all in all thats good stuff.
I want to let you know, that some other also answered my question. His solution is perfect and he also explained whats wrong. The Point is to delete parse_dates=[0,1,100,101]. Thats all
Check this: https://stackoverflow.com/questions/4558...-dataframe
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020