Add column headers to dataframe - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Data Science (https://python-forum.io/forum-44.html) +--- Thread: Add column headers to dataframe (/thread-10136.html) |
Add column headers to dataframe - chisox721 - May-14-2018 Hey guys-I'm working on a script that grabs weather information from different locations. The only thing I'm missing within the dataframe is replacing "0_x" and "0_y" (below) with a reference to the location. Ref 0_x 0_y 0 temp 83 71 1 humid 19% 45% 2 pressure 29 29 3 dew 37 49 4 wind_dir 157 227 5 wind_mph 6.5 1.2 6 wind_gust 9.8 3.7 Within the csv file that holds the location info, there is a 3rd column with specific location names. All I need to add is a line in the script that references the 3rd column and inserts it as a column header. Hopefully this all makes sense. Any help would be greatly appreciated. import requests import csv import pandas as pd stadiums = pd.read_csv('ParkLocations1.csv') stadiums.to_dict('series') all_loc_data = pd.DataFrame() col_1 = ['temp','humid','pressure','dew','wind_dir','wind_mph','wind_gust'] all_loc_data.insert(loc=0, column='Ref', value=col_1) url_template = "http://api.wunderground.com/api/88fb15b49e765a43/conditions/q/{state}/{city}.json" for state,city in zip(stadiums.loc[ : ,"State"],stadiums.loc[ : ,"City"]): url = requests.get(url_template.format(state=state,city=city)) parsed = url.json() temp = parsed['current_observation']['temp_f'] humid = parsed['current_observation']['relative_humidity'] pressure = parsed['current_observation']['pressure_in'] dew = parsed['current_observation']['dewpoint_f'] wind_dir = parsed['current_observation']['wind_degrees'] wind_mph = parsed['current_observation']['wind_mph'] wind_gust = parsed['current_observation']['wind_gust_mph'] data_list = [temp,humid,pressure,dew,wind_dir,wind_mph,wind_gust] loc_data = pd.DataFrame(data_list) loc_data.insert(loc=0, column='Ref', value=col_1) all_loc_data = all_loc_data.merge(loc_data, on='Ref',how='outer') all_loc_data.to_csv('WUEx.csv') RE: Add column headers to dataframe - volcano63 - May-15-2018 Without examples of state/city pairs (as int your file) not much I can do, but you really have to learn to work with DataFrame.apply. This is extremely inefficient use of pandas RE: Add column headers to dataframe - chisox721 - May-15-2018 (May-15-2018, 09:59 AM)volcano63 Wrote: Without examples of state/city pairs (as int your file) not much I can do, but you really have to learn to work with DataFrame.apply. This is extremely inefficient use of pandas Here is a sample of what the parklocations1.csv looks like: Team State City SFG CA San Francisco NYY NY Bronx RE: Add column headers to dataframe - volcano63 - May-16-2018 OK, here is a simple example how to use apply in your code url_template = "http://api.wunderground.com/api/88fb15b49e765a43/conditions/q/{state}/{city}.json" def get_weather(row, *fileds): url = requests.get(url_template.format(state=row['State'],city=row['City'])) observation = url.json()['current_observation'] return [observation[field] for field in fields] fields = ['temp_f', 'relative_humidity'] stadiums = pd.DataFrame([['SFG', 'CA', 'San Francisco'], ['NYY', 'NY', 'Bronx']], columns=['Team', 'State', 'City']) print(stadiums.apply(lambda r: get_weather(r, *fields), axis=1).values)And the result is now, stadium[fields] = pd.DataFrame(<result>)will update your stadiums DataFrame with new fields - but you have to convert the result to a form accepted by pandas.DataFrame constructor. I will let you experiment with that on your own. PS pandas is an amazing tool which I just started learning myself - but you have to understand, DataFrame and Series are objects intended for processing as a whole - otherwise, you abuse pandas. Learn it properly RE: Add column headers to dataframe - chisox721 - May-17-2018 (May-16-2018, 06:01 AM)volcano63 Wrote: PS pandas is an amazing tool which I just started learning myself - but you have to understand, DataFrame and Series are objects intended for processing as a whole - otherwise, you abuse pandas. Learn it properly Thank you for the response and advice. I knew I was taking a round-a-bout approach but wasn't really sure how to improve on it. Your input made things a lot clearer for me. RE: Add column headers to dataframe - volcano63 - May-20-2018 PS I just took a second look - I don't know why I did not suggest url_template = http://api.wunderground.com/api/88fb15b49e765a43/conditions/q/{State}/{City}.json" .... url = requests.get(url_template.format(**row)) |