Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 webscrapping lists to dataframe
#1
Hi i am doing some webscrapping and is stuck at the follow codes below:

For some reason the record_lost is not inserted into the dataframe. Unable to debug this portion and would appreciate any kind help on this.

record_list = [list(item) for item in list(zip(url_list, title_list, description_list))]
df = pd.DataFrame(data=record_list,columns=['URL','Title', 'Description'])

Quote
#2
Post a sample of output record_list.
Quote
#3
Isn't there needless conversions in creating record_list?

>>> a = [1, 2, 3]                                                              
>>> b = 'abc'                                                                  
>>> c = [10, 20, 30] 
>>> for row in zip(a, b, c): 
....    print(row) 
....                                                                           
(1, 'a', 10)
(2, 'b', 20)
(3, 'c', 30)
>>> for row in list(zip(a, b, c)): 
....    print(row) 
....                                                                           
(1, 'a', 10)
(2, 'b', 20)
(3, 'c', 30)
>>> [list(item) for item in list(zip(a, b, c))]                                
[[1, 'a', 10], [2, 'b', 20], [3, 'c', 30]]
>>> list(zip(a, b, c))                                                         
[(1, 'a', 10), (2, 'b', 20), (3, 'c', 30)]
DataFrame data source can be iterable (documentation: data : ndarray (structured or homogeneous), Iterable, dict, or DataFrame) therefore even converting to list is not necessary:

>>> df = pd.DataFrame(zip(a, b, c), columns = ('First', 'Second', 'Third'))
>>> df
   First Second  Third
0      1      a     10
1      2      b     20
2      3      c     30
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Quote
#4
Hi snippsat, thanks for your quick reply. Actually i am reusing code i found and editing it. I found out why the lists not working cus my webscrapping didnt pull the values successfully into the lists and they are empty which is why they didnt work. Once i fixed that it is working now. I will reread your dataframe advice again to understand it.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  webscrapping links from pandas dataframe Wolverin 1 224 Jun-19-2019, 11:22 PM
Last Post: Larz60+
  webscrapping links and then enter those links to scrape data kirito85 2 335 Jun-13-2019, 02:23 AM
Last Post: kirito85

Forum Jump:


Users browsing this thread: 1 Guest(s)