+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: webscrapping lists to dataframe (/thread-19002.html)
webscrapping lists to dataframe - kirito85 - Jun-10-2019
Hi i am doing some webscrapping and is stuck at the follow codes below:
For some reason the record_lost is not inserted into the dataframe. Unable to debug this portion and would appreciate any kind help on this.
record_list = [list(item) for item in list(zip(url_list, title_list, description_list))] df = pd.DataFrame(data=record_list,columns=['URL','Title', 'Description'])
RE: webscrapping lists to dataframe - snippsat - Jun-10-2019
Post a sample of output
record_list.
RE: webscrapping lists to dataframe - perfringo - Jun-10-2019
Isn't there needless conversions in creating record_list?
>>> a = [1, 2, 3] >>> b = 'abc' >>> c = [10, 20, 30] >>> for row in zip(a, b, c): .... print(row) .... (1, 'a', 10) (2, 'b', 20) (3, 'c', 30) >>> for row in list(zip(a, b, c)): .... print(row) .... (1, 'a', 10) (2, 'b', 20) (3, 'c', 30) >>> [list(item) for item in list(zip(a, b, c))] [[1, 'a', 10], [2, 'b', 20], [3, 'c', 30]] >>> list(zip(a, b, c)) [(1, 'a', 10), (2, 'b', 20), (3, 'c', 30)]DataFrame data source can be iterable (documentation: data : ndarray (structured or homogeneous), Iterable, dict, or DataFrame) therefore even converting to list is not necessary:
>>> df = pd.DataFrame(zip(a, b, c), columns = ('First', 'Second', 'Third'))
>>> df
First Second Third
0 1 a 10
1 2 b 20
2 3 c 30
RE: webscrapping lists to dataframe - kirito85 - Jun-10-2019
Hi snippsat, thanks for your quick reply. Actually i am reusing code i found and editing it. I found out why the lists not working cus my webscrapping didnt pull the values successfully into the lists and they are empty which is why they didnt work. Once i fixed that it is working now. I will reread your dataframe advice again to understand it.