+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: convert list compression to for loop (/thread-5510.html)
convert list compression to for loop - Prince_Bhatia - Oct-07-2017
hi,
i have a script which has so many list compression of for loop, can someone expand those list compression to for loop so that i can understand it much better
below is the script
from selenium import webdriver
from bs4 import BeautifulSoup
from collections import OrderedDict
import more_itertools
# open Firefox to get the data
driver = webdriver.Firefox()
driver.get('https://rotogrinders.com/grids/nfl-targets-1402017?site=draftkings')
soup = BeautifulSoup(driver.page_source, 'lxml')
driver.quit()
# extract data from BeautifulSoup object
player_data = soup.find_all('div', attrs={'class':'rgt-col'})
text = [y.text for x in player_data for y in x.descendants if y.name == 'div']
indices_to_delete = [i for i in range(0, len(text), 250)]
keys = [text[k] for k in indices_to_delete]
new_text = [x for x in text if not x in keys]
text = list(more_itertools.sliced(new_text, 249))
new_text = list(zip(*text))
# build the dict
players = OrderedDict()
for x in new_text:
y = list(zip(keys, x))
for key, val in y:
if key == 'Player':
players[val] = {}
current_player = val
else:
players[current_player][key] = val
RE: convert list compression to for loop - micseydel - Oct-08-2017
What have you tried?
RE: convert list compression to for loop - Prince_Bhatia - Oct-09-2017
hi sir,
this is what i have tried
for x in player_data:
for y in x.descendants:
if y.name == "div"
text = y.text
indices_to_delete = []
for i in range(0, len(text), 250):
indices_to_delete.append(i)
keys = []
for k in indices_to_delete:
keys.append(text[k])
new_text = []
for x in text:
if not x in keys:
new_text.append(x)
has no idea is it correct and how to club them together , all the loops? can someone please tell if it is right or wrong?
RE: convert list compression to for loop - micseydel - Oct-09-2017
The last two look right. The second looks unnecessary, could just be
indices_to_delete = range(0, len(text), 250). The first one isn't right.As general advice on question asking when it comes to programming, if you can test it yourself in a few seconds then it's not a good question to ask on a forum. Your first conversion is easy to test and verify as incorrect (even after the syntax errors are fixed), since the type isn't even right. The latter three were good questions, after the second post. As a hint on the first one, you should follow a similar pattern of the last three, creating a list and appending to it, not just doing an assignment conditionally.
RE: convert list compression to for loop - miltmobley - Oct-09-2017
This reply speaks to your understanding of list comprehensions, so that you might understand the code,
and see there is probably no need to convert it.
This code:
text = [y.text for x in player_data for y in x.descendants if y.name == 'div']is equivalent to:
text = []
for x in player_data:
for y in x.descendants:
if y.name == 'div':
text += [y.text]As you can see the list comprehension is more concise, and, according to python developers,executes more efficiently than the corresponding for loops. If the original code works,
and you change it, you risk breaking it without being able to fix it.
RE: convert list compression to for loop - micseydel - Oct-09-2017
As a note about miltmobley's comments:
I strongly recommend using the append() method instead of
+=. The latter requires creating many intermediate lists, which is unnecessary.List comprehensions can be faster, but "efficient" often has a technical meaning (complexity analysis, big-O), so just be aware of potential pushback from that wording.