"I'm Feeling Lucky" script problem (again) - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: "I'm Feeling Lucky" script problem (again) (/thread-19719.html) |
"I'm Feeling Lucky" script problem (again) - tab_lo_lo - Jul-11-2019 I didn't get any replies to the other related thread; and, because one other person in that thread had the same problem as me with no answers I thought it appropriate to start a new thread. Original question: "Man, I'm having trouble with this project too. I'm getting no search results after passing the search terms into the run window." My script returns this output: "Googling... None Press any key to continue . . ." My script: #! python3 # lucky.py - Opens several Google search results. import webbrowser, bs4, requests, sys print('Googling...') # display text while downloading the Google page res = requests.get('http://google.com/search?q=' + ' '.join(sys.argv[1:])) print(res.raise_for_status()) # Retreive top search result links. soup = bs4.BeautifulSoup(res.text, "html.parser") # Open a browser tab for each result linkElems = soup.select('.r a') numOpen = min(5, len(linkElems)) for i in range(numOpen): webbrowser.open('http://google.com' + linkElems[i].get('href')) RE: "I'm Feeling Lucky" script problem (again) - nilamo - Jul-11-2019 print(linkElems) # []If you look at the response you're getting, there are no links nested under a .r class. The page that google sends you is dependent upon whether or not you have javascript enabled. So, inspect what you're getting, and narrow your selector appropriately. RE: "I'm Feeling Lucky" script problem (again) - tab_lo_lo - Jul-16-2019 Hmmm. When I inspect the page I do see a .r <div class="r"><a href="https://www.python.org/" for each link listed in the search: "python." I added that code you posted to lines 19-20 of my script and I got the output: Googling... None [] Press any key to continue . . . I tried just eliminating the .r class and I got the 5 browsers to open, but predictably I just got the first 5 links under the a class. I checked in my Chrome settings and javascript was enabled Thanks!! RE: "I'm Feeling Lucky" script problem (again) - snippsat - Jul-16-2019 Look at Web-scraping part-2,under "God dammit JavaScript, why do i not get all content". I have done this task before look at hint on this line browser.page_source .soup = BeautifulSoup(browser.page_source, 'lxml')If do like this will get JavaScript(how in it look Browser) and can parse out links from Google as the look when inspect in browser. RE: "I'm Feeling Lucky" script problem (again) - tab_lo_lo - Jul-23-2019 I don't get why the script given to the readers in the Automate the Boring Stuff just doesn't work and why the author hasn't tried to provide additional guidance? I appreciate your help help here. I'm frustrated that I'm still having problems. Ugh. I get this message when I try to run the script with Selenium in it... This is the script from the "God dammit Javascript..." I tried to figure out how to add the Selenium piece and the browser.page_source to my script and got a similar message.
RE: "I'm Feeling Lucky" script problem (again) - snippsat - Jul-23-2019 Quote: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/...river/homeYou most have chromedriver.exe in your Windows Path.Also in Environment Variables Path i have C:\cmder\bin .Test code. from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.keys import Keys import time #--| Setup chrome_options = Options() #chrome_options.add_argument("--headless") browser = webdriver.Chrome(executable_path=r'C:\cmder\bin\chromedriver.exe') #--| Parse or automation browser.get('https://duckduckgo.com') input_field = browser.find_elements_by_css_selector('#search_form_input_homepage') input_field[0].send_keys('car' + Keys.RETURN) time.sleep(3) images_link = browser.find_elements_by_link_text('Images') # Or name your country use images_link[0].click() time.sleep(5) browser.quit() RE: "I'm Feeling Lucky" script problem (again) - tab_lo_lo - Jul-23-2019 Bummer. I don't have admin rights to add this .exe to my PATH. I may just skip this project and move on to the next one. Thanks for your help on this one! RE: "I'm Feeling Lucky" script problem (again) - snippsat - Jul-23-2019 (Jul-23-2019, 10:16 PM)tab_lo_lo Wrote: Bummer. I don't have admin rights to add this .exe to my PATH.An other solution that work is to have chrome driver an python script in same folder. browser = webdriver.Chrome(executable_path=r'chromedriver.exe') |