Project: “I’m Feeling Lucky” Google Search - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: Project: “I’m Feeling Lucky” Google Search (/thread-11674.html) |
Project: “I’m Feeling Lucky” Google Search - Truman - Jul-20-2018 This is web scraping project from Automate the Boring Stuff with Python. From some reason solution doesn't work and I have some doubts myself. The goal is to type a search term on the command line and have my computer automatically open a browser with all the top search results in new tabs. #! python3 # lucky.py - Opens several Google search results. import webbrowser, bs4, requests, sys print('Googling...') # display text while downloading the Google page res = requests.get('http://google.com/search?source=' + ' '.join(sys.argv[1:])) print(res.raise_for_status()) # Retreive top search result links. soup = bs4.BeautifulSoup(res.text, "html.parser") # Open a browser tab for each result linkElems = soup.select('.r a') numOpen = min(5, len(linkElems)) for i in range(numOpen): webbrowser.open('http://google.com' + linkElems[i].get('href'))I'm also not sure how to start this program. Should I type a search term in command prompt where I usually start my python files? RE: Project: “I’m Feeling Lucky” Google Search - Larz60+ - Jul-21-2018 Quote:I'm also not sure how to start this programyou can create a function, for example named test_it() which executes a test case, then start that with (untested): #! python3 # lucky.py - Opens several Google search results. import webbrowser import bs4 import requests import sys def my_func(search_str) : print('Googling...') # display text while downloading the Google page res = requests.get('http://google.com/search?source=' + ' '.join(search_str)) print(res.raise_for_status()) # Retreive top search result links. soup = bs4.BeautifulSoup(res.text, "html.parser") # Open a browser tab for each result linkElems = soup.select('.r a') numOpen = min(5, len(linkElems)) for i in range(numOpen): webbrowser.open('http://google.com' + linkElems[i].get('href')) def test_it(): my_func(sys.argv[1:]) if __name__ == '__main__': test_it()this code will only run if the program is started from within an IDE, or from the command line, but will not run if your main code is called from another program. RE: Project: “I’m Feeling Lucky” Google Search - Truman - Jul-21-2018 Now I just feel completely lost. lucky is my code filename and I named lucky1 what you Larz wrote. I used Thonny IDE.I installed bs4 of course. and when I run code from Notepad++ with command prompt I get this: my search term in command line is "novak djokovic" and it's not giving any result obviously.I'll try to find solution on stackoverflow or smth like that. RE: Project: “I’m Feeling Lucky” Google Search - Larz60+ - Jul-22-2018 whenever you see: ModuleNotFoundError: No module named '...', you need to install a package. In this case, you need to install BeautifulSoup4: first test to see that pip version is correct one: pip -VYou want to see something like: if you don't see proper python version, substitute pip3 for pip belowpip install BeautifulSoup4 RE: Project: “I’m Feeling Lucky” Google Search - Truman - Jul-22-2018 this is what I get...From some reason I still receive the same Traceback error with Thonny and the same output with command prompt.
RE: Project: “I’m Feeling Lucky” Google Search - Larz60+ - Jul-22-2018 that's good, that means you can use: pip install BeautifulSoup4 I see it's already installed. Then make the following changes on your code (by line number) In reverse order (so line numbers remain constant until done) line 15 # from: soup = bs4.BeautifulSoup(res.text, "html.parser") # to: soup = BeautifulSoup(res.text, "html.parser")line 5 # From: import bs4 # to: from bs4 import BeautifulSoup RE: Project: “I’m Feeling Lucky” Google Search - Truman - Jul-22-2018 I edited my last post in the meantime. Now the question is how to start this program. RE: Project: “I’m Feeling Lucky” Google Search - Larz60+ - Jul-22-2018 run from command line python myprogramname.py RE: Project: “I’m Feeling Lucky” Google Search - Truman - Jul-22-2018 The point of this program, if I understand it correctly, is to add search word in command line and immediately get 5 open tasks with first 5 search results. I tried with adding search words too after filename but the outcome is the same.
RE: Project: “I’m Feeling Lucky” Google Search - Larz60+ - Jul-22-2018 It appears to have run you are getting none from: print(res.raise_for_status())after line 18, add: for link in linkElems: print(f'link: {link}')so you can see if the soup.select (which is css select by the way) is finding anything also, you need to supply a value to the command line e.g. python -m myprog.py searchvalue |