Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Project: “I’m Feeling Lucky” Google Search
#1
This is web scraping project from Automate the Boring Stuff with Python. From some reason solution doesn't work and I have some doubts myself. The goal is to type a search term on the command line and have my computer automatically open a browser with all the top search results in new tabs.

#! python3
# lucky.py - Opens several Google search results.

import webbrowser, bs4, requests, sys

print('Googling...') # display text while downloading the Google page
res = requests.get('http://google.com/search?source=' + ' '.join(sys.argv[1:]))
print(res.raise_for_status())

# Retreive top search result links.
soup = bs4.BeautifulSoup(res.text, "html.parser")

# Open a browser tab for each result
linkElems = soup.select('.r a')
numOpen = min(5, len(linkElems))
for i in range(numOpen):
	webbrowser.open('http://google.com' + linkElems[i].get('href'))
I'm also not sure how to start this program. Should I type a search term in command prompt where I usually start my python files?
Quote
#2
Quote:I'm also not sure how to start this program
you can create a function, for example named test_it() which executes a test case, then
start that with (untested):
#! python3
# lucky.py - Opens several Google search results.
 
import webbrowser
import bs4
import requests
import sys

def my_func(search_str) :
    print('Googling...') # display text while downloading the Google page
    res = requests.get('http://google.com/search?source=' + ' '.join(search_str))
    print(res.raise_for_status())
 
    # Retreive top search result links.
    soup = bs4.BeautifulSoup(res.text, "html.parser")
 
    # Open a browser tab for each result
    linkElems = soup.select('.r a')
    numOpen = min(5, len(linkElems))
    for i in range(numOpen):
        webbrowser.open('http://google.com' + linkElems[i].get('href'))

def test_it():
    my_func(sys.argv[1:])

if __name__ == '__main__':
    test_it()
this code will only run if the program is started from within an IDE, or from the command line, but will not run if your main code is called from another program.
Truman likes this post
Quote
#3
Error:
Traceback (most recent call last): File "C:\Python36\kodovi\lucky.py", line 4, in <module> import webbrowser, bs4, requests, sys ModuleNotFoundError: No module named 'bs4' >>> %Run lucky1.py Traceback (most recent call last): File "C:\Python36\kodovi\lucky1.py", line 5, in <module> import bs4 ModuleNotFoundError: No module named 'bs4'
Now I just feel completely lost. lucky is my code filename and I named lucky1 what you Larz wrote. I used Thonny IDE.
I installed bs4 of course.

and when I run code from Notepad++ with command prompt I get this:
Output:
C:\Python36\kodovi>lucky novak djokovic Googling... None
my search term in command line is "novak djokovic" and it's not giving any result obviously.
I'll try to find solution on stackoverflow or smth like that.
Quote
#4
whenever you see: ModuleNotFoundError: No module named '...', you need to install a package.

In this case, you need to install BeautifulSoup4:

first test to see that pip version is correct one:
pip -V
You want to see something like:

Output:
pip 10.0.1 from ... (python 3.6)
if you don't see proper python version, substitute pip3 for pip below

pip install BeautifulSoup4
Quote
#5
Output:
C:\Users\user>pip -V pip 10.0.1 from c:\python36\lib\site-packages\pip (python 3.6) C:\Users\user>pip3 install BeautifulSoup4 Requirement already satisfied: BeautifulSoup4 in c:\python36\lib\site-packages ( 4.6.0)
this is what I get...From some reason I still receive the same Traceback error with Thonny and the same output with command prompt.
Quote
#6
that's good, that means you can use:
pip install BeautifulSoup4

I see it's already installed.
Then make the following changes on your code (by line number)
In reverse order (so line numbers remain constant until done)
line 15
# from:
    soup = bs4.BeautifulSoup(res.text, "html.parser")
# to:
    soup = BeautifulSoup(res.text, "html.parser")
line 5
# From:
import bs4
# to:
from bs4 import BeautifulSoup
Quote
#7
I edited my last post in the meantime. Now the question is how to start this program.
Quote
#8
run from command line
python myprogramname.py
Quote
#9
Output:
C:\Python36\kodovi>python lucky1.py Googling... None
The point of this program, if I understand it correctly, is to add search word in command line and immediately get 5 open tasks with first 5 search results. I tried with adding search words too after filename but the outcome is the same.
Quote
#10
It appears to have run
you are getting none from:
print(res.raise_for_status())
after line 18, add:
for link in linkElems:
    print(f'link: {link}')
so you can see if the soup.select (which is css select by the way) is finding anything

also, you need to supply a value to the command line e.g.
python -m myprog.py searchvalue
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  "I'm Feeling Lucky" script problem (again) tab_lo_lo 7 1,648 Jul-23-2019, 11:26 PM
Last Post: snippsat
  Jump to next empty column with Google Sheets & Python Biks 0 172 Jun-14-2019, 07:28 PM
Last Post: Biks
  How to fix and extract Google CSE JSON API using Python.exe venkatsrrecruiter 1 270 May-04-2019, 07:33 PM
Last Post: venkatsrrecruiter
  Interfacing Google Chrome & Python akornrumpf 5 541 Mar-16-2019, 10:20 PM
Last Post: Larz60+
  RESTful API in Google Cloud Functions fggmack 0 341 Dec-30-2018, 06:58 AM
Last Post: fggmack
  Web application with google maps api jomonetta 1 615 Sep-13-2018, 05:23 AM
Last Post: buran
  Needs search the text from attachments in WordPress marcellofisher 1 644 Jul-09-2018, 07:49 AM
Last Post: alex6
  ImportError: No module named django.core.exceptions in google app engine timtinan37 4 1,399 Jun-18-2018, 01:34 PM
Last Post: Larz60+
  Cant get financial data from google Adam 0 1,235 Apr-11-2018, 03:02 PM
Last Post: Adam
  Different Output of findall and search in re module shiva 1 599 Mar-12-2018, 08:39 PM
Last Post: snippsat

Forum Jump:


Users browsing this thread: 1 Guest(s)