An other one one,i drop to do the search with Chrome/Phantom and use
Have added next page search
search?q=
Have added next page search
Google_Search('python forum', page=1)
.from selenium import webdriver from bs4 import BeautifulSoup class Google_Search: def __init__(self, search, page=0): self.search = search self.page = page self.url = f'https://www.google.com/search?q={self.search}\ &ei=m3w9WuyXNJHMwALelovwAQ&start={str(self.page)+"0"}&sa=N&biw=848&bih=972' self.result() def result(self): browser = webdriver.PhantomJS() browser.get(self.url) soup = BeautifulSoup(browser.page_source, 'lxml') name_link = soup.find_all('h3', class_='r') link = soup.find_all('cite') for n_link, l in zip(name_link,link): print(f'{n_link.text}\n{l.text}') print('---------') if __name__ == '__main__': Google_Search('python forum') #Google_Search('python forum', page=1)
Output:Forums | Python.org
https://www.python.org/community/forums/
---------
Python Forum
https://python-forum.io/
---------
What are the best Python forums to hang out in? : Python - Reddit
https://www.reddit.com/.../Python/.../what_are_the_best_python_forums_to_ hang_out_in/
---------
Python Forum | Dream.In.Code
www.dreamincode.net/forums/forum/29-python/
---------
Python For Beginners Forum | Codecademy
https://www.codecademy.com/en/forums/python-for-beginners
---------
Python Syntax Forum | Codecademy
https://www.codecademy.com/.../forums/introduction-to-python-6WeG3
---------
Nytt Norsk Python Forum - Scriptingspråk (Python, Perl, Ruby o.l ...
https://www.diskusjon.no/index.php?showtopic=828151
---------
Python Programming - Dev Shed Forums
forums.devshed.com/python-programming-11/
---------
Python - Raspberry Pi Forums
https://www.raspberrypi.org/forums/viewforum.php?f=32
---------
Python - thenewboston Forum
https://thenewboston.com/forum/category.php?id=15
---------