Python Forum
pyGle - a tool for scrapping the web using Google - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: General (https://python-forum.io/forum-1.html)
+--- Forum: Code sharing (https://python-forum.io/forum-5.html)
+--- Thread: pyGle - a tool for scrapping the web using Google (/thread-12358.html)

pyGle - a tool for scrapping the web using Google - Javinator9889 - Aug-21-2018

Hi everyone Big Grin

I present you a tool I developed, pyGle, which aims to be the best tool for doing web scrapping through Google.
Basically, now you can do a search at:
  1. Google Search
  2. Google News
  3. Google Shops
  4. Google Images
  5. Google Patents
  6. Google Books
  7. Google Videos
The results are returned as a Future object, so when they are available, you can access a list which contains each result.

Everything is completely explained at the GitHub page (also the wiki is already completed).
You can contribute by adding your ideas or optimizing the app: it is completely free Tongue

Have fun programming, contribute and share Smile -pyGle - https://github.com/Javinator9889/pyGle

RE: pyGle - a tool for scrapping the web using Google - Larz60+ - Aug-22-2018

tried using your tool python 3.7.0, OpenSuse Leap 15 Linux
I tries to install errors:
Error:
Collecting errors
  Could not find a version that satisfies the requirement errors (from versions: )
No matching distribution found for errors
code:
from pyGle import PyGle


pSearch = PyGle(enable_history=True, use_session_cookies=True)


if __name__ == '__main__':
    pSearch.withQuery("what we want to search").withContainingTwoTerms("Alabaster, Alabama", "business list")
    ft = pSearch.doSearch()
    search_results = ft.result()
    pSearch.pprintHistory()
    print(search_results)
which gives following error:
Error:
Traceback (most recent call last):
  File ".../src/TryPyGle.py", line 10, in <module>
    search_results = ft.result()
  File ".../.pyenv/versions/3.7.0/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File ".../.pyenv/versions/3.7.0/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File ".../.pyenv/versions/3.7.0/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".../venv/lib/python3.7/site-packages/pyGle/extractor/__init__.py", line 270, in __extractor
    html, search_time = super().obtain_html_object(url)
  File ".../venv/lib/python3.7/site-packages/pyGle/extractor/__init__.py", line 37, in obtain_html_object
    built_url = url.build()
  File ".../venv/lib/python3.7/site-packages/pyGle/url/__init__.py", line 251, in build
    from errors import NullQueryError
ModuleNotFoundError: No module named 'errors'