Python Forum
urlib - to use or not to use ( for web scraping )?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urlib - to use or not to use ( for web scraping )?
#24
Requests has it in utils.
>>> import requests
>>> 
>>> url = 'http://www.cwi.nl:80/%7Eguido/Python.html'
>>> u = requests.utils.urlparse(url)
>>> u
ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html', params='', query='', fragment='')
>>> u.geturl()
'http://www.cwi.nl:80/%7Eguido/Python.html'
>>> u.scheme
'http'
>>> u.netloc
'www.cwi.nl:80'
There no problem to use urllib.parse for a so specific task as parse url.
requests.utils has stuff that was not the main goal of Requests project,so documentation is spare.
Reply


Messages In This Thread
RE: urlib - to use or not to use ( for web scraping )? - by snippsat - Nov-27-2018, 10:59 AM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020