Python Forum
Thread Rating:
  • 3 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web-scraping part-2
#7
So here gone look a getting source from the web with BeautifulSoup and lxml.
For both BS and lxml(aslo has it's own method) is advisable to use Requests.
So i install Requests into my virtual environment:

Gone use python.org as example.
We are getting the head tag which is <title>Welcome to Python.org</title>.
As mention before in part-1 use Developer Tools Chrome and FireFox(earlier FireBug) to navigate/inspect web-site.

So using method over XPath /html/head/title and CSS selector head > title,
to get the head title tag.

  • BeautifulSoup CSS selector

  • lxml XPath
  • lxml CSS selector

Reply


Messages In This Thread
Web-scraping part-2 - by snippsat - Oct-30-2016, 11:21 PM
RE: Web-scraping part-2 - by metulburr - Oct-31-2016, 12:56 AM
RE: Web-scraping part-2 - by metulburr - Oct-31-2016, 12:56 AM
RE: Web-scraping part-2 - by snippsat - Oct-31-2016, 01:17 AM
RE: Web-scraping part-2 - by metulburr - Jan-29-2017, 01:35 AM
RE: Web-scraping part-2 - by snippsat - Jan-29-2017, 03:49 PM
RE: Web-scraping part-2 - by snippsat - Jan-30-2017, 01:43 PM
RE: Web-scraping part-2 - by metulburr - Oct-21-2017, 12:10 AM
RE: Web-scraping part-2 - by snippsat - Apr-01-2018, 02:06 AM
RE: Web-scraping part-2 - by metulburr - Oct-15-2018, 11:59 PM
RE: Web-scraping part-2 - by snippsat - Oct-16-2018, 04:18 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Web-Scraping part-1 snippsat 2 34,611 Jun-08-2017, 10:55 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020