Python Forum
for web scraping, what library other than selenium and scrapy are still good to use?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
for web scraping, what library other than selenium and scrapy are still good to use?
#1
I use selenium and scrapy alot for web scraping.
Here are the main reasons I like those:

Selenium:
can access dynamic page
access iframe
it seems to have no robot.txt issue

script:
alternative much faster
automatic
handle a lot feature that I don't need to handwriting it

is there any other library that is good to use, and especially can combine both features?
is there any library can be fast, automatic, and solve dynamic page issue?
Reply
#2
beautifulsoup4
lxml
more
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020