Jan-29-2018, 07:05 AM
I worked with beautifulsoup to scrap data from webpages and used selenium to enter web pages that needs login.
Now my question is can I enter into webpages using selenium and then scrape data from there using beautifulsoup?
I observed that beautifulsoup never work on those pages needs login, so I need to login first to that website using selenium. If beautifulsoup will not work in this like of webpages where I need login, then how to scrap images or a texts associated with particular html tag on that page? I know it's very easy to scrap texts or images using beautifulsoup and urlretrieve from the normal websites.
Now my question is can I enter into webpages using selenium and then scrape data from there using beautifulsoup?
I observed that beautifulsoup never work on those pages needs login, so I need to login first to that website using selenium. If beautifulsoup will not work in this like of webpages where I need login, then how to scrap images or a texts associated with particular html tag on that page? I know it's very easy to scrap texts or images using beautifulsoup and urlretrieve from the normal websites.