Specifically facebook is hard by itself solely because they are trying to stop bots from doing anything on their site, where other sites cannot afford such measures, etc. Any site that has an API is going to try to stop bots, as they want people to use their API to limit their access. But facebook is probably the worst because of their "unlimited funds". Ive seen code try to break bots by randomizing their ID names each session, or they nest iframes to not show the code in the initial source, etc. Along with the usual javascript to break basic bot measures, and obfuscate their code. Usually though its not that, but your code.
If the site you are scraping is facebook, then yes you need selenium.
Show your code, and explain what page on facebook (if possible) that you are having trouble scraping and i can see try to see why. If you search these forums there has been previous discussions about scraping facebook and examples given of some basic tasks already. For example:
https://python-forum.io/Thread-facebook-...t=facebook
If the site you are scraping is facebook, then yes you need selenium.
(Jan-30-2018, 01:39 PM)sumandas89 Wrote: It happens that some contents are available and in the pages but not available in the page sourceSometimes they add things in iframes so that you have to switch to that window to be able to scrape it. But i am unsure as you have not said the exact page you are looking for.
Show your code, and explain what page on facebook (if possible) that you are having trouble scraping and i can see try to see why. If you search these forums there has been previous discussions about scraping facebook and examples given of some basic tasks already. For example:
https://python-forum.io/Thread-facebook-...t=facebook
Recommended Tutorials: