Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
facebook scraping
#1
I would like to scrape 100 or so facebook groups (from which my true account is in) to obtain specific data. Ive done this with other sites a lot via BeautifulSoup and if more extensive to bypass then with selenium. However im wondering if i login to my account and scrape 100 groups on a regular basis (once a day or two) via selenium, that the requests would trigger a lockdown on my account. Has anyone ever tried this?

I also know about the facebook API but it seems like a pain in the ass to use. I vaguely tried it before and there seemed to also be a lock on requests that just kicked you out when you exceeded them. Also it seems the API didnt use your account but a "bot account". In this case the bot account would not have access to all those groups?

What would be the best way to scrape facebook groups? And/or a little clarification on the process from someone who has done some facebook scraping before.
Recommended Tutorials:
Reply
#2
I can't speak from direct experience with scraping Facebook, but if you rate limit yourself a bit then that can help. Use randomized sleeps between requests and basically try to mimic real user behavior. I'm sure there are people who visit ~100 groups per day and are legit users that Facebook definitely doesn't want to discourage from being on the platform by limiting them.
Reply
#3
I have have tried to scrape Facebook a couple of time not that i have needed it.
just as quick help to answer question about like here.

Look like Get/Post with eg Requests to the Graph API /{groupId}
Can get info about groups.
Reply
#4
the more i look into the facebook API the more i hate it. Apparently since a version not too long ago, the API permissions were taken away to give you a list of all user groups in which you are a member. You can now only get a list of groups in which you are an admin of. So i have to go through and use selenium to get the group ID's anyways. It seems like their API is too restrictive. And their documentation is horrendous to someone not familiar with it and i see very little python examples of actual code usage. I could only assume because most people use selenium instead.  Tongue

using that example i get only this for a friends list
Quote:{u'data': [], u'summary': {u'total_count': 32}}
just shows the friends count, not anything else.
Recommended Tutorials:
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Automating Facebook Posts mieciow 2 1,335 Aug-07-2023, 12:34 PM
Last Post: Gaurav_Kumar
  Web scrapping login facebook credentials kosmas9 0 1,912 Aug-17-2020, 01:33 PM
Last Post: kosmas9
  How to click facebook message button JanelleGuthrie 2 2,369 May-14-2020, 06:02 PM
Last Post: Larz60+
  Application like simpler facebook wall in python framework seidman 1 3,273 Mar-11-2018, 05:07 PM
Last Post: wavic
  Need help with Django 2.0 + Facebook SDK avtrrr 4 4,520 Jan-31-2018, 08:36 AM
Last Post: Larz60+
  facebook friends crawler edithegodfather 12 20,889 Jan-15-2018, 07:07 AM
Last Post: qnkhuat
  I need request POST for Facebook in My Profile Kalet 4 4,244 Sep-27-2017, 05:53 PM
Last Post: Kalet
  Facebook AutoPost badpandahere 6 4,891 Sep-12-2017, 11:20 AM
Last Post: badpandahere
  Scrape Facebook page user posts text stockholm 6 8,282 May-08-2017, 12:24 PM
Last Post: Joseph_f2

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020