Python Forum
Easiest way to log into a site for web scraping? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Easiest way to log into a site for web scraping? (/thread-17038.html)



Easiest way to log into a site for web scraping? - ejected - Mar-26-2019

What is the easiest way to log into a site to web scrape some data?

I tried using RoboBrowser and I can't get it to log in. What are some other ways I can try?

I'm very new to Python so please dumb down everything in your reply :)

Thanks in advance


RE: Easiest way to log into a site for web scraping? - micseydel - Mar-26-2019

I tend to use mechanize. You could also look at Selenium, especially if you have to deal with Javascript. It would probably help to learn more about your use-case before suggesting anything else.


RE: Easiest way to log into a site for web scraping? - metulburr - Mar-26-2019

I tend to use selenium mostly now, just because if i run into javascript, i dont have to rewrite it.